In which channels is harmful Gen-AI content most prevalent, how does it spread, and how can friction be introduced to these channels?

Background

Although there is already material evidence on the types of serious harms individuals encounter online, there still remain a number of emerging harms, where the evidence base is still yet to mature (e.g. epilepsy trolling, online animal abuse). SOH would like to close this significant gap in understanding the impact of encountering different types of serious harms online and understanding the best approaches to measuring the impact of the Online Safety legislation.

SOH highlights the importance of Media Literacy in the digital age and asks for further studies to uncover barriers to engagement as well as the effectiveness of DSIT programmes. This issue closely relates to Counter-Disinformation interventions, which requires evidence for its effect on bystanders, topic specific disinformation and what tools can be used to combat this issue.

Research on Safety Technology would greatly develop SOH’s understanding of the relationship that DSIT online safety objectives have with the technology market today. A primary focus lands on improving Age Assurance (AA) measures. This includes ensuring transparency and assessing opportunities for the sector.

Next steps

If you are keen to register your interest in working and connecting with DSIT Digital Technology and Telecoms Group and/or submitting evidence, then please complete the DSIT-ARI Evidence survey - https://dsit.qualtrics.com/jfe/form/SV_cDfmK2OukVAnirs.
Please view full details: https://www.gov.uk/government/publications/department-for-science-innovation-and-technology-areas-of-research-interest/dsit-areas-of-research-interest-2024

Source

This question was published as part of the set of ARIs in this document:

DSIT Areas of Research Interest 2024 GOV UK

Related UKRI funded projects


  • Safe Internet surfing with an intelligent child-centred shield against harmful content

    The Internet provides high exposure to malicious content with direct impact on children's safety. Illicit, violent and pornographic material to name a few. The Internet is also an enabler for cyber victimisation such as ...

    Funded by: Innovate UK

    Why might this be relevant?

    The project focuses on creating a child-centered shield against harmful online content, directly addressing the issue of harmful Gen-AI content and its spread.

  • Rethinking Media Literacy and Digital Skills in Europe (REMEDIS)

    Over the past decade, we have witnessed major transformations in access to digital media platforms in Europe. However, navigating this digitally mediated world can be challenging as it requires operational, social, conte...

    Funded by: AHRC

    Why might this be relevant?

    The project addresses the importance of Media Literacy and Digital Skills, which are related to the issue of harmful Gen-AI content, but does not specifically focus on the prevalence, spread, and introduction of friction to these channels.

  • HORIZON-MSCA-2023-SE-01

    Digital technologies have become an important part of children's daily lives, regardless of cultural and geographical context. This is often due to the actions of parents who notoriously share information about their chi...

    Funded by: Horizon Europe Guarantee

    Lead research organisation: The Polish University Abroad (PUNO)

    Why might this be relevant?

    The project addresses the risks of sharenting and its impact on children's online safety, which is related to the broader issue of harmful online content.

  • PrivacyEye: Controlling Harmful Multimedia Sharing Among Children

    The increasing use of electronic devices and online applications among children in the UK has raised significant concerns about their online safety. Nearly 90% of children aged 0-18 go online daily, with those aged 5-15 ...

    Funded by: Innovate UK

    Lead research organisation: DE MONTFORT UNIVERSITY

  • Equally Safe Online

    We address the timely topic of online gender-based violence (GBV): Almost 1 in every 2 women and non-binary people (46%) reported experiencing online abuse since the beginning of COVID-19 (Glitch report, 2020). Our aim i...

    Funded by: EPSRC

    Why might this be relevant?

    Partially relevant as it addresses online abuse but not specifically Gen-AI content.

  • AI Safety Platform: Generative AI and Cybersecurity Training SaaS for Schools and Families

    **Problem statement:** The rapid rise of **generative AI** has introduced **unprecedented cybersecurity risks**, particularly for students and families. **Deepfakes, AI-driven scams, misinformation, identity theft, and c...

    Funded by: Innovate UK

    Lead research organisation: UNIVERSITY OF EAST LONDON