Does the cyber security of AI models need to follow any novel principles that aren’t set out under existing policy and technology security principles? If so, what are these measures and how do the differ from what exists? How do the vulnerabilities/risk of AI model security differ from existing cyber threats?

Background

In order to ensure the UK is well protected now and in the future we need to build upon our existing knowledge of cyber security risks and the resilience of software supply chains and are working to identify which mitigations the government can leverage to ensure the UK is well protected now and in the future.

We’re also assessing the impact of these mitigations in driving enhanced security measures. A key area of research interest therefore is understanding the consequences of requiring higher levels of cybersecurity for products and digital services sold in the UK. Strengthening this evidence base will help shape and inform future policy work around the regulation of products and business requirements for cyber security.
Some technologies are critical to cyberspace. To build and sustain competitive edge in cyber-related technologies we need a coordinated, rigorous and consistent approach to identify and analyse critical areas of science and technology and prioritise national effort. CSDI is interested inbeing able to better anticipate the science and technology developments most vital to UK cyber power and in analysing thebe opportunities and risks related to those developments.

It is crucial to ensure that digital identity solutions are both secure and inclusive. The directorate seeks to gather further evidence on how best to enable more inclusive digital identity services whilst maintaining robust security measures. Digital identity solutions enable a person to prove something about themselves for the purposes of a transaction, an eligibility check or accessing services. This includes verifying a person’s age when purchasing age regulated products, when undertaking pre-employment checks or opening a bank account. The government is not mandating specific approach, but instead has committed to setting outcomes-based standards in the form of the UK digital identity & attributes trust framework. (https://www.gov.uk/government/publications/uk-digital-identity-and-attributes-trust-frameworkbeta-version/uk-digital-identity-and-attributes-trust-framework-beta-version)
Organisations that adhere to these standards and agree to oversight from the Office for Digital Identities and Attributes (OfDIA) will receive a trust mark, so that businesses and individuals can be confident that their digital identity solution is safe and secure.

Next steps

If you are keen to register your interest in working and connecting with DSIT Digital Technology and Telecoms Group and/or submitting evidence, then please complete the DSIT-ARI Evidence survey - https://dsit.qualtrics.com/jfe/form/SV_cDfmK2OukVAnirs.
Please view full details: https://www.gov.uk/government/publications/department-for-science-innovation-and-technology-areas-of-research-interest/dsit-areas-of-research-interest-2024

Related UKRI funded projects


  • Academic Centre of Excellence in Cyber Security Research - Newcastle University

    The Newcastle Academic Centre of Excellence in Cyber Security Research pursues the research vision "Protecting Society's Fabric." Following the conviction that cyber security does not arise from protecting crit...

    Funded by: EPSRC

    Why might this be relevant?

    The project focuses on cyber security research, including the resilience of critical infrastructures and secure design of security protocols, which directly addresses the question about novel principles for AI model security.

  • Academic Centre of Excellence in Cyber Security Research - University of Northumbria at Newcastle

    Northumbria University applies knowledge from multiple disciplines, into digital security through the work of the Cyber Security Research Group (CSRG) - a cross university group that combines (i) technical research on bi...

    Funded by: EPSRC

    Why might this be relevant?

    The project addresses the human dimensions of cyber security and explores factors influencing behavior change, which are relevant to understanding vulnerabilities and risks in AI model security.

  • CyberSecDome

    Organisations across the sectors significantly benefit from digital transformation to support evolving business models, services and customer experience. Despite the benefits of digital infrastructure adoption, there are...

    Funded by: Horizon Europe Guarantee

    Why might this be relevant?

    Partially relevant as it addresses AI-enabled security solutions but does not specifically mention novel principles or how vulnerabilities/risk of AI model security differ from existing cyber threats.

  • Research Institute in Science of Cyber Security (RISCS) Phase 2

    The Digital Economy is a key part of the strategy for UK economic growth. But as more businesses move into the digital space, they need to be able to protect their assets (such as their Intellectual Property) and process...

    Funded by: EPSRC

  • Academic Centre of Excellence in Cyber Security Research - King's College London

    King's College London (KCL) is a research-led university established in 1829 (4th oldest in England), and one of the top 25 universities in the world (2017-18 QS international world rankings). The current King's Strategi...

    Funded by: EPSRC

    Why might this be relevant?

    The project specifically focuses on AI Cyber Security and the cyber security of AI models, addressing the novel principles and vulnerabilities mentioned in the question.

  • Academic Centre of Excellence in Cyber Security Research - Newcastle University

    Following the insight that cyber security does not only come from protecting critical infrastructures, the Centre for Cybercrime and Computer Security at Newcastle University (referred to as Cybercrime Centre in what fol...

    Funded by: EPSRC

  • AI - Cybersecurity Nexus

    The project **AI-Cybersecurity Nexus** is dedicated to addressing the critical intersection of cybersecurity and artificial intelligence (AI), specifically targeting small to medium-sized enterprises (SMEs) in Greater Ma...

    Funded by: Innovate UK

  • DiScriBe: Digital Security by Design Social Science Hub+

    Technological advances have done, and will do, much to improve cybersecurity. But, a technological approach is only part of the solution - achieving digital security is inherently a socio-technical endeavour. By combinin...

    Funded by: ISCF

  • Cyber-R: Securing Businesses through Generative AI-based Adaptive Cyber Resilience Service

    According to the UK Cyber Security Breaches Survey 2024, UK businesses faced approximately 7.78 million cybercrimes over the past year, with phishing attacks accounting for 84% of these incidents. Additionally, 50% of bu...

    Funded by: Innovate UK

    Lead research organisation: UNIVERSITY OF WOLVERHAMPTON

  • System Security Modeller

    "With online banking and shopping being ever more critical to our daily lives and health data increasingly going online, protecting the security of our IT systems has never been more important. We have a cyber-secu...

    Funded by: Innovate UK