Examining recent developments in AI

Over the last few years advancements in AI have been rapid, covering a range of applications across multiple industries. Some of the key developments that pertain to cyber security are explored below.

Proliferation of large language models

Large language models (LLMs - a subset of of Generative AI, see below) have been in development since the 1960s,3 but interest in them increased massively in June 2020, when OpenAI released GPT-3.4 Since then, media hype has driven adoption among technologists, bad actors, and the broader population to the point where LLMs offer several opportunities and challenges to organisations, security professionals, data privacy officers and AI developers.

LLMs such as GPT-35 and BERT6 have been leveraged in various cyber security applications to automate and optimise tasks such as threat intelligence gathering and analysis, natural language processing and report generation.7 Security teams can leverage LLMs to improve efficiency, automate routine tasks and focus on strategic initiatives.

However, the widespread use of LLMs across cyber security and other business functions has raised concerns about data privacy, inadvertent adoption of biases or use of incorrect information, and increased adversarial attacks that use leaked business and personnel data to exploit vulnerable systems and people, impersonate legitimate entities or spread misinformation.

Corporate policies, designed to restrict the use of LLMs and other AI technologies on business equipment, can prove effective but may lead to other risks in the form of Shadow AI, whereby employees make use of the services on their own devices connected to corporate networks.8

Related research

ISF research

This research is only available to ISF Members

Securing Operational Machine Learning Systems

This briefing paper clarifies information security responsibilities with respect to operational machine learning (ML) systems. It explores where the business should take up the responsibilities and sets out ways to secure these systems that resemble, and diverge from, established approaches.

ISF Analyst Insight Podcast

What ChatGPT Did Next

Patricia Rodrigues, Paul Watts and Mark Ward

The rise of Generative AI

Generative AI (GenAI) has had a significant impact on the cyber security landscape in recent times with deep learning-based models, such as Generative Adversarial Networks (GANs), being leveraged by cyber criminals to develop advanced and evasive malware, enhance phishing campaigns and create deepfakes that can bypass traditional security defences by deceiving both humans and machines.9 The rapid advances in GenAI capabilities have necessitated the development of more robust strategies, tools, and technologies capable of detecting, analysing, and mitigating new and evolving cyber threats effectively.

However, GenAI also offers AI-driven tools that can be used to detect and respond to cyber threats more effectively than traditional methods. AI algorithms can analyse vast amounts of data to identify patterns that may signify malicious activity, allowing proactive action to be taken against potential breaches. GenAI has also been instrumental in the development of adaptive security solutions that can respond to evolving security threats, providing dynamic defences that can keep pace with sophisticated attackers.10

Defensive AI

AI-powered tools have revolutionised defensive AI tactics through full or semi-automation, from leveraging predictive analytics to enriching threat intelligence with business context and enhancing threat detection and analysis. AI can also improve the speed and effectiveness of incident detection, response and remediation. By taking such a proactive approach, organisations can better prioritise resources, focus on high-risk areas and implement preventative measures to strengthen their defences.11

Retain human input

It is important to retain an element of human interaction and oversight to ensure that AI responses are aligned with desired and expected outcomes.

Predictive analytics

Predictive analytics can be utilised to analyse historical data, current trends, and emerging patterns to forecast potential future security risks, vulnerabilities, attack vectors and provide actionable insights, real-time alerts, and automated recommendations to help make informed decisions and take timely action.

Threat intelligence

AI has already had a huge impact on threat intelligence, leveraging advanced algorithms and natural language processing (NLP) techniques, to provide deeper insights into evolving cyber threats, global threat landscapes and adversarial tactics, techniques, and procedures (TTPs). By collating complex and unstructured data, AI-powered threat intelligence platforms can automatically collect, analyse, and correlate vast amounts of data from diverse sources, including open-source intelligence, dark web forums and closed-list sources. This helps security teams to identify and prioritise emerging threats, vulnerabilities, and indicators of compromise (IOCs) and make informed decisions, develop proactive defence strategies and respond promptly to potential security incidents.

Threat detection and analysis

AI-powered threat detection systems can analyse vast amounts of data in real-time, including network logs, user behaviour and system activities, to detect patterns, anomalies, and other IOCs that may indicate advanced persistent threats, targeted attacks, or insider threats. Through continuous learning, such systems can prioritise alerts, categorise threats and provide actionable intelligence to help security professionals make informed decisions and respond promptly to suspected security incidents. By improving the accuracy of threat analysis, and saving the time spent chasing down false positives, AI allows security teams to focus on more important things and organisations to become more resilient, improve incident response times, and mitigate the potential impact of cyber threats.

Automated incident response

AI-powered automated incident response allows organisations to detect, analyse and mitigate security incidents with greater speed and efficiency. These AI-driven platforms can automatically analyse and correlate security alerts, diverse threat intelligence feeds and network data to identify and triage potential security incidents. Optionally, and with great care and human oversight, they can also respond with predefined rule-based or adaptable reactions.

By integrating AI-powered incident response capabilities with existing security tools and orchestration platforms, organisations can minimise false positives and automate complex incident response actions, enabling them to respond to new threats with greater accuracy, consistency, and speed.

Return to the start

About this insights paper