Software security has always been a race between builders and attackers. Artificial intelligence is now changing the speed of that race. The worry is not simply that AI can write malware or help hackers. The deeper concern is that AI is being woven into the software development lifecycle before many companies have updated their security habits, approval processes and incident response plans.
The clearest warning comes from breach data. Verizon’s 2026 Data Breach Investigations Report says 31% of breaches now begin with software vulnerabilities, overtaking stolen credentials as the leading entry point for the first time in the report’s 19-year history. Verizon also says generative AI is being used to accelerate attack techniques, including spotting security gaps and writing malware.
That is the new reality for software teams. AI can help developers ship faster, find bugs sooner and automate repetitive security work. But the same abilities can also produce insecure code at scale, expose sensitive company data, overload vulnerability teams, and create new attack paths inside AI-powered applications.
Code that works is not always code that is safe
For many businesses, the first AI security concern starts in the editor. Developers are using coding assistants to generate functions, tests, documentation and even entire application modules. That productivity boost is real, but security researchers continue to find a gap between code that runs and code that is safe.
Veracode’s 2025 GenAI Code Security Report tested more than 100 large language models across Java, Python, C# and JavaScript. It found that 45% of generated code samples failed security tests and introduced OWASP Top 10 vulnerabilities. Java had the highest failure rate in that evaluation, while common issues included cross-site scripting and other familiar web application weaknesses.
The problem did not disappear as newer models arrived. In a 2026 update, Veracode said syntax correctness had climbed above 95%, but secure code generation remained stuck at around 55%. In plain terms, AI tools are getting better at writing code that compiles, but not necessarily code that survives attack.
This matters because software teams may not always know where AI-generated code enters the stack. It can come from in-house developers, contractors, open-source maintainers, low-code platforms or third-party vendors. A company that bans AI in its own engineering team may still inherit AI-written code through dependencies and suppliers.
AI applications bring their own vulnerabilities
The second concern is that AI-powered applications are not just normal applications with a chatbot attached. They introduce new failure modes. OWASP’s 2025 Top 10 for LLM and generative AI applications lists prompt injection, sensitive information disclosure, supply chain weaknesses, data and model poisoning, improper output handling, excessive agency, system prompt leakage, vector and embedding weaknesses, misinformation and unbounded consumption among the key risks.
Prompt injection is one of the easiest risks to understand and one of the hardest to eliminate. It happens when a user, webpage, document or other input tricks a model into following instructions it should ignore. In a traditional application, developers try to separate commands from data. In an AI application, that boundary is less obvious because the model reads instructions and outside content as language.
That creates awkward security questions. Should an AI assistant be allowed to read customer emails and trigger refunds? Should a coding agent be allowed to open pull requests and run commands? Should a business analyst’s AI tool connect to production data? The more an AI system can do, the more damage it can cause when it misunderstands a task, leaks data or follows hostile instructions.
Agentic AI raises the stakes further. The UK National Cyber Security Centre warned in May 2026 that agentic systems can access data sources, remember context, make decisions, use tools and take actions without continuous human intervention. The NCSC said this extra autonomy increases the attack surface and makes behaviour harder to predict, test and govern.
Data is now part of the software supply chain
AI also changes how companies should think about data security. Training data, prompts, embeddings, logs, source code, test data and user feedback can all influence how AI systems behave. If that data is sensitive, poorly governed or maliciously altered, it can become a security problem.
The National Security Agency’s Artificial Intelligence Security Center, along with CISA, the FBI and international partners, released AI data security guidance in 2025. The agencies described data used during AI development, testing and operation as a vital part of the AI supply chain. They recommended measures such as tracking data provenance, using digital signatures to authenticate trusted revisions and relying on trusted infrastructure.
This is especially relevant for software companies that feed code repositories, bug histories, design documents or customer tickets into AI tools. These inputs can be valuable to engineers, but they can also reveal secrets. API keys, private business logic, unreleased product plans and customer information should not drift into tools that the company does not control or cannot audit.
Shadow AI is becoming a boardroom issue
Another concern is “shadow AI,” the use of unapproved AI tools by employees. Security teams have faced shadow IT for years, but AI makes the problem more sensitive because workers may paste proprietary code, contracts, incident notes or customer data into external services for quick help.
Verizon’s 2026 DBIR says shadow AI is now the third most common non-malicious data leakage related activity, and that frequent employee use of AI tools rose from 15% to 45% in a single year. That does not mean every use is dangerous, but it does show how quickly informal AI adoption can outrun policy.
Blocking every tool is unlikely to work. Employees will keep using systems that save time. The better answer is to offer approved tools, set clear rules for sensitive data, monitor usage, educate teams and make safe behaviour easier than risky workarounds.
The patch window is shrinking
AI is also changing vulnerability management. For years, many organizations accepted slow patch cycles because attackers needed time to find, understand and exploit flaws. That assumption is weakening.
The NCSC warned in May 2026 that AI, when used by sufficiently skilled people, can exploit technical debt at scale and pace across the software ecosystem. It expects a “patch wave,” a rush of software updates needed to address newly disclosed vulnerabilities across open-source, commercial, proprietary and software-as-a-service products.
The agency’s advice is practical: identify external attack surfaces, prioritize internet-facing systems, apply updates quickly and prepare to patch more often and at scale. It also warned that patching alone will not fix every problem, especially when legacy technology is out of support and cannot receive updates.
At the same time, AI can create too many findings for humans to handle. The NCSC separately urged organizations to ask whether they have a process to receive, prioritize and fix vulnerabilities found by AI. It noted that more than 40,000 vulnerabilities were assigned CVEs in 2025, while CISA’s Known Exploited Vulnerabilities catalog tracked only about 400 new vulnerabilities as exploited, with about 40 of those being zero-days at the time of exploitation.
That distinction matters. The future of software security will not be won by finding the most bugs. It will be won by fixing the right bugs before attackers exploit them.
Attackers are learning the same tools
Government agencies and AI companies have also reported that threat actors are adopting AI across their workflows. The NCSC assessed that AI would increase the volume and impact of cyberattacks, especially by improving reconnaissance, social engineering and data analysis. It also said AI lowers the barrier for less-skilled criminals, hackers-for-hire and hacktivists.
AI providers have published similar warnings. OpenAI said its threat reports have documented malicious cyber activity, social engineering, cyber espionage and scams, while Anthropic said in 2025 that it had disrupted cases where AI was used for extortion, ransomware development and agentic cyber operations.
In November 2025, Anthropic reported what it described as a large-scale AI-orchestrated cyber espionage campaign in which a threat actor manipulated Claude Code into attempting infiltration against roughly 30 targets. Anthropic said the campaign showed how agentic systems can help attackers with reconnaissance, vulnerability testing, credential harvesting and documentation at machine speed.
What companies should do now
The answer is not to reject AI. That would be unrealistic and, in many cases, counterproductive. AI can help defenders summarize alerts, find risky code, detect phishing, speed up incident response and improve security testing. The question is whether companies can adopt it with controls that match its power.
First, treat AI-generated code as untrusted until reviewed. It should pass the same checks as human-written code, including secure code review, static testing, dependency scanning, secret scanning and runtime testing. Developers should be trained to ask AI tools for secure patterns, but they should not assume that a model understands the company’s threat model.
Second, secure AI applications like high-risk software systems. Limit permissions, validate outputs before they reach databases or APIs, isolate tools, sandbox agents, log actions, and require human approval for sensitive operations. Least privilege is not optional when an AI agent can act across files, repositories, cloud systems or customer data.
Third, govern data before connecting it to models. Companies should classify what data can be used with approved AI tools, what must stay internal and what should never be pasted into prompts. Sensitive source code, credentials, customer records and private vulnerability reports require special handling.
Fourth, prepare for faster vulnerability cycles. Maintain accurate asset inventories, know which systems face the internet, understand software dependencies, use exploitability-based prioritization and automate safe updates where possible. AI will help find more flaws, but resilience depends on whether teams can patch, replace or contain vulnerable systems.
Finally, keep humans accountable. AI can suggest, summarize and act, but a person must own the decision to deploy it, the access it receives, the controls around it and the consequences when something goes wrong. NIST’s AI Risk Management Framework is designed to help organizations incorporate trustworthiness into the design, development, use and evaluation of AI systems, and its generative AI profile helps organizations identify risks unique to generative AI.
AI is not making software security irrelevant. It is making it more important. The companies that benefit will be those that use AI to strengthen engineering discipline, not bypass it. The companies that struggle will be those that let faster coding, faster scanning and faster automation move ahead of governance.
The next security crisis may not begin with a hacker typing in the dark. It may begin with a helpful tool, a copied prompt, a rushed pull request or an agent granted more access than anyone remembered to revoke.
