Cisco tried using AI to write security incident reports and things didn't really go as planned

Date:
Fri, 22 May 2026 19:25:00 +0000

Description:
There are many challenges with generating long-form reports with AI, even
with specific sources and instructions.

FULL STORY ======================================================================Copy link Facebook X Whatsapp Reddit Pinterest Flipboard Threads Email Share this article 0 Join the conversation Follow us Add us as a preferred source on Google Newsletter Subscribe to our newsletter Cisco warns AIgenerated
incident reports are often inaccurate, inconsistent, and prone to data loss due to LLM limitations The company advises granular, singletask prompts,
fixed source documents, and strict formatting rules to improve reliability Crosscontamination between reports remains a challenge, with researchers recommending fresh sessions for each new incident report to avoid errors Any companies looking to utilize AI tools for their security reporting may want
to read a new report from Cisco outlining its experience using AI-generated incident reporting.

The company has warned those using AI to create long-form technical content should expect significant inaccuracies, unusual conclusions, and inconsistent writing styles, mostly because of the probability-driven nature of Large Language Models (LLM). These models generate output by predicting the next token, typically a word or sub-word, in a sequence, based on model weights
and training data, Cisco says or, as The Register puts it, theyre essentially a fancy autocomplete system that makes educated guesses. Latest Videos From You may like Microsoft scientists find most AI models struggle with long-running tasks AI can summarize meetings, but heres what it still cant do in 2026 Even the most advanced AI models fail more often than you think on structured outputs What works and what doesn't Since AI is basically just predicting the next word, it creates four key issues, according to Cisco:
LLMs use different data for each new query, making consistency and standardization a challenge Even if the same data is shared, the outcome will always be slightly different Every new document will have different structure and formatting, which is yet another standardization challenge AI often discards valuable data, changing the outcome This doesnt mean AI is unusable for long-form technical reporting - quite the contrary. It can still save enterprises plenty of data, but the tool needs to be properly set up and optimized. Cisco says a good approach is to give the AI granular, single-task instructions focusing on a specific, small portion of the report.

The company also said that the AI should not be free to choose their sources for the report but instead should be given specific documents. Finally, the
AI should have clear instructions regarding the formatting and the style.

"A blind test of the sample report in our quality assurance process showed no noticeable drop in overall writing quality," Cisco said. Are you a pro? Subscribe to our newsletter Sign up to the TechRadar Pro newsletter to get
all the top news, opinion, features and guidance your business needs to succeed! Contact me with news and offers from other Future brands Receive email from us on behalf of our trusted partners or sponsors By submitting
your information you agree to the Terms & Conditions and Privacy Policy and are aged 16 or over.

"The peer reviewer, professional editor, and management reviewer all made complimentary comments about the report while unaware that it was AI-generated. The peer reviewer commented that the incidence of typos and grammatical errors was far lower than in the average report."

Cisco further discovered another challenge - when the AI is asked to edit multiple sample reports in a single session, content from one reports source material gets cross-contaminated with another, even if the notes used to generate the first report were deleted from the projects reference documents.

To work around this issue, the researchers advised starting a new session and re-entering the prompts for each new incident report. The best antivirus for all budgets Our top picks, based on real-world testing and comparisons

Read our full guide to the best antivirus 1. Best overall: Bitdefender Total Security 2. Best for families: Norton 360 with LifeLock 3. Best for mobile: McAfee Mobile Security Follow TechRadar on Google News and add us as a preferred source to get our expert news, reviews, and opinion in your feeds.



======================================================================
Link to news story: https://www.techradar.com/pro/security/cisco-tried-using-ai-to-write-security- incident-reports-and-things-didnt-really-go-as-planned


--- Mystic BBS v1.12 A49 (Linux/64)
* Origin: tqwNet Technology News (1337:1/100)