AI Finds So Many Linux Bugs, Canonical Changes to a Two-Week Stable Release Update Cycle (nerds.xyz) 84
"Finding vulnerabilities faster also puts pressure on Linux distributions to fix and deliver patches faster," writes Slashdot reader BrianFagioli
AI has transformed bug discovery from "a manual, time-intensive process into a highly automated engine," notes Canonical's blog, leading to a "recent explosion in the volume of CVEs". Additionally, the upstream kernel community became its own CVE Numbering Authority (CNA) and assigned CVE (Common Vulnerabilities and Exposures) identifiers to thousands of bugs, arguing that at the kernel level, almost any type of bug that can affect a running system, could potentially be classified as a vulnerability. As a result, the volume of CVEs has skyrocketed exponentially, creating a massive backlog of alerts and forcing defenders to drastically increase the speed of their fixes to close the window of risk.
To address the growing volume of CVEs and the demand for faster security fixes, we are transitioning to a unified, 2-week release cycle...
While a patch is being prepared, Canonical aims to provide safe workarounds where applicable, so users aren't left exposed in the meantime. Where no safe workaround exists, Canonical will say so clearly and point users toward general hardening steps instead. The goal is to get environments into a defensible, safer state within 24 to 48 hours of public disclosure — well before a patch ships. This doesn't replace the patch; it buys the time needed to fix the vulnerability properly, without sacrificing security.
"Linux did not suddenly become wildly insecure overnight," notes the blog Nerds.xyz. "We are getting much better at finding and cataloging problems that may have previously gone unnoticed." There is something almost ironic about all of this. AI is routinely pitched as a tool that will make software development faster, but it is also making vulnerability discovery faster. That means maintainers now have to accelerate the other side of the equation too.
For Ubuntu users, that should ultimately be good news. More bugs being discovered is preferable to vulnerabilities sitting unnoticed in the Linux kernel.
AI has transformed bug discovery from "a manual, time-intensive process into a highly automated engine," notes Canonical's blog, leading to a "recent explosion in the volume of CVEs". Additionally, the upstream kernel community became its own CVE Numbering Authority (CNA) and assigned CVE (Common Vulnerabilities and Exposures) identifiers to thousands of bugs, arguing that at the kernel level, almost any type of bug that can affect a running system, could potentially be classified as a vulnerability. As a result, the volume of CVEs has skyrocketed exponentially, creating a massive backlog of alerts and forcing defenders to drastically increase the speed of their fixes to close the window of risk.
To address the growing volume of CVEs and the demand for faster security fixes, we are transitioning to a unified, 2-week release cycle...
While a patch is being prepared, Canonical aims to provide safe workarounds where applicable, so users aren't left exposed in the meantime. Where no safe workaround exists, Canonical will say so clearly and point users toward general hardening steps instead. The goal is to get environments into a defensible, safer state within 24 to 48 hours of public disclosure — well before a patch ships. This doesn't replace the patch; it buys the time needed to fix the vulnerability properly, without sacrificing security.
"Linux did not suddenly become wildly insecure overnight," notes the blog Nerds.xyz. "We are getting much better at finding and cataloging problems that may have previously gone unnoticed." There is something almost ironic about all of this. AI is routinely pitched as a tool that will make software development faster, but it is also making vulnerability discovery faster. That means maintainers now have to accelerate the other side of the equation too.
For Ubuntu users, that should ultimately be good news. More bugs being discovered is preferable to vulnerabilities sitting unnoticed in the Linux kernel.
Great! (Score:5, Funny)
Re:Great! (Score:5, Funny)
Re:Great! (Score:5, Funny)
I turned claude on windows 11. After running for a few weeks, it generated the final result. It turned out to be Linux.
Re: (Score:3)
The real problem with Windows 11 is every single division in Microsoft is required to be profitable because of course they are so you're fucking weather app uses 3 GB of RAM so it can serve up a limitless supply of advertisements in an effort to make money off a freaking weather app...
All those divisions aren't going to give up their RAM and resources without a fight. So I don't see
Not quite (Score:2, Informative)
I promise you that the weather app is not it's own division with a P&L statement at Microsoft. Instead, you have Experiences + Devices (which has a P&L) and under that, Windows + Devices (which has a P&L). I left MSFT several years ago so I don't know the org chart under W+D or whether there are P&Ls under that level. Typically the CVP (corporate vice president) and up at MSFT has a P&L.
Anyway point being that someone is absolutely putting ads in the weather app to meet his P&L obje
Re: (Score:3)
Microsoft can do a lean, efficient OS when they want to. Windows 7, for example.
They seem to be getting over AI now it has become clear that AI is going to stop people buying new computers and it will be mostly a cloud service. Hopefully with the threat from Linux they will get back on track.
Re: (Score:2)
Re: (Score:2)
Given the number of updates in the last few months, I'd say they already have.
Re: (Score:2)
They have, that's why Microsoft has been releasing so many patches lately.
Remember all the comments about how incompetent Microsoft developers are, and how Linux is so much better? Turns out nobody is safe from AI bug hunters.
For me it's more of a pain with Linux because now I have to support a whole Linux SBOM.
Re: (Score:3)
Remember all the comments about how incompetent Microsoft developers are, and how Linux is so much better? Turns out nobody is safe from AI bug hunters.
For me it's more of a pain with Linux because now I have to support a whole Linux SBOM.
I'll agree when Linux updates act like Windows and render the computer inoperable. Had it happen twice on my work computer, which is managed by my employer. It got to the point where I installed the needed software on my Windows computer, which I manage.
As well is it not time we get away from the idea of perfect security be eliminating all the possible vulnerabilities until it is not possible to ever suffer any issues? I know the modern computer is aimed at people who don't have a clue. The present Paradi
Re: (Score:2)
I've had Ubuntu updates brick VMs before.
Re: (Score:2)
I've had Ubuntu updates brick VMs before.
do you mean it destroyed the VM software? Or just restart it?
Re: (Score:2)
It buggered up the OS enough that it wouldn't boot, possibly because of the client side software for VM integration. I didn't investigate too much, I just made a new VM.
Re: (Score:2)
It buggered up the OS enough that it wouldn't boot, possibly because of the client side software for VM integration. I didn't investigate too much, I just made a new VM.
As a side note, my Mac just upgraded/updated to Golden Gate 27.0 from Tahoe. A huge upgrade. Unlike my Windows machines it is working 100 percent after the upgrade and updates.
I query the group. What is a better computer, a Windows machine you can't even log into and use (although you can see ads on teh login screen. Or a Mac that continues to function like before, only with improvements. And if you are on a hard deadline? This isn't about some obscure software that only runs on one or the other OS - tha
Re: Great! (Score:1)
Re: (Score:2)
what kind of bugs (Score:5, Interesting)
Re:what kind of bugs (Score:5, Insightful)
From other projects, I see about a third or half of bugs the AI finds are legit. Most of those are small corner cases or error handling issues that almost never trigger, but most are actionable. But I value even a 1 line change that clears a static analysis warning if even in practice it was impossible to trigger the issue in a real system.
Re: what kind of bugs (Score:2)
Re: what kind of bugs (Score:5, Interesting)
i can understand that, so it's more of a source code cleanup.
And one that increases security by keeping the computer from booting on occasion. A couple weeks ago, the plethora of updates on my work computer borked my camera so no face login, wouldn't take my PIN or my password, wouldn't let me reset the password, rejected the question set.
It did however serve up ads on the login screen. Tied my IT guy up most of a week cuz it affected the one thing in the Bios I couldn't change. Security through bricking.
Re:what kind of bugs (Score:5, Insightful)
But I value even a 1 line change that clears a static analysis warning if even in practice it was impossible to trigger the issue in a real system.
Yes, I agree. The problem with thinking "this bug seems harmless, because I can't imagine how anyone could exploit it" is in the "I can't imagine" part; my imagination is limited to what is covered by my mental model of how computers work, but an attackers' ingenuity is not.
In particular, the C/C++ optimizer is a devious beast, and will exploit any opportunity to make the code more efficient, even if that means doing things that are wildly unintuitive to a naive human reader -- and it sees any instance of undefined behavior as an opportunity to exploit.
Re: (Score:2)
But I value even a 1 line change that clears a static analysis warning if even in practice it was impossible to trigger the issue in a real system.
Yes, I agree. The problem with thinking "this bug seems harmless, because I can't imagine how anyone could exploit it" is in the "I can't imagine"....
All that said, more and more computers are being rendered perfectly secure, because the minor bug they dutifully patched in the Quixotic quest to create perfect security creates that security by borking the computers.
Nothin' is as secure as a computer that is unplugged and in a closet because it doesn't work. At that time, it's doing forensics. Barely got my computer back in time for my tasking, after it got the 500 vulnerabilities patch.
Since Linux is heading down that path, let's hope they don't su
Re: what kind of bugs (Score:2)
Re: (Score:1)
I'm not anti-AI but Canonical is run by a nutjob (Mark Shuttleworth) so any decision they make ought to be scrutinized heavily.
Re:what kind of bugs (Score:5, Interesting)
This argument makes zero sense.
First, people are asking what the number is, they already know that it's being used to justify a two week release schedule.
Second, I think most people here would like to know what on earth a volume of unknown bugs has to do with a two week release schedule. That's not how bug fixes works.
- If bugs are causing problems NOW for people and/or are security issues, you release ASAP, you don't want for a rolling release.
- If bugs are not causing problems now, you provide testers with a reasonable period of time to test the software out in the field before doing a release.
Two week release time isn't sane, it's PR that's proposing something apparently stupid in response to something people might be concerned about. It's like a company bragging it's making its employees work 23 hour a day shifts to get things done.
I already swore off Ubuntu because of the snap/Firefox fiasco. But if I hadn't, this dumbass fucking policy that all but guarantees hastily untested "bug fixes" are going to be foisted onto Ubuntu users who have made software choices that reflect a desire for stability and reliability, would be the reason I switch.
What a bunch of fucking cretins.
Re: (Score:1)
How is he 'full of shit'? He's got a point.
Anytime even the smallest vulnerability is found in Windows, the Linux crowd here acts like it's the final, devastating blow to M$ and that the solution is obvious; run Linux.
Now we have AI scanning the Linux source code and showing that it's vaunted security has been through obscurity for far too long, but the same crowd hand waves it away with "But are they real bugs?"
Re: (Score:3)
Either you don't understand what security through obscurity is, or your being incredibly disengenous. Linux's source has been on public display since the outset. Windows has never been on display (except for the occasional leak). Automatically and by definition Linuxes security has been less "by obscurity" than it would ever be possible for windows to have been, short of a full so
Re: (Score:2)
Re:what kind of bugs (Score:5, Interesting)
How many are realisically actionable vulnerabilities. The statistics have indicated so far that AI isn't really finding many non-minor bugs.
Seems to be a bit of selection bias baked into what AI is being asked to do. Tried AI (GLM-5.3) on new code that has never been executed. Also ran it against code that has been in production use for many years.
The types of bugs tended to be in obscure features, buggy error paths, parsing / protocol pedantry, cut and paste errors especially in various lookup tables, algorithm accuracy, inconsistencies, poor and obscure concurrency bugs. Can't really expect it to have found anything too important as it would have tripped up code and runtime analyzers or angry customers because all of that would have already been discovered and dealt with.
In the new code it found a couple of show stoppers that would be immediately obvious the second anyone tried it in addition to some more obscure things.
While I've not yet seen it discover any magical exploits it did get us to reconsider some questionable security related decisions and make improvements. Unfortunately tends to focus mostly on nuts and bolts rather than higher level machinery.
Been trying to get LLMs to do bug hunting for years and it has never worked. The AI just never had the depth to understand enough of what is going on to say anything useful. They still output quite a bit of crap... some of it isn't the models fault... for example tend to feed it source files one at a time to keep from blowing through too much context. This requires the models to make all kinds of inferences about dependencies it has no real knowledge of... sometimes it doesn't make the right assumptions. Sometimes it says nonsensical things or doesn't seem to "see" its own context perfectly misreading the code and complaining about something that isn't real... still well worth the effort. Amazing this shit works at all.
Re: (Score:2)
Yeah, the curl devs (to name one project I monitor) are still having to swat away the vast majority of bug reports from what I've seen even if they're being far more positive about slopreports than I would be.
I'm also interested in how many bugs are being added en-mass by people trying to fix minor issues an LLM identified.
Re: what kind of bugs (Score:3)
Re: (Score:3)
To go one step further... I wonder how many of these are the same kinds of 'bugs' you would get by turning on all the compiler warnings, running a linter, or bringing i a commercial tool like LDRA or Vector cover. All of those will complain about all sorts of 'bugs'.
Re: (Score:3)
If you're running Debian, there's no benefit to you, because Debian is getting the same warnings, and responding just as fast (or faster).
OTOH, I don't have the version name memorized. I think Trixie is the current one, so that just means be sure to run apt update frequently. (I do it nearly every night.)
Re: (Score:2)
OK. I don't have those, so I don't know about that. ... well, because I stopped doing it years ago, I forget why, and doing the update is no hassle.
I mainly don't do auto-update because
Re: (Score:2)
I think Trixie is the current one, so that just means be sure to run apt update frequently. (I do it nearly every night.)
Also maybe interesting: unattended-upgrades [debian.org].
Re:implications for other distributions (Score:4, Insightful)
Just one day (Score:2)
I'd like to go one day, just one day, without seeing an AI story. Christ!
Re: (Score:3)
For that, I think you'll need a time machine to take you back to well before the 2020's.
Re: (Score:2)
Or go forward to 2036. :P
Re:Just one day (Score:5, Funny)
I'd like to go one day, just one day, without seeing an AI story. Christ!
I am truly sorry my son, but I can do nothing about this. I am the ALmighty not the AImighty.
Re: (Score:2)
Stop reading tech news?
It got rusty. (Score:2)
/* klibc was better */
Total coverage. (Score:4, Interesting)
A thousand AI eyes makes bugs shallow.
Ultimately is this not all to the good ? (Score:4, Insightful)
Bugs (in new code) are hopefully being added more slowly than bugs found and squashed in existing code - so surely the number of remaining bugs will drop. Hopefully those running bug finding AIs are not keeping some remote exploitable bugs to themselves; I would not be surprised if government agencies were doing this.
Re: (Score:3)
You left out the category of new bugs created by the patches and "upgrades", especially for new features.
Which somehow made me think of an even worse race condition, though I'm not sure how to describe the level of abstraction here... If the AI search for bugs goes to a certain depth, and assuming there are no bugs in the search algorithms, then what happens when the AI gets a bit smarter and searches a bit more deeply?
Long time ago when I was first studying computer security, but I remember two fundamental
Re: (Score:2)
Government agencies have been caught red-handed weaponizing critical vulnerabilities and keeping that to themselves. Criminals managed to hack some and sell the vulnerabilities off on the black market, forcing disclosure for remediation.
Re: (Score:2)
Presumably if enough people are running different AI stuff, sooner or later someone else will find whatever bug you are trying to abuse.
Unbalancing the arms race? (Score:5, Interesting)
AI is routinely pitched as a tool that will make software development faster, but it is also making vulnerability discovery faster.
That was my first thought and I'm glad the story mentioned it. But I'm concerned that AI may tilt the table in favour of bad actors. Mightn't it lower the bar when it comes to expertise at finding and exploiting vulnerabilities, while simultaneously increasing the burden on already over-worked programmers?
In other words, does AI help black-hat hacking organizations more than it helps good-guy programmers and maintainers? I'm thinking of how a hammer is more easily used to destroy than to create, but IANAP so I don't have a good sense of this.
Re:Unbalancing the arms race? (Score:4, Interesting)
Long term, however, once this initial tsunami is worked through then hopefully more of these types of bugs will be found before new code ships, making it harder to find critical vulnerabilities in the wild.
Re: (Score:2)
That will only be true if "good-guy" programmers decide against using tools that can help them address problems as fast as or faster than the bad guys can find them.
Re: (Score:2)
As this site is ready testament towards, there are a lot of "open source" types who have their ass firmly planted on their face about AI. "It's a fad", "it's a gimmicky chatbot", "It's not real intelligence".
Yeah, OK. But it is a real tool, and one which is increasingly showing its value for... literally everything. Get onboard or get left behind.
Re: (Score:3)
That was my first thought and I'm glad the story mentioned it. But I'm concerned that AI may tilt the table in favour of bad actors. Mightn't it lower the bar when it comes to expertise at finding and exploiting vulnerabilities, while simultaneously increasing the burden on already over-worked programmers?
In other words, does AI help black-hat hacking organizations more than it helps good-guy programmers and maintainers? I'm thinking of how a hammer is more easily used to destroy than to create, but IANAP so I don't have a good sense of this.
Bouncing code off of AI is being baked into software lifecycles and will result in fewer bugs across the board. For open source you will have more people armed with AI finding even more bugs (while annoying the shit out of maintainers). The interesting thing about these models you can run them over and over and over again and sometimes different bugs fall out so the more AI eyeballs the better. Still I expect ultimately the computational cost of finding any remaining bugs to work against attackers.
On the
Re: (Score:3)
You can pretty easily fix the "annoying the shit out of maintainers" problem by having modern code commit/merge management practices.
You know, something I'd think "open source maintainers" would be pretty readily willing to do. Not only is it best practice (and has been a long time) to gate PR merge on CI completion, but it's trivially easy to have regression and e2e testing in place, as well as require specific artifacts in the commit as evidence that it "works on my machine".
Re: (Score:2)
The problem is the old code that isn't getting updated - operating systems no longer receiving updates, obsolete firmware on hardware still in use, perpetually licensed software n
If canonical is so worried (Score:2)
about the volume of work, they should be using LTS or SLTS kernels, instead of bespoke Kernels. For instance, Ubuntu 26.04 LTS uses kernel 7.0 (not LTS, and certainly not SLTS), which means canonical engineers have to do all the backporting and patching work...
Anywho. Good luck to canonical and their users.
Re:If canonical is so worried (Score:4, Interesting)
As someone who lives in the world of enterprise Linux distributions there's a couple of reasons for this.
One, the release timeline and lifecycle of a particular LTS distro like Ubuntu or RHEL pretty much never lines up with that of the upstream kernel maintainers. With an ELS add-on, you can get about 15 years of critical CVE support for RHEL. Upstream will have moved on a long time before that.
Two, these distros do their own QA and release management. A bug that they have reported will almost always get patched upstream, but it may not end up in the same kernel release branch. It's at the whim of the kernel maintainers. They also have to be concerned with what IS going into a particular kernel release branch. Does it break something? Does it change the behavior of something in an undesirable way? Tell Linus he's wrong about something and a particular patch is bad and you'll get cursed and insulted.
It's just easier to do your own release management.
Re: (Score:3)
Whether or not one lives in the enterprise kernel world (I do as well), a glance at the mainstream kernel's changelog indicates we're all largely relying on those companies' employees. Most patches are coming from people who are paid by Red Hat et. al.
Re: (Score:2)
straw man. no one is "worried" except you
Considering I use MacOS 15.8, bootcamp into Windows 10 for gaming, and nowadays my only linuxes are on my NAS (DSM 7.3) and 2 VMs, one with TAILS and one with Kali, I am not worried in the slightest about what happens with Canonical...
"It's a fad" (Score:2)
I've got to wonder about the people who are (still, even/particularly on this site) saying AI is not capable or useful when it's able to do things like this.
Those of you who still think AI is a glorified chatbot: what's your rationale?
Re: (Score:2)
Re: (Score:2)
What to do? (Score:2)
Ironic (Score:1)
Same Bugs as Before? (Score:2)
Arms race (Score:1)
Ah, you've discovered the arms race.
And yes, all you can do is complain about it. You actually can't opt out of it. But please, carry on ...
No, not overnight. (Score:3)
ha nope (Score:2)