HT to @wdormann here - somebody has backdoored the open source project XZ which has downstream impacts.
For example, although OpenSSH doesn’t use XZ, Debian patch OpenSSH and introduced a dependency which translates as the XZ changes introducing a sshd authentication bypass backdoor it appears.
One dude bothered to investigate in his free time about why ssh was running slow, so it was caught fairly early - i.e. hopefully before distros started bundling it.
As I said, the impact here will be very limited due to how quick it was caught. Everybody owes the finder a beer.
CISA advisory: Reported Supply Chain Compromise Affecting XZ Utils Data Compression Library, CVE-2024-3094 https://www.cisa.gov/news-events/alerts/2024/03/29/reported-supply-chain-compromise-affecting-xz-utils-data-compression-library-cve-2024-3094
Interesting find by @fuomag9 - the XZ repo person tried getting Ubuntu to update yesterday by filing a bug report https://bugs.launchpad.net/bugs/2059417
The Twilight zone time - a bug from 2015 comes back around in XZ incident, it appears https://github.com/google/sanitizers/issues/342
Back in 2022 a host of characters appeared and basically bullied the creator of the XZ project to hand it over to somebody else - at the time the guy cited mental health issues around not updating the project quickly.
At the time he was already talking about maybe handing over to the account who years later introduced the backdoor.
In mid 2023 said account introduced a change to Google’s OSS Fuzzer to weaken detection for XZ.
Somebody played a years long game of Jenga and lost.
Before everybody high fives each other, this is how the backdoor was found: somebody happened to look at why CPU usage had increased in sshd, and did all the research and notification work themselves. By this point the backdoor had been there for a month unnoticed.
https://mastodon.social/@AndresFreundTec/112180406142695845
I’ve made the joke before that if GCHQ aren’t introducing backdoors and vulns in open source that I want a tax refund. It wasn’t a joke. And it won’t be just be GCHQ.
Another two thoughts on XZ -
- sshd itself has no dependency on the XZ utils library. The streams got crossed in a way I don’t think anybody understood (except the threat actor).
- had that backdoor been performant with sshd, I don’t think anybody would have spotted it.
The way this played out opens a window of opportunity to go back and look at both issues.
Really good timeline of what is known to have happened so far. It looks like the rogue developer deliberately introduced a vulnerability in other package, too - I haven’t seen anybody else mention this.
Reading the dev’s GitHub history, they’ve been making changes to other open source projects too around compression. It also appears they/somebody involved has other accounts, too.
https://boehs.org/node/everything-i-know-about-the-xz-backdoor
How far the rabbit hole goes - back in 2021 they deliberately introduced an obvious vulnerability in the compression library libarchive. Nobody noticed. This is shipped in a ton of systems:
https://github.com/libarchive/libarchive/pull/1609
Whoever the threat actor is knows what they are doing as they’ve gone after chained dependencies around compression.
If anybody thinks this kind of thing is unique, it isn’t.
Example - CVE-2021-44529 in Ivanti Endpoint Manager. The cause?
Backdoor in open source code, was there for 7 years.
XZ Embedded Linux kernel module for IoT devices, 10 days ago had a change submitted to add Jia Tan as a maintainer.
https://lore.kernel.org/lkml/20240320183846.19475-2-lasse.collin@tukaani.org/
Linux kernel documentation: https://docs.kernel.org/staging/xz.html
Original maintainer of XZ repos has posted a short update:
https://tukaani.org/xz-backdoor/
HT @SamantazFox
Another weakening of XZ found: https://hachyderm.io/@danderson/112185746000358589
Also since there’s a lot going on here, up thread I mentioned a 2015 minor bug in Google’s OSS Fuzzer (security testing tool) - the threat actor deliberately introduced the bugged function into XZ, then used that to get an exception in OSS Fuzzer’s code to stop scanning of XZ.
I’ve just been looking at the actual backdoor for a few hours with greater minds than me, it’s incredibly complex - it basically piggy backs RSA key RCE inside sshd as a Trojan horse. Somebody/bodies spent $$ on this.
Also, to be super clear nobody should panic as the Postgres developer who found this basically caught it quick enough that almost no businesses or devices will be running the code.
So everybody should be chill about this specific issue as that guy saved everybody’s bacon.
To give an idea of the scale of OpenSSH usage, it’s absolutely huge, it dwarfs RDP by a huge margin (think ten times), and had this survived for a long period of time it would have been unbelievably bad.
@GossiTheDog It seems like deep supply chain attacks like this are going to be the thing for a while. This whole business is just totally depressing.