> The scary part? This attack vector is perfect for developers. We download and run code all day long. GitHub repos, npm packages, coding challenges. Most of us don't sandbox every single thing.
Embedded into this story about being attacked is (hopefully) a serious lesson for all programmers (not just OP) about pulling down random dependencies/code and just yolo'ing them into their own codebases. How do you know your real project's dependencies also don't have subtle malware in them? Have you looked at all of them? Do you regularly audit them after you update? Do you know what other SDKs they are using? Do you know the full list of endpoints they hit?
How long do we have until the first serious AI coding agent poisoning attack, where someone finds a way to trick coding assistants into inserting malware while a vibe-coder who doesn't review the code is oblivious?
Sadly, this is a lesson that we should have learned some time ago. But from our past failure to learn, we can reliably predict that people will continue avoiding learning.
Supply side attacks are real, and they're here. Attackers attack core developers, then get their code into repositories. As happened this year to the npm package eslint-config-prettier, and last year to the Cyberhaven Chrome extension. Attackers use social engineering to get developers to hand over control of lesser used packages, which they then compromise. As happened in 2021 with the npm package ua-parser-js, and separately with the Chrome extension The Great Suspender. (I'm picking on Chrome because I wanted examples that impact non-developers. I'm only picking on npm because it turned up quickly when I looked for examples.)
The exact social engineering attack described by the OP is also not new. https://www.csoonline.com/article/3479795/north-korean-cyber... was published last year, and describes this being used at scale by North Korea. Remember, even if you don't have direct access to anything important, a sophisticated attacker may still find you useful as part of a spearphishing campaign aimed at someone else. Because a phishing attack that actually comes from a legitimate friend's account may succeed, where a faked message would not. And a company whose LinkedIn shows real developers, is more compelling than one without.
I go to the repo and get a feel for how popular, how recent, and how active the project is. I then lock it and I only update dependencies annually or if I need to address a specific issue.
Risk gets managed, not eliminated. There is no one "correct" approach as risk is a sliding scale that depends on your project's risk appetite.
None of those methods are even remotely reliable for filtering out bad code. See e.g. this excellent write up on how many methods there are to infect popular repos and bypass common security approaches [1] (including Github "screening"). The only thing that works nowadays is sandbox, sandbox, sandbox. Assume everything may be compromised one day. The only way to prevent your entire company (or personal life) from being taken over is if that system was never connected to anything it didn't absolutely require for running. That includes network access. And regarding separation, even docker is not really safe [2]. VM separation is a bit better. Bare metal is best.
You'd have to write the standard libraries and OS as well. Not that it can't be done, but let's just say that people who tried that did not fare well in the mental health department.
If you go down this road there also isn't really much need to write anything yourself. After all, you'll be much more likely to include exploitable bugs yourself once you start messing with things you are not an expert in. So neither way is a good solution.
Yup, for sure. But part of risk management is considering how likely a failure mode might be and if it's really worth paying to mitigate. Developers are really good at imagining failure modes, but often not so good at estimating their likelihood/cost.
I have no "hard rules" on how to appraise a dependency. In addition to the above, I also like to skim the issue tracker, skim code for a moment to get a feel for quality, skim the docs, etc. I think that being able to quickly skim a project and get a feel for quality, as well as knowing when to dig deeper and how deep to dig are what makes someone a seasoned developer.
And beware of anyone who has opinions on right vs. wrong without knowing anything about your project and it's risk appetite. There's a whole range between "I'm making a microwave website" and "I'm making software that operates MRIs."
Of course. A malware-infected dependency has motivation to pay for GitHub stars and fake repo activity. I would never trust any metric that measures public "user activity". It can all be bought by bad actors.
Would totally depend on the project and what kinds of risks were appropriate to take given the nature of the project. But as a general principal, for all kinds of development: "Bringing in a new dependency should be A Big Deal." Whether you are writing a toy project or space flight avionics, you should not bring in unknown code casually. The level of vetting required will depend on the project, but you have to vet it.
Is there a market for a distributed audit infra with attestations? If I can have ChatGPT audit a file (content hash) with a known-good prompt, and then share the link as proof of the full conversation, would this be useful evidence to de-risk?
If each developer can audit some portion of their dep tree and reuse prior cached audits, maybe it’s tractable to actually get “eyeballs” on every bit of code?
Not as good as human audit of course, but could improve the Pareto-frontier for cost/effectiveness (ie make the average web dev no-friction usecase safer).
I think there is, definitely, and that will be a solid route out of this supply chain debacle we find ourselves in.
It will have to involve identity (public key), reputation (white list?), and signing their commits and releases (private key). All the various package managers will need to be validating this stuff before installing anything.
Then your attestation can be a manifest "here is everything that went into my product, and all of those components are also okay.
> Secondly, if you re-analyzed the same malicious script over and over again it would eventually pass inspection, and it only needs to pass once
You’d need some probabilistic signal rather than a binary one. Eg if some user with zero reputation submits a single session saying “all good”, this would be a very weak signal.
If one of the Python contributors submits a batch of 100 reasoning traces all showing green, you’d be more inclined to trust that. And of course you would prefer to see multiple scans from different package managers, infra providers, and OS distributions.
That's why from my perspective, almost everything is f'd up in tech at this point.
Any update I may do to any project dependencies I have on my workstation? Either I bet, pray and hope that there's no malicious code in these.
Either I have an isolated VM for every single separate project.
Either I just unplug the thing, throw it in the bin, and go make something truly lucrative and sustainable in the near future (plumber, electrician, carpenter) that let's me sleep at night.
>Either I have an isolated VM for every single separate project.
That's not too hard to do with devcontainers. Most IDEs also support remote execution of some kind so you can edit locally but all the execution happens in a VM/container.
What I'm wondering about is, if you have lots of dependencies, like in the hundreds or thousands, idk how many npm packages usually can have for the average web dev project, how do you even audit all of that manually? Sounds pretty infeasible? This is not to say we should not worry about it, I'm just genuinely curious what do you do in this situation? One could say well don't get that many dependencies to begin with, but the reality of web dev projects nowadays for instance, is that you get alot of dependencies that are hard to check manually for insecurities.
Some developers accept it as a reality, but it's only a reality if you're doing it. I think the time to figure this out is before your project gets a mess of hundreds or thousands of dependencies. Bringing in even a single dependency should be a big deal. Something you agonize over. Something you debate and study. Something you don't do unless you really, really mean it. Certainly not a casual decision. Some languages/environments make it too easy. Easy like: A single command line command and you now have a dependency. Total madness!
A good candidate is niche frameworks.. where most of the data about usage are limited to few domains and not many sources. Could maybe have middling popularity (popular lang, strong representation on its focused problem). Recent examples of this in my experience: Kafka connector and PowerPoint lib (marp). Few sources and the llm hallucinated on these. So maybe a poisoned source would be more likely to pop up in llm suggestions
Many of these projects are set to compile only on the latest OS' which makes sandboxing even more difficult and impossible on VM, which is actually the red flag
So I sandbox but I don't get to the place of being able to run it
so they can just assume I'm incompetent and I can avoid having my computer and crypto messed up
I develop everything on Linux VMs, it has desktop, editors, build tools...
It simplifies backups and management a lot.
Host OS does not even have Browser or PDF viewer.
You could have a command like "python3.14" that will run that version of Python in a Docker container, mounting the current directory, and exposing whatever ports you want.
This way you can specify the version of the OS you want, which should let you run things a bit more easily. I think these attacks rely largely on how much friction it is to sandbox something (even remembering the cli flags for Docker, for example) over just running one command that will sandbox by default.
> How long do we have until the first serious AI coding agent poisoning attack, where someone finds a way to trick coding assistants into inserting malware while a vibe-coder who doesn't review the code is oblivious?
I mean we had Shai-Hulud about a week ago - we don't need AI for this.
Embedded into this story about being attacked is (hopefully) a serious lesson for all programmers (not just OP) about pulling down random dependencies/code and just yolo'ing them into their own codebases. How do you know your real project's dependencies also don't have subtle malware in them? Have you looked at all of them? Do you regularly audit them after you update? Do you know what other SDKs they are using? Do you know the full list of endpoints they hit?
How long do we have until the first serious AI coding agent poisoning attack, where someone finds a way to trick coding assistants into inserting malware while a vibe-coder who doesn't review the code is oblivious?