Detecting whether something is written by an AI is a waste of time. Either someone will sign the statement as their own or they won't (and it should be treated as nonsense).
People lie. People tell the truth. Machines lie. Machines tell the truth. I bet our ability to detect when a person is lieing isn't any better than 50% either.
What matters is accountability, not method of generation.
In relevant studies, people attempt to discriminate lies from truths in real time with no special aids or training. In these circumstances, people achieve an average of 54% correct lie-truth judgments, correctly classifying 47% of lies as deceptive and 61% of truths as nondeceptive. [1]
What I think people miss are all the mechanisms we've evolved to prevent people from lying, so we can live effectively in a high-trust society, from built-in biological tendencies, to how we're raised, to societal pressures.
"People lie too" but in 95% of cases they don't. If someone on Hacker News say they prefer Zig to Rust or that they liked the Dune movie, they're likely telling the truth. There's no incentive either way, we've just evolved as social creatures that share little bits of information and reputation. And to lie, yes, and to expose the lies of others, but only when there's a big payoff to defect.
If you had a friend that kept telling you about their trips to restaurants that didn't actually exist, or a junior developer at work that made up fictional APIs when they didn't know the answer to a question, you'd tell them to stop, and if they kept at it you probably wouldn't care to hang out with them. ChatGPT seems to bypass those natural defenses for now.
Most people think they are hard to deceive. But I see plenty people here on HN with confidently wrong beliefs about how ChatGPT works, that they've gotten from asking ChatGPT about itself. It's not intuitive for us that ChatGPT actually knows very little about how itself works. It even took humanity a while to realize that "How does it feel like my body works" isn't a great way to figure out biology.
> If you had a friend that kept telling you about their trips to restaurants that didn't actually exist, or a junior developer at work that made up fictional APIs when they didn't know the answer to a question, you'd tell them to stop, and if they kept at it you probably wouldn't care to hang out with them. ChatGPT seems to bypass those natural defenses for now.
While this is a reasonable thing to hope for, I'd like to point out that former British Prime Minister Boris Johnson has been making things up for his entire career, repeatedly getting into trouble for it when caught, and yet somehow he managed to keep failing upwards in the process.
So even in humans, our defences assume the other person is capable of recognised the difference between truth and fiction; when they can't — and it is my opinion that Johnson genuinely can't tell rather than that he merely keeps choosing to lie, given how stupid some of the lies have been — then our defences are bypassed.
People like Johnson and Trump are exactly the exceptions that prove the rule. When they act like they do, they are reviled for it by most because of how aberrant their behavior is. They fail up because that revulsion is politically useful.
For humans there's a social cost to wild lies and fabrications, even if one is otherwise generally reliable. I would probably consider a person who is wrong 50% of the time but can reason about how they came to a conclusion and the limits of their knowledge/certainty to be more reliable than someone who is correct 90% of the time but lies/fabricates/hallucinates the other 10% of what they say.
If a human acting in good faith is pressed for the evidence for something they said that is untrue, they will probably give a hazy recollection of how they got the information ("I think I read it in a NYT article", etc). They might be indignant, but they won't fabricate an equally erroneous trail of citations.
ChatGPT produces some shockingly good text, but the rate of hallucinations and its inability to reliably reason about either correct or incorrect statements would be enough to mark a human as untrustworthy.
The fact that LLMs can produce plausible, authoritative text that appears well evidenced, and can convincingly argue its validity regardless of any actual truth does however mean that we might be entering an era of ever more accessible and convincing fraud and misinformation.
> ChatGPT produces some shockingly good text, but the rate of hallucinations and its inability to reliably reason about either correct or incorrect statements would be enough to mark a human as untrustworthy.
It's not even the rate, which is troubling enough. It's the kinds of things it gets wrong too. For instance, you can say to ChatGPT, "Tell me about X" where X is something you made up. Then it will say "I don't know anything about X, why don't you tell me about it?" So you proceed to tell it about X, and eventually you ask "Tell me about X" and it will summarize what you've said.
Here's where it gets strange. Now you start telling it more things about X, and it will start telling you that you're wrong. It didn't know anything about X before, now all of a sudden it's an authority on X, willing to correct actual an actual authority after knowing just a couple things.
It will even assert its authority and expertise: as "As a language model, I must clarify that this statement is not entirely accurate". The "clarification" that followed was another lie and a non sequitur. Such clarity.
What does ChatGPT mean by "As a language model, I must clarify". Why must it clarify? Why does its identity as "a language model" give it this imperative?
Well, in actuality it doesn't, it's just saying things. But to the listener, it does. Language Models are currently being sold as passing the bar, passing medical exams, passing the SAT. They are being sold to us as experts before they've even established themselves. And now these so called experts are correcting humans about something it literally said it has no knowledge.
If a 4-year old came up to you and said "As a four year old, I must clarify that this statement is not entirely accurate", you would dismiss them out of hand, because you know they just make shit up all the time. But not the language model that can pass the Bar, SAT, GRE, and MCATS?. Can you do that? No? Then why are you going to doubt the language model when it's trying to clear things up.
Language models are going to be a boon for experts. I can spot the nonsense and correct in real time. For non experts, they when LLMs work they will work great, and when they don't you'll be left holding the bag when you act on its wrong information.
I'm concerned that they'll prevent non-experts from becoming experts. Most of my learning is done through observation: if I'm observing an endless stream of subtly-wrong bullshit, what am I learning?
My wife and I were just talking about this exact thing earlier today. I was using an AI to assist in some boring and repetitive “programming” with yaml. It was wrong a good chunk of the time, but I was mostly working as a “supervisor.”
This would have been useless to the point of breaking things if a junior engineer had been using it. It even almost tripped me up a few times when it would write something correct, but with a punctuation in the wrong place. At least it made the repetitive task interesting.
Seems that this depends on the definition of “lie.” It might be true that humans aren’t trying to deceive others 95% of the time, just like it’s true that ChatGPT isn’t _trying_ to deceive people 100% of the time. But both of them have a habit of spreading a ton of misinformation.
For humans, there’s simply an alarming percent of the time they present faulting memories as facts, with no one questioning them and believing them entirely at face value. You mentioned Hacker News comments. I’ve been unsettled by the number of times someone makes a grand claim with absolutely no evidence, and people respond to it like it’s completely true. I sometimes think “well, that’s a serious claim that they aren’t presenting any evidence for, I’m sure people will either ignore it or ask for more evidence,” and then return to the topic later and the comments are all going, “Amazing, I never new this!”
Often when one looks it up, there seems to be no evidence for the claim, or the person is (intentionally or not) completely misrepresenting it. But it takes mere seconds to make a claim, and takes a much longer time for someone to fact check it (often the topic has fallen off the main page by then).
This is all over the internet. You’d think “don’t automatically believe grand claims made by strangers online and presented with zero evidence” would be common sense, but it rarely seems to be practiced. And not just the internet; there are plenty of times when I’ve tracked down the primary sources for articles and found that they painted a very different story from the one presented.
I actually think people have been more skeptical of ChatGPT responses than they have about confident human created nonsense.
> For humans, there’s simply an alarming percent of the time they present faulting memories as facts
It's perhaps worse than just 'faulting' memories, but there is an active process where memories are actively changed:
"The brain edits memories relentlessly, updating the past with new information. Scientists say that this isn't a question of having a bad memory. Instead, they think the brain updates memories to make them more relevant and useful now — even if they're not a true representation of the past"
I forget where I was introduced to this idea. In that source, I recall (FWIW!) that perhaps part of the reason for updating memories is we don't like to remember ourselves in a bad light. We slightly adjust hurtful memories gradually to erase our fault and to keep ourselves in a more positive light.
People believe lies, often. That's just an undeniable fact of human nature. AIs can produce lots of plausible lies very quickly, much more quickly and at much greater scale than humans could. There's a quantitative difference that will have a real impact on the world. Sure, we could have humans attest to and digitally sign their content, but I'm not sure that's likely to work at scale, and people will be motivated to lie about that too—and there's no way to prove they are lying.
Pretty sure there will be a cost to those people eventually for believing lies. Over time, evolution will take care of it.
By which I don’t just mean survival of the fittest people / brains, but also survival of better memes (in the Dawkins sense of the word) and better approaches for bullshit detection, and diminishing of worse approaches.
Machines lie very effectively. Machines plainly have more resources, while people give all kinds of metadata that they're lying. It used to be that if someone had a lot of details ready at hand they were probably truth-tellers, since details are tiresome to fabricate. But ChatGPT can talk math-into-code with me for an hour, occasionally asking for clarification (which makes me clarify my thinking) and still lead me to a totally nonsensical path, including realistic code that imports libraries I know to be relevant, and then relies on classes/functions that don't exist. Fool me once, shame on me.
Machines don't lie. There is no intention of misleading someone behind wrong statements from a machine.
I could lie to you while still stating something that is factually correct but intentionally misleading.
Imagine me standing in front of the White House, taking my phone and calling the Meta or Google press bureau. I could say, I am calling from the White House (factually correct) but would imply, that I am calling in an official capacity. And while I know that this is a contrived example, I hope it clarifies my point of intentional deception being the identifying element of a lie.
And this intentional misleading is what I deny machines to exhibit.
Still the quote authoritative sounding texts that AI produce (or human text farm monkeys for that matter) force us to think about how we evaluate factfulness and how we qualify sources. Not an easy task before AI and by far even more difficult after AI imho.
ChatGPT generates text based on input from a human who takes the output and does something with it. The machine is not really the one in control and lying or telling the truth. It's the person that does something with it.
You're right accountability but the issue goes even as far as copyright eligibility - only human authored works are eligible for copyright or patent protection so being able to detect ai writing is critical to keeping intellectual property from being flooded with non human generated spam that would have large corporations own pieces of potential human thinking in the future.
> What matters is accountability, not method of generation.
Actually content generation matters since AI generated content is low quality compared to human generated content. When is not blatantly false and misleading.
Yeah it's a binary proposition (AI or human) and if the success rate is 50/50 then it's pure chance and it means we likely can't identify AI vs human-generated at all.
Which is fine. I can't understand what the majority of the utter garbage humans put out is supposed to mean anyway. If humans are incomprehensible how can AI, which is trained on human output, be any better?
That does not follow, and how is trust even codified? Are you keeping a list of people and permissions?
Fundamentally though most of our society depends on a high degree of trust and stops functioning almost immediately if that trust becomes significantly tarnished. Going 'trustless' in human communities probably looks like small communities with strong initial distrust for strangers.
Copy paste is easily detected and removed. Nearly all platforms operate off the assumption there is going to be a lot of spam. They do not have a single tool to deal with decent text generation.
People lie. People tell the truth. Machines lie. Machines tell the truth. I bet our ability to detect when a person is lieing isn't any better than 50% either.
What matters is accountability, not method of generation.