ChatGPT is instructed to roleplay a cheesy cheery bot and so it responds accordingly, but it (and almost any LLM) can be instructed to roleplay any sort of character, none of which mean anything about the system itself.
Of course an AGI system could also be instructed to roleplay such a character, but that doesn't mean it'd be an inherent attribute of the system itself.
It (at least LLMs) can reproduce similar display of having these emotions, when instructed so, but if it matters or not depends on the context of that display and why the question is asked in the first place.
For example, if i ask an LLM to tell me the syntax of the TextOut function, it gives me the Win32 syntax and i clarify that i meant the TextOut function from Delphi before it gives me the proper result, while i know i'm essentially participating in a turn-based game of filling a chat transcript between a "user" (with my input) and an "assistant" (the chat transcript segments the LLM fills in), it doesn't really matter for the purposes of finding out the syntax of the TextOut function.
However if the purpose was to make sure the LLM understands my correction and is able to reference it in the future (ignoring external tools assisting the process as those are not part of the LLM - and do not work reliably anyway) then the difference between what the LLM displays and what is an inherent attribute of it does matter.
In fact, knowing the difference can help take better advantage of the LLM: in some inference UIs you can edit the entire chat transcript and when finding mistakes, you can edit them in place including both your requests and the LLM's response as if the LLM did not do any mistakes instead of trying to correct it as part of the transcript itself, thus avoiding the scenario where the LLM "roleplays" as an assistant that does mistakes you end up correcting.
Why would it care to stay alive? The discussion is pretty pointless as we have no knowledge about alien intelligence and there can be no arguments based on hard facts.
Any form of AI unconcerned about its own continued survival would be just be selected against.
Evolutionary principles/selection pressure applies just the same to artificial life, and it seems pretty reasonable to assume that drive/selfpreservation would at least be somewhat comparable.
That assumes that AI needs to be like life, though.
Consider computers: there's no selection pressure for an ordinary computer to be self-reproducing, or to shock you when you reach for the off button, because it's just a tool. An AI could also be just a tool that you fire up, get its answer, and then shut down.
It's true that if some mutation were to create an AI with a survival instinct, and that AI were to get loose, then it would "win" (unless people used tool-AIs to defeat it). But that's not quite the same as saying that AIs would, by default, converge to having a drive for self preservation.
Despite the false advertising in the Tears for Fears song, everybody does _not_ want to rule the world. Omohundro drives are a great philosophical thought experiment and it is certainly plausible to consider that they might apply to AI, but claiming as is common on LessWrong that unlimited power seeking is an inevitable consequence of a sufficiently intelligent system seems to be missing a few proof steps, and is opposed by the example of 99% of human beings.
> Instrumental convergence is the hypothetical tendency of most sufficiently intelligent, goal-directed beings (human and nonhuman) to pursue similar sub-goals (such as survival or resource acquisition), even if their ultimate goals are quite different. More precisely, beings with agency may pursue similar instrumental goals—goals which are made in pursuit of some particular end, but are not the end goals themselves—because it helps accomplish end goals.
'Running the planet' does not derive from instrumental convergence as defined here. Very few humans would wish to 'run the planet' as an instrumental goal in the pursuit of their own ultimate goals. Why would it be different for AGIs?
I think it's academic because I suspect we're much further from AGI than anyone thinks. We're especially far from AGI that can act in physical space without human "robots" to carry out its commands.
That's an interesting formulation. I'd actually be quite worried about a Manna-like world, where we have AGI and most humans don't have any economic value except as its "remote hands".
...Well, why would aliens care, when they take over the planet? Or the Tuatha De Danan come back and decide we've all been very wicked? Because right now, those are just about as likely as AGI taking over.
There's a bit of a circular argument here - even if we human always assign intrinsic value to ourselves and our kin, I don't see a clear argument that human capabilities will have external value to the economy at large.
If you "unwind" all the complexities in modern supply chains, there are always human people paying for something they want at the leaf nodes.
Take the food and clothing industries as obvious examples. In some AI singularity scenario where all humans are unemployed and dirt poor, does all the food and clothing produced by the automated factories just end up in big piles because we naked and starving people can't afford to buy them?
There's nothing definitional about the economy being driven by human need. In a future scenario where there are superintelligent AIs, there's no reason why they wouldn't run their own economy for their own needs, collecting and processing materials to service each other's goals, for example of space exploration.
For economic purposes, "the economy" also includes corporations and governments.
Corporations and governments have counted amongst their property entities that they did not grant equal rights to, sometimes whom they did not even consider to be people. Humans have been treated in the past much as livestock and guide dogs still are.
>Even if they automate all our current jobs uniquely human experiences will always be valuable to us and will always have demand.
I call this the Quark principle. On DS9, there are matter replicators that can perfectly recreate any possible drink imaginable instantly. And yet, the people of the station still gather at Quark's and pay him money to pour and mix their drinks from physical bottles. As long as we are human, some things will never go away no matter how advanced the technology becomes.
Yeah that feels like the right approach to enable the code generation to test itself which then becomes a matter of specification and functionality definition instead of worrying about code quality.
Are you doing all of this in cursor or something like Claude code?
There is a common theme of "the end result is all that matters" but there are pretty big long term repercussions of design and implementation choices. For side projects and POC experiments this feels like the right approach but the risk flip flops for large scale projects that have the more risk associated with them. Maybe it is just through testing and validation checks?
I posit that most software isn't large scale projects.
Certainly mine isn't. But I've still generated hundreds of thousands of lines of code.
But no one will ever read them. And solid engineering defines the interfaces between them. So we specify the ins and outs and let the rest take its course.
> there are pretty big long term repercussions of design and implementation choices
At least this part I am still specifying. It doesn't get to choose its own technologies. It generally includes the architecture in the plan that I review.
Me too. I specify everything like that. Which database to use, which style of database to use, etc. the sort of thing a Team Leader would pick (after consulting the team, of course).
I've been coding since the 8-bit days.
With the added benefit I can specify, "let's try using this stack this time." I haven't got to spend two months learning it to get to MVP.
Not trying to diminish the work at all because it works well but I hate this UX pattern. There is likely a reason this hasn't been done. If it wasn't for the pick -> scroll -> place instruction I would have no idea how to use it.
It’s possible, but maybe unlikely. A tower is hard-wired to fiber, and immensely cheaper than a satellite, per pound. Satellites are always going to be more expensive than land based equipment, and cost is critical here, plus fundamentally more powerful with better data connections to the trunk.
> Put simply, it's all about electrons. For data storage and transfer to happen on any device — smartphone, desktop PC or internet server — you need electrons. And while these particles aren't exactly massive, they do have weight: approximately 9.1 x 10^-31 kg. Take that and apply it to an ordinary email, which comes in at about 50 kilobytes: You need 8 billion electrons. Sounds like a lot but only comes in at two ten-thousandths of a quadrillionth of an ounce.
> Seitz scaled this up to determine the weight of all internet traffic and got 50g, or the weight of one strawberry. Applied to all the stored information online, which is around 5 million terabytes, the number is just 0.2 millionths of an ounce
reply