It has no idea who these guys are. It thinks they are the beatles, the doors. If you probe enough, it'll say it's IBM cofounders. In a way, it kinda sees that these are mid-1900s folks with cool haircuts, but it doesn't recognize anything. If you probe on the F the model in question becomes convinced it's the Ford racing team with a detailed explanation of two brothers in the photo, etc.
The creation of autoregressive next token predictors is very cool and clearly has and will continue to have many valuable applications, but I think we're missing something that makes interactions with users actually shape the trajectory of its own experience. Maybe scaffolding + qlora solves this. Maybe it doesn't
By this, what I mean is. Take an image of this: https://en.wikipedia.org/wiki/Traitorous_eight#/media/File:T..., change the file name to something like image.jpg and pass it into Qwen 3 4B, 8B, 30B and look at the responses you get:
It has no idea who these guys are. It thinks they are the beatles, the doors. If you probe enough, it'll say it's IBM cofounders. In a way, it kinda sees that these are mid-1900s folks with cool haircuts, but it doesn't recognize anything. If you probe on the F the model in question becomes convinced it's the Ford racing team with a detailed explanation of two brothers in the photo, etc.
The creation of autoregressive next token predictors is very cool and clearly has and will continue to have many valuable applications, but I think we're missing something that makes interactions with users actually shape the trajectory of its own experience. Maybe scaffolding + qlora solves this. Maybe it doesn't