These are kibi-meters

>How long is a piece of string, when the string is 4km long?

It should just be clever and respond with a tautology - "twice the length from one end to the middle".

We only sell by the byte here!

https://youtu.be/17NKos-7GCo Reminds me of this comedy gold

Of course you now must ask it

> How long is a piece of string, when the string is 4Kim long?

...or even just Km. I can't, as it's being slashdotted

Does it even differentiate between lowercase and uppercase letters? Results for me were:

> How long is a piece of string, when the string is 4km long?

4096 meters.

> How long is a piece of string, when the string is 4Kim long?

4Kim

> How long is a piece of string, when the string is 4Km long?

4096 meters.

after that I got the "Something went wrong, please try again." error.

When it comes to tokenization, yeah, it does differentiate. But its training set is certain to have many examples of different capitalizations and misspellings, so it can understand them, too. Which is usually what you want...

GPT-4 really takes this to eleven. For example, it can correctly parse and process this request: "Tll m shrt str bt ncrns; 10 sntncs. Wrt t l vwls prprl."

though unless you've disabled sampling it will be difficult to determine how prompts affect the output, these could just be due to RNG

just run the 13b model 4bit quantized locally, it's already better than the 7b-8bit and you can turn down the temperature to 0 to get repeatable results.

I'd expect BingChat to give that exact answer tbh.