You've just described a Playdate. It's a game console not a computer but it pretty much is designed how you've suggested. It also uses the same Sharp Memory LCD.
it's the sensible way to design a system around this display, but the playdate isn't what i'm looking for because it doesn't have a keyboard and also it's kind of bulky