Ask chatgpt to implement some of the things you worked on the last few months. I was very skeptical too until I tried this.
Here are some sample prompts that I tried and got full working code for:
- "write pytorch code to train a transformer model on common crawl data and an inference service using fastapi"
- "write react native code for a camera screen that can read barcodes and look them up using an API and then display info for matched results in a widget under the camera view"
- "write react code for a wedding website"
- "write code to deploy a django website on GCP using terraform and kubernetes"
- "how do I dockerize the app, it uses pytorch and faiss, also push it to a container registry"
- "implement a GPT style transformer model in pytorch", "write a training loop for it with distributed support and fp16"
- "how would you implement reinforcement learning with human feedback (RLHF)", "can you implement it in pytorch"
- "write code to compress a model trained in pytorch and export for inference on iOS"
- "how would you distill a large vision model to a small one"
- "what are the best CV architectures for mobile inference?"
For all of these it gave me code that was 95% usable, all in under 15 minutes, and which would have taken me a week or two to do on my own.
You know what's funny? I just asked ChatGPT to implement those exact same things and it shat all over itself producing embarrassing nonsense that won't compile, let alone do what they're expected to do. Bugs and incomplete code everywhere.
You'd have a much better time just Googling those asks and re-using a working examples from SO or GitHub. Which is ironic, given how ChatGPT is supposedly trained on those exact things.
I'm wondering how come we're both getting such vastly different results. Maybe your bar is just lower than mine? I don't know. I'm honestly shocked at the contrast between the PR given to ChatGPT, and the results on the ground.
Try this simple ask (the results of which you'll find plastered everywhere): produce a Python function that decodes a Base64 string and prints the results, without using any "imports" or libraries. Every single output I got back was embarrassing garbage, and I gave it something like 15 shots.
I tested the Base64 thing with GPT4 and it produces code that does seem to work. There have been other tasks I've given it (C++, Clojure, JS) that it doesn't get on the first try or in some cases doesn't get at all though. One task I tried in C++ it kept going in circles and ignoring requirements from prior prompts and I tried numerous ways to prompt it.
All that in mind, I'd be lying to say I'm not more than a little concerned with the progress from 3.5 -> 4. I'm only two years into my career and my fingers are crossed that it won't significantly impact the market for devs for as long as possible.
How can a large language model achieve that?