Awesome job with the simplicity, gonna play with it. Have you tried using yaml as the format with the models instead of JSOn? Feel like you'll use far fewer tokens to describe the same thing. Perhaps it's a bit more forgiving as well.
EDIT: Just tried using the decorator to output a fairly complex pydantic model and it failed with "magentic.chat_model.openai_chat_model.StructuredOutputError: Failed to parse model output. You may need to update your prompt to encourage the model to return a specific type."
I typically try to give examples in the pydantic Config class, perhaps those could be piped in for some few-shot methods, and also have some iteration if the model output is not perfectly parseable to correct the output syntax..
Yes, I'm working on allowing few-shot examples to be provided as part of defining the prompt-function, which should help in cases like this. Unfortunately from my testing just now it appears that OpenAI ignores examples added to the model config.
In the meantime, have a look at the ValidationError traceback which might highlight a specific field that is causing the issue. Some options to resolve the issue might be: the type for this field could be made more lenient (e.g. str); the `Annotated` type hint could be used to give the field a description to help correct the error [0]; the field could be removed. You could also try using gpt-4 by setting the env var MAGENTIC_OPENAI_MODEL [1].
If none of these help resolve it or it appears to be an issue with magentic itself please file a github issue with an example. Comments on how to improve error messages and debugging are also welcome! Thanks for trying it out.
EDIT: Just tried using the decorator to output a fairly complex pydantic model and it failed with "magentic.chat_model.openai_chat_model.StructuredOutputError: Failed to parse model output. You may need to update your prompt to encourage the model to return a specific type."
I typically try to give examples in the pydantic Config class, perhaps those could be piped in for some few-shot methods, and also have some iteration if the model output is not perfectly parseable to correct the output syntax..