Issue #49 : Use the pydantic model type in the dspy Signature #51
Conversation
…irectory-of-fertilizer-label-inspection-data-for-testing-pipeline-performance' into 49-dspy-pydantic-schema
There was a problem hiding this comment.
WARNING about the observability code added that is related to Phoenix:
Tread with caution as it will break the pipeline if the endpoint given is not valid (it will keep trying to send the traces in an infinite loop).
Also, we forgot to remove everything related to the performance assessment from this PR. Therefore, all the changes in to the following files need to be removed from the PR:
- .gitignore
- performance_assessment.py
- test_data/*
Otherwise, looks good.
| ) | ||
|
|
||
| DSPyInstrumentor().instrument(tracer_provider=tracer_provider) | ||
|
|
There was a problem hiding this comment.
I recommend exercising caution when adding lines 8-17 to the main branch. If the endpoint isn’t valid (i.e., Phoenix isn’t set up), it WILL cause the entire pipeline to stop functioning (I had that happen to me this morning).
To mitigate this, I suggest either removing the observability code for now or commenting it out by default. We can uncomment it when we need to use traces for debugging.
Additionally, I’m unsure if this feature should have its own issue and PR, or if it’s acceptable to include it in this PR.
| test_logs/ | ||
| reports/ | ||
| test_data/ | ||
| performance_assessment.py |
There was a problem hiding this comment.
I don't think adding test_data/ and performance_assessment.py into the gitignore was intended (nor desirable).
There was a problem hiding this comment.
We forgot to remove this from this PR.
There was a problem hiding this comment.
Same here, test_data stuff should not be merged into main yet.
There was a problem hiding this comment.
idem - should not be merged into main yet.
This PR doesn't fully resolve #49 but rather serves as a stepping stone for the use of annotation in the Pydantic model in the future.
We now leverage the TypedChainOfThough for better integration of the pydantic schema in DSPy. It allows us to simplifies the Signature and set the Pydantic model as the output type meaning DSPy must return an object that matches the schema.
The internals of the the GPT related was also refactored to match the changes in the API.
Related to #49