Add RouteLLM Pipeline

[justinh-rahb]

Introducing the RouteLLM Intelligent Routing Pipeline

This PR unveils a significant innovation in our processing capabilities with the RouteLLM Pipeline, now implemented in: routellm_pipeline.py. This pipeline is engineered to enhance efficiency by intelligently routing queries between different language models to balance cost and performance effectively.

RouteLLM Routing Features:

• The pipeline leverages RouteLLM to dynamically route queries between a strong model and a weak model based on a specified cost threshold.
• Seamlessly integrates with multiple language models to ensure high-quality responses while optimizing for resource usage.

---

This addition to our Pipelines framework empowers users with intelligent routing capabilities, allowing for a significant reduction in operational costs while maintaining the high-quality output required for various applications. Embrace the efficient, flexible, and scalable processing with the RouteLLM Intelligent Routing Pipeline.

[justinh-rahb]

It's got some issues right now:

• Need to save your valves again after restarting server to initialize the environment variables
• litellm is used for API adapters, so this could support other endpoints than OpenAI-spec Supports LiteLLM-supported providers now.

[justinh-rahb]

Still having the issue where the valves need to be re-saved after a server restart to take effect, as well as this when sending a message with an image:

ERROR:root:Error in pipe: Error code: 400 - {'error': {'message': "'$.input' is invalid. Please check the API reference: https://platform.openai.com/docs/api-reference.", 'type': 'invalid_request_error', 'param': None, 'code': None}}
routellm.openai

[justinh-rahb]

I feel better about merging this now, the startup initialization issue is fixed. It still doesn't support images, though I seem to recall we always had issues there with LiteLLM before.