[Feature] Functionality to create charts with commands in natural language

[ved-kashyap-samsung]

• User should be able to enable (using feature flag) the feature of creating charts with natural language commands
• There should be minimalistic UI to provide command in natural language in the website ( UI to be proposed by core UX team)
• The natural language command to be converted in SQL query format (we should be able to use externally hosted ML models through API calls)
• Superset should fetch data using connected database using SQL query
• Data to be converted in charts dynamically

@rusackas

[rusackas]

Is there a way to standardize/abstract these models, so people can choose which of the many AI services/models they want to use? Some might want to pay for OpenAI, some might want to use something more open-source friendly via Hugging Face, etc.

How do we set limits (or what even is the proposed approach) on sending sample data to the AI to give it context? This might be problematic for people worrying about various types of compliance.

I think pretty much everyone is excited about the prospect, but nervous about how to make it open-source friendly in terms of licensing, cost, flexibility, security, and maintainability. Would love to hear more about your thinking on this.

[Abhishek-kumar-samsung]

For now we can try to enable feature with llm endpoint variable. If anyone wants to use this feature then he can put llm endpoint somewhere in ui and then use the feature.
Similar to automated reporting using slack(where user has to put slack token to use automated reporting).

User can run llm anywhere in their system, and put endpoint in superset ui, and use the feature.

[rusackas]

If you mean a feature flag, then definitely yes :)

Most LLMs seem to use different request bodies -we may want a place to abstract things as configuration, or even several options, so people can choose the platform(s) and models they want to use. There should be a uniform interface, with "adapters" to connect to various LLM services/models and APIs, rather than being prescriptive about which of these we'll integrate in a more opinionated approach. This might apply to vector databases, too.

I think there are a LOT of people interested in this conversation about how the interface will work, from a UI standpoint, and a platform/abstraction perspective as well. Would you be open to creating a SIP (Superset Improvement Proposal) around this? You're welcome to open a PR to help frame/POC that approach, but without due process, I'm worried a PR might get stuck. It's something we're all interested, so we'd love to see someone like Samsung take the lead here! :)

[jyotirmay123]

Anything happening here? I was actually looking for this feature!

[gpchandran]

I'm also interested in contributing to this feature of creating chart using Gen AI LLM, can we all pull together and work on this feature.