Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resolve key assumptions #1

Open
4 tasks
aaroneden opened this issue Feb 18, 2024 · 0 comments
Open
4 tasks

Resolve key assumptions #1

aaroneden opened this issue Feb 18, 2024 · 0 comments
Labels
good first issue Good for newcomers question Further information is requested

Comments

@aaroneden
Copy link
Owner

aaroneden commented Feb 18, 2024

Our first goal is to enable realtime virtual speech to speech translation over the web. This would allow a webinar presenter to have their voice translated to another language in realtime to the audience. Additionally the reverse would be true for the audience.

The assumptions below must be proven in order for this project to be successful. Proving these will allow us to design the appropriate architecture to allow online events to occur without language barriers.

  • S2S inference lag is not disruptive to a conversation.
    If the lag is too long it will stall the conversation and it will not be valuable for participants.

  • an S2S model can be utilized via a web conferencing system
    We must have a web conferencing system with plugin support that allows the audio stream to be overridden so we can inject specific language audio streams.

  • Realtime S2S translation is not cost prohibitive.
    If the inference cost is too high then it won't make sense to implement this system. We need to have a sense of what the inference cost is per language for a 1 hour webinar.

  • Seamless m4t is the best model to utilize for this use case.
    It is possible that other S2S models would work as well or better than Seamless. We need to understand the tradeoffs between different models.

@aaroneden aaroneden added good first issue Good for newcomers question Further information is requested labels Feb 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers question Further information is requested
Projects
None yet
Development

No branches or pull requests

1 participant