Implement Basic Text to Speech #1143

ShaopengLin · 2024-07-18T07:11:12Z

Changes:

Reading the entire or selected text on a web page in different languages.
The reading can be triggered in the Edit Menu, Shortcut, and right-click on a page.
TTS UI to select languages and voices, and stop the reading.
Voice configurations are remembered between sessions.
Language determined by Zim's Locale.

ShaopengLin · 2024-07-18T07:34:13Z

I wasn't able to implement PAUSE, RESUME, and volume.

PAUSE/RESUME: Qt relies on the underlying system's audio engines to perform these operations and Qt has quite some buggy behavior on those engines. I tried for a long time to find solutions but none is compatible with all engines and Qt versions. This leads to the compromise of simply having Speak & STOP.

Volume: From what I tested, the Linux "speech" engines' audio doesn't change during the reading, which in my opinion makes it useless.

Considering we are shipping cross-platform, it is unreliable to have features that work on one but not on another. If we can find a way to package Qt with a working speech engine, we might be able to implement PAUSE/RESUME. However, I have no idea how to do that. @veloman-yunkan @mgautierfr Do you by any chance have an idea of how hard it would be to package an external library like Flite with Kiwix-Desktop? If it's a huge hurdle and we only get the PAUSE/RESUME, then I wouldn't say the return on the effort is great.

The priority for this PR would be to have Kiwix-Desktop be compatible with Qt6.4+, CI builds, and a satisfactory UI/UX given the existing feature.

kelson42 · 2024-07-19T06:14:05Z

@ShaopengLin Thank you very much for your PR. For the moment I'm not able to compile it on my Ubuntu 22.04 but should move soon to 24.04 and hope this will allow me to have Qt 6.4. In the meantime we need to secure the backward compatibility with older version of Qt (both version 5 and 6). Please fix the compilation script and the code to secure that this feature is only activated if Qt6.4 or higher is available, otherwise just deactive the feature in a way both compilation and run are fine.

ShaopengLin · 2024-08-05T07:57:12Z

@kelson42 The compilation is now for Qt 6.4 +.

kelson42 · 2024-09-08T22:03:51Z

@ShaopengLin I have moved to 24.04. Would you be able please to rebase on main so I can finally move forward with the testing?

ShaopengLin · 2024-09-09T06:57:28Z

@kelson42 Done.

From my own testing, TTS from Qt is rather unstable and sometimes crashes the application when used for no apparent reason/pattern at all. I have been trying my best to find out if there is a solution to the core problem, though little information really floats around the internet on this topic. I honestly hope it's just my own computer's issue... Please try it on your machine and see if any crashes or misbehaviors exist. If I cannot figure out why those things occur, rather than relying on Qt's flawed implementation, the goal should be improving screen reader compatibilities.

When you test, the dfki packages(if exists) are likely not going to work so select the other ones.

ShaopengLin · 2024-09-11T17:21:56Z

@kelson42 I have found the root cause to the crashes. Will put up a fixup commit soon...

ShaopengLin · 2024-09-13T20:01:19Z

@kelson42 Should be ready for testing now.

kelson42 · 2024-09-14T12:22:58Z

@ShaopengLin I see the menu entry but nothing happens if I click on it: no sound not bottom bar visible. I use Ubuntu 24.04 wieth Qt 5.15.13.

ShaopengLin · 2024-09-14T12:24:45Z

@kelson42 As mentioned in this comment we moved compilation to Qt6.4+. I will make a change so those menu entries are invisible in Qt 6.4 and below.

kelson42 · 2024-09-14T12:31:22Z

@kelson42 As mentioned in this comment we moved compilation to Qt6.4+. I will make a change so those menu entries are invisible in Qt 6.4 and below.

@ShaopengLin Then the option should not be available (or grayed)

kelson42 · 2024-09-14T12:49:48Z

@ShaopengLin For me, does not compile with Qt 6.4.2:

g++ -c -pipe -std=c++17 -Werror -I/usr/local/include/ -I/usr/local/include -I/usr/include/x86_64-linux-gnu -I/usr/include/p11-kit-1 -O2 -std=gnu++1z -Wall -Wextra -fPIC -D_REENTRANT -DVERSION=2.3.1 -DQT_DEPRECATED_WARNINGS -DQT_DISABLE_DEPRECATED_BEFORE=0x060000 -DQT_NO_DEBUG -DQT_WEBENGINEWIDGETS_LIB -DQT_PRINTSUPPORT_LIB -DQT_WIDGETS_LIB -DQT_WEBENGINECORE_LIB -DQT_QUICK_LIB -DQT_OPENGL_LIB -DQT_GUI_LIB -DQT_QMLMODELS_LIB -DQT_WEBCHANNEL_LIB -DQT_QML_LIB -DQT_NETWORK_LIB -DQT_QMLINTEGRATION_LIB -DQT_POSITIONING_LIB -DQT_CORE_LIB -I. -Isubprojects/QtSingleApplication/src -I/usr/include/x86_64-linux-gnu/qt6 -I/usr/include/x86_64-linux-gnu/qt6/QtWebEngineWidgets -I/usr/include/x86_64-linux-gnu/qt6/QtPrintSupport -I/usr/include/x86_64-linux-gnu/qt6/QtWidgets -I/usr/include/x86_64-linux-gnu/qt6/QtWebEngineCore -I/usr/include/x86_64-linux-gnu/qt6/QtQuick -I/usr/include/x86_64-linux-gnu/qt6/QtOpenGL -I/usr/include/x86_64-linux-gnu/qt6/QtGui -I/usr/include/x86_64-linux-gnu/qt6/QtQmlModels -I/usr/include/x86_64-linux-gnu/qt6/QtWebChannel -I/usr/include/x86_64-linux-gnu/qt6/QtQml -I/usr/include/x86_64-linux-gnu/qt6/QtNetwork -I/usr/include/x86_64-linux-gnu/qt6/QtQmlIntegration -I/usr/include/x86_64-linux-gnu/qt6/QtPositioning -I/usr/include/x86_64-linux-gnu/qt6/QtCore -I. -I. -I/usr/lib/x86_64-linux-gnu/qt6/mkspecs/linux-g++ -o contenttypefilter.o src/contenttypefilter.cpp
src/contenttypefilter.cpp: In constructor ‘ContentTypeFilter::ContentTypeFilter(QString, QWidget*)’:
src/contenttypefilter.cpp:16:35: error: ‘checkStateChanged’ is not a member of ‘QCheckBox’
   16 |         connect(this, &QCheckBox::checkStateChanged, this, &ContentTypeFilter::onStateChanged);
      |                                   ^~~~~~~~~~~~~~~~~
make: *** [Makefile:1437: contenttypefilter.o] Error 1

ShaopengLin · 2024-09-14T12:52:42Z

@kelson42 That is not my change. I believe this is from #1205 where I believe @sgourdas should've made the deprecation for only Qt 6.7+, and not Qt 6+. Source

kelson42 · 2024-09-17T09:31:16Z

@sgourdas Can you please fix compilation regression we have now with Qt6? This is blocking review of this PR.

sgourdas · 2024-09-17T09:52:31Z

@sgourdas Can you please fix compilation regression we have now with Qt6? This is blocking review of this PR.

@kelson42 was this not fixed with #1212 by @ShaopengLin?

kelson42 · 2024-09-17T12:30:23Z

Actually I don't have Qt6.7+ on my Ubuntu 24.04. How should i test this? @ShaopengLin @sgourdas Do you use a PPA to get the most recent version?

sgourdas · 2024-09-17T15:33:36Z

Actually I don't have Qt6.7+ on my Ubuntu 24.04. How should i test this? @ShaopengLin @sgourdas Do you use a PPA to get the most recent version?

If I remember correctly I had built it from the git source.

Would you like me to test something?

kelson42

LGTM from a user perspective

ShaopengLin · 2024-10-03T15:29:43Z

@veloman-yunkan will notify you when code is ready. We can focus on the bigger ones first.

ShaopengLin · 2024-11-13T08:51:09Z

@veloman-yunkan Ready for review.

ShaopengLin · 2024-11-13T08:59:10Z

@veloman-yunkan Also, Qt at CI / Linux (focal) does not have texttospeech. Will we be adding the package or will I have to disable TTS if the package does not exist?

kelson42 · 2024-11-13T19:50:16Z

@veloman-yunkan Also, Qt at CI / Linux (focal) does not have texttospeech. Will we be adding the package or will I have to disable TTS if the package does not exist?

What does that technically means "does not have texttospeech"? It means no boice is available?

ShaopengLin · 2024-11-13T19:54:54Z

@kelson42 The Qt TextToSpeech module is missing in that CI.

kelson42 · 2024-11-13T20:10:49Z

@ShaopengLin The problem is not really the CI but what will happen if for some reason we have Qt5 without all the necessary. My best guess is that we should be prepared to such a scenario and hide or desactivate the feature if this lack is detected at start. Or is that depending to a certain version of Qt5?

ShaopengLin · 2024-11-13T20:12:49Z

@kelson42 Don't believe it should depend on a certain version of Qt5, but I suppose having safeguards against this would be ideal. I will change this shortly.

Basic Reading of Articles or Selected Texts.

Added the bar at bottom of webpage. Intoduced the stop speaking button.

Bar is default closed and shown on any speech activation.

The line edit should not be edit/highlight/selectable.

Rather than only the arrow, clicking any place on the combobox should popup the selector

Tabs are by default Zim's Locale

Keep track of the voices so user do not need to reconfigure.

Some systems or QT versions might not have or the module is not available.

veloman-yunkan

Considering TTS as an accessibility feature for people with vision problems, it should be primarily accessible via keyboard. I know that kiwix-desktop's accessibility in that regard is quite poor, however introducing TTS without a way of controlling it via keyboard doesn't seem wise.

veloman-yunkan · 2024-11-14T07:28:44Z

src/kiwixapp.cpp

@@ -544,6 +544,11 @@ void KiwixApp::saveWindowState()
  mp_session->setValue("windowState", getMainWindow()->saveState());
 }

+void KiwixApp::saveVoiceName(const QString& langName, const QString& voiceName)
+{
+  mp_session->setValue(langName + "/voice", voiceName);


This way of generating the setting name from variable data introduces a possibility of conflict with another setting with a name generated in a different way. I think it is better to at least start setting names with a fixed known prefix (either move "voice" to the beginning, or prepend "tts:" or "text2speech:")

veloman-yunkan · 2024-11-14T07:33:55Z

src/texttospeechbar.cpp

+    /* We either stay with default voices or matches with the saved voice. */
+    int voiceIndex = 0;
+    for (int i = 0; i < ui->voiceComboBox->count(); i++)
+    {
+        if (m_voices[i].name() == currentVoice.name())
+            voiceIndex = i;
+        if (m_voices[i].name() == savedVoiceName)
+        {
+            voiceIndex = i;
+            break;
+        }
+    }


Please introduce a function int TextToSpeechBar::getVoiceIndex(const QString& voiceName) const and use it by trying to first obtain the saved voice, or the current voice as a fallback.

veloman-yunkan · 2024-11-14T07:42:27Z

src/texttospeechbar.cpp

+    if (m_speech.state() == QTextToSpeech::Speaking)
+        speak(m_text);


IMHO, that won't be good UX. If possible (given the limitation of pause/resuming functionality of QTextToSpeech) it would be better to pause the speech, change the voice and resume the speech.

My first thought as well, unfortunately, QTextToSpeech's pause and resume barely work.

ShaopengLin requested a review from kelson42 July 18, 2024 07:34

ShaopengLin force-pushed the Issue#44-tts branch 8 times, most recently from e06256d to ba921f6 Compare August 5, 2024 01:46

ShaopengLin force-pushed the Issue#44-tts branch from ba921f6 to f4ad9f8 Compare September 9, 2024 06:43

ShaopengLin force-pushed the Issue#44-tts branch 2 times, most recently from 6937c65 to 1b9265c Compare September 13, 2024 19:57

ShaopengLin mentioned this pull request Sep 14, 2024

Corrected version for checkStateChanged #1212

Merged

ShaopengLin force-pushed the Issue#44-tts branch from 1b9265c to 5ce8df9 Compare September 15, 2024 11:19

kelson42 approved these changes Oct 3, 2024

View reviewed changes

ShaopengLin force-pushed the Issue#44-tts branch 2 times, most recently from 6fb2227 to fb1ee94 Compare November 13, 2024 08:49

ShaopengLin requested a review from veloman-yunkan November 13, 2024 08:51

ShaopengLin force-pushed the Issue#44-tts branch 2 times, most recently from 52fac5b to c62418d Compare November 13, 2024 18:26

ShaopengLin force-pushed the Issue#44-tts branch from c62418d to 7b629ff Compare November 14, 2024 00:22

ShaopengLin added 14 commits November 13, 2024 19:23

Introduce texttospeechbar.{h,cpp}

b2f2e59

Basic Reading of Articles or Selected Texts.

TTS Actions Added to Right Click ContextMenu

a1ca60a

Introduce texttospeechbar.ui

8d9f51f

Added the bar at bottom of webpage. Intoduced the stop speaking button.

Add Close Button to TTS Bar

5e46e4a

Bar is default closed and shown on any speech activation.

ESC Key Closes TTS Bar

90b630e

Added Language Selector to TTS Bar

31c099b

TTS Lanugage Selector Entries Are Read-Only

71fdf03

The line edit should not be edit/highlight/selectable.

TTS Language Selector Popup On-Click Anywhere

4b42bf3

Rather than only the arrow, clicking any place on the combobox should popup the selector

Added Voice Selector to TTS Bar

7f7cb60

TTS Lanugage Matches Zim Locale

4ee6beb

Tabs are by default Zim's Locale

Save Voice Selected For Each Lanugage

e8dcdd0

Keep track of the voices so user do not need to reconfigure.

Restart Speech When TTS Language/Voice Changes

f1999ec

Proper Styling for TSS

18be972

Disable TTS If No Such QT Module

45a1139

Some systems or QT versions might not have or the module is not available.

ShaopengLin force-pushed the Issue#44-tts branch from 7b629ff to 45a1139 Compare November 14, 2024 00:29

veloman-yunkan requested changes Nov 14, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement Basic Text to Speech #1143

Implement Basic Text to Speech #1143

ShaopengLin commented Jul 18, 2024 •

edited

Loading

ShaopengLin commented Jul 18, 2024

kelson42 commented Jul 19, 2024

ShaopengLin commented Aug 5, 2024

kelson42 commented Sep 8, 2024

ShaopengLin commented Sep 9, 2024 •

edited

Loading

ShaopengLin commented Sep 11, 2024 •

edited

Loading

ShaopengLin commented Sep 13, 2024

kelson42 commented Sep 14, 2024

ShaopengLin commented Sep 14, 2024 •

edited

Loading

kelson42 commented Sep 14, 2024

kelson42 commented Sep 14, 2024

ShaopengLin commented Sep 14, 2024 •

edited

Loading

kelson42 commented Sep 17, 2024

sgourdas commented Sep 17, 2024

kelson42 commented Sep 17, 2024

sgourdas commented Sep 17, 2024

kelson42 left a comment

ShaopengLin commented Oct 3, 2024

ShaopengLin commented Nov 13, 2024

ShaopengLin commented Nov 13, 2024

kelson42 commented Nov 13, 2024

ShaopengLin commented Nov 13, 2024

kelson42 commented Nov 13, 2024

ShaopengLin commented Nov 13, 2024

veloman-yunkan left a comment

veloman-yunkan Nov 14, 2024

veloman-yunkan Nov 14, 2024

veloman-yunkan Nov 14, 2024

ShaopengLin Nov 14, 2024

		if (m_speech.state() == QTextToSpeech::Speaking)
		speak(m_text);

Implement Basic Text to Speech #1143

Are you sure you want to change the base?

Implement Basic Text to Speech #1143

Conversation

ShaopengLin commented Jul 18, 2024 • edited Loading

ShaopengLin commented Jul 18, 2024

kelson42 commented Jul 19, 2024

ShaopengLin commented Aug 5, 2024

kelson42 commented Sep 8, 2024

ShaopengLin commented Sep 9, 2024 • edited Loading

ShaopengLin commented Sep 11, 2024 • edited Loading

ShaopengLin commented Sep 13, 2024

kelson42 commented Sep 14, 2024

ShaopengLin commented Sep 14, 2024 • edited Loading

kelson42 commented Sep 14, 2024

kelson42 commented Sep 14, 2024

ShaopengLin commented Sep 14, 2024 • edited Loading

kelson42 commented Sep 17, 2024

sgourdas commented Sep 17, 2024

kelson42 commented Sep 17, 2024

sgourdas commented Sep 17, 2024

kelson42 left a comment

Choose a reason for hiding this comment

ShaopengLin commented Oct 3, 2024

ShaopengLin commented Nov 13, 2024

ShaopengLin commented Nov 13, 2024

kelson42 commented Nov 13, 2024

ShaopengLin commented Nov 13, 2024

kelson42 commented Nov 13, 2024

ShaopengLin commented Nov 13, 2024

veloman-yunkan left a comment

Choose a reason for hiding this comment

veloman-yunkan Nov 14, 2024

Choose a reason for hiding this comment

veloman-yunkan Nov 14, 2024

Choose a reason for hiding this comment

veloman-yunkan Nov 14, 2024

Choose a reason for hiding this comment

ShaopengLin Nov 14, 2024

Choose a reason for hiding this comment

ShaopengLin commented Jul 18, 2024 •

edited

Loading

ShaopengLin commented Sep 9, 2024 •

edited

Loading

ShaopengLin commented Sep 11, 2024 •

edited

Loading

ShaopengLin commented Sep 14, 2024 •

edited

Loading

ShaopengLin commented Sep 14, 2024 •

edited

Loading