add ability to translate the user guide on Crowdin (#17050)

Currently user documentation such as the user guide are made translatable via a custom (and very old) translation system hosted by NV Access. For many reasons we need to move away from this old system to something more mainstream and maintainable. We have already successfully moved translation of NVDA interface messages to Crowdin, and we should do the same for the user guide and other documentation.

Description of development approach
• Added markdownTranslate.py, which contains several commands for generating and updating xliff files from markdown files. These xliff files can then be uploaded to Crowdin for translation, and eventually downloaded again and converted back to markdown files. Commands include: 
◦ generateXliff: to generate an xliff file from a markdown file. Firstly a 'skeleton' of the markdown file is produced which is all the structure of a markdown file, but the translatable content on each line has been replaced by a special translation ID. Lines such as blank lines, hidden header rows, or table header separator lines are included in the skeleton in tact and are not available for translation. The xliff file is then produced, which contains one translatable string per translation unit, keyed by its respective translation ID. Each unit also contains translator notes to aide in translation, such as the line number, and any prefix or suffix markdown structure. E.g. a heading might have a prefix of ### and a suffix of {#Intro}. The skeleton is also embedded into the xliff file so that it is possible to update the xliff file keeping existing translation IDs, and or generate the existing markdown file from the xliff file.
◦ generateMarkdown: Given an xliff file, the original markdown file is reproduced from the embedded skeleton, using either the translated or source strings from the xliff file, depending on whether you want a translated or untranslated markdown file.
◦ updateXliff: to update an existing xliff file with changes from a markdown file, ensuring that IDs of existing translatable strings are kept in tact. This command extracts the skeleton from the xliff file, makes a diff of the old and new markdown files, then applies this diff to the skeleton file I.e. removes skeleton lines that were removed from the markdown file, and adds skeleton lines (with new IDs) for lines that are newly added to the markdown file. All existing lines stay as is, keeping their existing translation IDs. Finally a new xliff file is generated from the up to date markdown file and skeleton, resulting in an xliff file that contains all translatable strings from the new markdown file, but reusing translation IDs for existing strings.
◦ translateXliff: given an xliff file, and a pretranslated markdown file that matches the skeleton, a new xliff file is produced containing translations for all strings.
◦ pretranslateAllPossibleLangs: this walks the NVDA user_docs directory, and for each language, pretranslates the English xliff file using the existing pretranslated markdown file from the old translation system (if it matches the skeleton exactly) producing a translated xliff file that can be uploaded to Crowdin to bring an existing translation up to where it was in the old system.
• Added a generated xliff file for the current English user guide markdown file. Note that this has been uploaded to Crowdin for translation.
• Added a GitHub action that runs on the beta branch if English userGuide.md changes. The action regenerates the original markdown file from the current English user guide xliff, then updates the xliff file based on the changes from the original markdown file to the current markdown file. This xliff file is then uploaded to Crowdin, and also committed and pushed to beta.

.github/workflows/regenerate_english_userDocs_translation_source.yml

@@ -0,0 +1,94 @@
+name: Rebuild English User Documentation for Translation 
+
+on:
+  push:
+    branches:
+      - beta
+    paths:
+      - 'user_docs/en/*.md'
+
+jobs:
+  rebuild-translation-source:
+    runs-on: windows-latest
+
+    steps:
+    - name: Checkout repository
+      uses: actions/checkout@v3
+      with:
+        fetch-depth: 0
+
+    - name: Set up Python
+      uses: actions/setup-python@v4
+      with:
+        python-version: '3.11'
+
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install lxml requests
+
+    - name: update xliff files
+      shell: pwsh
+      run: |
+        # for any English markdown files changed within the commits of this push,
+        # update the corresponding xliff file (if one exists) to reflect the current markdown file,
+        # keeping existing translation IDs in tact. 
+        $ErrorActionPreference = 'Stop'
+        $changedFiles = git diff --name-only ${{github.event.before}}.. -- user_docs/en/*.md
+        foreach ($file in $changedFiles) {
+            Write-Host "$file has changed"
+            $baseName = [System.IO.Path]::GetFileNameWithoutExtension($file)
+            $xliff = "user_docs/en/$baseName.xliff"
+            $tempXliff = "user_docs/en/$baseName.xliff.temp"
+            $markdown = $file
+            if (Test-Path $xliff) {
+                Write-Host "Updating $xliff with changes from $markdown"
+                python user_docs/markdownTranslate.py updateXliff -x $xliff -m $file -o $tempXliff
+                Write-Host "Renaming $tempXliff to $xliff"
+                move-item -Path $tempXliff -Destination $xliff -Force
+            } else {
+                Write-Host "Ignoring $markdown as it does not have a corresponding xliff file"
+            }
+        }
+      if: success()
+
+    - name: Commit and Push changes
+      env:
+        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      run: |
+        $ErrorActionPreference = 'Stop'
+        git config --local user.name "GitHub Actions"
+        git config --local user.email "[email protected]"
+        git remote set-url origin https://x-access-token:${GITHUB_TOKEN}@github.com/${{ github.repository }}.git
+        $filesChanged = git diff --name-only -- *.xliff
+        if ($filesChanged) {
+          Write-Host "xliff files were changed. Committing and pushing changes."
+          foreach ($file in $filesChanged) {
+            git add $file
+            git commit -m "Update $file"
+          }
+          git push origin HEAD
+        } else {
+          Write-Host "No xliff files were changed. Skipping commit and push."
+        }
+      if: success()
+
+    - name: Crowdin upload
+      # This step must only be run after successfully pushing changes to the repository.
+      # Otherwise if the push fails, subsequent runs may cause new translation IDs to be created,
+      # which will cause needless retranslation of existing strings.
+      env:
+        crowdinProjectID: ${{ vars.CROWDIN_PROJECT_ID }}
+        crowdinAuthToken: ${{ secrets.CROWDIN_AUTH_TOKEN }}
+      run: |
+        # Check if we changed userGuide.xliff in this action.
+        # If we did, upload it to Crowdin.
+        $ErrorActionPreference = 'Stop'
+        $changed = git diff --name-only ${{GITHUB.SHA}}.. -- user_docs/en/userGuide.xliff
+        if ($changed) {
+            Write-Host "Uploading userGuide.xliff to Crowdin"
+            # 18 is the file ID for userGuide.xliff in Crowdin.
+            python appVeyor/crowdinSync.py uploadSourceFile 18 user_docs/en/userguide.xliff
+        } else {
+            Write-Host "Not uploading userGuide.xliff to Crowdin as it has not changed"
+        }

sconstruct

@@ -333,6 +333,17 @@ for po in env.Glob(sourceDir.path + "/locale/*/lc_messages/*.po"):
 styles = os.path.join(userDocsDir.path, "styles.css")
 numberedHeadingsStyle = os.path.join(userDocsDir.path, "numberedHeadings.css")
 
+# Create Non-english md files in user_docs from localized xliff and English skel files
+for xliffFile in env.Glob(os.path.join(userDocsDir.path, "*", "*.xliff")):
+	lang = os.path.basename(xliffFile.path).split(".")[0]
+	if lang == "en":
+		continue
+	mdFile = os.path.splitext(xliffFile.path)[0] + ".md"
+	env.Command(
+		mdFile,
+		xliffFile,
+		[f'@{sys.executable} user_docs/markdownTranslate.py generateMarkdown -x "{xliffFile}" -o "{mdFile}"'],
+	)
 # Allow all markdown files to be converted to html in user_docs
 for mdFile in env.Glob(os.path.join(userDocsDir.path, "*", "*.md")):
 	htmlFile = env.md2html(mdFile)