Improve Gherkin support

[Hirse]

Resolves #3638
Resolves #3639

Changes

• Move keywords to mode to support keywords with spaces and those ending with a colon
• Update list of keywords (see below)
• Change * from symbol to keyword since that is how it's described in the reference
• Tighten variable definition to disallow whitespace
• Only recognize keywords, comments, tags, and docstrings at the beginning of a line
• Support docstrings starting with triple backticks
• Remove (non-docstring) string mode

Reference

• https://cucumber.io/docs/gherkin/reference
  
• https://cucumber.io/docs/gherkin/languages/#gherkin-dialect-en-content
  

Checklist

• Added markup tests
• Updated the changelog at CHANGES.md

[pkufranky]

thanks very much

[joshgoebel]

Technically all features (except variables) are only valid at the beginning of a line.

Can't we do this with multi-match and ^ (start line)? Just use the first part of the match to grab the whitespace...

[joshgoebel]

and those ending with a colon

Is the colon part of the keyword though?

[Hirse]

Updated with a new version that removes the custom keyword pattern and fixes false positives.

I couldn't figure out a good way to match for "beginning of line, skipping the whitespace" as /^[ \t]*/ triggers the "zero-width regex" error so there's currently a bunch of duplication.

[joshgoebel]

This repetition is frustrating. Could a single top-level rule to eat tabs at the start of lines accomplish this same thing (probably would leave to false positives)? If not why not add it to each multi-match rule as an optional first component:

        begin: [
          LINE_BEGINS_WS,
          /(Feature|Rule|Examples?|Scenario(?:s| Outline| Template)?|Background)/,
          /:/
        ],
        beginScope: {
          2: 'keyword',
          3: 'punctuation',
        },

[Hirse]

Yes, eating whitespace of the beginning of the line with an empty mode seems to work. Is that what you had in mind? 😄

[joshgoebel]

The multi-match method I show above is what I had in mind, yes. That's the best way I know to handle this kind of thing (without putting the whitespace inside the highlight zone).

[pkufranky]

Any progress for this pr?

[joshgoebel]

@Hirse is this ready to go? sorry for long wait. coud you update changes.md?