Implement basic nested language detection support #1159
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
baronfel
force-pushed
the
support-nested-languages
branch
from
ddd5195
to
f414147
Compare
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| @@ -12,6 +11,14 @@ | |||
| open System.Threading | |||
| open IcedTasks | |||
|
|
|||
| type NestedLanguage = | |||
| { Language: string | |||
| Ranges: Types.Range[] } | |||
Check notice
Code scanning / Ionide.Analyzers.Cli
Detect if type[] is used instead of type array Note
|
|
||
| type TextDocumentNestedLanguages = | ||
| { TextDocument: VersionedTextDocumentIdentifier | ||
| NestedLanguages: NestedLanguage[] } |
Check notice
Code scanning / Ionide.Analyzers.Cli
Detect if type[] is used instead of type array Note
…ts as that function's named language
|
BIG update here - this works off of StringSyntax attributes for F# functions, but not for F#-defined class members or BCL-provided class members. There's a gap in the FCS APIs for providing Attributes for these members. I've raised it upstream. In the meantime, this is enough to get going on the Ionide-side for implementation! |
baronfel
commented
Dec 31, 2023
Comment on lines
+28
to
+104
| let private discoverRangesToRemoveForInterpolatedString | ||
| (stringKind: SynStringKind) | ||
| (parts: SynInterpolatedStringPart[]) | ||
| = | ||
| parts | ||
| |> Array.indexed | ||
| |> Array.collect (fun (index, part) -> | ||
| match part with | ||
| | SynInterpolatedStringPart.FillExpr(fillExpr = e) -> [| e.Range |] | ||
| // for the first part we have whatever 'leading' element on the left and a trailing interpolation piece (which can include a format specifier) on the right | ||
| | SynInterpolatedStringPart.String(range = range) when index = 0 -> | ||
| [| | ||
| // leading tokens adjustment | ||
| // GAP: we don't know how many interpolation $ or " there are, so we are guessing | ||
| match stringKind with | ||
| | SynStringKind.Regular -> | ||
| // 'regular' means $" leading identifier | ||
| range.WithEnd(range.Start.WithColumn(range.StartColumn + 2)) | ||
| | SynStringKind.TripleQuote -> | ||
| // 'triple quote' means $""" leading identifier | ||
| range.WithEnd(range.Start.WithColumn(range.StartColumn + 4)) | ||
| // there's no such thing as a verbatim interpolated string | ||
| | SynStringKind.Verbatim -> () | ||
|
|
||
| // trailing token adjustment- only an opening bracket { | ||
| // GAP: this is the feature gap - we don't know about format specifiers | ||
| range.WithStart(range.End.WithColumn(range.EndColumn - 1)) | ||
|
|
||
| |] | ||
| // for the last part we have a single-character interpolation bracket on the left and the 'trailing' string elements on the right | ||
| | SynInterpolatedStringPart.String(range = range) when index = parts.Length - 1 -> | ||
| [| | ||
| // leading token adjustment - only a closing bracket } | ||
| range.WithEnd(range.Start.WithColumn(range.StartColumn + 1)) | ||
|
|
||
| // trailing tokens adjustment | ||
| // GAP: we don't know how many """ to adjust for triple-quote interpolated string endings | ||
| match stringKind with | ||
| | SynStringKind.Regular -> | ||
| // 'regular' means trailing identifier " | ||
| range.WithStart(range.End.WithColumn(range.EndColumn - 1)) | ||
| | SynStringKind.TripleQuote -> | ||
| // 'triple quote' means trailing identifier """ | ||
| range.WithStart(range.End.WithColumn(range.EndColumn - 3)) | ||
| // no such thing as verbatim interpolated strings | ||
| | SynStringKind.Verbatim -> () |] | ||
| // for all other parts we have a single-character interpolation bracket on the left and a trailing interpolation piece (which can include a format specifier) on the right | ||
| | SynInterpolatedStringPart.String(range = range) -> | ||
| [| | ||
| // leading token adjustment - only a closing bracket } | ||
| range.WithEnd(range.Start.WithColumn(range.StartColumn + 1)) | ||
| // trailing token adjustment- only an opening bracket { | ||
| // GAP: this is the feature gap - we don't know about format specifiers here | ||
| range.WithStart(range.End.WithColumn(range.EndColumn - 1)) |]) | ||
|
|
||
| let private (|Ident|_|) (e: SynExpr) = | ||
| match e with | ||
| | SynExpr.Ident(ident) -> Some([ ident ]) | ||
| | SynExpr.LongIdent(longDotId = SynLongIdent(id = ident)) -> Some ident | ||
| | _ -> None | ||
|
|
||
| /// in order for nested documents to be recognized as their document types, the string quotes (and other tokens) need to be removed | ||
| /// from the actual string content. | ||
| let private removeStringTokensFromStringRange (kind: SynStringKind) (range: Range) : Range array = | ||
| match kind with | ||
| | SynStringKind.Regular -> | ||
| // we need to trim the double-quote off of the start and end | ||
| [| Range.mkRange range.FileName range.Start (range.Start.WithColumn(range.StartColumn + 1)) | ||
| Range.mkRange range.FileName (range.End.WithColumn(range.EndColumn - 1)) range.End |] | ||
| | SynStringKind.Verbatim -> | ||
| // we need to trim the @+double-quote off of the start and double-quote off the end | ||
| [| Range.mkRange range.FileName range.Start (range.Start.WithColumn(range.StartColumn + 2)) | ||
| Range.mkRange range.FileName (range.End.WithColumn(range.EndColumn - 1)) range.End |] | ||
| | SynStringKind.TripleQuote -> | ||
| // we need to trim the @+double-quote off of the start and double-quote off the end | ||
| [| Range.mkRange range.FileName range.Start (range.Start.WithColumn(range.StartColumn + 2)) | ||
| Range.mkRange range.FileName (range.End.WithColumn(range.EndColumn - 1)) range.End |] |
There was a problem hiding this comment.
@nojaf @vzarytovskii look at all of the nonsense string math I have to do in these two adjustment functions to get the 'real' content of the string :( Do you think the AST could/should learn any trivia here to make this kind of job easier? There are a few different pieces that are troublesome:
- string/interpolated string start - this can be
", or@", or""", or now with interpolated strings any number of$followed by""". this is untenable to get right currently. - string/interpolated string end - this can be
",""", or arbitrary"now. similarly bad :( - format specifiers - these are relevant to formatting, but are not part of the logical string content. currently they are part of the reported string range and this means to do it 'right' I'd need to get the full string's content and do a subsequent parse
- string interpolation close/open brackets - these can be any number of
{/}now with the expanded interpolation RFC
Essentially I'd love some kind of synthesized range for the logical content of a string in the AST, because all of this math is very error prong.
There was a problem hiding this comment.
In Fantomas, we currently rely on the ranges of SynInterpolatedStringParts to restore the string. So, as long as these are accurate, we can grab what we need from ISourceText and no longer deal with this. In your case, it is indeed very messy to get right.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
WHAT
🤖 Generated by Copilot at ddd5195
This pull request adds a new feature to detect and report nested languages in F# files, such as XML or SQL, using a compiler analyzer and a custom LSP notification. It modifies the
FsAutoComplete.CoreandFsAutoComplete.Lspprojects to implement the feature and theFsAutoComplete.Tests.Lspproject to test it. It also fixes some indentation and style issues in various files and adds a new configuration for running the tests in VS Code.🤖 Generated by Copilot at ddd5195
🌐🛠️🧪
WHY
Part of ionide/ionide-vscode-fsharp#1912, blocked by dotnet/fsharp#15925.
HOW
🤖 Generated by Copilot at ddd5195
SyntaxCollectorBasetype withAbstractClassattribute and default methods for convenience and clarity (link, link)previewlanguage features inFsAutoComplete.Core.fsprojto usenameofandopen typesyntax (link)launch.json(link)NestedLanguageTests.fsand include them ingeneralTestsinProgram.fs(link, link)Program.fsandServer.fsaccording to F# style guide (link, link, link, link, link, link, link, link, link, link, link, link, link, link, link, link, link, link, link, link, link, link, link, link, link, link)Server.fsiaccording to F# style guide (link, link)openstatement inFSharpLspClient.fs(link)findNestedLanguagesfunction as a potential configuration flag inAdaptiveFSharpLspServer.fs(link)