Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch sources to typed arrays instead of strings #242

Merged
merged 8 commits into from
Jan 11, 2024

Conversation

cd1m0
Copy link
Contributor

@cd1m0 cd1m0 commented Jan 8, 2024

Warning

This is an interface breaking change.

Change our implementation to keep source file contents as Uint8Arrays instead of string. This is needed since AST source map are actually byte offsets into the file. We were erroneously trearting them as string indices, which breaks in the presence of multi-byte Unicode characters.

This involves:

  1. Changing a bunch of the compile* interfaces to accept Map<string, Uint8Array> instead of Map<string, string>.
  2. Changing the StructuredDocumentation pass to correctly compute docs by doing byte indexing.
  3. Changing the ASTWriter to compute a correct map.
  4. Adding tests.

Todo (@blitz-1306 ): I havent looked at inline assembly. Is there anything to do there?

@cd1m0 cd1m0 requested a review from blitz-1306 January 8, 2024 19:37
Copy link
Contributor

@blitz-1306 blitz-1306 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work. Two things to consider:

  1. There is a coverage failure.
  2. One small nit in the code.

I will try to pick up issues tomorrow.

I havent looked at inline assembly. Is there anything to do there?

No, I think there is nothing left.

src/compile/utils.ts Outdated Show resolved Hide resolved
@blitz-1306 blitz-1306 added bug Something isn't working enhancement New feature or request labels Jan 10, 2024
@blitz-1306 blitz-1306 changed the title Switch sources to bufs Switch sources to typed arrays nstead of strings Jan 10, 2024
@blitz-1306 blitz-1306 added the breaking change Changes that would cause a backward compatibility break label Jan 10, 2024
…ve unnecessary Buffer reference in compile utils.
blitz-1306
blitz-1306 previously approved these changes Jan 10, 2024
Copy link
Contributor

@blitz-1306 blitz-1306 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks ready to merge after 73492b7.

@blitz-1306 blitz-1306 changed the title Switch sources to typed arrays nstead of strings Switch sources to typed arrays instead of strings Jan 10, 2024
@codecov-commenter
Copy link

codecov-commenter commented Jan 11, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (9ebcad8) 91.59% compared to head (685b922) 91.60%.

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #242   +/-   ##
=======================================
  Coverage   91.59%   91.60%           
=======================================
  Files         267      270    +3     
  Lines        6630     6634    +4     
  Branches     1353     1350    -3     
=======================================
+ Hits         6073     6077    +4     
  Misses        291      291           
  Partials      266      266           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

blitz-1306
blitz-1306 previously approved these changes Jan 11, 2024
…ment garbage at the begining of the fragment, including comments that are not of interest to us
@blitz-1306 blitz-1306 merged commit c11d592 into master Jan 11, 2024
3 checks passed
@blitz-1306 blitz-1306 deleted the switch_sources_to_bufs branch January 11, 2024 16:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking change Changes that would cause a backward compatibility break bug Something isn't working enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants