Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A minor overhaul #68

Merged
merged 19 commits into from
May 10, 2024
Merged

A minor overhaul #68

merged 19 commits into from
May 10, 2024

Conversation

gwynne
Copy link
Member

@gwynne gwynne commented May 2, 2024

These changes are now available in 1.9.0

Numerous improvements have been made to SQLiteNIO's functionality:

  • Blobs are now copied fewer times on their way to and from the database (and in some rare cases, they're copied several fewer times).
  • When decoding Strings, Floats, and Doubles via the SQLiteDataConvertible path (this primarily affects FluentSQLiteDriver and SQLiteKit), integer and real values are now converted to strings, and integer values are now converted to floats or doubles. Previously such conversions would be rejected.
  • The minimum supported Swift version is now 5.8.
  • All EventLoopFuture-based APIs now have explicit async counterparts, which in most cases saves at least a couple of excess thread hops. (This shows particular benefits in SQLiteKit - or rather, it will once Leverage improvements in SQLKit and SQLiteNIO sqlite-kit#108 lands.)
  • SQLite statement handles are now finalized correctly after encountering error conditions, rather than leaking until the overall connection is closed.
  • SQLite connection handles are no longer leaked when certain errors occur during SQLiteConnection.open().
  • SQLiteConnection.open() now throws more useful, less generic errors.
  • Strings and binary blobs larger than 4GB are now supported.
  • In several places where unexpected nil values would previously cause a crash, the failure is now handled gracefully.
  • Additional Sendable correctness has been implemented. In particular, SQLiteDatabase is now Sendable.
  • We now omit several SQLite features that aren't usable through SQLiteNIO. As we do not make the embedded CSQLite target available to downstream packages, this should not affect users functionally, and may slightly improve performance and output binary size.
  • As recommended by SQLite's documentation, custom user functions are now SQLITE_DIRECTONLY by default; this can be changed by passing indirect: true to the SQLiteCustomFunction initializer if needed.
  • SQLiteError now recognizes and uses SQLite's more specific "extended" result codes.
  • TSan no longer throws false positives when using Swift 5.8.
  • Database connections now default to using NIOThreadPool.singleton and MultiThreadedEventLoopGroup.singleton unless otherwise specified.

gwynne added 7 commits May 2, 2024 02:00
…s when being decoded, make `Double` and `Float` accept integer values, use recommended APIs for translating between ByteBuffer and Data
… error handling. Make sure statements get finalized when errors occur. Use sqlite3_bind_blob64() and sqlite3_bind_text64() instead of the 32-bit ones. Simplify reading of blob data.
…dd set of async assertions. Make tests resilient against array index range violations. Handle blobs more sensibly instead of torturing Data. Get rid of lots of force-unwraps. Use "throws error" assertion instead of messy do/catch clauses. Avoid crashing if opening a DB fails. Use singleton thread pool and MTELG always.
@gwynne gwynne added enhancement New feature or request semver-minor Contains new APIs labels May 2, 2024
@gwynne gwynne requested review from 0xTim, MahdiBM and ptoffy May 2, 2024 07:18
Copy link

codecov bot commented May 2, 2024

Codecov Report

Attention: Patch coverage is 55.09138% with 172 lines in your changes are missing coverage. Please review.

Project coverage is 65.60%. Comparing base (2102da7) to head (575470b).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #68      +/-   ##
==========================================
+ Coverage   65.03%   65.60%   +0.57%     
==========================================
  Files           8        9       +1     
  Lines         675      756      +81     
==========================================
+ Hits          439      496      +57     
- Misses        236      260      +24     
Files Coverage Δ
Sources/SQLiteNIO/SQLiteDataType.swift 0.00% <ø> (ø)
Sources/SQLiteNIO/SQLiteRow.swift 64.28% <100.00%> (ø)
Sources/SQLiteNIO/SQLiteData.swift 75.00% <50.00%> (+1.50%) ⬆️
Sources/SQLiteNIO/SQLiteStatement.swift 95.06% <93.61%> (+5.58%) ⬆️
Sources/SQLiteNIO/SQLiteCustomFunction.swift 82.48% <70.00%> (-1.20%) ⬇️
Sources/SQLiteNIO/SQLiteDataConvertible.swift 58.62% <30.43%> (-2.30%) ⬇️
Sources/SQLiteNIO/SQLiteConnection.swift 68.51% <63.15%> (+0.24%) ⬆️
Sources/SQLiteNIO/SQLiteDatabase.swift 30.00% <30.00%> (ø)
Sources/SQLiteNIO/SQLiteError.swift 48.98% <49.73%> (+22.17%) ⬆️

gwynne added 4 commits May 2, 2024 16:25
…convenience methods. Factor out the `open()` implementation so the core part can be shared by the ELF and async versions. Don't leak sqlite3* handles if sqlite3_busy_handler() fails for some reason. Throw more specific errors from `open()`. Don't log at error level. Use the async version of NIOThreadPool.runIfActive() when possible.
…n_v2(), so there's no point in setting multithread mode as the default during compilation; use serialized as the default instead.
Sources/SQLiteNIO/SQLiteStatement.swift Outdated Show resolved Hide resolved
Sources/SQLiteNIO/SQLiteStatement.swift Outdated Show resolved Hide resolved
Sources/SQLiteNIO/SQLiteData.swift Show resolved Hide resolved
Sources/SQLiteNIO/SQLiteData.swift Show resolved Hide resolved
gwynne added 3 commits May 4, 2024 08:30
…entation, plus omitting several APIs that aren't usable through this package anyway.
…y default (security hardening). Provide an initializer flag to override it if needed.
Package.swift Outdated Show resolved Hide resolved
Sources/SQLiteNIO/SQLiteConnection.swift Show resolved Hide resolved
@gwynne gwynne requested a review from 0xTim May 10, 2024 06:22
@gwynne gwynne changed the title Improve data handling for strings and blobs, plus general cleanup A minor overhaul May 10, 2024
@gwynne
Copy link
Member Author

gwynne commented May 10, 2024

The dependents test will not pass without the fix to FluentSQLiteDriver to account for the extended result codes in the DatabaseError conformance, but that is not a blocker for this PR.

@@ -54,6 +55,33 @@ public struct SQLiteError: Error, CustomStringConvertible, LocalizedError {
case warning
case row
case done

// SQLite "extended" result codes
case errorMissingCollatingSequence, errorRetry, errorMissingSnapshot
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Urgh, we need a blanket ban on enums in the next major releases

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or, you know, the core team could finally listen to people about the need to solve this problem 🤨

@gwynne gwynne merged commit 98d2894 into main May 10, 2024
9 of 11 checks passed
@gwynne gwynne deleted the overhaul branch May 10, 2024 10:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request semver-minor Contains new APIs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants