Skip to content

Commit

Permalink
A minor overhaul (#68)
Browse files Browse the repository at this point in the history
* Bump minimum Swift version to 5.8. Update CI. General package cleanup. General docs cleanup.
* Add explicit async versions of all ELF methods of SQLiteConnection
* Improve handling of blob and text data in SQLiteData
* Make `String` accept integer and real values as well as textual values when being decoded, make `Double` and `Float` accept integer values, use recommended APIs for translating between ByteBuffer and Data
* SQLiteStatement: Use sqlite_prepare_v3() instead of _v2(). Factor out error handling. Make sure statements get finalized when errors occur. Use sqlite3_bind_blob64() and sqlite3_bind_text64() instead of the 32-bit ones. Simplify reading of blob data.
* Misc minor cleanup (Style only, nothing functional)
* Tests: Make all tests async. Add helper to replace use of defer {}. Add set of async assertions. Make tests resilient against array index range violations. Handle blobs more sensibly instead of torturing Data. Get rid of lots of force-unwraps. Use "throws error" assertion instead of messy do/catch clauses. Avoid crashing if opening a DB fails. Use singleton thread pool and MTELG always.
* Remove incorrect new protocol requirements. Add passthroughs for the convenience methods. Factor out the `open()` implementation so the core part can be shared by the ELF and async versions. Don't leak sqlite3* handles if sqlite3_busy_handler() fails for some reason. Throw more specific errors from `open()`. Don't log at error level. Use the async version of NIOThreadPool.runIfActive() when possible.
* Split the SQLDatabase protocol into its own file. Group SQLiteConnnection's async methods together.
* We always pass SQLITE_OPEN_FULLMUTEX (serialized mode) to sqlite3_open_v2(), so there's no point in setting multithread mode as the default during compilation; use serialized as the default instead.
* Add test that validates we're using SQLite in the correct (safest) threading mode.
* Fix a couple of log messages
* Apply several SQLite compilation settings according to upstream documentation, plus omitting several APIs that aren't usable through this package anyway.
* As recommended in the docs, mark custom functions SQLITE_DIRECTONLY by default (security hardening). Provide an initializer flag to override it if needed.
* Remove not yet available upcoming/experimental feature flags from the 5.8 manifest
* Update dependency minimums
* Add support for SQLite's extended result codes
* Lots of documentation comments. Reorganize SQLiteConnection a little. SQLiteDatabase is Sendable.
* Try to silence TSan false positives in 5.8
  • Loading branch information
gwynne committed May 10, 2024
1 parent e79d223 commit 98d2894
Show file tree
Hide file tree
Showing 21 changed files with 1,696 additions and 866 deletions.
12 changes: 0 additions & 12 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
version: 2
enable-beta-ecosystems: true
updates:
- package-ecosystem: "github-actions"
directory: "/"
Expand All @@ -11,14 +10,3 @@ updates:
dependencies:
patterns:
- "*"
- package-ecosystem: "swift"
directory: "/"
schedule:
interval: "daily"
open-pull-requests-limit: 6
allow:
- dependency-type: all
groups:
all-dependencies:
patterns:
- "*"
3 changes: 2 additions & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ jobs:
dependents-check:
if: ${{ !(github.event.pull_request.draft || false) }}
runs-on: ubuntu-latest
container: swift:5.9-jammy
container: swift:5.10-jammy
steps:
- name: Check out package
uses: actions/checkout@v4
Expand All @@ -38,3 +38,4 @@ jobs:

unit-tests:
uses: vapor/ci/.github/workflows/run-unit-tests.yml@main
secrets: inherit
103 changes: 67 additions & 36 deletions Package.swift
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
// swift-tools-version:5.7
// swift-tools-version:5.8
import PackageDescription

let package = Package(
Expand All @@ -13,8 +13,8 @@ let package = Package(
.library(name: "SQLiteNIO", targets: ["SQLiteNIO"]),
],
dependencies: [
.package(url: "https://github.com/apple/swift-nio.git", from: "2.58.0"),
.package(url: "https://github.com/apple/swift-log.git", from: "1.0.0"),
.package(url: "https://github.com/apple/swift-nio.git", from: "2.65.0"),
.package(url: "https://github.com/apple/swift-log.git", from: "1.5.4"),
],
targets: [
.plugin(
Expand All @@ -25,38 +25,69 @@ let package = Package(
),
exclude: ["001-warnings-and-data-race.patch"]
),
.target(name: "CSQLite", cSettings: [
// Derived from sqlite3 version 3.43.0
.define("SQLITE_DQS", to: "0"),
.define("SQLITE_ENABLE_API_ARMOR"),
.define("SQLITE_ENABLE_COLUMN_METADATA"),
.define("SQLITE_ENABLE_DBSTAT_VTAB"),
.define("SQLITE_ENABLE_FTS3"),
.define("SQLITE_ENABLE_FTS3_PARENTHESIS"),
.define("SQLITE_ENABLE_FTS3_TOKENIZER"),
.define("SQLITE_ENABLE_FTS4"),
.define("SQLITE_ENABLE_FTS5"),
.define("SQLITE_ENABLE_MEMORY_MANAGEMENT"),
.define("SQLITE_ENABLE_PREUPDATE_HOOK"),
.define("SQLITE_ENABLE_RTREE"),
.define("SQLITE_ENABLE_SESSION"),
.define("SQLITE_ENABLE_STMTVTAB"),
.define("SQLITE_ENABLE_UNKNOWN_SQL_FUNCTION"),
.define("SQLITE_ENABLE_UNLOCK_NOTIFY"),
.define("SQLITE_MAX_VARIABLE_NUMBER", to: "250000"),
.define("SQLITE_LIKE_DOESNT_MATCH_BLOBS"),
.define("SQLITE_OMIT_DEPRECATED"),
.define("SQLITE_OMIT_LOAD_EXTENSION"),
.define("SQLITE_OMIT_SHARED_CACHE"),
.define("SQLITE_SECURE_DELETE"),
.define("SQLITE_THREADSAFE", to: "2"),
.define("SQLITE_USE_URI"),
]),
.target(name: "SQLiteNIO", dependencies: [
.target(name: "CSQLite"),
.product(name: "Logging", package: "swift-log"),
.product(name: "NIO", package: "swift-nio"),
]),
.testTarget(name: "SQLiteNIOTests", dependencies: ["SQLiteNIO"]),
.target(
name: "CSQLite",
cSettings: sqliteCSettings
),
.target(
name: "SQLiteNIO",
dependencies: [
.target(name: "CSQLite"),
.product(name: "Logging", package: "swift-log"),
.product(name: "NIOCore", package: "swift-nio"),
.product(name: "NIOPosix", package: "swift-nio"),
.product(name: "NIOFoundationCompat", package: "swift-nio"),
],
swiftSettings: swiftSettings
),
.testTarget(
name: "SQLiteNIOTests",
dependencies: [
.target(name: "SQLiteNIO"),
],
swiftSettings: swiftSettings
),
]
)

var swiftSettings: [SwiftSetting] { [
.enableUpcomingFeature("ConciseMagicFile"),
.enableUpcomingFeature("ForwardTrailingClosures"),
] }

var sqliteCSettings: [CSetting] { [
// Derived from sqlite3 version 3.43.0
.define("SQLITE_DEFAULT_MEMSTATUS", to: "0"),
.define("SQLITE_DISABLE_PAGECACHE_OVERFLOW_STATS"),
.define("SQLITE_DQS", to: "0"),
.define("SQLITE_ENABLE_API_ARMOR", .when(configuration: .debug)),
.define("SQLITE_ENABLE_COLUMN_METADATA"),
.define("SQLITE_ENABLE_DBSTAT_VTAB"),
.define("SQLITE_ENABLE_FTS3"),
.define("SQLITE_ENABLE_FTS3_PARENTHESIS"),
.define("SQLITE_ENABLE_FTS3_TOKENIZER"),
.define("SQLITE_ENABLE_FTS4"),
.define("SQLITE_ENABLE_FTS5"),
.define("SQLITE_ENABLE_NULL_TRIM"),
.define("SQLITE_ENABLE_RTREE"),
.define("SQLITE_ENABLE_SESSION"),
.define("SQLITE_ENABLE_STMTVTAB"),
.define("SQLITE_ENABLE_UNKNOWN_SQL_FUNCTION"),
.define("SQLITE_ENABLE_UNLOCK_NOTIFY"),
.define("SQLITE_MAX_VARIABLE_NUMBER", to: "250000"),
.define("SQLITE_LIKE_DOESNT_MATCH_BLOBS"),
.define("SQLITE_OMIT_AUTHORIZATION"),
.define("SQLITE_OMIT_COMPLETE"),
.define("SQLITE_OMIT_DEPRECATED"),
.define("SQLITE_OMIT_DESERIALIZE"),
.define("SQLITE_OMIT_GET_TABLE"),
.define("SQLITE_OMIT_LOAD_EXTENSION"),
.define("SQLITE_OMIT_PROGRESS_CALLBACK"),
.define("SQLITE_OMIT_SHARED_CACHE"),
.define("SQLITE_OMIT_TCL_VARIABLE"),
.define("SQLITE_OMIT_TRACE"),
.define("SQLITE_SECURE_DELETE"),
.define("SQLITE_THREADSAFE", to: "1"),
.define("SQLITE_UNTESTABLE"),
.define("SQLITE_USE_URI"),
] }
99 changes: 99 additions & 0 deletions [email protected]
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
// swift-tools-version:5.9
import PackageDescription

let package = Package(
name: "sqlite-nio",
platforms: [
.macOS(.v10_15),
.iOS(.v13),
.watchOS(.v6),
.tvOS(.v13),
],
products: [
.library(name: "SQLiteNIO", targets: ["SQLiteNIO"]),
],
dependencies: [
.package(url: "https://github.com/apple/swift-nio.git", from: "2.65.0"),
.package(url: "https://github.com/apple/swift-log.git", from: "1.5.4"),
],
targets: [
.plugin(
name: "VendorSQLite",
capability: .command(
intent: .custom(verb: "vendor-sqlite", description: "Vendor SQLite"),
permissions: [
.allowNetworkConnections(scope: .all(ports: [443]), reason: "Retrieve the latest build of SQLite"),
.writeToPackageDirectory(reason: "Update the vendored SQLite files"),
]
),
exclude: ["001-warnings-and-data-race.patch"]
),
.target(
name: "CSQLite",
cSettings: sqliteCSettings
),
.target(
name: "SQLiteNIO",
dependencies: [
.target(name: "CSQLite"),
.product(name: "Logging", package: "swift-log"),
.product(name: "NIOCore", package: "swift-nio"),
.product(name: "NIOPosix", package: "swift-nio"),
.product(name: "NIOFoundationCompat", package: "swift-nio"),
],
swiftSettings: swiftSettings
),
.testTarget(
name: "SQLiteNIOTests",
dependencies: [
.target(name: "SQLiteNIO"),
],
swiftSettings: swiftSettings
),
]
)

var swiftSettings: [SwiftSetting] { [
.enableUpcomingFeature("ExistentialAny"),
.enableUpcomingFeature("ConciseMagicFile"),
.enableUpcomingFeature("ForwardTrailingClosures"),
.enableUpcomingFeature("DisableOutwardActorInference"),
.enableExperimentalFeature("StrictConcurrency=complete"),
] }

var sqliteCSettings: [CSetting] { [
// Derived from sqlite3 version 3.43.0
.define("SQLITE_DEFAULT_MEMSTATUS", to: "0"),
.define("SQLITE_DISABLE_PAGECACHE_OVERFLOW_STATS"),
.define("SQLITE_DQS", to: "0"),
.define("SQLITE_ENABLE_API_ARMOR", .when(configuration: .debug)),
.define("SQLITE_ENABLE_COLUMN_METADATA"),
.define("SQLITE_ENABLE_DBSTAT_VTAB"),
.define("SQLITE_ENABLE_FTS3"),
.define("SQLITE_ENABLE_FTS3_PARENTHESIS"),
.define("SQLITE_ENABLE_FTS3_TOKENIZER"),
.define("SQLITE_ENABLE_FTS4"),
.define("SQLITE_ENABLE_FTS5"),
.define("SQLITE_ENABLE_NULL_TRIM"),
.define("SQLITE_ENABLE_RTREE"),
.define("SQLITE_ENABLE_SESSION"),
.define("SQLITE_ENABLE_STMTVTAB"),
.define("SQLITE_ENABLE_UNKNOWN_SQL_FUNCTION"),
.define("SQLITE_ENABLE_UNLOCK_NOTIFY"),
.define("SQLITE_MAX_VARIABLE_NUMBER", to: "250000"),
.define("SQLITE_LIKE_DOESNT_MATCH_BLOBS"),
.define("SQLITE_OMIT_AUTHORIZATION"),
.define("SQLITE_OMIT_COMPLETE"),
.define("SQLITE_OMIT_DEPRECATED"),
.define("SQLITE_OMIT_DESERIALIZE"),
.define("SQLITE_OMIT_GET_TABLE"),
.define("SQLITE_OMIT_LOAD_EXTENSION"),
.define("SQLITE_OMIT_PROGRESS_CALLBACK"),
.define("SQLITE_OMIT_SHARED_CACHE"),
.define("SQLITE_OMIT_TCL_VARIABLE"),
.define("SQLITE_OMIT_TRACE"),
.define("SQLITE_SECURE_DELETE"),
.define("SQLITE_THREADSAFE", to: "1"),
.define("SQLITE_UNTESTABLE"),
.define("SQLITE_USE_URI"),
] }
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@
<a href="https://docs.vapor.codes/4.0/"><img src="https://design.vapor.codes/images/readthedocs.svg" alt="Documentation"></a>
<a href="https://discord.gg/vapor"><img src="https://design.vapor.codes/images/discordchat.svg" alt="Team Chat"></a>
<a href="LICENSE"><img src="https://design.vapor.codes/images/mitlicense.svg" alt="MIT License"></a>
<a href="https://github.com/vapor/sqlite-nio/actions/workflows/test.yml"><img src="https://img.shields.io/github/actions/workflow/status/vapor/sqlite-nio/test.yml?event=push&style=plastic&logo=github&label=test&logoColor=%23ccc" alt="Continuous Integration"></a>
<a href="https://codecov.io/github/vapor/sqlite-nio"><img src="https://img.shields.io/codecov/c/github/vapor/sqlite-nio?style=plastic&logo=codecov&label=Codecov"></a>
<a href="https://swift.org"><img src="https://design.vapor.codes/images/swift57up.svg" alt="Swift 5.7+"></a>
<a href="https://github.com/vapor/sqlite-nio/actions/workflows/test.yml"><img src="https://img.shields.io/github/actions/workflow/status/vapor/sqlite-nio/test.yml?event=push&style=plastic&logo=github&label=tests&logoColor=%23ccc" alt="Continuous Integration"></a>
<a href="https://codecov.io/github/vapor/sqlite-nio"><img src="https://img.shields.io/codecov/c/github/vapor/sqlite-nio?style=plastic&logo=codecov&label=codecov"></a>
<a href="https://swift.org"><img src="https://design.vapor.codes/images/swift58up.svg" alt="Swift 5.8+"></a>
</p>

<br>
Expand Down
5 changes: 2 additions & 3 deletions Sources/SQLiteNIO/Docs.docc/Documentation.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,11 @@
@TitleHeading(Package)
}

🪶 Non-blocking, event-driven Swift client for SQLite with embedded libsqlite
🪶 Non-blocking, event-driven Swift client for SQLite with embedded `libsqlite`.

## Supported Versions

This package is compatible with all platforms supported by [SwiftNIO 2.x](https://github.com/apple/swift-nio/). It has
been specifically tested on the following platforms:
This package is compatible with all platforms supported by [SwiftNIO 2.x](https://github.com/apple/swift-nio/). It has been specifically tested on the following platforms:

- Ubuntu 20.04 ("Focal") and 22.04 ("Jammy")
- Amazon Linux 2
Expand Down
2 changes: 1 addition & 1 deletion Sources/SQLiteNIO/Docs.docc/theme-settings.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"theme": {
"aside": { "border-radius": "6px", "border-style": "double", "border-width": "3px" },
"aside": { "border-radius": "16px", "border-style": "double", "border-width": "3px" },
"border-radius": "0",
"button": { "border-radius": "16px", "border-width": "1px", "border-style": "solid" },
"code": { "border-radius": "16px", "border-width": "1px", "border-style": "solid" },
Expand Down
12 changes: 0 additions & 12 deletions Sources/SQLiteNIO/Exports.swift
Original file line number Diff line number Diff line change
@@ -1,17 +1,5 @@
#if swift(>=5.8)

@_documentation(visibility: internal) @_exported import struct NIOCore.ByteBuffer
@_documentation(visibility: internal) @_exported import class NIOPosix.NIOThreadPool
@_documentation(visibility: internal) @_exported import protocol NIOCore.EventLoop
@_documentation(visibility: internal) @_exported import protocol NIOCore.EventLoopGroup
@_documentation(visibility: internal) @_exported import class NIOPosix.MultiThreadedEventLoopGroup

#else

@_exported import struct NIOCore.ByteBuffer
@_exported import class NIOPosix.NIOThreadPool
@_exported import protocol NIOCore.EventLoop
@_exported import protocol NIOCore.EventLoopGroup
@_exported import class NIOPosix.MultiThreadedEventLoopGroup

#endif
Loading

0 comments on commit 98d2894

Please sign in to comment.