diff --git a/Documentation/Index.md b/Documentation/Index.md index e88ddc04d..18a41771e 100644 --- a/Documentation/Index.md +++ b/Documentation/Index.md @@ -93,6 +93,15 @@ - [Attaching and detaching databases](#attaching-and-detaching-databases) - [Logging](#logging) - [Vacuum](#vacuum) + - [Pragmas and WAL Mode](#pragmas-and-wal-mode) + - [Enabling WAL](#enabling-wal) + - [Why WAL](#why-wal) + - [Auto-skip rules](#auto-skip-rules) + - [Other journal modes](#other-journal-modes) + - [`synchronous`](#synchronous) + - [Checkpointing](#checkpointing) + - [Sidecar files](#sidecar-files) + - [File system caveats](#file-system-caveats) [↩]: #sqliteswift-documentation @@ -2324,5 +2333,164 @@ try db.vacuum() ``` +## Pragmas and WAL Mode + +SQLite.swift exposes typed accessors for the most common +[PRAGMAs](https://sqlite.org/pragma.html) on `Connection`, including +journaling, synchronization, and Write-Ahead Logging (WAL). + +### Enabling WAL + +The simplest way to opt into WAL is at connection time: + +```swift +let db = try Connection("path/to/db.sqlite3", journalMode: .wal) +``` + +This enables WAL **and** sets `synchronous = NORMAL`, the recommended +configuration: durable, fast, and safe against database corruption. + +You can also enable WAL on an existing connection: + +```swift +let mode = try db.enableWAL() +guard mode == .wal else { + // SQLite refused WAL on this database — fall back or log. + return +} +``` + +`enableWAL()` is idempotent: calling it on a database that is already in +WAL mode is a no-op. + +### Why WAL + +In WAL mode, writers append new pages to a `-wal` log file instead of +rewriting the main database. Readers and writers coordinate through a +shared-memory index (`-shm`). Compared to the default `DELETE` (rollback +journal) mode this gives you: + +- Concurrent reads and writes — readers do not block writers and vice + versa. Writes still serialize against each other. +- Fewer `fsync` calls per commit, so faster small transactions. +- Better SSD wear characteristics for write-heavy workloads. + +WAL is persisted in the database file header. Once enabled, subsequent +connections inherit the mode automatically; you do not need to set it +again per connection. + +### Auto-skip rules + +`Connection(_, journalMode:)` skips the configuration step automatically +for databases where WAL is meaningless or unsupported: + +- `.inMemory` and `.temporary` connections. +- URI locations resolving to an empty path or `:memory:`. +- Read-only connections — including those forced read-only by URI + parameters such as `.uri(path, parameters: [.mode(.readOnly)])` or + `.immutable(true)`, or by opening a file the process cannot write. + +In those cases the connection opens normally and the journal mode is left +untouched. + +### Other journal modes + +`JournalMode` covers all of SQLite's options: `.delete` (default), +`.truncate`, `.persist`, `.memory`, `.wal`, `.off`. Use the throwing +`setJournalMode(_:)` if you need to detect the actual mode SQLite applied +(for example, WAL on a network volume is silently downgraded): + +```swift +let actual = try db.setJournalMode(.wal) +if actual != .wal { + log("WAL not supported on this volume; running with \(actual)") +} +``` + +The non-throwing `journalMode` property is also available for read or +fire-and-forget set: + +```swift +db.journalMode = .truncate +print(db.journalMode) +``` + +### `synchronous` + +`Synchronous` mirrors +[`PRAGMA synchronous`](https://sqlite.org/pragma.html#pragma_synchronous): +`.off`, `.normal`, `.full`, `.extra`. Pair WAL with `.normal` (the default +applied by `enableWAL()`); only the most recent committed transaction may +be lost on power failure, and the database itself remains uncorrupted. + +```swift +try db.setSynchronous(.normal) // throwing +db.synchronous = .full // non-throwing setter +``` + +### Checkpointing + +SQLite checkpoints the WAL into the main database file automatically +once it reaches `walAutoCheckpoint` pages (default 1000, ~4 MB at the +default page size). You can tune or disable that: + +```swift +db.walAutoCheckpoint = 1000 // default +db.walAutoCheckpoint = 0 // disable automatic checkpoints +``` + +To force a checkpoint — for example before backing up the file or when +the application is moving to the background — call `walCheckpoint(...)`: + +```swift +let result = try db.walCheckpoint(mode: .truncate) +print("WAL frames: \(result.log), moved: \(result.checkpointed), busy: \(result.busy)") +``` + +The four checkpoint modes mirror SQLite's: + +| Mode | Behaviour | +|------|-----------| +| `.passive` | Best-effort; never blocks readers or writers. | +| `.full` | Waits for active readers to finish, then checkpoints fully. | +| `.restart` | Like `.full`, then resets the WAL file. | +| `.truncate` | Like `.restart`, plus shrinks `-wal` to zero bytes. | + +`.truncate` is the right choice for periodic clean-up since the WAL file +otherwise only ever grows. + +### Sidecar files + +WAL mode creates two sidecar files alongside the main database: + +``` +mydb.sqlite3 +mydb.sqlite3-wal ← write-ahead log +mydb.sqlite3-shm ← shared-memory index +``` + +When you copy, move, back up, or delete the database you must handle all +three (or use the [Online Database Backup](#online-database-backup) API, +which handles the coordination for you). When excluding the database +from iCloud Backup with `URLResourceKey.isExcludedFromBackupKey`, set the +key on every sidecar that exists. + +### File system caveats + +WAL relies on shared-memory locking primitives that are not available on +every storage backend. Avoid WAL on: + +- Network file systems such as NFS, SMB, or iCloud Drive sync folders — + using WAL there can corrupt the database. +- Read-only media (the connection has nowhere to write the `-wal`/`-shm` + files). + +On iOS, if the database file uses the `.complete` data protection class +the `-wal` and `-shm` files become inaccessible while the device is +locked, which causes background writes to fail. Use +`.completeUntilFirstUserAuthentication` (or a less strict class) for +databases that must be writable in the background. + + [ROWID]: https://sqlite.org/lang_createtable.html#rowid [SQLiteMigrationManager.swift]: https://github.com/garriguv/SQLiteMigrationManager.swift diff --git a/Documentation/Upgrading.md b/Documentation/Upgrading.md index 0e12aacf0..b2ea6c501 100644 --- a/Documentation/Upgrading.md +++ b/Documentation/Upgrading.md @@ -1,5 +1,25 @@ # Upgrading +## Unreleased + +WAL mode and related journaling APIs are now first-class: + +- Both `Connection.init(_ location:readonly:journalMode:)` and + `Connection.init(_ filename:readonly:journalMode:)` accept an optional + `journalMode:` parameter. When set to `.wal`, WAL is enabled with + `synchronous = NORMAL`. The configuration step is skipped automatically for + `.inMemory`, `.temporary`, empty URIs, and any connection that opens + read-only — including read-only URI parameters such as + `.uri(path, parameters: [.mode(.readOnly)])` and `.immutable(true)`. +- New `Connection.JournalMode`, `Connection.Synchronous`, and + `Connection.WALCheckpointMode` types in `Connection+Pragmas.swift`. +- New connection API: `journalMode`, `synchronous`, `walAutoCheckpoint` + properties, plus the throwing helpers `setJournalMode(_:)`, + `setSynchronous(_:)`, `enableWAL()`, and `walCheckpoint(mode:schema:)`. + +These additions are source-compatible with existing code; no migration is +required for callers that do not opt in. + ## 0.13 → 0.14 - `Expression.asSQL()` is no longer available. Expressions now implement `CustomStringConvertible`, diff --git a/README.md b/README.md index 126b7344b..71f2905a5 100644 --- a/README.md +++ b/README.md @@ -20,6 +20,9 @@ syntax _and_ intent. - Extensively tested - [SQLCipher][] support via Swift Package Manager - [Schema query/migration][] + - First-class WAL mode and journaling configuration via + `Connection(_, journalMode: .wal)` and the `enableWAL()` / `walCheckpoint(...)` + APIs - Works on [Linux](Documentation/Linux.md) (with some limitations) - Active support at [StackOverflow](https://stackoverflow.com/questions/tagged/sqlite.swift), diff --git a/Sources/SQLite/Core/Connection+Pragmas.swift b/Sources/SQLite/Core/Connection+Pragmas.swift index 2c4f0efbb..7342b474b 100644 --- a/Sources/SQLite/Core/Connection+Pragmas.swift +++ b/Sources/SQLite/Core/Connection+Pragmas.swift @@ -39,6 +39,121 @@ public extension Connection { set { setBoolPragma("defer_foreign_keys", newValue) } } + /// The journal mode for the main database. + /// See SQLite [PRAGMA journal_mode](https://sqlite.org/pragma.html#pragma_journal_mode) + /// + /// `WAL` is persistent across connections (stored in the database header), so + /// setting this once is sufficient. Setting may silently no-op for some + /// databases (e.g. `:memory:`, network file systems). Use `setJournalMode(_:)` + /// to verify the new mode. + var journalMode: JournalMode { + get { + guard let raw = (try? scalar("PRAGMA journal_mode")) as? String, + let mode = JournalMode(rawValue: raw.lowercased()) else { return .delete } + return mode + } + set { + _ = try? setJournalMode(newValue) + } + } + + /// Sets the journal mode and returns the mode actually in effect after the + /// change. SQLite reports the resulting mode; callers should compare against + /// `mode` to detect failures (e.g. WAL on a network file system). + @discardableResult + func setJournalMode(_ mode: JournalMode) throws -> JournalMode { + guard let raw = try scalar("PRAGMA journal_mode = \(mode.rawValue.uppercased())") as? String, + let result = JournalMode(rawValue: raw.lowercased()) else { + return .delete + } + return result + } + + /// Enables WAL journaling and pairs it with `synchronous = NORMAL`, which is + /// the recommended configuration: durable, fast, and safe against database + /// corruption (only the most recent committed transaction may be lost on + /// power failure). + /// + /// Idempotent. Safe to call on every connection — WAL is persisted in the + /// database header, so subsequent connections inherit it, but calling this + /// again is a no-op. + /// + /// SQLite silently downgrades WAL on databases that cannot support it + /// (`:memory:`, network file systems, read-only media). Inspect the return + /// value to detect this case; `synchronous` is only changed when WAL was + /// successfully applied. + /// + /// - Returns: The journal mode actually in effect after the call. + /// - Throws: `Result.Error` if the pragma cannot be executed. + @discardableResult + func enableWAL() throws -> JournalMode { + let mode = try setJournalMode(.wal) + if mode == .wal { + try setSynchronous(.normal) + } + return mode + } + + /// Throwing equivalent of the `synchronous` setter; surfaces SQLite errors + /// instead of silently swallowing them. + func setSynchronous(_ mode: Synchronous) throws { + try run("PRAGMA synchronous = \(mode.rawValue)") + } + + /// The disk synchronization mode. `NORMAL` is the recommended pairing with + /// WAL: durable, fast, and safe against corruption (only the most recent + /// committed transaction may be lost on power failure). + /// See SQLite [PRAGMA synchronous](https://sqlite.org/pragma.html#pragma_synchronous) + var synchronous: Synchronous { + get { + guard let value = (try? scalar("PRAGMA synchronous")) as? Int64, + let mode = Synchronous(rawValue: Int(value)) else { return .full } + return mode + } + set { + _ = try? run("PRAGMA synchronous = \(newValue.rawValue)") + } + } + + /// The WAL auto-checkpoint threshold in pages. SQLite checkpoints + /// automatically once the WAL reaches this many pages (default 1000). + /// Set to 0 (or negative) to disable automatic checkpoints. + /// See SQLite [PRAGMA wal_autocheckpoint](https://sqlite.org/pragma.html#pragma_wal_autocheckpoint) + var walAutoCheckpoint: Int { + get { + guard let value = (try? scalar("PRAGMA wal_autocheckpoint")) as? Int64 else { return 1000 } + return Int(value) + } + set { + _ = try? run("PRAGMA wal_autocheckpoint = \(newValue)") + } + } + + /// Runs a WAL checkpoint and returns its result. + /// + /// - Parameters: + /// - mode: The checkpoint mode. Defaults to `.passive`. + /// - schema: The attached database schema to checkpoint, or `nil` for all + /// attached databases. + /// - Returns: A tuple of: + /// - `busy`: `true` if the checkpoint could not complete because a reader + /// or writer prevented it. + /// - `log`: Number of frames in the WAL. + /// - `checkpointed`: Number of frames moved from the WAL to the database + /// file (or -1 if not in WAL mode). + /// - Throws: `Result.Error` if the pragma cannot be executed. + @discardableResult + func walCheckpoint(mode: WALCheckpointMode = .passive, schema: String? = nil) throws + -> (busy: Bool, log: Int, checkpointed: Int) { + let scope = schema.map { "\($0.quote())." } ?? "" + let stmt = try prepare("PRAGMA \(scope)wal_checkpoint(\(mode.rawValue))") + _ = try stmt.step() + let busy: Int64 = stmt.row[0] + let log: Int64 = stmt.row[1] + let checkpointed: Int64 = stmt.row[2] + return (busy != 0, Int(log), Int(checkpointed)) + } + private func getBoolPragma(_ key: String) -> Bool { guard let binding = try? scalar("PRAGMA \(key)"), let intBinding = binding as? Int64 else { return false } @@ -49,3 +164,34 @@ public extension Connection { _ = try? run("PRAGMA \(key) = \(newValue ? "1" : "0")") } } + +public extension Connection { + /// SQLite journal modes. + /// See + enum JournalMode: String, CaseIterable { + case delete + case truncate + case persist + case memory + case wal + case off + } + + /// SQLite `synchronous` settings. + /// See + enum Synchronous: Int, CaseIterable { + case off = 0 + case normal = 1 + case full = 2 + case extra = 3 + } + + /// WAL checkpoint modes. + /// See + enum WALCheckpointMode: String { + case passive = "PASSIVE" + case full = "FULL" + case restart = "RESTART" + case truncate = "TRUNCATE" + } +} diff --git a/Sources/SQLite/Core/Connection.swift b/Sources/SQLite/Core/Connection.swift index 00ba064c3..7529c6788 100644 --- a/Sources/SQLite/Core/Connection.swift +++ b/Sources/SQLite/Core/Connection.swift @@ -102,14 +102,33 @@ public final class Connection { /// /// Default: `false`. /// + /// - journalMode: If non-nil, sets the journal mode after opening. Skipped + /// for `.inMemory`, `.temporary`, empty URIs, and read-only connections, + /// where SQLite cannot persist a different journal mode. Use `.wal` to + /// opt into WAL with `synchronous = NORMAL`. + /// + /// Default: `nil` (leaves the database’s current journal mode intact). + /// /// - Returns: A new database connection. - public init(_ location: Location = .inMemory, readonly: Bool = false) throws { + public init(_ location: Location = .inMemory, readonly: Bool = false, journalMode: JournalMode? = nil) throws { let flags = readonly ? SQLITE_OPEN_READONLY : (SQLITE_OPEN_CREATE | SQLITE_OPEN_READWRITE) try check(sqlite3_open_v2(location.description, &_handle, flags | SQLITE_OPEN_FULLMUTEX | SQLITE_OPEN_URI, nil)) queue.setSpecific(key: Connection.queueKey, value: queueContext) + + // Use the live `readonly` flag rather than the init parameter so that + // URI-based read-only connections (e.g. `mode=ro`, `immutable=1`) and + // `SQLITE_OPEN_READONLY` opens of files lacking write permission are + // also skipped. + if let journalMode, !self.readonly, Connection.canConfigureJournalMode(for: location) { + if journalMode == .wal { + try enableWAL() + } else { + try setJournalMode(journalMode) + } + } } /// Initializes a new connection to a database. @@ -123,11 +142,22 @@ public final class Connection { /// /// Default: `false`. /// + /// - journalMode: See `init(_:readonly:journalMode:)`. Default: `nil`. + /// /// - Throws: `Result.Error` iff a connection cannot be established. /// /// - Returns: A new database connection. - public convenience init(_ filename: String, readonly: Bool = false) throws { - try self.init(.uri(filename), readonly: readonly) + public convenience init(_ filename: String, readonly: Bool = false, journalMode: JournalMode? = nil) throws { + try self.init(.uri(filename), readonly: readonly, journalMode: journalMode) + } + + private static func canConfigureJournalMode(for location: Location) -> Bool { + switch location { + case .inMemory, .temporary: + return false + case let .uri(path, _): + return !path.isEmpty && path != ":memory:" + } } deinit { diff --git a/Tests/SQLiteTests/Core/Connection+PragmaTests.swift b/Tests/SQLiteTests/Core/Connection+PragmaTests.swift index c039a7cc0..7333e4d7f 100644 --- a/Tests/SQLiteTests/Core/Connection+PragmaTests.swift +++ b/Tests/SQLiteTests/Core/Connection+PragmaTests.swift @@ -29,4 +29,143 @@ class ConnectionPragmaTests: SQLiteTestCase { db.deferForeignKeys = true XCTAssertTrue(db.deferForeignKeys) } + + // MARK: - journal_mode / synchronous / WAL + + func test_journalMode_defaults_to_memory_for_in_memory_database() { + // In-memory databases cannot use WAL; SQLite reports `memory`. + XCTAssertEqual(db.journalMode, .memory) + } + + func test_journalMode_round_trip_on_file_database() throws { + try withTemporaryDatabasePath(prefix: "sqlite-swift-wal") { path in + let fileDB = try Connection(path) + let applied = try fileDB.setJournalMode(.wal) + XCTAssertEqual(applied, .wal) + XCTAssertEqual(fileDB.journalMode, .wal) + + let reverted = try fileDB.setJournalMode(.delete) + XCTAssertEqual(reverted, .delete) + } + } + + func test_setJournalMode_returns_actual_mode_for_in_memory() throws { + // SQLite silently downgrades WAL to memory for `:memory:` databases. + let applied = try db.setJournalMode(.wal) + XCTAssertEqual(applied, .memory) + } + + func test_synchronous_round_trip() { + db.synchronous = .normal + XCTAssertEqual(db.synchronous, .normal) + db.synchronous = .full + XCTAssertEqual(db.synchronous, .full) + } + + func test_walAutoCheckpoint_round_trip() { + db.walAutoCheckpoint = 500 + XCTAssertEqual(db.walAutoCheckpoint, 500) + } + + func test_walCheckpoint_returns_zero_pages_when_not_in_wal_mode() throws { + // Not in WAL mode → log/checkpointed are -1. + let result = try db.walCheckpoint() + XCTAssertFalse(result.busy) + XCTAssertEqual(result.log, -1) + XCTAssertEqual(result.checkpointed, -1) + } + + func test_enableWAL_on_file_database_sets_wal_and_synchronous_normal() throws { + try withTemporaryDatabasePath(prefix: "sqlite-swift-enablewal") { path in + let fileDB = try Connection(path) + let mode = try fileDB.enableWAL() + XCTAssertEqual(mode, .wal) + XCTAssertEqual(fileDB.journalMode, .wal) + XCTAssertEqual(fileDB.synchronous, .normal) + } + } + + func test_enableWAL_is_idempotent() throws { + try withTemporaryDatabasePath(prefix: "sqlite-swift-enablewal-idem") { path in + let fileDB = try Connection(path) + XCTAssertEqual(try fileDB.enableWAL(), .wal) + XCTAssertEqual(try fileDB.enableWAL(), .wal) + } + } + + func test_enableWAL_does_not_touch_synchronous_when_wal_unsupported() throws { + // In-memory database cannot use WAL; synchronous should be left untouched. + db.synchronous = .full + let mode = try db.enableWAL() + XCTAssertNotEqual(mode, .wal) + XCTAssertEqual(db.synchronous, .full) + } + + // MARK: - init journalMode parameter + + func test_init_with_journalMode_wal_applies_on_file_database() throws { + try withTemporaryDatabasePath(prefix: "sqlite-swift-init-wal") { path in + let fileDB = try Connection(path, journalMode: .wal) + XCTAssertEqual(fileDB.journalMode, .wal) + XCTAssertEqual(fileDB.synchronous, .normal) + } + } + + func test_init_with_journalMode_wal_skipped_for_in_memory() throws { + let memoryDB = try Connection(.inMemory, journalMode: .wal) + XCTAssertEqual(memoryDB.journalMode, .memory) + } + + func test_init_with_journalMode_wal_skipped_for_temporary() throws { + let tempDB = try Connection(.temporary, journalMode: .wal) + XCTAssertNotEqual(tempDB.journalMode, .wal) + } + + func test_init_with_journalMode_wal_skipped_for_readonly() throws { + try withTemporaryDatabasePath(prefix: "sqlite-swift-init-readonly") { path in + // Create the database file first; readonly cannot create. + _ = try Connection(path) + let readonlyDB = try Connection(path, readonly: true, journalMode: .wal) + XCTAssertNotEqual(readonlyDB.journalMode, .wal) + } + } + + func test_init_with_journalMode_wal_skipped_for_uri_readonly_mode() throws { + try withTemporaryDatabasePath(prefix: "sqlite-swift-init-uri-ro") { path in + // Create the database first so the read-only URI open succeeds. + _ = try Connection(path) + // `readonly: false` here, but the URI parameter forces read-only. + // The init must consult the live `readonly` flag, not just its parameter. + let uriRO = try Connection(.uri(path, parameters: [.mode(.readOnly)]), + readonly: false, + journalMode: .wal) + XCTAssertTrue(uriRO.readonly) + XCTAssertNotEqual(uriRO.journalMode, .wal) + } + } + + func test_init_with_journalMode_truncate_applies_on_file_database() throws { + try withTemporaryDatabasePath(prefix: "sqlite-swift-init-truncate") { path in + let fileDB = try Connection(path, journalMode: .truncate) + XCTAssertEqual(fileDB.journalMode, .truncate) + } + } + + // MARK: - Helpers + + /// Creates a unique temporary database path, runs the block, then removes + /// the main file plus SQLite's `-wal` / `-shm` sidecars (which use a `-` + /// separator, not a `.` extension). + private func withTemporaryDatabasePath(prefix: String, + _ block: (String) throws -> Void) throws { + let path = FileManager.default.temporaryDirectory + .appendingPathComponent("\(prefix)-\(UUID().uuidString).sqlite3") + .path + defer { + for sidecar in ["", "-wal", "-shm", "-journal"] { + try? FileManager.default.removeItem(atPath: path + sidecar) + } + } + try block(path) + } }