Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
168 changes: 168 additions & 0 deletions Documentation/Index.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,15 @@
- [Attaching and detaching databases](#attaching-and-detaching-databases)
- [Logging](#logging)
- [Vacuum](#vacuum)
- [Pragmas and WAL Mode](#pragmas-and-wal-mode)
- [Enabling WAL](#enabling-wal)
- [Why WAL](#why-wal)
- [Auto-skip rules](#auto-skip-rules)
- [Other journal modes](#other-journal-modes)
- [`synchronous`](#synchronous)
- [Checkpointing](#checkpointing)
- [Sidecar files](#sidecar-files)
- [File system caveats](#file-system-caveats)

[↩]: #sqliteswift-documentation

Expand Down Expand Up @@ -2324,5 +2333,164 @@ try db.vacuum()
```


## Pragmas and WAL Mode

SQLite.swift exposes typed accessors for the most common
[PRAGMAs](https://sqlite.org/pragma.html) on `Connection`, including
journaling, synchronization, and Write-Ahead Logging (WAL).

### Enabling WAL

The simplest way to opt into WAL is at connection time:

```swift
let db = try Connection("path/to/db.sqlite3", journalMode: .wal)
```

This enables WAL **and** sets `synchronous = NORMAL`, the recommended
configuration: durable, fast, and safe against database corruption.

You can also enable WAL on an existing connection:

```swift
let mode = try db.enableWAL()
guard mode == .wal else {
// SQLite refused WAL on this database — fall back or log.
return
}
```

`enableWAL()` is idempotent: calling it on a database that is already in
WAL mode is a no-op.

### Why WAL

In WAL mode, writers append new pages to a `-wal` log file instead of
rewriting the main database. Readers and writers coordinate through a
shared-memory index (`-shm`). Compared to the default `DELETE` (rollback
journal) mode this gives you:

- Concurrent reads and writes — readers do not block writers and vice
versa. Writes still serialize against each other.
- Fewer `fsync` calls per commit, so faster small transactions.
- Better SSD wear characteristics for write-heavy workloads.

WAL is persisted in the database file header. Once enabled, subsequent
connections inherit the mode automatically; you do not need to set it
again per connection.

### Auto-skip rules

`Connection(_, journalMode:)` skips the configuration step automatically
for databases where WAL is meaningless or unsupported:

- `.inMemory` and `.temporary` connections.
- URI locations resolving to an empty path or `:memory:`.
- Read-only connections — including those forced read-only by URI
parameters such as `.uri(path, parameters: [.mode(.readOnly)])` or
`.immutable(true)`, or by opening a file the process cannot write.

In those cases the connection opens normally and the journal mode is left
untouched.

### Other journal modes

`JournalMode` covers all of SQLite's options: `.delete` (default),
`.truncate`, `.persist`, `.memory`, `.wal`, `.off`. Use the throwing
`setJournalMode(_:)` if you need to detect the actual mode SQLite applied
(for example, WAL on a network volume is silently downgraded):

```swift
let actual = try db.setJournalMode(.wal)
if actual != .wal {
log("WAL not supported on this volume; running with \(actual)")
}
```

The non-throwing `journalMode` property is also available for read or
fire-and-forget set:

```swift
db.journalMode = .truncate
print(db.journalMode)
```

### `synchronous`

`Synchronous` mirrors
[`PRAGMA synchronous`](https://sqlite.org/pragma.html#pragma_synchronous):
`.off`, `.normal`, `.full`, `.extra`. Pair WAL with `.normal` (the default
applied by `enableWAL()`); only the most recent committed transaction may
be lost on power failure, and the database itself remains uncorrupted.

```swift
try db.setSynchronous(.normal) // throwing
db.synchronous = .full // non-throwing setter
```

### Checkpointing

SQLite checkpoints the WAL into the main database file automatically
once it reaches `walAutoCheckpoint` pages (default 1000, ~4 MB at the
default page size). You can tune or disable that:

```swift
db.walAutoCheckpoint = 1000 // default
db.walAutoCheckpoint = 0 // disable automatic checkpoints
```

To force a checkpoint — for example before backing up the file or when
the application is moving to the background — call `walCheckpoint(...)`:

```swift
let result = try db.walCheckpoint(mode: .truncate)
print("WAL frames: \(result.log), moved: \(result.checkpointed), busy: \(result.busy)")
```

The four checkpoint modes mirror SQLite's:

| Mode | Behaviour |
|------|-----------|
| `.passive` | Best-effort; never blocks readers or writers. |
| `.full` | Waits for active readers to finish, then checkpoints fully. |
| `.restart` | Like `.full`, then resets the WAL file. |
| `.truncate` | Like `.restart`, plus shrinks `-wal` to zero bytes. |

`.truncate` is the right choice for periodic clean-up since the WAL file
otherwise only ever grows.

### Sidecar files

WAL mode creates two sidecar files alongside the main database:

```
mydb.sqlite3
mydb.sqlite3-wal ← write-ahead log
mydb.sqlite3-shm ← shared-memory index
```

When you copy, move, back up, or delete the database you must handle all
three (or use the [Online Database Backup](#online-database-backup) API,
which handles the coordination for you). When excluding the database
from iCloud Backup with `URLResourceKey.isExcludedFromBackupKey`, set the
key on every sidecar that exists.

### File system caveats

WAL relies on shared-memory locking primitives that are not available on
every storage backend. Avoid WAL on:

- Network file systems such as NFS, SMB, or iCloud Drive sync folders —
using WAL there can corrupt the database.
- Read-only media (the connection has nowhere to write the `-wal`/`-shm`
files).

On iOS, if the database file uses the `.complete` data protection class
the `-wal` and `-shm` files become inaccessible while the device is
locked, which causes background writes to fail. Use
`.completeUntilFirstUserAuthentication` (or a less strict class) for
databases that must be writable in the background.


[ROWID]: https://sqlite.org/lang_createtable.html#rowid
[SQLiteMigrationManager.swift]: https://github.com/garriguv/SQLiteMigrationManager.swift
20 changes: 20 additions & 0 deletions Documentation/Upgrading.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,25 @@
# Upgrading

## Unreleased

WAL mode and related journaling APIs are now first-class:

- Both `Connection.init(_ location:readonly:journalMode:)` and
`Connection.init(_ filename:readonly:journalMode:)` accept an optional
`journalMode:` parameter. When set to `.wal`, WAL is enabled with
`synchronous = NORMAL`. The configuration step is skipped automatically for
`.inMemory`, `.temporary`, empty URIs, and any connection that opens
read-only — including read-only URI parameters such as
`.uri(path, parameters: [.mode(.readOnly)])` and `.immutable(true)`.
- New `Connection.JournalMode`, `Connection.Synchronous`, and
`Connection.WALCheckpointMode` types in `Connection+Pragmas.swift`.
- New connection API: `journalMode`, `synchronous`, `walAutoCheckpoint`
properties, plus the throwing helpers `setJournalMode(_:)`,
`setSynchronous(_:)`, `enableWAL()`, and `walCheckpoint(mode:schema:)`.

These additions are source-compatible with existing code; no migration is
required for callers that do not opt in.

## 0.13 → 0.14

- `Expression.asSQL()` is no longer available. Expressions now implement `CustomStringConvertible`,
Expand Down
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,9 @@ syntax _and_ intent.
- Extensively tested
- [SQLCipher][] support via Swift Package Manager
- [Schema query/migration][]
- First-class WAL mode and journaling configuration via
`Connection(_, journalMode: .wal)` and the `enableWAL()` / `walCheckpoint(...)`
APIs
- Works on [Linux](Documentation/Linux.md) (with some limitations)
- Active support at
[StackOverflow](https://stackoverflow.com/questions/tagged/sqlite.swift),
Expand Down
146 changes: 146 additions & 0 deletions Sources/SQLite/Core/Connection+Pragmas.swift
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,121 @@ public extension Connection {
set { setBoolPragma("defer_foreign_keys", newValue) }
}

/// The journal mode for the main database.
/// See SQLite [PRAGMA journal_mode](https://sqlite.org/pragma.html#pragma_journal_mode)
///
/// `WAL` is persistent across connections (stored in the database header), so
/// setting this once is sufficient. Setting may silently no-op for some
/// databases (e.g. `:memory:`, network file systems). Use `setJournalMode(_:)`
/// to verify the new mode.
var journalMode: JournalMode {
get {
guard let raw = (try? scalar("PRAGMA journal_mode")) as? String,
let mode = JournalMode(rawValue: raw.lowercased()) else { return .delete }
return mode
}
set {
_ = try? setJournalMode(newValue)
}
}

/// Sets the journal mode and returns the mode actually in effect after the
/// change. SQLite reports the resulting mode; callers should compare against
/// `mode` to detect failures (e.g. WAL on a network file system).
@discardableResult
func setJournalMode(_ mode: JournalMode) throws -> JournalMode {
guard let raw = try scalar("PRAGMA journal_mode = \(mode.rawValue.uppercased())") as? String,
let result = JournalMode(rawValue: raw.lowercased()) else {
return .delete
}
return result
}

/// Enables WAL journaling and pairs it with `synchronous = NORMAL`, which is
/// the recommended configuration: durable, fast, and safe against database
/// corruption (only the most recent committed transaction may be lost on
/// power failure).
///
/// Idempotent. Safe to call on every connection — WAL is persisted in the
/// database header, so subsequent connections inherit it, but calling this
/// again is a no-op.
///
/// SQLite silently downgrades WAL on databases that cannot support it
/// (`:memory:`, network file systems, read-only media). Inspect the return
/// value to detect this case; `synchronous` is only changed when WAL was
/// successfully applied.
///
/// - Returns: The journal mode actually in effect after the call.
/// - Throws: `Result.Error` if the pragma cannot be executed.
@discardableResult
func enableWAL() throws -> JournalMode {
let mode = try setJournalMode(.wal)
if mode == .wal {
try setSynchronous(.normal)
}
return mode
}

/// Throwing equivalent of the `synchronous` setter; surfaces SQLite errors
/// instead of silently swallowing them.
func setSynchronous(_ mode: Synchronous) throws {
try run("PRAGMA synchronous = \(mode.rawValue)")
}

/// The disk synchronization mode. `NORMAL` is the recommended pairing with
/// WAL: durable, fast, and safe against corruption (only the most recent
/// committed transaction may be lost on power failure).
/// See SQLite [PRAGMA synchronous](https://sqlite.org/pragma.html#pragma_synchronous)
var synchronous: Synchronous {
get {
guard let value = (try? scalar("PRAGMA synchronous")) as? Int64,
let mode = Synchronous(rawValue: Int(value)) else { return .full }
return mode
}
set {
_ = try? run("PRAGMA synchronous = \(newValue.rawValue)")
}
}

/// The WAL auto-checkpoint threshold in pages. SQLite checkpoints
/// automatically once the WAL reaches this many pages (default 1000).
/// Set to 0 (or negative) to disable automatic checkpoints.
/// See SQLite [PRAGMA wal_autocheckpoint](https://sqlite.org/pragma.html#pragma_wal_autocheckpoint)
var walAutoCheckpoint: Int {
get {
guard let value = (try? scalar("PRAGMA wal_autocheckpoint")) as? Int64 else { return 1000 }
return Int(value)
}
set {
_ = try? run("PRAGMA wal_autocheckpoint = \(newValue)")
}
}

/// Runs a WAL checkpoint and returns its result.
///
/// - Parameters:
/// - mode: The checkpoint mode. Defaults to `.passive`.
/// - schema: The attached database schema to checkpoint, or `nil` for all
/// attached databases.
/// - Returns: A tuple of:
/// - `busy`: `true` if the checkpoint could not complete because a reader
/// or writer prevented it.
/// - `log`: Number of frames in the WAL.
/// - `checkpointed`: Number of frames moved from the WAL to the database
/// file (or -1 if not in WAL mode).
/// - Throws: `Result.Error` if the pragma cannot be executed.
@discardableResult
func walCheckpoint(mode: WALCheckpointMode = .passive, schema: String? = nil) throws
-> (busy: Bool, log: Int, checkpointed: Int) {
let scope = schema.map { "\($0.quote())." } ?? ""
let stmt = try prepare("PRAGMA \(scope)wal_checkpoint(\(mode.rawValue))")
_ = try stmt.step()
let busy: Int64 = stmt.row[0]
let log: Int64 = stmt.row[1]
let checkpointed: Int64 = stmt.row[2]
return (busy != 0, Int(log), Int(checkpointed))
}

private func getBoolPragma(_ key: String) -> Bool {
guard let binding = try? scalar("PRAGMA \(key)"),
let intBinding = binding as? Int64 else { return false }
Expand All @@ -49,3 +164,34 @@ public extension Connection {
_ = try? run("PRAGMA \(key) = \(newValue ? "1" : "0")")
}
}

public extension Connection {
/// SQLite journal modes.
/// See <https://sqlite.org/pragma.html#pragma_journal_mode>
enum JournalMode: String, CaseIterable {
case delete
case truncate
case persist
case memory
case wal
case off
}

/// SQLite `synchronous` settings.
/// See <https://sqlite.org/pragma.html#pragma_synchronous>
enum Synchronous: Int, CaseIterable {
case off = 0
case normal = 1
case full = 2
case extra = 3
}

/// WAL checkpoint modes.
/// See <https://sqlite.org/pragma.html#pragma_wal_checkpoint>
enum WALCheckpointMode: String {
case passive = "PASSIVE"
case full = "FULL"
case restart = "RESTART"
case truncate = "TRUNCATE"
}
}
Loading
Loading