Skip to content
Draft
Show file tree
Hide file tree
Changes from 75 commits
Commits
Show all changes
79 commits
Select commit Hold shift + click to select a range
feadf80
Initial commit
Choochmeque Mar 25, 2026
d797611
C++ code formatting
Choochmeque Mar 25, 2026
ec44a32
Add GitHub Actions workflow for Rust project with checks and tests (e…
Choochmeque Mar 25, 2026
daa1adf
Improve error handling in FFI callbacks to suppress panics
Choochmeque Apr 2, 2026
79eec1d
Update README with new FDB API usage examples and correct method calls
Choochmeque Apr 2, 2026
5c27e96
Refactor axes() to use dynamic mapping instead of hardcoded axis names
Choochmeque Apr 2, 2026
014c3de
Add safety comments for thread-safety of various iterators in FDB
Choochmeque Apr 2, 2026
6888dec
Add integration tests for FDB methods: archive_raw, read_uri, read_ur…
Choochmeque Apr 2, 2026
5d43c67
Fix CMake patching error handling and improve seek position calculations
Choochmeque Apr 2, 2026
0886506
Refactor FDB configuration accessors to return Option types for missi…
Choochmeque Apr 2, 2026
5f5b02b
Add concurrent archive and mixed read/write tests for FDB operations
Choochmeque Apr 2, 2026
4320482
Improve exception handling messages in fdb_bridge
Choochmeque Apr 2, 2026
0a4d9b0
Update rust CI workflow
Choochmeque Apr 7, 2026
b9e0404
Refactor FdbHandle to use direct FDB instance instead of unique_ptr
Choochmeque Apr 8, 2026
df834bc
Remove unnecessary drop statements from FDB test cases
Choochmeque Apr 8, 2026
af8d471
Add support for user configuration in FdbHandle initialization and API
Choochmeque Apr 8, 2026
ae1d491
Update iterator methods to return Result<bool> for hasNext checks
Choochmeque Apr 8, 2026
baa976d
Add Cargo configuration and improve control identifier handling in FDB
Choochmeque Apr 8, 2026
cbcc8b7
Update GRIB support and improve URI handling in FDB integration tests
Choochmeque Apr 8, 2026
990cb6e
Remove AxesIteratorHandle and related axes iterator functionality
Choochmeque Apr 8, 2026
0fe11aa
Update log message for empty axes case to clarify request context
Choochmeque Apr 8, 2026
7d2bcc3
Remove debug print statement for FDB name in example code
Choochmeque Apr 8, 2026
3a7672b
Update fdb_list example to format output as key-value pairs in braces
Choochmeque Apr 8, 2026
47ddce5
Remove FDB configuration methods and related data structures to simpl…
Choochmeque Apr 9, 2026
f03a576
Update documentation for HandleInner to clarify FDB instance usage
Choochmeque Apr 9, 2026
826f9e1
Update README to correct FDB name and improve description clarity
Choochmeque Apr 9, 2026
57ef2ed
Update README files to clarify feature flags and build strategies for…
Choochmeque Apr 9, 2026
117cf99
Update README and examples to clarify key requirements for FDB usage
Choochmeque Apr 9, 2026
a53a1c2
Add `memfs` feature to bake eccodes tables into libeccodes for easier…
Choochmeque Apr 9, 2026
1b1c6b5
Refactor version retrieval to use module-level functions instead of m…
Choochmeque Apr 9, 2026
cbddb3b
Replace std::runtime_error with eckit exceptions in DataReader and it…
Choochmeque Apr 9, 2026
76a9c50
Refactor Key struct to use fdb_sys::KeyData
Choochmeque Apr 9, 2026
3582a80
Add function to determine CMake build type based on Cargo profile set…
Choochmeque Apr 9, 2026
9d2cf2d
Update MARS request parsing to use metkit's parser and expansion logic
Choochmeque Apr 9, 2026
a096b5d
Add path-based constructors to FdbHandle for loading configurations d…
Choochmeque Apr 9, 2026
d1311fc
Add clap dependency and implement CLI for fdb_list example tool
Choochmeque Apr 9, 2026
08f099d
Refactor FDB handle creation to use `Fdb::open` instead of `Fdb::new`
Choochmeque Apr 9, 2026
6a9dffc
Add ListOptions struct for improved parameter handling in Fdb methods
Choochmeque Apr 9, 2026
9170f39
Add streaming support for archiving GRIB data from Rust `Read` sources
Choochmeque Apr 9, 2026
2d12394
Add examples for `fdb_read` and `fdb_write` tools with CLI usage inst…
Choochmeque Apr 9, 2026
55b958e
Add detailed index and database statistics reporting in FDB bridge
Choochmeque Apr 9, 2026
027ca66
Refactor DataReaderHandle to eckit::DataHandle shim functions
Choochmeque Apr 9, 2026
ae445d4
Add integration test for FDB axes to validate expected values returned
Choochmeque Apr 9, 2026
fd9b094
Add bindman-utils dependency and refactor build scripts
Choochmeque Apr 11, 2026
39232f3
Add fdb-hammer tool for benchmarking and stress testing FDB performance
Choochmeque Apr 13, 2026
5f86390
Add integration testing workflow for rust-bindings branch in CI
Choochmeque Apr 13, 2026
bfa19e7
Add compact listing functionality to ListIterator for MARS-request ag…
Choochmeque Apr 15, 2026
0834d5c
Remove MoveIterator and related move_data functionality from FDB API
Choochmeque Apr 15, 2026
bd495fd
Enhance rpath handling in fdb build scripts
Choochmeque Apr 15, 2026
1ca42b3
Update README.md to clarify binary execution without environment vari…
Choochmeque Apr 15, 2026
40de393
Update fdb_axes example to use structured argument parsing with clap
Choochmeque Apr 15, 2026
5984a13
Remove unnecessary FDB handle creation logs from examples
Choochmeque Apr 15, 2026
3705937
Add IndexMap dependency and update Request struct for key-value manag…
Choochmeque Apr 16, 2026
c20c3ec
Remove unnecessary FDB_DIR parameter from cmake_find_package call
Choochmeque Apr 16, 2026
4001f5d
Remove branch specification for bindman-utils in Cargo.toml
Choochmeque Apr 16, 2026
17e0fc1
Remove unnecessary integration test steps and update test commands
Choochmeque Apr 16, 2026
902007f
Add mutex lock to serialize GRIB ingest across Fdb instances to preve…
Choochmeque Apr 16, 2026
b519f16
Use eckit/metkit in -sys crate
Choochmeque Apr 20, 2026
dd0a86b
Add metkit as a workspace dependency and update retrieve method to us…
Choochmeque Apr 20, 2026
12144bf
Refactor FDB request handling to use MarsRequestWrapper
Choochmeque Apr 20, 2026
995578e
Refactor FDB to use DataHandleWrapper for data retrieval and reading
Choochmeque Apr 20, 2026
47c8cc0
Refactor FdbHandle constructors to use eckit_bridge::ConfigWrapper fo…
Choochmeque Apr 21, 2026
13fe913
Refactor read handle retrieval to include estimated size in examples …
Choochmeque Apr 21, 2026
d68b4fc
Update FDB benchmarks and tests to use new request building method
Choochmeque Apr 22, 2026
7e8179a
Update build scripts and dependencies for fdb and fdb-sys crates
Choochmeque Apr 22, 2026
e125c92
Fix path for eccodes-sys in Cargo.toml to correct directory structure
Choochmeque Apr 22, 2026
d15b078
Add bindman-utils dependency and update GRIB handling in fdb-hammer
Choochmeque Apr 22, 2026
96cfe4b
Update README files to clarify RPATH setup for binaries on macOS and …
Choochmeque Apr 22, 2026
22554ca
Update eccodes-sys dependency to include additional features for thre…
Choochmeque Apr 23, 2026
941922d
Remove unused LazyLock for mutex serialization in FDB handle operations
Choochmeque Apr 23, 2026
d94833d
Add UserConfig struct for per-instance FDB configuration options
Choochmeque Apr 27, 2026
cc3a6d3
Add MessageArchiverWrapper to interface with fdb5::MessageArchiver fo…
Choochmeque Apr 27, 2026
9a56a13
Add exception generation for fdb-sys to improve error handling integr…
Choochmeque May 14, 2026
df24d47
Merge remote-tracking branch 'origin/develop' into rust-bindings-v1
Choochmeque Jun 3, 2026
8ee4ec8
Remove unused DataReader and Request structs from the FDB crate
Choochmeque Jun 3, 2026
5e695fe
Update dependencies to use Git repositories for eckit and bindman crates
Choochmeque Jun 10, 2026
f790bea
Update metkit-sys dependency to point to the correct repository branch
Choochmeque Jun 11, 2026
e9e4b41
Merge branch 'develop' into rust-bindings-v1
Choochmeque Jun 11, 2026
aceb4e5
Refactor MessageArchiverWrapper constructor and new_message_archiver …
Choochmeque Jun 11, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 8 additions & 6 deletions rust/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -17,14 +17,16 @@ fdb-sys = { path = "crates/fdb-sys" }
fdb = { path = "crates/fdb" }

# Foundation crates
eckit-sys = { git = "ssh://git@github.com/ecmwf/rust-wrappers-playground.git", default-features = false }
metkit-sys = { git = "ssh://git@github.com/ecmwf/rust-wrappers-playground.git", default-features = false }
eccodes-sys = { git = "ssh://git@github.com/ecmwf/rust-wrappers-playground.git", default-features = false }
eckit-sys = { path = "../../rust-wrappers-playground/crates/eckit-sys", default-features = false }
eckit = { path = "../../rust-wrappers-playground/crates/eckit" }
metkit-sys = { path = "../../rust-wrappers-playground/crates/metkit-sys", default-features = false }
metkit = { path = "../../rust-wrappers-playground/crates/metkit" }
eccodes-sys = { path = "../../rust-wrappers-playground/crates/eccodes-sys", default-features = false, features = ["vendored", "eccodes-threads"] }

# Build tools
bindman = { git = "ssh://git@github.com/ecmwf/bindman.git" }
bindman-build = { git = "ssh://git@github.com/ecmwf/bindman.git" }
bindman-utils = { git = "ssh://git@github.com/ecmwf/bindman.git" }
bindman = { path = "../../bindman/bindman" }
bindman-build = { path = "../../bindman/bindman-build" }
bindman-utils = { path = "../../bindman/bindman-utils" }

# External
thiserror = "2"
Expand Down
136 changes: 66 additions & 70 deletions rust/crates/fdb-sys/build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,6 @@
//!
//! Both modes build the CXX bridge for C++ to Rust bindings.

use std::env;
use std::path::PathBuf;

const FDB_VERSION: &str = "5.19.1";

fn main() {
println!("cargo:rerun-if-changed=build.rs");
println!("cargo:rerun-if-changed=src/lib.rs");
Expand All @@ -26,29 +21,56 @@ fn main() {

bindman_utils::validate_build_mode(cfg!(feature = "system"), cfg!(feature = "vendored"));

generate_exceptions();

if cfg!(feature = "system") {
build_system();
} else {
build_vendored();
}
}

/// Generate `fdb_exceptions.{h,rs}` for fdb-sys's cxx bridge.
///
/// fdb-sys does not introduce its own exception subclasses (the higher-level
/// `fdb` crate maps `cxx::Exception` directly), so the `own` list is empty
/// and we inherit C++ catch blocks from upstream `-sys` crates (eckit-sys,
/// metkit-sys) via [`bindman_build::collect_dep_exception_sources`].
fn generate_exceptions() {
let out_dir = std::path::PathBuf::from(std::env::var("OUT_DIR").expect("OUT_DIR not set"));
let inherited = bindman_build::collect_dep_exception_sources();

bindman_build::generate_exception_bridge(&bindman_build::ExceptionBridgeConfig {
primary_namespace: "fdb",
out_dir: &out_dir,
own: &[],
inherited: &inherited,
});
}

/// Build using system-installed fdb5 via `CMake` `find_package`
#[cfg(feature = "system")]
fn build_system() {
use std::env;
use std::path::PathBuf;

let crate_dir =
PathBuf::from(env::var("CARGO_MANIFEST_DIR").expect("CARGO_MANIFEST_DIR not set"));

// Get dependency paths from -sys crates
let eckit_include = env::var("DEP_ECKIT_INCLUDE")
.expect("DEP_ECKIT_INCLUDE not set - eckit-sys must be a dependency");
let metkit_include = env::var("DEP_METKIT_INCLUDE")
.expect("DEP_METKIT_INCLUDE not set - metkit-sys must be a dependency");
let eccodes_include = env::var("DEP_ECCODES_INCLUDE")
.expect("DEP_ECCODES_INCLUDE not set - eccodes-sys must be a dependency");
let eckit_include = env::var("DEP_ECKIT_SYS_INCLUDE")
.expect("DEP_ECKIT_SYS_INCLUDE not set - eckit-sys must be a dependency");
let eckit_cpp_dir = env::var("DEP_ECKIT_SYS_CPP_DIR")
.expect("DEP_ECKIT_SYS_CPP_DIR not set - eckit-sys must be a dependency");
let metkit_include = env::var("DEP_METKIT_SYS_INCLUDE")
.expect("DEP_METKIT_SYS_INCLUDE not set - metkit-sys must be a dependency");
let metkit_cpp_dir = env::var("DEP_METKIT_SYS_CPP_DIR")
.expect("DEP_METKIT_SYS_CPP_DIR not set - metkit-sys must be a dependency");
let eccodes_include = env::var("DEP_ECCODES_SYS_INCLUDE")
.expect("DEP_ECCODES_SYS_INCLUDE not set - eccodes-sys must be a dependency");
let out_dir = PathBuf::from(env::var("OUT_DIR").expect("OUT_DIR not set"));

let (root, fdb_include, lib_dir) =
bindman_utils::cmake_find_package("fdb5", FDB_VERSION, Some("FDB_DIR"));
let (root, fdb_include, lib_dir) = bindman_utils::cmake_find_package("fdb5", "5.10.0");

println!("cargo:rustc-link-search=native={}", lib_dir.display());
println!("cargo:rustc-link-lib=dylib=fdb5");
Expand All @@ -58,19 +80,22 @@ fn build_system() {
.file(crate_dir.join("cpp/fdb_bridge.cpp"))
.include(&fdb_include)
.include(&eckit_include)
.include(&eckit_cpp_dir) // for eckit_bridge.h
.include(&metkit_include)
.include(&metkit_cpp_dir) // for metkit_bridge.h
.include(&eccodes_include)
.include(crate_dir.join("cpp"))
.include(&out_dir) // for fdb_exceptions.h (generated)
.flag_if_supported("-std=c++17")
.compile("fdb_sys_bridge");

// Link to eckit and metkit (bridge uses their symbols)
let eckit_root = env::var("DEP_ECKIT_ROOT")
.expect("DEP_ECKIT_ROOT not set - eckit-sys must be a dependency");
let metkit_root = env::var("DEP_METKIT_ROOT")
.expect("DEP_METKIT_ROOT not set - metkit-sys must be a dependency");
let eccodes_root = env::var("DEP_ECCODES_ROOT")
.expect("DEP_ECCODES_ROOT not set - eccodes-sys must be a dependency");
let eckit_root = env::var("DEP_ECKIT_SYS_ROOT")
.expect("DEP_ECKIT_SYS_ROOT not set - eckit-sys must be a dependency");
let metkit_root = env::var("DEP_METKIT_SYS_ROOT")
.expect("DEP_METKIT_SYS_ROOT not set - metkit-sys must be a dependency");
let eccodes_root = env::var("DEP_ECCODES_SYS_ROOT")
.expect("DEP_ECCODES_SYS_ROOT not set - eccodes-sys must be a dependency");

println!("cargo:rustc-link-search=native={eckit_root}/lib");
println!("cargo:rustc-link-lib=dylib=eckit");
Expand Down Expand Up @@ -105,13 +130,16 @@ fn build_system() {
#[cfg(feature = "vendored")]
#[allow(clippy::too_many_lines)]
fn build_vendored() {
use std::env;
use std::fs;
use std::path::PathBuf;
use std::process::Command;

const ECBUILD_REPO: &str = "https://github.com/ecmwf/ecbuild.git";
const ECBUILD_TAG: &str = "3.13.1";

const FDB_REPO: &str = "https://github.com/ecmwf/fdb.git";
const FDB_TAG: &str = "5.19.1";

let out_dir = PathBuf::from(env::var("OUT_DIR").expect("OUT_DIR not set"));
let src_dir = out_dir.join("src");
Expand All @@ -122,16 +150,16 @@ fn build_vendored() {
fs::create_dir_all(&build_dir).expect("Failed to create build directory");

// Get dependency paths from -sys crates
let eckit_root = env::var("DEP_ECKIT_ROOT")
.expect("DEP_ECKIT_ROOT not set - eckit-sys must be a dependency");
let metkit_root = env::var("DEP_METKIT_ROOT")
.expect("DEP_METKIT_ROOT not set - metkit-sys must be a dependency");
let eccodes_root = env::var("DEP_ECCODES_ROOT")
.expect("DEP_ECCODES_ROOT not set - eccodes-sys must be a dependency");
let eckit_root = env::var("DEP_ECKIT_SYS_ROOT")
.expect("DEP_ECKIT_SYS_ROOT not set - eckit-sys must be a dependency");
let metkit_root = env::var("DEP_METKIT_SYS_ROOT")
.expect("DEP_METKIT_SYS_ROOT not set - metkit-sys must be a dependency");
let eccodes_root = env::var("DEP_ECCODES_SYS_ROOT")
.expect("DEP_ECCODES_SYS_ROOT not set - eccodes-sys must be a dependency");

// Clone sources
let ecbuild_src = bindman_utils::git_clone(ECBUILD_REPO, ECBUILD_TAG, &src_dir.join("ecbuild"));
let fdb_src = bindman_utils::git_clone(FDB_REPO, FDB_VERSION, &src_dir.join("fdb"));
let fdb_src = bindman_utils::git_clone(FDB_REPO, FDB_TAG, &src_dir.join("fdb"));

// Patch CMakeLists.txt to remove tests subdirectory (buggy when ENABLE_TESTS=OFF)
let cmakelists = fdb_src.join("CMakeLists.txt");
Expand Down Expand Up @@ -211,15 +239,9 @@ fn build_vendored() {
bindman_utils::on_off(cfg!(feature = "sandbox"))
));

// Portable install names for dynamic libraries
// Use @rpath install names — the leaf binary sets rpaths via bindman_utils::emit_rpaths()
#[cfg(target_os = "macos")]
cmd.arg("-DCMAKE_INSTALL_NAME_DIR=@executable_path/fdb_libs");

#[cfg(target_os = "linux")]
{
cmd.arg("-DCMAKE_INSTALL_RPATH=$ORIGIN:$ORIGIN/../fdb_libs");
cmd.arg("-DCMAKE_BUILD_WITH_INSTALL_RPATH=ON");
}
cmd.arg("-DCMAKE_INSTALL_NAME_DIR=@rpath");

bindman_utils::run_command(&mut cmd, "ecbuild configure fdb");

Expand All @@ -244,35 +266,36 @@ fn build_vendored() {
// FDB source directory contains private headers that may be needed
let fdb_src_include = fdb_src.join("src");

// IMPORTANT: Copy resources FIRST, then link against the copied location.
let libs_dest = copy_resources_to_output(&install_dir, &eckit_root, &metkit_root);
let eckit_cpp_dir = env::var("DEP_ECKIT_SYS_CPP_DIR")
.expect("DEP_ECKIT_SYS_CPP_DIR not set - eckit-sys must be a dependency");
let metkit_cpp_dir = env::var("DEP_METKIT_SYS_CPP_DIR")
.expect("DEP_METKIT_SYS_CPP_DIR not set - metkit-sys must be a dependency");

// Build the CXX bridge
cxx_build::bridge("src/lib.rs")
.file(crate_dir.join("cpp/fdb_bridge.cpp"))
.include(&include_dir)
.include(&fdb_src_include)
.include(format!("{eckit_root}/include"))
.include(&eckit_cpp_dir) // for eckit_bridge.h
.include(format!("{metkit_root}/include"))
.include(&metkit_cpp_dir) // for metkit_bridge.h
.include(format!("{eccodes_root}/include"))
.include(crate_dir.join("cpp"))
.include(&out_dir) // for fdb_exceptions.h (generated)
.flag_if_supported("-std=c++17")
.compile("fdb_sys_bridge");

// Link against the copied location in target directory
println!("cargo:rustc-link-search=native={}", libs_dest.display());
// Link against the install directory
let fdb_lib_dir = bindman_utils::resolve_lib_dir(&install_dir);
println!("cargo:rustc-link-search=native={}", fdb_lib_dir.display());
println!("cargo:rustc-link-lib=dylib=fdb5");
println!("cargo:rustc-link-lib=dylib=eckit");
println!("cargo:rustc-link-lib=dylib=metkit");
bindman_utils::link_cpp_stdlib();

// Export for downstream crates (still point to install dir for headers)
println!("cargo:root={}", install_dir.display());
println!("cargo:include={}", include_dir.display());

// Emit RPATH flags for runtime library discovery
bindman_utils::emit_rpath_flags(&["fdb_libs"]);

// Check C++ API
bindman_build::check_cpp_api(&fdb_src_include, &crate_dir.join("src/lib.rs"));
}
Expand All @@ -281,30 +304,3 @@ fn build_vendored() {
fn build_vendored() {
unreachable!("build_vendored called without vendored feature");
}

/// Copy libraries to target directory for portable binaries.
/// Returns the path to the libs directory where libraries were copied.
#[cfg(feature = "vendored")]
fn copy_resources_to_output(
fdb_install_dir: &std::path::Path,
eckit_root: &str,
metkit_root: &str,
) -> PathBuf {
use std::path::Path;

let target_dir = bindman_utils::target_profile_dir();
let libs_dest = target_dir.join("fdb_libs");

let fdb_lib_dir = bindman_utils::resolve_lib_dir(fdb_install_dir);
let eckit_lib_dir = Path::new(eckit_root).join("lib");
let metkit_lib_dir = Path::new(metkit_root).join("lib");

bindman_utils::copy_shared_libs(&fdb_lib_dir, &libs_dest, "fdb5");
bindman_utils::copy_shared_libs(&eckit_lib_dir, &libs_dest, "eckit");
bindman_utils::copy_shared_libs(&metkit_lib_dir, &libs_dest, "metkit");

// Export resource directory name for runtime discovery
println!("cargo:rustc-env=FDB_LIBS_DIR=fdb_libs");

libs_dest
}
Loading
Loading