Skip to content

Handling NetCDF enums as julia enums - #316

Merged
Alexander-Barth merged 33 commits into
mainfrom
ab-julia-enums
Jul 24, 2026
Merged

Handling NetCDF enums as julia enums#316
Alexander-Barth merged 33 commits into
mainfrom
ab-julia-enums

Conversation

@Alexander-Barth

@Alexander-Barth Alexander-Barth commented Jul 18, 2026

Copy link
Copy Markdown
Member

This is an alternative way to implement netCDF/HDF5 enum types in NCDatasets as julia enum.

Reference to PR using CategoricalArrays:

What is works so far:

  • read/write Enum variables
  • read/write Enum attributes
  • Enum with compound variables
  • Enum with vlen-arrays
  • Enum in ncgen
  • [ ] Enum types displayed in show (doing it later with changes in CommonDataModel)

CC:
@lupemba @tiemvanderdeure @evetion

@Alexander-Barth

Alexander-Barth commented Jul 23, 2026

Copy link
Copy Markdown
Member Author

Using eval to generate the types (enums or structs) has the drawback that it lead to world age issues when this new datatype is called in a previously defined function and this function for example want to convert the enum into a string:

 ERROR: LoadError: MethodError: no method matching namemap(::Type{NCDatasets.ReconstructedTypes_2437958937.cloud_class_t})         

The applicable method may be too new: running in world age 39256, while current world is 39277.

The solution is to use invokelatest which is ok for NCDatasets to use internally, but not pretty when the users have to call it on their functions.

A drop-in alternative is to define a custom struct for enums (similarity for compound types):

struct NCEnum{T,typename,names,values}
    data::T # T is the base type of the enum
end

In my benchmark, it is as fast and memory efficient as regular enums. They have also the same memory layout for the C code and can be used as a drop-in replacement. The user will still be able to use its own julia enum if it registered before hand with the function:

NCDatasets.typemap!(ds,"cloud_class_t" => Clouds.cloud_class_t); # or similar API

As we own the type NCEnum is also easier to provide a conversion function to CatagoricalArrays as a weak-extension.

@Alexander-Barth
Alexander-Barth merged commit 0530bca into main Jul 24, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant