What is the expected timeline for MPI 5.0 / OpenMPI 6 ?

[amckinstry]

Background information

Hi. I'm doing MPI stack planning for Debian. We currently ship 5.0.10 in the development, and i'm wondering:

• When might the MPI ABI be ready (MPI 5.0 chp 20)
• Will it ship with v5 or v6
• When might v6 ship, and what major features are planned ?

I appreciate any answers would be approximate at best (bug dependent). But estimates and pointers to trackers would be appreciated.

Thanks
Alastair

[jsquyres]

The MPI ABI is coming in Open MPI v6.0.0.

Keep in mind that there are some significant limitations with the MPI ABI. For example, even if you compile an application with Open MPI v6.0.0 and then re-run-time link it to MPICH's library (e.g., via LD_LIBRARY_PATH or whatever), you still have to use MPICH's mpiexec, not Open MPI's. And there is limited compatibility between the CLI options in our two mpiexec's. All the run-time tunable parameters are different, too (e.g., MPICH has no MCA parameters -- they have different mechanisms). Also, there is no Fortran ABI for MPI.

Finally, you have to compile your application with mpicc-abi, not mpicc (which then causes it to link against a different library: libmpi_abi). Open MPI will still ship with our existing libmpi -- i.e., we'll still ship both the Open MPI ABI and the MPI Standard ABI. Which wrapper compiler you use essentially decides which ABI you link against. The API functionality is the same between the two.

Essentially: the library will cross-link at run-time, but run-time stuff is all still not portable between the two.

Does that make sense?

[rhc54]

Just for my own curiosity - isn't there at least a slight performance hit by going through the ABI? From what I've been seeing, it basically is a translation wrapper on top of the "real" OMPI library.

[jsquyres]

The Open MPI ABI and the MPI Forum ABI layers are both layers on top of a common back-end.

The main semantic difference between the two is that the MPI Forum ABI MPI handles are integers, and the Open MPI layer handles are the actual memory pointers. So the MPI Forum ABI layer has to do a lookup to resolve to the actual memory pointer. Yes, that's definitely a few more instructions / at least one extra de-reference. It's nonzero, but it's likely not too expensive (MPICH has been doing this for years, as has Open MPI's Fortran bindings -- because MPI defines Fortran handles to be integers).

That being said: we're not inspired to change Open MPI's C bindings model -- we like our handles being the actual back-end pointers. 😄

[rhc54]

Oh I'm not suggesting changing the handles! Just noting that users of this ABI stuff have to accept a performance penalty (of whatever size it turns out to be) for doing so.

[amckinstry]

Thanks.

Just to be clear, if I build application A against libmpi_abi from MPICH, and then relink using LD_LIBRARY_PATH to use OpenMPI libmpi_abi, do I use the MPICH mpiexec or OpenMPI mpiexec ? my understanding had been the OpenMPI version.

For Fortran, I am currently planning on using https://github.com/eschnett/mpif
As I understand it, the Fortran bindings go through libmpi_abi.

Is there an expected timeline for v6 ?

[rhc54]

do I use the MPICH mpiexec or OpenMPI mpiexec

You always have to use the one from the underlying library as it is expecting the information provided by the matching runtime.

[jsquyres]

You always have to use the one from the underlying library as it is expecting the information provided by the matching runtime.

Yes -- what @rhc54 said. Use the mpiexec (and corresponding runtime) of the MPI ABI library that you're run-time linking with.

[jsquyres]

As for timeframe for v6.0.0 -- that's a great question. 😄

We wanted to have shipped it months ago. That being said, we're closer than ever -- some big new features just went in recently, and there's at least one more on the cusp of going in (ABI -- #13280, if you want to follow it).

[amckinstry]

So it sounds like we should be tracking v6 and hope to get it in by February 2027.
Is there a PR/bug tracking v6.0 status?
Is there a list of new features? importantly other than mpi_abi, is there anything that might break compatability with v5.0 (in terms of testing workload and rebuilds).

[Growl1234]

I'm also looking forward to v6.0.0 👀

Is there a list of new features? importantly other than mpi_abi, is there anything that might break compatability with v5.0 (in terms of testing workload and rebuilds).

I guess, from https://github.com/open-mpi/ompi/wiki/6.0.x-Feature-List we can get some more information

[jsquyres]

So it sounds like we should be tracking v6 and hope to get it in by February 2027.

Is Feb 2027 a significant deadline for you? I ask because knowing the deadlines of our partners and developers is helpful in pushing ourselves to meet them.

Is there a PR/bug tracking v6.0 status? Is there a list of new features? importantly other than mpi_abi,

I don't know if we have a comprehensive v6.0.x epic, but we do have one for MPI-5.0 conformance tracking: #13987

I guess, from https://github.com/open-mpi/ompi/wiki/6.0.x-Feature-List we can get some more information

Yes, that wiki is a decent representation.

is there anything that might break compatability with v5.0 (in terms of testing workload and rebuilds).

Open MPI will ship with 2 ABI's starting in 6.0.x:

• The Open MPI (C) ABI: same one we've been shipping for years.
  • This is the libmpi library that Open MPI has been building for years.
  • We don't anticipate having any major breakages compared to the v5.0.x series.
• The MPI Forum (C) ABI: new, and defined in the MPI-5.0 standard (with some refinements coming in MPI-5.1).
  • This is the new libmpi_abi library; a name that is mandated by the MPI Forum ABI.
  • Remember: the guarantees about ABI are mostly about C, not Fortran.

Open MPI 6.0.0 builds both ABIs (libraries) by default.

To be 100% clear: these 2 ABIs are separate and incompatible with each other.
To be 1000% clear: if you compile and link against libmpi, you cannot switch that executable to use libmpi_abi at run time (and vice versa). I.e.: you build your executable for one ABI and that's the ABI it will always use. You can, obviously, re-compile / re-link your executable against the other ABI / library -- these two libraries are source compatible, just not run-time link compatible.

Does that help?

[Growl1234]

I'd wonder what's still missing for v6.0.0 (and new v5.0.11 or perhaps v5.1.0?) release, as it seems that wiki page has not been updated for many months.

[jsquyres]

Don't let the wiki page age fool you; unfortunately, some of these things have been under development for quite a while. This summer, a bunch of development got sped up, which is quite encouraging.

[jsquyres]

@Growl1234 @amckinstry Here's our OMPI docs page for the MPI Standard ABI: https://ompi--13280.org.readthedocs.build/en/13280/building-apps/mpi-forum-abi.html

This is a PR docs build, and will be part of https://docs.open-mpi.org/en/main/ when the PR lands in main (and then ultimately make its way to https://docs.open-mpi.org/en/v6.0.x-pre-release/).

That page gives a fairly realistic explanation of all the ins-and-outs of the Open MPI ABI and MPI Standard ABI. I'd like to put as much detail on it as possible because I'm anticipating that there will be many questions and some confusion about what is and what is not supported by the MPI Standard ABI. Comments welcome.