Skip to content

5.0.x FeatureList

Jeff Squyres edited this page Oct 20, 2018 · 63 revisions

Aiming for summer 2019.

Features deferred to v5.0.x from 4.0.x

  • Remove some or all of the deleted MPI-1 and MPI-2 functionality

    • As of Oct 2018, we all seem to agree that removing the C++ bindings is not a problem.
    • ...but we're not sure of whether we can remove all the MPI-1 and MPI-2 deleted functionality (e.g., MPI_ATTR_DELETE, MPI_UB/MPI_LB, ...etc.). I.e., have all apps adapted to use the new APIs? (I know, I know... sigh!).
    • Probably need to re-evaluate the state of the MPI community and applications to see if it's practically feasible to remove these in 2019 (if 5.0.0 is aiming for summer of 2019).
    • History:
      • All these functions/etc. were marked as "deprecated" (possibly in 2.0.x series? definitely by the 3.0.x series).
      • In the v4.0.x series, the C++ bindings are not built by default, and mpi.h/mpif.h/mpi+mpi_f08 modules do not have prototypes/declarations for all the MPI-1 deleted functions and globals (although all the symbols are still present in libmpi for ABI compatibility reasons, at the request of our packagers). v4.0.x does allow using --enable-mpi1-compat to restore the declarations in mpi.h (and friends).
        • Remember that distros and auto-packagers (e.g., Spack, EasyBuild, etc.) will almost certainly --enable-mpi1-compatibility, so some/many users may not even feel the pain of not enabling the functionality by default yet).
      • The idea was that the deprecated/don't-build-by-default warnings in 2.0.x/3.0.x/4.0.x would be enough to allow us to actually delete this stuff in v5.0.x. Specifically: the intent was to remove --enable-mpi-cxx, --enable-mpi1-compat, the C++ bindings, and all the deleted MPI-1 and MPI-2 functionality in v5.0.x.
      • ...but as stated above, as of Oct 2018, the feeling was that removing the MPI-1/MPI-2 deleted functionality in the v5.0.x timeframe (i.e., summer 2019) is going to be too soon. So it's likely that we'll defer actually ditching the deleted MPI-1/MPI-2 functionality until the v6.0.x timeframe (i.e., summer 2020).
  • Delete the openib BTL

    • It's effectively unmaintained
    • All networks supported by the openib BTL (IB, RoCE, iWARP) now supported by Libfabric and/or UCX.
    • There was talk of doing this for v4.0.0
    • Hence, it seems that the future of RoCE and iWARP is either or both of Libfabric and UCX.
      • ...but neither of those will be 100% ready for Open MPI v4.0.0.
      • It didn't seem to make sense to make iWARP users move from openib to iwarp in v4.0.0 (and potentially something similar for non-Mellanox RoCE users), and then move them again to something else in v5.0.0 ("How to annoy your users, 101").
      • The lowest cost solution for v4.0.0 was to disable IB support by default in openib (i.e., only iWARP and RoCE will use it by default), and punt the ultimate decision about potentially deleting the openib BTL to v5.0.0. Note that v4.0.0 also has a "back-door" MCA parameter to enable IB devices, for the "just in case" scenarios (where users, for whatever reason, who don't want to upgrade to UCX).
    • With all that, need to investigate and see what the Right course of action is for v5.0.0 (i.e., re-evaluate where Libfabric and/or UCX are w.r.t. RoCE support for non-Mellanox devices and iWARP support), and how to plumb that support into Open MPI / expose it to the user.
  • (UTK) Better multithreading. - George

    • In OB1 PML, normal OMP parallel Sections. Improved for injection and extraction rates.
    • Implications for other PMLs. Very OB1 specific Maybe a little bit in progress.
  • (UTK) ULFM support via new MPIX functions. Most is in MPIX, but some in PML.

    • Depends on PMIx v3.x
  • Want Nathan's fix for Vader and other BTL to allow us to have SOMETHING for OSC_RDMA for one-sided + MT runs.

    • something similar coming into BTL-TCP
    • If osc/rdma supports all possible scenarios (e.g., all BTLs support the RDMA methods osc/rdma needs), this should allow us to remove osc/pt2pt (i.e., 100% migrated to osc/rdma). Would be good if there was an osc/pt2pt alias in case anyone is scripting their mpirun's to select --mca osc pt2pt.
  • Change defaults for embedding libevent / hwloc (see this issue) - HELP NEEDED see PR 5395

Other new features

Clone this wiki locally