
                           Score-P OPEN ISSUES v9.4
                           ========================
                                                           Effective: Dec 2025

This file lists known limitations and unimplemented features of
various Score-P components.

--------------------------------------------------------------------------------

* Platform support

  - Score-P is intended to run on ppc64, x86_64, aarch64, or
    riscv architectures.

  - Score-P has been tested on the following platforms:

    + HPE/Cray EX systems with PrgEnvs cray, intel, nvidia, gnu, amd,
      aocc, cray-amd. PrgEnv-cray with Cray PE prior 21.05 fails to
      compiler-instrument functions.

    + various Linux (Intel, AMD, ARM) clusters with GNU, Intel,
      NVIDIA, AMD, and Fujitsu compilers. Note that some compilers
      are based on Clang. Note that Fujitsu cross-compilers haven't
      been tested.
    The provided configure options (see INSTALL) may provide a good
    basis for building and testing the toolset on other systems.

  - The following platforms have not been tested recently:
    + Intel Xeon Phi (KNL)
    + IBM Blue Gene and IBM Power systems
    + Cray XC, XT, XE, XK
    However the supplied build system might still work on these
    systems.

  - Each toolset installation can only support one MPI implementation
    (because MPI is only source-code but not binary compatible). If
    your systems support more than one MPI implementation (e.g. Linux
    clusters often do), separate builds for each MPI implementation
    have to be installed. The same applies for SHMEM.

  - The same is true if your system features more than one compiler
    supporting automatic function instrumentation. Additionally if
    the GNU compiler plug-in based instrumentation is used, then
    different major version of the GNU compiler suite are also
    incompatible. I.e., for each major version a separate Score-P
    installation must be provided.

  - To build Score-P, a C and C++ compiler as well as C and, optionally,
    Fortran compiler wrappers (e.g., MPI or SHMEM) are used. Compilers
    and compiler wrappers must be compatible.

     - Note (dated): PGI changed its default C++ compiler from 'pgCC'
       (which was entirely removed in 16.1) to 'pgc++'. For building
       Score-P with '--with-nocross-compiler-suite=pgi', 'CXX=pgc++'
       is used.  If your MPI and SHMEM compiler wrappers still use the
       old 'pgCC', please add 'CXX=pgCC' to your configure line to
       force Score-P to use 'pgCC' as well. This will prevent link
       failures.

  - (dated) Shared libraries on Cray systems: building Score-P's
    shared libraries works as normal through the '--enable-shared'
    configure option. But note that if the default Cray link mode is
    static (which is the case for CCE prior to 9.0.0) or if
    CRAYPE_LINK_TYPE=static is provided via the environment, you need
    to pass the '-dynamic' flag (if available) to the Cray compiler
    wrapper when building your application to link to Score-P's shared
    libraries.

  - To build with Fujitsu compilers, use the --with-custom-compilers
    configure option. For shared builds (the default), disable the Fortran
    compilers to prevent linker errors. Disable Fortran by providing
    'MPIFC=defunct' and 'MPIF77=defunct' in the platform-mpi-user-provided
    file. Do the same for FC and F77 in the platform-backend-user-provided
    file.

  - Experimental platforms:

    The following platforms are considered experimental and don't get
    much attention. They may break without notification. Help to
    maintain these is always welcomed.

    + MinGW
      - Address-to-line lookup might not work as link.h and
        dl_iterate_phdr might not be available.
    + macOS
      - Building shared/dynamic libraries may not work.
      - Address-to-line lookup (e.g., for clang-based compiler
        instrumentation) does not work as link.h and dl_iterate_phdr
        is not available.

--------------------------------------------------------------------------------

* Automatic compiler instrumentation via "scorep" based on (often
  undocumented) compiler switches

  - GNU    : tested with GCC 4.9 and higher
  - NVHPC  : tested with version 20.7 and higher
  - PGI    : tested with version 10.1 and higher
             Note that PGI 13.8 is currently not supported as it is not
             recognized as an PGI compiler anymore. PGI 16 supports
             only 64-bit platforms.
  - IBM    : only works for xlc/xlC version 7.0 and xlf version 9.1 and
             higher and corresponding bgxl compilers on Blue Gene systems
  - Intel  : only works with Intel icc/icpc/ifort version 11 and higher
             compilers or the Clang-based ixc/icpx compilers from
             Intel oneAPI toolkit. Note that the Fortran ifx compiler
             does not support compiler instrumentation.
  - Cray   : tested with version 8.1.8 and later, uses the GNU interface.
  - Fujitsu: tested with the language environment 1.2.41 in traditional
             and Clang mode on Fugaku, uses the GNU interface.
  - Clang  : tested with Clang 4.0 and higher. Note that several
             compiler vendors now use Clang for their C and C++
             compilers while the Fortran compiler stays
             vendor-specific.

  - The Intel compiler provides the function name and the source code
    location in one string separated by a colon.  Thus, if the path
    name of the source code file contains colons, Score-P will split
    the source file name and the function name improperly.
    Additionally, the provided function name makes it impossible to
    distinguish overloaded functions in C++. Thus, functions which
    differ only in the argument list will be mapped to the same
    function definition by Score-P.

  - When using any of the intrinsic headers (e.g., xmmintrin.h and friends)
    from GCC version up to 6.3, it is known that the result of the compiler
    instrumentation will fail at link time because of undefined reference.
    A work around is to pass

      -finstrument-functions-exclude-file-list=intrin.h

    as flag to GCC on the command line. This issue does not apply if the
    GNU compiler plug-in based instrumentation is used.

  - Measurement filtering can only be applied to functions
    instrumented by the IBM, GNU, Intel, NVHPC, Clang, or PGI compilers as
    well as functions instrumented by Kokkos and user functions.
    Filtering of MPI, SHMEM, and OpenMP runtime functions is always
    ineffective.

  - The GCC plug-in is supported from GCC 4.9 onwards. Though due to
    constant changes in the plug-in API of the GNU compiler
    infrastructure, it is unlikely that the Score-P instrumentation
    plug-in builds with versions newer than GCC snapshot 11-20210307.

  - The pgCC compiler versions 13.9 and higher preinclude omp.h for
    OpenMP codes. This results in multiple defined symbols if the
    source file is preprocessed before compilation. Since version 14.1
    an option is available to avoid preinclusion, which we can use for
    preprocessed source files.  For the pgCC versions 13.9 until 14.0,
    preprocessing is not possible for C++ codes.

  - Some compiler interfaces provide function pointers that get
    translated to region meta data via libbfd from the binutils
    package. If Score-P uses a system libbfd and region meta data is
    implausible, consider reinstalling Score-P while providing a
    recent libbfd via the configure option `--with-libbfd=download`.

  - The LLVM plug-in is supported from LLVM 13.0 and onwards. Due to
    regular changes in the plug-in API, bleeding edge releases of
    LLVM might not be able to compile the plug-in. In our testing,
    the plug-in did compile with LLVM 13.0 to LLVM 21.1.

  - When no support for LLVM demangle is detected during configure,
    compile-time filtering will only work for mangled names.

  - Compiling HIP code with the LLVM plug-in might cause error messages
    of the linker `lld` because plug-in parameters are passed to the
    linker. This issue is fixed in LLVM 17.0 and newer, as well as in
    ROCm 6.1 and newer.

  - Bootstrapped LLVM builds that use LLD as their linker might fail to enable
    the LLVM plug-in because static libraries are built with LLD, which is not
    communicated by `llvm-config`. To use the LLVM plug-in, pass this option
    to `configure`:

      LDFLAGS_FOR_BUILD_LLVM_PLUGIN="-fuse-ld=lld"

--------------------------------------------------------------------------------

* MPI support

  - Most functions added with MPI-3.0 and MPI-3.1 only have Enter
    and Exit event records, i.e., Score-P can measure the time spent
    in these routines, but no further analysis is possible.

  - C++ bindings for MPI are not supported. These were deprecated as
    of MPI-2.2 and removed with MPI-3.0. When using C++ bindings for
    MPI, Score-P will most likely indicate the failed library wrapping
    with the following warning: "If you are using MPICH1, please
    ignore this warning. If not, it seems that your interprocess
    communication library (e.g., MPI) hasn't been initialized. Score-P
    cannot generate output."
    If your MPI implements the C++ bindings in a separate library on
    top of the C bindings, the following workaround might work for
    you:
    1) Instrument your application with 'scorep --keep-files -v' It
       will make the instrumenter not remove the intermediate files
       and output the commands it executes.
    2) Copy the link command (the last command from the scorep
       output).
    3) Insert a link flag (e.g. -lmpicxx) for your C++ bindings
       library right before '-lscorep_adapter_mpi_events' and execute
       the modified link command.

  - When using derived data types in non-blocking communications, and
    no support for MPI_Type_dup() was detected, please ensure that the
    MPI_Datatype handle is still valid at the time the request
    finishes.

  - Currently, Score-P cannot handle MPI_Finalize() calls that occur
    after the end of main(), e.g., via a destructor of a static C++
    object. Please call MPI_Finalize() before the end of main(). The
    issue will be resolved in a future version of Score-P.

  - The IBM Platform MPI (not mpixlc!) compiler wrapper (the formerly
    HP-MPI) does not append its libraries at the very end of the
    original link command. Thus, instrumenting applications with
    Score-P fails at link time due to unresolved symbols in the
    Score-P libraries.

  - Multi-threaded MPI communication is limited/experimental in Score-P.
    It is not that it is not possible to measure communication
    from other threads. But it is nearly impossible to do reasonable analysis
    when only ranks but not the real communication partners (threads)
    are known. There are efforts in the MPI forum to address this
    issue (endpoints).
    The current version of Score-P is designed to stop the instrumented
    application from crashing under multi-threaded MPI communication
    but there are some limitations.
    When thread-local storage is not detected during the configure
    step `MPI_THREAD_MULTIPLE` is not supported.
    All recorded MPI communication events will still be reported on
    the master thread.
    MPI implementations optimize small messages and create non-unique
    requests, e.g., empty requests in OpenMPI, and can not be
    distinguished by Score-P. A workaround is implemented. However,
    the ordering and, therefore, association to the MPI calls is lost.
    This effect was also present in previous versions of Score-P but went
    unnoticed.

  - Open MPI 4.0.0 is known broken to build with Score-P, because it
    accidentally removed deprecated-only MPI functions, without
    removing the prototypes in `mpi.h`.

  - Score-P leaks memory in the MPI requests management as the Score-P
    memory management does not allow freeing individual allocations.
    We try to reduce this leakage. Gradually increasing the number of
    requests will increase this effect. However, in practice,
    it does not seem to have any impact on the measurement.

  - The MPI sessions model is not supported and `MPI_Session_init`
    aborts the measurement.

  - Fortran 2008 bindings:

    - When instrumented with a Score-P that lacks F08 support, applications that
      initialize MPI with the `mpi` module, but call functions from `mpi_f08`,
      might crash or produce inconsistent measurements.
      Verify with `scorep-info config-summary | grep "Fortran 2008 bindings"`
      that Score-P is built with F08 support.

    - The deprecated functions `MPI_KEYVAL_CREATE`, `MPI_KEYVAL_FREE`,
      `MPI_ATRR_PUT`, `MPI_ATTR_GET`, `MPI_ATTR_DELETE` are not supported. These
      functions are deprecated since MPI 2.0 and the MPI standard does not
      define an F08 binding for them. Please update your program to use the
      appropriate MPI 2.0 replacements.

    - Due to the delayed call to `MPI_FINALIZE`, user-defined attribute
      deletion callbacks that are invoked during MPI finalization might see
      invalid values for `extrastate`. Either invoke the deletion callback
      explicitly before `MPI_FINALIZE`, or declare the actual argument for
      `extrastate` as `SAVE`.

    - When sampling is active, the Score-P internal function
      `scorep_in_measurement_increment_fromF08` might be recorded in the
      profile.

    - `MPI_ALLTOALLW` allocates temporary arrays for type conversions from F08
      to C. On large communicators, this might consume a large amount of memory.

    - Configure will disable MPIF08 support with Intel MPI (several
      versions affected) if it detects an error in their F08 bindings. A
      workaround, thus enabling support in Score-P, is described here:

        https://community.intel.com/t5/Intel-MPI-Library/MPI-f08-with-polymorphic-argument-CLASS/m-p/1590421/highlight/true#M11660

--------------------------------------------------------------------------------

* OpenMP support

  Applies to OPARI2 and OMPT instrumentation:

  - Changes introduced with the OpenMP 6.0 specification are not supported.

  - Combining OPARI2 or OMPT with GOTCHA wrapped libraries is not supported.
    This affects namely memory tracking, POSIX I/O, and OpenCL. Please use
    Pthread instrumentation via '--thread=pthread' instead.

  Applies to OPARI2 instrumentation only:

  - Device directives, introduced with OpenMP 4 and later, are not supported.

  - Function instrumentation using the Intel compiler version 11.1 for
    codes using OpenMP tasking is erroneous.

  - When compiling with the PGI compiler version 10.1, local variables
    that are defined after a OpenMP for construct share the same memory
    address. This breaks the OPARI2 instrumentation for task tracking
    and may lead to segmentation faults in the measurement system.
    Our recommendation is to use a newer compiler version, According
    to our tests, late compiler versions have fixed this issues.
    We tested with PGI compiler version 11.7.

  - Currently, the instrumenter allows to switch off OPARI2
    instrumentation for OpenMP programs with the --noopenmp
    option. However parallel regions still need to be instrumented to
    ensure thread-safe execution of the measurement system. Currently
    combined constructs, such as parallel for/do, are still
    instrumented fully, i.e. the for/do appears in the measurements.

  - Due to a bug in the Cray compilers OpenMP instrumentation is
    broken if an OpenMP parallel pragma is used in combination with
    an if clause.

  - OPARI2-instrumented Fortran OpenMP codes that use compiler options
    to change the default name-mangling (XL compilers: -qextname, GNU:
    -fno-underscoring' and '-fsecond-underscore') might crash if
    OpenMP 3.0 ancestry API functions are not available (check with
    'scorep-info config-summary | grep ancestry'). In this case OPARI2
    instruments a thread private variable that, due to non-standard
    name-mangling, does not match the one used in the Score-P
    libraries. On AIX, a workaround is to manually rename it at link
    time (-brename:pomp_tpd_,pomp_tpd).

  - With PGI C++ v13.10 compiler, preprocessing of OpenMP codes using
    OPARI2 is not possible any longer as the compiler itself adds a
    '--preinclude omp.h' option to the call of pgcpp1. This leads to
    'invalid redeclaration' errors. As a workaround, use the
    '--nopreprocess' instrumenter option.

  - The OPARI2 instrumentation and the preprocessing for OPARI2
    instrumentation prepend some headers to source files which
    include stdint.h in C/C++ files. The behavior of this header changes
    whether macros the __STDC_CONSTANT_MACROS or __STDC_LIMIT_MACROS are
    defined. If these macros are defined in a header file or source file,
    the instrumentation will prepend the include directive before the macro
    definition. Thus, macros like UINT32_C are left undefined. As workaround,
    pass macros like __STDC_CONSTANT_MACROS or __STDC_LIMIT_MACROS on the
    command line. See also the Open Issues of OPARI2.

  - OPARI2-instrumented OpenMP code using user-defined reductions
    (UDR), introduced with OpenMP 4, might crash when the UDR includes
    execution of instrumented routines, which Score-P currently
    reports as events on invalid threads or mis-matching enter/exit
    events.

  - Score-P 5.0 introduced program-begin/-end events. In profiling
    mode, these manifest as enter/exit of an additional region which
    acts as the new root node for the program. Artificial regions like
    THREADS and TASKS are now recorded as child nodes of this new
    root node.
    When recording OpenMP tasks, the tasks are recorded as children of
    the TASKS node as well as children in the program's
    calltree. Metrics, time in particular, are double accounted.
    As a consequence, the new root node shows negative time values
    when expanded (the absolute value equals the inclusive TASKS
    value). The values shown for TASKS and the program's calltree are
    correct though. This situation will be improved in a future
    release of Score-P and/or Cube.

  - In C++, OpenMP clauses like private, firstprivate, and lastprivate
    call constructors, copy-constructors, and destructors for list
    elements of class type. Given OPARI2 instrumentation, if these
    functions trigger any event, Score-P will/might fail. For
    constructors and destructors we might see `TPD == 0` errors, for
    destructors it will be `Assertion 'parent != ((void *)0)' failed`.
    Preventing all events from these functions by runtime/compile-time
    filtering is a workaround. Future OpenMP instrumentation based on
    OMPT instead of OPARI2 does not show this problems.

  - Like other compilers, the NVIDIA HPC SDK compilers add outlined
    functions to implement OpenMP directives. These functions get
    potentially instrumented but need to be ignored by Score-P to
    prevent 'TPD==0' errors. Unfortunately, NVIDIA's naming convention
    for these functions isn't documented and changed between versions
    of the SDK. However, since version 21.2 of the SDK, the outlined
    functions appear to follow this naming scheme:
      __nv_*_F[0-9]*L[0-9]*_[0-9]*
    Outlined functions following this scheme are automatically
    filtered by Score-P.

    For SDKs predating 21.2, the outlined functions seem to be of the
    form <FUNCTION>_<SOMETHING>L<LINENO>. This isn't handled
    automatically by Score-P, but could be filtered with this very
    generic EXCLUDE rule:
      *_*L[0-9]*

  - With CMake, OpenMP might get enabled at link time not via a
    compiler flag like `-fopenmp`, but by explicit linking of the
    runtime, e.g., `-lgomp`. In this case, Score-P would not
    automatically add its OpenMP support libraries and the link would
    fail. As a workaround, please provide `--thread=omp` to the
    `scorep-wrapper` via
    `make SCOREP_WRAPPER_INSTRUMENTER_FLAGS="--thread=omp"`.

  - With CMake and NVHPC, the preparations for OPARI2 instrumentation
    might create empty source files due to the `-MD` flag added by
    CMake during compilation. The compilation will abort with the
    message `undefined reference to main`. For NVHPC 23.7 and newer,
    please use OMPT via `--thread=omp:ompt` instead.

  - With Cray compiler wrappers and an accelerator architecture loaded,
    CPE 23.09 or lower are not supported due to Clang-based compilers failing
    to compile the pre-processed output generated by the wrappers. Trying to
    build an OpenMP application with OPARI2 enabled will fail with compilation
    errors.

  Applies to OMPT instrumentation only:

  - When instrumenting an application using `#pragma omp sections` with
    Intel oneAPI compilers, enabling OpenMP via `-fiopenmp` or `-qopenmp`
    may result in an error reporting an `inconsistent profile`. This happens
    because the runtime reports the end of the sections construct incorrectly.
    For C / C++ only, you may switch to `-fopenmp` for instrumentation. For
    Fortran applications, please use OPARI2 via `--thread=omp:opari2`.

  - When instrumenting an application using `#pragma omp sections` with
    NVHPC 23.7, running the instrumented application will abort reporting an
    `inconsistent profile`. This happens because the runtime does not report
    the end of the sections construct. If you want to instrument applications
    using OpenMP sections with NVHPC, please use OPARI2 via `--thread=omp:opari2`
    instead.

  - When measuring an application using `#pragma omp ordered` with NVHPC,
    running the application may result in an error message like
    "Bug: 'task->workshare_regions_current == 0'" or crash with a segmentation
    fault. As this is a runtime issue and cannot be worked around, please use
    OPARI2 via `--thread=omp:opari2` to instrument applications using
    `#pragma omp ordered`.

  - When configuring Score-P with a Cray compiler and an accelerator
    architecture loaded on a node without the respective accelerator,
    configure checks might fail because no GPU was found. Please build
    on a node with the accelerator being present.

  - When instrumenting an application using OpenMP tasks within a teams region,
    using `#pragma omp single nowait` or `#pragma omp masked` might cause Score-P
    to abort when using LLVM based compilers due to missing implicit barriers
    in the runtime. As a workaround, please use `#pragma omp single` instead.

  - NVHPC 24.3 to 24.7 have been observed to dispatch `lock_init` callbacks
    before dispatching `thread_begin` when `lock_init` is the first event
    on this thread. This is non-conforming runtime behavior. Score-P will
    detect this and abort the measurement. As a workaround, please try to
    trigger an additional event on affected threads, e.g. by adding an empty
    parallel region.

  Applies to OpenMP target offloading:

  - When using an LLVM-based compiler in an application using OpenMP
    target directives, the `scorep` compile command may fail with a
    linker error and the following error message:

      Undefined reference to '__cyg_profile_func_enter'
      Undefined reference to '__cyg_profile_func_exit'

    The error is caused by the `-finstrument-functions-after-inlining`
    flag. As a work around, you may try to add the following function
    declarations to each of your compile units:

      #pragma omp begin declare target device_type(nohost)
      void __cyg_profile_func_enter(void *a, void *b)
      __attribute__((no_instrument_function));
      void __cyg_profile_func_exit(void *a, void *b)
      __attribute__((no_instrument_function));
      void __cyg_profile_func_enter(void *a, void *b) { }
      void __cyg_profile_func_exit(void *a, void *b) { }
      #pragma omp end declare target

    On a compiler using LLVM 16.0 or newer, you may try to use these flags:

      scorep --nocompiler [llvm-based compiler] [args] \
      -Xarch_host -finstrument-functions-after-inlining

  - Runtimes have been reported creating helper threads for certain target
    directives. These helper threads may dispatch events in a non-conforming
    way. In addition, non-conforming tasking behavior has been observed.
    In these cases, Score-P can't ignore the device events, but aborts.
    As a workaround, please turn off these helper threads by setting the
    environment variable `LIBOMP_USE_HIDDEN_HELPER_TASK` to `0`. Known
    nonconforming runtimes include LLVM/Clang (12.0.0 to 18.1.7) and
    oneAPI (2024.1 and earlier).

  - For instrumentation of target directives, only runtimes having the OpenMP
    Tools Interface device tracing interface available are supported.  This
    includes recent ROCm and AOMP compilers. For other runtimes, please try to
    use the native accelerator adapters for instrumentation.

  - For best support, it is advised to use the latest compiler versions to
    ensure the best support of the device tracing interface. In our testing,
    AMDs ROCm 6.2.x and AOMP 20.0-0 offer the best support, with the following
    limitations:
      - AOMP 19.0-0 will report incorrect times for data transfers between
        devices.
      - AOMP 18.0-1 may deadlock for short programs when multiple accelerators
        are initialized.
      - ROCm 6.1.3 and earlier and AOMP 18.0-0 and earlier do not support
        multiple devices per rank. If kernels are executed on more than one
        device per process, execution may abort. Otherwise events may be
        associated with the wrong accelerator.
      - AOMP 18.0-0 incorrectly maps identifiers between callbacks and the
        device tracing interface. This leads to data transfers being shown
        incorrectly between the host threads and devices.
      - ROCm 5.7.1 and earlier and AOMP 17.0-3 and earlier do not support
        accessing a device from multiple threads. This may lead to issues
        where events are associated with the incorrect host thread.
      - ROCm 5.6 to 5.7.1 do not dispatch all callbacks for `#pragma omp
        target enter/exit data`. Score-P will abort due to timestamp issues.
      - When utilizing multiple accelerators with ROCm 5.5, execution will
        dead lock at the end of the program execution when Score-P calls
        `stop_trace` for the device tracing interface.
      - ROCm 5.4 and earlier are not supported due to not having a way to
        translate the device time to host time.
    In addition, Cray CCE 17.0.1 and newer support the device tracing
    interface, with the following known limitations. These limitations will be
    improved in future Cray CCE versions:
      - Cray CCE 17.0.1 may not dispatch outstanding buffers at the end of the
        program, and may not dispatch a device finalization event before the
        measurement has already finished. This may lead to missing events.
      - Cray CCE 17.0.1 may dispatch buffers twice for the same event, when
        explicit flushes are done. Score-P tries to ignore these duplicate
        events. This issue is resolved in Cray CCE 19.0.0.
    If the application crashes or hangs because of the mentioned issues,
    please try to disable the device tracing interface by setting
    `SCOREP_OPENMP_TARGET_DEVICE_TRACING_ENABLE=false` and record events with
    the respective native accelerator adapter.

  - Not all runtimes offering support for the OpenMP Tools Interface support
    dispatching events for target directives and/or the device tracing
    interface to report accelerator events. In our testing, LLVM/Clang 17.0,
    oneAPI 2024.1, NVHPC 23.7, ROCm 5.4, and newer versions have at least
    partial support for target directives. If the device tracing interface
    cannot be enabled, Score-P will report a warning. In this case, please try
    to record events using the respective native accelerator adapter, e.g.,
    CUDA.

  - When instrumenting an application with NVHPC 24.11 or 25.1, the OpenMP
    runtime may dispatch a target-begin event before device initialization.
    This was observed with codes, where no data transfer is done, or small
    variables on the stack were copied. Since Score-P requires having
    OpenMP target devices initialized when a target-begin event is dispatched,
    the program will abort. Please disable the instrumentation of target
    directives via `SCOREP_OPENMP_TARGET_ENABLE=no` and record events with
    the CUDA adapter instead.

  - When instrumenting an application with Cray CPE 25.03 (CCE 19.0.0),
    the OpenMP runtime may dispatch nested target-enter-data-begin
    and target-data-end events using the same target_data pointer when
    using `#pragma omp declare mapper`. These nested events are not
    handled correctly by Score-P, causing the measurement to abort
    with an 'Inconsistent profile' error.

--------------------------------------------------------------------------------

* CUDA support

  - Support for CUDA 4.2 and prior versions was deprecated in Score-P 7.0 and
    removed in Score-P 8.0. Since 8.0, Score-P requires CUDA 7.0 or newer.
    If an older CUDA version is used, the CUDA adapter will not be built.
    This requirement will be bumped to CUDA 11.6 in a future release.

  - The `nvcc` compiler preincludes its CUDA runtime header. This results
    in multiple defined symbols if the source file is preprocessed
    before compilation. Thus enabling preprocessing and CUDA simultaneously
    is not allowed by the instrumenter.

  - Score-P's NVTX implementation does not support RangeStart/Stop APIs.
    They are supposed to cross locations and can overlap, which does not fit
    into Score-P's event model.

  - NVTX does consider different domains as disjoint range stacks, thus
    this does not fit into Score-P's event model as well. Score-P expect
    that regions from disjoint domains are properly nested.
  - Score-P's NVTX implementation might require setting `LD_PRELOAD`
    for instrumentation to work correctly if Score-P is built with NVTX
    v2 or lower (CUDA 9.0 and lower). This happens because `libnvToolsExt.so`
    might be loaded before Score-P's CUDA adapter. The required flags can
    be obtained via `scorep-preload-init --cuda [app]`.

  - Thread migration of a context is not supported. Affected memory copies
    are not recorded. This will be improved in future Score-P versions.

  - Unified memory is not supported. The CUDA adapter might report that
    memory allocations could not be found or memory was freed that has
    not been allocated. Data transfers are not shown. This will be
    improved in future Score-P versions.

  - CUDA graphs are not supported. The CUDA adapter might report that
    hash values could not be retrieved or matching failed. Kernels
    are still recorded, but allocations and data transfers are not.
    This will be improved in future Score-P versions.

  - CUDA calls transferring memory introduced after CUDA 7.0
    might not be supported correctly. The CUDA adapter might report
    that the memory copy kind could not be identified. This includes
    2D/3D memory transfers. This will be improved in future Score-P versions.

  - The CUDA adapter is not able to record kernels associated with
    OpenMP target directives to NVIDIA GPUs with LLVM 18.1 or newer, as the
    runtime is creating streams before the Score-P CUDA adapter can be
    initialized. Future versions of Score-P will handle this more gracefully.

  - When configuring Score-P with the option `--with-libNvToolsExt` and CUDA
    12.9 or newer, configure may abort due to `libNvToolsExt.so` not being
    found. This happens due to the removal of the NVTX v1/v2 libraries, which
    Score-P expects to find during configure. This will be resolved in future
    Score-P versions. For now, please use `--with-libcudart`, which should
    pick up NVTX correctly with standard CUDA directory structures.

--------------------------------------------------------------------------------

* SHMEM support

  - When using the OpenSHMEM reference implementation and building
    Score-P as a shared library, ensure that the GASNet library is
    build with -fPIC on platforms which need this flag for shared
    library code. For example, run make with the MANUAL_LIBCFLAGS and
    MANUAL_CFLAGS variables set:

      make MANUAL_LIBCFLAGS="-fPIC -DPIC" MANUAL_CFLAGS="-fPIC -DPIC"

    on GNU/Linux platforms.
    Also, if you encounter segmentation faults when running the
    instrumented application with Score-P attached, but the error
    disappears magically on subsequent runs, then the OpenSHMEM
    reference implementation may not consider some of your global or
    static variables as symmetric objects.  Please allocate these
    objects with shmalloc() to ensure that they are in the symmetric
    heap.

  - Since version 1.0g the OpenSHMEM library is compiled as a statically
    linked archive, rather than as a shared object. If you want to build
    Score-P as a shared library, ensure that the OpenSHMEM archieve is
    build with -fPIC.

  - Currently tested SHMEM implementations:
    + OpenSHMEM reference implementation 1.0h
    + Open MPI implementation up to 4.1.1
    + SGI SHMEM
    + Cray SHMEM

  - When using the Cray SHMEM versions 7.2.2 or 7.2.3, then Score-P fails
    to detect a number of SHMEM functions.  Please use a newer version of
    the module to circumvent this problem.

--------------------------------------------------------------------------------

* POSIX threads support

  - Pthread support is currently not available on systems that use
    linkers other than the GNU linker, e.g., AIX systems.

  - Please note that on systems where Pthreads or runtimes based on
    Pthreads don't need extra flags (like e.g., -pthread) to be
    compiled and linked (e.g., Cray, Blue Gene/Q), Score-P cannot
    instrument Pthreads automatically. You need to enable Pthread
    instrumentation manually via scorep's --thread=pthread option.

--------------------------------------------------------------------------------

* Sampling support

  - Sampling is mostly tested on x86-64 CPUs. The compiler needs to support
    thread-local storage, either via the language extension __thread or the C11
    keyword _Thread_local. This excludes PGI compilers prior to 'Version 2015'.

  - Sampling uses libunwind as external library. Please use version
    1.2 or later as earlier versions occasionally segfaulted.
    - libunwind used in combination with Intel MPI, even when sampling
      is not active, may mysteriously alter the application output
      just by linking libunwind. Thus, sampling/libunwind support is
      disabled when Intel MPI is detected.

  - It is possible to activate sampling for programs that were
    instrumented with compiler instrumentation. In this case the
    events originating from the compiler instrumentation will be
    suppressed. Please note that depending on the compiler
    instrumentation used, the overhead can still be significant as
    the compiler instrumentation might affect inlining. If this is
    the case, consider rebuilding your application using Score-P's
    --nocompiler switch.

  - Thread management events and intermediate trace buffer flushes may result
    in unexpected callpaths.

  - CUDA API calls are represented twice in the calling context.

  - If PAPI is used as the interrupt source for sampling, then it is not
    possible to measure also PAPI metrics.

  - The timer interrupt source only works for non-threaded applications.

  - Flushing the trace buffer from inside a sample signal is not allowed and
    thus aborts the measurement immediately.

  - Untested feature combinations with sampling:
     - OpenMP tasks
     - Trace rewinding
     - SCOREP_RECORDING_ON/SCOREP_RECORDING_OFF user instrumentation macros
     - Selective recording (SCOREP_SELECTIVE_CONFIG_FILE)

--------------------------------------------------------------------------------

* User library wrapping

  - Functions returning function pointers are not supported, e.g.:

      int ( *fn_x_return_function_pointer( void ) )( double );

    A typedef works around this limitation, e.g.:

      typedef int ( function )( void );
      function fn_x_return_function_pointer( double );

  - Nested Function pointer arguments are not supported, e.g.:

      int sion_generic_register_gather_and_execute_cb(
          int aid,
          int gather_execute_cb( const void *, long long*, int,
              long long, void *, int, int, int, int,
              int process_cb( const void *, long long *, int ) ) );

    Typedefs work around this limitation, e.g.:

      typedef int ( process_function )( const void *, long long *, int );
      typedef int ( gather_execute_function )( const void *, long long*, int,
              long long, void *, int, int, int, int,
              process_function process_cb );
      int sion_generic_register_gather_and_execute_cb(
          int aid, gather_execute_function gather_execute_cb );

  - Typedefs from classes need to be fully qualified when used, e.g., convert:

      class C {
          typedef int mytype;
          void f(mytype a);
      };

    into:

      class C {
          typedef int mytype;
          void f(C::mytype a);
      };

  - LLVM upto version 3.9 does not support the ABI tag attribute from GCC 5
    and onwards.  Thus creating library wrappers for symbols with ABI tags is
    impossible.  The issue is tracked in LLVM here:

      https://llvm.org/bugs/show_bug.cgi?id=23529

  - Building and using library wrappers for Intel Xeon Phi co-processors is
    untested.

--------------------------------------------------------------------------------

* I/O recording

  - ISO C I/O support
    - fsetpos: Seek events provides only the resulting offset
    - ungetc not supported yet

  - POSIX I/O support
    - Redirecting stdout/stderr to a file via dup2 will not affect the ISO C I/O.
      The recorded I/O of e.g. puts will target the stdout instead of the file.
    - O_NONBLOCK and O_NDELAY flags are not handled in open operations because of
      the following note (see manpage):

        Note that this flag has no effect for regular files and  block  devices;
        that is,  I/O  operations  will  (briefly)  block when device activity
        is required, regardless of whether O_NONBLOCK is set. Since O_NONBLOCK
        semantics  might eventually be implemented, applications should not
        depend upon blocking behavior when specifying this flag for regular
        files and block devices.

    - The mode of an open operation will not be tracked.
      e.g. open(..., mode_t mode);
    - putc will not be recorded because on most system it is macro. Furthermore,
      it is difficult to detect it as a macro on Cray systems.

--------------------------------------------------------------------------------

* Kokkos support

  - Currently only one accelerator device is supported. To ensure that for this
    device a Score-P location was created before first deep-copy, enable
    recording of memory allocations in the corresponding accelerator adapter
    of Score-P.
  - Kokkos/OpenMP support should be considered experimental until OMPT
    support is added to Score-P. This affects principally the Kokkos adapter.
  - Kokkos/Pthreads should be considered unsupported. It will produce a
    measurement, but probably not a meaningful one. This affects
    instrumentation of Kokkos-based codes in general, with or without use of
    the Kokkos tools interface.
  - In order for non-GPU-based back ends to support Kokkos-based allocation
    tracking and memory copy instrumentation, it is necessary to ensure that
    the threads where memory is allocated or copied are instrumented. For best
    chances of success there, we recommend using Score-P's '--thread=pthread'
    option for the OpenMP back end and a static build of Kokkos. This affects
    use of the SCOREP_KOKKOS_ENABLE=memcpy,malloc settings in the Kokkos
    adapter. If instrumenting a different configuration (e.g. a shared Kokkos
    library) is important for your use case, please contact
    support@score-p.org for help.

--------------------------------------------------------------------------------

* HIP support

  - Score-P is only able to use one compiler suite for one installation.
    Because `hipcc` is Clang based, it also only accepts the Clang compiler
    instrumentation. Thus Score-P must be build with a Clang-based compiler
    suite.

    If the compile step for device code errors out with:

      lld: error: undefined symbol: __cyg_profile_func_enter
      >>> referenced by <filename>:<linenumber>

    then the heuristic to detect the usage of `hipcc` failed and must be
    enabled explicity with `scorep --hip`.

  - Device-to-device data transfers are only recorded on the API level, not
    as a device activity.

  - Setting compile flags via the environment variable
    `HIPCC_COMPILE_FLAGS_APPEND` or link flags via the environment variable
    `HIPCC_LINK_FLAGS_APPEND` may interfere with correct instrumentation, most
    likely resulting in an empty experiment directory when MPI is used.
    Please pass the desired compile/link flags explicitly instead. If this is
    not possible, try to preload the Score-P libraries via
    `LD_PRELOAD=<Score-P prefix>/lib/libscorep_adapter_mpi_event.so`.

--------------------------------------------------------------------------------

* OpenACC support

  - Score-P may generate OpenACC related events if the compiler supports
    the OpenACC Profiling (and Error) Callback Interface. It is a Score-P
    requirement that the OpenACC runtime registers a tool by linking a
    library that contains the function `acc_register_library`. Tool
    registration via `ACC_PROFLIB` or `LD_PRELOAD` is not supported.
    Currently, only NVIDIA compilers are known to support the interface
    and to provide events to Score-P. With GNU compilers, a tool can be
    compiled, but it won't be registered by linking `acc_register_library`
    to the program.

--------------------------------------------------------------------------------

* Score-P misc.

  - Score-P does not support MPMD style programs where the executables
    are instrumented differently. I.e., it is best to compile/link all
    sources with the same set of flags for the Score-P instrumenter, so
    that the instrumenter itself cannot automatically decides what
    paradigms the program uses. This applies to the --mpp and --thread
    flags.

  - There might be a performance impact when instrumenting code
    without explicitly given optimization flags. The instrumenter adds
    compiler flags to enable additional debugging
    information. Depending on the compiler this may turn off
    optimization unless optimization flags are explicitly specified.

  - For autotools or CMake based build systems, please consult the usage
    of `scorep-wrapper` to learn about the recommended way to apply
    instrumentation.

  - Literal file-filter rules like "INCLUDE bt.f" for Fortran files
    that will be processed by OPARI2 (i.e., files containing OpenMP or
    POMP user pragmas) do not work as expected as OPARI2 changes the
    file name (here to bt.opari.f)

  - Rusage-based metrics are not supported on Blue Gene systems.

  - Traces generated by applications compiled with the CCE Fortran
    compiler on Cray X series systems are inconsistent as there is no
    exit event for 'main'. I.e., such traces are not analyzable by
    Scalasca.

  - Running make check fails if SCOREP_EXPERIMENT_DIRECTORY is set to
    'scorep'. One workaround is to run '(unset
    SCOREP_EXPERIMENT_DIRECTORY; make check;)'.

  - In some cases the PGI 11.7 compiler fails to link C++ programs
    that use user-instrumentation macros due to an undefined reference
    to '_Unwind_Resume'. Adding -lstdc++ to the original link line
    solves this issue.

  - The Cube metric hierarchy remapping specification file shipped with
    Score-P currently classifies all MPI request finalization calls (i.e.,
    MPI_Test[all|any|some] and MPI_Wait[all|any|some]) as point-to-point,
    regardless of whether they actually finalize a point-to-point request,
    a file I/O request, or a non-blocking collective operation. This will
    be addressed in a future release.

  - When instrumenting memory API calls with the Cray CCE, Score-P uses
    the '-h system_alloc' flag and therefore not the default tcmalloc
    library (note that -h tcmalloc was deprecated in CCE-15).

  - Tracking memory allocations in Fortran programs may only work, if the
    compiler generate calls to malloc/free. This is only known for the
    GNU compiler.

  - Score-P uses an 'atexit' handler to finalize the measurement and produces
    the output files.  In case a run-time system also uses an 'atexit' handler
    but does not allow previous registered 'atexit' handlers to run, then one
    can try to re-install the 'atexit' handler again, so that Score-P's will
    run before the malicious handler.  If that still does not work one can
    also trigger the finalization manually.  Here are the C prototypes for
    these two functions:

      void SCOREP_RegisterExitHandler( void );

      void SCOREP_FinalizeMeasurement( void );

    One known malicious run-time system is NetDRMS.

  - Instrumentation of files that include system headers via local
    headers of the same name fails for compilers that don't support
    the '-iquote' option (e.g., PGI). In this case, renaming of the
    local header is the only workaround.

  - Recording arguments of the executable was disabled due to unreliable values
    on some old HPC platforms now decommissioned. If there are program aborts
    inside Score-P, please do make check and installcheck and exercise the
    constructor checks:

      $ make
      $ make -C build-backend check-build
      $ make -C build-backend check-serial TESTS_SERIAL="test_constructor_check_c test_constructor_check_cxx test_constructor_f"
      $ ./build-backend/test_constructor_check_c 1 2 3
      $ ./build-backend/test_constructor_check_cxx 1 2 3
      $ ./build-backend/test_constructor_check_f 1 2 3
      $ make install
      $ make -C build-backend constructor-checks
      $ ./installcheck/constructor_checks/bin/run_constructor_checks.sh

  - For AMD's AOCC 4.1.0 and newer (tested till 5.0.0), AOCC's clang
    compiler prepends its library directory to user provided library
    directories if -fopenmp is given. As a consequence, AOCC's
    incompatible libunwind is linked instead of the one provided to
    Score-P's configure. This fails with undefined symbol errors. One
    could configure --without-libunwind to work around this compiler
    bug. Still, there might be subtle differences, as not only
    libunwind is found and silently linked in AOCC's library
    directory.
    The same applies to AMD's ROCm compiler amdclang in version 6.1.2
    and newer.

--------------------------------------------------------------------------------

Please report bugs, wishes, and suggestions to <support@score-p.org>.
