macOS xterm-256color fish 174 views

Env

module load  compiler/devtoolset/7.3.1   compiler/rocm/dtk/24.04.3   mpi/hpcx/2.11.0/gcc-7.3.1   compiler/cmake/3.24.1  

code version: https://git.hpcer.dev/source-flow/source-flow/-/commit/af0eca9b5ba9836bb6db4fb6062376c0568618e7

Code prepare on your local environment

git clone ssh://git@git.hpcer.dev:2222/source-flow/source-flow.git  
cd source-flow
pkg fetch
vi dep.cmake  # change openblas path

Then, we can pack the code and upload them to remote.

Build

export C_INCLUDE_PATH=$C_INCLUDE_PATH:/public/home/wangxg/public/software/openblas/0.3.8-gcc7.3.1/include/
export CC=gcc CXX=g++ FC=gfortran
pkg install -j 4
cmake -B ./build -S ./ \
   -DNVWA_ENABLE_HIP_EXECUTOR=ON \
   -DHIP_HIPCC_FLAGS="-std=c++11 -fgpu-rdc" \
   -DCMAKE_LINKER=hipcc \
   -DCMAKE_CXX_LINK_EXECUTABLE="<CMAKE_LINKER> <FLAGS> -fgpu-rdc <CMAKE_CXX_LINK_FLAGS> <LINK_FLAGS> <OBJECTS> -o <TARGET> <LINK_LIBRARIES>"
cmake --build ./build -j 4

这里:

  • 采用 hipcc 进行可执行程序的链接,因为需要链接系统 dtk 相关的库;
  • 启用 HIP 加速(NVWA_ENABLE_HIP_EXECUTOR 选项);
  • GNU 下采用 0x00000001 作为 Fortran 的 .true. (现在这个版本,不用特殊设置,会自动检测);
  • 链接加上参数:-fgpu-rdc,避免错误:对‘__hip_fatbin’未定义的引用;
  • hipcc 编译加上参数 -fgpu-rdc

run

export HIP_KERNEL_PRINTF=1
srun --mpi=pmix_v3 -p normal -n 12 --gres=dcu:4 ../../build/src/nvwa_c top_flow

参考:类似的错误,可以参见这里解决:https://misa-md.github.io/MDoc/docs/compiling-with-hip/#常见问题-rocm

More recordings by genshen