Building the kaiju software (serial only) on a Mac

Introduction

These instructions will walk you through the process of building and installing the kaiju software on an Intel-based Macintosh computer running the MacOS Ventura operating system.

These instructions assume that the user is using the bash shell, and that no modifications have been made to the user “dotfiles” ($HOME/.bashrc, $HOME/.bash_profile). If you have customized either of these files for your account, please carefully save and inspect the output from each command in the build process to ensure that no unexpected problems have crept in. To facilitate this practice, all of the commands shown below will illustrate how to save command output, and how to measure how long each step takes. The latter is a useful bit of information which can help identify build problems early in the process, avoiding much wasted time and effort later.

Unlike most HPC systems, a Mac does not use the module system to manage the versions of software packages available to the user. Compilers and libraries must be installed by the user before starting on building the kaiju software.

Building the serial version of the kaiju software

Step 1: Configure the required software

Set environment variables for the Intel fortran compiler (adjust the path as needed for your machine):

export INTEL_HOME=/opt/intel
export PATH=$INTEL_HOME/oneapi/compiler/latest/mac/bin/intel64:$PATH

Set environment variables for CDF (adjust the paths as needed for your machine):

source $HOME/ventura/local/cdf/3.9.0/bin/definitions.B

Set environment variables for HDF5 (adjust the paths as needed for your machine):

export HDF5_DIR=$HOME/ventura/local/hdf5/1.14.1-2
export PATH=$HDF5_DIR/bin:$PATH
export HDF5_INCLUDE_DIRS=$HDF5_DIR/include
export HDF5_LIBRARIES=$HDF5_DIR/lib
export CPATH=$HDF5_INCLUDE_DIRS

Step 2: Create the build directory

Create a ventura-specific build directory.

cd $HOME
mkdir -p ventura
cd ventura

Then make an additional subdirectory level for the branch of the code you are building (the development branch is used as an example). This arrangement is useful when you need to maintain simultaneous builds of different branches.

mkdir development
cd development

Step 3: Clone the kaiju repository from BitBucket

NOTE: This step assumes you have been granted access to the kaiju repository on BitBucket, and that you have configured an SSH key pair for use with BitBucket. If you need help with these tasks, please contact a CGS team member for assistance.

Clone the kaiju repository (or “repo”) from BitBucket:

git clone git@bitbucket.org:aplkaiju/kaiju.git

This process should take a minute or so. When complete, verify that the kaiju code exists in your directory (the actual directory contents may differ slightly from what is shown below):

ls kaiju
analysis  cmake  CMakeLists.txt  examples  external  gitHookScripts  kaiju.sublime-project  kaipy  places  pytests  quickstart  README.md  scripts  setup.py  src  testingScripts  tests  xml

Now move down into the cloned repo, and switch to the branch of the code you wish to use. By default, the cloned repository provides the master branch, but we want the development branch:

cd kaiju
git switch development

Step 4: Run cmake to create the Makefile needed to build the software

Since the kaiju code can be built in serial and MPI forms, we first make a directory in which to build the serial version of the code (use whatever name you prefer, but build_serial is simple and unambiguous):

mkdir build_serial
cd build_serial

Now run the cmake command. Save the cmake output, and use timestamps for each step. The options shown below direct the build process to use a recent version of the Intel Fortran compiler:

date; time FC=`which ifort` cmake -DALLOW_INVALID_COMPILERS=ON >& cmake.out; date

This command usually takes less than 10 seconds, depending on system activity. Examine the output file cmake.out for problems. It should look something like this:

-- The Fortran compiler identification is Intel 2021.9.0.20230302
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Check for working Fortran compiler: /opt/intel/oneapi/compiler/latest/mac/bin/intel64/ifort - skipped
-- Found HDF5: /Users/winteel1/ventura/local/hdf5/1.14.1-2/lib/libhdf5_fortran.dylib;/Users/winteel1/ventura/local/hdf5/1.14.1-2/lib/libhdf5.dylib;/Library/Developer/CommandLineTools/SDKs/MacOSX13.3.sdk/usr/lib/libz.tbd;/Library/Developer/CommandLineTools/SDKs/MacOSX13.3.sdk/usr/lib/libdl.tbd;/Library/Developer/CommandLineTools/SDKs/MacOSX13.3.sdk/usr/lib/libm.tbd (found version "1.14.1-2") found components: Fortran
-- Found OpenMP_Fortran: -qopenmp (found version "5.0")
-- Found OpenMP: TRUE (found version "5.0") found components: Fortran
CMake Warning at cmake/compilers.cmake:61 (message):
  Setting default optimization to O2 to avoid certain Intel compiler bugs
Call Stack (most recent call first):
  CMakeLists.txt:99 (include)


-------------------------
Configuration summary ...
System: C02FC1QCMD6T-ML
OS: Darwin
Processor: x86_64
Compiler: Intel / 2021.9.0.20230302
          /opt/intel/oneapi/compiler/latest/mac/bin/intel64/ifort
HDF5 Wrapper: /Users/winteel1/ventura/local/hdf5/1.14.1-2/bin/h5fc
Version: 339707e4 / ewinter-python_environment_upgrade
Build Type: Release
Base Flags:  -fPIC -Wl,-stack_size,0x40000000,-stack_addr,0xf0000000 -xHost -free -implicitnone -qopenmp
Build Flags: -O2 -align array64byte -align rec32byte -no-prec-div -fast-transcendentals
-------------------------

Adding CHIMP module ...
    EB IC file is /Users/winteel1/ventura/cgs/kaiju/ewinter-python_environment_upgrade/kaiju/src/chimp/ebICs/ebICstd.F90
    TP IC file is /Users/winteel1/ventura/cgs/kaiju/ewinter-python_environment_upgrade/kaiju/src/chimp/tpICs/tpICstd.F90
    Adding executable project.x
    Adding executable psd.x
    Adding executable push.x
    Adding executable slice.x
    Adding executable chop.x
    Adding executable trace.x
    Adding executable sctrack.x
    Adding executable calcdb.x
    Adding executable wpicheck.x
Adding Gamera module ...
    Bricksize is 16
    IC file is /Users/winteel1/ventura/cgs/kaiju/ewinter-python_environment_upgrade/kaiju/src/gamera/ICs/null.F90
    Adding executable gamera.x
Adding Gamera Helio module ...
    IC file is /Users/winteel1/ventura/cgs/kaiju/ewinter-python_environment_upgrade/kaiju/src/gamera/ICs/helio/wsa.F90
    Adding executable gamhelio.x
Adding ReMIX module ...
    Adding executable remix.x
Adding RCM module ...
    RCM Grid is of size 180 x 361 x 160
    Adding executable rcm.x
Adding Voltron module ...
    IC file is /Users/winteel1/ventura/cgs/kaiju/ewinter-python_environment_upgrade/kaiju/src/voltron/ICs/earthcmi.F90
    Adding executable voltron.x
-- Configuring done (7.6s)
-- Generating done (0.3s)
-- Build files have been written to: /Users/winteel1/ventura/cgs/kaiju/ewinter-python_environment_upgrade/kaiju/build_serial

Step 5: Compile the kaiju software

Now use make to build the kaiju software, time-stamping and saving the output:

date; time make >& make.out; date

This command should complete in about 3-4 minutes on a modern Mac. When the command is finished, check the output file make.out. The file is long, but the last few lines should look something like this:

[ 99%] Linking Fortran static library libvoltlib.a
[ 99%] Built target voltlib
[100%] Building Fortran object CMakeFiles/voltron.x.dir/src/drivers/voltronx.F90.o
[100%] Linking Fortran executable bin/voltron.x
[100%] Built target voltron.x
[100%] Built target voltron

To verify that all of the kaiju programs have been built, examine the bin subdirectory of your build_serial directory (this list will evolve as more programs are added):

ls bin
calcdb.x  chop.x  gamera.x  gamhelio.x  kaitoy.x  project.x  psd.x  push.x  rcm.x  remix2rcm.x  remix2remix.x  remix.x  sctrack.x  slice.x  trace.x  voltron.x  wpicheck.x

Using the kaiju software

Once built, you must run the setup script before using the kaiju software:

source $KAIJU_HOME/scripts/setupEnvironment.sh

This script will set environment variables needed by the kaiju software, including the KAIJUHOME environment variable. However, the path to the compiled programs is not added - you will need to specify the complete path when using compiled programs. For example, to run the serial version of gamera.x:

$KAIJUHOME/build_serial/bin/gamera.x