Modules Compiling

EasyBuild Recipe Commands Reference

Overview

EasyBuild Modules Compiling automates the process of installing scientific software modules on computing clusters or systems. It simplifies complex software builds and installations, providing a unified and automated approach.

Basic Build Commands

# Build a recipe
eb MySoftware-1.0.0-foss-2023a.eb

# Build with dependencies using robot mode
eb MySoftware-1.0.0-foss-2023a.eb --robot

# Robot mode with current directory as search path
eb -r . MySoftware-1.0.0-foss-2023a.eb

# Build ignoring checksums (use with caution)
eb -r . --ignore-checksums path/MySoftware-1.0.0-foss-2023a.eb

# Dry run (show what would be built)
eb MySoftware-1.0.0-foss-2023a.eb --dry-run

# Build multiple recipes
eb *.eb --robot

How to Run EasyBuild Module Compiling

1. Select Software

Choose the software package you want to install. For example, "GROMACS".

2. Build the Module

Use the eb command with robot mode (-r) and dependency search path (.):

eb -r . path/GROMACS-2023.1-foss-2023a.eb

The dot (.) is the starting search path for dependency modules. The -r stands for Robot mode - it searches for dependencies starting from the search path.

3. Load the Module

After the module is built, load it into your environment:

module load GROMACS/2023.1-foss-2023a

4. Verify Installation

Check that the software is installed correctly:

gmx --version

Recipe Creation Commands

# Create new recipe from template
eb --new-recipe MySoftware 1.0.0

# Copy existing recipe as template
cp existing-recipe.eb MySoftware-1.0.0-foss-2023a.eb

# Search for existing recipes
eb --search MySoftware

# Find recipe files
find $EASYBUILD_CONFIGFILES -name "*MySoftware*"

Debugging Commands

# Build with debug output
eb MySoftware-1.0.0-foss-2023a.eb --debug

# Stop at specific step
eb MySoftware-1.0.0-foss-2023a.eb --stop=configure
eb MySoftware-1.0.0-foss-2023a.eb --stop=build
eb MySoftware-1.0.0-foss-2023a.eb --stop=install

# Keep build directory
eb MySoftware-1.0.0-foss-2023a.eb --stop=build --try-update-deps

# Force rebuild
eb MySoftware-1.0.0-foss-2023a.eb --force --rebuild

# Skip sanity check
eb MySoftware-1.0.0-foss-2023a.eb --skip-test-step

Testing Commands

# Test existing installation
eb MySoftware-1.0.0-foss-2023a.eb --sanity-check-only

# Run tests during build
eb MySoftware-1.0.0-foss-2023a.eb --run-all-tests

# Extended dry run with tests
eb MySoftware-1.0.0-foss-2023a.eb --extended-dry-run

Dependency Management

# Check dependencies
eb MySoftware-1.0.0-foss-2023a.eb --dry-run --robot

# Missing dependencies only
eb MySoftware-1.0.0-foss-2023a.eb --missing-modules

# Resolve dependencies automatically
eb MySoftware-1.0.0-foss-2023a.eb --robot --dry-run

# Build dependencies only
eb MySoftware-1.0.0-foss-2023a.eb --robot --stop=ready

Validation Commands

# Check recipe syntax
eb MySoftware-1.0.0-foss-2023a.eb --check-syntax

# Validate checksums
eb MySoftware-1.0.0-foss-2023a.eb --check-checksums

# Check for conflicts
eb MySoftware-1.0.0-foss-2023a.eb --check-conflicts

Information Commands

# Show recipe details
eb MySoftware-1.0.0-foss-2023a.eb --show-config

# List toolchains
eb --list-toolchains

# List available easyblocks
eb --list-easyblocks

# Show software details
eb --software-list | grep -i mysoftware

Build Path Management

# Use temporary build directory
eb MySoftware-1.0.0-foss-2023a.eb --tmpdir=/tmp/eb_build

# Specify build directory
eb MySoftware-1.0.0-foss-2023a.eb --buildpath=/scratch/build

# Clean build directory after success
eb MySoftware-1.0.0-foss-2023a.eb --cleanup-builddir

Advanced Commands

# Use specific Python version
eb MySoftware-1.0.0-foss-2023a.eb --try-toolchain=GCC,12.3.0

# Update toolchain
eb MySoftware-1.0.0-foss-2023a.eb --try-update-deps

# Use different installation path
eb MySoftware-1.0.0-foss-2023a.eb --installpath=/opt/software

# Parallel builds
eb MySoftware-1.0.0-foss-2023a.eb --parallel=8

Environment Setup

# Set EasyBuild environment
export EASYBUILD_CONFIGFILES=/path/to/config
export EASYBUILD_BUILDPATH=/tmp/easybuild
export EASYBUILD_INSTALLPATH=/opt/easybuild
export EASYBUILD_SOURCEPATH=/opt/easybuild/sources

# Module environment
module use /opt/easybuild/modules/all
module load EasyBuild

Recipe Example Structure

Basic EasyBuild Recipe (.eb file)

easyblock = 'AutotoolsMake'
name = 'example-software'
version = '1.0'
homepage = 'https://example.com'
description = "Example software package"

toolchain = {'name': 'GCC', 'version': '9.3.0'}

sources = [SOURCE_TAR_GZ]
source_urls = ['https://example.com/downloads/']

dependencies = [('GCC', '9.3.0')]

patches = [
    ('patch1.patch', 1),
    ('patch2.patch', 1)
]

preconfigopts = './configure --prefix=$EBROOTEXAMPLE_SOFTWARE'

buildopts = 'VERBOSE=1'

postinstallcmds = [
    'echo "Example software installation complete."'
]

sanity_check_paths = {
    'files': ['bin/example-software'],
    'dirs': ['lib', 'include']
}

moduleclass = 'tools'

EasyBlock Types

The easyblock variable specifies the build method:

AutotoolsMake - For Autotools-based packages
CMakeMake - For CMake-based packages
PythonPackage - For Python packages
MakeFile - For simple Makefile packages
ConfigureMake - For configure + make packages

cat > MySoftware-1.0.0-foss-2023a.eb << 'EOF'
easyblock = 'ConfigureMake'
name = 'MySoftware'
version = '1.0.0'
homepage = 'https://example.com'
description = "Description here"
toolchain = {'name': 'foss', 'version': '2023a'}
source_urls = ['https://github.com/user/repo/archive/']
sources = ['v%(version)s.tar.gz']
checksums = ['checksum_here']
configopts = '--enable-shared'
sanity_check_paths = {'files': ['bin/mysoftware'], 'dirs': []}
moduleclass = 'tools'
EOF

CMake Recipe

cat > MySoftware-1.0.0-foss-2023a.eb << 'EOF'
easyblock = 'CMakeMake'
name = 'MySoftware'
version = '1.0.0'
homepage = 'https://example.com'
description = "Description here"
toolchain = {'name': 'foss', 'version': '2023a'}
source_urls = ['https://github.com/user/repo/archive/']
sources = ['v%(version)s.tar.gz']
checksums = ['checksum_here']
configopts = '-DCMAKE_BUILD_TYPE=Release'
sanity_check_paths = {'files': ['bin/mysoftware'], 'dirs': []}
moduleclass = 'tools'
EOF

Python Package Recipe

cat > MySoftware-1.0.0-foss-2023a.eb << 'EOF'
easyblock = 'PythonPackage'
name = 'MySoftware'
version = '1.0.0'
homepage = 'https://pypi.org/project/mysoftware'
description = "Description here"
toolchain = {'name': 'foss', 'version': '2023a'}
source_urls = [PYPI_SOURCE]
sources = [SOURCELOWER_TAR_GZ]
checksums = ['checksum_here']
dependencies = [('Python', '3.11.3')]
use_pip = True
download_dep_fail = True
sanity_pip_check = True
moduleclass = 'tools'
EOF

Common Workflow

# 1. Create recipe
eb --new-recipe MySoftware 1.0.0

# 2. Edit recipe file
vim MySoftware-1.0.0-foss-2023a.eb

# 3. Test syntax
eb MySoftware-1.0.0-foss-2023a.eb --check-syntax

# 4. Dry run
eb MySoftware-1.0.0-foss-2023a.eb --dry-run --robot

# 5. Build
eb MySoftware-1.0.0-foss-2023a.eb --robot

# 6. Test installation
eb MySoftware-1.0.0-foss-2023a.eb --sanity-check-only

Troubleshooting Commands

# Check build log
tail -f /tmp/eb-*/easybuild-MySoftware-*.log

# Find build directory
find /tmp -name "*MySoftware*" -type d

# Check module availability
module avail MySoftware

# Debug failed build
eb MySoftware-1.0.0-foss-2023a.eb --debug --stop=configure

# Clean failed build
eb MySoftware-1.0.0-foss-2023a.eb --force --cleanup-builddir

References and Resources

Documentation

EasyBuild Documentation: https://easybuild.readthedocs.io
EasyBuild GitHub Repository: https://github.com/easybuilders/easybuild
EasyBuild Website: https://www.easybuild.io/
EasyBuild Community Wiki: https://easybuild.io/

High-Performance Computing (HPC)
Environment Modules
Lmod
Software Development
Computational Science
Build Automation