Skip to main content

Modules Compiling

EasyBuild Recipe Commands Reference

Overview

EasyBuild Modules Compiling automates the process of installing scientific software modules on computing clusters or systems. It simplifies complex software builds and installations, providing a unified and automated approach.

Basic Build Commands

# Build a recipe
eb MySoftware-1.0.0-foss-2023a.eb

# Build with dependencies using robot mode
eb MySoftware-1.0.0-foss-2023a.eb --robot

# Robot mode with current directory as search path
eb -r . MySoftware-1.0.0-foss-2023a.eb

# Build ignoring checksums (use with caution)
eb -r . --ignore-checksums path/MySoftware-1.0.0-foss-2023a.eb

# Dry run (show what would be built)
eb MySoftware-1.0.0-foss-2023a.eb --dry-run

# Build multiple recipes
eb *.eb --robot

How to Run EasyBuild Module Compiling

    1.

  1. Select Software:

    2. Choose the software package you want to install. For example, let's say you want to install the software package "example-software"GROMACS".

    2.

  2. Build the Module:

    3. Use the eb command followedwith byrobot themode name of the software package to build it. You can optionally include the (--ignore-checksumsr) optionand todependency ignoresearch checksumpath verification. For example:(.):

    eb -r . (--ignore-checksums) path/example-software.GROMACS-2023.1-foss-2023a.eb

theThe dot (.) is the starting searchingsearch path for the Dependencydependency modules. The -r stands for Robot mode - it searches for the Dependenciesdependencies starting searchingfrom the search path.

3. Load the Module:

After the module is built, load it into your environment using the module load command. For example:environment:

module load example-module.ebGROMACS/2023.1-foss-2023a

    4.
  1. Verify Installation:

    2. You can verify

    Check that the software is installed correctly by checking its version or running a basic command associated with it.

Example:

Suppose you want to install the software package "GROMACS" while ignoring checksum verification. Here's how you would do it:

  1. Build the Module:correctly:

    eb -r .gmx --ignore-checksums path/GROMACS.ebversion

    2. Recipe
  2. Creation

    Load the Module:

    Commands

    3. # Create new recipe from template
eb --new-recipe MySoftware 1.0.0

# Copy existing recipe as template
cp existing-recipe.eb MySoftware-1.0.0-foss-2023a.eb

# Search for existing recipes
eb --search MySoftware

# Find recipe files
find $EASYBUILD_CONFIGFILES -name "*MySoftware*"

    Debugging Commands

    # Build with debug output
eb MySoftware-1.0.0-foss-2023a.eb --debug

# Stop at specific step
eb MySoftware-1.0.0-foss-2023a.eb --stop=configure
eb MySoftware-1.0.0-foss-2023a.eb --stop=build
eb MySoftware-1.0.0-foss-2023a.eb --stop=install

# Keep build directory
eb MySoftware-1.0.0-foss-2023a.eb --stop=build --try-update-deps

# Force rebuild
eb MySoftware-1.0.0-foss-2023a.eb --force --rebuild

# Skip sanity check
eb MySoftware-1.0.0-foss-2023a.eb --skip-test-step

    Testing Commands

    # Test existing installation
eb MySoftware-1.0.0-foss-2023a.eb --sanity-check-only

# Run tests during build
eb MySoftware-1.0.0-foss-2023a.eb --run-all-tests

# Extended dry run with tests
eb MySoftware-1.0.0-foss-2023a.eb --extended-dry-run

    Dependency Management

    # Check dependencies
eb MySoftware-1.0.0-foss-2023a.eb --dry-run --robot

# Missing dependencies only
eb MySoftware-1.0.0-foss-2023a.eb --missing-modules

# Resolve dependencies automatically
eb MySoftware-1.0.0-foss-2023a.eb --robot --dry-run

# Build dependencies only
eb MySoftware-1.0.0-foss-2023a.eb --robot --stop=ready

    Validation Commands

    # Check recipe syntax
eb MySoftware-1.0.0-foss-2023a.eb --check-syntax

# Validate checksums
eb MySoftware-1.0.0-foss-2023a.eb --check-checksums

# Check for conflicts
eb MySoftware-1.0.0-foss-2023a.eb --check-conflicts

    Information Commands

    # Show recipe details
eb MySoftware-1.0.0-foss-2023a.eb --show-config

# List toolchains
eb --list-toolchains

# List available easyblocks
eb --list-easyblocks

# Show software details
eb --software-list | grep -i mysoftware

    Build Path Management

    # Use temporary build directory
eb MySoftware-1.0.0-foss-2023a.eb --tmpdir=/tmp/eb_build

# Specify build directory
eb MySoftware-1.0.0-foss-2023a.eb --buildpath=/scratch/build

# Clean build directory after success
eb MySoftware-1.0.0-foss-2023a.eb --cleanup-builddir

    Advanced Commands

    # Use specific Python version
eb MySoftware-1.0.0-foss-2023a.eb --try-toolchain=GCC,12.3.0

# Update toolchain
eb MySoftware-1.0.0-foss-2023a.eb --try-update-deps

# Use different installation path
eb MySoftware-1.0.0-foss-2023a.eb --installpath=/opt/software

# Parallel builds
eb MySoftware-1.0.0-foss-2023a.eb --parallel=8

    Environment Setup

    # Set EasyBuild environment
export EASYBUILD_CONFIGFILES=/path/to/config
export EASYBUILD_BUILDPATH=/tmp/easybuild
export EASYBUILD_INSTALLPATH=/opt/easybuild
export EASYBUILD_SOURCEPATH=/opt/easybuild/sources

# Module environment
module use /opt/easybuild/modules/all
module load GROMACSEasyBuild

    Recipe

Example Structure

Basic EasyBuild Recipe Example

When writing a recipe for EasyBuild, you'll create a (.eb file for each software package you want to build. This file contains instructions for EasyBuild on how to download, configure, build, and install the software package. Here's a basic example of what an EasyBuild recipe file might look like:

file) 
easyblock = 'AutotoolsMake'
name = 'example-software'
version = '1.0'
homepage = 'https://example.com'
sourcesdescription = [SOURCE_TAR_GZ]"Example dependenciessoftware package"

toolchain = [('GCC', '9.3.0')]

moduleclass = 'tools'

sanity_check_paths = {
    'files': [],
    'dirs': []
}

buildopts = {'toolchain': {'name': 'GCC', 'version': '9.3.0'}}

sources = [SOURCE_TAR_GZ]
source_urls = ['https://example.com/downloads/']

dependencies = [('GCC', '9.3.0')]

patches = [
    ('patch1.patch', 1),
    ('patch2.patch', 1)
]

moduleclass = 'tools'

preconfigopts = [ './configure --prefix=$EBROOTEXAMPLE_SOFTWARE',

]

moduleclassbuildopts = 'tools'VERBOSE=1'

postinstallcmds = [
    'echo "Example software installation complete."'
]

sanity_check_paths = {
    'files': ['bin/example-software'],
    'dirs': ['lib', 'include']
}

moduleclass = 'tools'

EasyBlock Types

The easyblock variable specifies the EasyBuildbuild easyblockmethod:

class
    for
  • AutotoolsMake building- theFor softwareAutotools-based package.packages
    • Easyblocks
  • CMakeMake are- For CMake-based packages
  • PythonPackage - For Python classespackages
    • that
  • MakeFile handle the build process.- For example:

    simple Makefile packages
  • ConfigureMake - For configure + make packages
cat > MySoftware-1.0.0-foss-2023a.eb << 'EOF'
easyblock = 'AutotoolsMake'ConfigureMake'
name = 'MySoftware'
version = '1.0.0'
homepage = 'https://example.com'
description = "Description here"
toolchain = {'name': 'foss', 'version': '2023a'}
source_urls = ['https://github.com/user/repo/archive/']
sources = ['v%(version)s.tar.gz']
checksums = ['checksum_here']
configopts = '--enable-shared'
sanity_check_paths = {'files': ['bin/mysoftware'], 'dirs': []}
moduleclass = 'tools'
EOF

This

CMake lineRecipe

sets
cat the> MySoftware-1.0.0-foss-2023a.eb << 'EOF'
easyblock class= to'CMakeMake'
AutotoolsMake,name suitable= for'MySoftware'
packagesversion using= the'1.0.0'
Autotoolshomepage = 'https://example.com'
description = "Description here"
toolchain = {'name': 'foss', 'version': '2023a'}
source_urls = ['https://github.com/user/repo/archive/']
sources = ['v%(version)s.tar.gz']
checksums = ['checksum_here']
configopts = '-DCMAKE_BUILD_TYPE=Release'
sanity_check_paths = {'files': ['bin/mysoftware'], 'dirs': []}
moduleclass = 'tools'
EOF

Python Package Recipe

cat > MySoftware-1.0.0-foss-2023a.eb << 'EOF'
easyblock = 'PythonPackage'
name = 'MySoftware'
version = '1.0.0'
homepage = 'https://pypi.org/project/mysoftware'
description = "Description here"
toolchain = {'name': 'foss', 'version': '2023a'}
source_urls = [PYPI_SOURCE]
sources = [SOURCELOWER_TAR_GZ]
checksums = ['checksum_here']
dependencies = [('Python', '3.11.3')]
use_pip = True
download_dep_fail = True
sanity_pip_check = True
moduleclass = 'tools'
EOF

Common Workflow

# 1. Create recipe
eb --new-recipe MySoftware 1.0.0

# 2. Edit recipe file
vim MySoftware-1.0.0-foss-2023a.eb

# 3. Test syntax
eb MySoftware-1.0.0-foss-2023a.eb --check-syntax

# 4. Dry run
eb MySoftware-1.0.0-foss-2023a.eb --dry-run --robot

# 5. Build
eb MySoftware-1.0.0-foss-2023a.eb --robot

# 6. Test installation
eb MySoftware-1.0.0-foss-2023a.eb --sanity-check-only

Troubleshooting Commands

# Check build system.log
Othertail common-f classes/tmp/eb-*/easybuild-MySoftware-*.log

include# CMake,Find PythonPackage,build directory
find /tmp -name "*MySoftware*" -type d

# Check module availability
module avail MySoftware

# Debug failed build
eb MySoftware-1.0.0-foss-2023a.eb --debug --stop=configure

# Clean failed build
eb MySoftware-1.0.0-foss-2023a.eb --force --cleanup-builddir

References and MakeFile, each for different types of software packages. This choice ensures EasyBuild applies the correct build logic and commands for successful compilation and installation.

This is just a basic example, and the contents of the .eb file can vary depending on the specific requirements of the software package you're building.

Resources

ReferencesDocumentation

  • High-Performance Computing (HPC)
  • Environment Modules
  • Lmod

Categories

  • High-Performance Computing
  • Software Development
  • Computational Science
  • Build Automation
Back to top