Deploying Modules

In order to test your module, you should learn Jarvis, which is the tool that is used to deploy the iowarp runtime.

In the previous section, we built a module repo named "my_mod_repo" with the namespace "example" and a module named "compressor" in that repo. This section will give detail on how to build a jarvis package to test the compressor module.

The Current Example

my_mod_repo
├── chimaera_repo.yaml  # Repo metadata
├── CMakeLists.txt      # Repo cmake
└── compressor
    ├── chimaera_mod.yaml  # Module metadata
    ├── CMakeLists.txt     # Module cmake
    ├── include
    │   └── compressor
    │       ├── compressor_client.h      # Client API
    │       ├── compressor_lib_exec.h    # (autogenerated from *methods.yaml)
    │       ├── compressor_methods.h     # (autogenerated from *methods.yaml)
    │       ├── compressor_methods.yaml  # Task declarations 
    │       └── compressor_tasks.h       # Task struct definitions 
    └── src
        ├── CMakeLists.txt          # Builds compressor_client and runtime  
        ├── compressor_client.cc    # Client API source
        ├── compressor_monitor.py   # Used for monitoring
        └── compressor_runtime.cc   # Runtime API source

Create a simple unit test

mkdir ~/my_mod_repo/compressor/test

Edit the file ~/my_mod_repo/compressor/test/test.cc:

#include "compressor/compressor_client.h"

int main() {
  CHIMAERA_CLIENT_INIT();
  chi::compressor::Client client;
  client.Create(
      HSHM_MCTX,
      chi::DomainQuery::GetDirectHash(chi::SubDomainId::kGlobalContainers, 0),
      chi::DomainQuery::GetGlobalBcast(), "ipc_test");

  size_t data_size = hshm::Unit<size_t>::Megabytes(1);
  hipc::FullPtr<char> orig_data =
      CHI_CLIENT->AllocateBuffer(HSHM_MCTX, data_size);
  client.Compress(HSHM_MCTX, chi::DomainQuery::GetLocalHash(0), orig_data.shm_,
                  data_size);
  return 0;
}

Add the tester CMake

Edit the file ~/my_mod_repo/compressor/test/CMakeLists.txt:

add_executable(compress_test test.cc)
target_link_libraries(compress_test example::compressor_client)

install(
  TARGETS
  compress_test
  EXPORT
  ${CHIMAERA_EXPORTED_TARGETS}
  LIBRARY DESTINATION ${CHIMAERA_INSTALL_LIB_DIR}
  ARCHIVE DESTINATION ${CHIMAERA_INSTALL_LIB_DIR}
  RUNTIME DESTINATION ${CHIMAERA_INSTALL_BIN_DIR}
)

This will create an executable named compress_test when make is executed.

Connect the tester CMake to the overall module

And then edit compressor's root cmake ~/my_mod_repo/compressor/CMakeLists.txt:

# ------------------------------------------------------------------------------
# Build compressor module
# ------------------------------------------------------------------------------
include_directories(include)
add_subdirectory(src)
add_subdirectory(test)  # ADD ME

# -----------------------------------------------------------------------------
# Install compressor headers
# -----------------------------------------------------------------------------
install(DIRECTORY include DESTINATION ${CMAKE_INSTALL_PREFIX})

Build and Install

scspkg create compressor
cd ~/my_mod_repo
mkdir build
cd build
cmake .. -DCMAKE_INSTALL_PREFIX=$(scspkg pkg root compressor)
make -j32 install

Create a Jarvis Repo

The following will create a repo named jarvis_example.

mkdir -p ~/my_mod_repo/jarvis_example/jarvis_example
mkdir -p ~/my_mod_repo/jarvis_example/pipelines
jarvis repo add ~/my_mod_repo/jarvis_example

Create a Jarvis Pkg

The following will bootstrap a compressor package in the most recently added repo (jarvis_example).

jarvis repo create compressor app

You will then have the following structure:

.
├── chimaera_repo.yaml
├── CMakeLists.txt
├── compressor
│   ├── chimaera_mod.yaml
│   ├── CMakeLists.txt
│   ├── include
│   │   └── compressor
│   │       ├── compressor_client.h
│   │       ├── compressor_client.temp_h
│   │       ├── compressor_lib_exec.h
│   │       ├── compressor_methods.compiled.yaml
│   │       ├── compressor_methods.h
│   │       ├── compressor_methods.yaml
│   │       ├── compressor_tasks.h
│   │       └── compressor_tasks.temp_h
│   ├── src
│   │   ├── CMakeLists.txt
│   │   ├── CMakeLists.txt.backup
│   │   ├── compressor_client.cc
│   │   ├── compressor_monitor.py
│   │   ├── compressor_runtime.cc
│   │   └── compressor_runtime.temp_cc
│   └── test
│       ├── CMakeLists.txt
│       └── test.cc
└── jarvis_example
    ├── jarvis_example
    │   └── compressor
    │       └── pkg.py
    └── pipelines

Build the Jarvis Pkg

Begin editing ~/my_mod_repo/jarvis_example/jarvis_example/compressor/pkg.py

The unedited code looks like this:

"""
This module provides classes and methods to launch the Compressor application.
Compressor is ....
"""
from jarvis_cd.basic.pkg import Application
from jarvis_util import *


class Compressor(Application):
    """
    This class provides methods to launch the Compressor application.
    """
    def _init(self):
        """
        Initialize paths
        """
        pass

    def _configure_menu(self):
        """
        Create a CLI menu for the configurator method.
        For thorough documentation of these parameters, view:
        https://github.com/scs-lab/jarvis-util/wiki/3.-Argument-Parsing

        :return: List(dict)
        """
        return [
            {
                'name': None,  # The name of the parameter
                'msg': '',  # Describe this parameter
                'type': str,  # What is the parameter type?
                'default': None,  # What is the default value if not required?
                # Does this parameter have specific valid inputs?
                'choices': [],
                # When type is list, what do the entries of the list mean?
                # A list of dicts just like this one.
                'args': [],
            },
        ]

    def _configure(self, **kwargs):
        """
        Converts the Jarvis configuration to application-specific configuration.
        E.g., OrangeFS produces an orangefs.xml file.

        :param kwargs: Configuration parameters for this pkg.
        :return: None
        """
        pass

    def start(self):
        """
        Launch an application. E.g., OrangeFS will launch the servers, clients,
        and metadata services on all necessary pkgs.

        :return: None
        """
        pass

    def stop(self):
        """
        Stop a running application. E.g., OrangeFS will terminate the servers,
        clients, and metadata services.

        :return: None
        """
        pass

    def kill(self):
        """
        Forcibly a running application. E.g., OrangeFS will terminate the servers,
        clients, and metadata services.

        :return: None
        """
        pass

    def clean(self):
        """
        Destroy all data for an application. E.g., OrangeFS will delete all
        metadata and data directories in addition to the orangefs.xml file.

        :return: None
        """
        pass

Edit configure menu

    def _configure_menu(self):
        """
        Create a CLI menu for the configurator method.
        For thorough documentation of these parameters, view:
        https://github.com/scs-lab/jarvis-util/wiki/3.-Argument-Parsing

        :return: List(dict)
        """
        return []

Our package will take no arguments.

Edit the start function

    def start(self):
        """
        Launch an application. E.g., OrangeFS will launch the servers, clients,
        and metadata services on all necessary pkgs.

        :return: None
        """
        Exec('compress_test',
             LocalExecInfo(env=self.env))

• Exec will run the command compress_test.
• LocalExecInfo indicates that this will be executed on this machine.
• env=self.env ensures that the pipeline's environment is used for this command. The environment is stored in a YAML file in the pipeline's config directory and is built using commands like jarvis ppl env build and jarvis ppl env copy.

More documentation on Jarvis PKGs

This guide goes over in more detail each jarvis pkg method.

This guide goes over more ways to execute distributed programs with tools like PSSH and MPI.

This guide goes over more about the argument parser.

Build a Pipeline Script

Next we must build a pipeline for the tool. Below is an example pipeline script that you should place in

~/my_mod_repo/compressor/jarvis_example/pipelines/test.yaml

name: test_pipeline
env: chimaera
pkgs:
  - pkg_type: chimaera_run
    pkg_name: chimaera_run
    sleep: 5
    do_dbg: false
    dbg_port: 4000
    port: 6000
    modules: ['example_compressor']
  - pkg_type: compressor
    pkg_name: compressor

The directory ~/my_mod_repo/compressor/jarvis_example/pipelines is known as a pipeline index. It is simply a place where jarvis knows pipelines are located. This means you don't have to remember the full path to the script.

• env: chimaera: This will copy the named environment named chimaera. We will create this environment in the next section.
• modules: The modules that the iowarp runtime (Chimaera) searches for at launch. By using example_compressor here, the runtime will search for various shared objects, such as libexample_compressor.so in LD_LIBRARY_PATH.
• pkg_type: compressor: This corresponds to the jarvis repo create compressor command.
• pkg_name: compressor: This can be whatever string you want. However, I typically just keep it the same as pkg_type.

Build pipeline environment

spack load iowarp
module load compressor
jarvis env build chimaera

Load + Run the Pipeline

Here is how we would run the pipeline

jarvis ppl index copy jarvis_example.test
jarvis ppl load yaml test.yaml
jarvis ppl run