camber.mpi

Camber has a built-in engine to run workloads for OpenMPI, a library for high-performance, parallel computing.

import camber.mpi

You can access the MPI engine through camber.mpi with the following methods

Methods

`create_job`

Creates a CamberJob using the MPI engine.

Args

command: str: Base command in string form; e.g. mpirun -np 2 athena/bin/athena -i athinput.orszag-tang
engine_size: Optional[str]: Size of engine, one of XMICRO, MICRO, XXSMALL, XSMALL, SMALL, MEDIUM, LARGE.; Default is XSMALL.
extra_env_vars: Optional[Dict[str, str]]: Extra environment variables to give the engine.; e.g. {"MY_ENV_VAR":"my_value"}
tags: Optional[Union[str, List]]: Tags to help identify the job.; e.g. ["tag1", "tag2"] or "tag1".

Returns

CamberJob: The CamberJob object representing the created job.

`create_scatter_job`

Creates a list of jobs by scattering the parameters grid. The params_grid must be a dictionary of lists. The keys of the dictionary are the parameter names and the values are the list of possible values for that parameter. The Cartesian product of all values from all parameters is used to create individual jobs. The limit of jobs that can be created is 10.

The engine_size, job_type and tags hold constant for all jobs created. The command template should have the ability to format the parameters using the python string formatting syntax. For example, this scatter job runs every combination of name, age, and favorite animal:

params={
  "name":["John","Sally"],
   "age":[21,22,23],
   "animal": ["llama","tortoise"]
}

mpi.create_scatter_job(
    engine_size="XSMALL",
    command_template="echo \
      {name} said: \"When I was {age}, my favorite animal was the {animal}.",
    template_params_grid=params

)

# Job1 runs "echo John said: When I was 21, my favorite animal was the llama."
# Job2 runs "echo Sally said: When I was 21, my favorite animal was the llama."
# ...
# Job12 runs "echo Sally said: When I was 23, my favorite animal was the tortoise."

Args

template_params_grid: Dict[str, List]: A dictionary of parameters to scatter.
command_template: str: The base command in string form
engine_size: Optional[str]: Size of engine. One of: XMICRO, MICRO, XXSMALL, XSMALL, SMALL, MEDIUM, LARGE; Default is XSMALL
extra_env_vars: Optional[Dict[str, str]]: Extra environment variables to give the engine.; e.g. {"MY_ENV_VAR":"my_value"}
tags: Optional[Union[str, List]]: Tags to help identify the job.; e.g. ["tag1", "tag2"] or "tag1".

Returns

List[CamberJob]: The list of CamberJob objects representing the created jobs.

`delete_job`

Deletes an MPI based CamberJob from your account. Supply the id of the CamberJob.

Args

job_id: Union[int, str]: The id of the CamberJob object to delete; e.g. 1234

`get_job`

Retrieves an MPI based CamberJob from CamberCloud. Supply the id of the CamberJob.

Args

job_id: Union[int, str]: The id of the CamberJob object to retrieve; e.g. 1234

Returns

CamberJob: The retrieved CamberJob object.

`list_jobs`

Lists all MPI based CamberJobs in CamberCloud. Optionally supply the tags to filter the jobs.

Args

tags: Optional[Union[str, List]]: Tags to help identify the job.; e.g. ["tag1", "tag2"] or "tag1".

Returns

List[CamberJob]: The list of CamberJob objects.

camber.mesa camber.spark