camber.mpi
Camber has a built-in engine to run workloads for OpenMPI, a library for high-performance, parallel computing.
import camber.mpi
You can access the MPI engine through camber.mpi
with the following methods
Methods
create_job
Creates a CamberJob using the MPI engine.
Args
command
: str- Base command in string form
- e.g.
mpirun -np 2 athena/bin/athena -i athinput.orszag-tang
engine_size
: Optional[str]- Size of engine, one of
XMICRO
,MICRO
,XXSMALL
,XSMALL
,SMALL
,MEDIUM
,LARGE
. - Default is
XSMALL
. extra_env_vars
: Optional[Dict[str, str]]- Extra environment variables to give the engine.
- e.g.
{"MY_ENV_VAR":"my_value"}
tags
: Optional[Union[str, List]]- Tags to help identify the job.
- e.g.
["tag1", "tag2"]
or"tag1"
.
Returns
CamberJob
- The
CamberJob
object representing the created job.
create_scatter_job
Creates a list of jobs by scattering the parameters grid.
The params_grid
must be a dictionary of lists.
The keys of the dictionary are the parameter names and the values are the list of possible
values for that parameter.
The Cartesian product of all values from all parameters is used to
create individual jobs.
The limit of jobs that can be created is 10.
The engine_size
, job_type
and tags hold constant for all jobs created.
The command template should have the ability to format the parameters using the python string formatting syntax.
For example, this scatter job runs every combination of name, age, and favorite animal:
params={
"name":["John","Sally"],
"age":[21,22,23],
"animal": ["llama","tortoise"]
}
mpi.create_scatter_job(
engine_size="XSMALL",
command_template="echo \
{name} said: \"When I was {age}, my favorite animal was the {animal}.",
template_params_grid=params
)
# Job1 runs "echo John said: When I was 21, my favorite animal was the llama."
# Job2 runs "echo Sally said: When I was 21, my favorite animal was the llama."
# ...
# Job12 runs "echo Sally said: When I was 23, my favorite animal was the tortoise."
Args
template_params_grid
: Dict[str, List]- A dictionary of parameters to scatter.
command_template
: str- The base command in string form
engine_size
: Optional[str]- Size of engine. One of:
XMICRO
,MICRO
,XXSMALL
,XSMALL
,SMALL
,MEDIUM
,LARGE
- Default is
XSMALL
extra_env_vars
: Optional[Dict[str, str]]- Extra environment variables to give the engine.
- e.g.
{"MY_ENV_VAR":"my_value"}
tags
: Optional[Union[str, List]]- Tags to help identify the job.
- e.g.
["tag1", "tag2"]
or"tag1"
.
Returns
List[CamberJob]
- The list of
CamberJob
objects representing the created jobs.
delete_job
Deletes an MPI based CamberJob from your account. Supply the id of the CamberJob.
Args
job_id
: Union[int, str]- The id of the
CamberJob
object to delete - e.g.
1234
get_job
Retrieves an MPI based CamberJob from CamberCloud. Supply the id of the CamberJob.
Args
job_id
: Union[int, str]- The id of the CamberJob object to retrieve
- e.g.
1234
Returns
CamberJob
- The retrieved
CamberJob
object.
list_jobs
Lists all MPI based CamberJobs in CamberCloud. Optionally supply the tags to filter the jobs.
Args
tags
: Optional[Union[str, List]]- Tags to help identify the job.
- e.g.
["tag1", "tag2"]
or"tag1"
.
Returns
List[CamberJob]
- The list of
CamberJob
objects.