Difference between revisions of "Grid5000"
From HCL
(→Compilation and running jobs) |
|||
| Line 1: | Line 1: | ||
| − | https://www.grid5000.fr/mediawiki/index.php/Grid5000:Home | + | https://www.grid5000.fr/mediawiki/index.php/Grid5000:Home |
| + | |||
| + | == Login, job submission, deployment of image == | ||
| + | |||
| + | *Select sites and clusters for experiments, using information on the [https://www.grid5000.fr/mediawiki/index.php/Grid5000:Network#Grid.275000_Sites Grid5000 network] and the [https://www.grid5000.fr/mediawiki/index.php/Status Status page] | ||
| + | *Access is provided via access nodes '''access.SITE.grid5000.fr''' marked [https://www.grid5000.fr/mediawiki/index.php/External_access here] as ''accessible from '''everywhere''' via ssh with '''keyboard-interactive''' authentication method''. As soon as you are on one of the sites, you can directly ssh frontend node of any other site: | ||
| − | |||
| − | |||
| − | |||
<source lang="bash"> | <source lang="bash"> | ||
access_$ ssh frontend.SITE2 | access_$ ssh frontend.SITE2 | ||
| − | </source> | + | </source> |
| − | * There is no access to Internet from computing nodes (external IPs should be registered on proxy), therefore, download/update your stuff at the access nodes. Several revision control clients are available. | + | |
| − | * Each site has a separate NFS, therefore, to run an application on several sites at once, you need to copy it '''scp, sftp, rsync''' between access or frontend nodes. | + | *There is no access to Internet from computing nodes (external IPs should be registered on proxy), therefore, download/update your stuff at the access nodes. Several revision control clients are available. |
| − | * Jobs are run from the frondend nodes, using a [http://en.wikipedia.org/wiki/OpenPBS PBS]-like system [https://www.grid5000.fr/mediawiki/index.php/Cluster_experiment-OAR2 OAR]. Basic commands: | + | *Each site has a separate NFS, therefore, to run an application on several sites at once, you need to copy it '''scp, sftp, rsync''' between access or frontend nodes. |
| − | ** '''oarstat''' - queue status | + | *Jobs are run from the frondend nodes, using a [http://en.wikipedia.org/wiki/OpenPBS PBS]-like system [https://www.grid5000.fr/mediawiki/index.php/Cluster_experiment-OAR2 OAR]. Basic commands: |
| − | ** '''oarsub''' - job submission | + | **'''oarstat''' - queue status |
| − | ** '''oardel''' - job removal | + | **'''oarsub''' - job submission |
| − | Interactive job on deployed images: | + | **'''oardel''' - job removal |
| − | <source lang="bash"> | + | |
| + | Interactive job on deployed images: <source lang="bash"> | ||
fontend_$ oarsub -I -t deploy -l [/cluster=N/]nodes=N,walltime=HH[:MM[:SS]] [-p 'PROPERTY="VALUE"'] | fontend_$ oarsub -I -t deploy -l [/cluster=N/]nodes=N,walltime=HH[:MM[:SS]] [-p 'PROPERTY="VALUE"'] | ||
| − | </source> | + | </source> Batch job on installed images: <source lang="bash"> |
| − | Batch job on installed images: | ||
| − | <source lang="bash"> | ||
fontend_$ oarsub BATCH_FILE -t allow_classic_ssh -l [/cluster=N/]nodes=N,walltime=HH[:MM[:SS]] [-p 'PROPERTY="VALUE"'] | fontend_$ oarsub BATCH_FILE -t allow_classic_ssh -l [/cluster=N/]nodes=N,walltime=HH[:MM[:SS]] [-p 'PROPERTY="VALUE"'] | ||
| − | </source> | + | </source> |
| − | * The image to deploy can be created and loaded with help of a [http://wiki.systemimager.org/index.php/Main_Page Systemimager]-like system [https://www.grid5000.fr/mediawiki/index.php/Deploy_environment-OAR2 Kadeploy]. Creating: [https://www.grid5000.fr/mediawiki/index.php/Deploy_environment-OAR2#Tune_an_environment_to_build_another_one:_customize_authentification_parameters described here] | + | |
| − | Loading: | + | *The image to deploy can be created and loaded with help of a [http://wiki.systemimager.org/index.php/Main_Page Systemimager]-like system [https://www.grid5000.fr/mediawiki/index.php/Deploy_environment-OAR2 Kadeploy]. Creating: [https://www.grid5000.fr/mediawiki/index.php/Deploy_environment-OAR2#Tune_an_environment_to_build_another_one:_customize_authentification_parameters described here] |
| − | <source lang="bash"> | + | |
| + | Loading: <source lang="bash"> | ||
fontend_$ kadeploy3 -a PATH_TO_PRIVATE_IMAGE_DESC -f $OAR_FILE_NODES | fontend_$ kadeploy3 -a PATH_TO_PRIVATE_IMAGE_DESC -f $OAR_FILE_NODES | ||
| − | </source> | + | </source> A Linux distribution lenny-x64-nfs-2.1 with mc, subversion, autotools, doxygen, MPICH2, GSL, Boost, R, gnuplot, graphviz, X11, evince is available at Orsay /home/nancy/alastovetsky/grid5000. |
| − | A Linux distribution lenny-x64-nfs-2.1 with mc, subversion, autotools, doxygen, MPICH2, GSL, Boost, R, gnuplot, graphviz, X11, evince is available at Orsay /home/nancy/alastovetsky/grid5000. | + | |
| + | == Compiling and running MPI applications == | ||
| + | |||
| + | *Compilation should be done on one of the reserved nodes (e.g. ssh `head -n 1 $OAR_NODEFILE`) | ||
| + | *Running MPI applications is described [https://www.grid5000.fr/mediawiki/index.php/Run_MPI_On_Grid%275000 here] | ||
| + | **mpirun/mpiexec should be run from one of the reserved nodes (e.g. ssh `head -n 1 $OAR_NODEFILE`) | ||
| + | |||
| + | Setting up new deploy image | ||
| + | |||
| + | ... launch image ... | ||
| + | |||
| + | <source> | ||
| + | deb http://ftp.fr.debian.org/debian/ sid main contrib non-free | ||
| − | + | <source> | |
| − | |||
| − | |||
| − | |||
Revision as of 10:35, 1 April 2011
https://www.grid5000.fr/mediawiki/index.php/Grid5000:Home
Login, job submission, deployment of image
- Select sites and clusters for experiments, using information on the Grid5000 network and the Status page
- Access is provided via access nodes access.SITE.grid5000.fr marked here as accessible from everywhere via ssh with keyboard-interactive authentication method. As soon as you are on one of the sites, you can directly ssh frontend node of any other site:
access_$ ssh frontend.SITE2
- There is no access to Internet from computing nodes (external IPs should be registered on proxy), therefore, download/update your stuff at the access nodes. Several revision control clients are available.
- Each site has a separate NFS, therefore, to run an application on several sites at once, you need to copy it scp, sftp, rsync between access or frontend nodes.
- Jobs are run from the frondend nodes, using a PBS-like system OAR. Basic commands:
- oarstat - queue status
- oarsub - job submission
- oardel - job removal
fontend_$ oarsub -I -t deploy -l [/cluster=N/]nodes=N,walltime=HH[:MM[:SS]] [-p 'PROPERTY="VALUE"']
fontend_$ oarsub BATCH_FILE -t allow_classic_ssh -l [/cluster=N/]nodes=N,walltime=HH[:MM[:SS]] [-p 'PROPERTY="VALUE"']
- The image to deploy can be created and loaded with help of a Systemimager-like system Kadeploy. Creating: described here
fontend_$ kadeploy3 -a PATH_TO_PRIVATE_IMAGE_DESC -f $OAR_FILE_NODES
Compiling and running MPI applications
- Compilation should be done on one of the reserved nodes (e.g. ssh `head -n 1 $OAR_NODEFILE`)
- Running MPI applications is described here
- mpirun/mpiexec should be run from one of the reserved nodes (e.g. ssh `head -n 1 $OAR_NODEFILE`)
Setting up new deploy image
... launch image ...
<source> deb http://ftp.fr.debian.org/debian/ sid main contrib non-free
<source>
