HCL - User contributions [en]

UTK multicores + GPU

2012-08-10T11:36:24Z

Quintin:

== List of machines ==
http://icl.cs.utk.edu/iclhelp/custom/index.html?lid=97&slid=180

== Display a list of available GPUs ==
$ nvidia-smi -L

== Using Fupermod on hybrid multicore/GPUs node ==
* Compiling : Create two separate directories for configuration with selected CPU cblas (e.g. gsl, acml, mkl) and GPU cblas (e.g. cublas).

- For example: configuring with [http://developer.amd.com/libraries/acml/pages/default.aspx acml] for CPU and [http://developer.nvidia.com/cublas cublas] for GPU

$ cd fupermod/

$ mkdir acml_config
$ cd acml_config
$ ../configure --with-cblas=acml
$ make

$ mkdir cuda_config
$ cd cuda_config
$ ../configure --with-cblas=cuda
$ make

* Building performance model:

- Rankfile is for [http://www.open-mpi.org/doc/v1.6/man1/mpirun.1.php#toc8 process binding] (TODO correction this is a dead link), and appfile tells mpirun what programs to launch
$ mpirun -rf rankfile -app appfile_fpm

- Example of a rankfile:
rank 0=ig.icl.utk.edu slot=0:0
rank 1=ig.icl.utk.edu slot=0:1
...

- Example of an appfile for building functional permanence model (appfile_fpm):
# GPU
# e.g. Linking against cublas, and fupermod is configured under cublas_config
# suboption g=0 means device 0 is selected for computing
-host localhost -np 1 $HOME/fupermod/cublas_config/tools/builder -l $HOME/fupermod/cublas_config/routines/mxm/.libs/libmxm_col.so -o k=640,g=0 -U10000 -s10
# CPU
# e.g. Linking against acml, and fupermod is configured under acml_config
-host localhost -np 47 $HOME/fupermod/acml_config/tools/builder -l $HOME/fupermod/acml_config/routines/mxm/.libs/libmxm_col.so -o k=640 -U10000 -s10

- TODO this library doesn't exist anymore which one has to be used?
-- when we execute it's complaining about the file ./nodetype.hostname.dev what is it?
-- I got the error rank_intra >= nodesize, 1 >= 0 could you tell me what is wrong?

* Data partitioning

- Matrix size D = N x N, and machinefile lists the nodes participating in the computing

$ fupermod/tools/partitioner -l fupermod/routines/mxm/.libs/libmxm_col.so -D10000 -o N=100 -m machinefile

* Running matrix multiplication

$ mpirun -rf rankfile -app appfile_mxm

- Example of an appfile for matrix multiplication (appfile_mxm)
# GPU
# Assuming fupermod is configured under cublas_config, linking against cublas
# -g0 means device 0 is selected for computing
-host localhost -np 1 $HOME/fupermod/cublas_config/routines/mxm/mxm_col -k640 -g0 -m machinefile
# CPU
# Assuming fupermod is configured under acml_config, linking against acml
-host localhost -np 47 $HOME/fupermod/acml_config/routines/mxm/mxm_col -k640 -m machinefile

Talk:UTK multicores + GPU

2012-08-10T11:29:34Z

Quintin: Created page with "Is it possible to get the same thing with mpich?"

Is it possible to get the same thing with mpich?

UTK multicores + GPU

2012-08-10T11:29:11Z

Quintin:

== List of machines ==
http://icl.cs.utk.edu/iclhelp/custom/index.html?lid=97&slid=180

== Display a list of available GPUs ==
$ nvidia-smi -L

== Using Fupermod on hybrid multicore/GPUs node ==
* Compiling : Create two separate directories for configuration with selected CPU cblas (e.g. gsl, acml, mkl) and GPU cblas (e.g. cublas).

- For example: configuring with [http://developer.amd.com/libraries/acml/pages/default.aspx acml] for CPU and [http://developer.nvidia.com/cublas cublas] for GPU

$ cd fupermod/

$ mkdir acml_config
$ cd acml_config
$ ../configure --with-cblas=acml
$ make

$ mkdir cuda_config
$ cd cuda_config
$ ../configure --with-cblas=cuda
$ make

* Building performance model:

- Rankfile is for [http://www.open-mpi.org/doc/v1.6/man1/mpirun.1.php#toc8 process binding] (TODO correction this is a dead link), and appfile tells mpirun what programs to launch
$ mpirun -rf rankfile -app appfile_fpm

- Example of a rankfile:
rank 0=ig.icl.utk.edu slot=0:0
rank 1=ig.icl.utk.edu slot=0:1
...

- Example of an appfile for building functional permanence model (appfile_fpm):
# GPU
# e.g. Linking against cublas, and fupermod is configured under cublas_config
# suboption g=0 means device 0 is selected for computing
-host localhost -np 1 $HOME/fupermod/cublas_config/tools/builder -l $HOME/fupermod/cublas_config/routines/mxm/.libs/libmxm_col.so -o k=640,g=0 -U10000 -s10
# CPU
# e.g. Linking against acml, and fupermod is configured under acml_config
-host localhost -np 47 $HOME/fupermod/acml_config/tools/builder -l $HOME/fupermod/acml_config/routines/mxm/.libs/libmxm_col.so -o k=640 -U10000 -s10

* Data partitioning

- Matrix size D = N x N, and machinefile lists the nodes participating in the computing

$ fupermod/tools/partitioner -l fupermod/routines/mxm/.libs/libmxm_col.so -D10000 -o N=100 -m machinefile

* Running matrix multiplication

$ mpirun -rf rankfile -app appfile_mxm

- Example of an appfile for matrix multiplication (appfile_mxm)
# GPU
# Assuming fupermod is configured under cublas_config, linking against cublas
# -g0 means device 0 is selected for computing
-host localhost -np 1 $HOME/fupermod/cublas_config/routines/mxm/mxm_col -k640 -g0 -m machinefile
# CPU
# Assuming fupermod is configured under acml_config, linking against acml
-host localhost -np 47 $HOME/fupermod/acml_config/routines/mxm/mxm_col -k640 -m machinefile

Pgfplot

2012-05-09T11:53:49Z

Quintin:

PgfPlot is a package which let you do the same things as gnuplot but directly inside your latex file.
And this has a lot of advantages:
- no additional file
- no conversion from eps or ps or png to the pdf
- the quality is really improved
- the picture text has the same font as the rest of the document

=Preambule=
<source lang="latex">
\usepackage{tikz}
\usepackage{pgfplots}
</source>

=example=

<source lang="latex">
\begin{tikzpicture}[transform shape,scale=0.9]
\begin{axis}[
ylabel style={yshift=-15pt}, % place of the ylabel
width=\textwidth,height=7.4cm, %size of the picture
xlabel=Number of processors,
ylabel=Execution time ($10^3$ s),
legend style={at={(.9,.9)}}, % place of the legend
ymin=0, ymax=8.8, % range for the y axe
mark size=1.5 % size of the points
]
\addplot+[] table[header=false,x index=0,y expr=\thisrowno{2}/1000]{data};
\addplot+[] table[header=false,x index=0,y expr=\thisrowno{2}/1000]{data};
\addplot+[] table[header=false,x index=0,y expr=\thisrowno{2}/1000]{data};

\legend{WSCOM$_{\text{pf}}$,WSCOM,list\_min};

\end{axis}\end{tikzpicture}

</source>

a part of the file "data" is shown just after:

<source lang="text">
1 list_min 8241.04532500001 0.592721444672621 400
2 list_min 4276.264 0.541485563775196 400
3 list_min 2840.83145 0.240012784271948 400
4 list_min 2171.53825 0.204766932333766 400
5 list_min 1771.9397 0.187049380812677 400
</source>

Pgfplot

2012-05-09T11:49:59Z

Quintin:

PgfPlot is a package which let you do the same things as gnuplot but directly inside your latex file.
And this has a lot of advantages:
- no additional file
- no conversion from eps or ps or png to the pdf
- the quality is really improved
- the picture text has the same font as the rest of the document

=Preambule=
<source lang="latex">
\usepackage{tikz}
\usepackage{pgfplots}
</source>

=example=

<source lang="latex">
\begin{tikzpicture}[transform shape,scale=0.9]
\begin{axis}[ ylabel
style={yshift=-15pt},width=\textwidth,height=7.4cm,
xlabel=Number of processors,
ylabel=Execution time ($10^3$ s),legend
style={at={(.9,.9)}},ymin=0,ymax=8.8,mark size=1.5]
\addplot+[] table[header=false,x index=0,y
expr=\thisrowno{2}/1000]{data};
\addplot+[] table[header=false,x index=0,y
expr=\thisrowno{2}/1000]{data};
\addplot+[] table[header=false,x index=0,yexpr=\thisrowno{2}/1000]{data};
\legend{WSCOM$_{\text{pf}}$,WSCOM,list\_min};

\end{axis}\end{tikzpicture}
</source>

a part of the file "data" is shown just after:

<source lang="text">
1 list_min 8241.04532500001 0.592721444672621 400
2 list_min 4276.264 0.541485563775196 400
3 list_min 2840.83145 0.240012784271948 400
4 list_min 2171.53825 0.204766932333766 400
5 list_min 1771.9397 0.187049380812677 400
</source>

Pgfplot

2012-05-09T11:49:38Z

Quintin:

PgfPlot is a package which let you do the same things as gnuplot but directly inside your latex file.
And this has a lot of advantages:
- no additional file
- no conversion from eps or ps or png to the pdf
- the quality is really improved
- the picture text has the same font as the rest of the document

=Preambule=
<source lang="latex">
\usepackage{tikz}
\usepackage{pgfplots}
</source>

=example=

<source lang="latex">
\begin{tikzpicture}[transform shape,scale=0.9]
\begin{axis}[ ylabel
style={yshift=-15pt},width=\textwidth,height=7.4cm,
xlabel=Number of processors,
ylabel=Execution time ($10^3$ s),legend
style={at={(.9,.9)}},ymin=0,ymax=8.8,mark size=1.5]
%, ytick={1000,2000,3000,4000,5000,6000,7000,8000}]
%,
%xtick={0,5,10,15,20,30,50},ytick={1000,2000,4000,8000}]
\addplot+[] table[header=false,x index=0,y
expr=\thisrowno{2}/1000]{data};
\addplot+[] table[header=false,x index=0,y
expr=\thisrowno{2}/1000]{data};
\addplot+[] table[header=false,x index=0,yexpr=\thisrowno{2}/1000]{data};
\legend{WSCOM$_{\text{pf}}$,WSCOM,list\_min};

\end{axis}\end{tikzpicture}
</source>

a part of the file "data" is shown just after:

<source lang="text">
1 list_min 8241.04532500001 0.592721444672621 400
2 list_min 4276.264 0.541485563775196 400
3 list_min 2840.83145 0.240012784271948 400
4 list_min 2171.53825 0.204766932333766 400
5 list_min 1771.9397 0.187049380812677 400
</source>

Pgfplot

2012-05-09T11:48:46Z

Quintin:

PgfPlot is a package which let you do the same things as gnuplot but directly inside your latex file.
And this has a lot of advantages:
- no additional file
- no conversion from eps or ps or png to the pdf
- the quality is really improved
- the picture text has the same font as the rest of the document

=Preambule=
<source lang="latex">
\usepackage{tikz}
\usepackage{pgfplots}
</source>

=example=

<source lang="latex">
\begin{tikzpicture}[transform shape,scale=0.9]
\begin{axis}[ ylabel
style={yshift=-15pt},width=\textwidth,height=7.4cm,
xlabel=Number of processors,
ylabel=Execution time ($10^3$ s),legend
style={at={(.9,.9)}},ymin=0,ymax=8.8,mark size=1.5]
%, ytick={1000,2000,3000,4000,5000,6000,7000,8000}]
%,
%xtick={0,5,10,15,20,30,50},ytick={1000,2000,4000,8000}]
\addplot+[] table[header=false,x index=0,y
expr=\thisrowno{2}/1000]{data};
\addplot+[] table[header=false,x index=0,y
expr=\thisrowno{2}/1000]{data};
\addplot+[] table[header=false,x index=0,yexpr=\thisrowno{2}/1000]{data};
\legend{WSCOM$_{\text{pf}}$,WSCOM,list\_min};

\end{axis}\end{tikzpicture}
</source>

<source lang="text">
1 list_min 8241.04532500001 0.592721444672621 400
2 list_min 4276.264 0.541485563775196 400
3 list_min 2840.83145 0.240012784271948 400
4 list_min 2171.53825 0.204766932333766 400
5 list_min 1771.9397 0.187049380812677 400
</source>

Pgfplot

2012-05-09T11:48:30Z

Quintin:

PgfPlot is a package which let you do the same things as gnuplot but directly inside your latex file.
And this has a lot of advantages:
- no additional file
- no conversion from eps or ps or png to the pdf
- the quality is really improved
- the picture text has the same font as the rest of the document

=Preambule=
<source lang="latex">
\usepackage{tikz}
\usepackage{pgfplots}
</source>

=example=

<source lang="latex">
\begin{tikzpicture}[transform shape,scale=0.9]
\begin{axis}[ ylabel
style={yshift=-15pt},width=\textwidth,height=7.4cm,
xlabel=Number of processors,
ylabel=Execution time ($10^3$ s),legend
style={at={(.9,.9)}},ymin=0,ymax=8.8,mark size=1.5]
%, ytick={1000,2000,3000,4000,5000,6000,7000,8000}]
%,
%xtick={0,5,10,15,20,30,50},ytick={1000,2000,4000,8000}]
\addplot+[] table[header=false,x index=0,y
expr=\thisrowno{2}/1000]{data};
\addplot+[] table[header=false,x index=0,y
expr=\thisrowno{2}/1000]{data};
\addplot+[] table[header=false,x index=0,yexpr=\thisrowno{2}/1000]{data};
\legend{WSCOM$_{\text{pf}}$,WSCOM,list\_min};

\end{axis}\end{tikzpicture}
</source>

<source lang="plain">
1 list_min 8241.04532500001 0.592721444672621 400
2 list_min 4276.264 0.541485563775196 400
3 list_min 2840.83145 0.240012784271948 400
4 list_min 2171.53825 0.204766932333766 400
5 list_min 1771.9397 0.187049380812677 400
</source>

Pgfplot

2012-05-09T11:48:15Z

Quintin:

PgfPlot is a package which let you do the same things as gnuplot but directly inside your latex file.
And this has a lot of advantages:
- no additional file
- no conversion from eps or ps or png to the pdf
- the quality is really improved
- the picture text has the same font as the rest of the document

=Preambule=
<source lang="latex">
\usepackage{tikz}
\usepackage{pgfplots}
</source>

=example=

<source lang="latex">
\begin{tikzpicture}[transform shape,scale=0.9]
\begin{axis}[ ylabel
style={yshift=-15pt},width=\textwidth,height=7.4cm,
xlabel=Number of processors,
ylabel=Execution time ($10^3$ s),legend
style={at={(.9,.9)}},ymin=0,ymax=8.8,mark size=1.5]
%, ytick={1000,2000,3000,4000,5000,6000,7000,8000}]
%,
%xtick={0,5,10,15,20,30,50},ytick={1000,2000,4000,8000}]
\addplot+[] table[header=false,x index=0,y
expr=\thisrowno{2}/1000]{data};
\addplot+[] table[header=false,x index=0,y
expr=\thisrowno{2}/1000]{data};
\addplot+[] table[header=false,x index=0,yexpr=\thisrowno{2}/1000]{data};
\legend{WSCOM$_{\text{pf}}$,WSCOM,list\_min};

\end{axis}\end{tikzpicture}
</source>

<source lang="data">
1 list_min 8241.04532500001 0.592721444672621 400
2 list_min 4276.264 0.541485563775196 400
3 list_min 2840.83145 0.240012784271948 400
4 list_min 2171.53825 0.204766932333766 400
5 list_min 1771.9397 0.187049380812677 400
</source>

Pgfplot

2012-05-09T11:43:47Z

Quintin:

Pgfplot

2012-05-09T11:43:11Z

Quintin: Created page with "PgfPlot is a package which let you do the same things as gnuplot but directly inside your latex file. And this has a lot of advantages: - no additional file - no conversion fro…"

Boost

2012-04-11T14:13:37Z

Quintin:

http://www.boost.org/

== Installation from sources ==
1. By default, boost is configured with all libraries. To save time on building boost, you can configure it ony with the libraries you need:
<source lang="bash">
$ ./b2 --prefix=DIR --with-graph --with-serialization --with-mpi
</source>
2. Default installation:
- DIR/include/boost_version/boost
- DIR/lib/libboost_library_versions.*
Create symbolic links:
<source lang="bash">
$ cd DIR/include; ln -s boost_version/boost
$ cd DIR/lib; ln -s libboost_[library]_[version].[a/so] libboost_[library].[a/so]
$ export LD_LIBRARY_PATH=DIR/lib:$LD_LIBRARY_PATH
</source>

To install boost with mpi library, you need to add "using mpi ;" in the file "tools/build/v2/user-config.jam"

== Documentation ==
* [http://www.boost.org/doc/libs/1_42_0/libs/graph/doc/table_of_contents.html Graph]
* [http://www.boost.org/doc/libs/1_42_0/libs/serialization/doc/index.html Serialization]

Boost

2012-04-11T11:42:25Z

Quintin:

http://www.boost.org/

== Installation from sources ==
1. By default, boost is configured with all libraries. To save time on building boost, you can configure it ony with the libraries you need:
<source lang="bash">
$ ./b2 --prefix=DIR --with-libraries=graph,serialization
</source>
2. Default installation:
- DIR/include/boost_version/boost
- DIR/lib/libboost_library_versions.*
Create symbolic links:
<source lang="bash">
$ cd DIR/include; ln -s boost_version/boost
$ cd DIR/lib; ln -s libboost_[library]_[version].[a/so] libboost_[library].[a/so]
$ export LD_LIBRARY_PATH=DIR/lib:$LD_LIBRARY_PATH
</source>

To install boost with mpi library, you need to add "using mpi ;" in the file "tools/build/v2/user-config.jam"

== Documentation ==
* [http://www.boost.org/doc/libs/1_42_0/libs/graph/doc/table_of_contents.html Graph]
* [http://www.boost.org/doc/libs/1_42_0/libs/serialization/doc/index.html Serialization]

Boost

2012-04-11T11:41:59Z

Quintin:

http://www.boost.org/

== Installation from sources ==
1. By default, boost is configured with all libraries. To save time on building boost, you can configure it ony with the libraries you need:
<source lang="bash">
$ ./b2 --prefix=DIR --with-libraries=graph,serialization
</source>
To install boost with mpi library, you need to add "using mpi ;" in the file "tools/build/v2/user-config.jam"
2. Default installation:
- DIR/include/boost_version/boost
- DIR/lib/libboost_library_versions.*
Create symbolic links:
<source lang="bash">
$ cd DIR/include; ln -s boost_version/boost
$ cd DIR/lib; ln -s libboost_[library]_[version].[a/so] libboost_[library].[a/so]
$ export LD_LIBRARY_PATH=DIR/lib:$LD_LIBRARY_PATH
</source>

== Documentation ==
* [http://www.boost.org/doc/libs/1_42_0/libs/graph/doc/table_of_contents.html Graph]
* [http://www.boost.org/doc/libs/1_42_0/libs/serialization/doc/index.html Serialization]

PGF/Tikz

2012-01-30T17:43:30Z

Quintin:

=Write a figure=
The preamble of the latex file must contain:
<source lang="latex">\usepackage{tikz}</source>
Some optional libraries could be add like this:
<source lang="latex">\usetikzlibrary{calc}</source>

To start a figure, the code must be inside the tikzpicture environment like this:
<source lang="latex">\begin{tikzpicture} ... TikZ code here... \end{tikzpicture}</source>

=exemple=
<source lang="latex">
% Author: Quintin Jean-Noël
% <http://moais.imag.fr/membres/jean-noel.quintin/>
\documentclass{article}
\usepackage{tikz}
\usetikzlibrary[topaths]
% A counter, since TikZ is not clever enough (yet) to handle
% arbitrary angle systems.
\newcount\mycount
\begin{document}
\begin{tikzpicture}[transform shape]
%the multiplication with floats is not possible. Thus I split the loop
%in two.
\foreach \number in {1,...,8}{
% Computer angle:
\mycount=\number
\advance\mycount by -1
\multiply\mycount by 45
\advance\mycount by 0
\node[draw,circle,inner sep=0.125cm] (N-\number) at (\the\mycount:5.4cm) {};
}
\foreach \number in {9,...,16}{
% Computer angle:
\mycount=\number
\advance\mycount by -1
\multiply\mycount by 45
\advance\mycount by 22.5
\node[draw,circle,inner sep=0.125cm] (N-\number) at (\the\mycount:5.4cm) {};
}
\foreach \number in {1,...,15}{
\mycount=\number
\advance\mycount by 1
\foreach \numbera in {\the\mycount,...,16}{
\path (N-\number) edge[->,bend right=3] (N-\numbera) edge[<-,bend
left=3] (N-\numbera);
}
}
\end{tikzpicture}
\end{document}
</source>

PGF/Tikz

2012-01-30T17:26:54Z

Quintin: Created page with "=PGF/Tikz="

=PGF/Tikz=

Main Page

2012-01-30T17:26:11Z

Quintin:

This site is set up for sharing ideas, findings and experience in heterogeneous computing. Please, log in and create new or edit existing pages. How to format wiki-pages read [[Help:Editing|here]].

== HCL software for heterogeneous computing ==
* Extensions for [[MPI]]: [http://hcl.ucd.ie/project/mpC mpC] [http://hcl.ucd.ie/project/HeteroMPI HeteroMPI] [http://hcl.ucd.ie/project/libELC libELC]
* Extensions for [[GridRPC]]: [http://hcl.ucd.ie/project/SmartGridSolve SmartGridSolve] [http://hcl.ucd.ie/project/NI-Connect NI-Connect]
* Computation benchmarking, modeling, dynamic load balancing: [http://hcl.ucd.ie/project/fupermod FuPerMod] [http://hcl.ucd.ie/project/pmm PMM]
* Communication benchmarking, modeling, optimization: [http://hcl.ucd.ie/project/cpm CPM] [http://hcl.ucd.ie/project/mpiblib MPIBlib]

== Heterogeneous mathematical software ==
* [http://hcl.ucd.ie/project/HeteroScaLAPACK HeteroScaLAPACK]
* [http://hcl.ucd.ie/project/Hydropad Hydropad]

== Operating systems ==
* [[Linux]]
* [[Windows]]

== Development tools ==
* [[C/C++]], [[Python]], [[UML]], [[FORTRAN]]
* [[Autotools]]
* [[GDB]], [[OProfile]], [[Valgrind]]
* [[Doxygen]]
* [[ChangeLog]], [[Subversion]]
* [[Eclipse]]

== [[Libraries]] ==
* [[GNU C Library]]
* [[MPI]]
* [[STL]], [[Boost]]
* [[GSL]]
* [[BLAS LAPACK ScaLAPACK]]
* [[NLOPT]]
* [[BitTorrent (B. Cohen's version)]]
* [[CUDA SDK]]

== Data processing ==
* [[gnuplot]], [[pgfplot]], [[matplotlib]]
* [[Graphviz]]
* [[Octave]], [[R]]
* [[G3DViewer]]

== Paper & Presentation Tools ==
* [[Dia]], [[PGF/Tikz]]
* [[LaTeX]], [[Beamer]]
* [[BibTeX]], [[JabRef]]

== Hardware ==
* [[HCL cluster]]
* [[Other UCD Resources]]
* [[UTK multicores + GPU]]
* [[Grid5000]]

[[SSH|How to connect to cluster via SSH]]

[[hwloc|How to find information about the hardware]]

== Mathematics ==
* [http://en.wikipedia.org/wiki/Confidence_interval Confidence interval (Statistics)], [http://en.wikipedia.org/wiki/Student's_t-distribution Student's t-distribution] (implemented in [[GSL]])
* [http://en.wikipedia.org/wiki/Linear_regression Linear regression] (implemented in [[GSL]])
* [http://en.wikipedia.org/wiki/Binomial_tree#Binomial_tree Binomial tree] (use [[Graphviz]] to visualize trees)
* [http://en.wikipedia.org/wiki/Spline_interpolation Spline interpolation], [http://en.wikipedia.org/wiki/B-spline Spline approximation] (implemented in [[GSL]])