HCL cluster - Revision history

Davepc at 13:53, 9 September 2013

2013-09-09T13:53:24Z

Davepc at 13:28, 9 September 2013

2013-09-09T13:28:22Z

Davepc: /* Software packages available on HCL Cluster 2.0 */

2012-07-11T23:13:03Z

‎Software packages available on HCL Cluster 2.0

Davepc: /* Cluster Administration */

2011-08-04T12:35:02Z

‎Cluster Administration

Rhiggins: /* Useful Tools */

2010-12-01T15:32:23Z

‎Useful Tools

Rhiggins: /* Useful Tools */

2010-11-15T19:07:18Z

‎Useful Tools

Davepc: /* Creating new user accounts */

2010-10-18T18:01:52Z

‎Creating new user accounts

Davepc: /* Access and Security */

2010-10-18T18:01:28Z

‎Access and Security

Davepc: /* Software packages available on HCL Cluster 2.0 */

2010-10-15T14:14:44Z

‎Software packages available on HCL Cluster 2.0

Davepc: /* Software packages available on HCL Cluster 2.0 */

2010-10-15T14:10:55Z

‎Software packages available on HCL Cluster 2.0

@@ Line 175: / Line 175: @@
 == Paging and the OOM-Killer ==
-Please read [[virtual memory overcommit]] for details.
+Please read [[Virtual Memory Overcommit]] page for details. For reasons given overcommit has been disabled on the cluster.
   cat /proc/sys/vm/overcommit_memory
@@ Line 191: / Line 187: @@
 == Manually Limit the Memory on the OS level ==
+Memory can be manually limited in the grub. This should be reset when you are finished.
-as root edit /etc/default/grub
+If you are doing memory exaustive experiments test check this has not being adjusted by someone else.
- GRUB_CMDLINE_LINUX_DEFAULT="quiet mem=128M"
+See [[Memory size, overcommit, limit]] for more detail.

← Older revision		Revision as of 13:28, 9 September 2013
Line 175:		Line 175:

	== Paging and the OOM-Killer ==		== Paging and the OOM-Killer ==
		+	Please read [[virtual memory overcommit]] for details.
	Due to the nature of experiments run on the cluster, we often induce heavy paging and complete exhaustion of available memory on certain nodes. Linux has a pair of strategies to deal with heavy memory use. First, is overcommitting. This is where a process is allowed allocate or fork even when there is no more memory available. You can seem some interesting numbers here:[http://opsmonkey.blogspot.com/2007/01/linux-memory-overcommit.html]. The assumption is that processes may not use all memory that they allocate and failing on allocation is worse than failing at a later date when the memory use is actually required. More processes may be supported by allowing them to allocate memory (provided they do not use it all). The second part of the strategy is the Out-of-Memory killer (OOM Killer). When memory has been exhausted and a process tries to use some 'overcommitted' part of memory, the OOM killer is invoked. It's job is to rank all processes in terms of their memory use, priority, privilege and some other parameters, and then select a process to kill based on the ranks.		Due to the nature of experiments run on the cluster, we often induce heavy paging and complete exhaustion of available memory on certain nodes. Linux has a pair of strategies to deal with heavy memory use. First, is overcommitting. This is where a process is allowed allocate or fork even when there is no more memory available. You can seem some interesting numbers here:[http://opsmonkey.blogspot.com/2007/01/linux-memory-overcommit.html]. The assumption is that processes may not use all memory that they allocate and failing on allocation is worse than failing at a later date when the memory use is actually required. More processes may be supported by allowing them to allocate memory (provided they do not use it all). The second part of the strategy is the Out-of-Memory killer (OOM Killer). When memory has been exhausted and a process tries to use some 'overcommitted' part of memory, the OOM killer is invoked. It's job is to rank all processes in terms of their memory use, priority, privilege and some other parameters, and then select a process to kill based on the ranks.

@@ Line 74: / Line 74: @@
 * libboost-graph-dev
 * libboost-serialization-dev
 * r-cran-strucchange
 * graphviz

@@ Line 28: / Line 28: @@
 == Cluster Administration ==
 ===Useful Tools===
 <code>root</code> on <code>heterogeneous.ucd.ie</code> has a number of [http://expect.nist.gov/ Expect] scripts to automate administration on the cluster (in <code>/root/scripts</code>). <code>root_ssh</code> will automatically log into a host, provide the root password and either return a shell to the user or execute a command that is passed as a second argument. Command syntax is as follows:

@@ Line 40: / Line 40: @@
 The above is sequential. To run parallel jobs, for example: <code>apt-get update && apt-get -y upgrade</code>, try the following trick with [http://www.gnu.org/software/screen/ screen]:
-  # for i in `seq -w 1 16`; do screen -L -d -m root_ssh hcl$i apt-get update \&\& apt-get -y upgrade'; done
+  # for i in `seq -w 1 16`; do screen -L -d -m root_ssh hcl$i apt-get update \&\& apt-get -y upgrade; done
 You can check the screenlog.* files for errors and delete them when you are happy. Sometimes all logs are sent to screenlog.0, not sure why.

@@ Line 37: / Line 37: @@
 Example usage, to login and execute a command on each node in the cluster (note the file <code>/etc/dsh/machines.list</code> contains the hostnames of all compute nodes of the cluster):
-  # for i in `cat /etc/dsh/machines.list`; do root_ssh $i ps ax \| grep pbs; done
+  # for i in `seq -w 1 16`; do root_ssh hcl$i ps ax \| grep pbs; done
 The above is sequential. To run parallel jobs, for example: <code>apt-get update && apt-get -y upgrade</code>, try the following trick with [http://www.gnu.org/software/screen/ screen]:
-  # for i in `cat /etc/dsh/machines.list`; do screen -L -d -m root_ssh $i apt-get update \&\& apt-get -y upgrade'; done
+  # for i in `seq -w 1 16`; do screen -L -d -m root_ssh hcl$i apt-get update \&\& apt-get -y upgrade'; done
 You can check the screenlog.* files for errors and delete them when you are happy. Sometimes all logs are sent to screenlog.0, not sure why.

@@ Line 92: / Line 92: @@
 === Creating new user accounts ===
   adduser <username>
   make -C /var/yp

← Older revision		Revision as of 18:01, 18 October 2010
Line 90:		Line 90:

	Access from outside the UCD network is only allowed once you have gained entry to a server that allows outside connections (such as csserver.ucd.ie)		Access from outside the UCD network is only allowed once you have gained entry to a server that allows outside connections (such as csserver.ucd.ie)
		+
		+	=== Creating new user accounts ===
		+	adduser <username>
		+	make -C /var/yp

	=== Access to the nodes is controlled by Torque PBS.===		=== Access to the nodes is controlled by Torque PBS.===

@@ Line 75: / Line 75: @@
 * colorgcc
-[[HCL_cluster/heterogeneous.ucd.ie_install_log|new hcl node install & configuration log]]
+[[HCL_cluster/hcl_node_install_configuration_log|new hcl node install & configuration log]]
-[[/new heterogeneous.ucd.ie install log]]
+[[HCL_cluster/heterogeneous.ucd.ie_install_log|new heterogeneous.ucd.ie install log]]
 ===APT===