• What's the Cluster-enabled OpenMP?
• Platform
• Install
• How to compile and run your program on your cluster
• Omni OpenMP extensions and performance tuning
  • Performance tuning for Omni/SCASH
  • Data mapping directive
  • Affinity scheduling clause for parallel loops
  • Omni/SCASH shared memory allocator "ompsm_galloc"
• Current status and restriction, known problems and bugs
• Implementation notes

• The SCASH system in SCore Cluster Software 3.3.2 supports SMP clusters. The processors in SMP nodes shares data through hardware shared memory.
• SCore 3.3.2 also supports SCASH over ethernet clusters as well as Myrinet clusters.
• In Omni 1.3, Omni extended directives are available to tune the performance of Omni/SCASH OpenMP programs.

For programming distributed memory multiprocessors such as clusters of PC/WS and MPP, message passing is usually used. A message passing system requires programmers to explicitly code the communication and makes writing parallel programs cumbersome.

OpenMP is attracting wide-spread interests because of its easy-to-use parallel programming model. Our goal is to enable OpenMP to transparently program for distributed memory environment such as a cluster of PC without any modification of a OpenMP source program.

While OpenMP is designed as a programming model for shared memory hardware, one way to support OpenMP in a distributed memory environment is to use a software distributed shared memory system (SDSM) as an underlying runtime system for OpenMP. Cluster-enabled Omni OpenMP on SCASH is an implementation of Omni OpenMP compiler for a software distributed shared memory system SCASH running under SCore Cluster System Software.

• Your platform must be running under the RWCP SCore 3.3.2 (later) which supports SCASH. It is available at http://www.rwcp.or.jp/lab/pdslab.
• The current SCASH system strongly recommands the Myrinet network hardware. In the current SCASH, a single processor in nodes can be used in case of SMP cluster. This problem will be removed soon in the next release.
• The backend compiler of Omni must support ".ctor" sections for linking. We have tested "gcc" in Intel-based Linux clusters. "gcc" collect2 command supports this feature.

• Download RWCP SCore 3.3.2 (and later) Cluster System software, and install them for your cluster.
• Enabling SCASH feature. Add the option '--enable-scash' when configure. For example,

          % ./configure --enable-scash
          % make 
          % make install
  

• The default install process assumes that SCore is installed in the standard directory "/opt/score", in which SCASH library is included. If Score is not installed in the standard directory, you should specifiy the SCore directory as follows:

   
          % ./configure --enable-scash -with-scoreDir=SCore_directory
  

• To compile and link your program, you should use multi-configuration driver ( omcc and omf77 ) with "--omniconfig=scash" option. For example:

          % omcc -omniconfig=scash foo.c
  

• To reserve a partition of your cluster, use "scout" in advance. Then, run your program by "scrun". For example, suppose the host groups "pcc2x" and 4 processors,

          % scout -g pcc2x
          % scrun -nodes=4 a.out
  

  For more detail about SCore environment, see the SCore documents. In case of SCASH, the number of processors is given by "scrun".

To test your installation, "Omni/tests/scash-test" directory contains some sample programs.