High Performance Computing
FLOW-3D users have two options for speeding up their runtimes via parallel computing: shared memory and distributed memory (clusters). Contact us for more information about parallel computing.
Shared-Memory Computers
FLOW-3D users with shared-memory processor (SMP) workstations can take advantage of the implementation of OpenMP protocols which enable the use of two or more SMP processors to run a single calculation.
Advantages of SMP Version
Unlike the distributed memory version, where a mesh block must be assigned for each cpu, no changes need to be made to a simulation input file to run in SMP. All that is required to run a multiprocessor simulation with the SMP parallel version is the appropriate license file.
Another important advantage lies in the rapidly declining cost of dual-processor PCs such AMD's Opteron® and Intel's Xeon®. Add 64-bit capability, and the attractiveness of OpenMP, especially for certain types of simulations, is clear.
Limitations of SMP Version
OpenMP scaling is limited by both hardware and software. Because of the limitations of OpenMP, some of FLOW-3D’s modeling algorithms could not be parallelized. More important, the maximum memory of a single computer puts a cap on the size of flow problems.
Due to the nature of the SMP approach, scaling tends to drop off after three or four processors. In fact, beyond eight processors, simulations will actually take progressively longer to run.
Additionally, since shared memory regions must be created by FLOW-3D to allow inter-thread data exchange, the memory required by the SMP approach increases as more processes are added. Finally, since the arrays in the SMP approach are contiguous, the larger the arrays become, the more inefficient the CPU cache becomes, and therefore, solutions to large problems (> 8 million cells) become inefficient.
FLOW-3D/MP for Linux clusters
FLOW-3D users with access to Linux clusters can take advantage of the scaling potential of distributed memory computing using an MPI version of FLOW-3D. FLOW-3D/MP 3.0 is based on version 9.2.1 of FLOW-3D, having all of the powerful features of that release. FLOW-3D/MP 3.0 is available for both 32-bit and 64-bit clusters.
Advantages of Cluster Simulations
FLOW-3D/MP is a new Linux cluster processing version of FLOW-3D. In benchmark tests it achieves an average speed-up of 5-6 times on an 8-processor cluster. This capability allows for larger simulations to be run and results to be obtained much more quickly.
Limitations of FLOW-3D/MP
Scaling with the MPI version is not dependent, as it is with the SMP version, on the models being run. Nevertheless, some problems will scale better than others. For example, smaller problems will not scale well. Moreover, filling simulations will not scale as well as those where the entire domain is in play at all times during the simulation.
Requirements
The SMP parallel version is available for Windows XP & Vista (32-bit and 64-bit) and Linux RedHat Enterprise 3.0 and above (32-bit and 64-bit). To run simulations using the SMP parallel version, all that is required is a multiprocessor computer and the appropriate license from Flow Science. The current SMP version of FLOW-3D is Version 9.2.
To run FLOW-3D/MP, users must have a cluster of computers (which may consist of a group of similarly configured x86-32 or x86-64 processors) running Redhat Enterprise 3 or higher, Suse Enterprise 9 or higher, or Windows CCS 2003, a high speed network interconnect such as Gigabit Ethernet or Infiniband, and a large shared NFS disk accessible from all nodes in cluster. Required software includes FLOW-3D/MP and MPICH2 or Intel-MPI.
