High Performance Computing
FLOW-3D users have two options for speeding up their runtimes via parallel computing: shared memory and distributed memory (clusters). For more information, contact us at Flow Science!
Shared-Memory Computers
FLOW-3D users with shared-memory processor (SMP) workstations can take advantage of the implementation of OpenMP protocols which enable the use of two or more SMP processors to run a single calculation.
Advantages of SMP Version
No changes need to be made to a simulation input file to run in SMP. For simulations that have difficulty converging, the new GMRES pressure solver in Version 9.0 can often provide a solution much quicker than SOR or ADI. The GMRES pressure solver is SMP parallelized in Version 9.0 and scales much better than SOR or ADI.
Another important advantage lies in the rapidly declining cost of dual-processor PCs such AMD's Opteron® and Intel's Xeon®. Add 64-bit capability, and the attractiveness of OpenMP, especially for certain types of simulations, is clear.
Finally, as opposed to distributed memory computing, no special efforts are required in order to set up and run the SMP version of FLOW-3D.
Limitations of SMP Version
OpenMP scaling is limited by both hardware and software. Because of the limitations of OpenMP, some of FLOW-3D’s modeling algorithms could not be parallelized. More important, the maximum memory of a single computer puts a cap on the size of flow problems.
Due to the nature of the SMP approach, scaling tends to drop off after three or four processors. In fact, beyond eight processors, simulations will actually take progressively longer to run.
Additionally, since shared memory regions must be created by FLOW-3D to allow inter-thread data exchange, the memory required by the SMP approach increases as more processes are added. Finally, since the arrays in the SMP approach are contiguous, the larger the arrays become, the more inefficient the CPU cache becomes, and therefore, solutions to large problems (> 8 million cells) become inefficient.
FLOW-3D/MP for Linux clusters
FLOW-3D users with access to Linux clusters can take advantage of the scaling potential of distributed memory computing using an MPI version of FLOW-3D. FLOW-3D/MP 3.0 is based on version 9.2.1 of FLOW-3D, having all of the powerful features of that release. FLOW-3D/MP 3.0 is available for both 32-bit and 64-bit clusters.
Advantages of Cluster Simulations
FLOW-3D/MP is a new Linux cluster processing version of FLOW-3D. In benchmark tests it achieves an average speed-up of 5-6 times on an 8-processor cluster. This capability allows for larger simulations to be run and results to be obtained much more quickly.
Limitations of FLOW-3D/MP
Scaling with the MPI version is not dependent, as it is with the SMP version, on the models being run. Nevertheless, some problems will scale better than others. For example, smaller problems will not scale well. Moreover, filling simulations will not scale as well as those where the entire domain is in play at all times during the simulation.
Requirements
The SMP parallel version is available for Windows XP & 2000 (32-bit and 64-bit), Linux RedHat Enterprise 3.0 and above (32-bit and 64-bit), SGI IRIX, HP-UX 64-bit, IBM 64-bit and Compaq Tru64 Unix. To run simulations using the SMP parallel version, all that is required is a multiprocessor computer and the appropriate license from Flow Science. The current SMP version of FLOW-3D is Version 9.2.
To run FLOW-3D/MP, users must have a cluster of computers (which may consist of a group of similarly configured Xeons®, Pentiums®, Athlons® or Opterons®) running Redhat Enterprise 3 or higher or Suse Enterprise 9 (or higher), a high speed network interconnect such as Gigabit Ethernet and a large shared NFS disk accessible from all nodes in cluster. Required software includes FLOW-3D/MP and MPICH2 (freeware).
