This is an old revision of the document!
The temperature cross section and a composition isosurface are shown. A dense chemical layer is at the base of the mantle initially. As the layer heats up and convection develops, the layer gets entrained into the ambient mantle. (original)
CitcomS: Evolution of Plume Conduit in the Coupled Model
The containing solver (not shown) is a global model, with plate motion imposed on the top surface. The embedded Solver (shown in the animation) is a high-resolution regional model, with boundary conditions retrieved from the containing solver. The black line is the past hotspot location. The red segment is the assumed melt conduit, starting at 160 km depth. The velocity vector is in yellow. The temperature BC at the CMB is shown in color. The plume is visualized as an iso-surface (T=0.08). The numbers of grid points of both meshes are reduced for visualization purposes.
Model Description
These scalability tests were run using CitcomS 3.2.0 with default configuration. The mesh for these tests is a regional cap with 129x129x129 nodes. Total velocity unknowns is 129^3 x 3 = 6.4 million. The model is run for 11 time steps. The result reported is the total wall clock time. Each node on this cluster has 2 Xeon 5680 series 3.33GHz hex-core processors with a 12MB unified L3 cache and 24GB RAM, for a total of 12 cores per node. The interconnect is QDR InfiniBand.
Partition | Total Procs | Wall Time (sec) | Speedup | Scalability |
---|---|---|---|---|
1x1x1 | 1 | 47217 | 1.000 | 1.000 |
1x1x2 | 2 | 25466 | 1.854 | 0.927 |
1x1x4 | 4 | 14645 | 3.224 | 0.806 |
2x2x1 | 4 | 14438 | 3.270 | 0.818 |
2x2x2 | 8 | 8980 | 5.258 | 0.657 |
2x2x4 | 16 | 4432 | 10.654 | 0.666 |
4x4x1 | 16 | 5367 | 8.798 | 0.550 |
4x4x2 | 32 | 2460 | 19.194 | 0.600 |
4x4x4 | 64 | 1346 | 35.079 | 0.548 |
8x8x2 | 128 | 583 | 80.990 | 0.633 |
8x8x4 | 256 | 337 | 140.110 | 0.547 |
The input file is available here. It is currently configured for 1x1x1 processors, to do different processor divisions you must change the nprocx, nprocy, and nprocz parameters. You must create a folder named “scratch” in the working directory for the output files. The input file uses the non-Python version of CitcomS, located at CitcomS-3.2.0/bin/CitcomSRegional.
Q: What are the dependencies of CitcomS-2.2 or later?
CitcomS-2.2 or later depends on the MPI library. In addition, if you like to use the Pyre framework (which provides an ease-of-use environment), you will need Python 2.3 or newer. (Note that if your machine is 64-bit, you will need Python 2.4.)
Q: I want to use CitcomS in the old way (pure C code, no Python). How can I do that?
You can configure CitcomS to only use C code by running:
./configure --without-pyre
before compiling the code.
Q: What does it mean when “configure” reports this error message
OverflowError: signed integer is greater than maximum
Are you on a 64-bit machine? If so, you will need to upgrade your Python to v2.4 or newer. Python 2.3 or earlier is not 64-bit safe.
Q: The gcc/python/mpi on my system is too old, but I don't have the privilege to update the software. How can I install gcc/python/mpi under my home directory?
1 Get the tarball of gcc/python/mpi and untar it
2 cd into the untarred directory
3 Run the following to install the package under the directory $HOME/opt
./configure --prefix=$HOME/opt && make && make install
4 Add $HOME/opt/bin
into your $PATH
and (maybe) $HOME/opt/lib
into your $LD_PATH_LIBRARY
.
Q: I want a specific temperature boundary condition. Where to modify?
Look at lib/ directory, the function full_temperature_boundary_conditions() in Full_boundary_conditions.c or regional_temperature_boundary_conditions in Regional_boundary_conditions.c is the function you need to modify.
Q: I want a specific velocity boundary condition. Where to modify?
Look at lib/ directory, the function full_velocity_boundary_conditions() in Full_boundary_conditions.c or regional_velocity_boundary_conditions() in Regional_boundary_conditions.c.
Q: I want a specific rheology law. Where to modify?
Look at lib/ directory, the function get_system_viscosity() in Viscosity_structures.c.
Q: I want a new equation of state or a new law for heat expansivity, heat capacity, or thermal conductivity. Where to modify?
Look at lib/ directory, the function () in Material_properties.c.
CitcomS Tutorial PDF slides of talk given by Eh Tan, CIG Training Session, EarthScope 2009, Boise, ID (May 12, 2009), on large scale models
Modified Legendre Functions A technical note by Shijie Zhong.