Computational Infrastructure for Geodynamics Wiki

User Tools

Site Tools

Trace:
software:citcoms:start

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
software:citcoms:start [2014/09/09 17:59]
rkk [Preparation] 		software:citcoms:start [2021/01/01 23:17] (current)
ljhwang
Line 2: Line 2:
  
 [[http://​geodynamics.org/​cig/​software/​citcoms|CitcomS Software Page]] [[http://​geodynamics.org/​cig/​software/​citcoms|CitcomS Software Page]]
 +
 +[[software:​citcoms:​User Group Meetings|User Group Meetings]]
 +
 +//This content has been moved to hubzero. LJ Hwang 2021 Jan 1//
 +
 +===== Current Community Development Projects ======
 +
 +Please let the community know what features your are currently working on in CitcomS and projected release date by adding a page here.
  
 ===== Benchmarks ===== ===== Benchmarks =====
Line 11: Line 19:
 [[software:​citcoms:​tutorials:​preparation|Preparation]] [[software:​citcoms:​tutorials:​preparation|Preparation]]
  
-[[software:​citcoms:​tutorials:​tutorialOne|Tutorial One]]+[[software:​citcoms:​tutorials:​tutorialOne|Tutorial One --- Global Model]]
  
-[[software:​citcoms:​tutorials:​tutorialTwo|Tutorial Two]] +[[software:​citcoms:​tutorials:​tutorialTwo|Tutorial Two --- Velocity Boundary Conditions]]
- +
-===== Teleconference Summaries and Action Items ====== +
-=== April 2nd, 2014 === +
-== Discussion == +
- +
-  ​- Rajesh reported on the CitcomS benchmarking progress. The "​A1"​ benchmark is running on Stampede, and preliminary results are expected in the next couple of days. +
-  ​- Nicola expressed an interest in timing the Gaia code on Stampede. This code has already been benchmarked. The question is should we create an account on Stampede so that Nicola can run this code himself, or will he provide the code to us (CIG staff) and we run the code on Stampede.  +
-  ​- It was suggested that a mantle convection focussed meeting of Citcom and Aspect users should be organized, perhaps in Davis before the AGU Fall meeting. We will discuss this in further detail at a later date +
-  - The following CitcomS work plan was reiterated:​ +
-    - Benchmarking CitcomS on Stampede. +
-    - Complete the removal of python and caps_per_proc from the CitcomS code +
-    - Incorporate PETSc changes that have been (partially?​) implemented for the CitcomCU code base +
-    - Possible merging of the CitcomS and CitcomCU codes (to be discussed further) +
- +
-=== February 19th, 2014 === +
-== Discussion == +
- +
-  - Rajesh reported on the Cookbook 4 "​solver not converging"​ warnings. The problem has been identified and further testing is in progress. +
-  - CitcomS manual is being updated. It was agreed that adding a '​Multigrid HOWTO' section would be a good idea. Scott,​Margarette and Katrina agreed to provide config files, MATLAB scripts, etc. to faciliate this   +
- +
-== Action Items == +
-  - Identify and fix the source of the warning in cookbook 4. +
-  - Update manual to include the Multigrid versions of the cookbooks and the benchmarks.  +
-  - Validating CitcomS 3.2 using cookbooks and benchmarks.  +
-   +
-=== February 5th, 2014 === +
- +
-== Discussion == +
- +
-  - Rajesh reported on the conversion of cookbooks 1-8 using the Python to C configuration file conversion tool, and the testing of cookbooks 1-4. The conversion tool and the converted config files have been checked into new CitcomS branch, called "​python_removal"​  +
-  - Cookbook 4 has several "​solver not converging"​ warnings and Scott suggested running this cookbook with a uniform mesh to check if the warnings are eliminated. +
-  - We disucssed if only the multigrid versions of the cookbooks should be presented in the manual. While this would be useful in introducing the concepts of multigrid to new Citcom users, there are many subtleties with setting up a multigrid problems which need to be carefully presented. +
-  - Scott suggested that after adding topography to Aspect, we need to check if topography results can be improved by higher order elements. +
-  - Margarete brought up the technique of "​manufactured solutions"​ and how they can be used to verify code that can't be tested analytically. She provided the following papers on this subject ​  +
-    - [[http://​fluidsengineering.asmedigitalcollection.asme.org/​article.aspx?​articleid=1429463]] ​  +
-    - [[http://​gji.oxfordjournals.org/​content/​early/​2013/​01/​09/​gji.ggs070]] +
-    - [[http://​onlinelibrary.wiley.com/​doi/​10.1029/​2011GC003567/​abstract]] +
-  - Shijie noted that we need to be careful regarding the initial condition spherical harmonics for CitcomS +
- +
-== Action Items == +
-  - Identify and fix the source of the warning in cookbook 4 [Rajesh] +
-  - Update the manual to refer to the new C based versions of the config files. Eliminate all references to Python from the CitcomS manual [Rajesh] +
-  - Conver the benchmark configuration files to use multigrid, rather than conjugate gradient. [Rajesh] +
-  - Continue work on the spherical convection benchmarks for CitcomS and Aspect.  +
-   +
-=== January 22nd, 2014 === +
-== Discussion == +
- +
-  - The priority list for CitcomS was discussed. Specifically,​ the status of the energy solver was discussed. To this end we discussed Entropy Viscosity method, Discontinuous Galerkin methods, Finite Volume methods, etc. The issue of cleanup of tracer code in CitcomS was also discussed. It was agreed that the initial focus should be on improving the Stokes solver by incorporating PETSc into Citcom, and eliminating Python dependencies from CitcomS +
-  - Spherical convection benchmarks for Citcom and Aspect were discussed. It was agreed that the initial focus should be on the benchmarks from Zhong et. al (2008) paper. The target for the initial benchmarking is the Banff meeting in early May.  +
- +
-== Action Items == +
-  - The C-based version of CitcomS, and the tool for converting Python-based .cfg files to C-based files, need to output the full parameter list, indicating if each parameter has the default value, or if the value has been overriden via the config file. [Rajesh] +
-  - Test the consistency of the cookbooks in CitcomS. This will be accomplished by comparing the results of running the cookbooks under CitcomS 3.1 and 3.2 [Rajesh] +
-  - Start collecting the materials needed for benchmarking CitcomS and Aspect. Eric will setup a Google doc to start gathering the information about spherical convection benchmarks for Citcom and Aspect. [Eric] +
- +
- +
- +
- +
- +
-===== Tutorials ===== +
- +
- +
-==== Tutorial 1: Global Model ==== +
- +
-This example solves for thermal convection within a full spherical shell domain. +
- +
-=== Problem === +
- +
-This example solves for thermal convection within a full spherical shell domain. The full spherical version of CitcomS.py is designed to run on a cluster that decomposes the spherical shell into 12 equal "​caps"​ and then distributes the calculation for caps onto separate processors. To run CitcomS.py with the full solver parameter set, it is recommended that you have a minimum of 12 processors available on your cluster.  +
- +
-{{ :​software:​citcoms:​cookbook1.gif?​direct |}} +
- +
-=== Solution === +
- +
- +
-You will use cookbook1.cfg. The first set of parameters specifies that you are going to use the full spherical version of the solver. The default is to use the regional spherical version, so this must be set: +
- +
-      solver = full +
- +
-The second set of parameters specifies the number of time steps (101), how often full outputs of the computation are created (25), and the prefix of output filenames (cookbook1):​ +
- +
-      steps = 101 +
-      monitoringFrequency = 25 +
-      datafile = cookbook1 +
- +
-The solver.ic facility controls the temperature field for the initial conditions. The last set of parameters includes the number of perturbations to the initial temperature (1), the number of nodal lines of the perturbation in the longitudinal direction, e.g., the spherical harmonic order (3), the number of nodal lines in the latitudinal direction, e.g., the spherical harmonic degree (2), which layer the pertubation is on (5), and the amplitude of the perturbation (0.05). Note that although the number of perturbations is assigned here as num_perturbations=1,​ that is actually the default value: +
- +
-      num_perturbations = 1 +
-      perturbl = 3 +
-      perturbm = 2 +
-      perturblayer = 5 +
-      perturbmag = 0.05 +
- +
-This example is executed by typing '$ citcoms cookbook1.cfg'​ +
- +
-Example: Global Model, cookbook1.cfg:​ +
- +
-      [CitcomS] +
-      solver = full +
-      steps = 101                 ; number of time steps +
-       +
-      [CitcomS.controller] +
-      monitoringFrequency = 25    ; how often outputs are created +
-       +
-      [CitcomS.solver] +
-      datafile = cookbook1 ​       ; prefix of output filenames +
-       +
-      [CitcomS.solver.ic] +
-      num_perturbations = 1 +
-      perturbl = 3 +
-      perturbm = 2 +
-      perturblayer = 5 +
-      perturbmag = 0.05  +
- +
-Once you have run the model, you can visualize the results using OpenDX, described in the previous chapter. When you invoke "Edit Visual Program,"​ select visFull.net.  +
- +
-{{ :​software:​citcoms:​cookbook1.2.gif?​direct |}} +
- +
-=== Discussion === +
- +
-You have generated a simple example of the full CitcomS.py model, with minimal parameter alterations. With a default Rayleigh number of 10<​sup>​5</​sup>​ and perturbation on the initial temperature field which has a degree-3 and an order-2 pattern, after 100 time steps, the convection pattern remains relatively steady. +
- +
-As a side note, it is not required that 12 processors, with one spherical cap per processor, be used. As an end-member possibility,​ for example, 12 different jobs could be run on a single computer (n001 in this example) by invoking: +
- +
-        $ citcoms cookbook1.cfg --launcher.nodegen="​n%03d"​\ +
-        --launcher.nodelist=[1,​1,​1,​1,​1,​1,​1,​1,​1,​1,​1,​1] +
- +
-This is not particularly efficient, but it does illustrate the flexibility of both mpi and Pyre. +
- +
- +
- +
-==== Tutorial 2: Velocity Boundary Conditions ​==== +
- +
-=== Problem === +
- +
- +
-This example solves for thermal convection with velocity boundary conditions imposed on the top surface within a given region of a sphere. This requires using the regional version of CitcomS.py.  +
- +
-=== Solution === +
- +
-You will use cookbook2.cfg. The parameters specify the number of time steps (61), the prefix of the output filenames (cookbook2),​ and how often outputs will be created (30): +
- +
-      steps = 61 +
-      monitoringFrequency = 30 +
-      datafile = cookbook2 +
- +
-The solver.mesher facility has several properties involved in the generation of the computational mesh. Continue to use the default values for the physical portion of the domain in which you are interested. However, try modifying the layout of the mesh as shown: +
- +
-      nprocx =  2 +
-      nprocy =  2 +
-      nodex  = 17 +
-      nodey  = 17 +
-      nodez  =  9 +
- +
-The solver.bc facility allows you to impose a uniform velocity across the top surface. You have a velocity which is purely in the colatitude direction with a non-dimensional velocity of 100 (see Equation 1.6 in the [[http://​geodynamics.org/​cig/​software/​citcoms/​citcoms.pdf|CitcomS User Manual]] for how to dimensionalize it): +
- +
-      topvbc ​   =   1 +
-      topvbxval = 100 +
- +
-In addition, the initial temperature field has a linear conductive profile. The amplitude of initial temperature perturbation is set to zero using the solver.ic facility  +
- +
-      num_perturbations = 1 +
-      perturbmag ​       = 0.0 +
- +
-Example: Velocity Boundary Conditions, cookbook2.cfg:​ +
- +
-      [CitcomS] +
-      steps = 61                  ; number of time steps +
-       +
-      [CitcomS.controller] +
-      monitoringFrequency = 30    ; how often outputs are created +
-       +
-      [CitcomS.solver] +
-      datafile = cookbook2 ​       ; prefix of output filenames +
-       +
-      # Modify the layout of the mesh. +
-      [CitcomS.solver.mesher] +
-      nprocx =  2 +
-      nprocy =  2 +
-      nodex  = 17 +
-      nodey  = 17 +
-      nodez  =  9 +
-       +
-      # Impose a uniform velocity across the top surface. +
-      [CitcomS.solver.bc] +
-      topvbc ​   =   1 +
-      topvbxval = 100 +
-       +
-      # Modify the layout of the mesh. +
-      [CitcomS.solver.mesher] +
-      nprocx =  2 +
-      nprocy =   2 +
-      nodex  = 17 +
-      nodey  = 17 +
-      nodez  =  9 +
-       +
-      # Impose a uniform velocity across the top surface. +
-      [CitcomS.solver.bc] +
-      topvbc ​   =   1 +
-      topvbxval = 100 +
-      topvbyval =   0 +
-       +
-      # In addition, set the initial temperature perturbation to zero. +
-      [CitcomS.solver.ic] +
-      num_perturbations = 1 +
-      perturbmag ​       = 0.0  +
- +
-=== Discussion === +
- +
-Using OpenDX to visualize the results (Figure 3 below), this model allows you to create a plate-driven convection in which there is a thermal upwelling on one wall, a thermal downwelling on another, and uniform horizontal velocity across the top. The downwelling is not exactly subduction because the default boundary conditions are close to zero shear stress on the boundaries. This means that there is a symmetrical downwelling in a vertical domain on the other side.  +
- +
-{{ :​software:​citcoms:​cookbook2.2.gif?​direct |}} +
- +
-//More tutorials are found in Chapter 6 of the [[http://​geodynamics.org/​cig/​software/​citcoms/​citcoms.pdf|CitcomS User Manual]] +
-//+
  
 ===== Animations ===== ===== Animations =====
 +[[software:​citcoms:​animations|Animations]]
  
-{{url>​http://​www.youtube.com/​embed/​uzd0zn2qJMU 420px,315px noborder |CitcomS Thermo-Chemical Convection for the Mantle.}} 
- 
-The temperature cross section and a composition isosurface are shown. A dense chemical layer is at the base of the mantle initially. As the layer heats up and convection develops, the layer gets entrained into the ambient mantle. ([[http://​geodynamics.org/​static/​citcoms_thermochemical-convection.mov|original]]) 
- 
-[[http://​geodynamics.org/​static/​citcoms_movie.gif|CitcomS:​ Evolution of Plume Conduit in the Coupled Model]] 
- 
-The containing solver (not shown) is a global model, with plate motion imposed on the top surface. The embedded Solver (shown in the animation) is a high-resolution regional model, with boundary conditions retrieved from the containing solver. The black line is the past hotspot location. The red segment is the assumed melt conduit, starting at 160 km depth. The velocity vector is in yellow. The temperature BC at the CMB is shown in color. The plume is visualized as an iso-surface (T=0.08). The numbers of grid points of both meshes are reduced for visualization purposes. 
 ===== Scalability Results ===== ===== Scalability Results =====
 +[[software:​citcoms:​scalabilityresults|Scalability Results]]
  
-Model Description+===== Teleconference Summaries and Action Items ====== 
 +[[software:​citcoms:​teleconsummaries|Teleconference Summaries and Action Items]]
  
-These scalability tests were run using CitcomS 3.2.0 with default configuration. The mesh for these tests is a regional cap with 129x129x129 nodes. Total velocity unknowns is 129^3 x 3 = 6.4 million. The model is run for 11 time steps. The result reported is the total wall clock time. Each node on this cluster has 2 Xeon 5680 series 3.33GHz hex-core processors with a 12MB unified L3 cache and 24GB RAM, for a total of 12 cores per node. The interconnect is QDR InfiniBand. 
- 
-^Partition^Total Procs^Wall Time (sec)^Speedup^Scalability^ 
-|1x1x1|1|47217|1.000|1.000| 
-|1x1x2|2|25466|1.854|0.927| 
-|1x1x4|4|14645|3.224|0.806| 
-|2x2x1|4|14438|3.270|0.818| 
-|2x2x2|8|8980|5.258|0.657| 
-|2x2x4|16|4432|10.654|0.666| 
-|4x4x1|16|5367|8.798|0.550| 
-|4x4x2|32|2460|19.194|0.600| 
-|4x4x4|64|1346|35.079|0.548| 
-|8x8x2|128|583|80.990|0.633| 
-|8x8x4|256|337|140.110|0.547| 
- 
-The input file is {{:​software:​citcoms:​input.sample.zip|available here}}. It is currently configured for 1x1x1 processors, to do different processor divisions you must change the nprocx, nprocy, and nprocz parameters. You must create a folder named "​scratch"​ in the working directory for the output files. ​ The input file uses the non-Python version of CitcomS, located at CitcomS-3.2.0/​bin/​CitcomSRegional. 
 ===== FAQ ===== ===== FAQ =====
 +[[software:​citcoms:​faq|FAQ]]
  
-=== Install CitcomS ​===+===== Other Documents ​===== 
 +[[software:​citcoms:​documents:​tutorials|Tutorial Talks]]
  
-Q: What are the dependencies of CitcomS-2.2 or later?+[[software:citcoms:​documents:​technotes|Technical Notes]]
  
-CitcomS-2.2 or later depends on the MPI library. In addition, if you like to use the Pyre framework (which provides an ease-of-use environment),​ you will need Python 2.3 or newer. (Note that if your machine is 64-bit, you will need Python 2.4.) 
  
----- 
- 
-Q: I want to use CitcomS in the old way (pure C code, no Python). How can I do that? 
- 
-You can configure CitcomS to only use C code by running: 
- 
-    ./configure --without-pyre 
- 
-before compiling the code. 
- 
----- 
- 
-Q: What does it mean when "​configure"​ reports this error message 
- 
-    OverflowError:​ signed integer is greater than maximum 
- 
-Are you on a 64-bit machine? If so, you will need to upgrade your Python to v2.4 or newer. Python 2.3 or earlier is not 64-bit safe. 
- 
----- 
- 
-Q: The gcc/​python/​mpi on my system is too old, but I don't have the privilege to update the software. How can I install gcc/​python/​mpi under my home directory? 
- 
-1 Get the tarball of gcc/​python/​mpi and untar it 
- 
-2 cd into the untarred directory 
- 
-3 Run the following to install the package under the directory $HOME/opt 
- 
-    ./configure --prefix=$HOME/​opt && make && make install ​ 
- 
-4 Add ''​$HOME/​opt/​bin''​ into your ''​$PATH''​ and (maybe) ''​$HOME/​opt/​lib''​ into your ''​$LD_PATH_LIBRARY''​. 
- 
-=== Documentation for Developers === 
- 
-**Q: I want a specific temperature boundary condition. Where to modify?** 
- 
-Look at lib/ directory, the function full_temperature_boundary_conditions() in Full_boundary_conditions.c or regional_temperature_boundary_conditions in Regional_boundary_conditions.c is the function you need to modify. 
- 
----- 
- 
-**Q: I want a specific velocity boundary condition. Where to modify?** 
- 
-Look at lib/ directory, the function full_velocity_boundary_conditions() in Full_boundary_conditions.c or regional_velocity_boundary_conditions() in Regional_boundary_conditions.c. 
- 
----- 
- 
-**Q: I want a specific rheology law. Where to modify?** 
- 
-Look at lib/ directory, the function get_system_viscosity() in Viscosity_structures.c. 
- 
----- 
- 
- 
-**Q: I want a new equation of state or a new law for heat expansivity,​ heat capacity, or thermal conductivity. Where to modify?** 
- 
-Look at lib/ directory, the function () in Material_properties.c. 
- 
-===== Other Documents ===== 
  
-{{:​software:​citcoms:​citcoms-earthscope09.pdf|CitcomS Tutorial}} PDF slides of talk given by Eh Tan, CIG Training Session, EarthScope 2009, Boise, ID (May 12, 2009), on large scale models 
  
-{{:​software:​citcoms:​modified_legendre_functions.pdf|Modified Legendre Functions}} A technical note by Shijie Zhong. 
software/citcoms/start.1410285546.txt.gz · Last modified: 2014/09/09 17:59 by rkk

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 3.0 Unported
CC Attribution-Share Alike 3.0 Unported Donate Powered by PHP Valid HTML5 Valid CSS Run on Debian Driven by DokuWiki