Computational Infrastructure for Geodynamics Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Pythia ======

[[http://geodynamics.org/cig/software/pythia|Pythia Software Page]]

===== Meeting Notes June 5, 2006 =====

Attendees: Michael Aivazis, Luis Armendariz, Patrick Hung, Mike McKerns, Leif Strand

summary of last week:

set meeting time
discussed parts of pyre to keep
components, framework, inventory
agenda:

discuss the design issues for pyre
map people to the work
What are the issues?:

Application support --Leif
databases -- Luis
non-trivial UI (web & gui) -- Patrick
distributed computing --Mike
documentation --Cassie, Michael, everyone
PH> Documentation is going to be wiki style?

MA> I envisioned a pair of O'Reilly-style books. Pyre has always lacked a good structure for documentation. I thought we'd use docbook, but maybe that's not the case. Most projects use teX.

Application support:

Component
what is it? [thing to allow easy way to have things talk to each other]
what does it need to function? [what do I provide to make it work correctly]
what is it's enviroment (i.e. interactions with other code)
        / Property
  Trait \ Facility

  Inventory, Registry
  Curator
   ...
We need a map of classes under component & the classes that derive from them. We need a map of the descendants of component & how they interact.

MA> Lief, any additions at the top-level?

LS> I don't see anything further right now at the top level.

MM> What about bootstrapping of components when you are constructing apps?

MA> You mean how you wake up components?  The class below Script assumes a normalized way that components wake up.  You should also be able to do something weird.  Can all of these weird things be done in the current implementation?  I don't know.  In many cases, I'll start something that does this bootstrapping, however the design has always been wrong.  I can use subclasses in every case I've come across so far.

MM> I'd maybe use bootstrapping for GUI/web usage and distributed components.

MA> Ok, that's a good possibility.  I currently write a configuration file for a proxy to tell it where the thing lives.  Maybe this is not the best?

LS> What about nested components?

MA> To me a component is a box. What's inside is not my problem. Inside you could have other components that you are managing.

LS> What I want is lookup. I want to have a component lookup a scratch directory, or a clock cycle... instead of having to wire it into all of the components that I write, I want a global lookup.

MA> There is this association between component and inventory that is too tight... but let's neglect that right now. There is, in pyre, support to say "I'm not the keeper of a property"... There is something called a "delagate", that when the property is set to None, [...MISSED...] then a runtime instance [...MISSED...] A nested inventory delegate is a subset of this. It was being used in 0.7, I don't know if it is used in 0.8 -- someone will have to check. We need to talk to about freeing components from the inventory. It may be a trivial change.  Just by saying "self.inventory", this forces us to have a single inventory... we need to move from this. Anything else?

MM> What about using .odb files from more than one package?

MA> Yes, odb is also something to talk about. I don't know if it came from a conversation with you or otherwise, but I've recently made a change that allows .odb files to associate with more than one component, and be used with n-components instead.

PH> Is there a central page where all documentation can be found?

MA> No, there is a lot of it, but it is scattered between DANSE, CIG, the pyre page, and elsewhere.  There are also talks, tutorials, and a bunch of examples.

MM> I've been compiling talks, notes, tutorials, and have started a repository for a pyre user manual.  There's not much to the manual so far, but some notes on the desired content we discussed a few months ago.

PH> I'd like to see this documentation for pyre done.  There are some things like traits that I'm not so familiar with.  What do you suggest to remedy this?

MA> At these meetings, maybe we get cassie to take notes. Then I can go through use of the critical pieces of pyre and how it was intended to be used.  We will delegate all of the other items (UI, database, distributed, application) to the back for now. Anyway, if we can do the distributed computing, it solves all of these problems.  It is so complex, that we should be able to use the solution to do all of the other things like databases and UIs.

PH> Doesn't Twisted do a lot of these things?

MA> Yes, we should steal a good bit of their code, but it is not enough. I also have a simple implementation in pyre, but it is not enough. [...MISSED...]

MM> It's the same story, but as soon as you have three parties, things become difficult.

MA> Let's talk about databases...

Databases:

philosophy
- object store on top of RDBMS

mapping ==>     handle to object is row in table
and attributes are columns in table
data object model (dom)
set of objects that are representable in the DB
all of the things that inventory types have, these should have
(when I put this into pyre, I cut and pasted from inventory)
inventories are never needed to be blank, but here this is needed
operations on the container object
ask the manager of the table to get the information that is requested
We need to... Look at the design, build the DOM. Take apart the low level bindings to the developer libdevXXX (XXX = mysql, ...) and see what is useable.

mysql, postgressql, and etc are using tuples... this is crap. We need to use a DOM.  To not produce brittle code, the only way to make the code maintainable is to use a DOM.  I mean, if you add an extra row to a table and you are using tuples, all of your code breaks... how foolish is that?  A DOM does not have this problem.

UI:

easy event driven support
- initial intent for pyre was for asynchronous event driven support

GUI, web
- forms for user interactions

. collect properties  (user -> server)
. configure components (server -> components)
. invoke actions
- opal = for the web we have a working web example (also uses a database)

- blade = older, built to construct GUI in backend independent way

Opal works by building structure document [...MISSED...]

short term: enthought.ui & enthought.traits
 built on traits & wxWindows
their traits is our inventory
MA> It works by generating dialouge boxes to take information that will be used to construct the GUI, then setting the properties.

I think that looking at enthought would be the first thing to do. Can you look at wxWindows on the mac, Patrick?

I need to make sure it doesn't suck on the mac. We have a real call from users to build a short terms solution for how GUI construction is suggested to be done.  Right now, I have several groups that are all building GUIs and they have all selected the wrong package, and they are mixing up their functionallity.  I need to be able to tell them this is the way I suggest to do it.

Distributed component control:

We don't have anything pressing here for DANSE right now, but next year we will.  Mike you can help out on other things for now.
MM> Distributed component control in DANSE begins the last month or so of 2006.

Documentation:

I'd like to have all of this turn into a book. I've had the wrong approach to documentation all of these years... we can have documentation first, where we have our converstaions turn into documentation -- then take that documentation turn into code.
Action items:

Lief, classify your pyre notes as feature requests, bugs, ...
Luis, look at psychopg
Patrick, play with traits on the mac

===== Batch System Configuration =====

Instructions on configuring the CIG edition of Pythia for a batch system such as LSF or PBS.

The following examples assume you are using CitcomS.  If you are using PyLith v0.8.2, replace 'CitcomS' with 'pylith3d'.

The configuration options below are suitable for inclusion in a ''CitcomS.cfg'' configuration file; for example:

  ~/.pyre/CitcomS/CitcomS.cfg

Be sure to set the ''command'' option under ''[CitcomS.launcher]'' to the proper command for launching MPI jobs on your cluster. This command will vary from cluster to cluster; the commands shown here are provided only as examples.

==== LSF ====

Here are example configuration options for LSF. These were tested on TACC's Lonestar system.

  [CitcomS]
  scheduler = lsf
  
  [CitcomS.job]
  queue = normal
  #queue = development
  
  [CitcomS.launcher]
  command = ibrun

==== PBS ====

Here are example configuration options for PBS. These were tested on NCSA's Itanium cluster.

You will have to set ''ppn'' to a value suitable for your cluster.

  [CitcomS]
  scheduler = pbs
  
  [CitcomS.launcher]
  command = mpirun -np ${nodes} -machinefile ${PBS_NODEFILE}
  
  [CitcomS.pbs]
  ppn = 2  ; processors per node
Computational Infrastructure for Geodynamics Wiki

User Tools

Site Tools

Page Tools
