Difference: Pathena (1 vs. 2)

Revision 218 Feb 2009 - FZU.MartinZeman

Line: 1 to 1
 
META TOPICPARENT name="AtlasSoftware"

Using PanDA? Athena

Changed:
<
<

1 Atlas TWiki

1.1 The PanDA? Production and Distributed Analysis System

General info about panda: https://twiki.cern.ch/twiki/bin/view/Atlas/Panda
>
>
 
Changed:
<
<

0.1 Client tools for Panda analysis jobs

Installation, setup, release notes: https://twiki.cern.ch/twiki/bin/view/Atlas/PandaTools
>
>

1 Atlas TWiki Links

The PanDA? Production and Distributed Analysis System: https://twiki.cern.ch/twiki/bin/view/Atlas/Panda
 
Changed:
<
<

0.1 How to submit Athena jobs to Panda

Job Submission: https://twiki.cern.ch/twiki/bin/view/Atlas/PandaAthena
>
>		Client tools for Panda analysis jobs: https://twiki.cern.ch/twiki/bin/view/Atlas/PandaTools

How to submit Athena jobs to Panda: https://twiki.cern.ch/twiki/bin/view/Atlas/PandaAthena

 

1 Pathena Full Chain

Changed:
<
<		Athena 14.5.0.2 Jobs in <i<pathena are run just as any job in Athena on a JobOptions file:
>
>		Used Athena Version: 14.5.0.2

Jobs in pathena are run just as any job in Athena on a JobOptions file:

 
> athena.py <JobOptions>
using the pathena command on a Job Options file. OutputDataset and InputDataset is all that you have to provide in addition:
Changed:
<
<		pathena
>
>		> pathena
  [--inDS ] --outDS
Line: 45 to 45
 

0.0.1 How to change minimum number of events:

Changed:
<
<		The default value of the MC8.105144.PythiaZee.py is 5000 events, therefore if you choose your > MAX EVENTS $lt below 5000, you will get into problems and generation will crash.
>
>		The default value of the MC8.105144.PythiaZee.py is 5000 events, therefore if you choose your below 5000, you will get into problems and generation will crash.
  If you want to change this, you need to edit the JobOptions? .py file (e.g. MC8.105144.PythiaZee.py) and add this line to the end:
Line: 54 to 54
  For pathena we can of course leave the default 5000, since pathena standard pathena jobs get killed after 90 minutes rather than 45 minutes on the LXPLUS. (pathena --long allows you to submit to a long queue).
Changed:
<
<

0.0.1 Running Athena on JobOptions?

>
>

0.0.1 Running Pathena on JobOptions?

is basically the same as running athena on JobOptions:
 
Changed:
<
<		pathena pomwig.py --outDS user09.MartinZeman.pomwig.evgen.pool.v14000502 --noBuild
>
>		> pathena pomwig.py --outDS user09.MartinZeman.pomwig.evgen.pool.v14000502 --noBuild
 
Changed:
<
<

0.0.1 Running Generation using Job Transformation

>
>

0.0.1 Running Generation using CSC Job Transformation on Pathena

 Locally you can run Evgen using Job Transformation by issuing:
Changed:
<
<		> csc_evgen08_trf.py ./
>
>		> csc_evgen08_trf.py <RunNo.> ./<JobOptions.py> <Output.evgen.pool.root>
 

In our case, we have MC8.105144.PythiaZee.py so the transformation will be locally run like this:

Changed:
<
<		> csc_evgen08_trf.py 105144 1 110 1324354657 ./MC8.105144 .PythiaZee.py PythiaZee? .110.Local.pool.root
>
>		> csc_evgen08_trf.py 105144 1 110 1324354657 ./MC8.105144.PythiaZee.py PythiaZee? .110.evgen.pool.root
 

Submission through pathena works similarly:

Changed:
<
<		> csc_evgen08_trf.py 105144 1 110 1324354657 ./MC8.105144 .PythiaZee.py PythiaZee? .110.Local.pool.root
>
>		> pathena --trf "csc_evgen08_trf.py <RunNo.> ./<JobOptions.py> %OUT.evgen.pool.root"
--outDS --noBuild
 

and our example requires little change:

Changed:
<
<		> pathena --trf "csc_evgen08_trf.py 105144 1 110 1324354657 ./MC8.105144 .PythiaZee.py PythiaZee? .110.Local.pool.root
>
>		> pathena --trf "csc_evgen08_trf.py 105144 1 10000 1324354657 ./MC8.105144 .PythiaZee.py %OUT.evgen.pool.root"
--outDS user09.MartinZeman.MC8.105144.PythiaZee.evgen.v14000502 --noBuild
 
Changed:
<
<		pathena --trf "csc_reco_trf.py %IN %OUT.ESD.pool.root %OUT.AOD.pool.root %OUT.NTUP.root 250 0 ATLAS-CSC-05-00-00 DEFAULT ./NoTrackSlimming.py %DB=ddo.000001.Atlas.Ideal.DBRelease.v050101:DBRelease-5.1.1.tar.gz NONE" --inDS ... --outDS ...

pathena jobOptions.pythia.py --outDS user.TadashiMaeno.456.ccaho.evgen.pool.v5 --split 3

>
>		Dont forget to change the name of the OutputDataset to user09.. otherwise pathena will not submit the job.
 
Changed:
<
<

0.1 Simulation + Digitization

>
>

0.1 Simulation

Simulation is run just as any job in Athena using the athena.py script on a Job Options file. You can run simulation together with digitization using Geant4 by running csc_simul_trf.py script (accessible after sourcing Athena).
 
Changed:
<
<

0.0.1 CSC Simulation Job Transformation

>
>

0.0.1 Running Simulation using CSC Job Transformation on Pathena

Locally you can run Simul using Job Transformation by issuing:
 
Changed:
<
<		pathena --trf "csc_simul_trf.py %IN %OUT.hits.pool.root %OUT.rdo.pool.root 100 0 1324354656 ATLAS-CSC-02-00-00 100 1000" --inDS user09.MartinZeman.pomwig.evgen.pool.v14000502 --outDS user09.MartinZeman.pomwig.simul.test.v14000502 --noBuild --long
>
>		> csc_simul_trf.py

Submission through pathena: 

> pathena --trf "csc_simul_trf.py <inputEvgenFile> <outputHitsFile> <outputRDOFile> <maxEvents> <skipEvents> 
       <randomSeed> <geometryVersion> <digiSeedOffset1> <digiSeedOffset2>" 
       --inDS <InputDataset>
       --outDS <OutputDataset>
       --noBuild --long

Following with the example: 

> pathena --trf "csc_simul_trf.py %IN %OUT.hits.pool.root %OUT.rdo.pool.root 100 0 1324354656 ATLAS-CSC-02-00-00 100 1000" 
       --inDS user09.MartinZeman.MC8.105144.PythiaZee.evgen.v14000502
       --outDS user09.MartinZeman.MC8.105144.PythiaZee.simul.v14000502
       --noBuild --long
 

0.0.1 Running Athena on JobOptions?

Changed:
<
<		This does not seem to work very well. General guidelines here: https://twiki.cern.ch/twiki/bin/view/Atlas/PandaAthena#example_2_g4sim
>
>		This does not seem to work very well so I would encourage using the JobTransformation? . General guidelines here: https://twiki.cern.ch/twiki/bin/view/Atlas/PandaAthena#example_2_g4sim.
 
Added:
>
>

0.1 Digitization

Digitization is run together with simulation if using csc_simul_trf.py job transformation. You can use csc_digi_trf.py to run just digitization in the very same way.
 
Added:
>
>
 

0.1 Reconstruction

Changed:
<
<

0.0.1 CSC Reconstruction Job Transformation

>
>		Reconstruction generally runs on the Reconstruction/RecExample/RecExCommon package. More information about how it works and ho write your JobOptions? can be found here: https://twiki.cern.ch/twiki/bin/view/Atlas/RunningReconstruction

Documentation: https://twiki.cern.ch/twiki/bin/view/Atlas/ReconstructionDocumentation

0.0.1 Running Reconstruction using CSC Job Transformation on Pathena

You can run reconstruction using Job Transformation as follows: 
> csc_reco_trf.py <InputRDOFile> esd.pool.root aod.pool.root ntuple.root <MaxEvents> <Skip> 
       <GeometryVersion> <TriggerSettings>

which in pathena is: 

> pathena --trf "csc_reco_trf.py <outputESDFile> <outputAODFile> <outputCBNTFile> <MaxEvents> <SkipEvents> 
       <GeometryVersion> <TriggerSettings>" 
       --inDS <InputDataset>
       --outDS <OutputDataset>
       --noBuild

The example: 

> pathena --trf "csc_reco_trf.py %IN %OUT.ESD.pool.root %OUT.AOD.pool.root %OUT.CBNT.pool.root 100 0
       ATLAS-CSC-02-00-00 NONE" 
       --inDS valid1.005200.T1_McAtNlo_Jimmy.digit.RDO.e322_s405/ 
       --outDS user09.MartinZeman.pathenafiles.CSC.005200.Ideal.valid1.T1_McAtNlo_Jimmy.recon 
       --noBuild

Using non-CSC reconstruction Job Trasnformations does not seem to be supported by pathena.

1 Running multiple jobs

How this works has not yet been discovered. 
--split
--nEventsPerJob
--nFiles
  -- MartinZeman - 17 Feb 2009

Revision 117 Feb 2009 - FZU.MartinZeman

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="AtlasSoftware"

Using PanDA? Athena

1 Atlas TWiki

1.1 The PanDA? Production and Distributed Analysis System

General info about panda: https://twiki.cern.ch/twiki/bin/view/Atlas/Panda

1.2 Client tools for Panda analysis jobs

Installation, setup, release notes: https://twiki.cern.ch/twiki/bin/view/Atlas/PandaTools

1.3 How to submit Athena jobs to Panda

Job Submission: https://twiki.cern.ch/twiki/bin/view/Atlas/PandaAthena


2 Pathena Full Chain

Athena 14.5.0.2 Jobs in <i<pathena are run just as any job in Athena on a JobOptions file: 
> athena.py <JobOptions>
using the pathena command on a Job Options file. OutputDataset and InputDataset is all that you have to provide in addition: 
pathena <JobOptions>
       [--inDS <InputDataset>]
       --outDS <OutputDataset>

Pathena copies the contents of the entire directory it is ran from to the remote site.


2.1 Generation

More information can be found here: https://twiki.cern.ch/twiki/bin/view/Atlas/WorkBookGeneration
General pathena example here: https://twiki.cern.ch/twiki/bin/view/Atlas/PandaAthena#example_1_evgen

2.1.1 How to get Job Options files:

You need to write or get the JobOptions you run pathena on. You can choose from a variety of files available on http://reserve02.usatlas.bnl.gov/lxr/source/atlas/Generators/EvgenJobOptions/share/:
  • MC8.105145.PythiaZmumu.py for Z->mu,mu decay,
  • MC8.105144.PythiaZee.py for Z->e,e decay,
  • and many others

Use the following command to get Job Options you want, we are going to use the Z->e+,e- decay: 

> get_files -jo MC8.105144.PythiaZee.py

2.1.2 How to change minimum number of events:

The default value of the MC8.105144.PythiaZee.py is 5000 events, therefore if you choose your > MAX EVENTS $lt below 5000, you will get into problems and generation will crash.

If you want to change this, you need to edit the JobOptions? .py file (e.g. MC8.105144.PythiaZee.py) and add this line to the end: 

evgenConfig.minevents = 100 # default is 5000

For pathena we can of course leave the default 5000, since pathena standard pathena jobs get killed after 90 minutes rather than 45 minutes on the LXPLUS. (pathena --long allows you to submit to a long queue).

2.1.3 Running Athena on JobOptions? 

pathena pomwig.py --outDS user09.MartinZeman.pomwig.evgen.pool.v14000502 --noBuild

2.1.4 Running Generation using Job Transformation

Locally you can run Evgen using Job Transformation by issuing: 
> csc_evgen08_trf.py <RUN no.> <FIRST EVENT> <MAX EVENTS> <RANDOM SEED> ./<JOB OPTIONS .py> <OUTPUT run.pool.root>

In our case, we have MC8.105144.PythiaZee.py so the transformation will be locally run like this: 

> csc_evgen08_trf.py 105144 1 110 1324354657 ./MC8.105144 .PythiaZee.py MC8.PythiaZee.110.Local.pool.root

Submission through pathena works similarly: 

> csc_evgen08_trf.py 105144 1 110 1324354657 ./MC8.105144 .PythiaZee.py MC8.PythiaZee.110.Local.pool.root

and our example requires little change: 

> pathena --trf "csc_evgen08_trf.py 105144 1 110 1324354657 ./MC8.105144 .PythiaZee.py MC8.PythiaZee.110.Local.pool.root

pathena --trf "csc_reco_trf.py %IN %OUT.ESD.pool.root %OUT.AOD.pool.root %OUT.NTUP.root 250 0 ATLAS-CSC-05-00-00 DEFAULT ./NoTrackSlimming.py %DB=ddo.000001.Atlas.Ideal.DBRelease.v050101:DBRelease-5.1.1.tar.gz NONE" --inDS ... --outDS ...

pathena jobOptions.pythia.py --outDS user.TadashiMaeno.456.ccaho.evgen.pool.v5 --split 3


2.2 Simulation + Digitization

2.2.1 CSC Simulation Job Transformation 

pathena --trf "csc_simul_trf.py %IN %OUT.hits.pool.root %OUT.rdo.pool.root 100 0 1324354656 ATLAS-CSC-02-00-00 100 1000" --inDS user09.MartinZeman.pomwig.evgen.pool.v14000502 --outDS user09.MartinZeman.pomwig.simul.test.v14000502 --noBuild --long

2.2.2 Running Athena on JobOptions?

This does not seem to work very well. General guidelines here: https://twiki.cern.ch/twiki/bin/view/Atlas/PandaAthena#example_2_g4sim

2.3 Reconstruction

2.3.1 CSC Reconstruction Job Transformation

-- MartinZeman - 17 Feb 2009

View topic | History: r2 < r1 | More topic actions...
 
This site is powered by the TWiki collaboration platformCopyright &Š by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback