Custom Cosmic Muon Simulations in Athena 13.0.10

Customized Athena Full Chain step by step:

I. Simulation

Preparing the lxplus environment

1. Login to lxplus.cern.ch using your credentials.

2. Prepare the necessary directories. Create your CMT_HOME directory for your configuration management (see http://www.cmtsite.org):

> cd $HOME
> mkdir cmt-cosmics

Create your TestArea, where your packages will be installed:

> mkdir testarea
> mkdir testarea/cosmics

3. Download the CMT enviroment to your CMT_HOME. The v1r20p20070208 (an iteration for the v1r20 release) is tested well-working environment.

> cd cmt-cosmics
> source /afs/cern.ch/sw/contrib/CMT/v1r20p20070208/mgr/setup.sh

4. Create the requirements environment for the CMT.

> touch requirements
> mcedit requirements

or use pico or whichever editor you prefer and copy this

#---- CMT HOME REQUIREMENTS FILE ---------------------------------
set   CMTSITE  CERN
set   SITEROOT /afs/cern.ch
macro ATLAS_DIST_AREA ${SITEROOT}/atlas/software/dist
macro ATLAS_TEST_AREA /afs/cern.ch/user/m/mzeman/testarea/cosmics

use AtlasLogin AtlasLogin-* $(ATLAS_DIST_AREA)

apply_tag oneTest  # use ATLAS working directory
apply_tag setup    # use working directory
apply_tag 32       # use 32-bit

#----------------------------------------------------------------

5. Configure the CMT to your AFS account (you do this only once per each new CMT_HOME you are using):

> cmt config

6. Logout.

7. Login. Enter your CMT_HOME directory and source the Athena setup with release specifications.

> cd cmt-cosmics
> source setup.sh -tag=13.0.10,runtime

8. Download the packages for cosmics simulation using cmt co command (CMT checkout).

> cd testarea/cosmics
> cmt co -r CosmicGenerator-00-00-24 Generators/CosmicGenerator
> cmt co -r G4AtlasApps-00-02-15 Simulation/G4Atlas/G4AtlasApps
> cmt co -r LArG4HitManagement-00-00-25 LArCalorimeter/LArG4/LArG4HitManagement

Package Description

• Generators/CosmicGenerator is a generator using a well-tested FORTRAN algorithm which can simulate muons according to real distributions on the surface (Beware! The Generator has reversed coordinates system, but this is fixed in our setup)
• Simulation/G4Atlas/G4AtlasApps is the main Geant4 simulation package.
• LArCalorimeter/LArG4/LArG4HitManagement is something obscure.

9. Download the modified G4UserActions? package to your Simulation area
from: /afs/cern.ch/user/j/jez/public/G4PixelCosmics/ or /afs/cern.ch/user/m/mzeman/public/Cosmics/Simulation/G4PixelCosmics/
to: testarea/cosmics/Simulation/

10. Add the following line to the G4AtlasApps? CMT requirements file (do not change anything else).

> cd testarea/cosmics/Simulation/G4Atlas/G4AtlasApps/cmt/
> mcedit requirements

#---- CMT G4AtlasApps REQUIREMENTS FILE -------------------------
.
.
.
use G4AtlasRun...
use G4PixelCosmics G4PixelCosmics-* Simulation=  # <<< this line
.
.
.
#----------------------------------------------------------------

11. Check with CMT broadcast which packages are being used, all of them should be connected.

> cmt broadcast

12. Configure and compile the packages. All the requirements files must be properly configured by now. Execute the following commands within each cmt directory of each package (in this case four times):

> cmt config
> source setup.sh
> gmake

This will create a new directory InstallArea.

You can also use this sophisticated command which does it all for all packages using two threads (so it is faster, harder, scooter): > cmt broadcast cmt config && source setup.sh && gmake -j 3

At this point you should have everything compiled and ready to use. You can verify that the package is compiled correctly if the last line the outputs print out states all ok. You should see this for each package, so in our case four times. If that is not the case, use:

> gmake clean

to delete all package binaries and start over.

13. Copy a sample jobOptions file from G4AtlasApps/share/ to G4Atlas/Apps/run which is the main run directory. It should be some simple file.

> cp testarea/cosmics/Simulation/G4Atlas/G4AtlasApps/share/jobOptions.G4Cosmic.py testarea/cosmics/Simulation/G4Atlas/G4AtlasApps/run/.

14. Modify the file so it uses the G4PixelCosmics package and its text outputs.

Add these declarations to the beginning of your jobOptions. They intertwine the script with our custom G4PixelCosmics package:

#--- Declarations -------------------------------------------
text_output = "text.output.txt"
text_output_floor = "floor.output.txt"
hits_output = "hits.output.txt"
hits_short_output = "hits.short.txt"

Add this line to the end of Simulation flags. This enables the usage of user actions (G4PixelCosmics package):

#--- Simulation flags -----------------------------------------
.
.
.
SimFlags.init_Level.set_Value(1) # enable usage of UserActions

Add this user action configuration to the end of jobOptions:

#==============================================================
# Job configuration (configure the simulation by user actions)
# ***>> Do not add flags or simulation options below this line
#==============================================================
from G4AtlasApps import SimKernel # calls simulation kernel
from G4AtlasApps import AtlasG4Eng # comms betwwen Geant4 & Athena
theApp.initialize()
from G4AtlasApps import PyG4Atlas

MyAction= PyG4Atlas.UserAction( 'G4PixelCosmics','G4PixelCosmics',
       ['BeginOfEvent','EndOfEvent','BeginOfRun','EndOfRun','Step'])

actionProperties={=output_file_name=: text_output ,
       =output_file_name_floor=: text_output_floor ,
       =hits_file_name=: hits_output ,
       =hits_short_file_name= : hits_short_output *

MyAction.set_Properties(actionProperties)

AtlasG4Eng.G4Eng.menu_UserActions.add_UserAction(MyAction)
AtlasG4Eng.G4Eng.init_Simulation(3)


theApp.nextEvent( theApp.EvtMax )
theApp.finalize()
theApp.exit()

#--- End jobOptions.G4Cosmic.py file  -------------------------

Feel free to change the number of events in the AthenaCommon flags as you like. For LXbatch simulations we use about 30000 events which proved stable. On local trying no more than 50 is the thing to do:

athenaCommonFlags.EvtMax=50 # number of events

15. Run Athena and enjoy the text output.

> athena jobOptions.G4Cosmic.bigGun.py |tee output.txt

What to do every time you log on?

Go to your CMT_HOME directory:

> cd cmt-cosmics

Load Athena environment:

> source setup.sh -tag=13.0.10,runtime

Running on lxbatch

NOT FINISHED and probably will not be for some time

In order to simulate larger numbers of events, we need to use the lxbatch because all jobs on lxplus get killed after 40 minutes. This requires modification of our jobOptions:

This submits 10 simulations to the LXbatch. Make sure you have enough disk space or the runs will fail. > fs lq

One successful run text file outputs has about 10 MB.

All the ROOT files are saved on CASTOR which can be listed by:

> rfdir /castor/cern.ch/user/m/mzeman

% RECONSTRUCTION % source /afs/cern.ch/sw/contrib/CMT/v1r20p20080222/mgr/setup.sh %cmt co -r InDetCosmicRecExample? -00-03-62 %bacha castor 'afs' % DetDescrTag? = 'ATLAS-CommNF-02-00-00' % InputFile? = '/afs/cern.ch/user/j/jez/scratch0/newCosmics13/Simulation/G4Atlas/G4AtlasApps/run/digiID.pool.root' % athena ../share/InDetCosmicRec_topOptions.py % JiveXML? = True

-- MartinZeman - 24 Jul 2008