Difference: Tier2StorageAccess (1 vs. 11)

Revision 112015-05-29 - AndrzejOlszewski

Line: 1 to 1
 
META TOPICPARENT name="CyfronetWeb"

Access to disk storage for LHC experiments at Cyfronet

Line: 45 to 45
 

Data handling commands on grid enabled storage elements - direct access

XROOTD

Changed:
<
<
In ATLAS a global file access is being developed under name of FAX. This allows access of files without needing to know where the files are physically. This way of access is using a redirection mechanism available in xrootd and Scalla services of ROOT.
>
>
Access to files in ATLAS global name space is described at FAX.
 
Changed:
<
<
The xrootd access plugins have been installed on DPM servers at Polish Tier2 sites at Cyfronet and PSNC. Until FAX is fully operational one can use direct access to files available on these sites using this xrootd protocol which is fast an reliable.
>
>
The xrootd access plugins have been installed on DPM servers at Polish Tier2 sites at Cyfronet and PSNC.
 
Changed:
<
<
To get a path to any files on srm:
>
>
To use this mode of access:
 
  1. setup environment
    export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase
    source ${ATLAS_LOCAL_ROOT_BASE}/user/atlasLocalSetup.sh
    localSetupFAX
Changed:
<
<
  1. find site name of a dataset replica
    dq2-ls -r "dataset name"
  2. get path to files on srm
    dq2-ls -fp -L "site name" "dataset name"
  3. replace srm prefix by xrootd prefix
    CYFRONET: root://dpm.cyf-kr.edu.pl:1094/
    PSNC: root://se.reef.man.poznan.pl:1094/
  4. Example:
    dq2-ls -fp -L CYFRONET-LCG2_DATADISK user.ivukotic.xrootd.cyfronet-lcg2
    srm://dpm.cyf-kr.edu.pl/dpm/cyf-kr.edu.pl/home/atlas/atlasdatadisk/rucio/user/ivukotic/9b/43/user.ivukotic.xrootd.cyfronet-lcg2-1M
    convert ->
    root://dpm.cyf-kr.edu.pl:1094//dpm/cyf-kr.edu.pl/home/atlas/atlasdatadisk/rucio/user/ivukotic/9b/43/user.ivukotic.xrootd.cyfronet-lcg2-1M
    (note // after :1094)
To get a path to files from registered datasets:
  1. setup environment
    export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase
    source ${ATLAS_LOCAL_ROOT_BASE}/user/atlasLocalSetup.sh
    localSetupFAX
  2. check site where dataset replica exists
    dq2-ls -r "dataset name"
  3. set STORAGEPREFIX according to site you want to access
    CYFRONET: export STORAGEPREFIX=root://dpm.cyf-kr.edu.pl:1094
    PSNC: export STORAGEPREFIX=root://se.reef.man.poznan.pl:1094
    Global: export STORAGEPREFIX=root://atlas-xrd-eu.cern.ch:1094 (may not work yet)
  4. get file paths
    dq2-list-files -g "dataset name"
    convert them adding manuallY $STORAGEPREFIX
  5. Example:
    export STORAGEPREFIX=root://dpm.cyf-kr.edu.pl:1094
    for f in `dq2-list-files -g user.ivukotic.xrootd.cyfronet-lcg2`; do echo $STORAGEPREFIX$f'; done
To use a file in ROOT:
  • setup environment
    localSetupFAX
  • TFile::Open("xrootdpath")
In Athena job (probably, not tried):
InputCollections = [ 'xrootdpath' ]
>
>
  1. STORAGEPREFIX will be set to local gateway
    at CYFRONET: STORAGEPREFIX=root://dpm.cyf-kr.edu.pl:1094
    at PSNC: STORAGEPREFIX=root://se.reef.man.poznan.pl:1094
    Global: export STORAGEPREFIX=root://atlas-xrd-eu.cern.ch:1094 (may not work yet)
  2. RUCIO_ACCOUNT should be set your ATLAS nickname (by default set to local account name which may be wrong)
  3. VOMS proxy should be valid

Example of use:

  1. check if dataset is available in FAX
    fax-is-dataset-covered data12_8TeV.periodH2.physics_Muons.PhysCont.NTUP_SMWZ.grp13_v01_p1067/
  2. listpath to files of the dataset
    fax-get-gLFNs data12_8TeV.periodH2.physics_Muons.PhysCont.NTUP_SMWZ.grp13_v01_p1067/
  3. copy file to local storage
    xrdcp root://dpm.cyf-kr.edu.pl:1094//atlas/rucio/data12_8TeV:NTUP_SMWZ.01014068._000133.root.1 /tmp/ao.test
  4. or use fax-get tool
 
Deleted:
<
<
-- AndrzejOlszewski - 1 Dec 2013
 \ No newline at end of file
Added:
>
>
-- AndrzejOlszewski - 1 June 2015
 \ No newline at end of file

Revision 102014-03-05 - AndrzejOlszewski

Line: 1 to 1
 
META TOPICPARENT name="CyfronetWeb"

Access to disk storage for LHC experiments at Cyfronet

Line: 25 to 25
 
  • temporary scratch at /scratch-lustre/< user >

Grid authentication

Added:
>
>
  • set your proxy file to a fixed location, this will allow to use this authorization also from batch jobs
    export X509_USER_PROXY=/people/$USER/.globus/.vomsproxy
 
  • atlas voms proxy initilaization
    voms-proxy-init -valid 72:0 -voms atlas
  • or for proxy with "polish" group
    voms-proxy-init -valid 72:0 -voms atlas:/atlas/pl
Added:
>
>
  • then in batch jobs it is enough to define same X509_USER_PROXY in order to use the initialized proxy
 

Data handling commands on grid enabled storage elements - remote access

  • DPM SRM - dpm.cyf-kr.edu.pl - supports DPNS, RFIO, gridftp, srm protocols

Revision 92014-02-20 - AndrzejOlszewski

Line: 1 to 1
 
META TOPICPARENT name="CyfronetWeb"

Access to disk storage for LHC experiments at Cyfronet

Line: 52 to 52
 
  1. find site name of a dataset replica
    dq2-ls -r "dataset name"
  2. get path to files on srm
    dq2-ls -fp -L "site name" "dataset name"
  3. replace srm prefix by xrootd prefix
    CYFRONET: root://dpm.cyf-kr.edu.pl:1094/
    PSNC: root://se.reef.man.poznan.pl:1094/
Changed:
<
<
  1. Example:
    dq2-ls -fp -L CYFRONET-LCG2_DATADISK user.hito.xrootd.cyfronet-lcg2
    srm://dpm.cyf-kr.edu.pl/dpm/cyf-kr.edu.pl/home/atlas/atlasdatadisk/rucio/user/hito/6a/10/user.hito.xrootd.cyfronet-lcg2-1M
    convert ->
    root://dpm.cyf-kr.edu.pl:1094//dpm/cyf-kr.edu.pl/home/atlas/atlasdatadisk/rucio/user/hito/6a/10/user.hito.xrootd.cyfronet-lcg2-1M test
    (note // after :1094)
>
>
  1. Example:
    dq2-ls -fp -L CYFRONET-LCG2_DATADISK user.ivukotic.xrootd.cyfronet-lcg2
    srm://dpm.cyf-kr.edu.pl/dpm/cyf-kr.edu.pl/home/atlas/atlasdatadisk/rucio/user/ivukotic/9b/43/user.ivukotic.xrootd.cyfronet-lcg2-1M
    convert ->
    root://dpm.cyf-kr.edu.pl:1094//dpm/cyf-kr.edu.pl/home/atlas/atlasdatadisk/rucio/user/ivukotic/9b/43/user.ivukotic.xrootd.cyfronet-lcg2-1M
    (note // after :1094)
 To get a path to files from registered datasets:
  1. setup environment
    export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase
    source ${ATLAS_LOCAL_ROOT_BASE}/user/atlasLocalSetup.sh
    localSetupFAX
  2. check site where dataset replica exists
    dq2-ls -r "dataset name"
Changed:
<
<
  1. set STORAGEPREFIX according to site you want to access
    CYFRONET: export STORAGEPREFIX=root://dpm.cyf-kr.edu.pl:1094/
    PSNC: export STORAGEPREFIX=root://se.reef.man.poznan.pl:1094/
    Global: export STORAGEPREFIX=root://atlas-xrd-eu.cern.ch:1094/ (may not work yet)
  2. get file paths
    dq2-list-files -p "dataset name"
  3. Example:
    export STORAGEPREFIX=root://dpm.cyf-kr.edu.pl:1094/
    dq2-list-files -p user.hito.xrootd.cyfronet-lcg2
    root://dpm.cyf-kr.edu.pl:1094//atlas/dq2/user/hito/xrootd/user.hito.xrootd.cyfronet-lcg2/user.hito.xrootd.cyfronet-lcg2-1M
To copy a file:
  • setup environment
  • xrdcp root://"xrootdpath" "local path"
  • Example:
    xrdcp root://dpm.cyf-kr.edu.pl:1094//atlas/dq2/user/hito/xrootd/user.hito.xrootd.cyfronet-lcg2/user.hito.xrootd.cyfronet-lcg2-1M test
>
>
  1. set STORAGEPREFIX according to site you want to access
    CYFRONET: export STORAGEPREFIX=root://dpm.cyf-kr.edu.pl:1094
    PSNC: export STORAGEPREFIX=root://se.reef.man.poznan.pl:1094
    Global: export STORAGEPREFIX=root://atlas-xrd-eu.cern.ch:1094 (may not work yet)
  2. get file paths
    dq2-list-files -g "dataset name"
    convert them adding manuallY $STORAGEPREFIX
  3. Example:
    export STORAGEPREFIX=root://dpm.cyf-kr.edu.pl:1094
    for f in `dq2-list-files -g user.ivukotic.xrootd.cyfronet-lcg2`; do echo $STORAGEPREFIX$f'; done
 To use a file in ROOT:
Changed:
<
<
  • setup environment
    localSetupROOT
>
>
  • setup environment
    localSetupFAX
 
  • TFile::Open("xrootdpath")
In Athena job (probably, not tried):
InputCollections = [ 'xrootdpath' ]

Revision 82013-12-04 - AndrzejOlszewski

Line: 1 to 1
 
META TOPICPARENT name="CyfronetWeb"

Access to disk storage for LHC experiments at Cyfronet

Line: 41 to 41
 LCG commands allow action on both storage and LFC catalogs that keep info about files on the Grid.

Data handling commands on grid enabled storage elements - direct access

Changed:
<
<

XROOTD

>
>

XROOTD

  In ATLAS a global file access is being developed under name of FAX. This allows access of files without needing to know where the files are physically. This way of access is using a redirection mechanism available in xrootd and Scalla services of ROOT.

Revision 72013-12-02 - AndrzejOlszewski

Line: 1 to 1
 
META TOPICPARENT name="CyfronetWeb"

Access to disk storage for LHC experiments at Cyfronet

Line: 37 to 37
 
  • on srmV1 (port 8443):
    srm-get-metadata srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk/Andrzej.Olszewski/hijing.1GB.root.1
    srm-advisory-delete srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk/Andrzej.Olszewski/hijing.1GB.root.1
  • or on srmV2 (port 8446)
    srmls srm://dpm.cyf-kr.edu.pl:8446/dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk/Andrzej.Olszewski/hijing.1GB.root.1
    srmrm srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk/Andrzej.Olszewski/hijing.1GB.root.1
Changed:
<
<
  • lcg examples:
    lcg-cp --vo atlas --dst ATLASSCRATCHDISK file:////tmp/hijing.1GB.root \
    srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk/Andrzej.Olszewski/hijing.1GB.root.1
    lcg-cp --vo atlas srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlassratchdisk/Andrzej.Olszewski/hijing.1GB.root.1 file:////tmp/hijing.1GB.root
>
>
  • lcg examples:
    lcg-cp --vo atlas --dst ATLASSCRATCHDISK file:////tmp/hijing.1GB.root \
    srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk/Andrzej.Olszewski/hijing.1GB.root.1
    lcg-cp --vo atlas srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlassratchdisk/Andrzej.Olszewski/hijing.1GB.root.1 file:////tmp/hijing.1GB.root
 LCG commands allow action on both storage and LFC catalogs that keep info about files on the Grid.

Data handling commands on grid enabled storage elements - direct access

Deleted:
<
<

RFIO

export DPNS_HOST=dpm.cyf-kr.edu.pl
export DPM_HOST=dpm.cyf-kr.edu.pl
export LCG_RFIO_TYPE=dpm

ln -s $LCG_LOCATION/lib/libdpm.so libshift.so.2.1 (2.0 in older athena)
export LD_LIBRARY_PATH=$PWD:$LD_LIBRARY_PATH

In ROOT job:
f = new TRFIOFile("rfio:/dpm/cyf-kr.edu.pl/home/atlas/test.root")

In Athena job:
EventSelector.InputCollections = [ 'rfio:/dpm/cyf-kr.edu.pl/home/atlas/test.root' ]

 

XROOTD

In ATLAS a global file access is being developed under name of FAX. This allows access of files without needing to know where the files are physically. This way of access is using a redirection mechanism available in xrootd and Scalla services of ROOT.

The xrootd access plugins have been installed on DPM servers at Polish Tier2 sites at Cyfronet and PSNC. Until FAX is fully operational one can use direct access to files available on these sites using this xrootd protocol which is fast an reliable.

Changed:
<
<
To get a path to files:
>
>
To get a path to any files on srm:
  1. setup environment
    export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase
    source ${ATLAS_LOCAL_ROOT_BASE}/user/atlasLocalSetup.sh
    localSetupFAX
 
  1. find site name of a dataset replica
    dq2-ls -r "dataset name"
  2. get path to files on srm
    dq2-ls -fp -L "site name" "dataset name"
  3. replace srm prefix by xrootd prefix
    CYFRONET: root://dpm.cyf-kr.edu.pl:1094/
    PSNC: root://se.reef.man.poznan.pl:1094/
Added:
>
>
  1. Example:
    dq2-ls -fp -L CYFRONET-LCG2_DATADISK user.hito.xrootd.cyfronet-lcg2
    srm://dpm.cyf-kr.edu.pl/dpm/cyf-kr.edu.pl/home/atlas/atlasdatadisk/rucio/user/hito/6a/10/user.hito.xrootd.cyfronet-lcg2-1M
    convert ->
    root://dpm.cyf-kr.edu.pl:1094//dpm/cyf-kr.edu.pl/home/atlas/atlasdatadisk/rucio/user/hito/6a/10/user.hito.xrootd.cyfronet-lcg2-1M test
    (note // after :1094)
To get a path to files from registered datasets:
  1. setup environment
    export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase
    source ${ATLAS_LOCAL_ROOT_BASE}/user/atlasLocalSetup.sh
    localSetupFAX
  2. check site where dataset replica exists
    dq2-ls -r "dataset name"
  3. set STORAGEPREFIX according to site you want to access
    CYFRONET: export STORAGEPREFIX=root://dpm.cyf-kr.edu.pl:1094/
    PSNC: export STORAGEPREFIX=root://se.reef.man.poznan.pl:1094/
    Global: export STORAGEPREFIX=root://atlas-xrd-eu.cern.ch:1094/ (may not work yet)
  4. get file paths
    dq2-list-files -p "dataset name"
  5. Example:
    export STORAGEPREFIX=root://dpm.cyf-kr.edu.pl:1094/
    dq2-list-files -p user.hito.xrootd.cyfronet-lcg2
    root://dpm.cyf-kr.edu.pl:1094//atlas/dq2/user/hito/xrootd/user.hito.xrootd.cyfronet-lcg2/user.hito.xrootd.cyfronet-lcg2-1M
 To copy a file:
Changed:
<
<
  • example: xrdcp
>
>
  • setup environment
  • xrdcp root://"xrootdpath" "local path"
  • Example:
    xrdcp root://dpm.cyf-kr.edu.pl:1094//atlas/dq2/user/hito/xrootd/user.hito.xrootd.cyfronet-lcg2/user.hito.xrootd.cyfronet-lcg2-1M test
To use a file in ROOT:
  • setup environment
    localSetupROOT
  • TFile::Open("xrootdpath")
In Athena job (probably, not tried):
InputCollections = [ 'xrootdpath' ]
 -- AndrzejOlszewski - 1 Dec 2013 \ No newline at end of file

Revision 62013-12-02 - AndrzejOlszewski

Line: 1 to 1
 
META TOPICPARENT name="CyfronetWeb"
Changed:
<
<

Access to disk storage for LHC experiments at Cyfronet

>
>

Access to disk storage for LHC experiments at Cyfronet

 
Changed:
<
<

Resources requiring valid grid credentials to access their resources

  • DPM server providing access to large disk arrays with some 240 TB total disk space available.
    In 2009 some 160TB is assigned for Atlas experiment.
    Server address: srm://dpm.cyf-kr.edu.pl/dpm/cyf-kr.edu.pl/home/atlas
  • Atlas space is divided into space token reserved areas:
    • Production space: ATLASMCDISK, ATLASDATADISK, ATLASPRODDISK
      namespace
      /dpm/cyf-kr.edu.pl/home/atlas/(atlasmcdisk)(atlasdatadisk)(atlasproddidsk)
      access only for atlas users with production role
    • Physics group space: ATLASGROUPDISK
      namespace
      /dpm/cyf-kr.edu.pl/home/atlas/(atlasgroupdisk)
      transfer requests allowed only via Web form
    • User space: ATLASSCRATCHDISK
      namespace
      /dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk
      user scratch space, cleaned automatically after 30 days
    • Polish user space: ATLASLOCALGROUPDISK
      namespace
      /dpm/cyf-kr.edu.pl/home/atlas/atlaslocalgroupdisk
      permanent space for users with polish group certificate: /atlas/pl'
      Replication to this space is requested via WebForm
    • Polish user space: ATLASLOCALGROUPDISK
      namespace
      /dpm/cyf-kr.edu.pl/home/atlas/dq2localgroupdisk
      This namespace should be used for personal transfers using dq2 (lcg) client tools
>
>

Resources requiring valid grid credentials to access their resources

  • DPM server providing access to large disk arrays with some 240 TB total disk space available.
    In 2009 some 160TB is assigned for Atlas experiment.
    Server address: srm://dpm.cyf-kr.edu.pl/dpm/cyf-kr.edu.pl/home/atlas
  • Atlas space is divided into space token reserved areas:
    • Production space: ATLASMCDISK, ATLASDATADISK, ATLASPRODDISK
      namespace
      /dpm/cyf-kr.edu.pl/home/atlas/(atlasmcdisk)(atlasdatadisk)(atlasproddidsk)
      access only for atlas users with production role
    • Physics group space: ATLASGROUPDISK
      namespace
      /dpm/cyf-kr.edu.pl/home/atlas/(atlasgroupdisk)
      transfer requests allowed only via Web form
    • User space: ATLASSCRATCHDISK
      namespace
      /dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk
      user scratch space, cleaned automatically after 30 days
    • Polish user space: ATLASLOCALGROUPDISK
      namespace
      /dpm/cyf-kr.edu.pl/home/atlas/atlaslocalgroupdisk
      permanent space for users with polish group certificate: /atlas/pl'
      Replication to this space is requested via WebForm
    • Polish user space: ATLASLOCALGROUPDISK
      namespace
      /dpm/cyf-kr.edu.pl/home/atlas/dq2localgroupdisk
      This namespace should be used for personal transfers using dq2 (lcg) client tools
 

Resources requiring only local authentication

  • local /home and /tmp directories on WNs
Line: 31 to 24
 
  • not backed up data storage at /storage/< user >
  • temporary scratch at /scratch-lustre/< user >
Changed:
<
<

Grid authentication

  • atlas voms proxy initilaization
    voms-proxy-init -valid 72:0 -voms atlas
  • or for proxy with "polish" group
    voms-proxy-init -valid 72:0 -voms atlas:/atlas/pl
>
>

Grid authentication

  • atlas voms proxy initilaization
    voms-proxy-init -valid 72:0 -voms atlas
  • or for proxy with "polish" group
    voms-proxy-init -valid 72:0 -voms atlas:/atlas/pl
 

Data handling commands on grid enabled storage elements - remote access

  • DPM SRM - dpm.cyf-kr.edu.pl - supports DPNS, RFIO, gridftp, srm protocols
Changed:
<
<
  • Use rfio commands only for reading and storage cleanup
    In ~/.bash_profile
    export DPNS_HOST=dpm.cyf-kr.edu.pl
    export DPM_HOST=dpm.cyf-kr.edu.pl
    examples:
    rfdir /dpm/cyf-kr.edu.pl/home/atlas/dq2localgroupdisk/Andrzej.Olszewski/mc08
    rfcp /dpm/cyf-kr.edu.pl/home/atlas/dq2localgroupdisk/Andrzej.Olszewski/mc08/test.pool.root .
    quick remove of the whole directory tree:
    rfrm -fr /dpm/cyf-kr.edu.pl/home/atlas/dq2localgroupdisk/Andrzej.Olszewski/mc08/
  • Users should write only to spacetoken areas with commands like srmcp, lcg-cp.
    Space outside spacetoken will be cleaned from time to time without a Warning!
  • Using dq2 client tool to replicate dataset to ATLASLOCALGROUPDISK
    dq2-get -L CYFRONET-LCG2_LOCALGROUPDISK -k ATLASLOCALGROUPDISK \
    -S srm://dpm.cyf-kr.edu.pl:8446/srm/managerv2?SFN=/dpm/cyf-kr.edu.pl/home/atlas/dq2localgroupdisk/Andrzej.Olszewski/mc08 \
    mc08.008801.Hijing_PbPb_5p5TeV_MinBias.simul.HITS.e379_s501_tid040379
  • srm examples:
    srmcp -space_token=ATLASSCRATCHDISK file:////tmp/hijing.1GB.root \
    srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk/Andrzej.Olszewski/hijing.1GB.root.1
    srmcp srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk/Andrzej.Olszewski/hijing.1GB.root.1 file:////tmp/hijing.1GB.root

  • on srmV1 (port 8443):
    srm-get-metadata srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk/Andrzej.Olszewski/hijing.1GB.root.1
    srm-advisory-delete srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk/Andrzej.Olszewski/hijing.1GB.root.1
  • or on srmV2 (port 8446)
    srmls srm://dpm.cyf-kr.edu.pl:8446/dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk/Andrzej.Olszewski/hijing.1GB.root.1
    srmrm srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk/Andrzej.Olszewski/hijing.1GB.root.1
  • lcg examples:
    lcg-cp --vo atlas --dst ATLASSCRATCHDISK file:////tmp/hijing.1GB.root \
    srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk/Andrzej.Olszewski/hijing.1GB.root.1
    lcg-cp --vo atlas srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlassratchdisk/Andrzej.Olszewski/hijing.1GB.root.1 file:////tmp/hijing.1GB.root

>
>
  • Use rfio commands only for reading and storage cleanup
    In ~/.bash_profile
    export DPNS_HOST=dpm.cyf-kr.edu.pl
    export DPM_HOST=dpm.cyf-kr.edu.pl
    examples:
    rfdir /dpm/cyf-kr.edu.pl/home/atlas/dq2localgroupdisk/Andrzej.Olszewski/mc08
    rfcp /dpm/cyf-kr.edu.pl/home/atlas/dq2localgroupdisk/Andrzej.Olszewski/mc08/test.pool.root .
    quick remove of the whole directory tree:
    rfrm -fr /dpm/cyf-kr.edu.pl/home/atlas/dq2localgroupdisk/Andrzej.Olszewski/mc08/
  • Users should write only to spacetoken areas with commands like srmcp, lcg-cp.
    Space outside spacetoken will be cleaned from time to time without a Warning!
  • Using dq2 client tool to replicate dataset to ATLASLOCALGROUPDISK
    dq2-get -L CYFRONET-LCG2_LOCALGROUPDISK -k ATLASLOCALGROUPDISK \
    -S srm://dpm.cyf-kr.edu.pl:8446/srm/managerv2?SFN=/dpm/cyf-kr.edu.pl/home/atlas/dq2localgroupdisk/Andrzej.Olszewski/mc08 \
    mc08.008801.Hijing_PbPb_5p5TeV_MinBias.simul.HITS.e379_s501_tid040379
  • srm examples:
    srmcp -space_token=ATLASSCRATCHDISK file:////tmp/hijing.1GB.root \
    srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk/Andrzej.Olszewski/hijing.1GB.root.1
    srmcp srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk/Andrzej.Olszewski/hijing.1GB.root.1 file:////tmp/hijing.1GB.root

  • on srmV1 (port 8443):
    srm-get-metadata srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk/Andrzej.Olszewski/hijing.1GB.root.1
    srm-advisory-delete srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk/Andrzej.Olszewski/hijing.1GB.root.1
  • or on srmV2 (port 8446)
    srmls srm://dpm.cyf-kr.edu.pl:8446/dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk/Andrzej.Olszewski/hijing.1GB.root.1
    srmrm srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk/Andrzej.Olszewski/hijing.1GB.root.1
  • lcg examples:
    lcg-cp --vo atlas --dst ATLASSCRATCHDISK file:////tmp/hijing.1GB.root \
    srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk/Andrzej.Olszewski/hijing.1GB.root.1
    lcg-cp --vo atlas srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlassratchdisk/Andrzej.Olszewski/hijing.1GB.root.1 file:////tmp/hijing.1GB.root
  LCG commands allow action on both storage and LFC catalogs that keep info about files on the Grid.

Data handling commands on grid enabled storage elements - direct access

Added:
>
>

RFIO

export DPNS_HOST=dpm.cyf-kr.edu.pl
export DPM_HOST=dpm.cyf-kr.edu.pl
export LCG_RFIO_TYPE=dpm

ln -s $LCG_LOCATION/lib/libdpm.so libshift.so.2.1 (2.0 in older athena)
export LD_LIBRARY_PATH=$PWD:$LD_LIBRARY_PATH

 
Changed:
<
<
export DPNS_HOST=dpm.cyf-kr.edu.pl
export DPM_HOST=dpm.cyf-kr.edu.pl
export LCG_RFIO_TYPE=dpm
>
>
In ROOT job:
f = new TRFIOFile("rfio:/dpm/cyf-kr.edu.pl/home/atlas/test.root")
 
Changed:
<
<
ln -s $LCG_LOCATION/lib/libdpm.so libshift.so.2.1 (2.0 in older athena)
export LD_LIBRARY_PATH=$PWD:$LD_LIBRARY_PATH
>
>
In Athena job:
EventSelector.InputCollections = [ 'rfio:/dpm/cyf-kr.edu.pl/home/atlas/test.root' ]

XROOTD

 
Changed:
<
<
In ROOT job:
f = new TRFIOFile("rfio:/dpm/cyf-kr.edu.pl/home/atlas/test.root")
>
>
In ATLAS a global file access is being developed under name of FAX. This allows access of files without needing to know where the files are physically. This way of access is using a redirection mechanism available in xrootd and Scalla services of ROOT.
 
Changed:
<
<
In Athena job:
EventSelector.InputCollections = [ 'rfio:/dpm/cyf-kr.edu.pl/home/atlas/test.root' ]
>
>
The xrootd access plugins have been installed on DPM servers at Polish Tier2 sites at Cyfronet and PSNC. Until FAX is fully operational one can use direct access to files available on these sites using this xrootd protocol which is fast an reliable.
 
Deleted:
<
<
-- AndrzejOlszewski - 16 Mar 2009
 \ No newline at end of file
Added:
>
>
To get a path to files:
  1. find site name of a dataset replica
    dq2-ls -r "dataset name"
  2. get path to files on srm
    dq2-ls -fp -L "site name" "dataset name"
  3. replace srm prefix by xrootd prefix
    CYFRONET: root://dpm.cyf-kr.edu.pl:1094/
    PSNC: root://se.reef.man.poznan.pl:1094/
To copy a file:
  • example: xrdcp
-- AndrzejOlszewski - 1 Dec 2013
 \ No newline at end of file

Revision 52010-06-23 - AndrzejOlszewski

Line: 1 to 1
 
META TOPICPARENT name="CyfronetWeb"

Access to disk storage for LHC experiments at Cyfronet

Line: 26 to 26
  This namespace should be used for personal transfers using dq2 (lcg) client tools

Resources requiring only local authentication

Changed:
<
<
  • IDE disk: total ~100 GB mounted under common /home/people on all machines from EEGE cluster
  • Internal IDE disks on WN
  • access to local /home and /tmp directories, but only for temporary use during job execution
>
>
  • local /home and /tmp directories on WNs
  • backed up home at /people/< user >
  • not backed up data storage at /storage/< user >
  • temporary scratch at /scratch-lustre/< user >
 

Grid authentication

  • atlas voms proxy initilaization

Revision 42009-10-08 - AndrzejOlszewski

Line: 1 to 1
 
META TOPICPARENT name="CyfronetWeb"

Access to disk storage for LHC experiments at Cyfronet

Line: 72 to 72
 

Data handling commands on grid enabled storage elements - direct access

Added:
>
>
export DPNS_HOST=dpm.cyf-kr.edu.pl
export DPM_HOST=dpm.cyf-kr.edu.pl
 export LCG_RFIO_TYPE=dpm
Added:
>
>
ln -s $LCG_LOCATION/lib/libdpm.so libshift.so.2.1 (2.0 in older athena)
export LD_LIBRARY_PATH=$PWD:$LD_LIBRARY_PATH
 In ROOT job:
Changed:
<
<
f = new TGFALFile("gfal:rfio:////dpm/cyf-kr.edu.pl/home/atlas/test.root")
>
>
f = new TRFIOFile("rfio:/dpm/cyf-kr.edu.pl/home/atlas/test.root")
 In Athena job:
Changed:
<
<
pool_insertFileToCatalog gfal:rfio:////dpm/cyf-kr.edu.pl/home/atlas/test.root
then in script:
EventSelector.InputCollections = [ 'gfal:rfio:////dpm/cyf-kr.edu.pl/home/atlas/test.root' ]
>
>
EventSelector.InputCollections = [ 'rfio:/dpm/cyf-kr.edu.pl/home/atlas/test.root' ]
  -- AndrzejOlszewski - 16 Mar 2009 \ No newline at end of file

Revision 32009-03-27 - AndrzejOlszewski

Line: 1 to 1
 
META TOPICPARENT name="CyfronetWeb"

Access to disk storage for LHC experiments at Cyfronet

Line: 38 to 38
 

Data handling commands on grid enabled storage elements - remote access

  • DPM SRM - dpm.cyf-kr.edu.pl - supports DPNS, RFIO, gridftp, srm protocols
Added:
>
>
  • Use rfio commands only for reading and storage cleanup
    In ~/.bash_profile
    export DPNS_HOST=dpm.cyf-kr.edu.pl
    export DPM_HOST=dpm.cyf-kr.edu.pl
    examples:
    rfdir /dpm/cyf-kr.edu.pl/home/atlas/dq2localgroupdisk/Andrzej.Olszewski/mc08
    rfcp /dpm/cyf-kr.edu.pl/home/atlas/dq2localgroupdisk/Andrzej.Olszewski/mc08/test.pool.root .
    quick remove of the whole directory tree:
    rfrm -fr /dpm/cyf-kr.edu.pl/home/atlas/dq2localgroupdisk/Andrzej.Olszewski/mc08/
 
  • Users should write only to spacetoken areas with commands like srmcp, lcg-cp.
    Space outside spacetoken will be cleaned from time to time without a Warning!
  • Using dq2 client tool to replicate dataset to ATLASLOCALGROUPDISK
Line: 55 to 64
  srmls srm://dpm.cyf-kr.edu.pl:8446/dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk/Andrzej.Olszewski/hijing.1GB.root.1
srmrm srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk/Andrzej.Olszewski/hijing.1GB.root.1
  • lcg examples:
Changed:
<
<
lcg-cp --vo atlas cp --dest ATLASSCRATCHDISK file:////tmp/hijing.1GB.root \
srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk/Andrzej.Olszewski/hijing.1GB.root.1
>
>
lcg-cp --vo atlas --dst ATLASSCRATCHDISK file:////tmp/hijing.1GB.root \
srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk/Andrzej.Olszewski/hijing.1GB.root.1
  lcg-cp --vo atlas srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlassratchdisk/Andrzej.Olszewski/hijing.1GB.root.1 file:////tmp/hijing.1GB.root

LCG commands allow action on both storage and LFC catalogs that keep info about files on the Grid.

Revision 22009-03-19 - AndrzejOlszewski

Line: 1 to 1
 
META TOPICPARENT name="CyfronetWeb"

Access to disk storage for LHC experiments at Cyfronet

Line: 14 to 14
  * Physics group space: ATLASGROUPDISK
namespace
/dpm/cyf-kr.edu.pl/home/atlas/(atlasgroupdisk)
transfer requests allowed only via Web form
Changed:
<
<
* User space: ATLASUSERDISK, ATLASSCRATCHDISK
namespace
/dpm/cyf-kr.edu.pl/home/atlas/(atlasuserdisk)(atlasscratchdisk)
>
>
    • User space: ATLASSCRATCHDISK
      namespace
      /dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk
  user scratch space, cleaned automatically after 30 days
Changed:
<
<
* Polish user space: ATLASUSERDISK, ATLASLOCALGROUPDISK
>
>
    • Polish user space: ATLASLOCALGROUPDISK
 
namespace
/dpm/cyf-kr.edu.pl/home/atlas/atlaslocalgroupdisk
Changed:
<
<
permanent space for users with polish group certificate: /atlas/pl'
>
>
permanent space for users with polish group certificate: /atlas/pl'
Replication to this space is requested via WebForm
    • Polish user space: ATLASLOCALGROUPDISK
      namespace
      /dpm/cyf-kr.edu.pl/home/atlas/dq2localgroupdisk
      This namespace should be used for personal transfers using dq2 (lcg) client tools
 

Resources requiring only local authentication

  • IDE disk: total ~100 GB mounted under common /home/people on all machines from EEGE cluster
Line: 36 to 40
 
  • DPM SRM - dpm.cyf-kr.edu.pl - supports DPNS, RFIO, gridftp, srm protocols
  • Users should write only to spacetoken areas with commands like srmcp, lcg-cp.
    Space outside spacetoken will be cleaned from time to time without a Warning!
Added:
>
>
  • Using dq2 client tool to replicate dataset to ATLASLOCALGROUPDISK
    dq2-get -L CYFRONET-LCG2_LOCALGROUPDISK -k ATLASLOCALGROUPDISK \
    -S srm://dpm.cyf-kr.edu.pl:8446/srm/managerv2?SFN=/dpm/cyf-kr.edu.pl/home/atlas/dq2localgroupdisk/Andrzej.Olszewski/mc08 \
    mc08.008801.Hijing_PbPb_5p5TeV_MinBias.simul.HITS.e379_s501_tid040379
 
  • srm examples:
Changed:
<
<
srmcp -space_token=ATLASUSERDISK file:////tmp/hijing.1GB.root \
srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasuserdisk/Andrzej.Olszewski/hijing.1GB.root.1
srmcp srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasuserdisk/Andrzej.Olszewski/hijing.1GB.root.1 file:////tmp/hijing.1GB.root
>
>
srmcp -space_token=ATLASSCRATCHDISK file:////tmp/hijing.1GB.root \
srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk/Andrzej.Olszewski/hijing.1GB.root.1
srmcp srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk/Andrzej.Olszewski/hijing.1GB.root.1 file:////tmp/hijing.1GB.root
 
  • on srmV1 (port 8443):
Changed:
<
<
srm-get-metadata srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasuserdisk/Andrzej.Olszewski/hijing.1GB.root.1
srm-advisory-delete srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasuserdisk/Andrzej.Olszewski/hijing.1GB.root.1
>
>
srm-get-metadata srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk/Andrzej.Olszewski/hijing.1GB.root.1
srm-advisory-delete srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk/Andrzej.Olszewski/hijing.1GB.root.1
 
  • or on srmV2 (port 8446)
Changed:
<
<
srmls srm://dpm.cyf-kr.edu.pl:8446/dpm/cyf-kr.edu.pl/home/atlas/atlasuserdisk/Andrzej.Olszewski/hijing.1GB.root.1
srmrm srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasuserdisk/Andrzej.Olszewski/hijing.1GB.root.1
>
>
srmls srm://dpm.cyf-kr.edu.pl:8446/dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk/Andrzej.Olszewski/hijing.1GB.root.1
srmrm srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk/Andrzej.Olszewski/hijing.1GB.root.1
 
  • lcg examples:
Changed:
<
<
lcg-cp --vo atlas cp --dest ATLASUSERDISK file:////tmp/hijing.1GB.root \
srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasuserdisk/Andrzej.Olszewski/hijing.1GB.root.1
lcg-cp --vo atlas srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasuserdisk/Andrzej.Olszewski/hijing.1GB.root.1 file:////tmp/hijing.1GB.root

>
>
lcg-cp --vo atlas cp --dest ATLASSCRATCHDISK file:////tmp/hijing.1GB.root \
srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasscratchdisk/Andrzej.Olszewski/hijing.1GB.root.1
lcg-cp --vo atlas srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlassratchdisk/Andrzej.Olszewski/hijing.1GB.root.1 file:////tmp/hijing.1GB.root

  LCG commands allow action on both storage and LFC catalogs that keep info about files on the Grid.
Line: 55 to 63
 

Data handling commands on grid enabled storage elements - direct access

export LCG_RFIO_TYPE=dpm

Deleted:
<
<
 In ROOT job:
Changed:
<
<
f = new TGFALFile("gfal:rfio:////dpm/cyf-kr.edu.pl/home/atlas/test.root")
>
>
f = new TGFALFile("gfal:rfio:////dpm/cyf-kr.edu.pl/home/atlas/test.root")
 In Athena job:
Changed:
<
<
pool_insertFileToCatalog gfal:rfio:////dpm/cyf-kr.edu.pl/home/atlas/test.root

then in script:

EventSelector.InputCollections = [ 'gfal:rfio:////dpm/cyf-kr.edu.pl/home/atlas/test.root' ]
>
>
pool_insertFileToCatalog gfal:rfio:////dpm/cyf-kr.edu.pl/home/atlas/test.root
then in script:
EventSelector.InputCollections = [ 'gfal:rfio:////dpm/cyf-kr.edu.pl/home/atlas/test.root' ]
  -- AndrzejOlszewski - 16 Mar 2009 \ No newline at end of file

Revision 12009-03-16 - AndrzejOlszewski

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="CyfronetWeb"

Access to disk storage for LHC experiments at Cyfronet

Resources requiring valid grid credentials to access their resources

  • DPM server providing access to large disk arrays with some 240 TB total disk space available.
    In 2009 some 160TB is assigned for Atlas experiment.
    Server address: srm://dpm.cyf-kr.edu.pl/dpm/cyf-kr.edu.pl/home/atlas
  • Atlas space is divided into space token reserved areas:
    * Production space: ATLASMCDISK, ATLASDATADISK, ATLASPRODDISK
    namespace
    /dpm/cyf-kr.edu.pl/home/atlas/(atlasmcdisk)(atlasdatadisk)(atlasproddidsk)
    access only for atlas users with production role
    * Physics group space: ATLASGROUPDISK
    namespace
    /dpm/cyf-kr.edu.pl/home/atlas/(atlasgroupdisk)
    transfer requests allowed only via Web form
    * User space: ATLASUSERDISK, ATLASSCRATCHDISK
    namespace
    /dpm/cyf-kr.edu.pl/home/atlas/(atlasuserdisk)(atlasscratchdisk)
    user scratch space, cleaned automatically after 30 days
    * Polish user space: ATLASUSERDISK, ATLASLOCALGROUPDISK
    namespace
    /dpm/cyf-kr.edu.pl/home/atlas/atlaslocalgroupdisk
    permanent space for users with polish group certificate: /atlas/pl'

Resources requiring only local authentication

  • IDE disk: total ~100 GB mounted under common /home/people on all machines from EEGE cluster
  • Internal IDE disks on WN
  • access to local /home and /tmp directories, but only for temporary use during job execution

Grid authentication

  • atlas voms proxy initilaization
    voms-proxy-init -valid 72:0 -voms atlas
  • or for proxy with "polish" group
    voms-proxy-init -valid 72:0 -voms atlas:/atlas/pl

Data handling commands on grid enabled storage elements - remote access

  • DPM SRM - dpm.cyf-kr.edu.pl - supports DPNS, RFIO, gridftp, srm protocols
  • Users should write only to spacetoken areas with commands like srmcp, lcg-cp.
    Space outside spacetoken will be cleaned from time to time without a Warning!
  • srm examples:
    srmcp -space_token=ATLASUSERDISK file:////tmp/hijing.1GB.root \
    srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasuserdisk/Andrzej.Olszewski/hijing.1GB.root.1
    srmcp srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasuserdisk/Andrzej.Olszewski/hijing.1GB.root.1 file:////tmp/hijing.1GB.root

  • on srmV1 (port 8443):
    srm-get-metadata srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasuserdisk/Andrzej.Olszewski/hijing.1GB.root.1
    srm-advisory-delete srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasuserdisk/Andrzej.Olszewski/hijing.1GB.root.1
  • or on srmV2 (port 8446)
    srmls srm://dpm.cyf-kr.edu.pl:8446/dpm/cyf-kr.edu.pl/home/atlas/atlasuserdisk/Andrzej.Olszewski/hijing.1GB.root.1
    srmrm srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasuserdisk/Andrzej.Olszewski/hijing.1GB.root.1
  • lcg examples:
    lcg-cp --vo atlas cp --dest ATLASUSERDISK file:////tmp/hijing.1GB.root \
    srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasuserdisk/Andrzej.Olszewski/hijing.1GB.root.1
    lcg-cp --vo atlas srm://dpm.cyf-kr.edu.pl:8443/dpm/cyf-kr.edu.pl/home/atlas/atlasuserdisk/Andrzej.Olszewski/hijing.1GB.root.1 file:////tmp/hijing.1GB.root

LCG commands allow action on both storage and LFC catalogs that keep info about files on the Grid.

Data handling commands on grid enabled storage elements - direct access

export LCG_RFIO_TYPE=dpm

In ROOT job:

f = new TGFALFile("gfal:rfio:////dpm/cyf-kr.edu.pl/home/atlas/test.root")

In Athena job:

pool_insertFileToCatalog gfal:rfio:////dpm/cyf-kr.edu.pl/home/atlas/test.root

then in script:

EventSelector.InputCollections = [ 'gfal:rfio:////dpm/cyf-kr.edu.pl/home/atlas/test.root' ]

-- AndrzejOlszewski - 16 Mar 2009

 
This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback