Introduction

This is a CREAM test-suite developed using Robot Framework and python. It provides functional testing of CREAM services and related utilities. It is a work in progress and more test cases will be included in the future. The test suite supports Scientific Linux 5 and 6, as well as EMI 1 and EMI 2 releases. Also, it can easily be ported to any Linux platform. (anyone wanting to use the test suite in a non supported environment, please feel free to contact me)

For news and more info, you can follow us on Twitter and Posterous!

Scientific Linux 6 Installation

The test-suite has to be run in a host which has the capabilities to submit jobs to a CREAM endpoint. Namely, an EMI-UI is ideal for the job.

In order to install the test-suite, execute as root:

wget http://yum.gridctb.uoa.gr/repository/robot_testing_sl6.repo -O /etc/yum.repos.d/robot_testing_sl6.repo
yum install cream_test

(please note that the dependencies are automatically taken care of. Also,in previous versions, the robot_testing repository had to take precedence over the standard SL repos. This is no longer the case.)

If you have previously installed either the cream_test package, remove cream_test and then reinstall it:

yum remove cream_test
yum clean all
yum install cream_test

You can also install the RIDE editor. RIDE is a wysiwyg editor for Robot Framework's test case files. The available packages are for Scientific Linux only, but it can be easily installed in any Linux environment. In the case of SL, in order to install it, you first need to install wxPython with unicode support:

yum install wxPython
yum install robotframework-ride

If you want to use it by X11 forwarding, you must also have the xorg-x11-xauth package installed:

yum install xorg-x11-xauth

An alternative would be, to install it on your local machine and edit the remote test case files by mounting the remote host's file system through ssh for example.

If you only want to install robotframework,:

yum install robotframework

Scientific Linux 6 Dependencies

The available packages and their dependencies are:

Package	Dependency
cream_test	glite-ce-cream-cli
	lcg-util
	lfc
	uberftp
	subversion
	newt
	python-paramiko
	pexpect
	robotframework
robotframework	python >= 2.6 & < 3
Paramiko:	python-crypto
RIDE	python >= 2.6 & <3
	wxPython >= 2.8.6 with Unicode support

The packages needed by the test suite and their dependencies are provided in the aforementioned repository. Since the default Python for SL6 is version 2.6, there is no need for any external packages. All dependencies are either installed by default, or available in the standard repos (SL, EPEL etc)

Scientific Linux 5 Installation

The test-suite has to be run in a host which has the capabilities to submit jobs to a CREAM endpoint. Namely, an EMI-UI is ideal for the job.

In order to install the test-suite, execute as root:

wget http://yum.gridctb.uoa.gr/repository/robot_testing_sl5.repo -O /etc/yum.repos.d/robot_testing_sl5.repo
yum install cream_test

(please note that the dependencies are automatically taken care of. Also,in previous versions, the robot_testing repository had to take precedence over the standard SL repos. This is no longer the case.)

If you have previously installed either the cream_test package, remove cream_test and then reinstall it:

yum remove cream_test
yum clean all
yum install cream_test

You can also install the RIDE editor. RIDE is a wysiwyg editor for Robot Framework's test case files. The available packages are for Scientific Linux only, but it can be easily installed in any Linux environment. In the case of SL, in order to install it, you first need to install wxPython for python26 with unicode support:

yum --disablerepo "*" --enablerepo "robot_testing_sl5" install wxPython
yum install robotframework-ride

If you want to use it by X11 forwarding, you must also have the xorg-x11-xauth package installed:

yum install xorg-x11-xauth

An alternative would be, to install it on your local machine and edit the remote test case files by mounting the remote host's file system through ssh for example.

If you only want to install robotframework, you must first install python26:

yum install python26
yum install robotframework

Scientific Linux 5 Dependencies

The available packages and their dependencies are:

Package	Dependency
cream_test	glite-ce-cream-cli
	lcg-util
	lfc
	uberftp
	subversion
	newt
	python26-paramiko
	python26-pexpect
	robotframework
robotframework	python >= 2.6 & < 3
Paramiko:	python26-crypto
RIDE	python >= 2.6 & <3
	wxPython >= 2.8.6 with Unicode support

The packages needed by the test suite are provided in the aforementioned repository. Also, python26-pexpect and wxPython with unicode support for python26 are provided by our repository. The test suite needs python26 installed, since the default python for Scientific Linux 5 is 2.4. All other dependencies are either installed by default, or available in the standard repos (SL, EPEL etc)

Installed files

The CREAM testsuite installs the following files:

File	Description
/opt/cream_test/testsuite/cream_test_suite.html	The html testing module for Robot Framework
/opt/cream_test/lib/cream_testing.py	The python module with the methods/keywords needed by the test
/opt/cream_test/lib/vars.py	Python module, containing the variables needing to be set for the test
/opt/cream_test/bin/setup_keys.sh	Bash script, utilizing newt, to automatically set up ssh key authentication with needed hosts
/opt/cream_test/docs/cream_testing_keywords.html	Html documentation of the keywords provided by the test library
/opt/cream_test/docs/cream_testing_libdoc.html	Html Python module documentation for the test library
/opt/cream_test/docs/Cream_Test_Suite-doc.html	Html test suite documentation
/opt/cream_test/docs/COPYING	Copyright document
/opt/cream_test/docs/cream_test.7.gz and /usr/share/man/man7/cream_test.7.gz	The test suite's man page
/opt/cream_test/docs/CHANGELOG	The test suite's changelog

Also, once installed, the man page "cream_test" is available.

Deployment

Once the cream test package group is installed, you have to deploy the test to your environment. First of all,you can do a:

man cream_test

to view information about the testsuite and how to deploy it. There are also available these tutorial videos, but please note that they are obsolete. New ones will be uploaded as soon as possible:

• Repository Installation
• Test Suite Installation
• Test Suite Deployment

In order to deploy the test a set of variables that depend on the testing environment must be configured accordingly.

SSH Key Authentication

In order to use the test suite, the host executing it must have automatic ssh access through keys, to some remote service hosts, namely CREAM, LRMS and ARGUS. A special, interactive dialog script is provided in order to help with creating and deploying those keys quickly and effortlessly:

/opt/cream_test/bin/setup_keys.sh

Standard Deployment Method

In order to do so, edit the file

 /opt/cream_test/lib/vars.py 

with the desired values. Examples are provided in the file itself, in the man page and in later parts of this document. Then you can execute the test suite.

Deploy with RIDE

If you have a graphical environment available or the ability to X-forward GUI applications from the testing host to a graphical environment, you can install the RIDE editor. RIDE is a WYSIWYG editor for Robot Framework test cases. This might be the most intuitive way to create a new test, since it exposes the test-suite logic to the user.

1. Create a copy of the file /opt/cream_test/lib/vars.py
2. Edit it, assigning the desired values
3. Create a copy of the file /opt/cream_test/testsuite/cream_test_suite.html
4. Edit it with the RIDE editor, changing the path pointing to the variables file (by selecting the test suite's name in the list at the left part of the screen)
5. Save the produced file

Test Execution

In the directory in which the test-suite HTML file resides,run:

pybot -W $COLUMNS /relative/or/absolute/path/to/test/html/file

The -W flag is used to tell pybot how many columns of text to print. The variable COLUMNS contains the number of columns in the current shell by default, so it is used for convenience. Unless changed from the original deployment scenario, the test suite file is /opt/cream_test/testsuite/cream_test_suite.html.

Advanced Test Execution

1. If you want to execute a certain test case in the test suite,run:

pybot -i test_case_tag testsuite.html

Multiple test cases can be included in the same way:

pybot -i tag1 -i tag2 testsuite.html

2. If you want to exclude a test case from execution,run:

pybot -e test_case_tag testsuite.html

Multiple test cases can be excluded in the same way:

pybot -e tag1 -e tag2 testsuite.html

3. You can set the log level with the -L flag. Possible values are (from lowest to highest): WARN, INFO, DEBUG, TRACE. Example:

pybot -L TRACE testsuite.html

4. You can set whether a test case is critical or not. This means that if a test case isn't critical (by default all are), if it fails, the test-suite will not fail. If a critical test case fails, the test-suite fails. This doesn't affect whether the rest of the test cases will be executed, it affects only the final test report. Examples:

pybot --critical tag testsuite.html
pybot --noncritical tag testsuite.html

Test Suite Description

Variables

The test's variables have explanations and examples in both the test-suite's HTML file and the variables.txt file. Here follows the same info:

Variable Name	Description
ce_endpoint	The cream endpoint to be used (e.g.: ctb04.gridctb.uoa.gr:8443 )
cream_queue	The cream queue to be used (e.g.: cream-pbs-see )
deleg_endpoint	The CREAM endpoint in use for delegation (e.g.: https://ctb04.gridctb.uoa.gr:8443//ce-cream/services/gridsite-delegation)
vo	The user’s submitting the jobs virtual organisation (e.g.: see )
proxy_pass	The user’s submitting the jobs proxy password (e.g.: p4sSw0rD )
gridftp_server	Gridftp server used for data transfers (e.g.: se01.isabella.grnet.gr )
lfn_dir	LFC directory for data transfers. This directory will be created, it doesn’t have to allready exist. Example: /grid/dteam/cream_testing
deleg_id	A delegation id string (e.g.: me_deleg_id )
log_level	The log level used during the test.Default is INFO.For extra output,set to DEBUG or TRACE. (possible values: NONE FAIL WARN INFO DEBUG TRACE)
batch_system	The underlying batch system of the CREAM endpoint.Either pbs or lsf.
gridftp_dir	A directory in the gridftp server.This directory has to allready exist and your vo have write access to it. Used for OSB file storage. Example: /tmp
lrms_host	The hostname where TORQUE is running. Example: ctb07.gridctb.uoa.gr
tmp_dir	The path in which temporary files will reside. They will be automatically cleaned up unless you set the variable delete_files to "False" or explicitely don’t run the cleanup test case. The path will be created -with its parents-, it doesn’t have to exist. You can leave it empty and a temporary directory will be created for you. In order to know which temp random directory is used, it is printed in standard output and in the final test suite report. Warning: any parent directories created, are not removed! All in all, unless needed for specific reasons, you should leave this variable empty.
delete_files	Delete temporary files (jdl and script files created during the test) or not. Possible values: True False. Defaults to "True"
sec_cert	Path to a second user certificate.
sec_key	Path to a second user key.
sec_proxy_pass	Password for the aforementioned second certificate.
creamdb_host	The host of the cream database.
creamdb_port	The port in which the cream database is listening for connections.
creamdb_user	The user of the cream data base
creamdb_pass	The database password for the aforementioned user
authz_model	The authorization model in use by CREAM. Either gjaf or argus.
argus_host	The host of the argus service, if applicable.
middleware_version	The version of the middleware being tested. Either EMI1 or EMI2.
cream_ldap_port	The ldap port for the CREAM resource bdii. Defaults to 2170.
lfc_host	The LFC_HOST environmental variable to be set on the WN for output data operations.
lcg_gfal_infosys	The LCG_GFAL_INFOSYS environmental variable to be set on the WN for outputdata operations
blparser_host	The (old) blparser host. Defaults to the ce_endpoint. Could also be lrms_host, or a whole different host

Test Cases

Expand All + - TEST SUITE: Cream Test Suite

Full Name: Cream Test Suite Documentation: This is the main testing module,needed for testing the CREAM functionality.For more information check these urls: CREAM User's Guide: http://wiki.italiangrid.org/twiki/bin/view/CREAM/UserGuide JDL Guide: http://wiki.italiangrid.org/twiki/bin/view/CREAM/JdlGuide CREAM Administrator's Guide: http://wiki.italiangrid.org/twiki/bin/view/CREAM/SystemAdministratorGuideForEMI1 CREAM Test Suite Guide: https://twiki.cern.ch/twiki/bin/view/EMI/CREAMRobotFuncTests EMI Robot Framework Quickstart Guide: https://twiki.cern.ch/twiki/bin/view/EMI/RobotFrameworkQuickstartGuide Temporary Files Location: ${tmp_dir} Source: /home/dfiore/Desktop/untitled folder/cream_test_suite.html Number of Tests: 39 - + TEST CASE: Set Log Level Full Name: Cream Test Suite.Set Log Level Documentation: Set the log level used for the test suite. This changes the verbosity of the final test suite log files. Tags: loglevel, setup - + TEST CASE: Check Executables Full Name: Cream Test Suite.Check Executables Documentation: Check whether the needed executables for the test are available in the underlying system. Tags: check_exe, setup - + TEST CASE: Simple Submit Full Name: Cream Test Suite.Simple Submit Documentation: Execute /bin/uname -a.Submit the jdl and wait for final job status to be done-ok. Tags: authz, blparser, simple - + TEST CASE: ISB Base URI Full Name: Cream Test Suite.ISB Base URI Documentation: Execute a bash shell script.The script is stored in a gridftp server.The jdl attribute InputSandboxBaseURI is used.First upload the executable script,then submit the jdl and the final state should be done-ok. Tags: isb_baseuri - + TEST CASE: ISB Client to CE Full Name: Cream Test Suite.ISB Client to CE Documentation: Execute a bash shell script.The script is stored on the UI.The jdl attribute InputSandbox is used.Firstly submit the jdl and then wait for final job state to be done-ok. Tags: isb_client - + TEST CASE: ISB gsiftp to CE Full Name: Cream Test Suite.ISB gsiftp to CE Documentation: Execute a bash shell script.The script is stored in a gridftp server.The jdl attribute InputSandbox is used.First upload the executable script,then submit the jdl and the final state should be done-ok. Tags: isb_gsiftp - + TEST CASE: OSB localhost Full Name: Cream Test Suite.OSB localhost Documentation: Execute /bin/uname -a.Store std out and error streams on "gsiftp://localhost" .Firstly submit the jdl,wait for final job state done-ok,download the produced files,get the output file list from the jdl file and compare the two file lists. Tags: osb_localhost - + TEST CASE: OSB gsiftp BaseDestURI Full Name: Cream Test Suite.OSB gsiftp BaseDestURI Documentation: Execute /bin/uname -a.Store std out and error streams on a gridftp server.The jdl attribute OutputSandboxBaseDestURI is used. 2) Get output file list from jdl,base dest uri. 3) lcg-ls base dest uri and files shouldn't exist 4) Execute jdl 5) lcg-ls base dest uri and files should exist 6) Delete files (cleanup) 7) lcg-ls base dest uri and files shouldn't exist Tags: osb_basedesturi - + TEST CASE: OSB gsiftp DestURI Full Name: Cream Test Suite.OSB gsiftp DestURI Documentation: Execute /bin/uname -a.Store std out and error streams on a gridftp server.The jdl attribute OutputSandboxDestURI is used. 2) Get output file list from jdl,base dest uri. 3) lcg-ls base dest uri and files shouldn't exist 4) Execute jdl 5) lcg-ls base dest uri and files should exist 6) Delete files (cleanup) 7) lcg-ls base dest uri and files shouldn't exist Tags: osb_desturi - + TEST CASE: Environment Full Name: Cream Test Suite.Environment Documentation: Execute a bash shell script.The script is stored locally.The jdl sets an environmental variable.The script prints the value of that variable. Submit the jdl,the final state should be done-ok and the output file should have the designated variable. Tags: env - + TEST CASE: Epilogue Full Name: Cream Test Suite.Epilogue Documentation: Execute two bash shell scripts,one for the job and the other as an epilogue.The epilogue script creates a file with a certain string,which is collected by CREAM as an output file.Submit the jdl,the final state should be done ok,and the output file should have the designated string. Tags: epilogue - + TEST CASE: Prologue Full Name: Cream Test Suite.Prologue Documentation: Execute two bash shell scripts,one for the job and the other as a prologue.The prologue script creates a file with a certain string,which is collected by CREAM as an output file.Submit the jdl,the final state should be done ok,and the output file should have the designated string. Tags: prologue - + TEST CASE: Explicit Delegation Full Name: Cream Test Suite.Explicit Delegation Documentation: Test job submission with explicit delegation.Three cases are tested: 1) Delegate a proxy and submit a job (job final state should be done-ok) 2) Create an allready existing delegation id (failure is treated as correct behaviour) 3) Submit job with explicit delegation of a non existant delegation id (failure is treated as correct behaviour) Tags: deleg - + TEST CASE: Job Cancel Full Name: Cream Test Suite.Job Cancel Documentation: Test job cancellation.Two cases are tested: 1) Submit and then cancel a job.The final state should be cancelled. 2) Cancel the aforementioned job again (failure is treated as correct behaviour). Tags: blparser, cancel - + TEST CASE: Job List Full Name: Cream Test Suite.Job List Documentation: Test the job listing cli utility.Store the list of the user's jobs,before and after a job submission.The returned job id shouldn't exist in the job list command output before the submission and should exist afterwards. Tags: list - + TEST CASE: Job Suspend - Resume Full Name: Cream Test Suite.Job Suspend - Resume Documentation: Submit a job and suspend it.Wait until it's state is REALLY-RUNNING. Case LSF batch system: It's state should be HELD.Then resume the same job.It's status should be REALLY-RUNNING.The final status should be DONE-OK. Case Torque batch system: It's state should still be REALLY-RUNNING.Resume the job and expect an error.It's status should be REALLY-RUNNING.The final status should be DONE-OK. Tags: susp_res - + TEST CASE: Manually Cancel Job Full Name: Cream Test Suite.Manually Cancel Job Documentation: Submit a job and then manually cancel it on the LRMS host. The final status should be either "DONE-FAILED" or "CANCELLED" Internally, ssh is used to connect and execute qstat and qdel on the Torque host by a torque admin user. Tags: man_cancel - + TEST CASE: Verify CE service info Full Name: Cream Test Suite.Verify CE service info Documentation: This test case verifies the output of the glite-ce-service-info command for both verbosity levels. Tags: ce_info - + TEST CASE: Enable - Disable Submission Full Name: Cream Test Suite.Enable - Disable Submission Documentation: This test case tests the proper functionality of the glite-ce-allowed-submission and glite-ce-enable/disable-submission commands, for admin and non-admin users. The test executes the following steps: 1) Checks if initially the submission is allowed (it should) 2) Executes a simple job (it should succeed) 3) Disables submission (the current user executing the test case is considered to NOT be a CREAM admin, thus this step should fail) 4) Add the current user as a CREAM admin 5) Disable submission (this time it should succeed) 6) Check if submission is allowed (it shouldn't) 7) Execute a simple job submission (it should fail) 8) Enable submission 9) Execute a simple job (it should succeed) 10) Remove current user from CREAM admin 11) Disable submission (should fail, since the user isn't a CREAM admin anymore) This test case internally uses root ssh access to the cream host, in order to add/remove a cream administrator. Tags: en_dis_sub - + TEST CASE: Job Purge Full Name: Cream Test Suite.Job Purge Documentation: This test case verifies the proper functionality of the glite-ce-job-purge command. In order to do that, it purges a job and then searches for its status and its sandbox dirs on the cream node. The job shouldn't be found by the status command and the sandbox dirs shouldn't exist. Tags: purge - + TEST CASE: Admin Manage Job Full Name: Cream Test Suite.Admin Manage Job Documentation: This test case checks whether a user can manage other people's jobs when being or not being an admin. Test case algorithm: 1) Submit a sleep job and wait until it reaches the REALLY-RUNNING state 2) Create a new proxy with different credentials that the default (a second cert and key are required for this, pointed to by the relevant variables) 3) Try to cancel the job (should fail, since the user with the second proxy isn't an admin) 4) Make the user with the second proxy a cream admin 5) Try to cancel the job (should succeed, since the user with the second proxy is an admin now) 6) Remove the user with the second proxy from the admin list. Tags: job_man - + TEST CASE: Limiter Full Name: Cream Test Suite.Limiter Documentation: NOTE: This test case is about 25 minutes long! This test case, tests the functionality of the CREAM self limiting behaviour. It has the following logic: 1) Set a limiter threshold to a value which will triger it 2) Wait for 11 minutes for the limiter script to be run automatically 3) Submit and job and expect failure 4) Reset the limiter to the original values 5) Wait for 11 minutes for the limiter script to be run automatically 6) Submit a command and succeed Tags: limit - + TEST CASE: Ban User Full Name: Cream Test Suite.Ban User Documentation: This test case tests if the banning of user works. The test case has the following logic: 1) Add the user's DN to /etc/lcas/ban_users.db on the cream host -OR- 1) Create a ban rule for the user's DN on the Argus host 2) Submit job and expect failure 3) Remove DN from the aforementioned file 4) Submit job and succeed Tags: authz, ban - + TEST CASE: Sandbox Staging Full Name: Cream Test Suite.Sandbox Staging Documentation: This test case tests the two different Sandbox staging operations available for the /etc/glite-ce-cream/cream-config.xml, LRMS and GSIFTP. The test case has the following logic: 1) Change the setting (if GSIFTP, set to LRMS, if LRMS, set to GSIFTP) 2) Run the osb_localhost test case (it uses input and output sandboxes) 3) Reset the setting to its original and rerun the osb_localhost test case. Tags: sb_stage - + TEST CASE: Validate Glue Schema Full Name: Cream Test Suite.Validate Glue Schema Documentation: Run the glue validator on the cream host. The following two cases are tested: 1) o=grid glue1 2) o=glue glue2 Tags: glue - + TEST CASE: Automatic Proxy Purging Full Name: Cream Test Suite.Automatic Proxy Purging Documentation: This test case checks the automatic proxy purging functionality. Test case algorithm: 1) Modify the cream config xml file to set the delegation purge rate to value 1 2) Submit a simple job and wait for it to finish 3) Read the delegation id info from the CREAM service's delegation endpoint (before the purger runs) 4) List the proxy in the job's sandbox (before the purger runs) 5) Wait 6 minutes for the purger to run 6) Read the delegation ids info from the CREAM service's delegation endpoint (the purger now has ran) 7) List the proxy in the job's sandbox (the purger now has ran) 8) Reset the cream config xml to its original value 9) Check that the info about the delegation id is valid before the purger has ran and it isn't in storage after the purger has ran. 10) Check tha the delegation id of the job exists in the contents of the job's sandbox before the purger ran and doesn't exist after the purger ran Tags: proxy_purge - + TEST CASE: Verify Job Status Output Full Name: Cream Test Suite.Verify Job Status Output Documentation: This test is checking the output of the glite-ce-job-status command. It uses the following status cases: 1) verbosity 0 2) verbosity 1 3) verbosity 2 4) use the "from" filter 5) use the "from" and "to" filters 6) use the "to" filter 7) use the "status" filter For this test it is crucial that the ce host, the ui host and the testing host are time-synced. If the aren't synced, most probably this test case will fail when checking the "from" and "to" filters. Tags: status - + TEST CASE: OutputData Full Name: Cream Test Suite.OutputData Documentation: This test case tests the OutputData jdl attribute. The testing algorithm is the following: 1) Create and submit three jdls containing the output data attribute, in three different combinations (simple, with gridftp server, with gridftp server and lfn directory) 2) Check the final status of those jobs, which should be DONE-OK 3) Gather the output DSUpload files for each job and extract from them the relevant guids of the uploaded files 4) Download the uploaded files and check that they are not empty Note: The jdl is executing a simple script which runs "/bin/uname -a" and stores its output in a text file "script_output.txt" Tags: outputdata - + TEST CASE: Automatic Job Purge Full Name: Cream Test Suite.Automatic Job Purge Documentation: This test case is testing the functionality of the JobDBAdminPurger script. Test case algo: 1) Submit a job 2) Wait for it to finish 3) Purge it with the use of the JobDBAdmin purger script 4) Do an uberftp ls operation on its sandbox dir (which should fail) 5) Do a status request operation (glite-ce-job-status, which should fail) Tags: auto_job_purge - + TEST CASE: Proxy Renewal Full Name: Cream Test Suite.Proxy Renewal Documentation: Submit a job lasting 20 minutes using a proxy valid for 10 minutes. After 5 minutes renew the proxy with the glite-ce-proxy-renew command. Verify the final job status which should eventually be DONE-OK Tags: renew - + TEST CASE: CPU Allocation 1 Full Name: Cream Test Suite.CPU Allocation 1 Documentation: Validate the contents of the batch job submission script, according to: https://wiki.italiangrid.it/twiki/bin/view/CREAM/CreamTestWorkPlan cpu allocation test specifications. Per test information follows: Wherever it appears,S is the value published as GlueHostArchitectureSMPSize Test type 1 Jdl Attributes: WholeNodes=true SMPGranularity=G HostNumber=H, H>1 Verification for LSF: BSUB -n S*H, BSUB -R "span[ptile=S]", BSUB -x Verification for PBS: PBS -l nodes=H:ppn=S, PBS -W x=NACCESSPOLICY:SINGLEJOB Test type 2 JDL Attributes: WholeNodes=true SMPGranularity=G Verification for LSF: BSUB -n S, BSUB -R "span[jpsts=1]", BSUB -x Verification for PBS: PBS -l nodes=1:ppn=S, PBS -W x=NACCESSPOLICY:SINGLEJOB Test type 3 JDL Attributes: WholeNodes=true, HostNumber=H Verification for LSF: BSUB -n S*H, BSUB -R "span[ptiles=S]", BSUB -x Verification for PBS: PBS -l nodes=H:ppn=S, PBS -W x=NACCESSPOLICY:SINGLEJOB Test type 4 JDL Attributes: WholeNodes=false, SMPGranularity=G, CPUNumber=C Verification for LSF: BSUB -n C, BSUB -R "span[ptile=G]" Verification for PBS: PBS -l nodes=N:ppn=G { [+1:ppn=R] if r>0 } Test type 5 JDL Attributes: WholeNodes=false, HostNumber=H, CPUNumber=C, H>1 Verification for LSF: BSUB -n C, BSUB -R "span[ptile={ N if R=0 ; N+1 if R>0 }]" Verification for PBS: PBS -l nodes=H-R:ppn=N { [+R:ppn=N+1] if R>0 } Test type 6 JDL Attributes: WholeNodes=false, CPUNumber=C Verification for LSF: BSUB -n C Verification for PBS: PBS -L nodes=C Tags: cpu_alloc, cpu_alloc1 - + TEST CASE: CPU Allocation 2 Full Name: Cream Test Suite.CPU Allocation 2 Documentation: Validate the contents of the batch job submission script, according to: https://wiki.italiangrid.it/twiki/bin/view/CREAM/CreamTestWorkPlan cpu allocation test specifications. Per test information follows: Wherever it appears,S is the value published as GlueHostArchitectureSMPSize Test type 1 Jdl Attributes: WholeNodes=true SMPGranularity=G HostNumber=H, H>1 Verification for LSF: BSUB -n S*H, BSUB -R "span[ptile=S]", BSUB -x Verification for PBS: PBS -l nodes=H:ppn=S, PBS -W x=NACCESSPOLICY:SINGLEJOB Test type 2 JDL Attributes: WholeNodes=true SMPGranularity=G Verification for LSF: BSUB -n S, BSUB -R "span[jpsts=1]", BSUB -x Verification for PBS: PBS -l nodes=1:ppn=S, PBS -W x=NACCESSPOLICY:SINGLEJOB Test type 3 JDL Attributes: WholeNodes=true, HostNumber=H Verification for LSF: BSUB -n S*H, BSUB -R "span[ptiles=S]", BSUB -x Verification for PBS: PBS -l nodes=H:ppn=S, PBS -W x=NACCESSPOLICY:SINGLEJOB Test type 4 JDL Attributes: WholeNodes=false, SMPGranularity=G, CPUNumber=C Verification for LSF: BSUB -n C, BSUB -R "span[ptile=G]" Verification for PBS: PBS -l nodes=N:ppn=G { [+1:ppn=R] if r>0 } Test type 5 JDL Attributes: WholeNodes=false, HostNumber=H, CPUNumber=C, H>1 Verification for LSF: BSUB -n C, BSUB -R "span[ptile={ N if R=0 ; N+1 if R>0 }]" Verification for PBS: PBS -l nodes=H-R:ppn=N { [+R:ppn=N+1] if R>0 } Test type 6 JDL Attributes: WholeNodes=false, CPUNumber=C Verification for LSF: BSUB -n C Verification for PBS: PBS -L nodes=C Tags: cpu_alloc, cpu_alloc2 - + TEST CASE: CPU Allocation 3 Full Name: Cream Test Suite.CPU Allocation 3 Documentation: Validate the contents of the batch job submission script, according to: https://wiki.italiangrid.it/twiki/bin/view/CREAM/CreamTestWorkPlan cpu allocation test specifications. Per test information follows: Wherever it appears,S is the value published as GlueHostArchitectureSMPSize Test type 1 Jdl Attributes: WholeNodes=true SMPGranularity=G HostNumber=H, H>1 Verification for LSF: BSUB -n S*H, BSUB -R "span[ptile=S]", BSUB -x Verification for PBS: PBS -l nodes=H:ppn=S, PBS -W x=NACCESSPOLICY:SINGLEJOB Test type 2 JDL Attributes: WholeNodes=true SMPGranularity=G Verification for LSF: BSUB -n S, BSUB -R "span[jpsts=1]", BSUB -x Verification for PBS: PBS -l nodes=1:ppn=S, PBS -W x=NACCESSPOLICY:SINGLEJOB Test type 3 JDL Attributes: WholeNodes=true, HostNumber=H Verification for LSF: BSUB -n S*H, BSUB -R "span[ptiles=S]", BSUB -x Verification for PBS: PBS -l nodes=H:ppn=S, PBS -W x=NACCESSPOLICY:SINGLEJOB Test type 4 JDL Attributes: WholeNodes=false, SMPGranularity=G, CPUNumber=C Verification for LSF: BSUB -n C, BSUB -R "span[ptile=G]" Verification for PBS: PBS -l nodes=N:ppn=G { [+1:ppn=R] if r>0 } Test type 5 JDL Attributes: WholeNodes=false, HostNumber=H, CPUNumber=C, H>1 Verification for LSF: BSUB -n C, BSUB -R "span[ptile={ N if R=0 ; N+1 if R>0 }]" Verification for PBS: PBS -l nodes=H-R:ppn=N { [+R:ppn=N+1] if R>0 } Test type 6 JDL Attributes: WholeNodes=false, CPUNumber=C Verification for LSF: BSUB -n C Verification for PBS: PBS -L nodes=C Tags: cpu_alloc, cpu_alloc3 - + TEST CASE: CPU Allocation 4 Full Name: Cream Test Suite.CPU Allocation 4 Documentation: Validate the contents of the batch job submission script, according to: https://wiki.italiangrid.it/twiki/bin/view/CREAM/CreamTestWorkPlan cpu allocation test specifications. Per test information follows: Wherever it appears,S is the value published as GlueHostArchitectureSMPSize Test type 1 Jdl Attributes: WholeNodes=true SMPGranularity=G HostNumber=H, H>1 Verification for LSF: BSUB -n S*H, BSUB -R "span[ptile=S]", BSUB -x Verification for PBS: PBS -l nodes=H:ppn=S, PBS -W x=NACCESSPOLICY:SINGLEJOB Test type 2 JDL Attributes: WholeNodes=true SMPGranularity=G Verification for LSF: BSUB -n S, BSUB -R "span[jpsts=1]", BSUB -x Verification for PBS: PBS -l nodes=1:ppn=S, PBS -W x=NACCESSPOLICY:SINGLEJOB Test type 3 JDL Attributes: WholeNodes=true, HostNumber=H Verification for LSF: BSUB -n S*H, BSUB -R "span[ptiles=S]", BSUB -x Verification for PBS: PBS -l nodes=H:ppn=S, PBS -W x=NACCESSPOLICY:SINGLEJOB Test type 4 JDL Attributes: WholeNodes=false, SMPGranularity=G, CPUNumber=C Verification for LSF: BSUB -n C, BSUB -R "span[ptile=G]" Verification for PBS: PBS -l nodes=N:ppn=G { [+1:ppn=R] if r>0 } Test type 5 JDL Attributes: WholeNodes=false, HostNumber=H, CPUNumber=C, H>1 Verification for LSF: BSUB -n C, BSUB -R "span[ptile={ N if R=0 ; N+1 if R>0 }]" Verification for PBS: PBS -l nodes=H-R:ppn=N { [+R:ppn=N+1] if R>0 } Test type 6 JDL Attributes: WholeNodes=false, CPUNumber=C Verification for LSF: BSUB -n C Verification for PBS: PBS -L nodes=C Tags: cpu_alloc, cpu_alloc4 - + TEST CASE: CPU Allocation 5 Full Name: Cream Test Suite.CPU Allocation 5 Documentation: Validate the contents of the batch job submission script, according to: https://wiki.italiangrid.it/twiki/bin/view/CREAM/CreamTestWorkPlan cpu allocation test specifications. Per test information follows: Wherever it appears,S is the value published as GlueHostArchitectureSMPSize Test type 1 Jdl Attributes: WholeNodes=true SMPGranularity=G HostNumber=H, H>1 Verification for LSF: BSUB -n S*H, BSUB -R "span[ptile=S]", BSUB -x Verification for PBS: PBS -l nodes=H:ppn=S, PBS -W x=NACCESSPOLICY:SINGLEJOB Test type 2 JDL Attributes: WholeNodes=true SMPGranularity=G Verification for LSF: BSUB -n S, BSUB -R "span[jpsts=1]", BSUB -x Verification for PBS: PBS -l nodes=1:ppn=S, PBS -W x=NACCESSPOLICY:SINGLEJOB Test type 3 JDL Attributes: WholeNodes=true, HostNumber=H Verification for LSF: BSUB -n S*H, BSUB -R "span[ptiles=S]", BSUB -x Verification for PBS: PBS -l nodes=H:ppn=S, PBS -W x=NACCESSPOLICY:SINGLEJOB Test type 4 JDL Attributes: WholeNodes=false, SMPGranularity=G, CPUNumber=C Verification for LSF: BSUB -n C, BSUB -R "span[ptile=G]" Verification for PBS: PBS -l nodes=N:ppn=G { [+1:ppn=R] if r>0 } Test type 5 JDL Attributes: WholeNodes=false, HostNumber=H, CPUNumber=C, H>1 Verification for LSF: BSUB -n C, BSUB -R "span[ptile={ N if R=0 ; N+1 if R>0 }]" Verification for PBS: PBS -l nodes=H-R:ppn=N { [+R:ppn=N+1] if R>0 } Test type 6 JDL Attributes: WholeNodes=false, CPUNumber=C Verification for LSF: BSUB -n C Verification for PBS: PBS -L nodes=C Tags: cpu_alloc, cpu_alloc5 - + TEST CASE: CPU Allocation 6 Full Name: Cream Test Suite.CPU Allocation 6 Documentation: Validate the contents of the batch job submission script, according to: https://wiki.italiangrid.it/twiki/bin/view/CREAM/CreamTestWorkPlan cpu allocation test specifications. Per test information follows: Wherever it appears,S is the value published as GlueHostArchitectureSMPSize Test type 1 Jdl Attributes: WholeNodes=true SMPGranularity=G HostNumber=H, H>1 Verification for LSF: BSUB -n S*H, BSUB -R "span[ptile=S]", BSUB -x Verification for PBS: PBS -l nodes=H:ppn=S, PBS -W x=NACCESSPOLICY:SINGLEJOB Test type 2 JDL Attributes: WholeNodes=true SMPGranularity=G Verification for LSF: BSUB -n S, BSUB -R "span[jpsts=1]", BSUB -x Verification for PBS: PBS -l nodes=1:ppn=S, PBS -W x=NACCESSPOLICY:SINGLEJOB Test type 3 JDL Attributes: WholeNodes=true, HostNumber=H Verification for LSF: BSUB -n S*H, BSUB -R "span[ptiles=S]", BSUB -x Verification for PBS: PBS -l nodes=H:ppn=S, PBS -W x=NACCESSPOLICY:SINGLEJOB Test type 4 JDL Attributes: WholeNodes=false, SMPGranularity=G, CPUNumber=C Verification for LSF: BSUB -n C, BSUB -R "span[ptile=G]" Verification for PBS: PBS -l nodes=N:ppn=G { [+1:ppn=R] if r>0 } Test type 5 JDL Attributes: WholeNodes=false, HostNumber=H, CPUNumber=C, H>1 Verification for LSF: BSUB -n C, BSUB -R "span[ptile={ N if R=0 ; N+1 if R>0 }]" Verification for PBS: PBS -l nodes=H-R:ppn=N { [+R:ppn=N+1] if R>0 } Test type 6 JDL Attributes: WholeNodes=false, CPUNumber=C Verification for LSF: BSUB -n C Verification for PBS: PBS -L nodes=C Tags: cpu_alloc, cpu_alloc6 - + TEST CASE: Resource BDII Full Name: Cream Test Suite.Resource BDII Documentation: Check if the resource bdii publishes all the relevant objectclasses for both glue1 and glue2. Tags: bdii - + TEST CASE: Logging and Bookkeeping Full Name: Cream Test Suite.Logging and Bookkeeping Documentation: Check if the logging and bookkeeping service reports 3 events sourcing from LRMS. Tags: lb - + TEST CASE: Old BLParser Restart Full Name: Cream Test Suite.Old BLParser Restart Documentation: Configure a CREAM CE using the old blah blparser. Stop the blparser and after a few seconds (e.g. 10) restart it. Wait for 2 minute and try a job submission. The job should run and complete properly. Tags: old_blah

Library Reference

Shortcuts

Keywords

Keyword	Arguments	Documentation
Allowed Submission	ce_endpoint	Description: Return the output of the glite-ce-allowed-submission command Arguments: ce_endpoint the cream endpoint Returns: the command's output.
Ban User Argus	dn, argus_host	Description: Ban the specified DN using pap-admin cli on the argus host Arguments: dn Distinguished Name to be added argus_host the argus host Returns: Nothing (raises exception upon error)
Ban User Gjaf	dn, ce_endpoint	Description: Add the specified DN as a CREAM banned user (add it to /etc/lcas/ban_users.db) Arguments: dn Distinguished Name to be added ce_endpoint the cream endpoint Returns: Nothing (raises exception upon error)
Bkill Job	jid, lsf_host	Description: Manually bkill a job. Arguments: jid job identifier lsf_host the server hosting lsf, either ip or name Returns: Nothing
Cancel All Jobs	ce_endpoint	Description: Cancel all the user's jobs in the given cream endpoint. Arguments: ce_endpoint cream endpoint. Returns: nothing.
Cancel Job	job_id	Description: Cancel the given job. Arguments: job_id as returned by the submit operation. Returns: nothing.
Ce Service Info	ce_endpoint, verbosity	Description: Get the service info of a CREAM ce. Arguments: ce_endpoint the CREAM endpoint verbosity how much information will be shown, either 1 or 2 Returns: The output of the command
Change Sandbox Transfer Method	ce_endpoint	Description: Change the SANDBOX_TRANSFER_METHOD between GSIFTP and LRMS (@ /etc/lcas/ban_users.db) Arguments: ce_endpoint the cream endpoint Returns: Nothing (raises exception upon error)
Check Delegation Id In Filesystem	contents_before, contents_after, deleg_id	Description: Check the presense (or not) of a delegation id in the respective cream sandbox dir Arguments: contents_before fs contents before running the purger contents_after fs contents after running the purger deleg_id the delegation id searched for Returns: nothing (raises exception if the desired outcome isn't met)
Check Osb Basedesturi Files	jdl_path, gridftp_server, gridftp_path	Description: Read from a jdl file the list of files contained in the OutputSandBox and OutputSandboxBaseDestURI attribute. Then check if these files exist or not. Arguments: jdl_path path to the jdl file gridftp_server the gridftp server in use gridftp_path the path within the gridftp server Returns: True or False.
Check Osb Desturi Files	jdl_path, gridftp_server, gridftp_path	Description: Read from a jdl file the list of files contained in the OutputSandboxDestURI attribute. Then check if these files exist or not. Arguments: jdl_path path to the jdl file gridftp_server the gridftp server in use gridftp_path the path within the gridftp server Returns: True or False.
Check Proxy	time_left=None	Description: Check whether the proxy exists and if it has any time left. Arguments: Without any arguments,it checks if the proxy exists and has any time left With one argument,it checks if the proxy exists and has greater than or equal to the requested time left. Returns: nothing
Check Resource Bdii Published Values	results	Description: Validate the CREAM resource BDII published values according to https://wiki.italiangrid.it/twiki/bin/view/CREAM/CreamTestWorkPlan#Resource_BDII_test Arguments: results list with ldapsearch results Returns: Nothing. Raises exception upon non-confirmation.
Check Wms Logging Info	output	Description: Validates the output of a glite-wms-job-logging-info command Arguments: output the output of a wms job status logging ingo command Returns: Nothing (raises exception uppon non-validation)
Combo Jdl	vo, wholenodes, hostnumber, smpgranularity, output_dir	Description: Attribute checking jdl file.Sets the jdl attributes "WholeNodes","HostNumber" and "SMPGranularity" to the given numbers/values. It is used to test whether these attributes can be combined. In theory, when WholeNodes is "False",the other two shouldn't be able to be set and an error should be produced during jdl submission. This should be tested for arbitrary values of hostnumber and smpgranularity, and for both cases of wholenodes (set to "True" and set to "False"). Arguments: vo virtual organisation smpgranularity jdl attribute hostnumber jdl attribute wholenodes jdl attribute output_dir the directory to put the file in Returns: Temporary file name.
Cpu Allocation Jdl	output_dir, vo, wholenodes=None, hostnumber=None, smpgranularity=None, cpunumber=None	Description: Attribute checking jdl file.Sets the jdl attributes "WholeNodes","HostNumber" and "SMPGranularity" to the given numbers/values. If a value isn't given, then it is ignored in the final jdl file. Arguments: output_dir the directory to put the file in vo virtual organisation smpgranularity jdl attribute hostnumber jdl attribute wholenodes jdl attribute cpunumber jdl attribute Returns: Temporary file name.
Cpu Allocation Test	jid, ce_endpoint, test_type=0, wholenodes=0, cpunumber=0, smpgranularity=0, hostnumber=0, smpsize=0, batch_system=0	Description: Validate the contents of the batch job submission script, according to: https://wiki.italiangrid.it/twiki/bin/view/CREAM/CreamTestWorkPlan cpu allocation test specifications. Per test information follows: Wherever it appears,S is the value published as GlueHostArchitectureSMPSize Test type 1 Jdl Attributes: WholeNodes=true SMPGranularity=G HostNumber=H, H>1 Verification for LSF:  BSUB -n S*H, BSUB -R "span[ptile=S]", BSUB -x Verification for PBS:  PBS -l nodes=H:ppn=S, PBS -W x=NACCESSPOLICY:SINGLEJOB Test type 2 JDL Attributes: WholeNodes=true SMPGranularity=G Verification for LSF: BSUB -n S, BSUB -R "span[jpsts=1]", BSUB -x Verification for PBS: PBS -l nodes=1:ppn=S, PBS -W x=NACCESSPOLICY:SINGLEJOB Test type 3 JDL Attributes: WholeNodes=true, HostNumber=H Verification for LSF: BSUB -n S*H, BSUB -R "span[ptiles=S]", BSUB -x Verification for PBS: PBS -l nodes=H:ppn=S, PBS -W x=NACCESSPOLICY:SINGLEJOB Test type 4 JDL Attributes: WholeNodes=false, SMPGranularity=G, CPUNumber=C Verification for LSF: BSUB -n C, BSUB -R "span[ptile=G]" Verification for PBS: PBS -l nodes=N:ppn=G { [+1:ppn=R] if r>0 } Test type 5 JDL Attributes: WholeNodes=false, HostNumber=H, CPUNumber=C, H>1 Verification for LSF: BSUB -n C, BSUB -R "span[ptile={ N if R=0 ; N+1 if R>0 }]" Verification for PBS: PBS -l nodes=H-R:ppn=N { [+R:ppn=N+1] if R>0 } Test type 6 JDL Attributes: WholeNodes=false, CPUNumber=C Verification for LSF: BSUB -n C Verification for PBS: PBS -L nodes=C NOTE: Default values are used due to robot framework limitations for named arguments. These default values serve no other purpose. Arguments: jid CREAM job identifier ce_endpoint the CREAM endpoint test_type one of the aforementioned test types, as an integer wholenodes jdl attribute in use (either true or false!) cpunumber jdl attribute in use smpgranularity jdl attribute in use hostnumber jdl attribute in use smpsize the value published as GlueHostArchitectureSMPSize batch_system the underlying lrms system Returns: Nothing (raises exception if anything is out of order)
Cpunumber Jdl	vo, cpunumber, output_dir	Description: Attribute checking jdl file.Sets the jdl attribute "CPUNumber" to the given number. Arguments: vo virtual organisation cpunumber jdl attribute output_dir the directory to put the file in Returns: Temporary file name.
Create Delegation	cream_endpoint, delegId	Description: Delegate user's proxy credentials,to be used later for job submissions. Arguments: cream_endpoint the cream endpoint delegId the delegation id string Returns: nothing
Create Dummy Script	output_dir	Description: Create a dummy bash script executing ls. Arguments: output_dir the directory to put the file in Returns: Temporary file name
Create Proxy	password, vo, cert=None, key=None, time=None	Description: Create a user proxy. Arguments: password the user's proxy password vo for the voms extention. cert non standard certificate path key non standard key path time the validity period of the proxy. Form: HH:MM Returns: nothing.
Delegation Info	deleg_endpoint, deleg_id	Description: Fetch the information regarding a delegation. Arguments: deleg_endpoint the delegation endpoint of a cream service deleg_id the delegation id Returns: The commands output or the string "NOT IN STORAGE" if the specified delegation id doesn't exist in the specified endpoint
Delete Osb Basedesturi Files	jdl_path, gridftp_server, gridftp_path	Description: Read from a jdl file the list of files contained in the OutputSandBox and OutputSandboxBaseDestURI attribute. Then delete these files with lcg-del. Arguments: jdl_path path to the jdl file gridftp_server the gridftp server in use gridftp_path the path within the gridftp server Returns: nothing.
Delete Osb Desturi Files	jdl_path, gridftp_server	Description: Read from a jdl file the list of files contained in the OutputSandboxDestURI attribute. Then delete these files with lcg-del. Arguments: jdl_path path to the jdl file gridftp_server the gridftp server in use Returns: nothing.
Destroy Delegation	deleg_endpoint, deleg_id	Description: Delete a delegation. s Arguments: deleg_endpoint the delegation endpoint of a cream service deleg_id the delegation id Returns: nothing.
Destroy Proxy		Description: Delete a user's proxy. Arguments: none. Returns: nothing.
Disable Cream Admin	dn, ce_endpoint	Description: Remove the specified DN as a CREAM administrator (add it to /etc/grid-security/admin-list) Arguments: dn Distinguished Name to be added ce_endpoint the cream endpoint Returns: Nothing (raises exception upon error)
Disable Submission	ce_endpoint	Description: Disable the submission to the designated CREAM endpoint. Arguments: ce_endpoint the cream endpoint Returns: nothing (raises exception in case of error)
Enable Cream Admin	dn, ce_endpoint	Description: Add the specified DN as a CREAM administrator (add it to /etc/grid-security/admin-list) Arguments: dn Distinguished Name to be added ce_endpoint the cream endpoint Returns: Nothing (raises exception upon error)
Enable Submission	ce_endpoint	Description:    Enable the submission to the designated CREAM endpoint. Arguments:      ce_endpoint the cream endpoint Returns:        nothing (raises exception in case of error)
Environment Jdl	vo, output_dir	Description: Attribute checking jdl file. Set an environmental variable and check its value during runtime. Arguments: vo virtual organisation output_dir the directory to put the file in Returns: Temporary jdl file name,temporary shell script file name.
Epilogue Jdl	vo, output_dir	Description: Attribute checking jdl file. Use the epilogue argument and execute an epilogue script. Arguments: vo virtual organisation output_dir the directory to put the file in Returns: Temporary jdl file name,temporary shell script file name.,temporary epilogue script.
Execute Noninteractive Ssh Com	command, host, user, port=22	Description: Execute a non interactive command through ssh on another host NOTE: Shell metacharacters are NOT recognized. Call them through 'bash -c'. Example: '/bin/bash -c "ls -l grep LOG > log_list.txt"' Arguments: host ssh host port ssh port, 22 by default user the user name to use for the ssh connection command the command to execute Returns: the command's output
Execute Uberftp Command	uberftp_command, gridftp_server, gridftp_path	Description: Execute an uberftp command on a gridftp url Arguments: uberftp_command one of cat,chgrp,chmod,dir,ls,mkdir,rename,rm,rmdir,size,stage gridftp_server the gridftp server hostname gridftp_path the path in the gridftp server Returns: The output of the command
Fetch Output Files	dest_dir, job_id, timeout=0	Description: Gather the files from a certain job,store them in the specified directory and return the list of files transfered as they exist on the disk after the transfer operation. Note: Target directory must be empty!Any existing output files with the same name are overwritten! Arguments: dest_dir directory to store the files locally job_id the job's cream job id timeout time to wait before fetching output (used to wait until files are uploaded to CE's gridftp server) Returns: a list with the files transfered
File Should Contain	file_path, search_string	Description: Check whether the given file contains the given string. Note that this must be used for small files,since it reads it all in memory. Arguments: file_path the path pointing to the file search_string the string searching for Returns: True or False.
Files Should Not Be Empty	input_files	Description: Copy a file from an SE to localhost,using lcg-cp. Arguments: files file(s) to be checked for being non-empty this argument can either be a list of strings or a single string Returns: Nothing - raises error if any of the files given are empty
Get Arch Smp Size	ce_endpoint, ldap_port	Description: Get the published value of the GlueHostArchitectureSMPSize attribute Arguments: ce_endpoint the CREAM endpoint ldap_port the ldap port to use Returns: the value of the attribute
Get Current Status	job_id	Description: Return the current status of a job,with the use of the glite-ce-job-status command. This function will NOT wait until the job is in a final state. Arguments: job_id the job id,as returned by the submit operation. Returns: job's status as a string.
Get Current Status Verbose	job_id, verbosity=2	Description: Return the current status of a job,with the use of the glite-ce-job-status command,with the given verbosity This function will NOT wait until the job is in a final state. Arguments: job_id as returned by the submit operation. verbosity 0,1 or 2 Returns: a string.
Get Deleg Id From Status	status	Description: Return the delegation id in use by a job Arguments: status the job's verbose status Returns: the delegation id (string)
Get Delegation Id	jid	Description: Get the delegation id in use by a job Arguments: jid CREAM job identifier Returns: the delegation id string
Get Final Status	job_id	Description: Return the final status of a job,with the use of the glite-ce-job-status command. This command will wait until the job is in a final state. Arguments: job_id the job id,as returned by the submit operation. Returns: job's final status as a string.
Get Guid	file_path, prefix=None	Description: Find all the guids of the format "guid:36 char string" in a file Arguments: file_path the path to the file containing the guids (most probanly a DSUpload file) this argument can either be a list of strings or a single string prefix common prefix for all the files of the first argument, e.g.: /tmp/ This argument isn't mandatory Returns: list with corresponding guids
Get Job Sb	jid	Description: Find the gridftp url of the ISB of the given job Arguments: jid job id returned by job submit operation Returns: (gridftp server, gridftp path)
Get Osb	jdl_path	Description: Read from a jdl file the list of files contained in the OutputSandBox attribute. Arguments: jdl_path        path to the jdl file. Returns: a list with the files in the output sandbox
Get Proxy Dn		Description:    Get user's proxy DN. Arguments:      none. Returns:        The user's proxy DN.
Get Status With Filter	ce_endpoint, filter_type1, filter_data1, filter_type2=None, filter_data2=None	Description: Return the current statuses of jobs,with the use of the glite-ce-job-status command and a filter This function will NOT wait until the job is in a final state. filter_type2 and filter_data2 are optional Arguments: ce_endpoint the cream endpoint being questioned filter_type1/2 the filter being used, one of "from", "to", or "status" filter_data1/2 the corresponding's filter data, either a date (as accepted from the glite status command "YYYY-MM-DD HH:mm:ss" or a set of job statuses e.g. "DONE-OK:DONE-FAILED" Returns: list with corresponding job ids and statuses
Get Time In Status Format		Description: Returns the time in the format accepted by glite-ce-job-status Arguments: N/A N/A Returns: string (containing the time-date in format YYYY-MM-DD HH:mm:ss)
Hostnumber Jdl	vo, hostnumber, output_dir	Description: Attribute checking jdl file.Sets the jdl attribute "HostNumber" to the given number. Arguments: vo virtual organisation hostnumber jdl attribute output_dir the directory to put the file in Returns: Temporary file name.
Isb Baseuri Jdl	vo, gridftp_server, gridftp_path, uploaded_file_name, output_dir	Description: File transfer checking jdl file. Store input files to a gridftp server,and use the jdl attribute InputSandboxBaseURI to fetch them. Arguments: vo virtual organisation gridftp_server gridftp server to set ISBBaseURI gridftp_path gridftp path to set ISBBaseURI uploaded_file_name file name of the uploaded file output_dir the directory to put the file in Returns: Temporary file name.
Isb Client To Ce Jdl	vo, output_dir	Description: File transfer checking jdl file. Stage input files from client to CE node. Arguments: vo virtual organisation output_dir the directory to put the file in Returns: Temporary jdl file name,temporary shell script file name.
Isb Gsiftp To Ce Jdl	vo, gridftp_server, gridftp_path, uploaded_file_name, output_dir	Description: File transfer checking jdl file. Store input files to a gridftp server,and use the jdl attribute InputSandbox to fetch them. Arguments: vo virtual organisation gridftp_server gridftp server to set ISBBaseURI gridftp_path gridftp path to set ISBBaseURI uploaded_file_name file name of the uploaded file output_dir the directory to put the file in Returns: Temporary file name.
Job Status Should Be	jid, status	Description: Check that the given job's status is the one given. Arguments: jid job identifier status the status comparing to Returns: Nothing
Jobdbadminpurger	jid, ce_endpoint, db_user, db_pass, middleware_version	Description: Purge a job using the JobDBAdminPurger.sh script, run on cream host Arguments: jid the job to purge job id as returned by glite-ce-job-submit ce_endpoint the host of the cream service db_user cream's db user db_pass aforementioned user's db pass middleware_version the middleware version, either "EMI1" or "EMI2" Returns: nothing
Lcg Cp	vo, identifier, local_fname	Description: Copy a file from an SE to localhost,using lcg-cp. Note: without the LCG_GFAL_INFOSYS variable set, this keyword will NOT work. Arguments: vo virtual organisation identifier lfn or guid to download Must include lfn: or guid: at the start local_fname local file name to use Returns: Nothing.
Lcg Cr	vo, local_fname, storage_element, lfn	Description: Upload a file to an SE,using lcg-cr. Arguments: vo virtual organisation local_fname local file's name to upload storage_element storage element to be used lfn the logical file name to use Returns: Nothing.
Lcg Del	vo, lfn	Description: Delete all replicas of a file and its LFC entry,using lcg-del. Arguments: vo virtual organisation lfn logical file name to delete Returns: Nothing.
Ldap Search	host, port, search_base, search_filter	Description: Perform a simple ldapsearch. The option -LLL is always set. Arguments: host the ldap host port the ldap listening port search_base the search base search_filter the search filter Returns: The commands output.
Lfc Ls	lfn_path	Description: List an lfn,using lfc-ls. Arguments: lfn_path imaginary lfc path Returns: List containing all the files.
Lfc Mkdir	lfn_path	Description: Create a folder in the LFC,using lfc-mkdir. Arguments: lfn_path imaginary lfc path Returns: Nothing.
Lfc Rmdir	lfn_path	Description: Delete a folder in the LFC,using lfc-rm -r. Arguments: lfn_path imaginary lfc path Returns: Nothing.
List Jobs	ce_endpoint	Description: List the jobs not purged by the specified cream endpoint for the user executing the command. Arguments: ce_endpoint cream endpoint. Returns: a list containing all the relevant job ids
List Proxy Sandbox	ce_endpoint, job_status	Description: Check the presense (or not) of a delegation id in db queries contents Arguments: ce_endpoint the cream endpoint job_status a job's verbose status Returns: a list with the contents of the directory
Localhost Output Jdl	vo, output_dir	Description: File transfer checking jdl file. Stage files to the CE node (gsiftp://localhost). Arguments: vo virtual organisation output_dir the directory to put the file in Returns: Temporary file name.
Modify Cream Config Xml	ce_endpoint, attr, val	Description: Modify the given attribute of the cream config xml file Arguments: ce_endpoint the cream endpoint attr the option to change it's value val the value to set it Returns: Nothing (raises exception upon error or if attr isn't found)
Multiple Lcg Cp	vo, identifiers, id_type, output_dir	Description: Copy a file from an SE to localhost,using lcg-cp. Arguments: vo virtual organisation identifiers list of lfns or guids to download id_type either lfn or guid output_dir the directory where the files will be placed Returns: List of files downloaded
Osb Basedesturi Jdl	vo, gridftp_server, gridftp_path, output_dir	Description: File transfer checking jdl file. Stage output files to a gridftp server,with the use of the OutputSandboxBaseDestURI jdl attribute. Arguments: vo virtual organisation gridftp_server gridftp server for BaseDestURI gridftp_path gridftp path for BaseDestURI output_dir the directory to put the file in Returns: Temporary file name.
Osb Desturi Jdl	vo, gridftp_server, gridftp_path, output_dir	Description: File transfer checking jdl file. Stage output files to a gridftp server,with the use of the OutputSandboxDestURI jdl attribute. Arguments: vo virtual organisation gridftp_server gridftp server to set DestURI gridftp_path gridftp path to set DestURI output_dir the directory to put the file in Returns: Temporary file name.
Output Data Jdl	vo, output_dir, gridftp_server=None, lfn_dir=None, lfc=None, infosys=None	Description: Attribute checking jdl file. Uses a combination of the output data attributes. Arguments: vo virtual organisation output_dir the directory to put the file in lfc the value to set the environmental variable LFC_HOST on the WN infosys the value to set the environmental variable LCG_GFAL_INFOSYS on the WN Returns: Temporary jdl file name,temporary shell script file name.
Prologue Jdl	vo, output_dir	Description: Attribute checking jdl file. Use the prologue argument and execute a prologue script. Arguments: vo virtual organisation output_dir the directory to put the file in Returns: Temporary jdl file name,temporary shell script file name.,temporary prologue script.
Proxy Info		Description:    Read user's proxy info. Arguments:      None. Returns:        Command's output.
Purge All Jobs	ce_endpoint	Description: Purge all the user's jobs in the given cream endpoint. Arguments: ce_endpoint cream endpoint. Returns: nothing.
Purge Job	jid, ce_endpoint	Description: Purge the job with the corresponding jid at the given CREAM endpoint Arguments: jid job id returned by job submit operation ce_endpoint the CREAM endpoint Returns: nothing.
Qdel Job	jid, torque_host	Description: Manually qdel a job. Arguments: jid job identifier cream_host the server hosting torque, either ip or name cream_admin_user a user exiting on the torque host, having admin priviledges Returns: Nothing
Renew Proxy	ce_endpoint, deleg_id	Description: Renew a delegated proxy in a CREAM endpoint Arguments: ce_endpoint the CREAM endpoint deleg_id delegation id to renew Returns: nothing
Reset Cream Config Xml	ce_endpoint	Description: Reset the cream config xml file to the original Arguments: ce_endpoint the cream endpoint Returns: Nothing
Reset Limiter Threshold	ce_endpoint, middleware_version	Description: Reset the cream limiter perl script to the original unmodified one Arguments: ce_endpoint The cream host middleware_version The middleware version, either "EMI1" or "EMI2" Returns: Nothing
Restart Cream	ce_endpoint	Description: Restart the tomcat service, thus restarting cream Arguments: ce_endpoint the cream endpoint Returns: Nothing (raises exception upon error or if attr isn't found)
Resume All Jobs	ce_endpoint	Description: Suspend all the user's jobs in the given cream endpoint. Arguments: ce_endpoint cream endpoint. Returns: nothing.
Resume Job	job_id	Description: Resumes the given (previously suspended) job. Arguments: job id as returned by the submit operation. Returns: nothing.
Save Batch Job Submission Script Off	ce_endpoint, version	Description: Modify blah_common_submit_functions.sh in order to NOT save the job submission script Arguments: ce_endpoint the cream endpoint version EMI1 or EMI2 (to determine the path of the script file) Returns: Nothing (raises exception upon error)
Save Batch Job Submission Script On	ce_endpoint, version	Description: Modify blah_common_submit_functions.sh in order to save the job submission script Arguments: ce_endpoint the cream endpoint version EMI1 or EMI2 (to determine the path of the script file) Returns: Nothing (raises exception upon error)
Send Signal To Process	pid, sigNo	Description: Send a process a certain signal. Arguments: pid the proc's pid sigNo signal number to send Returns: nothing
Set Limiter Threshold	thresh_name, thresh_value, ce_endpoint, middleware_version	Description: Set one threshold of the cream limiter perl script Arguments: thresh_name The threshold's name thresh_value The threshold's value ce_endpoint The cream host middleware_version The middleware version, either "EMI1" or "EMI2" Returns: Nothing
Simple Jdl	vo, output_dir	Description: Simple jdl file.Executes /bin/uname -a. Arguments: vo virtual organisation output_dir the directory to put the file in Returns: Temporary file name.
Sleep Jdl	vo, secs, output_dir	Description: Simple jdl file.Executes /bin/sleep for the defined number of seconds. Arguments: vo virtual organisation secs seconds to sleep output_dir the directory to put the file in Returns: Temporary file name.
Smpgranularity Jdl	vo, smpgranularity, output_dir	Description: Attribute checking jdl file.Sets the jdl attribute "SMPGranularity" to the given number. Arguments: vo virtual organisation smpgranularity jdl attribute output_dir the directory to put the file in Returns: Temporary file name.
Start Old Blparser	host	Description: Start the old blparser service Arguments: host the service's host Returns: Nothing (raises exception upon error)
Stop Old Blparser	host	Description: Stop the old blparser service Arguments: host the service's host Returns: Nothing (raises exception upon error)
String Should Contain	string, search_string	Description: Check whether the given string contains the given substring. Arguments: string string haystack search_string string needle Returns: True or raise error
Submit And Wait	jdl_path=None, ce_endpoint=None, delegId=None	Description: Submit a job and wait until it finishes. Arguments: Same as submit_job. Returns: (int,string) as jid and final job state.
Submit Job	jdl_path, ce_endpoint, delegId=None	Description: Submit a job with automatic or explicit delegation and return it's job id. Arguments: jdl_path path to the jdl file ce_endpoint the cream endpoint,containing the queue delegId if specified,uses the given delegation id, else it uses automatic delegation Returns: the resulting cream job id as a string
Suspend All Jobs	ce_endpoint	Description: Suspend all the user's jobs in the given cream endpoint. Arguments: ce_endpoint cream endpoint. Returns: nothing.
Suspend Job	job_id	Description: Suspends the given job. Arguments: job id as returned by the submit operation. Returns: nothing.
Uberftp Upload	gridftp_server, gridftp_path, local_file_path	Description: Upload a file to a gridftp server with uberftp Arguments: gridftp_server the gridftp server hostname gridftp_path the path in the gridftp server local_file_path the full path of the local path Returns: The output of the command
Unban User Argus	dn, argus_host	Description: Un-Ban the specified DN using pap-admin cli on the argus host Arguments: dn Distinguished Name to be added argus_host the argus host Returns: Nothing (raises exception upon error)
Unban User Gjaf	dn, ce_endpoint	Description: Remove the specified DN as a CREAM banned user (remove it from /etc/lcas/ban_users.db) Arguments: dn Distinguished Name to be added ce_endpoint the cream endpoint Returns: Nothing (raises exception upon error)
Validate Ce Service Info	output	Description: Validates the output of a glite-ce-service-info command Arguments: output  the output of the method ce_service_info() Returns: Nothing (raises exception uppon non-validation)
Validate Glue	ce_endpoint, port, bind, glue_version	Description: Run the glue validator for the designated CREAM endpoint Arguments: ce_endpoint the cream endpoint port the ldap listening port bind the point where glue validator should bind glue_version the glue version to be tested by glue validator Returns: Nothing (raises exception upon error)
Validate Job Status	output, verbosity	Description: Validates the output of a glite-ce-job-status command Arguments: output the output of a job status command verbosity the verbosity of the status command Returns: Nothing (raises exception uppon non-validation)
Wait For Status	jid, status, timeout=3600	Description: Wait for the given job to reach the given status. Arguments: jid jod identifier status the status waiting for timeout timeout after which the operation fails (in seconds) Defaults in one hour timeout. Returns: Nothing
Wholenodes Jdl	vo, wholenodes, output_dir	Description: Attribute checking jdl file.Sets the jdl attribute "WholeNodes" to the given value. Arguments: vo              virtual organisation wholenodes      jdl attribute output_dir      the directory to put the file in Returns: Temporary file name.
Wms Get Final Job Status	job_id, verbosity=1	Description: Return the final status of a wms job,with the use of the glite-wms-job-status command. This function WILL wait until the job is in a final state. Arguments: job_id the job id,as returned by the submit operation. verbosity the status operation's verbosity. One of 0,1,2,3 Returns: job's status as a string.
Wms Get Job Status	job_id, verbosity=1	Description: Return the current status of a wms job,with the use of the glite-wms-job-status command. This function will NOT wait until the job is in a final state. Arguments: job_id the job id,as returned by the submit operation. verbosity the status operation's verbosity. One of 0,1,2,3 Returns: job's status as a string.
Wms Job Logging Info	job_id, verbosity=1	Description: Return the current status of a wms job,with the use of the glite-wms-job-status command. This function will NOT wait until the job is in a final state. Arguments: job_id the job id,as returned by the submit operation. verbosity the status operation's verbosity. One of 0,1,2,3 Returns: job's status as a string.
Wms Job Submit	jdl_path, wmproxy_endpoint=None, ce_endpoint=None, delegId=None	Description: Submit a job to the WMS, with automatic or explicit delegation and return it's job id. NOTE: this method only supports some basic options of the Workload Management System! Arguments: jdl_path path to the jdl file ce_endpoint the cream endpoint,containing the queue wmproxy_endpoint the wmproxy endpoint delegId if specified,uses the given delegation id, else it uses automatic delegation Returns: the resulting cream job id as a string

Changelog

!!!!!!!!!!!!!!!
Version 1.8 (1st EMI2 version)
!!!!!!!!!!!!!!!

====
LIBRARY
====
> Added "LCG_GFAL_VO..." to the outputdata jdl. Apparently, doesn't work without it
=> get_guid() returns an empty list if no guids are found, instead of raising an error.
=> restart_cream() auto-detects tomcat version (through SL version)
=> Added ldap_search() keyword
=> Added check_resource_bdii_published_values() keyword
=> Added wms_job_submit() keyword
=> Added wms_get_job_status() keyword
=> Added wms_get_final_job_status() keyword
=> Added wms_job_logging_info() keyword
=> Added check_wms_logging_info() keyword
=> Added stop_old_blparser() keyword
=> Added start_old_blparser() keyword
=> Added _get_ssh_connection() method
=> _enisc doesn't utilize password's anymore. It expects to be authenticated automagically, through ssh keys.
=> Removed execute_remote_mysql_query()


===========
TEST SUITE
===========
=> OutputData test case: check guids for existence and log them explicitely (was only implicit till now)
=> Automatic Proxy Purging Teardown test case: fixed delegation name
=> Added Explicit Delegation Teardown (used in the test case with the same name, to handle delegation destruction in case of error)
=> Added Resource BDII test case
=> Added Logging and Bookkeeping test case
=> Added Old BLParser Restart test case


==========
VARIABLES
==========
=> Added blparser_host
=> Added blparser_root_pass
=> Removed lrms_admin_pass and eliminated its usage
=> Removed lrms_admin_user and replaced it with root
=> Removed cream_root_pass and eliminated its usage
=> Removec blparser_root_pass and eliminated its usage


===================
DEPENDENCIES ADDED
===================
=> openldap-clients
=> whiptail

Check Also

Robot Framework Quick Start Guide https://twiki.cern.ch/twiki/bin/view/EMI/RobotFrameworkQuickstartGuide

Robot Framework Advanced Guide https://twiki.cern.ch/twiki/bin/view/EMI/RobotFrameworkAdvancedGuide

Robot Framework http://code.google.com/p/robotframework/

RIDE Editor for Robot Framework test suites http://code.google.com/p/robotframework-ride/

pybot --help

man cream_test

-- DimosthenesFioretos - 07-Oct-2011