Introduction

Robot Framework is a Python-based, extensible keyword-driven test automation framework for end-to-end acceptance testing and acceptance-test-driven development (ATDD). It can be used for testing distributed, heterogeneous applications, where verification requires touching several technologies and interfaces.

Why Robot Framework

  • Enables easy-to-use tabular syntax for creating test cases in a uniform way.
  • Provides ability to create reusable higher-level keywords from the existing keywords.
  • Provides easy-to-read result reports and logs in HTML format.
  • Is platform and application independent.
  • Provides a simple library API for creating customized test libraries which can be implemented natively with either Python or Java.
  • Provides a command line interface and XML based output files for integration into existing build infrastructure (continuous integration systems).
  • Provides support for Selenium for web testing, Java GUI testing, running processes, Telnet, SSH, and so on.
  • Supports creating data-driven test cases.
  • Has built-in support for variables, practical particularly for testing in different environments.
  • Provides tagging to categorize and select test cases to be executed.
  • Enables easy integration with source control: test suites are just files and directories that can be versioned with the production code.
  • Provides test-case and test-suite -level setup and teardown.
  • The modular architecture supports creating tests even for applications with several diverse interfaces.

High-level architecture

Robot Framework is a generic, application and technology independent framework. It has a highly modular architecture illustrated in the diagram below.

architecture.png Robot Framework architecture

The test data is in simple, easy-to-edit tabular format. When Robot Framework is started, it processes the test data, executes test cases and generates logs and reports. The core framework does not know anything about the target under test, and the interaction with it is handled by test libraries. Libraries can either use application interfaces directly or use lower level test tools as drivers.

Getting more information

Project Pages
Robot Framework http://code.google.com/p/robotframework/
Robot Framework SSH Library http://code.google.com/p/robotframework-sshlibrary/
Robot Framework RIDE Editor http://code.google.com/p/robotframework-ride/
Paramiko SSH Python Module http://www.lag.net/paramiko/
Pycrypto Cryptography Python Toolkit https://www.dlitz.net/software/pycrypto/
Contact
Dimosthenes Fioretos dfiore -at- noc -dot- edunet -dot- gr

Copyright and Licence

Each respective tool has its own copyright.
Robot Framework related: Apache
Pycrypto: Public Domain
Paramiko: LGPLv2.1
Cream Test: GPLv3

Installation

Precondition

Robot Framework runs on Python, and you need to have it installed to be able to use it. However, a recent Python version is required,so installing python26 from the epel repository is a necessary step.

Installation

There are two main ways with which you can install robot framework and any relevant usefull software.The first is by downloading and installing them by hand.The other is by installing the robot_testing repository residing at yum.gridctb.uoa.gr and using the yum tool.Both ways,should be preceded by the installation of python26 from the epel repository. The second and most easy way,requires the following steps:
yum install python26
wget http://yum.gridctb.uoa.gr/repository/robot_testing.repo -O /etc/yum.repos.d/robot_testing.repo
yum install robotframework pycrypto paramiko SSHLibrary
yum install wxPython-common-gtk2-unicode wxPython2.8-gtk2-unicode robotframework-ride
The other available packages are:
  • robotframework-ride: a wysiwyg editor/IDE for robot framework test data.
  • cream_test: a test for the CREAM service,using the robot framework.
  • wxPython-common: library needed by wxPython2.8
  • wxPython2.8-gtk2-unicode: library needed by RIDE editor

Where files are installed

Paramiko:
  • under /usr/lib/python2.6/site-packages/paramiko/
Pycrypto:
  • under /usr/lib64/python2.6/site-packages/Crypto/
  • /usr/lib64/python2.6/site-packages/pycrypto-2.3-py2.6.egg-info
Robot Framework:
  • /usr/bin/jybot
  • /usr/bin/pybot
  • /usr/bin/rebot
  • /usr/lib/python2.6/site-packages/robotframework-trunk_20110727-py2.6.egg-info
  • and under /usr/lib/python2.6/site-packages/robot/
SSHLibrary:
  • under /usr/lib/python2.6/site-packages/SSHLibrary/
RIDE:
  • under /usr/lib/python2.6/site-packages/robotide/
  • /usr/bin/ride.py

Verifying installation

For robot framework,execute:
 pybot --version 
If everything is ok,it should output the version of robot framework.
For the RIDE editor,execute:
 ride.py 
If everything is ok,the editor environment should appear.
For the ssh library,create the following file:
| *Setting* |     *Value*  |
| Library   | SSHLibrary   |

| *Test Case* |  *Action*              |           *Argument*                    |    *Argument*             |
| Test1       | Open Connection        |   put ssh host here                     |                           |
|             | Login                  |   put ssh user here                     | put ssh password here     |
Then save it -lets assume the name is test.txt- and execute:
 pybot test.txt 
The output should be something like:
==============================================================================
Test                                                                          
==============================================================================
Test1                                                                 | PASS |
------------------------------------------------------------------------------
Test                                                                  | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================
Output: /home/dfiore/robot/output.xml
Log: /home/dfiore/robot/log.html
Report: /home/dfiore/robot/report.html

Uninstallation

All the modules presented can be uninstalled by manually deleting the files the installed (the list is available in a previous section of this document).
Other than that,if the installation was done by the tools provided by the OS,they can by uninstalled by the same means.For example:
 yum remove robotframework 

Creating Test Data

Structure

The hierarchical structure for arranging test cases is built as follows:
  • Test cases are created in test case files.
  • A test case file automatically creates a test suite containing the test cases in that file.
  • A directory containing test case files forms a higher-level test suite. Such a test suite directory has suites created from test case files as its sub test suites.
  • A test suite directory can also contain other test suite directories, and this hierarchical structure can be as deeply nested as needed.
  • Test suite directories can have a special initialization file.
In addition to this, there are:
  • Test libraries containing the lowest-level keywords.
  • Resource files with variables and higher-level user keywords.
  • Variable files to provide more flexible ways to create variables than resource files.

Supported File Formats

Robot Framework test data is defined in tabular format, using either the hypertext markup language (HTML), tab-separated values (TSV), plain text, or reStructuredText (reST) formats. Robot Framework selects a parser for the test data based on the file extension. The extension is case-insensitive, and the recognized extensions are .html, .htm and .xhtml for HTML, .tsv for TSV, .txt for plain text, and .rst or .rest for reStructuredText.For ease of use and generally easier development and maintainability,txt and html files are mostly used -note that the RIDE editor can save the test case files in either txt,html or tsv-.Test case files can be edited with any editor.Still,the provided RIDE editor is recomended for graphical environments. For the HTML format,the test data is defined in separate tables (see the example below). Robot Framework recognizes these test data tables based on the text in their first cell. Everything outside recognized tables is ignored.
For the plain text format,test data tables are recognized from one or more asterisks ( ), followed by a normal table name and an optional closing asterisks. Everything before the first recognized table is ignored similarly as data outside tables in HTML data. For a separator between cells you can use either two or more spaces or a pipe character surrounded with spaces ( | ). Also,everything before the first table is ignored.Example:

Test Case Action Argument Argument
Simple Test Should Contain Hello, world! world
This test case checks whether the second argument ("world") is contained within the first argument ("Hello, world!").

Test Data Tables

Test data is structured in four types of tables listed below. These test data tables are identified by the first cell of the table, and the last column in the table below lists different aliases that can be used as a table name.
Table name	Used for	                                                                Aliases
Setting table	1) Importing test libraries, resource files and variable files                  Setting, Settings, Metadata
                2) Defining metadata for test suites and test cases

Variable table	Defining variables that can be used elsewhere in the test data	                Variable, Variables
Test case table	Creating test cases from available keywords	                                Test Case, Test Cases
Keyword table	Creating user keywords from existing lower-level keywords	                Keyword, Keywords, User Keyword, User Keywords

Parsing Data Rules

Parsing

When Robot Framework parses the test data, it ignores:
  • All tables that do not start with a recognized table name in the first cell.
  • Everything else on the first row of a table apart from the first cell.
  • Data outside tables in HTML and data before the first table in TXT.
  • All empty rows, which means these kinds of rows can be used to make the tables more readable.
  • All empty cells at the end of rows; you must add a backslash (\) to prevent such cells from being ignored.
  • All single backslashes (\); they are used as an escape character.
  • All characters following a hash mark (#), if it is the first character of a cell; this means that hash marks can be used to enter comments in the test data.
  • All formatting in the HTML test data.

Escaping

When Robot Framework ignores some data, this data is not available in any resulting reports and, additionally, most tools used with Robot Framework also ignore them. To add information that is visible in Robot Framework outputs, or available to, for example, RIDE, place it to the documentation or other metadata of test cases or suites, or log with the Log or Comment keywords available from the BuiltIn library. The escape character for the parser is the backslash (\). The escape character can be used as follows:
  • To escape special characters so that their literal values are used:
  • \${notvar} means a literal string ${notvar} that looks like a variable
  • \\ means a single backslash (for example, C:\\Temp)
  • \# means a literal hash (#) mark, even at the beginning of a cell
  • To affect the parsing of whitespaces.
  • To prevent ignoring empty cells at the end of a row (this requires \ to be in the appropriate cell). Another possibility is using built-in variable ${EMPTY}.
Note that these escaping rules are applied only to arguments to keywords and values to settings. They are not used, for example, with keyword and test case names.

Whitespaces

Robot Framework handles whitespace, such as spaces, newlines and tabs, the same way as they are handled in HTML. This means that Robot Framework:
  • Removes leading and trailing whitespace in all cells.
  • Changes multiple consecutive spaces into single spaces.
  • Converts all newlines and tabs into spaces.
To prevent Robot Framework from parsing data according to these rules, a backslash can be used:
  • Before leading spaces, for example \ some text.
  • Between consecutive spaces, for example text \ \ more text.
  • After trailing spaces, for example some text \ \.
  • As \n to create a newline, for example first line\n2nd line.
  • As \t to create a tab character, for example text\tmore text.
  • As \r to create a carriage return, for example text\rmore text.
Another, and often clearer, possibility for representing leading, trailing, or consecutive spaces is using built-in variable ${SPACE}. The extended variable syntax even allows syntax like ${SPACE * 8} which makes handling consecutive spaces very simple.

Dividing test data to several rows

If there is more data than readily fits a row, it possible to use ellipsis (...) to continue the previous line. In test case and user keyword tables, the ellipsis must be preceded by at least one empty cell. In settings and variable tables, it can be placed directly under the setting or variable name. In all tables, all empty cells before the ellipsis are ignored. Additionally, values of settings that take only one value (mainly documentations) can be split to several columns. These values will be then catenated together with spaces when the test data is parsed. Example:
Test Case Action Argument Argument Argument Argument Argument Argument
Example Do Something value1 value2 value3 value4 value5 value6
Can be turned to:
Test Case Action Argument Argument
Example Do Something value1 value2
  ... value3 value4
  ... value5 value6

Creating Test Cases

Test Case Syntax

Basic Syntax

Test cases are constructed in test case tables from the available keywords. Keywords can be imported from test libraries or resource files, or created in the keyword table of the test case file itself. The first column in the test case table contains test case names. A test case starts from the row with something in this column and continues to the next test case name or to the end of the table. It is an error to have something between the table headers and the first test. The second column normally has keyword names. An exception to this rule is setting variables from keyword return values, when the second and possibly also the subsequent columns contain variable names and a keyword name is located after them. In either case, columns after the keyword name contain possible arguments to the specified keyword. Example:
Setting Value
Library /path/to/cream/library
Test Case Action Argument Argument Argument
Submit Job Test Create Proxy dteam p4sSw0rD  
  ${job_id}= Submit Job /path/to/jdl cream_endpoint.cern.ch:8443/queue
  ${final_status}= Get Final Status ${job_id}  
  Should Be Equal ${final_status} DONE-OK  
  Destroy Proxy      
This test imports a library,and then uses the keywords defined within to submit a job to a cream ce and it checks its final status.

Settings in the Test Case Table

Test cases can also have their own settings. Setting names are always in the second column, where keywords normally are, and their values are in the subsequent columns. Setting names have square brackets around them to distinguish them from keywords. The available settings are listed below and explained later in this section. [Documentation] Used for specifying a test case documentation. [Tags] Used for tagging test cases. [Setup], [Teardown] Specify test setup and teardown. Have also synonyms [Precondition] and [Postcondition], respectively. [Timeout] Used for setting a test case timeout. Timeouts are discussed in their own section. Following is the last example,enhanced with the use of settings.
Setting Value
Library /path/to/cream/library
Test Case Action Argument Argument Argument
Submit Job Test [Documentation] Submit a job to a cream ce  
  [Tags] job_submit    
  Create Proxy dteam p4sSw0rD  
  ${job_id}= Submit Job /path/to/jdl cream_endpoint.cern.ch:8443/queue
  ${final_status}= Get Final Status ${job_id}  
  Should Be Equal ${final_status} DONE-OK  
  Destroy Proxy      

Using Arguments

The earlier examples have already demonstrated keywords taking different arguments, and this section discusses this important functionality more thoroughly. How to actually implement user keywords and library keywords with different arguments is discussed in separate sections. Keywords can accept zero or more arguments, and some arguments may have default values. What arguments a keyword accepts depends on its implementation, and typically the best place to search this information is keyword's documentation.Arguments can either be:
  • Required: An argument must always be passed to the keyword.An example of this could be a logging keyword.
  • Have default value: The argument may or may not be given.In the first case,it has the explicit value given.In the latter case,it has its default value.An example of this could be a keyword creating a temporary file name.
  • Variable number of arguments: The arguments provided may be from zero (in case of a default value being used at the same time),to any number.An example of this could be a keyword which deletes files. Important Note: The variable arguments must always be the last ones!
  • Named: Arguments can be given their values,by naming them.In this way,the arguments can be given in any order.
  • Embedded: Arguments may finally be incorporated inside user level keywords,thus being able to call a user keyword and in this way implicitely assign values to variables.

Test Case Name and Documentation

The test case name comes directly from the Test Case table: it is exactly what is entered into the test case column. Test cases in one test suite should have unique names. Pertaining to this, you can also use the automatic variable ${TEST_NAME} within the test itself to refer to the test name. It is available whenever a test is being executed, including all user keywords, as well as the test setup and the test teardown. The [Documentation] setting allows you to set a free documentation for a test case. That text is shown in the command line output, as well as the resulting test logs and test reports. If the documentation is long, it can be split into several cells that are catenated together with spaces. It is possible to use simple HTML formatting and variables can be used to make the documentation dynamic. Possible non-existing variables in documentation are just left unchanged. It is important that test cases have clear and descriptive names, and in that case they normally do not need any documentation. If the logic of the test case needs documenting, it is often a sign that keywords in the test case need better names and they are to be enhanced, instead of adding extra documentation.

Tagging Test Cases

Using tags in Robot Framework is a simple, yet powerful mechanism for classifying test cases. Tags are free text and they can be used at least for the following purposes:
  • Tags are shown in test reports, logs and, of course, in the test data, so they provide metadata to test cases.
  • Statistics about test cases (total, passed, failed are automatically collected based on tags).
  • With tags, you can include or exclude test cases to be executed.
  • With tags, you can specify which test cases are considered critical.
In this section it is only explained how to set tags for test cases, and different ways to do it are listed below. These approaches can naturally be used together.
  • Force Tags in the Setting table
    • All test cases in a test case file with this setting always get specified tags. If it is used in the test suite initialization file, all test cases in sub test suites get these tags.
  • Default Tags in the Setting table
    • Test cases that do not have a [Tags] setting of their own get these tags.
  • [Tags] in the Test Case table
    • A test case always gets these tags. Additionally, it does not get the possible tags specified with Default Tags, so it is possible to override the Default Tags by using empty value. It is also possible to use value NONE to override default tags.
  • --settag command line option
    • All executed test cases get tags set with this option in addition to tags they got elsewhere.
  • Set Tags and Remove Tags keywords
These BuiltIn keywords can be used to modify tags during the test execution. Tags are free text, but they are normalized so that they are converted to lowercase and all spaces are removed. If a test case gets the same tag several times, other occurrences than the first one are removed. Tags can be created using variables, assuming that those variables exist. Example:
Setting Value Value Value
Force Tags req-42    
Default Tags owner-john smoke  
Variable Value Value Value
${HOST} 10.0.1.42    
Test Case Action Argument Argument
No own tags [Documentation] This test has tags owner-john, smoke, req-42
  No Operation    
       
With own tags [Documentation] This test has tags not_ready, owner-mrx, req-42
  [Tags] owner-mrx not_ready  
  No Operation    
       
Own tags with variables [Documentation] This test has tags host-10.0.1.42, req-42
  [Tags] host-${HOST}  
  No Operation    
       
Empty own tags [Documentation] This test has tags req-42
  [Tags]    
  No Operation    
       
Set Tags and Remove Tags Keywords [Documentation] This test has tags mytag, owner-john
  Set Tags mytag    
  Remove Tags smoke req-*  

Test Setup and Teardown

Robot Framework has similar test setup and teardown functionality as many other test automation frameworks. In short, a test setup is something that is executed before a test case, and a test teardown is executed after a test case. In Robot Framework setups and teardowns are just normal keywords with possible arguments.
Setup and teardown are always a single keyword. If they need to take care of multiple separate tasks, it is possible to create higher-level user keywords for that purpose. An alternative solution is executing multiple keywords using the BuiltIn keyword Run Keywords. The test teardown is special in two ways. First of all, it is executed also when a test case fails, so it can be used for clean-up activities that must be done regardless of the test case status. All the keywords in the teardown are also executed even if one of them fails. This continue on failure functionality can be used also with normal keywords, but inside teardowns it is on by default.
The easiest way to specify a setup or a teardown for test cases in a test case file is using the Test Setup and Test Teardown settings in the Setting table. Individual test cases can also have their own setup or teardown. They are defined with the [Setup] or [Teardown] settings in the test case table and they override possible Test Setup and Test Teardown settings. Having no keyword after a [Setup] or [Teardown] setting means having no setup or teardown.
Following is the previous cream job submission example,enhanced with the use of setups/teardowns
Setting Value Value Value
Library /path/to/cream/library    
Test Setup Create Proxy dteam p4sSw0rD
Test Teardown Destroy Proxy    
Test Case Action Argument Argument Argument
Submit Job Test [Documentation] Submit a job to a cream ce  
  [Tags] job_submit    
  ${job_id}= Submit Job /path/to/jdl cream_endpoint.cern.ch:8443/queue
  ${final_status}= Get Final Status ${job_id}  
  Should Be Equal ${final_status} DONE-OK  
Our sample test,now uses setup and teardown facilities. In the case of the proxy creation,in case we had any more tests in this testsuite,we wouldn't have to create the proxy again and again for each seperate test case. In the case of proxy destruction,we also ensure that in case of test failure,the proxy is always destroyed.

Using Test Libraries

Using Custom Libraries

Test libraries contain those lowest-level keywords, often called library keywords, which actually interact with the system under test. All test cases always use keywords from some library, often through higher-level user keywords. This section explains how to take test libraries into use and how to use the keywords they provide. Creating test libraries is described in a separate section.
Libraries can be imported,as allready has been seen in previous examples,in the settings table of a test.They can also be imported with the builtin keyword "Import Library" used within a test case -this can be usefull for example,when a library isn't availlable until test execution- . Using the full path name to the library is the safest and most reliable way to import it,but it can also be imported by its name if it is availavle in PYTHONPATH.All the libraries dependencies -if any- must be made available in the same way -i.e. by their absolute or relative name inside the imported library- . Libraries can be imported with arguments,in this way:
Setting Value Value Value
Library /path/to/cream/library arg1 arg2
It is possible to use default values,variable number or named arguments,exactly like keyword arguments. Libraries can be imported with a custom name,in this way:
Setting Value Value Value
Library /path/to/cream/library WITH NAME libname
This is usefull in the following cases:
  • There is a need to import the same library several times with different arguments. This is not possible otherwise.
  • The library name is inconveniently long. This can happen, for example, if a Java library has a long package name.
  • You want to use variables to import different libraries in different environments, but refer to them with the same name.
  • The library name is misleading or otherwise poor. In this case, changing the actual name is, of course, a better solution.

Standard Libraries

Some test libraries are distributed with Robot Framework and these libraries are called standard libraries. These are the available standard libraries: The BuiltIn library is special, because it is taken into use automatically and thus its keywords are always available. Other standard libraries need to be imported in the same way as any other libraries, but there is no need to install them. Additionally, they work when running tests both with Python and Jython (with the Screenshot library as an exception). New standard libraries can, and will, be added in the future. If you have an idea for a new standard library, or even have one that could be incorporated immediately, you can contact Robot Framework developers. In general, a library is a good candidate to be added into standard libraries if it is generic, works on both Python and Jython without any external dependencies, and is adequately tested and documented.

BuiltIn library
The BuiltIn library provides a set of generic keywords needed often. It is imported automatically and thus always available. The provided keywords can be used, for example, for verifications (e.g. Should Be Equal, Should Contain), conversions (e.g. Convert To Integer) and for various other purposes (e.g. Log, Sleep, Run Keyword If, Set Global Variable). For more information, see the BuiltIn library documentation.

OperatingSystem library
The OperatingSystem library enables various operating system related tasks to be performed in the system where Robot Framework is running. It can, among other things, execute commands (e.g. Run), create and remove files and directories (e.g. Create File, Remove Directory), check whether files or directories exists or contain something (e.g. File Should Exist, Directory Should Be Empty) and manipulate environment variables (e.g. Set Environment Variable). For more information, see the OperatingSystem library documentation.

Telnet library
The Telnet library makes it possible to connect to Telnet servers and execute commands on the opened connections. For more information, see the Telnet library documentation.

Collections library
The Collections library provides a set of keywords for handling Python lists and dictionaries. This library has keywords, for example, for modifying and getting values from lists and dictionaries (e.g. Append To List, Get From Dictionary) and for verifying their contents (e.g. Lists Should Be Equal, Dictionary Should Contain Value). For more information, see the Collections library documentation.

String library
The String library enables manipulating strings (e.g. Replace String With Regexp, Split To Lines) and verifying their contents (e.g. Should Be String). For more information, see the String library documentation.

Dialogs library
The Dialogs library provides means for pausing the test execution and getting input from users. The dialogs are slightly different depending on are tests run on Python or Jython but they provide the same functionality. For more information, see the Dialogs library documentation.

Screenshot library
The Screenshot library has keywords to capture and store screenshots of the whole desktop. This library is implemented with Java AWT APIs, so it can be used only when running Robot Framework on Jython. For more information, see the Screenshot library documentation.

Remote library
The Remote library is totally different than the other standard libraries. It does not have any keywords of its own but it works as a proxy between Robot Framework and actual test library implementations. These libraries can be running on other machines than the core framework and can even be implemented using languages not supported by Robot Framework natively. See separate Remote library interface section for more information about the concept.

Variables

Introduction

Variables are an integral feature of Robot Framework, and they can be used in most places in test data. Most commonly, they are used in arguments for keywords in test case tables and keyword tables, but also all settings allow variables in their values. A normal keyword name cannot be specified with a variable, but the BuiltIn keyword Run Keyword can be used to get the same effect. Robot Framework itself has two kinds of variables, scalars and lists, and they have the syntaxes ${SCALAR} and @{LIST}, respectively. In addition to this, environment variables can be used directly with the syntax %{VARIABLE}. If a nonexistent variable is used in the test data, the keyword using it fails. If the same syntax that is used for variables is needed as a literal string, it must be escaped with a backslash as in \${NAME}.

Variable Types

Robot Framework variables, similarly as keywords, are case-insensitive, and also spaces and underscores are ignored. However, it is recommended to use all capital letters with global variables (for example, ${PATH} or ${TWO_WORDS}) and small letters with variables that are only available in certain test cases or user keywords (for example, ${my_var} or ${myVar}). Much more importantly, though, cases should be used consistently.
Unlike in some programming languages where similar variable syntax is used, curly braces ({ and }) are mandatory in Robot Framework test data. Basically, variable names can have any characters between the curly braces. However, using only alphabetic characters from a to z, numbers, underscore and space is recommended, and it is even a requirement for using the extended variable syntax.

Scalar Variables

When scalar variables are used in the test data, they are replaced with the value they are assigned to. While scalar variables are most commonly used for simple strings, you can assign any objects, including lists, to them. The scalar variable syntax, for example ${NAME}, should be familiar to most users, as it is also used, for example, in shell scripts and Perl programming language. The example below illustrates the usage of scalar variables.It is the well known example we have allready used,which submits a job to a cream ce.
Setting Value Value Value
Library /path/to/cream/library    
Test Setup Create Proxy dteam p4sSw0rD
Test Teardown Destroy Proxy    
Variable Value
${ce} cream_endpoint.cern.ch:8443/queue
Test Case Action Argument Argument Argument
Submit Job Test [Documentation] Submit a job to a cream ce  
  [Tags] job_submit    
  ${job_id}= Submit Job /path/to/jdl ${ce}
  ${final_status}= Get Final Status ${job_id}  
  Should Be Equal ${final_status} DONE-OK  
This is the same test as before,submiting a job to a cream ce and then checking its final status. The pros of using variables should be well known by anyone reading this page smile .
When a scalar variable is used as the only value in a test data cell, the scalar variable is replaced with the value it has. The value may be any object. When a scalar variable is used in a test data cell with anything else (constant strings or other variables), its value is first converted into a Unicode string and then catenated to whatever is in that cell. Converting the value into a string means that the object's method unicode (in Python, with str as a fallback) is called.Variable values are used as-is without conversions when passing arguments to keywords using the named arguments syntax like argname=${var}.
Note: Converting variables to Unicode obviously fails if the variable cannot be represented as Unicode. This can happen, for example, if you try to use byte sequences as arguments to keywords so that you catenate the values together like ${byte1}${byte2}. A workaround is creating a variable that contains the whole value and using it alone in the cell (e.g. ${bytes}) because then the value is used as-is.

List Variables

List variables are compound variables that can have several values assigned to them. In short, they are always lists and can contain an unlimited number of entries (also empty lists are possible). The main benefit of list variables is that they allow you to assign a name for a larger data set. While list variables normally contain only strings, other content is also possible.
When you use a list variable in test data, then the elements of the list are inserted as new cells in the test data. Thus, if the list variable contains two elements, the cell containing the list variable is turned into two cells with the content of the list variable. Note that cells with list variables should not contain other content. The list variable syntax, @{NAME}, is borrowed from Perl.
Assuming that the list variable @{LST} is set to the value ['/path/to/jdl','cream_endpoint.cern.ch:8443/queue'], the following two test cases are equivalent:
Test Case Action JDL Path CREAM CE
Strings Submit Job /path/to/jdl cream_endpoint.cern.ch:8443/queue
Strings Submit Job @{LST}  
It is also possible to access a certain value from the list variable with the syntax @{NAME}[i], where i is the index of the selected value. Indexes start from zero, and trying to access a value with too large an index causes an error. List items accessed in this manner can be used similarly as scalar variables:
Test Case Action JDL Path CREAM CE
Strings Submit Job @{LST}[0] @{LST}[1]

Environment Variables

Robot Framework allows using environment variables in the test data using the syntax %{ENV_VAR_NAME}. They are limited to string values. Environment variables set in the operating system before the test execution are available during it, and it is possible to create new ones with the keyword Set Environment Variable or delete existing ones with the keyword Delete Environment Variable, both available in the OperatingSystem library. Because environment variables are global, environment variables set in one test case can be used in other test cases executed after it. However, changes to environment variables are not effective after the test execution.

Creating Variables

Variables can spring into existence from different sources.Those include variable tables in test case files and resource files,variable files,return values from keywords and some builtin special purpose keywords.

Variable Table

The simplest possible variable assignment is setting a string into a scalar variable. This is done by giving the variable name (including ${}) in the first column of the Variable table and the value in the second one. If the second column is empty, an empty string is set as a value. Also an already defined variable can be used in the value.Example:
Variable Value Value
${test_name} cream_test  
${version} 1.2  
${test_descr} ${name} ${version}  
@{lst} val1 val2
@{lst2} @{lst} val3
@{empty}    
@{many} val1 val2
... val3 val4

Variable File

Variable files are the most powerful mechanism for creating different kind of variables. It is possible to assign variables to any object using them, and they also enable creating variables dynamically. The variable file syntax and taking variable files into use is explained in a later section.

Command Line

Variables can be set from the command line either individually with the --variable (-v) option or using a variable file with the --variablefile (-V) option. Variables set from the command line are globally available for all executed test data files, and they also override possible variables with the same names in the Variable table and in variable files imported in the test data.
The syntax for setting individual variables is --variable name:value, where name is the name of the variable without ${} and value is its value. Several variables can be set by using this option several times. Only scalar variables can be set using this syntax and they can only get string values. Many special characters are difficult to represent in the command line, but they can be escaped with the --escape option.
--variable EXAMPLE:value
--variable HOST:localhost:7272 --variable USER:robot
--variable ESCAPED:Qquotes_and_spacesQ --escape quot:Q --escape space:_
In the examples above, variables are set so that:
  • ${EXAMPLE} gets the value value
  • ${HOST} and ${USER} get the values localhost:7272 and robot
  • ${ESCAPED} gets the value "quotes and spaces"
  • The basic syntax for taking variable files into use from the command line is --variablefile path/to/variables.py, and Taking variable files into use section has more details. What variables actually are created depends on what variables there are in the referenced variable file.
If both variable files and individual variables are given from the command line, the latter have higher priority.

Return Values from Keywords

Return values from keywords can also be set into variables. This allows communication between different keywords even in different test libraries. The syntax for a simple case is illustrated in the example below: Assigning values from keywords to variables
Test Case Action Argument Argument
Returning ${x}= Get X arg
Log We got ${x}!    
In the example above, the value returned by the Get X keyword is first set into the variable ${x} and then used by the Log keyword. This syntax works in all cases where a keywords returns something, and the variable is set to whatever value returned by the keyword. Having the equals sign = after the variable name is not obligatory, but recommended, because it makes the assignment more explicit. If a keyword returns a list, it is also possible to assign the return value into several scalar variables and/or one list variable. Assigning multiple values at once
Test Case Action Argument Argument Argument
Return Multiple ${scalar}= Get 3    
${a} ${b} ${c} = Get 3  
${first} @{rest} = Get 3    
@{list} = Get 3      
Assuming that the keyword Get 3 returns a list [1, 2, 3], the following variables are created: ${scalar} with the value [1, 2, 3] ${a}, ${b} and ${c} with the values 1, 2, and 3, respectively ${first} with the value 1, and @{rest} with the value [2, 3] @{list} with the value [1, 2, 3] Variables set in this manner are otherwise similar to any other variables, but they are available only within the scope of the test case or keyword where they are created. Thus it is not possible, for example, to set a variable in one test case and use it in another. This is because, in general, automated test cases should not depend on each other, and accidentally setting a variable that is used elsewhere could cause hard-to-debug errors. If there is a genuine need for setting a variable in one test case and using it in another, it is possible to use built-in keywords as explained in the next section.

Using built-in Set Test/Suite/Global Variable keywords

The BuiltIn library has keywords Set Test Variable, Set Suite Variable and Set Global Variable which can be used for setting variables dynamically during the test execution. If a variable already exists within the new scope, its value will be overwritten, and otherwise a new variable is created.
Variables set with Set Test Variable keyword are available everywhere within the scope of the currently executed test case. For example, if you set a variable in a user keyword, it is available both in the test case level and also in all other user keywords used in the current test. Other test cases will not see variables set with this keyword.
Variables set with Set Suite Variable keyword are available everywhere within the scope of the currently executed test suite. Setting variables with this keyword thus has the same effect as creating them using the Variable table in the test data file or importing them from variable files. Other test suites, including possible child test suites, will not see variables set with this keyword.
Variables set with Set Global Variable keyword are globally available in all test cases and suites executed after setting them. Setting variables with this keyword thus has the same effect as creating from the command line using the options --variable or --variablefile. Because this keyword can change variables everywhere, it should be used with care.

Built-in Variables

Robot Framework provides some built-in variables that are available automatically.

Operating System Variables

Built-in variables related to the operating system ease making the test data operating-system-agnostic. Available operating-system-related built-in variables: Variable Explanation ${CURDIR} An absolute path to the directory where the test data file is located. This variable is case-sensitive. ${TEMPDIR} An absolute path to the system temporary directory. In UNIX-like systems this is typically /tmp, and in Windows c:\Documents and Settings\\Local Settings\Temp. ${EXECDIR} An absolute path to the directory where test execution was started from. ${/} The system directory path separator. / in UNIX-like systems, \ in Windows. ${:} The system path element separator. : in UNIX-like systems and ; in Windows. Using operating-system-related built-in variables
Test Case Action Argument Argument
Example Create File ${CURDIR}${/}input.data Some text here
  Set Environment Variable CLASSPATH ${TEMPDIR}${:}${TEMPDIR}${/}foo.jar

Number Variables

The variable syntax can be used for creating both integers and floating point numbers, as illustrated in the example below. This is useful when a keyword expects to get an actual number, and not a string that just looks like a number, as an argument. Example:
Test Case Action Argument Argument Comment
Example 1A Connect example.com 80 # Connect gets two strings as arguments
Example 1B Connect example.com ${80} # Connect gets a string and an integer
Example 2 Do X ${3.14} ${-1e-4} # Do X gets floating point numbers 3.14 and -0.0001
It is possible to create integers also from binary, octal, and hexadecimal values using 0b, 0o and 0x prefixes, respectively. The syntax is case insetive. Example:
Test Case Action Argument Argument
Example Should Be Equal ${0b1011} ${11}
Should Be Equal ${0o10} ${8}  
Should Be Equal ${0xff} ${255}  
Should Be Equal ${0B1010} ${0XA}  

Boolean and None Variables

Also Boolean values and Python None can be created using the variable syntax similarly as numbers. Example:
Test Case Action Argument Argument Comment
Boolean Set Status ${true}   # Set Status gets Boolean true as an argument
  Create Y something ${false} # Create Y gets a string and Boolean false
None Do XYZ ${None}   # Do XYZ gets Python None as an argument
Null ${ret} = Get Value arg # Checking that Get Value returns python none
  Should Be Equal ${ret} ${None}  
These variables are case-insensitive, so for example ${True} and ${true} are equivalent. Additionally, ${None} and ${null} are synonyms, because when running tests on the Jython interpreter, Jython automatically converts None and null to the correct format when necessary.

Space and Empty Variables

It is possible to create spaces and empty strings using variables ${SPACE} and ${EMPTY}, respectively. These variables are useful, for example, when there would otherwise be a need to escape spaces or empty cells with a backslash. If more than one space is needed, it is possible to use the extended variable syntax like ${SPACE * 5}. In the following example, Should Be Equal keyword gets identical arguments but those using variables are easier to understand than those using backslashes. Example:
Test Case Action Argument Argument
One Space Should Be Equal ${SPACE} \ \
Four Spaces Should Be Equal ${SPACE * 4} \ \ \ \ \
Ten Spaces Should Be Equal ${SPACE * 10} \ \ \ \ \ \ \ \ \ \ \
Quoted Space Should Be Equal "${SPACE}" " "
Quoted Spaces Should Be Equal "${SPACE * 2}" " \ "
Empty Should Be Equal ${EMPTY} \

Automatic Variables

Some automatic variables can also be used in the test data. These variables can have different values during the test execution and some of them are not even available all the time. Available automatic variables

Variable Explanation Available
${TEST NAME} The name of the current test case. Test case
@{TEST TAGS} Contains the tags of the current test case in alphabetical order. Test case
${TEST STATUS} The status of the current test case, either PASS or FAIL. Test teardown
${TEST MESSAGE} The possible error message of the current test case. Test teardown
${PREV TEST NAME} The name of the previous test case, or an empty string if no tests have been executed yet. Everywhere
${PREV TEST STATUS} The status of the previous test case: either PASS, FAIL or an empty string when no tests have been executed. Everywhere
${PREV TEST MESSAGE} The possible error message of the previous test case. Everywhere
${SUITE NAME} The full name of the current test suite. Everywhere
${SUITE SOURCE} An absolute path to the suite file or directory. Everywhere
${SUITE STATUS} The status of the current test case, either PASS or FAIL. Suite teardown
${SUITE MESSAGE} The full message of the current test suite, including statistics. Suite teardown
${OUTPUT FILE} An absolute path to the output file. Everywhere
${LOG FILE} An absolute path to the log file or string NONE when no log file is created. Everywhere
${REPORT FILE} An absolute path to the report file or string NONE when no report is created. Everywhere
${DEBUG FILE} An absolute path to the debug file or string NONE when no debug file is created. Everywhere
${OUTPUT DIR} An absolute path to the output directory. Everywhere

Variable Priorities and Scopes

Variables coming from different sources have different priorities and are available in different scopes.

Variable Priorities

Variable priorities
  • Variables from the command line
    • Variables set in the command line have the highest priority of all variables that can be set before the actual test execution starts. They override possible variables created in Variable tables in test case files, as well as in resource and variable files imported in the test data.
    • Individually set variables (--variable option) override the variables set using variable files (--variablefile option). If you specify same individual variable multiple times, the one specified last will override earlier ones. This allows setting default values for variables in a start-up script and overriding them from the command line. Notice, though, that if multiple variable files have same variables, the ones in the file specified first have the highest priority.
  • Variable table in a test case file
    • Variables created using the Variable table in a test case file are available for all the test cases in that file. These variables override possible variables with same names in imported resource and variable files. Variables created in the variable tables are available in all other tables in the file where they are created. This means that they can be used also in the Setting table, for example, for importing more variables from resource and variable files.
  • Imported resource and variable files
    • Variables imported from the resource and variable files have the lowest priority of all variables created in the test data. Variables from resource files and variable files have the same priority. If several resource and/or variable file have same variables, the ones in the file imported first are taken into use.
    • If a resource file imports resource files or variable files, variables in its own Variable table have a higher priority than variables it imports. All these variables are available for files that import this resource file. Note that variables imported from resource and variable files are not available in the Variable table of the file that imports them. This is due to the Variable table being processed before the Setting table where the resource files and variable files are imported.
  • Variables set during test execution
    • Variables set during the test execution either using return values from keywords or built-in keywords Set Test/Suite/Global Variable always override possible existing variables in the scope where they are set. In a sense they thus have the highest priority, but on the other hand they do not affect variables outside the scope they are defined.
  • Built-in variables
    • Built-in variables like ${TEMPDIR} and ${TEST_NAME} have the highest priority of all variables. They cannot be overridden using Variable table or from command line, but even they can be reset during the test execution. An exception to this rule are number variables, which are resolved dynamically if no variable is found otherwise. They can thus be overridden, but that is generally a bad idea. Additionally ${CURDIR} is special because it is replaced already during the test data processing time.

Variable Scopes

Depending on where and how they are created, variables can have a global, test suite, test case or user keyword scope.
  • Global scope
    • Global variables are available everywhere in the test data. These variables are normally set from the command line with the --variable and --variablefile options, but it is also possible to create new global variables or change the existing ones with the BuiltIn keyword Set Global Variable anywhere in the test data. Additionally also built-in variables are global.
    • It is recommended to use capital letters with all global variables.
  • Test suite scope
    • Variables with the test suite scope are available anywhere in the test suite where they are defined or imported. They can be created in Variable tables, imported from resource and variable files, or set during the test execution using the BuiltIn keyword Set Suite Variable.
    • The test suite scope is not recursive, which means that variables available in a higher-level test suite are not available in lower-level suites. If necessary, resource and variable files can be used for sharing variables.
    • Since these variables can be considered global in the test suite where they are used, it is recommended to use capital letters also with them.
  • Test case scope
    • Variables created in test cases from the return values of keywords have a test case scope and they are available only in that test case. Another possibility to create them is using the BuiltIn keyword Set Test Variable anywhere in that particular test case. Test case variables are local and should use lower-case letters.
  • User keyword scope
    • User keywords get their own variables from arguments passed to them and return values from the keywords they use. Also these variables are local and should use lower-case letters.

Advanced Variable Features

Extended variable syntax

Extended variable syntax can be used with objects set into scalar variables. It allows accessing the attributes of the object (for example, ${obj.name} or ${obj.some_attr}), and even calling its methods (for example, ${obj.get_name()} or ${obj.getSomething('arg')}). Extended variable syntax is a powerful feature, but it should be used with care. Accessing attributes is normally not a problem, on the contrary, as one variable with an object having several attributes is often better than having several variables. On the other hand, calling methods, especially when they are used with arguments, can make the test data complicated. If that happens, it is recommended to move the code into a test library. The most common usages of extended variable syntax are illustrated in the example below. First assume that we have the following variable file and test case:
class MyObject:
    def __init__(self, name):
        self.name = name
    def greet(self, who):
        return '%s says hello to %s' % (self.name, who)
    def __str__(self):
        return self.name
OBJECT = MyObject('Robot')
DICTIONARY = { 1: 'one', 2: 'two', 3: 'three'}

Test Case Action Argument Argument
Example KW 1 ${OBJECT.name}    
  KW 2 ${OBJECT.greet('Fit')}    
  KW 3 ${DICTIONARY[2]}    
When this test data is executed, the keywords get the arguments as explained below: KW 1 gets string Robot
KW 2 gets string Robot says hello to Fit
KW 3 gets string two
The extended variable syntax is evaluated in the following order:
  • The variable is searched using the full variable name. The extended variable syntax is evaluated only if no matching variable is found.
  • The real name of the base variable is created. The body of the name consists of all the characters after ${ until the first occurrence of a non-alphanumeric character or a space (for example, OBJECT in ${OBJECT.name} and DICTIONARY in ${DICTIONARY[2]}).
  • A variable matching the body is searched. If there is no match, an exception is raised and the test case fails.
  • The expression inside the curly brackets is evaluated as a Python expression, so that the base variable name is replaced with its value. If the evaluation fails because of an invalid syntax or that the queried attribute does not exist, an exception is raised and the test fails.
  • The whole extended variable is replaced with the value returned from the evaluation.
Many standard Python objects, including strings and numbers, have methods that can be used with the extended variable syntax either explicitly or implicitly. Sometimes this can be really useful and reduce the need for setting temporary variables, but it is also easy to overuse it and create really cryptic test data. Following examples show few pretty good usages. Example:
Test Case Action Argument Argument
String ${string} = Set Variable abc
  Log ${string.upper()} # Logs 'ABC'
  Log ${string * 2} # Logs 'abcabc'
Number ${number} = Set Variable ${-2}
  Log ${number * 10} # Logs -20
  Log ${number.__abs__()} # Logs 2
Note that even though abs(number) is recommended over number.__abs__() in normal Python code, using ${abs(number)} does not work. This is because the variable name must be in the beginning of the extended syntax. Using xxx methods in the test data like this is already a bit questionable, and it is normally better to move this kind of logic into test libraries.

Variables inside variables

Variables are allowed also inside variables, and when this syntax is used, variables are resolved from the inside out. For example, if you have a variable ${var${x}}, then ${x} is resolved first. If it has the value name, the final value is then the value of the variable ${varname}. There can be several nested variables, but resolving the outermost fails, if any of them does not exist. In the example below, Do X gets the value ${JOHN HOME} or ${JANE HOME}, depending on if Get Name returns john or jane. If it returns something else, resolving ${${name} HOME} fails. Example:
Variable Value
${JOHN HOME} /home/john
${JANE HOME} /home/jane
Test Case Action Argument
Example ${name} = Get Name
Do X ${${name} HOME}  

Creating User Keywords

Basic Syntax

In many ways, the overall user keyword syntax is identical to the test case syntax. User keywords are created in keyword tables which differ from test case tables only by the name that is used to identify them. User keyword names are in the first column similarly as test cases names. Also user keywords are created from keywords, either from keywords in test libraries or other user keywords. Keyword names are normally in the second column, but when setting variables from keyword return values, they are in the subsequent columns. Example:
Keyword Action Argument Argument
  Open Login Page Open Browser http://host/login.html
  Title Should Be Login Page  
       
Title Should Start With [Arguments] ${expected}  
  ${title} = Get Title  
  Should Start With ${title} ${expected}
Most user keywords take some arguments. This important feature is used already in the second example above, and it is explained in detail later in this section, similarly as user keyword return values. User keywords can be created in test case files, resource files, and test suite initialization files. Keywords created in resource files are available for files using them, whereas other keywords are only available in the files where they are created.

Settings in the Keyword Table

User keywords can have similar settings as test cases, and they have the same square bracket syntax separating them from keyword names. All available settings are listed below and explained later in this section.
  • [Documentation] Used for setting a user keyword documentation.
  • [Arguments] Specifies user keyword arguments.
  • [Return] Specifies user keyword return values.
  • [Teardown] Specify keyword teardown.
  • [Timeout] Sets the possible user keyword timeout. Timeouts are discussed in a section of their own.

User Keyword Arguments

Most user keywords need to take some arguments. The syntax for specifying them is probably the most complicated feature normally needed with Robot Framework, but even that is relatively easy, particularly in most common cases. Arguments are normally specified with the [Arguments] setting, and argument names use the same syntax as variables, for example ${arg}.

Positional arguments

The simplest way to specify arguments (apart from not having them at all) is using only positional arguments. In most cases, this is all that is needed. The syntax is such that first the [Arguments] setting is given and then argument names are defined in the subsequent cells. Each argument is in its own cell, using the same syntax as with variables. The keyword must be used with as many arguments as there are argument names in its signature. The actual argument names do not matter to the framework, but from users' perspective they should should be as descriptive as possible. It is recommended to use lower-case letters in variable names, either as ${my_arg}, ${my arg} or ${myArg}.
Keyword Action Argument Argument Argument
One Argument [Arguments] ${arg_name}    
  Log Got argument ${arg_name}    
         
Three Arguments [Arguments] ${arg1} ${arg2} ${arg3}
  Log 1st argument: ${arg1}    
  Log 2nd argument: ${arg2}    
  Log 3rd argument: ${arg3}    
Default values and variable numbers of arguments can be used as well.Example:
Keyword Action Argument Argument Argument
Default Arguments [Arguments] ${arg1}=val1 ${arg2} ${arg3}
  Do something ${arg1} ${arg2} ${arg3}
         
Var Arguments [Arguments] @{varargs}    
  Do something @{varargs}    
         
Combi Arguments [Arguments] ${req} ${opt}=1924 @{rest}
  Do something ${req} ${opt} @{rest}
         

Return Values

Similarly as library keywords, also user keywords can return values. Return values are defined with the [Return] setting. The values can then be assigned to variables in test cases or other user keywords.
In a typical case, a user keyword returns one value and it can be set to a scalar variable. This is done by having the return value in the next cell after the [Return] setting. User keywords can also return several values, which can then be assigned into several scalar variables at once, to a list variable, or to scalar variables and a list variable. Several values can be returned simply by specifying those values in different cells after the [Return] setting. Example:
Keyword Action Argument Argument Argument
Return One Value [Arguments] ${arg}    
  Do Something ${arg}    
  ${value} = Get Some Value    
  [Return] ${value}    
         
Return Three Values [Return] foo bar zap

Resource and Variable Files

User keywords and variables in test case files and test suite initialization files can only be used in files where they are created, but resource files provide a mechanism for sharing them. Since the resource file structure is very close to test case files, it is easy to create them. Variable files provide a powerful mechanism for creating and sharing variables. For example, they allow values other than strings and enable creating variables dynamically. Their flexibility comes from the fact that they are created using Python code, which also makes them somewhat more complicated than Variable tables.

Resource Files

How to use

Resource files are imported using the Resource setting in the Settings table. The path to the resource file is given in the cell after the setting name.
If the path is given in an absolute format, it is used directly. In other cases, the resource file is first searched relatively to the directory where the importing file is located. If the file is not found there, it is then searched from the directories in PYTHONPATH. The path can contain variables, and it is recommended to use them to make paths system-independent (for example, ${RESOURCES}/login_resources.html or ${RESOURCE_PATH}). Additionally, slashes ("/") in the path are automatically changed to backslashes ("\") on Windows. Example:
Setting Value
Resource myresources.html
Resource ../data/resources.html
Resource ${RESOURCES}/common.tsv
The user keywords and variables defined in a resource file are available in the file that takes that resource file into use. Similarly available are also all keywords and variables from the libraries, resource files and variable files imported by the said resource file.

Resource File Structure

The higher-level structure of resource files is the same as that of test case files otherwise, but, of course, they cannot contain Test Case tables. Additionally, the Setting table in resource files can contain only import settings (Library, Resource, Variables) and Documentation. The Variable table and Keyword table are used exactly the same way as in test case files.
If several resource files have a user keyword with the same name, they must be used so that the keyword name is prefixed with the resource file name without the extension (for example, myresources.Some Keyword and common.Some Keyword). Moreover, if several resource files contain the same variable, the one that is imported first is taken into use.

Documenting Resource Files

Keywords created in a resource file can be documented using [Documentation] setting. Also the resource file itself can have Documentation in the Setting table similarly as test suites.
Both libdoc.py and RIDE use these documentations, and they are naturally available for anyone opening resource files. The first line of the documentation of a keyword is logged when it is run, but otherwise resource file documentations are ignored during the test execution. Example Resource File:
Setting Value
Documentation An example resource file
Library SeleniumLibrary
Resource ${RESOURCES}/common.html
Variable Value
${HOST} localhost:7272
${LOGIN_URL} http://${HOST}/
${WELCOME_URL} http://${HOST}/welcome.html
${BROWSER} Firefox
Keyword Action Argument Argument
Open Login Page [Documentation] Opens browser to login page
  Open Browser ${LOGIN_URL} ${BROWSER}
  Title Should Be Login Page  
       
Input Name [Arguments] ${name}  
  Input Text username_field ${name}
       
Input Password [Arguments] ${password}  
  Input Text password_field ${password}

Variable Files

Variable files contain variables that can be used in test data. Variables can also be created using variable tables or set from the command line, but variable files allow creating them dynamically and their variables can contain any objects.
Variable files are created using Python code, and technically they are Python modules. There are two different possibilities for creating variables:
Creating variables directly
Variables are specified as module attributes. In simple cases, the syntax is so simple that no real programming is needed. For example, MY_VAR = 'my value' creates a variable ${MY_VAR} with the specified text as the value.
Getting variables from a special function
Alternatively, variable files can have a special get_variables (or getVariables) method that returns variables as either a Python dictionary or a Java Map. This method can even take arguments, which makes this approach quite flexible.

How to include Variable Files

All test data files can import variables using the Variables setting in the Setting table, in the same way as resource files are imported using the Resource setting. Similarly to resource files, the path to the imported variable file is considered relative to the directory where the importing file is, and if not found, it is searched from the directories in PYTHONPATH. The path can also contain variables, and slashes are converted to backslashes on Windows. If an argument file takes arguments, they are specified in the cells after the path and also they can contain variables. Example:
Setting Value Value Value
Variables myvariables.py    
Variables ../data/variables.py    
Variables ${RESOURCES}/common.py    
Variables taking_arguments.py arg1 ${ARG2}
All variables from a variable file are available in the test data file that imports it. If several variable files are imported and they contain a variable with the same name, the one in the earliest imported file is taken into use. Additionally, variables created in Variable tables and set from the command line override variables from variable files.
Another way to take variable files into use is using the command line option --variablefile. Variable files are referenced using a path to them, and possible arguments are joined to the path with a colon (:):
--variablefile myvariables.py
--variablefile path/variables.py
--variablefile /absolute/path/common.py
--variablefile taking_arguments.py:arg1:arg2
Variables in these files are globally available in all test data files, similarly as individual variables set with the --variable option. If both --variablefile and --variable options are used and there are variables with same names, those that are set individually with --variable option take precedence.

Creating Variables Directly

Basic Syntax

When variable files are taken into use, they are imported as Python modules and all their global attributes that do not start with an underscore ( ) are considered to be variables. Because variable names are case-insensitive, both lower- and upper-case names are possible, but in general, capital letters are recommended for global variables and attributes.

Example:

VARIABLE = "An example string"
ANOTHER_VARIABLE = "This is pretty easy!"
INTEGER = 42
STRINGS = ["one", "two", "kolme", "four"]
NUMBERS = [1, INTEGER, 3.14]
In the example above, variables ${VARIABLE}, ${ANOTHER_VARIABLE}, and so on, are created. The first two variables are strings, the third one is an integer and the last two are lists. All these variables are scalar variables, even the ones containing lists as values. To create list variables, the variable name must be prefixed with LIST_ (note the two underscores). Example:
LIST__STRINGS = ["list", "of", "strings"]
LIST__MIXED = ["first value", -1.1, None, True]
The variables in both the examples above could be created also using the Variable table below.

Variable Value Value Value *Value
${VARIABLE} An example string      
${ANOTHER_VARIABLE} This is pretty easy!      
${INTEGER} ${42}      
${STRINGS} one two three four
${NUMBERS} ${1} ${INTEGER} ${3.14}  
@{STRINGS} list of strings  
@{MIXED} first value ${-1.1} ${None} ${True}

Using Objects as values

Variables in variable files are not limited to having only strings or other base types as values like variable tables. Instead, their variables can contain any objects. In the example below, the variable ${TELEPHONES} is a python dictionary containing two pairs of values: TELEPHONES = {'jack': 4098, 'sape': 4139}

Creating Variables Dynamically

Because variable files are created using a real programming language, they can have dynamic logic for setting variables.
import os
import random
import time
USER = os.getlogin()                # current login name
RANDOM_INT = random.randint(0, 10)  # random integer in range [0,10]
CURRENT_TIME = time.asctime()       # timestamp like 'Thu Apr  6 12:45:21 2006'
if time.localtime()[3] > 12:
    AFTERNOON = True
else:
    AFTERNOON = False
The example above uses standard Python libraries to set different variables, but you can use your own code to construct the values. The example below illustrates the concept, but similarly, your code could read the data from a database, from an external file or even ask it from the user.
import math
def get_area(diameter):
    radius = diameter / 2
    area = math.pi * radius * radius
    return area
AREA1 = get_area(1)
AREA2 = get_area(2)

Selecting which Variables to Include

When Robot Framework processes variable files, all their attributes that do not start with an underscore are expected to be variables. This means that even functions or classes created in the variable file or imported from elsewhere are considered variables. For example, the last example would contain the variables ${math} and ${get_area} in addition to ${AREA1} and ${AREA2}. Normally the extra variables do not cause problems, but they could override some other variables and cause hard-to-debug errors. One possibility to ignore other attributes is prefixing them with an underscore:
import math as _math
def _get_area(diameter):
    radius = diameter / 2.0
    area = _math.pi * radius * radius
    return area
AREA1 = _get_area(1)
AREA2 = _get_area(2)
If there is a large number of other attributes, instead of prefixing them all, it is often easier to use a special attribute all and give it a list of attribute names to be processed as variables.
import math
__all__ = ['AREA1', 'AREA2']
def get_area(diameter):
    radius = diameter / 2.0
    area = math.pi * radius * radius
    return area
AREA1 = get_area(1)
AREA2 = get_area(2)
Note that the all attribute is also, and originally, used by Python to decide which attributes to import when using the syntax "from modulename import *".

Getting Variables from a Special Function

An alternative syntax for getting variables is having a special get_variables function (also camelCase syntax getVariables is possible) in the variable file. In this case, Robot Framework calls that function and it returns variables as a Python dictionary or its subclass or a Java Map, with variable names as keys and variable values as values. Variables are considered to be scalars, unless prefixed with LIST__, and values can contain anything. The example below is identical to the first examples of creating variables directly.
def get_variables():
    variables = { "VARIABLE ": "An example string",
                  "ANOTHER_VARIABLE": "This is pretty easy!",
                  "INTEGER": 42,
                  "STRINGS": ["one", "two", "kolme", "four"],
                  "NUMBERS": [1, 42, 3.14],
                  "LIST__STRINGS": ["list", "of", "strings"],
                  "LIST__MIXED": ["first value", -1.1, None, True] }
    return variables
get_variables can also take arguments, which facilitates changing what variables actually are created. Arguments to the function are set just as any other arguments for a Python function. When taking variable files into use in the test data, arguments are specified in cells after the path to the variable file, and in the command line they are separated from the path with a colon. The dummy example below shows how to use arguments with variable files. In a more realistic example, the argument could be a path to an external text file or database where to read variables from.
variables1 = { 'scalar': 'Scalar variable',
               'LIST__list': ['List','variable'] }
variables2 = { 'scalar' : 'Some other value',
               'LIST__list': ['Some','other','value'],
               'extra': 'variables1 does not have this at all' }
def get_variables(arg):
    if arg == 'one':
        return variables1
    else:
        return variables2
The most notable benefit of using get_variables instead of defining variables directly as global attributes of the variable file is the ability to use arguments. On the other hand, the main drawback is that this approach always requires some actual programming.

Advanced Features

Handling keywords with same names

Keywords that are used with Robot Framework are either library keywords or user keywords. The former come from standard libraries or external libraries, and the latter are either created in the same file where they are used or then imported from resource files. When many keywords are in use, it is quite common that some of them have the same name, and this section describes how to handle possible conflicts in these situations.

Keyword scopes

When only a keyword name is used and there are several keywords with that name, Robot Framework attempts to determine which keyword has the highest priority based on its scope. The keyword's scope is determined on the basis of how the keyword in question is created:
  • Created as a user keyword in the same file where it is used. These keywords have the highest priority and they are always used, even if there are other keywords with the same name elsewhere.
  • Created in a resource file and imported either directly or indirectly from another resource file. This is the second-highest priority.
  • Created in an external test library. These keywords are used, if there are no user keywords with the same name. However, if there is a keyword with the same name in the standard library, a warning is displayed.
  • Created in a standard library. These keywords have the lowest priority.

Specifying a keyword explicitly

Scopes alone are not a sufficient solution, because there can be keywords with the same name in several libraries or resources, and thus, they provide a mechanism to use only the keyword of the highest priority. In such cases, it is possible to use the full name of the keyword, where the keyword name is prefixed with the name of the resource or library and a dot is a delimiter.
With library keywords, the long format means only using the format LibraryName.Keyword Name. For example, the keyword Run from the OperatingSystem library could be used as OperatingSystem.Run, even if there was another Run keyword somewhere else. If the library is in a module or package, the full module or package name must be used (for example, com.company.Library.Some Keyword). If a custom name is given to a library using the WITH NAME syntax, the specified name must be used also in the full keyword name.
Resource files are specified in the full keyword name, similarly as library names. The name of the resource is derived from the basename of the resource file without the file extension. For example, the keyword Example in a resource file myresources.html can be used as myresources.Example. Note that this syntax does not work, if several resource files have the same basename. In such cases, either the files or the keywords must be renamed. The full name of the keyword is case-, space- and underscore-insensitive, similarly as normal keyword names.

Specifying library search order

It is also possible to use the keyword Set Library Search Order from BuiltIn library to specify the priority order between test libraries that are in use. If multiple keywords with the same name are found, libraries implementing them are gone through in the library search order. The first library containing the keyword is selected and the keyword from that library is used. If keyword is not found, test execution fails.
There is no need to use the long format LibraryName.Keyword Name notation when library search order is set. This makes it easier to to use multiple instances of the same library as Set Library Search Order keyword can be used to switch between the libraries. This enables using same user keywords instead of duplicating those for all the library instances.
Library search order is valid in the suite level meaning it is available only in the suite where it was set.

Timeouts

Warning: Using timeouts might cause test to be executed slower if the Python interpreter is used. The slowing down is caused by Python's threading implementation.
Keywords may be problematic in situations where they take exceptionally long to execute or just hang endlessly. Robot Framework allows you to set timeouts both for test cases and user keywords, and if a test or keyword is not finished within the specified time, the keyword that is currently being executed is forcefully stopped. Stopping keywords in this manner may leave the library or system under test to an unstable state, and timeouts are recommended only when there is no safer option available. In general, libraries should be implemented so that keywords cannot hang or that they have their own timeout mechanism, if necessary.

Test Case Timeout

The test case timeout can be set either by using the Test Timeout setting in the Setting table or the [Timeout] setting in the Test Case table. Test Timeout in the Setting table defines a default test timeout value for all the test cases in the test suite, whereas [Timeout] in the Test Case table applies a timeout to an individual test case and overrides the possible default value.
Using an empty [Timeout] means that the test has no timeout even when Test Timeout is used. It is also possible to use value NONE for this purpose.
Regardless of where the test timeout is defined, the first cell after the setting name contains the duration of the timeout. The duration must be given in Robot Framework's time format, that is, either directly in seconds or in a format like 1 minute 30 seconds. It must be noted that there is always some overhead by the framework, and timeouts shorter than one second are thus not recommended.
The default error message displayed when a test timeout occurs is Test timeout <time> exceeded. It is also possible to use custom error messages, and these messages are written into the cells after the timeout duration. The message can be split into multiple cells, similarly as documentations. Both the timeout value and the error message may contain variables.
If there is a timeout, the keyword running is stopped at the expiration of the timeout and the test case fails. However, keywords executed as test teardown are not interrupted if a test timeout occurs, because they are normally engaged in important clean-up activities. If necessary, it is possible to interrupt also these keywords with user keyword timeouts.
Test Case timeout examples:
Setting Value Value
Test Timeout 2 minutes  
Test Case Action Argument Argument Argument
Default Timeout [Documentation] Timeout from the Setting table is used    
  Some Keyword argument    
         
Override [Documentation] Override default, use 10 seconds timeout    
  [Timeout] 10    
  Some Keyword argument    
         
Custom Message [Documentation] Override default and use custom message    
  [Timeout] 1min 10s This is my custom error. It continues here.
  Some Keyword argument    
         
Variables [Documentation] It is possible to use variables too    
  [Timeout] ${TIMEOUT}    
  Some Keyword argument    
         
No Timeout [Documentation] Empty timeout means no timeout even when Test Timeout has been used  
  [Timeout]      
  Some Keyword argument    
         
No Timeout 2 [Documentation] Empty timeout using NONE    
  [Timeout] NONE    
  Some Keyword argument    

User Keyword Timeout

A timeout can be set for a user keyword using the [Timeout] setting in the Keyword table. The syntax for setting it, including how timeout values and possible custom messages are given, is identical to the syntax used with test case timeouts. If no custom message is provided, the default error message Keyword timeout
Edit | Attach | Watch | Print version | History: r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r1 - 2011-10-17 - unknown
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    EMI All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright &© 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback