header

SOAP Interface

The AMGA implementation uses TCP streaming to communicate between client and server which shows a very promising performance. To meet the EGEE requirements, we have also implemented an alternative SOAP-based frontend.

The interface provide by the SOAP frontend is compliant to OGF WS-DAI & WS-DAIR specification.

Installation

SOAP client installation

To install the command line soap client tools and C++ api, you need to install glite-amga-cli and glite-amga-soapclient packages. By default the packages will be installed in /usr.

  $ rpm -i glite-amga-cli-${version}.${platform}.rpm
  $ rpm -i glite-amga-soapclient-${version}.${platform}.rpm

The configuration file(mdclient.conf) used in the soap client tools are the same as one used in the mdclient. If you want to change the configuration, you should go on and read Configuring Configuration of the C++ and Java command line clients .

SOAP server installation

The AMGA SOAP server depends on the AMGA server package. To install the soap server, you need to install glite-amga-server and glite-amga-soapserver packages.

  $ rpm -i glite-amga-server-${version}.${platform}.rpm
  $ rpm -i glite-amga-soapserver-${version}.${platform}.rpm

To setup a database and ODBC driver, refers Server installation . You should now be able to start the service and verify whether it is running by doing

  $ /etc/init.d/mdsoapservice start
  $ tail /var/log/amgasoap.log

The AMGA SOAP server is configured using the mdsoapserver.config file. The options are almost same as the options in amgad.config file. For the configuration on amgad.config file, you should go on and read Configuring the AMGA Server and the Replicatin Daemon . For a soap server, the following additional options are supported.

How to use SOAP client tools

You can access the metadata catalogue with the soap client tools. These tools are:

The wsdair_access_execute tool is very similar to mdclient tool. The following is an example with the the wsdair_access_execute. The default port number SOAP server uses is 8833. However, you can change the port number you want to connect with "-p" option.

  $ wsdair_access_execute -p 8833
  Query> ls
  /STAFF
  /PROJ
  /WORKS
  Query> SELECT * FROM STAFF
  STAFF.FILE
  STAFF.EMPNUM
  STAFF.EMPNAME
  STAFF.GRADE
  STAFF.CITY
  S1
  E1
  Alice
  12
  Deale
  S2
  E2
  Betty
  10
  Vienna

The wsdair_access_cli tool is also similar to mdcli tool, which allows you to directly issue metadata commands on the shell. The following is an example with the the wsdair_access_cli.

  $ wsdair_access_cli -p 8833 "SELECT * FROM WORKS"
  WORKS.FILE
  WORKS.EMPNUM
  WORKS.PNUM
  WORKS.HOURS
  W1
  E1
  P1
  40
  W2
  E1
  P2
  20
  W4
  E1
  P4
  20
  W5
  E1
  P5
  12

The wsdair_access_factory tool creates an indirect data resource with the given command. When creating "indirect data resouce", a user does not receive the results in the response to a request made, and the results is made available to the user indirectly as a new data resource. The following is an example with the the wsdair_access_execute.

  $ wsdair_access_factory -p 8833 "SELECT * FROM WORKS"
  http://localhost:8833/SQLResponse/responses186_4
  DRAN : http://localhost:8833/SQLResponse/responses186_4

The newly created indirect data resource can be accessed with wsdair_rowset command. The follwoing example retrieves two rows from the indirect data resource with wsdair_rowset command. "-P" option defines the location where the retrieval starts and "-C" option is the number of rows to be retrieved.

  $ wsdair_rows -p 8833 -P 0 -C 2 http://localhost:8833/SQLResponse/responses186_4
  /SQLResponse/responses186_4.FILE
  /SQLResponse/responses186_4.EMPNUM
  /SQLResponse/responses186_4.PNUM
  /SQLResponse/responses186_4.HOURS
  W1
  E1
  P1
  40
  W2
  E1
  P2
  20

The wsdair_list lists all the data resources including indirect data resources created by wsdair_access_factory.

  $ wsdair_list -p 8833
  Address : http://localhost:8833/SQLResponse/responses186_4
  DRAN : http://localhost:8833/SQLResponse/responses186_4
  Address : http://localhost:8833/SQLAccess/Metadata
  DRAN : http://localhost:8833/SQLAccess/Metadata

The wsdair_destory utility removes an indirect data resource. The following is an example with the the wsdair_destroy.

  $ wsdair_destroy -p 8833  http://localhost:8833/SQLResponse/responses186_4

  $ wsdair_list -p 8833
  Address : http://localhost:8833/SQLAccess/Metadata
  DRAN : http://localhost:8833/SQLAccess/Metadata

WS-DAIR Architecture

Data access service and data resources

WS-DAI defines that a data access service is a web service that implements one or more of the WS-DAI specified interfaces to provide access to data resources. A single standalone service manages all the WS-DAI & WS-DAIR interfaces in AMGA WS-DAIR.

There are two types of data accesses defined in WS-DAI. The first one is "direct data access", which means that a consumer receives a direct response, containing the requested data, following a request made to a data access service. The second is "indirect data access". It means that a consumer does not receive the results in the response to a request made to a data access service. The request to access data will be processed by the data access service and data resource, with the results being made available to the consumer indirectly as a new data resource, often through a different data access service that may support a different set of interfaces.

To support these two data access types, there are two types of resources in AMGA WS-DAIR. The first one is SQLAccess type, which is the data source to support "direct data access." In AMGA WS-DAIR, it is possible to have only one SQLAccess resource; a relational database which is managed by AMGA for a metadata catalogue service. A consumer accesses it through the SQLAccess interface. And the second one is a SQLRowset type. It is a data resource newly created by "indirect data access." A SQLRowset type resource is a data source that AMGA WS-DAIR service accesses to handle requests from both SQLResponse interface and SQLRowset interface. Because AMGA does not support a stored procedure, the queried result is always a single dataset or number of updated rows. Therefore, it would be redundant if we separate SQLRowset resource from SQLResponse resource.

WS-DAI consider two kinds of data resources: externally managed data resource and service managed data resource. In AMGA WS-DAIR, data resources are externally managed. This means that the managed data resources exist outside the WS-DAIR services and their lifetime can be managed in ways that are not specified in the WS-DAI specifications. AMGA WS-DAIR services do not support WSRF, which is optional in WS-DAI.

Data Resource Abstract Name & Data Resource Address

A data resource abstract name is defined as a unique and persistent name for a data resource. The WS-DAI specification mandates an abstract name to use the form of a URI. A URI has two specializations known as URL and URN. The AMGA WS-DAIR implementation uses the form of a URL. A data resource address is a name that specifies the locations of a data resource as accessed through a data access service. A data resource address is the same as a data resource abstract name in the AMGA WS-DAIR implementation. A data resource abstract name on the initial data resource for "direct data access" can be configured in configuration file, which is usually located in etc directory of installed root.

Supported Dataset and Language

The WS-DAIR specification mandates the SQLAccess interface provide at least WebRowSet format. The AMGA WS-DAIR implementation supports WebRowSet format. Supported Languages are <direct_data_statements> in SQL-92 entry level & and some of AMGA metadata languages

Installation from Source

To install the SOAP server from source, you will need to frist download the source distribution from the download directory.

For compilation you need to install gsoap package.

  $ wget http//sourceforge.net/projects/gsoap2/files/gSOAP%20/gsoap_2.7.7.tar.gz
  $ tar xzvf gsoap_2.7.7.tar.gz
  $ cd gsoap-2.7
  $ ./configure --prefix=$PWD/out
  $ make
  $ make install
  $ cd ..

Now you should be ready to compile and install the AMGA server

  $ wget
http://amga.web.cern.ch/amga/downloads/glite-amga-server-${version}.tar.gz
  $ tar xzvf glite-amga-server-${version}.tar.gz
  $ cd org.glite.amga.server
  $ sh bootstrap
  $ ./configure --with-readline --with-editline-prefix=/usr --with-server
    --with-soapserver --with-gsoap-location=../gsoap-2.7/out --enable-rpm-rules
  $ make 
  $ make install

You also need a database and the appropriate ODBC driver To setup a database and ODBC driver, refers Server installation .

Howto make WSDAIR clients

WS-DAIR is a soap-based interface, that allows you to create clients by creating interface bindings in any language out of the so called wsdl files, that describe the soap interface.

In the following an example is given how to create a client using the Ruby programming language for AMGA from the wsdl. Creating other interfaces for other languages is similar.

You need the 1.8 version of Ruby and the 1.5.8 version of soap4r, that provides the soap support for Ruby. The instructions are for debian, but this should work similarly on other OSs.

Install the gem support package 

$ apt-get install rubygems
$ gem install soap4r
$ export PATH=/var/lib/gems/1.8/bin/:$PATH


$ mkdir wsdair
$ cd wsdair
$ cp org.glite.amga.server/src/soap/base/*.xsd .
$ cp amga/org.glite.amga.server/src/soap/base/*.wsdl .

Unfortunately there is an incompatibility between the gsoap soap implementation and the python/ruby implementations that were studied. You will therefore need to slightly modify the imple.wsdl file: Please insert the folling lines before the part <binding> starts in imple.wsdl

<wsdl:import namespace="http://www.ggf.org/namespaces/2005/12/WS-DAIR"
location="./sqlaccess.wsdl"/>
<wsdl:import namespace="http://www.ggf.org/namespaces/2005/12/WS-DAIR"
location="./sqlresponse.wsdl"/>
<wsdl:import namespace="http://www.ggf.org/namespaces/2005/12/WS-DAIR"
location="./sqlrowset.wsdl"/>
<wsdl:import namespace="http://www.ggf.org/namespaces/2005/12/WS-DAIR"
location="./wsdai_core_porttypes.wsdl"/>

Now the Ruby client can be created

$ wsdl2ruby.rb -d --type client --force --wsdl imple.wsdl

Several .rb ruby interface files are being created. If you have to use a gem for the 1.5.8 soap4r packages, like on debian, you need to edit wsdairDriver.rb and add the following 2 lines at the top

require 'rubygems'
gem 'soap4r'

so that the soap4r packages are being found.

Create the following small demonstration client: 

$ cat amgaclient.rb
#!/usr/bin/ruby
# Start me with -d to get debug output
#$DEBUG = true

require 'wsdairDriver.rb'

endpoint_url = 'http://localhost:8833/'
obj = SQLAccessPT.new(endpoint_url)
obj.wiredump_dev = STDERR if $DEBUG

expression = SQLExpressionType.new("SELECT * FROM /tutorial/cities")
request = SQLExecuteRequest.new("AMGA_Metadata",
"http://java.sun.com/xml/ns/jdbc", expression)

response = obj.sQLExecute(request)
print response

dataset=response.sQLDataset() # Type: SQLDatasetType
print "Dataset format URL >#{dataset.datasetFormatURI()}<\n"
print "Dataset data >#{dataset.datasetData()}<\n"
print "Update count >#{dataset.sQLUpdateCount()}<\n"
print "Output parameter >#{dataset.sQLOutputParameter()}<\n"
print "Return value >#{dataset.sQLReturnValue()}<\n"
print "Communication >#{dataset.sQLCommunicationsArea()}<\n"

returnValue = dataset.sQLReturnValue()
data = dataset.datasetData()
print "Return value >#{returnValue}<\n"
print "Data retrieved >#{data}<\n"

wrs = data.webRowSet()
print "Web Row set >#{wrs}<\n"

wrsMetadata = wrs.metadata()
print "Web Row set metadata >#{wrsMetadata}<\n"
wrsColumnCount = wrsMetadata.column_count()
print "Web Row set columns >#{wrsColumnCount}<\n"

for i in 0...wrsColumnCount.to_i()
  print "#{wrsMetadata.column_definition[i].table_name} :
#{wrsMetadata.column_definition[i].column_name}\n"
  end
  
  currentRow = wrs.data().currentRow()
  
  print "WRS Data >#{wrs.data()}<\n"
  currentRow.each { |v| p v }

And change the query in the expression variable as well as the endpoint_url according to your setup.

Start the soap-server: 

$ ./mdsoapserver -c mdsoapserver.config

now you can run the client: 

$ chmod +x amgaclient.rb
$ ./amgaclient.rb
Generated on Mon Apr 16 13:59:18 2012 for AMGA by  doxygen 1.4.7