Federation in AMGA

This section presents the Federation mechanisms that are part of AMGA since version 2.1.

Overview

The federation mechanism in AMGA provides a user a virtualized view on metadata as if one metadata server has all data which are actually distributed at multiple sites.

Federation in AMGA is very similar to mounting in NFS. That is, if a remote directory is mounted to a local directory, a user may access it as it is located at local site. Federation actually redirects user's query to other multiple AMGA nodes and integrates the results. The list of other AMGA nodes to redirect user's query should be set properly on a per-directory basis.

AMGA provides two types of federation methods; server-side federation and client-side federation. In server-side federation, an AMGA server does the actual federation, which redirects a query and integrates results. Meanwhile, in client-side federation, an AMGA server provides a client with a list of AMGA nodes to redirect user's query and a client side does the actual federation. The client-side federation puts little overhead on the AMGA server caused by federation, but it only works with C++ APIs currently. The server-side federation may have high overhead caused by federation at the server-side, but it works with all the clients APIs. This type can be set at the server configuration as follows.

   $ cat amgad.config
   ...
   ServerSideFed = 1 # Federation at Server Sider 
   ...

Setup

It is necessary to have one central AMGA server which maintains all the information about federation. That is, it needs to be configured with the settings of other AMGA nodes that it might be required to contact for federation. The configuration is defined on a per-node basis, which is done through the site_* commands, which can be used to define sites and set their parameters like the login name the central AMGA server will use. Please consult Section Site management for how to configure sites.

   Query> site_add SiteA hostA:8822
   Query> site_set_properties SiteA login root
   Query> site_add SiteB hostB:8822
   Query> site_set_properties SiteB login root

The next step is to initiate federation, that is, to execute the fed_mount commands. Only the root user is allowed to initiate or to control federation.

   Query> fed_mount SiteA /dir
   Query> fed_mount SiteB /dir

Authentication

In federation, connections to other AMGA nodes use the same protocol as the ones from clients to an AMGA server, including the support for authentication and encryption. With the client-side federation, all the necessary configuration settings should be stored at the mdclient.config file. And with the server-side federation, site information should hold necessary configuration settings (e.g., use SSL, authenticate with password, certificates or grid proxies,...).

Accessing Patterns

Accessing patterns differ depending on a command type. For example, if the "listattr" command accesses a federated directory, it is redirected to one of random federated hosts (access random type). However, in case of the "SELECT" command, a query is redirected into all the federated hosts (access all type). The following list types and corresponding commands. Other commands not listed below may result an error if they access a federated directory.

• Access random type : listAttr, addentry, addentries, stat, insert, INSERT
• Access all type : getAttr, find, listentries, ls, dir, chmod, chown, acl_*, removeEntries, selectattr, SELECT, DELETE, setAttr, updateattr, update, UPDATE, rm

Fault Handling

It is possible to have a failure while connecting and executing a command for federation. In case of "access random type" commands, it reports a success if there is any host which executed the given command successfully. If no hosts have a success, a failure is reported to a user. In case of "access all type" commands, it is allowed to define policies. It is possible to set two policies; AllowFedConnFail and AllowFedExecFail. This policies can be set at the server configuration file as follows.

   $ cat amgad.config
   ...
   AllowFedConnFail = 1   # Allows connection failures in federation
   AllowFedExecFail = 1   # Allows execution failures in federation
   ...

AllowFedConnFail allows connection failures in federation. If AllowFedConnFail is set to 1, then it reports a success if there is any one host having a successful connection. If it is set to 0, then all federated hosts should be connectable in order not to have a failure. AllowFedExecFail is very similar to AllowFedConnFail, but it is related to execution failures. By default, these policies are set to 1.

Even though there was a success to execute a command, a user may want to know there was failures with federation. There are two commands supporting this purpose. With these commands, it is possible to check partial failures when the policy is set to ignore the connection or execution failures. The fed_policy command shows the current policy about federation. The first line of the result shows whether connection failures are allowed in federation, and the second line shows whether execution failures are allowed. The fed_check_failure command shows whether last command had failures. The first line of the result shows whether connection failures happened, and the second line shows whether execution failures happened.

   Query> fed_policy
   >> 1
   >> 1
   Query> SELECT * FROM /dir
   >> ...
   Query> fed_check_failure
   >> 1
   >> 0

Limitations

The current limitions related to federations are as follows

• No Schema Heterogeneity is allowed
• Federating Directories with different Names is not allowed
• No Distributed Join is allowed
• No Transaction is Supported

---