Document Link: [ e-IRG White Paper 2009 (PDF) ]

NA3.2 Sustainability and Collaboration

• Foreword
  • The e-IRG White Paper is a live document summarising on-going discussions around key e-Infrastructure areas and topics that require immediate policy actions.
  • The reflections identified three key target audiences for this document:
    • Policy makers on the governmental and inter-governmental levels, dealing with funding, privacy and other issues that are becoming more and more crucial due to the broader uptake of the e-Infrastructure–enabled technologies and processes.
    • Service-providers on the e-Infrastructure domain, such as organisations operating research networks or computing centres – or projects that build on this base in order to provide higher-level multidisciplinary services.
    • Existing and new user communities, looking for a broad overview of the capacities and capabilities that the current and near-future e-Infrastructure can provide.
  • Translating the users’ needs into coherent technological, political and organisational plans is an iterative process and this White Paper should be a powerful instrument for fulfilling this goal.

• Introduction
  • The goal of this White Paper is to identify strategic topics in need of policy-level action from a torrent of novel ideas, technologies and paradigms.
  • Cloud Computing and Virtualisation illustrates the constant need for awareness of the new opportunities provided by innovative technologies that can be integrated into the overall e-Infrastructure. (ECB Report)
  • Security, a holistic approach examines security from different angles (“as a whole”) covering all e-Infrastructure aspects in an effort to minimize overlaps and promote synergies among different e-Infrastructure service providers (e.g. network and grid providers) for minimising a problem concerning all e-Infrastructures and all users.

• Terminology
  • e-IRG Education and Training Task Force (ETTF) – a task force created in the framework of e-IRG investigating and proposing strategic actions in the area of education and training in e-Infrastructure and its curricula.
  • e-IRG Sustainable e-Infrastructure task force (SeITF) - a task force created in the framework of e-IRG investigating and proposing strategic actions in the area of e-Infrastructure sustainability. A related report has been published available at the e-IRG site;
  • e-Science – the invention and application of ICT-enabled methods to achieve new, better, faster or more efficient research, innovation, decision support or diagnosis in any discipline. It draws on advances in computing science, computation and digital communications;
  • electronic Very Long Baseline Interferometry (eVLBI) is the process of using high speed networks to connect radio telescopes separated by large distances (100-1000s of km) instead of the traditional method of recording onto magnetic tape and shipping the recorded data to a central correlator. It is part of a collaboration of the major radio astronomical institutes in Europe, Asia and South Africa performing high angular resolution observations of cosmic radio sources; (ECB Report)
  • Education and Training Community Group (ET-CG) – is an OGF group aiming at bringing together practitioners in grid-related education and training (E&T) to share and develop best practice, to stimulate greater investment in Grid-related E&T and above all to build a mutually supportive community of grid trainers and educators;
  • European Grid Initiative (EGI) is the next phase of the implementation of capacity computing in Europe. EGI will manage no compute power, but guarantees transnational access to data and services;
  • Grid is a system that federates, shares and coordinates distributed resources from different organisations which are not subject to centralized control, using open, general-purpose and in some cases standard protocols and interfaces to deliver non-trivial qualities of service. The Grid is used by Virtual Organisations, i.e. thematic groups of users crossing administrative and geographical boundaries;
  • International Thermonuclear Experimental Reactor (ITER) – is a joint international research and development project that aims to demonstrate the scientific and technical feasibility of fusion power. Fusion is the energy source of the sun and the stars. On earth, fusion research is aimed at demonstrating that this energy source can be used to produce electricity in a safe and environmentally benign way, with abundant fuel resources, to meet the needs of a growing world population; (ECB Report)
  • Research and Education Federations (REFEDs) wish to address the need of existing and emerging e-identity federations operating in the field of education and research in Europe, America and Asia to collaborate on policy issues. Collaboration in this area is needed to learn from each other, to align policy in order to facilitate inter-federation work, and to reach outside the research and education community to enable federated access in the larger world;
  • White paper - an authoritative report that often addresses problems and how to solve them. White papers are used to educate readers and help people make decisions. They are often used in politics and business. Commission White Papers are documents containing proposals for Community action in a specific area. In some cases they follow a Green Paper published to launch a consultation process at European level. When a White Paper is favourably received by the Council, it can lead to an action programme for the Union in the area concerned;

• Executive Summary of the e-IRG White Paper
  • The document presents key e-Infrastructure issues and topics requiring policy action at the national and EU levels.
  • The purpose of the e-IRG White Papers is to provide a “snapshot” of organisational, technological, economical, and social developments in the e-Infrastructure domain.
  • Seven topics are examined in the e-IRG White Paper 2009: Global Collaboration, Education and Training in the Use of e-Infrastructure, Grid and Cloud Computing, Security - A Holistic Approach, Service-centric e-Infrastructures through Virtualisation, Remote Instrumentation, and Sustainability of the Computing-Related e-Infrastructure.
  • The Grid approach is based on requirements that were both quantitatively and qualitatively very challenging, necessitating a solution for managing remote resources that are not controlled by any single organisation in a secure and comprehensive manner.
  • An issue crosscutting all of the above areas is the sustainability of e-Infrastructure. If relevant communities do not trust that services will be available in the future, the issues of integrating new components into the existing e-Infrastructure and improving security cannot be approached in a stable manner;

• 1. Global Collaboration
  • It has been long recognised that e-Infrastructures are not restricted to country borders; on the contrary they must be planned and set up as global infrastructures in order to create an effective and competitive scientific ecosystem.
  • The engagement in worldwide collaborations between the various e-Infrastructures will increase cross fertilization of novel ideas across large scientific communities and harmonise policies and best practices between trans-continental large scale e-Infrastructures towards the worldwide Knowledge Society.
  • Services should aim to hide the complexity of the resources while providing ubiquitous access and enhanced usability;
  • Collaboration for innovation and harmonisation of e-Infrastructures worldwide is the main goal of this topic with multiple advantages regarding the e-Infrastructures themselves, being the ease of use and the resulting socio-economic impact.
  • Global collaboration will also add value to the European e-Infrastructure by increasing the efficiency and competitiveness of the European scientific community and by favouring innovation and the resulting socio-economic impact. (ECB Report - extend to user base case)
  • The USA and Japan are at the top of the list of countries that have a mature e-Infrastructure.
  • Other countries that can offer new perspectives due to their emerging technologies and fast growth are China and India, the last one especially as an HPC technology developer.
  • Also some large European projects, fundamental for the development of science and industry in Europe, which strongly depend on e-Infrastructures for achieving their full potential, should be included in the list of potential collaborations.
  • The Computing Community Consortium – CCC (NSF) – was created by the Computing Research Association (CRA) in 2006 to catalyse the computing research community to debate more audacious research challenges; to build consensus around major, long-term research directions; to articulate research agendas; to evolve the most promising visions toward clearly defined initiatives; and to work with funding organizations to move those initiatives toward funded programs.
  • The Advanced Scientific Computing Research – ASCR (DoE)
  • In Japan, the Cyber Science Infrastructure – CSI (NII) – is the foundation for Japan's scientific research and education activities aiming at promoting the strengthening of international competitiveness in order to maintain cutting-edge scientific development.
  • In China, the Chinese Academy of Sciences – CAS) – is responsible for undertaking government assigned projects with regard to key S&T problems in the process of social and economic development;
  • In the greater Asian region the EC is promoting e-Infrastructure bonds with Europe through funding projects in the area of networking (such as the TEIN series, currently TEIN3 which connects also Australia and Japan) and Grid computing (such as EuChinaGrid and EuAsiaGrid).
  • In India the Centre for Development of Advanced Computing – C-DAC – aims at continuing to create and deploy the finest talent for further expanding the frontiers of High Performance Computing and Communication Technologies and its applications
  • In Latin America, CLARA – Cooperación Latino Americana de Redes Avanzadas – is constituted by 17 Latin American countries and its Assembly – where each country has a representative – meets every six months to define courses of action and the policies to be implemented.
  • In addition, the EELA project series are also an important link to the distributed computing e-Infrastructures in Latin America.
  • Global collaborations are becoming an important and urgent issue for worldwide harmonisation and development of e-Infrastructures. The following prompt actions are proposed:
    • Initial interaction should be followed up by regular and formal contacts. Setting up a specific e-IRG Task Force should be investigated. The Task Force would be in charge of not just investigating new possible collaborations but also actively engaging in the collaborations and regularly reporting on the results achieved to the e-IRG forum during its meetings;
  • RECOMMENDATION:
    • The e-IRG recommends that Global Collaboration should move from its ad-hoc character to a more structured and continued mode adequately supported by the EC and national funding agencies; Europe should continue to take a leading role in global collaborations. (ECB Report - international science software foundation case)

NA3.3 Standardization and Integration

• Executive Summary of the e-IRG White Paper
  • Cloud computing and Virtualisation, are among the most promising and innovative ICT technologies. (ECB Report)
  • Seeking a way to combine best aspects of the Grid and Cloud Computing requires resolving technological and policy level conflicts between two paradigms that have started based on very different requirements, use cases and basic assumptions. (ECB Report)
  • The analysis of the state of the art of both of these computing paradigms indicates that the Cloud offerings are not yet sophisticated enough to support complex “Grid-like” use cases directly; (ECB Report)
  • therefore, it is important to study both opportunities provided by the integration of cloudbased offerings into research e-Infrastructure and by promoting the reuse of the results of the Grid initiatives in the Cloud context. (ECB Report)
  • In other words, the e-IRG believes that the long-term interests of the research community would be best served by increasing access to and use of a mixture of grid- and cloud-based services and technologies, and encourages the integration of cloudbased services into the existing e-Infrastructure. (ECB Report)
  • However the growing interest on the technology, mainly due to the economical benefits that brings along, advocate for some first policy actions: integration of virtualisation technologies in the current research e-Infrastructure and encouragement of open standards are such initial actions.
  • The current multitude of interfaces from different vendors may otherwise endanger the e-IRG vision of an open e-Infrastructure that allows optimal use of all electronically available resources.

• 1. Global Collaboration
  • Open Grid Forum – OGF - is an open community committed to driving the rapid evolution and adoption of applied distributed computing. Applied distributed computing is critical to developing new, innovative and scalable applications and infrastructures that are essential to productivity in the enterprise and within the science community. OGF accomplishes its work through open forums that build the community, explore trends, share best practices and consolidate these best practices into standards.

Document Link: [ e-IRG Blue Paper 2010 (PDF) ]

NA3.2 Sustainability and Collaboration

• Executive Summary
  • There are many real benefits that stem from a common e-Infrastructure across the European Research Area (ERA), including:
    • Benefits: '2. avoiding unnecessary duplication in provision of ICT solutions'
    • Benefits: '3. leveraging existing expertise and experience'
  • Collaboration between RI and e-Infrastructures should be actively supported at all levels, to their mutual benefit.
  • E-Infrastructure tools and resources must be developed in a global context to support researchers’ global endeavours.
  • Today, research is increasingly dependent on computational resources, and many scientific domains increasingly require large scale computing for their work.
  • This trend is driven by the growing use of computational methods, simulation, and data analysis, and is sustained by falling unit costs in this area.
  • VRCs are also known as Virtual Organisations (VOs) and facilitate integration of distributed research capacities/resources, virtual mobility of physically distant researchers, better access to research results, and regional and global collaboration and partnerships.
  • The over-riding themes that emerge from the Blue Paper are the increasingly global requirements placed on researchers and RI, and the need to evolve Europe’s e-Infrastructure through active engagement of service developers, users and providers, driven by the emerging requirements of the broader research community.

• 1. Foreword - Scope of the document
  • The first ESFRI roadmap already identified the importance of e-Infrastructure, highlighting research communication networks, distributed grids, high-performance computing facilities, digital repositories, data storage, data management, data curation, and the software required to operate, integrate and exploit all of these components (referred to as middleware).
  • The emergence of a large number of ESFRI projects (referred to as Research Infrastructures or RI) involving different sets of the above components, albeit in widely differing fields, has further highlighted the potential for a common e-Infrastructure.
  • There are many potential benefits of a common e-Infrastructure across the European Research Area (ERA), including:
    • avoiding duplication in construction and operation of ICT solutions
    • leveraging expertise and experience to take advantage of previous work
    • facilitating the integration and interoperation of different communities and RI
  • Key questions for research e-Infrastructure include:
    • Balance: Maintaining equilibrium between stability of services and innovation in the e-Infrastructure itself, especially as technology and user requirements evolve
  • Europe’s existing e-Infrastructure is also a research infrastructure
  • What perhaps sets it apart from other RI is its ability to deliver services across a broad spectrum of RI user communities
  • However, the ongoing evolution of e-Infrastructure is central to its ultimate success.
  • Close cooperation across RI and e-Infrastructure will drive this evolution to their mutual benefit.
  • Efficient e-Infrastructure is vital if Europe and European researchers are to exploit global opportunities, and Europe should play a major role in shaping these global solutions.

• 2. e-Infrastructure Services to support Research
  • These modern users increasingly demand ease of use, interoperability between different resources, and guaranteed stability or longevity, such that the effort required to adopt new ICT tools is perceived to be worthwhile.
  • However, the construction of effective, relevant and user-focused services is challenging and depends critically upon understanding user needs. The process is further complicated by
    • poor understanding of users’ requirements (on the part of both users and providers),
    • a lack of common (technical) terminology
    • poor understanding of constraints and opportunities
    • competitive funding environments which often mix research with infrastructure, and
    • a degree of mistrust from all parties.
  • 2.1 - e-Infrastructure layers - Computing
    • They can choose from a variety of computing infrastructures: their laptop or desktop, local computer services at the department or institute level, as well as national or international computing infrastructure.
    • While locally controlled computing services are important and more easily tuned to individual needs, national and international computing infrastructures can provide prioritised research projects with access to resources at scales not otherwise achievable.
    • Increasingly, the boundaries between local, national and international resources are blurring as researchers pool their resources to reach the resource scales required for their research.
    • EGI will also focus on working with new user communities to deliver and enhance services tailored to their research needs.
    • Both EGI and PRACE aim to offer sustainable services that will evolve with changing technology and user demand. User engagement with these initiatives is vital to their vision.
  • 2.1 - e-Infrastructure layers - Middleware
    • In a distributed computing system, for example, middleware is defined as the software layer lying between the operating system and the applications.
    • In a broader sense, middleware is computer software that connects components or applications.
    • Middleware is also used to refer to the ‘glue’ that enables virtualisation technology and services.
  • 2.2 Towards European e-Infrastructure as a Service
    • The computing needs of research user communities are often well ahead of the general ICT-market, and usually impossible to fulfil using generally available ‘commodity’ solutions in standard configurations.
    • In the past, research user communities have been spurred to develop new ICT components or specific solutions (e.g. the Internet and the World Wide Web), of which they have also been the first customers.
    • These services are often unavailable on the open market and thus can anticipate market trends.
    • Another example is the international grid infrastructure, which connects several hundred compute facilities from multiple research entities across the globe to provide a geographic distribution of services and solutions that is difficult to match in the commercial world.
    • The e-Infrastructure is acting both as a new integrating framework or facility (from the e-Infrastructure provider side) and as a new service platform (from the users’ side). (ECB Report - application enabling case)
    • Constructive and collaborative interaction between these stakeholders is key to the success of the e-Infrastructure, and of its users. (ECB Report - more direct user contact case)
    • Using the advanced software services available today, users can automatically find the computing power that best corresponds to their needs, manage their workload, search through data distributed in different sites, handle security and related user authentication and authorisation, provide accounting services and real-time monitoring of their activities, and much more.
    • Thanks to these services, it becomes easy to access and share distributed clusters and archives.
    • In all e‑Infrastructure domains there is a move towards service-orientation and away from the traditional technology- or product-orientation.
    • Thanks to virtualisation, e-Infrastructure can now provide on-demand web-based access to shareable services, reproducing exactly the environment required by a specific user.
    • For example, users can request and be provisioned with N computing nodes on operating system X for M days, or a data access service for K days to be shared by the users of the Virtual Organisation Z.
    • The adoption of such a service-oriented model – in which every component is offered via the web as a service customised according to user demand – increases the sustainability of e-Infrastructure by expanding the pool of potential users to encompass all of society, and by facilitating innovative solutions not yet available commercially.
    • e-IRG has previously documented a set of recommendations related to the move towards service-oriented e-Infrastructure, covering the main policy issues in the fields of grid and cloud computing, Service-centric e-Infrastructures through virtualisation, Remote instrumentation, and Sustainability of computing-related e‑Infrastructure.

NA3.3 Standardization and Integration

• Executive Summary
  • There are many real benefits that stem from a common e-Infrastructure across the European Research Area (ERA), including:
    • Beneifts '4. facilitating the integration and interoperation of different communities and RI'
    • Beneifts: '5. broadening engagement across Europe and internationally'
  • However, full exploitation of these opportunities requires more effective integration of data acquisition infrastructures with data processing infrastructures through standard interfaces to sensor networks and remote instrumentation, the so-called ‘Internet of Things’.

• 1. Foreword - Scope of the document
  • There are many potential benefits of a common e-Infrastructure across the European Research Area (ERA), including:
    • facilitating the integration and interoperation of different communities and RI
    • broadening engagement across Europe and internationally

• 2. e-Infrastructure Services to support Research
  • These modern users increasingly demand ease of use, interoperability between different resources, and guaranteed stability or longevity, such that the effort required to adopt new ICT tools is perceived to be worthwhile.
  • 2.1 - e-Infrastructure layers - Network
    • To facilitate the seamless networking increasingly demanded (and required) by users, networks must interoperate, and thus network providers must work together to develop globally accepted standards, an ongoing process taking place in global forums like IETF, IEEE, ITU, OGF.
  • 2.1 - e-Infrastructure layers - Computing
    • EGI is working to deliver standards-based services to support collaborative distributed computing at the local through to international scale.
  • 2.1 - e-Infrastructure layers - Middleware
    • A central theme of middleware development is the promotion of interoperability and standardisation of networked resources through a common base of protocols and services.
      • Authenticate and authorise secure access to resources (compute, storage, data, etc.).
      • Support open standards to allow the interoperability of different implementations (e.g. middleware repositories and parameter registration).
    • As an example, the European Middleware Initiative (EMI) project is expected to have a challenging time ahead in its effort to define and adopt the standard interfaces that will provide users with a unified access environment.
    • New, more general middleware solutions continue to emerge, virtualising computing resources and making them available through different interfaces: local, grid, or cloud.
  • 2.2 Towards European e-Infrastructure as a Service
    • As yet there is no single unified computing system in the sense that there is a single World Wide Web; rather there is a (relatively small) number of grid, HPC and cloud services, each with different interfaces.
    • Global standardisation efforts are slowly improving interoperability between these components.

Document Link: [ e-IRG White Paper 2011 (PDF) ]

NA3.2 Sustainability and Collaboration

• Introduction and Summary
  • Sustainability of e-Infrastructures is a major concern.
  • In the 2009 White Paper, e-IRG tackled this subject for computing e-Infrastructures and recommended funding and interworking as basic foundations of their sustainability.
  • Who should be the players in the governance model besides the service providers: the governments, the private sector, the user or a combination of these?
  • The 2009 White Paper included a chapter on security. The current White Paper extends this topic by concentrating on authentication, authorisation and accounting.
  • But can we go from T-flops to P-flops and can we really and efficiently use such a computing power?
  • Does a software revolution need to take place before supercomputing can make a leap forward?

• 1 e-Infrastructure Governance
  • 1.2 Context: Current practices - achievements and limitations
    • Maximising the benefit of the investments in research infrastructures for their intrinsic beneficiaries, i.e. the users, requires that the users must be taken into account in governance policies
    • Examples of this development are the demand for and the necessity of Green IT, the need for massive computational power (exascale computing), the increasing amount of data, the seamless access to services for users, the internationalisation of scientific research and the involvement of the user communities in governance of e-Infrastructures.
    • The shift from mere resource provisioning to a system of infrastructure services will have a considerable impact on how such infrastructures are funded and financed.
    • Different financing schemes may be appropriate to allow for the specific situation of given e-Infrastructures.
    • Different modes are currently in use. An accepted approach in international e-Infrastructure cost-sharing between countries is based on GDP5 (such as in GÉANT and EGI).
    • In many countries, High-Performance Computing is funded via the resource owners, but in some cases partly via the user budgets.
    • In some projects central funding is used to compensate for users in disadvantaged regions.
    • If e-infrastructures are to be operated in the medium to longterm in a sustainable manner, then users will need to be given a choice for the best available services regardless of national boundaries (ERA).
    • To guarantee that timely innovation takes place in e-Infrastructures for science and higher education so as to fulfil the legitimate demands for advanced services ahead of what commercial market offerings can provide, the active participation of users with leading edge service requirements in strategic governance decisions concerning e-infrastructures is essential.
    • In EGI, European organisations such as CERN and EMBL are members of the EGI Council and user representation is also provided in other policy groups within EGI.
    • In some NRENs universities and scientific institutes are directly in charge of the multiyear strategic plans, whereas in others, user involvement is organised through a council of users.
    • Some NRENs are associations of user organisations and several are fully relying on user funding for both innovation and the provision of services.
    • At any given moment in time these two groups may have different requirements.
    • As many cases in the past have shown, however, the requirements of today’s leading edge user communities will be the needs of all users tomorrow.
    • Any model should therefore be able to keep timely innovation alive.
  • 1.3 Proposed Approach
    • The essence of the proposed approach is that e-Infrastructure governance will have to show a shift towards a user-driven approach.
    • There will be no one size fits all solution.
    • Different technical, political and commercial developments, such as the virtualisation of services, the emergence of cloud computing, the ambition of establishing an ERA and the ever increasing need of leading edge user communities for services far beyond what the commercial market can offer, will drive this process.
    • Such changes should include provisions that allow users to switch to services with different funding and charging mechanisms, e.g. moving from a centrally funded service to one that is partially funded from the user budgets.
    • A User-centric Approach:... (ECB Report - More aligning with users)
      • Users and user communities, in particular the leading edge users, will acquire a key role in setting the strategy because the quality and continuity of their research depend on the long-term availability of the right e-Infrastructure ecosystem.
      • Timely e-Infrastructure innovation to serve user communities ahead of what the commercial markets can provide will remain a public responsibility at both the national and European levels. It will continue to require a concerted effort and large-scale funding on all levels. For national e-Infrastructures with international significance, national financing could be matched with appropriate EU funding.
      • The funding of the use of e-Infrastructures services should increasingly be paid out of the budgets of users and user projects.
    • Strategic and financial concerns:...
      • The governance system should be supported by an elaborate system of metrics to establish the value and costs of the services and the service delivery systems.
      • Long-term financial guarantees will be necessary, coupled with continuing benchmarking mechanisms to keep the players in the system eager and keen.
      • The different territorial and functional roles of the various e-Infrastructure resources, like networks, computing resources and data storage facilities should be carefully considered. This should also allow for management of the different life cycles of the various components of e-Infrastructures and for the investment of the underlying IT resources.
  • 1.4 Recommendations
    • Establish a user-community-centric approach in strategic e-Infrastructure governance. This should include the appropriate funding mechanisms that make a clear distinction between the funding of service provision and the funding of innovation activities.
    • Define the long-term financial strategy for e-Infrastructures aimed at a sustainable operation of services in a flexible and open environment that includes offers from commercial service providers.
    • Encourage important players in the use of e-Infrastructures, like ESFRI, Virtual Research Communities, … to investigate the impact of strategic changes in e-Infrastructure governance and financing on the operation of and access to international Research Infrastructures.

• 2 Future of research networking
  • 2.2 Context: Current practices - achievements and limitations
    • Moreover, the data deluge will change the relations between e-Infrastructures and their end users, demanding further internationalisation and coordination of the research area (e.g. ESFRI, EIT) and increasing cooperation between public and private research, i.e. public-private partnerships.
  • 2.3 Proposed approach
    • However, funding instruments should be used correctly: innovation funding should be used for innovation while structural funds could be used to address the digital divide.

NA3.3 Standardization and Integration

• 1 e-Infrastructure Governance
  • 1.2 Context: Current practices - achievements and limitations
    • Across Europe, seamlessly interworking and easily accessible e-infrastructure services will be the foundation for the ERA which is targeted to be completed by 2014.
  • 1.3 Proposed Approach
    • Openness, neutrality and diversity of resources and services:
      • Activities for the development of open standards for using the e-Infrastructure services should be promoted and supported on all functional levels and in all application areas. IPR on ICT products and services should not pose a barrier to such openness. This would make e-Infrastructure a flexible and living system and would avoid the occurrence of legal and financial roadblocks for later developments into unforeseen directions. (ECB Report - Standards and End-users)
    • Legal structures supporting the strategy
      • Appropriate arrangements for IPR and copyright should be in place, enhancing the value of public investments for all European user and service providers and avoiding barriers to interoperability.
  • 1.4 Recommendations
    • Address the problems of barriers to cross-border service delivery and quickly remove as many of these as possible.