Quick Links
last EMI-0 build results
EMI-0 repository

EMI-0 (Zugspitze) - Activity

EMI-0 - An ‘exercise’ release designed to understand how to apply the agreed procedures, find any problem about tools and processes and in general fine tune the EMI software engineering process before the EMI 1 release. The outcome of EMI 0 is not expected to be released to external users. The goal of EMI 0 is to prepare a consistent, coherent repository of non-conflicting packages by the end of October 2010 without any specific commitment on functionality.

Mandatory Platforms

• Platforms/architectures:
  • SL5 on 32 and 64bit architectures
  • Debian 5 on 32 and 64bit architectures

Milestones:

End of Sept, 2010 - configuration ready in ETICS:

General Guidelines - to be applied to all EMI Releases

• A single tool will be used to perform integration builds from source (nightly builds). All build tools will be supported and integrated via this unified integration system.
• Even though ETICS cannot be considered the optimal tool for each environment, it is the only one able to handle an integration build from source of all the four middleware distributions. Therefore it will be used to produce a first unified integration system.
• Globus packages for EMI shall be taken from existing standard repository like EPEL whenever available. All components in EMI shall adopt this strategy
• EMI is to use a common build, test and QA system as much as practically possible. Currently the system of choice is ETICS. All components to be released as part of EMI must have configurations in ETICS
• EMI is released on two sets of platforms, mandatory and optional. Mandatory platforms must be supported by all components in EMI, optional platforms can be supported by individual PTs depending on needs of technical capabilities. Currently EMI 0 and 1 are going to be supported on RHEL5 and compatible platforms and Debian 5 and compatible platforms on 32 and 64bit platform
• All components in EMI must be provided in the standard packaging formats typical of the supported mandatory platforms. Currently this means rpms for RHEL5 and compatible platforms and debs for Debian 5 and compatible platforms
• The EMI Repository must contain always non conflicting packages and only packages that are not commonly available on standard OS repositories or other mainstream repositories such as Fedora or EPEL

EMI-0 Specific Guidelines

• The starting platform will be Scientific Linux 5 x86_64 with the following enabled YUM repositories: SL5 X86_64, EPEL 5 x86_64, glite 3.2 X86_64
• A new ETICS platform and Worker Node is available: "sl5_x86_64_gcc412EPEL" which will have all the RPMs required to build the whole EMI distribution already installed in the VM image. Full RPM list: EMI_SL5_x86_64_EPEL_rpmlist.txt
• A new EMI project is available where all the EMI software will be built. All the EMI configurations required to build will be part of this project.
• Configurations already existing in other parts of the system will be cloned and moved in the EMI project if needed
• Product Teams which are interested in re-organizing subsystems or components are given the opportunity to change their software structure in ETICS. Please contact the ETICS Support for this.
• ETICS dependencies are currently not explicitly required for EMI-0 because all the dependencies are already installed in the VM. They will be needed for EMI-1. If PTs want to add them also in EMI-0, then it will be one thing less to do for EMI-1.
• It is not necessary to remove dependencies if a configuration already has them since they will be soon required again.
• All the build configuration guidelines present in the "Configuration & Integration Guidelines" already apply for EMI-0.
• No .DEFAULT properties are allowed in component or subsystem configurations. .DEFAULT properties are only present in the project configuration.
• No STATIC dependencies allowed unless requested to the EMT upon emergency and approved.
• Until a new version of the ETICS client, currently available version = 1.4.14-1, is released, EMI builds must be executed with this line in the "Append Requirements" field when submitting a build:

client_release=1.4.15-1_EMI_fixes@default

• ETICS project-configuration to be used for EMI-0 is "emi_R_0"

Hints on using globus default variables & pkg-config (EMT phone-conf 15.11.2010)

Details Hide Details

dcapBuildPackage_EPEL_i386_CO_5.4
./configure --with-globus-lib=/usr/lib/ --with-globus-include=/usr/lib/globus/include/ 
dcapBuildPackage_EPEL_x64_CO_5.4
./configure --with-globus-lib=/usr/lib64/ --with-globus-include=/usr/lib64/globus/include/ 
dcapBuildPackage_VDT_i386_SL_5.4
./configure --with-globus-include=/opt/globus/include/gcc32/ --with-globus-lib=/opt/globus/lib/ 
dcapBuildPackage_VDT_x64_SL_5.4
./configure --with-globus-include=/opt/globus/include/gcc64/ --with-globus-lib=/opt/globus/lib/

dcap EPEL specfile:
%configure \ --with-globus-include="%{_includedir}/globus -I%{_libdir}/globus/include" \ --with-globus-lib=/dummy 

Example for the WA use:

Details Hide Details

etics -checkout --ignorelocking --config "your_configuration" --project "emi" --project-config "emi_R_0" --runtimedeps --forcecheckout --verbose --continueonerror
etics -build  --config "emi-glite-apel_R_3_2_5_1" --verbose --target postpublish --continueonerror
platform:   sl5_x86_64_gcc412EPEL
client version:   1.4.15-1_EMI_fixes@default

Example using the etics-client:

Details Hide Details

    python etics-client-setup -r 1.4.15-1_EMI_fixes --volatile default

    etics-checkout --c <component_configuration_name> --project-config <project_configuration_name> <component_name>

    etics-build --platform sl5_x86_64_gcc412EPEL <component_name> 

ETICS-configuration schedule

• Sequence of ETICS_configurations steps:
  1. VOMS:
     • glite-security-voms-[api|api-c|api-cpp|api-clients]
     • "old" subsystem org.glite.voms
  2. CESNETSecurity:
     • gridsite-[shared|apache], glite-security-gsoap-plugin, glite-security-gss.
     • "old" subsystems: org.glite.security, org.gridsite
  3. gLiteSecurity:
     • glite-security-[lcas|lcmaps] glite-security-[trustmanager|util-java]
     • "old" subsystem org.glite.security
  4. gLiteInformationSystem:
     • bdii, glue-schema, glite-info-[templates,generic], glite-info-provider-[release|service|ldap]
     • "old" subsystems org.glite.bdii, org.glite.info
  5. DataManagement
     • "old" subsystem org.glite.lcg, org.glite.data, org.glite.lcgdm
  6. gLiteL&B, DGAS, APEL, ARGUS
  7. gLiteJobManagement (CREAM+WMS), StoRM

• Timelines, discussed on EMT-meeting - 27.09.2010:

EMI-0 Build status

Status Details Hide Status Details

PT	EMI-Subsystem	ETICS Components	Priority	Status	Worklog
VOMS	emi.voms	org.gltie.security.voms-api-cpp org.glite.security.voms-clients org.glite.security.voms-api org.glite.security.voms-server org.glite.security.voms-api-java EMI.voms.javadoc	Very High		see VOMS - changes for EMI - important for PTs dependng on voms see conf.nams <=> rpm in EPEL IMPORTANT - org.glite.security.voms-api-c was replaced/included in/by org.glite.security.voms-api-cpp, details on links above 15.11.2010 - ALL voms component build, except voms-mysql (to investigate) 22.11.2010 - ALL components build
CESNETSecurity	emi.gridsite emi.jobid emi.px	-	Very High		communicated the subsystem/components/configs to clone 22.10.2010 - configurations imported 10.11.2010 - have to solve globus dependencies 15.10.2010 - ALL gridsite builds, except gridsite-core (globus problems); ALL jobid components fail; ALL px components fail 30.11.2010 - emi.gridsite - OK, emi.jobid - OK, emi-px - requested myproxy in externals (!) 06.12.2010 - emi.px - In progress, will be done soon
gLiteSecurity	emi.delegation emi.hydra emi.java-security emi.sac	-	Very High		reorganization of the subsystem 22.10.2010 - configurations imported 08.10.2010 - correction required on voms-api-c dependency 15.11.2010 - builds only delegation-interface; hydra.ssss - errors on vdt.globus; emi.sac - 8 (of 30) build, the rest problems with globus & m4
gLiteInformationSystem	emi.bdii	-	Very High		communicated list of subsyst/compon/configs to clone , 22.10.2010 - configurations imported 10.11.2010 - CVS to SVN migration - for which "Failed" emi.bdii.glite-info-provider-service, emi.bdii.glite-info-provider-release, emi.bdii.yaim-bdii, emi.bdii.glite-info-plugin-fcr 14.11.2010 - ALL components in emi.bdii are building
UNICORE Target System Access	emi.unicore	unicore-unicorex	Normal		building
UNICORE Web Service Interfaces	emi.unicore	unicore-unicorex unicore-registry	Normal		building
UNICORE Client and APIs	emi.unicore	unicore-ucc unicore-hila-unicore6 unicore-hila-shell	Normal		building
UNICORE Service Hosting	emi.unicore	unicore-unicorex	Normal		building
UNICORE Security	emi.unicore	unicore-gateway unicore-xuudb	Normal		29.11.2010 - problems with gateway, caused by the specific WN were build was run
dCache	emi.dcache	-	Normal		communicated the subsystem/components/configs to clone 22.10.2010 - configurations imported 10.11.2010 corrected configuration-names, updated emi-dcache_B_HEAD, errors to correct for emi.dcache.srmclient 15.11.2010 - emi-dcache-srmclient_R_HEAD error on check-out command 30.11.2010 - dcache-server builds fail because of unsatisfied dependencies on SunJDK 09.12.2010 - ALL components build Action #18383
DataManagement	emi.lcgdm emi.lcgutil emi.fts	-	HIGH		communicated list of subsyst/compon/configs to clone 22.10.2010 - configurations imported 15.11.2010 - lfc-devel builds, but there are others failing (18 from 46 lcgdm)
ARC	emi.arc	arc0 arc1	Normal		test-builds with problem on dependencies on voms 10.11.2010 - problems with lfc, lfc-devel because of missing globus dependencies 15.11.2010 - arc0, arc1 build
gLite L&B	emi.lb emi.lbjp-common	-	Normal		communicated the subsystem/components/configs to clone 22.10.2010 - configurations imported 15.11.2010 - have to correct globus dependencies 30.11.2010 - emi.lbjp-common - waiting for postgresql-devel, emi.lb - in progress 06.12.2010 - emi.lbjp-common - OK; emi.lb - mostly OK, emi.lb.server failing on emi.sac.lcas
DGAS	emi.dgas	-	Normal		11.10.2010 - configurations present, have to update globus dependencies 30.11.2010 - ALL components build
APEL	emi.apel	-	Normal		communicated the subsystem/components/configs to clone 22.10.2010 - configurations imported 11.10.2010 - have to correct build commands, problems with java 08.12.2010 - All APEL modules built successfully in the NB Action #18382
ARGUS	emi.argus	-	Normal		configurations will be done by PT test-builds - problems with globus 11.10.2010 - no problems 14.11.2010 components build
gLite JobManag	emi.cream-ce emi.cream-ui emi.wms emi.wms-ui	-	Normal		Working ETICS Configurations errors on building components:
MPI	emi.mpi	-	Normal		configurations will be done by PT test-builds 15.11.2010 - ALL components build
StoRM	emi.storm	storm-checksum storm-common storm-.dynamic-info-provider glite-yaim-storm storm-backend storm-frontend storm-client storm-gridftp-dsi	Normal		15.11.2010 - some components don't have dependencies
Miscellaneous Components	emi.misc	glite-wms-utils-classad cleanup-grid-accounts edg-mkgridmap glite-info-generic glite-info-templates glite-version emi.glite.build-common-cpp emi-jobman-exception lcg-expiregridmapdir lcg-service-proxy	-		components that not have "other" etics-subsystem 15.11.2010 - ALL components build, except emi.misc.classad-utils

EMI-0 nightly builds

EMI-0 automatic builds

Problems encountered

1. change of VCS checkout command from:

checkout:   cvs -d ${vcsroot} co -r ${tag} ${moduleName}

checkout:   cvs -d ${vcsroot} co -r ${tag} -d ${moduleName} org.glite.apel.core

ETICS-responsible persons & external-dependencies lists

• ETICS-resposible persons

Details Hide Details

• List of dependencies that are not in the OS; different versions that the one provided by OS

Details Hide Details

• Common list of dependencies
  • list of external dependencies SL5/EPEL (x86_64)
    • attached list_externals_names_64_v7.txt
  • list of external dependencies Debian 5 - stable (Lenny), testing (Squeeze)
    • attached list_externals_names_debian_lenny.txt, list_externals_names_debian_squeeze_sid.txt

End of Dec, 2010 - packages ready in repository

• EMI Baseline

• List of components and their build status XLS, PDF