CERN Accelerating science

Migration of VM User Mini-Disks and VMARCHIVE Files

  Harry Renshall CN/PDP

There are several different types of disk files on CERNVM from the perspective of backup and archiving. Users personal VM mini-disks, the 191 or A-disk and often a secondary 192 or D-disk and also the group 197 or G-disk are all backed up by the VMBACKUP product on a daily basis, and versions can be recovered from up to a year ago by the VMBACKUP RESTORE command. These files are on disks labelled VMUK or VMUW in the VOLSER column shown by the command DISKLIST loginid. A second type of VM disk are maxi-disks, usually allocated to an experiment for disk copies of frequently accessed tape data but sometimes given to individual users. These were always given with addresses of 400 and above and are on volumes called VMUX, VMIX or VMUT. These disks are not backed up, this being the users' responsibility. Files from either of these types of disk could, however, be put into a long-term archive store, which allowed multiple copies of the same file name, by explicit use of the VMARCH ARCHIVE

command. From early 1996 the ARCHIVE function will be disabled on CERNVM. Users will be able to QUERY, RECALL and PURGE their archived files but not to archive more files. This function will instead be available for AFS files from any of the public or private Unix Work Group Servers or other AFS client machine and more information on this will be given in the next CNL. There is no archive function within the NICE PC environment.

In the CERN AFS file system, all user home directories are also backed up. Each night a complete copy (technically, an afs clone) of a home directory is made from the file system /afs/cern.ch/user/first-letter/loginid

(first-letter is the first letter of the loginid) into /afs/cern.ch/ubackup/first-letter/loginid. Thus users accidentally losing a file can always get a copy from the previous day. Files are also backed-up to tape but recall from tape is currently only by request to the User Consultancy Office. In addition to AFS backup, a product called ADSM is available to users with personal workstations with local disk files. This product does a nightly incremental backup of such local disks and also includes a user-driven archive function.

CERNVM users should use the tools that will be provided (see article ) to migrate the files that they need from CERNVM to AFS. Note that executable files (MODULE and TXTLIB) will not be usable and binary data in IBM format will have to be converted. These should be rewritten on CERNVM in a portable format such as ZEBRA or EPIO, or simply rewritten as formatted data (EBCDIC) which can be automatically converted to the Unix ASCII standard. However, we will provide a safety net for users who fail to move all their files, perhaps because of a long absence from CERN or by discovering later that a file was really needed.

When the CERNVM service stops at the end of June 1996 the VM disks will be switched off permanently. The current copies of the backed up mini-disks will first be migrated into the Unix ADSM product with a file name including the VM loginid, the disk address and the full file name. An interface into ADSM will be made available on the central unix services so that users can inspect and recall such files. PC users will have to request the User Consultancy Office for help. We will migrate all files even those which cannot be directly used, such as modules, to allow for the possibility that a user may transfer them into another VM system. They will be stored in an IBM format called 'terse' which will both compress the files and allow users to translate them from EBCDIC to ASCII or not on recalling them (we do not know which VM files are already ASCII so cannot do this conversion automatically). We will NOT save copies of VM files stored into VMBACKUP , we will only keep the last, current copy of a file. Files belonging to deleted CERNVM accounts will not be preserved and users who have deleted files that are still stored in VMBACKUP and that they wish to preserve must do so themselves by first recalling the file into CERNVM.

CERNVM maxi-disk files have never been backed up and they will not be preserved. It is the users' responsibility to migrate any such files.

The CERNVM VMARCHIVE files will be treated like user mini-disk files in that we will also migrate them into a Unix ADSM file base with a naming structure that makes their origin clear. It is hoped to switch on the existing ADSM user archive function in the central Unix services in January, initially with a command line interface, so users can archive AFS files as they now do CERNVM ones. We will have to make an ownership mapping between old VMARCHIVE loginids and new AFS loginids and sometimes this will need users to make an initial `registration' contact with us. It would, of course, reduce our workload if VMARCHIVE users could systematically delete files they no longer want in the archive. You can see your archived files with the command

VMARCH Q FILES * * (FOR loginid address

(the default is your loginid and address 191) and purge them with VMARCH PURGE .

Please contact me if you have particular questions concerning your CERNVM disk files.