Task packages for Enrichment Scripts

About

A place for discussion about the Task Packages that make up the work described in SystemDesign.

When finalized, could be put to Savannah.

Original musings (30 Jun 2008)

Possible modules allocation overview...should be replicated(moved) to SystemDesign:

• SystemDesignBibHarvest - Piotr Praczyk - started -> 31/08/2008
• SystemDesignBibUpload
• SystemDesignBibCheck
• SystemDesignBibEdit - Lars Christian Raae - now -> 12/10/2008 (started)
• SystemDesignDeDup - Lars Christian Raae - -> 01/02/2009
• SystemDesignWebCategorize
• SystemDesignBibExport - Radoslav Ivanov - now -> 31/08/2008
• SystemDesignWebCatalog - Radoslav Ivanov - 09/2008 -> 15/12/2008
• SystemDesignExtractions - SLAC? -1/08/08????
• SystemDesignBibAuthority - Kyriakos ? - 01/09/2008 ->
• SystemDesignBibSched - Samuele Kaplun - 01/09/2008 -

Tibor's comments:

• Some of the above are in fact "tasks" that may live inside existing modules, or that may touch relationship between more than one module. It may be better to think of "task packages" here.
• We may have more task packages, e.g. the creation of the holding pen database with "approval" etc tools, which we may want to exist independently on its own, though modules like BibEdit, BibHarvest and WebCatalogue will make heavy use of it. *We can make this modules vs "task packages" relationship clearer on the SystemDesign (Something we should do rather sooner than later now that we are starting with the enrichment scripting coding real soon now.)
• The people allocation may not be necessarily done on a per-module basis. With many programmers around, it may be more advantageous to specialize. E.g. Kyriakos has an interest in UI so he may play a role of an UI expert, working on the web interface layers of many modules/tasks, while Lars/Rado will be working on the backend business logic which is something they like more. Some other person can play the Common Lisp expert, so tackling various CLI tools we may keep in Common Lisp (see also the CernBibCheck wiki page musings). In other words, if we design nice APIs between our layers, then we can allocate people not only vertically (by modules) but also horizontally (by layers) which could lead to faster development e.g. when people do not have to think about making their own web interfaces right from scratch. Just an example.

Update (11 Dec 2008)

Now that Rado, Marko and Benoit are finishing with their bigger projects, here are some more free task possibilies in the enrichment department to muse about:

Larger tasks, ~month(s):

• multi-edit -> Rado
• bibcheck record similarity plugin (CL)
• bibcheck conf file management (web-based) -> Marko
• bibcatalogue and RT -> Travis
• larger special editing modes; see also Lars's list
• turning OAI holding pen into a general holding pen; non-OAI sources

Smaller tasks, ~week(s):

• bibcheck backend messages in XML (CL) -> Tibor
• bibcheck enriched by bad author name detection procedure (CL) -> Tibor
• bibcheck web api, e.g. /record/123/check?config=foo.cfg
• authority daemon based on bibknowledge and bibcheck (aka bibcheck CLI)
• search engine reading suggestions from authority files -> Marko
• smaller special editing modes; see also Lars's list