Some confusion exists about the terminology associated with linguistically annotated corpora. This is partly because the term tagset is used differently by two different communities. For the traditional corpus linguist, a tagset is the set of possible values used to explicitly annotate a text with a linguistic analysis; for example, the CLAWS tagset comprises a set of values such as NN1, VVD etc., each of which has a specific significance (singular common noun, verb past tense, etc.) For the mark-up specialist however, the term tagset refers to any kind of annotation, in particular the collection of SGML tags corresponding with the elements defined in a particular DTD: for example, the TEI defines a number of tagsets, each containing definitions for specific SGML elements and attributes.

Both usages reflect the fact that all markup introduced into a text is identical, at some level of analysis, in the sense that it serves to record or assert an association between stretches of text and values taken from some externally defined set of interpretations. However most people seem to categorize an analysis such as this is a paragraph differently from the formally equivalent judgement this is a noun. The former judgement is said to be structural and the latter interpretative. This kind of categorization also underlies the notion of level of annotation as exemplified by (inter alia) the Corpus Encoding Specification (Ide 1998), where the distinction is further justified by the observation that the addition of so-called structural markup is generally easier to automate than that of interpretive markup, since the latter (almost) invariably requires human judgement and knowledge, while the former rarely does. Particularly in the case of textual markup, interpretative judgements tend to be more controversial than structural ones, if only because the latter relate to aspects of a text which are accepted as intrinsic to its substance by the community of text readers. Structural interpretations form part of the contracts of literacy (Snow and Ninio, 1986) which form the precondition of a text's recognition as meaningful by the members of a particular community of readers.

For purposes of validation, however, the distinction seems unhelpful. All markup introduced into an corpus should be validated in the same way, and the validity of the corpus overall is equally affected by each type of markup used. Nevertheless, we have subdivided our discussion into two parts, reflecting the division currently made by most practitioners between structural and interpretative markup, and which are consequently reflected in actual practices. Structural markup is most generally to be validated with reference to an abstract model of textual components and features which is either entirely intuitive and common sense based, or defined in terms of some consensus-based model such as that of the TEI, restated as an SGML DTD. Interpretative markup may be similarly theory-free (see, for example, Leech 1993, but it is more customary to define it with reference to some explicitly stated analytic model, and hence to facilitate both automatic validation of the corpus itself (to check that it is valid in its own terms) and comparison of two corpora using different markup schemes derived from a common abstract model.

In section we discuss the process by which the structural markup defined for a given corpus may be validated. The formal mechanism used for this purpose is an SGML document type definition. In section we discuss in more detail one particular kind of interpretative markup: that which seeks to make explicit morpho-syntactic analysis of a text. We present here an SGML scheme for the formal expression of an abstract model that may be used to validate such analyses both internally and externally. Finally, in section we suggest some ways in which the result of either validation exercise may be formally documented. We begin, however, by describing the model of formal validation which underlies both descriptions. (For a more detailed discussion of the principles adumbrated here, see Sperberg-McQueen and Burnard 1995).

We begin by positing the existence of textual features or abstractions, instances of which are predicated at various positions within a document. The function of markup is to indicate unambiguously the presence of instances of such features. For example, a document may contain instances of the feature segment, whose presence might be signalled by such markup conventions as: the start of a new input record;the presence of some distinguishing code or sign such as a star, not otherwise present in the text; the presence of some predefined symbol such as the tag s

As noted above, the presence and scope of a feature such as singular noun may be predicated in exactly the same way.

We further assume that it is possible to define a grammar for such markup symbols: that is, a grammar which defines which combinations of such symbols in a document are to be regarded as legal. Such grammars generally have regard only to the markup language itself, rather than its extension to the underlying feature set. A markup grammar may simply enumerate all legal markup tokens, or simply specify an algorithm for the identification of markup tokens with no consideration of which markup tokens might be permitted. A more complex grammar (such as SGML) may also be used, enabling the formulation of contextual rules such as the tag X is only legal within the scope of the component identified by tag Y in addition to these kinds of rules. Note however that legality is still defined here in terms of syntax: only informal legislation can determine whether the content of an SGML element is correct with reference to some semantic model. Publications such as the typically extend the syntactic definitions embodied in their DTDs by more or less detailed discussion of the intended semantics of elements, but rarely provide a formally verifiable abstract model of such semantics, nor is it entirely clear what such a model might resemble. Nevertheless, throughout our discussion we will use the term feature (and derivatives) to refer to components of such a model, and the term tag (and derivatives) to refer to components of the markup system used to assert their existence.

This distinction seems to us crucial to the feasibility of validation: A corpus is a collection of utterances, and therefore a sample of actual linguistic behaviour. However, even if we do not believe that the distinction between competence and performance is valid, a corpus is not itself the behaviour, but a record of this behaviour (Stubbs, 1996). The function of the markup in the corpus is to make explicit, and hence accessible to comparative study, the recording process for both structural and interpretative encoding in a corpus text. Without this, neither comparative studies of different corpora, nor any assessment of the validity of the corpus record with respect to what it records will be possible.

We define the process of validation as follows: for each feature of interest, does the document contain any tagging?is the tagging of the document syntactically correct?is the tagging of the document consistently applied (i.e. is every occurrence of a given feature tagged in the same way)?is the tagging of a document correctly applied, with reference to some externally (or internally) defined abstract model?if correct, is the tagging of a document complete, with reference to some externally (or internally) defined list of mandatory features?

Taking these in reverse order, it is clear that, in the general case, the last two of these stages are automatable only to the extent that an abstract model can be formally specified for both the feature system itself and for the intended correspondence between that and the tagging employed. We present in section below one such abstract model, the EAGLES Guidelines for morpho-syntactic annotation (Leech and Wilson, 1994), re-expressed as a TEI-conformant feature system, against which any other set of morpho-syntactic annotations using the same representation may be validated, without necessarily having to conform to the EAGLES model. We also discuss the somewhat simpler abstract model proposed by EAGLES itself in section below.

Equally clearly, however, neither the third nor the first of the stages above can in principle be automated, since both depend on a human judgement to the effect that such and such a feature is in fact present, whether or not it is signalled by the tagging in a text. Such text-comprehension abilities still seem to be somewhat beyond the state of the art in NLP, despite some advances.

The second of the three stages above is however automatable, to the extent that the tagging syntax of the document is fully specified. In an SGML context, this implies the existence of a DTD against which candidate documents can be verified using an SGML parser. For other forms of markup, validation may involve other forms of verification, some of which may be intimately tied in to the behaviour of particular application software. For example, a document marked up in RTF or LaTeX may be considered valid so long as Microsoft Word or LaTeX does not reject it, irrespective of its output. Technical documentation will often specify what markup should be found in a document: where the markup syntax is arbitrary or application specific, clearly special purpose software must be developed to validate it.

Language corpora are made by combining together whole texts or extracts from pre-existing documents, usually according to some specific design criteria. The structure of the corpus itself may thus be described (and hence marked up) at two levels: internal, relating to the way the parts of the corpus fit together, and external, relating to compositional features of the originals. This distinction holds good whether the corpus under consideration is a fixed document or a dynamic or monitor corpus; in the latter case, as well as generally requiring dictate the use of whole texts rather than extracts, the internal design criteria may be further extended to include such topics as the rate at which new documents enter the corpus, the criteria for determining that they should be discarded from it, etc.

The internal structural features of a corpus are largely self-evident, and require little validation: common practice requires only the clear delimitation of individual text fragments, and to associate with each an appropriate level of description or metadata. In the TEI model, the former constitutes the text proper, and the latter its header. In older corpora, it was common practice to provide such metadata (if at all) as a separate documentary component, with only an informal association between the two, often depending on such artifices as file-naming conventions or sequencing to identify descriptive features of each component. The TEI model uses the power of SGML (in particular, its hierarchic structure and the consequent ability to specify property inheritance) to build more sophisticated structures. (For an account of some of these, see the discussion in e.g. Chapter 23 of the TEI Guidelines.)

The scope of the external features to be found marked up in language corpora varies greatly, depending both on the diverse nature of the materials they include and the diversity of applications envisaged for them. In large corpora, economic considerations alone preclude any attempt at modelling in the markup the full diversity of structures which a detailed textual feature analysis might indicate as possible: in the earliest corpora, for example the Lancaster/Oslo/Bergen corpus, even such basic organizational features as paragraphs or subheadings are rarely distinguished as such. Even today, the corpus designer is always forced to make pragmatic decisions about which structural features will have sufficient usefulness in the intended applications to warrant the expense of identifying them consistently and correctly. For many purposes, division into discrete segments, corresponding with identifiable locations in the original source, is adequate. For other purposes (for example, the study of discourse-related phenomena or text-grammar) a richer approach will be desirable.

Standards such as the CES provide a rich set of feature descriptions from which the corpus builder can select, together with specific tagging rules about how the presence of selected features can be made explicit. There is, however, considerable (and understandable) reluctance to make recommendations about which particular selections are appropriate or mandatory, since this will inevitably depend on the intended application for the corpus.

To validate such corpora therefore, a necessary first step is to identify the intentions of the designer. A corpus which does not mark up paragraph divisions is not necessarily less valid or useful than one which does; a corpus which claims to mark such divisions but which does so inconsistently or inaccurately is. Unfortunately, as WP2 demonstrates, it is often hard for corpus builders to specify their intentions in this respect, and harder for the validator to determine the extent to which these intentions have been carried out. Documentation and the provision of a DTD go some way to simplifying the task, as further discussed below.

As noted above, the extent to which the syntactic consistency of the structural markup in a corpus can be validated depends on the extent to which that markup uses a formally verifiable syntax. The great merit of SGML as a markup language is precisely that it makes this automatic verification simply a matter of defining an appropriate grammar (a document type definition) and checking the corpus against it. The most widely used software for this purpose is currently the freely available SGML parser SP, particularly its DOS incarnation NSGMLS [SP]. With the growing take up of SGML and of its simplified version XML, the number and sophistication of such systems is likely to increase greatly.

SP and similar programs typically perform a number of other functions on a document, but for validation purposes, the key functions may be summarized as follows: are the tags present in the corpus all defined in its DTD?are the tags in the corpus all present in syntactically correct contexts?do all attributes specified for the tags in the corpus conform to the value ranges specified for them in the DTD?are any cross references specified by the SGML markup satisfied?

The output from an SGML parser is thus typically either simply confirmation that the document does in fact conform to the DTD, or a list of instances where it does not conform. At the risk of stating the obvious, it should be emphasized that a corpus which does not conform to its DTD, or which lacks a DTD, cannot be validated, no matter how closely its markup appears to be modelled on that of the SGML standard. The notion SGML-like or unvalidated SGML is not a helpful one in this context.

For corpora which do not use SGML markup, validation will require the provision of some DTD-like set of formal rules, and the production of some parser-like software to check them against the corpus itself. Such procedures are eminently feasible, and for simple markup schemes may be considered preferable to the expense of converting the markup to true SGML. For a variety of reasons not necessary to summarize, we do not recommend this approach: in the long run, the use of a widely accepted standardized markup language should always be less expensive than the maintenance of an idiosyncratic or application-limited scheme.

The list of questions to which an SGML parser will provide answers given in the previous section falls some way short of what we would like to know before deciding that a given corpus is suitable for our purposes in the general case. In particular, a parser cannot tell us whether every item tagged as an instance of some feature is in fact such an instancewhether every instance of some feature is in fact tagged as such

To a large extent, however, these are limitations inherent in the whole markup enterprise; they also touch on fundamental problems of naming and ontology which have exercised philosophers since the time of Aristotle, and for which it would be unreasonable to expect immediate answers. Nevertheless, it is possible to make some pragmatic observations, additional to those provided in section below concerning the semantic validation of analytic tagging.

Although not formally presented as such, pre-defined feature lists such as those provided by the TEI and CES may be regarded as constituting a kind of abstract model for the structural components of texts. They thus provide a useful reference point against which the validator may check both that the objects tagged as representing some feature appear to conform with the definitions supplied there, and conversely that no features conformant with those definitions are present but untagged or tagged inappropriately. This remains however an entirely manual process.

Few corpora are small enough to permit the luxury of a close reading, and so in the general vase this kind of manual validation can only be done by sampling. Typical procedures are thus to inspect some random sample of the corpus for the presence of specific tagged features, for example, the paragraph boundaries or headings. Provided that the location of these samples within the original documents is known, an attempt can then be made to assess the accuracy with which the tagging of structural features has been carried out across the corpus with respect to the original source. In the absence of an original source, such accuracy can be assessed only in statistical terms, for example by comparing the distribution of certain tagged features in the sample with their distribution across the whole, where a correct distribution can be hypothesized on the basis of a priori reasoning (e.g. the number of paragraphs per text of a given type should be reasonably stable) or by applying other statistically derived heuristics.

Whatever form of markup is employed, morphosyntactic tagging is usually supplied at the level of individual tokens in a text and is thus usually self-evident. In the absence of any documentation, it is likely to be a generally a simple matter to extract from a document all the unique tokens constituting the markup, and also to identify the lexemes to which they are attached, as was done, for example, by Garside and McEnery 1993. In this example, annotations were separated from words by underscore characters. Other schemes place the markup and lexeme in separate fields, or on alternate lines within the text proper. In SGML documents, annotations may be represented as attribute values, or as distinct elements, and the association between lexical item and annotation may be made by means of pointer or link.

It will be rather less easy (in the absence of documentation) to determine what feature or combination of features each markup token is intended to represent. The list of all markup tokens, together with an index of their occurrences, and the associated lexical item, might be collated with an annotated corpus in which the same lexical items are associated with annotations whose feature equivalences are known, thus providing a kind of latter-day Rosetta Stone for the purpose. Such a process is hardly likely to be easily automated. This is one good reason for insisting on the availability of such documentation, preferably in a form which can be readily mapped to agreed standards.

Such mapping requires the predefinition of an agreed set of morphosyntactic features, independent of markup. Such a set is provided in the context of several western European languages (such as Danish, English, French, German, Greek and Spanish) by the EAGLES morphosyntactic annotation guidelines (Leech and Wilson, 1994), which we have therefore adopted as a test case for our recommendations. The procedures described here and the conclusions we reach would be equally applicable to any other set of Guidelines. However, as the EAGLES guidelines have been published on the basis of a wide-ranging review of corpus builders, recommendations derived from it are likely both to reflect, and potentially have a wide impact on, current practice.

The EAGLES recommendations have a dual focus: as well as providing an abstract model of the feature sets against which any particular combination of the features tagged in some corpus may be validated, the Recommendations specify explicitly a subset of recommended features which it is assumed should always be marked. Validation at this level thus becomes a matter of simply checking that the recommended features are in fact present [mdash ] in the terms we introduced in section above, validation that the tagging is not only syntactically correct, but also complete.

The aim of this level of validation is to ensure that the form of tags is consistent. Specifically, it should check that: each appropriate lexical item receives an appropriate annotation;each appropriate lexical item receives a single annotation;each annotation used is documented and corresponds with a known feature, i.e. there are no typographic errors;the annotation is presented using a consistent and correct syntax.

We use the phrase lexical item above to indicate that the tokens to which annotation is attached need not correspond with orthographic words. Although many commonly used annotation schemes for English do in fact attempt to make this correspondence, it is unnecessary where a single formalism such as SGML or something of equivalent power is used to represent both structure and analysis.

Thus, the CLAWS scheme uses a special form of annotation known as ditto tags to indicate that the annotation for one token applies also to another. For example, the English conjunction so that should properly be regarded as a single conjunction, although it is orthographically represented as two tokens. Early versions of CLAWS tagged this phrase as so_CS21 that_CS22 or, using the equivalent SGML formalism, as <w CS21>so <w CS22>that. The actual annotation for conjunction is CS, the following digit 2 indicates the number of tokens to which it is to be attached, and the final 1 and 2 indicate the number of this token within the sequence. A more natural approach would be to revise the tokenization rules so that the token so that might be treated as a single unit, tagging it as <w CS2>so that.. Uncoupling the annotation structure from the orthographic structure also enables a consistent approach to be taken for the case where the morphosyntactic units to be tagged are smaller than orthographic words.

We recommend above that a single annotation be attached to each lexical token, recognizing that in production systems it may be necessary to retain deliberately ambiguous or polyvalent annotations to avoid incorrect deterministic disambiguation. Such exceptions to the one word, one tag rule, should be clearly documented to aid validation; ideally each possible combination of multiple annotations can be represented as a distinct choice within the feature set. The FSD notation recommended below supports this possibility.

The majority of these tasks can be achieved using a series of procedures aided by simple Unix tools such as awk and grep. Checking SGML requires an SGML parser, and a number of these are available. As part of this workpackage, we reviewed the SGML validation that had been undertaken on the corpora covered in the WP2 review. For most part, the results (summarized in section below) indicate that as yet only a few corpus builders are taking advantage of the availability of tools such as SGML parsers to validate formally-defined markup schemes.

This is unsurprising, given the fact that such schemes have only begun to gain wide acceptance in the last few years. However, it does seem strange that the topic of validation is rarely touched on in the extant literature concerning corpus design and construction; where it is, the topic appears to relate almost exclusively to the statistical validity of a given sample as representative of some aspect of language (see for example Clear 1992, Atkins et al 1990). Corpora such as the LOB and Brown have been so exhaustively studied and analysed that it would be surprising if such errors as they contain had not come to light; furthermore, where they have, however, corpus designers and builders seem to have been uninterested in their status or implications. A plausible reason for this is that it is only with the advent of really large corpora, often produced by automatic or semi-automatic methods of data capture such as optical character recognition or as a by-product of electronic typesetting, that questions of accuracy and authenticity have arisen.

In addition to the strictly morphosyntactic analysis discussed so far, the EAGLES Guidelines also envisage two generic forms of syntactic analysis: phrase structure and dependency. Phrase structure grammars require the ability to model well-balanced trees in a markup language, while structural dependency grammar requires the ability to describe directed acyclic graphs.

Both abilities are intrinsic to the SGML abstract model, and the tasks of first representing, and then validating the correctness of such structures, is thus comparatively trivial. Furthermore, it is clear that the fundamental problems of semantic validation are the same whether analyses are attached to high level structural units such as those identified by syntactic analysis or to lower level word-like tokens.

The generality of the SGML model leads to its being suitable for the tagging of a semantically highly diverse set of textual features. For example, the TEI recommendations propose that SGML tagging be applied to mark inter alia the following features: orthographic and presentational features of the transcriptionlinks to corresponding objects (for example digitized recordings of transcribed speech, digitised page images of transcribed writing etc.) explicit disambiguation of features such as proper nouns, dates, times, etc.part-of-speech and morphologysyntactic analysisdiscourse analysiscontextual, bibliographic, and topically related featureseditorial correction, normalization, commentary, or annotation

While there is no doubt that an SGML encoding can cope with all of these forms of analysis individually, the difficulty of distinguishing them in combination rapidly increases, particularly if they are all located in the same data stream. There is an increasing tendency therefore towards so-called out-of-line annotation, in which potentially many, possibly contradictory, annotations or analytic interpretations are stored independently of the text itself, but linked to it by means of hypertext pointers. Similar techniques are required for the alignment of the structural components of multilingual or multimedia corpora.

Such techniques have much to recommend them, but place additional constraints on the ease with which the semantic and syntactic correctness of any one analysis can be validated. As well as checking that the analysis is internally consistent, it must be possible to check that the targets of each link are correctly specified. This may be difficult, if a non-portable or non-robust method has been used to specify them, or impossible entirely if the corpus text has been changed. Reliable standards for the specification of robust and application-independent linking mechanisms (e.g. HyTime, XLL) have a degree of acceptance within the computing sector, but are not yet widely accepted or understood within the community of corpus creators. An obvious exception to this generalization is in the special case of multilingual or multimedia aligned corpora where such mechanisms are essential.

We have restricted ourselves primarily to morphosyntax and syntax, partly because these are the most widely encountered forms of annotation and are also the only ones for which, at present, EAGLES guidelines exist. Other forms of annotation are sparser and more diverse, with insufficient examples of each type to make generally acceptable recommendations, even where consensus exists as to the scope or application of such analyses. This situation is like to change over time and consideration should be given on a rolling basis to validation procedures as the application of annotation types and the development of standards proceeds.

With this said, it is likely that many of the issues for validation of, say, pragmatic annotation, will be similar to those for morphosyntax. While the precise details of the scope of annotations and the interpretative nature of the schemes may differ, basic issues such as idiosyncratic v. widely accepted annotation schemes and questions of rigid v. fluid analysis schemes will most likely remain the same. So future work on the validation of such further annotations will be able to refer to this document for guidance, if not a complete solution.

This is a complete FSD for the EAGLES Guidelines for morphosyntactic analysis, using the formalism defined in chapter 26 of the TEI Guidelines. It consists of a series of declarations for feature structures, each represented as a fsDecl element, and each corresponding with an EAGLES recommended feature. Each fsDecl contains a series of fDecl elements, each corresponding with a set of the feature-value pairs defined for that feature structure in the EAGLES scheme. The values (vRange) are specified as a set of alternate values using the vAlt element, indicating that EAGLES does not permit multi-valued features, but a system-dependent default value (dft) is permitted for use in cases where none of the specified values is applicable. <!DOCTYPE teiFsd2 system "teifsd2.dtd"> <TEIfsd2> <teiHeader> <fileDesc> <titleStmt> <title>Feature System Declaration for the EAGLES tagset</title> </titleStmt> <publicationstmt> <p>Prepared for ELRA WP3 </publicationstmt> <sourcedesc><p>No source: this is an original work</sourcedesc> </filedesc> <revisiondesc> <change><date>2 apr 1997</date> <respstmt><resp>ed</resp><name>LB</name></respstmt> <item>Minor changes for validation; added header</item> </change> <change> <date>31 mar 1997</date> <respstmt><resp></resp><name>APM</name></respstmt> <item>First complete draft</item> </change> </revisiondesc> </teiHeader> <!-- Feature system for Nouns --> <fsDecl type = Noun> <fDecl name = Type> <fDescr>Range types associated with a noun</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Common --> <sym value=2><!-- Proper --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Gender> <fDescr>Range genders associated with a noun</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Masculine --> <sym value=2><!-- Feminine --> <sym value=3><!-- Neuter --> <sym value=4><!-- Common FOR USE WITH DUTCH AND DANISH ONLY --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Number> <fDescr>Range number associated with a noun</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Singular --> <sym value=2><!-- Plural --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Case> <fDescr>Range case associated with a noun</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Nominative --> <sym value=2><!-- Genitive --> <sym value=3><!-- Dative --> <sym value=4><!-- Accusative --> <sym value=5><!-- Vocative --> <sym value=6><!-- Indeclinable VALUE FOR GREEK ONLY --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Countability> <fDescr>Optional attribute counatbility</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Count --> <sym value=2><!-- Mass --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Countability> <fDescr>Language Specific Attribute Definiteness for Danish</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Definite --> <sym value=2><!-- Indefinite --> <sym value=3><!-- Unmarked --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> </fsDecl> <!-- Feature system for Verbs --> <fsDecl type = Verb> <fDecl name = Person> <fDescr>Range person associated with a verb</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- First Person --> <sym value=2><!-- Second person --> <sym value=3><!-- Third person --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Gender> <fDescr>Range genders associated with a verb</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Masculine --> <sym value=2><!-- Feminine --> <sym value=3><!-- Neuter --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Number> <fDescr>Range number associated with a verb</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Singular --> <sym value=2><!-- Plural --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Finiteness> <fDescr>Range finiteness associated with a verb</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Finite --> <sym value=2><!-- Non Finite --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = FormOrMood> <fDescr>Range form/mood associated with a verb</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Indicative --> <sym value=2><!-- Subjunctive --> <sym value=3><!-- Imperative --> <sym value=4><!-- Conditional --> <sym value=5><!-- Infinitive --> <sym value=6><!-- Participle --> <sym value=7><!-- Gerund --> <sym value=8><!-- Supine --> <sym value=9><!-- Ing Form VALID FOR ENGLISH ONLY --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Tense> <fDescr>Range tense associated with a verb</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Present --> <sym value=2><!-- Imperfect --> <sym value=3><!-- Future --> <sym value=4><!-- Past --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Voice> <fDescr>Range voice associated with a verb</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Active --> <sym value=2><!-- Passive --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Status> <fDescr>Range status associated with a verb</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Main --> <sym value=2><!-- Auxiliary --> <sym value=3><!-- Optional Attribute Semi Auxiliary --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Aspect> <fDescr>Optional Aspect attribute</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Perfective --> <sym value=2><!-- Imperfective --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Separability> <fDescr>Optional Separability Attribute</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Non Separable --> <sym value=2><!-- Separable --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Reflexivity> <fDescr>Optional Reflexivity Attribute</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Reflexive --> <sym value=2><!-- Non reflexive --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Auxiliary> <fDescr>Optional Auxiliary Attribute</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Have --> <sym value=2><!-- Be --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = AuxiliaryFunction> <fDescr>Auxiliary Function Attribute Applicable ONLY TO ENGLISH</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Primary --> <sym value=2><!-- Modal --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> </fsDecl> <!-- Feature system for Adjectives --> <fsDecl type = Adjective> <fDecl name = Degree> <fDescr>Range degree associated with an adjective</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Positive --> <sym value=2><!-- Comparative --> <sym value=3><!-- Superlative --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Gender> <fDescr>Range genders associated with an adjective</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Masculine --> <sym value=2><!-- Feminine --> <sym value=3><!-- Neuter --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Number> <fDescr>Range number associated with an adjective</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Singular --> <sym value=2><!-- Plural --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Case> <fDescr>Range case associated with an adjective</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Nominative --> <sym value=2><!-- Genitive --> <sym value=3><!-- Dative --> <sym value=4><!-- Accusative --> <sym value=5><!-- Vocative GREEK ONLY--> <sym value=6><!-- Indeclinable GREEK ONLY --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = InflectionType> <fDescr>Optional Inflection Type Attribute</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Weak flection --> <sym value=2><!-- Strong flection --> <sym value=3><!-- Mixed --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Use> <fDescr>Optional Use Attribute</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Attributive--> <sym value=2><!-- Predicative --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = NPFunction> <fDescr>Optional NP Function Attribute</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Premodifying --> <sym value=2><!-- Postmodifying --> <sym value=3><!-- Head function --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> </fsDecl> <!-- Feature system for Pronoun-Determiners --> <fsDecl type = PronounDeterminer> <fDecl name = Person> <fDescr>Range person associated with a pronoun/determiner</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- First Person --> <sym value=2><!-- Second person --> <sym value=3><!-- Third person --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Gender> <fDescr>Range genders associated with a pronoun/determiner</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Masculine --> <sym value=2><!-- Feminine --> <sym value=3><!-- Neuter --> <sym value=4><!-- Common DANISH ONLY --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Number> <fDescr>Range number associated with a pronoun/determiner</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Singular --> <sym value=2><!-- Plural --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Case> <fDescr>Range case associated with a pronoun/determiner</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Nominative --> <sym value=2><!-- Genitive --> <sym value=3><!-- Dative --> <sym value=4><!-- Accusative --> <sym value=5><!-- Non Genitive --> <sym value=6><!-- Oblique --> <sym value=7><!-- Prepositional case SPANISH ONLY --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Category> <fDescr>Range category associated with a pronoun/determiner</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Pronoun --> <sym value=2><!-- Determiner --> <sym value=3><!-- Both Pronoun and Determiner --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = PronounType> <fDescr>Range pronoun type associated with a pronoun/determiner</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Demonstrative --> <sym value=2><!-- Indefinite --> <sym value=3><!-- Possessive --> <sym value=4><!-- Int/Rel --> <sym value=5><!-- Personal/Reflexive --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = DeterminerType> <fDescr>Range determiner type associated with a pronoun/determiner</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Demonstrative --> <sym value=2><!-- Indefinite --> <sym value=3><!-- Possessive --> <sym value=4><!-- Int/Rel --> <sym value=5><!-- Partitive --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Strength> <fDescr>Range strength associated with a pronoun/determiner in FRENCH DUTCH AND GREEK ONLY</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Weak --> <sym value=2><!-- Strong --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = SpecialPronounType> <fDescr>Optional Special Pronoun Type Attribute</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Personal --> <sym value=2><!-- Reflexive --> <sym value=3><!-- Reciprocal --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = WHType> <fDescr>Optional WH Type Attribute</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Interogative --> <sym value=2><!-- Relative --> <sym value=3><!-- Exclamatory --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Politeness> <fDescr>Optional Politeness Attribute</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Polite --> <sym value=2><!-- Familiar --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> </fsDecl> <!-- Feature system for Articles --> <fsDecl type = Articles> <fDecl name = ArticleType> <fDescr>Range types associated with an article</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Definite --> <sym value=2><!-- Indefinite --> <sym value=3><!-- Partitive FRENCH ONLY --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Gender> <fDescr>Range genders associated with an article</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Masculine --> <sym value=2><!-- Feminine --> <sym value=3><!-- Neuter --> <sym value=4><!-- Common DANISH ONLY --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Number> <fDescr>Range number associated with an article</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Singular --> <sym value=2><!-- Plural --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Case> <fDescr>Range case associated with an article</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Nominative --> <sym value=2><!-- Genitive --> <sym value=3><!-- Dative --> <sym value=4><!-- Accusative --> <sym value=5><!-- Vocative GREEK ONLY --> <sym value=6><!-- Indeclinable GREEK ONLY --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> </fsDecl> <!-- Feature system for Adverbs --> <fsDecl type = Adverbs> <fDecl name = Degree> <fDescr>Range degree associated with an adverb</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Positive --> <sym value=2><!-- Comparative --> <sym value=3><!-- Superlative --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = AdverbType> <fDescr>Optional Adverb Type Attribute</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- General --> <sym value=2><!-- Degree --> <sym value=3><!-- Particle ENGLISH GERMAN DUTCH ONLY --> <sym value=4><!-- Pronominal ENGLISH GERMAN DUTCH ONLY --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Polarity> <fDescr>Optional Polarity Attribute</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- WH Type --> <sym value=2><!-- Non Wh Type --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = WHType> <fDescr>Range degree associated with an adverb</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Interogative --> <sym value=2><!-- Relative --> <sym value=3><!-- Exclamatory --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> </fsDecl> <!-- Feature system for Adpositions --> <fsDecl type = Adposition> <fDecl name = Type> <fDescr>Range types associated with an adposition</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Preposition --> <sym value=2><!-- Optional Fused Prepositional Article Value --> <sym value=3><!-- Postposition ENGLISH GERMAN ONLY --> <sym value=4><!-- Circumposition ENGLISH GERMAN ONLY --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> </fsDecl> <!-- Feature system for Conjunctions --> <fsDecl type = Conjunction> <fDecl name = Type> <fDescr>Range types associated with a conjunction</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Coordinating --> <sym value=2><!-- Subordinating --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = CoordType> <fDescr>Optional Coordination Type Attribute</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Simple --> <sym value=2><!-- Correlative--> <sym value=3><!-- Inital --> <sym value=4><!-- Non Initial--> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = SubordType> <fDescr>Subordination Type for GERMAN ONLY</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- With finite --> <sym value=2><!-- With infinite--> <sym value=3><!-- Comparative--> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> </fsDecl> <!-- Feature system for Numerals --> <fsDecl type = Numerals> <fDecl name = Type> <fDescr>Range types associated with a numeral</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Cardinal --> <sym value=2><!-- Ordinal --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Gender> <fDescr>Range genders associated with a numeral</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Masculine --> <sym value=2><!-- Feminine --> <sym value=3><!-- Neuter --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Number> <fDescr>Range number associated with a numeral</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Singular --> <sym value=2><!-- Plural --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Case> <fDescr>Range case associated with a numeral</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Nominative --> <sym value=2><!-- Genitive --> <sym value=3><!-- Dative --> <sym value=4><!-- Accusative --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Function> <fDescr>Range function associated with a numeral</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Pronoun --> <sym value=2><!-- Determiner --> <sym value=3><!-- Adjective --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> </fsDecl> <!-- Feature system for Unique tags --> <fsDecl type = unique> <fdecl name="interjection"> <fDescr>Range of types associated with interjections</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Interjection --> </vAlt></vRange> <vDefault><dft></vDefault> </fdecl></fsDecl> <fsDecl type = Unique> <fDecl name = InfinitiveMarker> <fDescr>Range types associated with an infinitive marker</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- German marker zu GERMAN ONLY --> <sym value=2><!-- Danish marker at DANISH ONLY --> <sym value=3><!-- Dutch marker DUTCH ONLY --> <sym value=4><!-- English marker ENGLISH ONLY --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = NegativeParticle> <fDescr>Negative particles ENGLISH ONLY</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- full form not --> <sym value=2><!-- contracted form of not --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = ExistentialMarker> <fDescr>Existential Markers</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- English existential marker ENGLISH ONLY --> <sym value=2><!-- Danish existential marker DANISH ONLY --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = SecondNegativeParticle> <fDescr>Second negative particles FRENCH ONLY</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- French pas --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Anticipatory> <fDescr>Anticipatory Marker er DUTCH only</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- er --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Mediopassive> <fDescr>Mediopassive PORTUGESE ONLY</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Mediopassive marker se --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = PreverbalParticle> <fDescr>Preverbal Particle GREEK ONLY</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Preverbal particle --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> </fsDecl> <!-- Feature system for Residuals --> <fsDecl type = Residual> <fDecl name = Type> <fDescr>Range types associated with a residual</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Foreign Word --> <sym value=2><!-- Formula --> <sym value=3><!-- Symbol --> <sym value=4><!-- Acronym --> <sym value=5><!-- Abbreviation --> <sym value=6><!-- Unclassified --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Number> <fDescr>Range number associated with a residual</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Singular --> <sym value=2><!-- Plural --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Gender> <fDescr>Range genders associated with a residual</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Masculine --> <sym value=2><!-- Feminine --> <sym value=3><!-- Neuter --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> </fsDecl> <!-- Feature system for Punctuation --> <fsDecl type = Punctuation> <fDecl name = Period> <fDescr>Range types associated with a fullstop</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Period --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Comma> <fDescr>Range types associated with a comma</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Comma --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> <fDecl name = Question> <fDescr>Range types associated with a question mark</fDescr> <vRange><vAlt> <sym value=0><!-- Value not relevant for a language --> <sym value=1><!-- Question mark --> </vAlt></vRange> <vDefault><dft></vDefault> </fDecl> </fsDecl> </TEIfsd2>

The following tables illustrate how a particular set of analytic tags, in this case the CLAWS7 tagset, can be re-expressed in terms of the EAGLES intermediate representation. In cases where the CLAWS7 tag underspecies, each possible EAGLES value is given as an alternation.

The tables are organized as follows. Each table relates to an EAGLES obligatory feature, within which appear entries for all of the CLAWS tags categorised as being grouped with that feature. These tags are then further analysed in terms of their recommended features. Mapping list for NounsND1N1010NNN1000NN1N1010NN2N1020NNAN1000NNBN1000NNJN1000NNJ2N1020NNL1N1010NNL2N1020NN0N1000NN02N1020NNT1N1010NNT2N1020NNUN1000NNU1N1010NNU2N1020NPN2000NP1N2010NP2N2020NPD1N2010NPD2N2020NPM1N2010NPM2N2020 Mapping list for VerbsVB0V00012|31|000VBDRV2|0|001|2|011|1|2400VBDZV-20111400VBGV00029000VBIV00025000VBMV10111100VBNV00026400VBRV2|001|211100VBZV30111100VD0V-3|0|0|001|2|0|011|1|2|31|1|1|000VDDV00011400VDGV00029001VDIV00025001VDNV00026401VDZV30111100VH0V-3|0|0|001|2|0|011|1|2|31|1|1|000VHDV00011400VHNV00026401VHGV00029000VHIV00025000VHZV30111100VMV00011002VMKV00011003VV0V-3|0|0|0|001|2|0|0|01|1|1|1|01|1|2|3|01|1|1|0|101VVDV00011401VVGV00029001VVGKV00029001VVIV00025001VVNV00026401VVNKV00026403VVZV30111101 Mapping list for Pronoun-DeterminersAPPGEPD1|201|2|00|1|20203DAPD0|301|200302DA1PD0|30100302DA2PD0|30200302DARPD0|30000304DATPD0|30000304DBPD0|30000304DB2PD0|30200301DDPD0|30000302DD1PD0|30100301DD2PD0|30200301DDQPD0|30000304DDQGEPD0|3001|20303DDQVPD0|30000304PNPD0|30000120PN1PD0|30100120PNQ0PD0|30006140PNQSPD0|30001140PNQVPD0|30100140PNX1PD30100150PPGEPD1|2|31|2|301|2|00130PPH1PD33101|6150PPH01PD31|2106150PPH02PD30206150PPHS1PD31|2101150PPHS2PD30201150PPI01PD10106150PPI02PD10206150PPIS1PD10101150PPIS2PD10201150PPX1PD1|2|31|2|3100150PPX2PD1|2|31|2|3200150PPYPD20001|6150 Mapping list for AdjectivesJJAJ1000JJRAJ2000JJTAJ3000JKAJ1000 Mapping list for AdverbsRAAV121REXAV121RGAV122RGQAV112RGQVAV112RGRAV222RGTAV322RLAV121RPAV123RPKAV123RRAV121RRQAV111RRQVAV111RRRAV221RRTAV321RTAV121 Mapping list for ArticlesATAT1000AT1AT2010 Mapping list for Adposition tagsIIAP1IOAP1IWAP1GEAP3 Mapping list for ConjunctionsBCLC120CCC11|40CCBC110CSC201CSAC203CSNC203CSTC201CSWC201|2 Mapping list for NumeralsMCNU10000MC1NU10100MC2NU10200MCGENU10000MCMCNU10000MDNU20000 Mapping list for ResidualsFOR200FUR600FWR100ZZ1R310ZZ2R320MFR300 Mapping list for Unique tagsUHI InterjectionEXUEExistential thereTOUTInfinitive markerXXUXNegative particlePUQRPunctuation mark (quotation)PUNRPunctuation mark (non-quotation)

In the following list, we summarize claims made by the builders of several of the corpora analysed in Work Package 2 regarding how the encoding of their corpus was validated. The information here is only partial, and has not been reviewed by our informants. BNCSGML parser used to validate all markup against the CDIF (Corpus Document Interchange Format) dtd; all tagging errors reported are then hand-corrected. Some semantic validation (on a portion of each text) also performed for errors such as incorrect or missing headings, with limited manual correction. All addition of analytic tagging was automatic. but its syntactic validity was checked again, using an SGML parser. As a separate exercise, a 2 percent sample of the corpus was hand-checked for accuracy of analytic tagging, and the results used to improve the original part-of-speech tagging. (Results of this are not yet publicly available, but are due in 1998).LOB and Brown No SGML mark-up used, but structure indicated by means of a simple and automatically verifiable coding. Typographic errors are retained unchanged. Analytic coding performed using similar techniques to those of the BNC.London Lund Corpus No SGML mark-up used, but detailed indication of prosodic features using idiosyncratic markup scheme; no information available as to how this was verified. Penn Treebank No SGML mark-up used, but detailed indication of syntactic features using idiosyncratic markup scheme;validated by own analytic tools. ICE Originally used own SGML-like markup scheme, validated by suite of WordPerfect macros which inserted text unit markup after full stops etc. This system generally ensures that markup symbols are closed, and reminds users to do so should they try opening the same symbol again before closing it. Nelson 1996, p 65-66. After developing further software tools to check validity, the project has reportedly converted to an SGML system, but we have been unable to obtain further details of this.Multext and CRATER Where applicable, automatic conversion of preexisting header data was carried out. As for primary data in most cases division and/or paragraph level markup of some kind already existed in the texts we received, so getting P and DIV was a matter of conversion or automatic insertion. However, corrections were made by hand to P level markup. Since they were dealing with issues of alignment the accuracy of sentence level (and above) tags was crucial, so, while automatic means where used for as many of the steps as practical, hand-checking was also performed on sentence and above (p, quote, div etc) markup. All texts were parsed against their respective DTDs. PlatoAccording to our informant, The corpora were produced all over Europe in various formats and by people with varying amounts of experience and expertise in such work. Many started with a paper text, which was then scanned or even keyboarded. So this was clearly an issue to be tackled, especially since we wanted to align the texts and needed the markup to be not just accurate and SGML-wise correct, but also similar enough to assist the aligner. Parsers (nsgmls/xemacs) were used to check and correct the SGML, and most of the hands-on dirty work was done recently at the workshop in Nancy with Laurent Romary and his team. Most of the TELRI-ers who had prepared texts came along and we had the chance to really check and compare the texts. Some of the texts very initially sliced into sentences using tools that has been developped at our sites and which, being SGML aware can base their work upon an existing [lt ]p[gt ] structure. The Lampeter Corpus Originally prepared using word processor macros to insert minimal tagging for font changes and some structural features, use of different languages etc. The texts were then converted to true SGML by a combination of automatic and manual means, and have been proof read several times. Correction and validation carried out using emacs, PSGML, SP, and Author/Editor. ENPCValidated against the TEI P3 DTD twice, once after proofreading, and then again after alignment to check that the values of the id and corresp attributes are unique and that the value of the corresp attribute points to an existing id in the parallel text. All validation performed by SP; project has developed its own SGML-aware software for further analysis.UAMSCUses SGML-like coding for speaker identification and vocalic effects but not validated during data capture; some subsequent SGML-based analysis and validation.HelsinkiUses simply OCP-style markup only; validated only by analytic tools.MUCSome use of SGML-style tagging, e.g. for anaphor markup. No formal validation, other than by analytic tools.Speech Thought and Writing Presentation Corpus Some use of SGML-style tagging but no formal validation, other than by analytic tools. Tagging all manually added.PAROLEMinimal TEI-conformant dtd defined at start of project against which all corpora are eventually to be validated. Considerable variation in encoding practices reported amongst partners, no detailed information currently available.