Resource Profiles

Stephen Downes

E-Learning Group
National Research Council
Moncton, NB
Canada
Stephen@Downes.ca

Abstract: The idea of a resource profile is that it is a multi-faceted, wide ranging description of a resource. A resource profile conforms to no particular XML schema, nor is it authored by any particular author. Additionally, unlike traditional resource descriptions, which are presumed to be instantiated as a single digital file and located in a particular place, a resource profile may be distributed, in pieces, across a large number of locations. And there is no single canonical or authoritative resource profile for a given resource. This paper describes the need for resource profiles, outlines their major conceptual properties, describes different types of constituent metadata, and examines the use of resource profiles in practice.

1. Introduction

1.1 What is a Resource?

Much of the thinking and design behind the concepts outlined in this paper are based on the idea of learning objects. This paper deliberately abstracts from the more usual mode of discourse, not in order to introduce unnecessary ambiguity, but to capture some of the ambiguity already inherent in the concept of the learning object and to place it in a light where it may be examined without a predefined conception.

The term 'learning objects' is based on the merger of two distinct concepts, neither of which are universally endorsed by practitioners in the field. The first term, 'learning', seems to imply that the item in question must have some pedagogical value. ( Magee and Friesen, 2001) But either a statement of this requires that a particular theoretical approach be presupposed, in which case proponents of different theories will not be in concord, or a tacit assumption of some common definition leaves the term so vague as to allow almost anything to qualify. The second term, 'object', presupposes a specific type of software entity, derived originally from the concept of object oriented programming, in which the resulting digital asset would support the concepts of inheritance, internal variables, and internal functions. (Downes, 2001) A great number of learning objects do not satisfy any of these criteria, and the original conception is now long lost in practice.

Instead, the approach taken in this paper will to discuss 'resources' generally, and it will be stipulated that a 'resource' may be anything that may be described in a 'resource profile'. This latter term is the subject of the paper as a whole, but in brief, what may be said of a resource profile is that it is an aggregate description of a resource. A 'resource', therefore, is anything that, for whatever reason, someone has found necessary or useful to describe, where the recommended structure for such descriptions is outlined in this paper.

The discussion and debate surrounding learning objects is but one instance of many attempts to identify what may be considered to be 'basic' or fundamental classes of resources. The term 'learning objects' presupposes, in other words, that resources may be divided into categories in two ways: 'learning' and 'non-learning'; 'objects' and 'non-objects'. A slight examination of the field suggests many more ways of classifying resources: 'digital' and 'non-digital' (IEEE, 2002), 'data' and 'metadata', 'text-based' and 'multimedia', and more. No doubt each of these distinctions will be useful within a given context. But it is by no means a straightforward matter to make such distinctions or to use them in a productive manner, much less obtain universal agreement that one, rather than the other, is a fundamental or essential categorization of resources.

For example, consider the distinction between 'data' and 'metadata', a commonly used and widely understood lexicon. How do we determine whether a given resource is a piece of data or a piece of metadata? It is said that metadata is 'data about data'. But metadata may itself be described, which is why the IEEE-LOM standard, for example, has a category titled 'meta-metadata'. Do we thence consider the metadata in the IEEE-LOM file to be data? (Bray, 2003) Obviously, there is a sense in which it is useful to think of it as data, and another sense in which it makes sense to think of it as metadata. This is a general issue. It is not possible to determine, based on the format or even the contents of a given file, whether it is a piece of data or a piece of metadata, because in a trivial sense, all data is 'about' something and can, in turn, have something that is 'about' it.

In this paper, therefore, no prior assumption is made regarding what may, or may not be, a resource, and no prior assumption is made regarding the structural, physical, or other characteristics of a resource. What makes something a resource is nothing more than the fact that somebody, at some time, considers it to be a resource. The definition of 'resources' thus offered in this paper is an ostensive definition: those things that we can and in fact do treat as resources, are what will be considered resources.

2. Describing Resources

2.1 Getting the Description Right

The purpose of this section of the paper is to state the problems to be addressed in the discussion to follow. We assume for the sake of argument that the purpose of a resource network is to enable people to be able to create, store, locate, retrieve resources. (IMS, 2003, Oliver, 2003) It is thus necessary at each stage of the process to be able in some way to distinguish one resource from another in a reliable manner; otherwise access to resources would be random. A common means of distinguishing items one from another is to give them a name, and this will be discussed below. However, while the practice of naming resources allows us avoid confusing them with each other, naming alone will not support the functions required of a resource network. If we had, say, only the names {'1','2', ..., '10025452'} to work from, we would have no means of deciding whether resource '2' would be a better candidate for a given purpose than resource '3545'.

We need to describe resources, that is, we need to be able to associate the having (or not having) of a given property to a set of resources. At first, the practice of describing resources may appear to be simple and straightforward, however, when a system of description is pressed a bit it becomes evident that it is fraught with difficulties. To take a simple, suppose that resource '23255' is what we commonly call an 'apple'. The use of the term 'apple' is itself the beginning of a description; it places the resource into a specific category based on a certain set of properties presumed to be had by the resource, that it is a 'pome', for example, that it 'contains' a 'core' and 'seeds'. The use of this vocabulary in turn presupposes not only a set of logical relations ('is a type of', 'contains') but also a specific vocabulary generally agreed upon by a linguistic community.

Compounding the difficulties in assigning descriptions to resources is the expectation that the description will be 'right', that is, that the description we apply to a resource will in some way be 'true' or 'accurate' or even 'useful'. This requirement introduces a host of new issues to the description of objects, a factor that is compounded by the use of differing metrics for the evaluation of the 'rightness' of a description. Though the philosophical literature is replete with models and strategies, a short survey will be sufficient to make the point. On one theory, a description is 'right' if the object, in fact, has the property being described. This theory, however, leaves open the question of the description of fictional objects ('Narnia', 'unicorns') and the attribution of subjective properties ('beautiful','honest'). A second theory proposes that a description is 'right' if it coheres with a logical or linguistic structure of descriptions. This theory, however, leaves open the possibility of systemic error or theoretical bias ('phlogiston','drives'). A third approach requires that a distinction be 'useful'. This theory, however, begs the question of what counts as 'useful' (does it mean 'cash value', does it mean 'utility'?).

These larger questions will be set aside as essentially unsolvable. What this means, for all practical purposes, is that the system of description we adopt cannot presuppose any of three major sets of criteria: the vocabularies used to name either objects themselves or properties of objects; the set of logical relations between logics; and the standard of 'rightness' of a description. None of these are presupposed because there is no means to pick between one or another, and while we may each of us express preferences in our work and our day-to-day lives, it is only a remote possibility that we would ever reach consensus on any of them.

To draw out and illustrate this point, please allow me to expand on some major areas where the 'rightness' of a description poses significant problems for current approaches to learning object metadata. I would point out that these are difficulties that cannot be addressed through better practice; they are structural flaws in the current system employed to describe learning objects.

2.2 Multiple Descriptions

There is a presumption implicit in the structure of learning object metadata that there exists a one-to-one relationship between a 'learning object' and the metadata used to describe that object. Even the slightest examination of the nature of digital resources shows that this is not the case.

Technology now exists to take the same 'resource' and to output it in a variety of formats. The application software 'Cocoon', for example, uses as input resources described in XML and outputs instances of the resource in HTML, PDF, plain text, or any of a number of formats. (Levitt, 2000) Moreover, Cocoon will output, on request, either the entire content of an resource, or only partial representations of the resource. Thus, for example, we may obtain an HTML version of the full text of 'The Red Headed League' or we may obtain a PDF version of the outline of the Conan Doyle short story. Which of these constitutes 'the' resource? It should be clear that there is no correct answer to the question. In a related case, image archives often use the same digital contents to produce an 'image' and a 'thumbnail' of the image. Norman, 2003. Which of these constitutes 'the' resource?

The possibility that works may have distinct representations is already a matter that has been addressed by the publishing industry. In the FRBR standard, for example, a four-level description of published works is employed: a 'work' is realized through an 'expression', which is embodied in a 'manifestation', which is exemplified by an 'item'. (Madison, 1997, Oliver, 2003) Each of these, in turn, has a set of associated properties. A 'work', for example, will have a 'title', 'form', 'date', and more. In the FRBR, "A Work is an abstract entity; there is no single material object one can point to as the work. We recognize the work through individual realizations or expressions of the work, but the work itself exists only in the commonality of content between and among the various expressions of the work." (Oliver, 2003)

Another source for a multiplicity of description arises in the case of what may be 'subjective' descriptions. Take, for example, the Kevin Costner film, 'The Postman', widely derided by the critical press and described as the worst film of 1991. (Ryan, 1998.) The Razzies have their opinion; I have mine, and would rank 'The Postman' as one of the better films of the year. Leaving aside the question of which assessment is 'right', we have a case here in which two distinct descriptions exist for a single film, one in which the film is classified as 'worst' and another in which it is classified 'not worst'. It is clear that there can be no single value for any given subjective description, by definition.

Much of the metadata in IEEE-LOM could be classified as subjective metadata. IEEE-LOM 5.3, 'interactivity', is a measure that, without an agreed upon metric, "can only yield subjective entries from the developers of learning systems." Schulmeister, 2001 In addition, IEEE-LOM 5.4, 'semantic density', for example, is a "subjective measure of the resource's usefulness as compared to its size or duration." (Sutton, 1999) In any case where such a subjective assessment is called for (and there are many more), we are automatically presented with the possibility of differing descriptions for any given resource. One observer may describe a learning object (or a movie) as 'too complicated for average viewers', while another may say it is 'challenging but accessible'.

2.3 The Problem of Trust

A second major problem regarding the description of resources revolves around the assumption that the person or organization providing the description will be motivated to privide an accurate description. The history of metadata is not reassuring on this point, even when it comes to what may be construed as 'objective' accounts of resource properties.

The HTML standard included the option for developers to include in document heads 'Meta' tags in order to provide content descriptions. The purpose of Meta tags in HTML documents was (and remains) exactly the same as the purpose of contemporary metadata. Meta tags were used by search engines in order to locate and organize web contents. Their use proved to be an unmitigated failure.

In "Death Of A Meta Tag," for example, Danny Sullivan summarizes, "Experience with the tag has showed it to be a spam magnet. Some web site owners would insert misleading words about their pages or use excessive repetition of words in hopes of tricking the crawlers about relevancy." () And Andrew Goodman offers this assessment: "Metatags, as many in the industry are aware, were an early victim, succumbing to the opportunism of web site owners. Marketers, particularly operators of porn sites, which made up much of the money-making power of Internet commerce circa 1995, made search engines like Altavista look pretty silly. Search engines which looked at and took metatags seriously were riddled with spam (insincere pages which manipulated their metatags in order to rank higher in searches) until they began more aggressively filtering spam with increasingly sophisticated ranking methods and filters." (Goodman, 2002) As Cory Doctorow comments, "When poisoning the well confers benefits to the poisoners, the meta-waters get awfully toxic in short order." (Doctorow, 2001)

In the field of metadata proper, the signs of similar information pollution are beginning to be noticed. The author of the Paintball Channel on the Internet Topic Exchange, an index of RSS feeds organized by topic, complains for example that "some suckers are using this media to air their dirty spam." (Jotajota, 2003) And while some suggest that, due to spam-blockers and harvester filters, that RSS solves the spam crisis (Naraine, 2003), it should be evident that it does not. There is no guarantee inherent in the RSS format -- or any XML format -- that the information placed into the file will be accurate. As Kevin A. Burton writes, "RSS is not the solution to the spam problem. The solution to the spam problem is a distributed trust metric. The major problem here is that this would require a lot of overhaul to the existing email infrastructure." (Burton, 2003)

In the field of learning object metadata there exist numerous openings for resource providers to insert false or misleading data. This will become evident once the use of metadata to distribute commercial learning content for sale becomes more widespread. A common value for 'typical age range', for example, will be '2-99' (on how many games for sale in stores have we seen this already?). Categorizations will be needlessly broad. 'Interactivity' will always be 'high', even if the resource is a static web page. Should the range of learning object expand (as I will suggest below) and more overtly evaluative metadata be included, vendors will consistently rate their material as 'best', 'cheapest' and 'most effective'. While there is no doubt that there is a great deal of honesty in the academic community, there is just enough dishonesty to undermine a system of descriptive metadata based on trust.

Untrustworthy metadata is already beginning to be seen in learning object metadata. Friesen and Anderson (2003) report observing metadata descriptions that are "more promotional than descriptive." IEEE-LOM and similar metadata standards have no means of addressing this. The presumption behind IEEE-LOM seems to be that reliable content authors or professional indexers would create metadata, leaving normal human error as the only major cause of disinformation in learning object metadata. If this was the presumption, it was not well considered.

3. Resource Profiles

3.1 Overview of the Concept

The idea of a resource profile is that it is a multi-faceted, wide ranging description of a resource. A resource profile conforms to no particular XML schema, nor is it authored by any particular author. Additionally, unlike traditional resource descriptions, which are presumed to be instantiated as a single digital file and located in a particular place, a resource profile may be distributed, in pieces, across a large number of locations. And there is no single canonical or authoritative resource profile for a given resource.

The term 'profile' was chosen because it allows an easy analogy to be drawn between a resource profile and the profile that might be created of a person. The traditional resource description (such as a learning object metadata record) may be seen as similar to a person's resume or curriculum vitae. Typically authored by the person it describes, it contains some essential information and selected highlights from that person's career and volunteer life. But when, say, an investigative agency is trying to come to a complete understanding of a person, a resume would be only one piece of the puzzle. A large number of additional records would be consulted, such as the person's driver's license, driving history, academic transcripts, credit record, criminal record. Friends may be interviewed, a bill payments examined, mail on and offline about the person may be read. A much more complete picture -- a profile -- is constructed from these various sources.

The difference between the completeness and accuracy of the information obtained in a resume as compared to a personal profile is striking. While a resume consists of a small set of information and is authored by the person, a profile consists of a large set of information authored by many people. While the trustworthiness of a resume may be cast into question, particularly if the person has something to gain from a glowing report, the trustworthiness of a profile is much higher, because data are submitted by people with no particular stake, and because different claims may be correlated with each other and with the original resume. If we wished to consider someone for a teaching position, we would be much better guided by reference to a profile than a resume; even the most minimal scrutiny involves the checking of references, and a more thorough examination would review citations, reviews and other commentary regarding the person's work. The same reasoning applies when considering the selection of a learning resource: it is the profile, not the description, that will best meet the objectives set out above, of being able to to create, store, locate, and retrieve resources.

In this section of the paper we will look at some of the defining characteristics of resource profiles. In the next section, we will survey some of the major components of resource profiles. The final section will consider questions surrounding the generation of resource profile data and its organization into a metadata network.

3.2 Vocabularies

A major underlying principle of resource profiles, drawn from the Resource Description Format (RDF), is that resource profiles may be constructed from multiple vocabularies. Any statement within a resource profile is at its core what RDF calls a 'triple' having the following form: . The is the resource being described by the profile, and is generally assumed. Thus, a profile will contain statements of the form <attribute><value>. In common parlance, the attribute is a metadata 'tag' while the value is the 'value' of that tag. Thus, in a metadata statement such as <title>As You Like It</title>, the attribute is 'title' and the value is 'As You Like It'.

The principle of multiple vocabularies has therefore two instances. The first instance is that multiple vocabularies may be used to define the range of possible attributes (tags). This is formalized in RDF through the use of 'namespaces' or schemas. The RDF schema "specifies mechanisms that may be used to name and describe properties and the classes of resource they describe." (W3C, 2003) The second instance is that multiple vocabularies may be used to define the range of possible objects (values). This is formalized in RDF through the use of 'ontologies'. "An explicit formal specification of how to represent the objects, concepts and other entities that are assumed to exist in some area of interest and the relationships that hold among them." (Paskin, 2003) In other words, "An ontology... is constituted by a specific vocabulary used to describe a certain reality, plus a set of explicit assumptions regarding the intended meaning of the vocabulary." Bechhofer, 2003, Introduction)

In practice, these two are typically combined. That is, the nature of the property may inherently define the set of possible values; this is part of the purpose of ontologies. For example, if we have a tag called <colour> then the range of possible values is clear: {'red','orange','blue',...}. But in many cases this is not (yet) defined, and in many cases, the relationship is not clear. Therefore, it is useful to think (at least conceptually) of the two types of vocabularies as being separate. So, suppose we had a tag such as <colour>red</colour>. The use of the <colour> tag is then specified by a schema, and the list of possible colours is obtained from a vocabulary. In general (still thinking conceptually), the format is <schema:attribute><vocabulary:value>. Extended, we could thus represent a statement as follows: <appearances:colour>spectrum:red</appearances:colour>.

By glossing over the technical details, we are able to extract from the preceding example the essential point: that a research profile is not confined to the use of only one schema or the use of only one vocabulary. This becomes clearer when we look at the profile of a person. It is clear that there are many ways to describe a person. So, too, with resources in general.

A person, for example, may have an 'appearance'. What we mean by 'appearance' is defined in a schema, and may include various properties such as 'colour', 'height', 'width', and more. But a person may also have an 'education', and which may include such properties as 'degrees', 'certificates', and 'workshops'. Not all schemas apply to all people. A driving person would having a 'driving record' while a non-driver would have none; a criminal person would have 'priors' while such a description makes no sense for the law-abiding. Similarly, a particular property may be described in a number of ways. A person's height, for example, may be described in terms of 'feet' and 'inches', or it may be defined in terms of 'centimeters'. Their 'identification number' may use the Canadian 'Social Insurance Number' or the American 'Social Security Number'.

None of these descriptions would be useful when describing a learning resource, of course; it does not even make sense to think of a learning resource as having a criminal record (only humans have criminal records, a fact that would, at some point, be recorded in an ontology). A learning resource might have a 'height' and a 'width', though, but typically only if it is an image; a text document does not have dimensional properties. While both people and learning resources may have a property called 'size', the person's size will be expressed as (say) a diameter, while an image's size will be expressed in bytes. Sometimes learning resources have more in common with people than they do with each other; a law professor and a law book may both have as a 'location' the Law Library, but a digital transcript will have as a 'location' only a URL.

It should be clear from this discussion, then, different sets of properties apply to different types of resources. Because there are many types of learning resources, it follows that learning resources ought to be described differently, with different sets of properties. An approach, therefore, such as that taken by IEEE-LOM, where every resource is described with a single set of properties, is inappropriate for this domain.

That said, IEEE-LOM already recognizes that there are different types of metadata. This may be seen by the division of the LOM into ten separate categories: general, lifecycle, technical, and like. IEEE, 2003 Each of these different categories may be viewed as being defined by a separate schema. This is in fact exactly the approach taken by Nilsson (2003) in the RDF binding of LOM metadata. Where appropriate, he replaces IEEE-LOM schema elements with Dublin Core elements. In RSS-LOM (Downes, 2003) different schemas defined as part of the RSS 1.0 protocol (Swartz, 2000) in order to create a combined format.

IEEE-LOM also allows for various vocabularies. The classification element, for example, contains two distinct components: a reference to the taxonomy being used, and the value of the current resource within that taxonomy. As the CanCore guidelines explain, "Classification element category is sophisticated and complex, providing elements for identifying and describing the purpose of the classification, the source, taxonomic value and identifier associated with the classification." (Friesen, 2003 The use of external vocabularies in IEEE-LOM is restricted, however. In a resource profile, the use of external vocabularies is unrestricted.

3.3 Authorship

Although in a certain sense a criminal is the author of his own misfortune, the authorship of the person's criminal record is not left to the person described, for the reason that such people will be motivated to falsely report their prior convictions. In a similar manner, a person's academic transcript is authored by the university registrar, and not the person being described. The same reasoning extends to description of other types of resources. Except in certain notable cases, movie reviews are not authored by movie studios, book reviews are penned by people other than the author. Some descriptions are not authored by a person at all. A person's power or water usage is recorded by a meter and fed directly into a central database, where it is used to issue power and water bills or to suggest targets for possible police investigations.

Learning object metadata files, however, are like many others assumed to have a single author. As we can see from the CanCore metadata guidelines, there is no provision for different authorship for different bits of information, save what (little) could be gleaned from the 'role'. (Friesen, 2003a) A learning resource profile, however, may have may authors. In principle, each statement within a learning profile could have a different author (though in practice, different authors will create different sets of tags).

The idea of attributing comments to authors is called 'reification'. Wikipedia (as of October 31, 2003) defines the concept: "In knowledge representation, reification is sometimes used to represent facts that must then be manipulated in some way, for example to compare logical assertions from different witnesses to determine their credibility. The message "John is six feet tall" is an assertion of truth that commits the sender to the fact, whereas the reified statement, "Mary reports that John is six feet tall" defers this commitment to Mary. In this way, the statements can be incompatible without creating contradictions in reasoning." Wikipedia, 2003 The concept of reification is explicitly discussed as such by Tim Berners-Lee. (Berners-Lee, 1999) And it is already instantiated in various semantic web implementations, such as Annotea. (Miller, 2003)

Tracking the authorship of metadata statements requires that author information be contained in the metadata. Author information must be contained in two places: in the first place, to designate the author of a given metadata file, which I'll call the 'metadata author'; and in the second place, to designate the author of tag contents, which I'll call the 'element author'. In IEEE-LOM, the metadata author is indicated in the metametadata. Other contributors may also be indicated in this area. Attributing element authors is not so straightforward; while Annotea describes the use of additional metadata tags, a more direct approach is preferred here: place a 'source' attribute within the tag pointing to the original metadata where the assertion was first made. Hence, for example, if we are depending on a second author for information about the resource's classification, we could describe it as follows: <schema:classification source="http://theothermetadata.com/feed.xml"> Information about the authorship of the classification metadata in this example would therefore be obtained by dereferencing the source and locating it within the metametadata.

In previous work I have referred to metadata authored in this way as 'third party metadata' [http://education.qld.gov.au/staff/learning/courses/sdownesnov.html], the idea being that metadata authored by the resource creator is first party metadata and that authored by the resource consumer is second party metadata. This term has been used in other work, sometimes as 'third party annotation' (Bartlett, 2001) or 'third party labeling' (Eysenback, 2001). Recker and Wiley (2001) use the term 'nonauthoritative metadata' to describe third party metadata: "metadata that describe the variety of real world cases in which a given resource has been reused, what we have termed 'nonauthoritative metadata', can be extremely helpful in facilitating the efficient and effective reuse of existing resources." The term 'third party' is preferred here as while there is no doubt of the source, there may be, as suggested above, some doubt of the trustworthiness of first party metadata.

3.4 Distributed Metadata

Alluded to in passing in the previous section, this principle of resource profiles allows that the metadata for a given resource may be stored in different locations across the internet. That is, there is no single metadata file describing any given resource; metadata about the resource may be found in numerous online locations. A metadata profile is therefore constructed by aggregating the metadata available at these different locations in order to form a particular view of the resource. It follows that there may be different metadata profiles for a given resource, as different aggregators harvest different metadata from different locations, though one could define an ideal (and usually fictional) 'total' metadata profile composed of all possible metadata from all possible sources.

Again, this corresponds with the manner in which information about a person is distributed. A person's health records are stored at a hospital, their driving record at the Department of Motor Vehicles, their academic transcripts at a university, they birth information at a bureau of statistics, and the like. Very little information about a person is actually obtained from the person himself, usually only easily verifiable data such as the person's current address and telephone number. Even though a person may assert additional information in, say, a resume, this information is in fact subject to verification through reference to the originators of that information or through the production of certificates, such as a driver's license or university diploma, and not taken at face value.

In the world of learning resources, a very similar pattern may be expected and, indeed, has begun to take shape already. For example, the learning resource titled The Fugues of the Well-Tempered Clavier, by Timothy A. Smith and David Korevaar, is located in one place. (Smith, 2003) This resource has been reviewed by the MERLOT Music Review Panel, and the review is located in another place. (MERLOT, 2003) An aggregator seeking to obtain a complete profile of this resource may be therefore to obtain information from two separate locations in order to form a complete picture.

3.5 Resource Identifiers

In some discussion to follow, it will be seen that a resource cannot be identified by its location on the internet. A resource may take one of several technical forms, or a resource may be mirrored to lower distribution costs. Additionally, resource metadata may have no single internet location. Because metadata descriptions of a given resource may have different authors, and may be located in different places, there needs to be a means of knowing when two metadata resources are describing the same resource. It should be clear that the title of a work cannot serve as an identifier either. For example, the title of this paper is duplicated by a description of services available to senior citizens (Senior Citizen's Guide, 2003), an account of agriculture in Kyrgyzstan (Fitzherbert, 2000), and a mainframe applications utility. (Leroy, 2002.) This difficulty is resolved by means of a resource identifier.

The same sort of difficulties exist in the realm of personal identification. Though a person may have a name, just as a resource has a title, this name may be a duplicate. There exists a Stephen Downes who is a restaurant critic in Melbourne, a Stephen Downes who works for the National Research Council, a Stephen Downes who is a visual artist in New York, a Stephen Downes who is a professor of philosophy at the University of Utah, and a Stephen Downes who was an NDP candidate in Nova Scotia. Any individual's physical address may change over time, and other identifying information, such as website addess, email address, or phone number, may also change.

Organizations respond to this difficulty by assigning each person a unique identifier. Examples of identifiers in Canada include Social Insurance Numbers, health care numbers, and driver's license numbers. Additionally, organizations, such as universities, will also assign their own unique identifiers. What is common about these identification systems is that each identifier is unique, and each identifier is stored in a canonical location (which may be called a 'registry'). In turn, these identifiers are associated with (what may be) less permanent information about a person, such as the person's name or address. When a less permanent feature of a person changes, the person is required to update the registry with the new information. Mechanisms are in place in order to prevent the fraudulent change of a registry.

In the realm of digital resources, the idea of resource identifiers has been proposed on numerous occasions. Books, for example, may be identified by their ISBN (ISBN, 2003); serials by their ISSN. (ISSN, 2003) A prominent initiative, the Digital Object Identifier system, (DOI, 2003) "provides a framework for managing intellectual content, for linking customers with content suppliers, for facilitating electronic commerce, and enabling automated copyright management for all types of media." The DOI syntax is an ANSI standard, Z39.84, (NISO, 2000) and is defined in two parts: a prefix, which identifies the identity of the registration agency, and a suffix, which is the unique code assigned by that agency. (NISO, 2000) Because the DOI registration system is a commercial enterprise, however (OASIS, 2003), organizations such as eduSource have adopted their own format, but again with the same two-part structure.

There is from time to time a call for a single standard for digital resource identification, just as there is from time to time a similar demand for a single standard for identifying people. Over time, some such standard may become a de facto universal standard (as has Canada's Social Insurance Number for Canadians), however, such calls should be resisted. Organizations may find it more convenient to employ an internal identifier scheme, employing a public scheme only when the resource is published or made public. Additionally, the use of multiple identifier systems is more able to withstand a catastrophic corruption, as even if one registry is corrupted, reference to additional registries may be employed to establish the original identity.

3.6 Models

A model is an XML description that is used for multiple purposes. The purpose of a model is to store information in one place in order to allow it to be used in multiple places. A model functions in much the same way as a Cascading Style Sheet (CSS) (W3C, 2003a) A full definition of a given style is stored in a CSS file; the CSS style is imported by the web page in which the style will be used, and HTML in the web page invokes the style by referring to it by name. In resource profiles the second step is omitted; the external resource is involved and implemented within the body of the XML.

In a certain sense, models are already supported in RDF. For any given property value, instead of using a string to indicate the value, an XML author may instead refer to an external resource. For example, the 'creator' of a document may be 'Stephen Downes'. However, this reference is vague (there may be, as suggested above, other people named 'Stephen Downes') and it is incomplete (what is the current 'email address' for the author?). RDF allows the 'creator' of the document to be identified as an external 'resource' using the following syntax: <s:Creator rdf:resource="http://www.w3.org/staffId/85740"/> (W3C, 1999) This is functionally equivalent to embedding vcard information into the XML (as proposed by IEEE-LOM).

In general, the use of external resources in this manner should be encouraged, and in reliable metadata networks, should be mandatory (conversely, XML which does not refer to external resources in this way should not be deemed trustworthy). The use of string data to refer to and describe external resources, such as authors and organizations, even if it is encoded in (say) vcard format, is fraught with danger. Such information will almost certainly change. Aside from the ambiguity of reference, pointed to above, people change email addresses and organizations (such as Docent and Click2Learn) merge and change names.

Because even the URLs of such resource metadata files will change over time, it is desirable when referring to an external resource to employ a permanent URL such as is provided by PURL (PURL, 2003) or a similar registry of resource locations. In such a case, the mechanism for referring to external resources would come to resemble (and, in fact, be a part of the same system as) the resource identification service described in the previous section. Hence, the reference to the external resource would be described in two parts: the name of the resource registry, and the unique identifier held by that registry. The registry, in turn, would either redirect an enquiry to the current location of the resource (as PURL does) or would return a set of metadata, which ought to include the location of that resource.

In a resource profile, a 'model' is employed in the same manner. A model has two parts: the name of the registry holding the model, and the unique identifier for the model. A model differs from an external resource, however, in that it is a partial metadata file and it does not describe any given resource. Rather, it describes a resource type, and the data contained in the model is intended to be descriptive of the current resource.

It is useful to think of resource models in much the same way we think of stereotypes as applied to people (but without the negative connotations). For example, if we have a person named 'Salty', we could add to the description of this person by invoking a specific model: 'sea captain'. Knowing that Salty is a sea captain immediately tells us many things about him: that he wears a captain's hat, that he has a peg leg, that he sings sea shanties. These details are not inferred (as would be the case with an ontology), these details are contained in the model itself. The model 'sea captain' just is the following XML: "<hat>captain's</hat><leg>peg</leg><sings>sea shanties</sings>". The model does not describe any person in particular, but when included as part of a resource profile, adds specific details to the description of the resource.

A model is used by a metadata author for several reasons. The use of metadata models may greatly simplify the creation of metadata. For example, in describing the digital rights associated with a resource, the associated ODRL file may run into several pages of detail. (W3C, 2002) However, if the relevant digital rights model is given a (recognizable) name, then this information may be very simply added to a metadata description.

A model may also be used to apply similar descriptions to multiple resources. For example, some properties of images offered by an image repository may be the same for each of 10,000 images: they may all be .gif images, 800 x 1400 pixels in dimension, with a colour depth of 16 bytes (or 256 colours). This information could be stored in a model called 'portrait', and then each of the 10,000 images could declare these technical specifications as a single line of XML code: <format model="our_models:portrait" />.

A third reason to use a model is to withhold metadata that may be subject to subsequent change, while at the same time making the current value of this metadata available to aggregators. For example, the price of a resource offered by a commercial provider may change over time. If digital rights metadata is included in the resource metadata or content package metadata, as proposed by COLIS (Iannella, 2002), then the digital rights associated with an object cannot be changed. While this is useful (and necessary) for objects that have already been transacted, such a system is unsustainable for the delivery of rights metadata prior to the conclusion of a transaction. Once a resource is offered a $50.00, it would have to be offered at that price forever, since no reliable means would exist for changing the price once the metadata had been harvested by third parties.

3.7 The Concept in Retrospect

The use of metadata to describe learning resources is, in essence, an effort to create a distributed and integrated system of data management and application. The concept of the resource profile, as described immediately above, represents what could be viewed as a set of best practices for such enterprises. While on the one hand the details of the concept may be subject to further amendment and elaboration by those more familiar with the details of data management and application, they are nonetheless built on known and widely applied principles, principles that may be viewed in other applications of data management, but unfortunately, not to learning object metadata.

The metaphor of a system for the organization of personal information was used throughout for illustrative purposes, but the reference standard for the elaboration of the concept of resource profiles ought to be data management theory. Several of the properties of resource profiles described immediately above are instances of data management theory. In particular, the use of resources and models conforms to sound practices of database design and object oriented programming. The former corresponds with the principle of data normalization (Gilfillan, 2000), which could, in a nutshell, be expressed as a variant of Ockham's razor (Britannica, 2003): do not multiply entities without necessity. The use of a resource identifier corresponds with the requirement for the use of a primary key for all data; the use of external resources instead of strings corresponds to the requirement of the use of (what are sometimes called) lookup tables instead of manually entered referents.

The use of metedata models enables inheritance. (Sun, 2003) Inheritance is a common (even necessary) feature of object oriented programming. Not only does the use of inheritance reduce the complexity of applications programming, it reduces the possibility of error and eases the work required to maintain application integrity. Inheritance also facilitates the identification of groups or classes or objects, and allows developers to predict the behaviour of objects even when information about that behaviour is not present. In more practical terms, the use of inheritance is an instance of 'not reinventing the wheel'. To force metadata authors, even automated metadata authors, to input, say, author data over and over again is a violation of that principle on a massive scale.

Finally, the development of a distributed system of metadata authoring is an instance of the aphorism 'two heads are better than one' and draws from the design and architecture of the world wide web itself. Centralized and sole source information networks have been found to present insurmountable bottlenecks to the aggregation and distribution of data. (Shirkey, 2003) Even closed data management systems presuppose multiple authors; an examination of university data systems such as Banner or Colleague will show that, even though the data itself is centralized, authorship is distributed. In addition single point authoring of metadata has shown itself to be unusable in a world-wide network; this method, employed in the early days of Yahoo, has been superseded by services such as Google, which employ an aggregation rather than a data entry system.

The aggregation of information about a given resource from many sources has proven to be a formidible application. Google's Page rank system, for example, depends on what are here called third party resources. One aspect of this system is to rank a page according to the number of links to that page are contained in other pages. (Google, 2003) This provides a system of ordering search results which could not have been imagined using a system in which individual authors provide all and only the metadata describing their own pages.

The concept of the resource profile itself draws on numerous existing concepts in web design and metadata, including most clearly the Resource Description Format, but also Digital Object Identifiers and object registries, reification, FRBR, Annotea, and more. The concept described in these pages is not intended to replace any of these prior initiatives or specifications, but rather to draw on them and to convince readers to look at the concept of resource description using metadata from a different frame of reference. The most difficult part of designing metadata descriptions, including IEEE-LOM, lies in understanding exactly what it is we are trying to do, and a failure to grasp the wider picture leads to errors in specific implementations, such as the errors in IEEE-LOM that have been alluded to in passing in this paper.

4. Concluding Remarks

In the preceding section the major elements of resource profiles were surveyed. This model needs in turn to be elaborated with a description of the major types of metadata constituting a resource profile, for example, bibliographic metadata, technical metadata, classification metadata, evaluative metadata and educational metadata. Different types of metadata should be authored by the appropriate agency -- evaluative metadata by evaluators, educational metadata by educators, for example. There is not room in this paper to present such a survey, however, this extended work is available online in an extended version of the <a href="http://www.downes.ca/files/resource_profiles.htm">Resource Profiles</a> paper. This paper also described how resource profile metadata should be used, describing the lifecycle of a resource as it emerges with only a minimal profile and as it acquires, through use, a richer and more complete metadata description. The picture that emerges of resource metadata is not that of a static, sterile description, but of a varied and textured profile.

4.1 The Future of Metadata

The science of metadata has been traditionally depicted as ordering the unordered, that "the purpose of metadata is to impose some order in a disordered information universe." (Lagoze, 2003) For the most part, however, this objective is misplaced. This is not because the desire to order the universe is misplaced; indeed, without the order inherent in natural laws and classifications the universe could not be comprehended at all. Rather, it is because the task of ordering information is best understood as something that is not accomplished in the creation of information, but rather, in the use of information. And the use of information is something that, like its object, almost defies order.

The central thread running through the concepts and mechanisms described in this paper is the recognition that the ordering of the universe, if it is to be accomplished at all, will not be accomplished in one place, in one way, or by one person. It is a recognition that a resource, like the proverbial elephant, may be viewed from different perspectives by different people. This is especially the case in more practical environments: a person buying an elephant, or seeking to use an elephant to pull a cart, will be interested only in a narrow set of properties, properties that might even be satisfied by certain oxen or horses better than some other elephants.

The second major thread running through this paper is the idea that, in order to be useful, these myriad descriptions must be communicated and connected one to the other. The idea is that, although there is no single common system of description, neither are there millions of individual descriptions. One person's description of a resource may have a great deal in common with another's, and these descriptions could usefully be clustered. groups of people with a similar perspective on a resource will adopt a similar vocabulary. Hence the need for a two-way flow of description, to enable people with such common interests to draw from and support each other.

This essay is a description of the technical and conceptual infrastructure underlying a system of metadata that adheres to these two threads. As mentioned above, it attempts to employ existing protocols and processes rather than redefine the concept of resource profiles from scratch. That this is possible without major modifications to any of the existing protocols and processes described shows that, to a significant degree, the properties essential to the creation of a resource profiles network have already begun to be embedded in the metadata network. However, until the nature of resource profiles is widely understood and widely shared by practitioners, these initiatives will continue to operate in silos, in isolation from each other, and the longer term benefits of metadata will not be realized.

4.2 The Intelligent Network

One might ask, what are the longer term benefits of metadata? Where is the payoff? Near the beginning of this paper, it was suggested that the purpose of metadata was to enable people to be able to create, store, locate and retrieve resources. In this final section we will look at how a network as described above realizes these objectives.

A great deal has been written about applications and systems that will use metadata in order to accomplish, say, the task of searching for resources online. Some authors, for example, propose that intelligent agents will work with metadata in order to organize and filter online information. "Resource discovery by agents can enable qualitatively more flexible applications than those in existence today, due to the fact that systems can be built to intelligently react to situations and environment not known at the time of system design." (Lassila, 1997)

The use of intelligent agents, however, simply places on computer software the onus to perform tasks that humans have thus far not been able to do. There is no reason to suppose that agents will be more successful, because agents will face the same problems humans do. There are too many resources to search, too many possible interactions, uncertainties in vocabulary, and trust issues. If the organization of information remains unchanged, agents will have no more success than humans. But conversely, if the organization is modified, then humans themselves may be able to perform the tasks previously assigned to agents.

To understand how this is possible, it is necessary to shift one's point of view from the idea that the network of information needs to be organized to the idea that what we want is a self-organizing network of information. That is not to say that no human intervention is required: people will, of course, have to create resources, describe resources, and use resources. But it is to say that the impossible task of organizing, sorting, filtering and retrieving these resources will be performed not by agents working on the network, but by the network itself.

We are already familiar with self-organizing networks. The human brain is one such system: constituted of billions of interconnected neural cells responding to and comprehending myriad sensory input, the human brain, with no particular design or program (and certainly no homunculi) manages to arrange all that data into an understanding of the world. (Loder, 1996) The study of the functioning of the human brain has led to the development of neural networks as a theory of computation. Today, connectionist systems are widely understood and studied, and though they have evolved far beyond their original biological basis, the fundamental principles remail constant.

The first principle of neural network design is that it is a form of distributed processing. No one node, no one neuron, corresponds to a macro phenomenon such as 'understanding' or 'our idea of the city of Paris'. Each neuron, by itself, with only a partial understanding of the process, manages only one aspect of the total function or concept. And the second major principle is connectivity. Neurons send information to each other, not at random, but as input to layers of additional neurons. Thus, for example, in the human visual processing system we observe layers of interconnected neurons performing the task of resolving random visual data into what Marr called the "2 1/2 dimensional sketch". (Glennerster, 2002)

The network of resource metadata described in this paper enulates the neural network. Layers of raw, disorganized input are provided by resource creators. This information flows, via aggregation, to a secondary layer, which performs a preliminary sort and filtering. Metadata may flow through additional layers as necessary. Finally, it reaches the output layer, where the resources are used. Data from the use and through what neural network theorists would call 'back propogation' this usage metadata is used to fine tune the connections and processing in the resource network. The result is that no individual or organization 'organizes' the network; it organizes itself.

How do we know this will work? We know, because it does work: it works in human cognition, and it works in artificially developed neural networks. Moreover, we have seen evidence of it working already on the web, through such phenomena as PageRank and blogging networks. The self-organizing network is not merely a pipe-dream, it is here already, and to see it those working in the field need only perform that hardest of all tasks, to recognize it.

References

Al-Muhajabah, 2003. What Is Trackback? Al-Muhajabah's Islamic Pages, 2003. http://www.muhajabah.com/islamicblog/what-is-tb.htm

APEGGA, 2001. PEGGAsus. The Association of professional Engineers, Geologists and Geophysicists of Alberta. November 24, 2003. http://www.peggasus.ca

Bartlett, 2001. Backlash vs. Third-Party Annotations from MS Smart Tags. Kynn Bartlett. WWW-Annotation Mailing List, World Wide Web Consortium. June 15, 2001. http://lists.w3.org/Archives/Public/www-annotation/2001JanJun/0115.html [cited]

Bechhofer, 2003. Tutorial on OWL. Sean Bechhofer, Ian Horrocks and Peter F. Patel-Schneider. 2nd International Semantic Web Conference, October 20, 2003. http://www.cs.man.ac.uk/~horrocks/ISWC2003/Tutorial/ [cited]

Bennett and Metros, 2001. The Promise and Pitfalls of Learning Objects: Current Status of Digital Repositories. Kathy Bennett and Susan Metros. EDUCAUSE, October 21, 2001. http://itc.utk.edu/educause2001/default.htm

Berners-Lee, 1999. The Semantic Toolbox: Building Semantics on top of XML-RDF. Tim Berners-Lee. World Wide Web Consortium, June 18, 1999. http://www.w3.org/DesignIssues/Toolbox.html [cited]

Bray, 2003. On Resources. Tim Bray. Ongoing, July 24, 2003. http://www.tbray.org/ongoing/When/200x/2003/07/24/HTTP-14 [cited]

Britannica, 2003. Ockham's Razor. Encyclopædia Britannica. 2003. Encyclopædia Britannica Premium Service. November 23, 2003 http://www.britannica.com/eb/article?eu=58133 [cited]

Burton, 2003. RSS Is Not The Solution To Spam. Kevin A. Burton. PeerFear.Org, September 2, 2003. http://www.peerfear.org/rss/permalink/2003/09/02/RSSIsNotTheSolutionToSpam/ [cited]

CreativeCommons, 2003. Creative Commons. Website, 2003. http://creativecommons.org/

Crossref, 2003. CrossRef. Website. http://www.crossref.org

Crossref, 2003a. DOI Information & Guidelines. Crossref, 2003. http://www.crossref.org/02publishers/15doi_guidelines.html

Doctorow, 2001. Metacrap: Putting the torch to seven straw-men of the meta-utopia, Version 1.3. Cory Doctorow. August 26, 2001. http://www.well.com/~doctorow/metacrap.htm [cited]

DOI, 2003. The Digital Object Identifier System. http://www.doi.org/ [cited]

Downes, 2001. Learning Objects: Resources for Distance Education Worldwide. Stephen Downes. International Review of Research in Open and Distance Learning: 2, 1, 2001. http://www.irrodl.org/content/v2.1/downes.html [cited]

Downes, 2002. Paying for Learning Objects in a Distributed Repository Model. Stephen Downes. http://www.downes.ca/cgi-bin/website/view.cgi?dbs=Article&key=1041124246

Downes, 2003. RSS-LOM. Stephen Downes. http://www.downes.ca/xml/RSS_LOM.htm [cited]

Downes, 2003a. Edu_RSS Topics. Stephen Downes. 2003. http://www.downes.ca/cgi-bin/xml/topics.cgi

Downes, 2003b. Design, Standards and Reusability. Stephen Downes. July 31, 2003. http://www.downes.ca/cgi-bin/website/view.cgi?dbs=Article&key=1059622263

Downes, 2003c. Edu_RSS Ratings. Stephen Downes. 2003. http://www.downes.ca/cgi-bin/xml/edu_ratings.cgi

Downes, 2003d. Meaning, Use and Metadata. Stephen Downes. August 25, 2003. http://www.downes.ca/cgi-bin/website/view.cgi?dbs=Article&key=1061841698

Downes, 2003e. Design and Reusability of Learning Objects in an Academic Context: A New Economy of Education?. Stephen Downes. USDLA Journal, Volume 17, Number 1, January, 2003. http://www.usdla.org/html/journal/JAN03_Issue/article01.html

Editeur, 2003. Website. http://www.editeur.org/

Eysenback, 2001. A metadata vocabulary for self- and third-party labeling of health web-sites: Health Information Disclosure, Description and Evaluation Language (HIDDEL). G. Eysenbach, C. Köhler, G. Yihune, K. Lampe, P. Cross and D. Brickley. AIMA, 2001. http://www.medcertain.org/pdf/AMIA2001-final-edited-hiddel.pdf [cited]

Fitzherbert, 2000. Country Pasture/Forage Resource Profiles. Anthony R. Fitzherbert. Food and Agriculture Organization of the United Nations, 2000. http://www.fao.org/WAICENT/FAOINFO/AGRICULT/AGP/AGPC/doc/Counprof/kyrgi.htm [cited]

FOAF, 2003. The Friend of a Friend (FOAF) project. Website. http://www.foaf-project.org/

Franklin, 1994. Perl Regular Expression Tutorial. Carl Franklin and Gary Wisniewski. http://www.english.uga.edu/humcomp/perl/regex2a.html

Friesen, 2003. CanCore Guidelines Version 1.9: Classification Category. Norm Friesen, Susan Fisher, Anthony Roberts, Susan Hesemeier and Scott Habkirk. The Canadian Core Learning Object Metadata Guidelines. CanCore, 2003. http://www.cancore.org/guidelines/1.9/CanCore_guidelines_Classification_1.9.pdf [cited]

Friesen, 2003a. CanCore Guidelines Version 1.9: Meta-Metadata Category. Norm Friesen, Susan Fisher, Anthony Roberts, Susan Hesemeier and Scott Habkirk. The Canadian Core Learning Object Metadata Guidelines. CanCore, 2003. http://www.cancore.ca/guidelines/1.9/CanCore_guidelines_Meta-metadata_1.9.pdf [cited]

Friesen, 2003b. CanCore Guidelines Version 1.9: General Category. Norm Friesen, Susan Fisher, Anthony Roberts, Susan Hesemeier and Scott Habkirk. The Canadian Core Learning Object Metadata Guidelines. CanCore, 2003. http://www.cancore.ca/guidelines/1.9/CanCore_guidelines_General_1.9.pdf

Friesen, 2003c. CanCore Guidelines Version 1.9: Life-Cycle Category. Norm Friesen, Susan Fisher, Anthony Roberts, Susan Hesemeier and Scott Habkirk. The Canadian Core Learning Object Metadata Guidelines. CanCore, 2003. http://www.cancore.ca/guidelines/1.9/CanCore_guidelines_Life_Cycle_1.9.pdf

Friesen, 2003d. CanCore Guidelines Version 1.9: Technical Category. Norm Friesen, Susan Fisher, Anthony Roberts, Susan Hesemeier and Scott Habkirk. The Canadian Core Learning Object Metadata Guidelines. CanCore, 2003. http://www.cancore.ca/guidelines/1.9/CanCore_guidelines_Technical_1.9.pdf

Friesen, 2003e. CanCore Guidelines Version 1.9: Annotation Category. Norm Friesen, Susan Fisher, Anthony Roberts, Susan Hesemeier and Scott Habkirk. The Canadian Core Learning Object Metadata Guidelines. CanCore, 2003. http://www.cancore.ca/guidelines/1.9/CanCore_guidelines_Annotation_1.9.pdf

Friesen, 2003f. CanCore Guidelines Version 1.9: Educational Category. Norm Friesen, Susan Fisher, Anthony Roberts, Susan Hesemeier and Scott Habkirk. The Canadian Core Learning Object Metadata Guidelines. CanCore, 2003. http://www.cancore.ca/guidelines/1.9/CanCore_guidelines_Educational_1.9.pdf

Friesen and Anderson, 2003. vPreliminary LOM Survey. Norm Friesen and Terry Anderson. Academic ADL Co-Lab Learning Repository Summit, October 8, 2003. http://www.academiccolab.org/events/oct78/lom_prelim_survey_NormTerry.ppt [cited]

Gilfillan, 2000. Database Normalization. Ian Gilfillan. Database Journal, March 22, 2000. http://www.databasejournal.com/sqletc/article.php/1428511 [cited]

Gillmor, 2003. RSS Hitting Critical Mass. Dan Gillmor. SiliconValley.Com, August 17, 2003. http://weblog.siliconvalley.com/column/dangillmor/archives/001285.shtml

Glennerster, 2002. Computational theories of vision. Andrew Glennerster. Current Biology, 12, R682-685, 2002. http://www.physiol.ox.ac.uk/~ag/pub/g2002/myg2002.pdf [cited]

Goodman, 2002. An End to Metatags (Enough Already, Part 1). Andrew Goodman. Traffik, September 2, 2002. http://www.traffick.com/article.asp?aID=102 [cited]

Google, 2003. Our Search: Google Technology. Google, 2003. http://www.google.com/technology/index.html [cited]

Google, 2003a. Google News Alerts (BETA). Google, 2003. http://www.google.com/newsalerts

Handschuh, 2001. CREAM ? Creating relational metadata with a component-based, ontology-driven annotation framework. Siegfried Handschuh, Steffen Staab, and Alexander Maedche. Semantic Web Working Symposium, July 30, 2003. http://www.semanticweb.org/SWWS/program/full/paper4.pdf

Hot or Not, 2003. Hot or Not. Website, 2003. http://www.hotornot.com

Hunter and Armstrong, 1999. A Comparison of Schemas for Video Metadata Representation. Proceedings of the Eighth International World Wide Web Conference (WWW8), May 11-14, 1999. http://archive.dstc.edu.au/RDU/staff/jane-hunter/www8/paper.html

IANA, 2002. MIME Media Types. Internet Assigned Numbers Authority, January 2, 2002. http://www.iana.org/assignments/media-types/

Iannella, 2003. The Open Digital Rights Language Initiative. Renato Iannella. Website, 2003. http://odrl.net/

Iannella, 2002. COLIS ODRL Metadata Profile. Renato Iannella. COLIS. July 4, 2002. http://workshops.eduworks.com/EdMedia2003/Docs/COLIS/COLIS-ODRL-Profile-04.pdf [cited]

IEEE, 2002. Position Statement on 1484.12.1-2002 Learning Object Metadata (LOM) Standard Maintenance/Revision. December, 2002. http://ltsc.ieee.org/wg12/ [cited]

IEEE, 2003. Learning Object Metadata (LOM) Final Draft. 2003. Was at http://ltsc.ieee.org/doc/wg12/LOM_1484_12_1_v1_Final_Draft.pdf but has now been stolen from the commons. [cited]

IFLA, 2003. Related efforts - Working Group on FRBR (Functional Requirements for Bibliographic Records) - Section on Cataloguing. International Federation of Library Associations and Institutions, 2003. http://www.ifla.org/VII/s13/wgfrbr/related_efforts.htm.

IMS, 2002. IMS Simple Sequencing Best Practice and Implementation Guide. October 17, 2002. http://imsproject.org/simplesequencing/v1p0pd/imsss_bestv1p0pd.html

IMS, 2003. IMS Digital Repositories Specification. October 21, 2003. http://www.imsglobal.org/digitalrepositories/index.cfm [cited]

ISBN, 2003. ISBN.Org. Website. http://www.isbn.org/standards/home/index.asp [cited]

ISSN, 2003. ISSN Home page: Navigate the ocean of periodicals with the ISSN Website. http://www.issn.org:8080/pub/ [cited]

Jotajota, 2003. RSS Spam. Jotajota. rnd(Thoughts). September 9, 2003. http://www.jotajota.org/archives/000131.html [cited]

Karieauskas, 2002. Text Categorization Using Hierarchical Bayesian Network Classifiers. Guytis Karieauskas. 2002. http://citeseer.nj.nec.com/karieauskas02text.html

Lagoze, 2003. Metadata Challenges for Libraries. Carl Lagoze. Preprints of the Metadiversity Conference Proceedings, 2003. http://www.nfais.org/publications/metadiversity_preprints24.htm [cited]

Lananas, 2002. Alternative Architectural Concept 2 - Federated Integration. Stephen Lahanas. CETIS, November 12, 2002. http://www.cetis.ac.uk/content/20021112133437

Lassila, 1997. RDF Metadata and Agent Architectures. Ora Lassila. November 21, 1997. http://www.objs.com/workshops/ws9801/papers/paper056.html [cited]

Leroy, 2002. Resource profiles utility. Patrick Leroy. Mainframe Week, January 30, 2002. http://www.mainframeweek.com/journals/articles/0004/Resource+profiles+utility [cited]

Levine, 2002. Syndicating Learning Objects with RSS and Trackback. Alan levine, Brian Lamb and D'Arcy Norman. MERLOT, August 8, 2003. http://www.mcli.dist.maricopa.edu/show/merlot03/

Levitt, 2000. Cocoon: Sanity For Web-Site Management. Jason Levitt. Information Week, May 22, 2000. http://www.informationweek.com/787/cocoon.htm [cited]

Loder, 1996. Neural Networks: An Overview. Chad Loder. February 28, 1996. http://www.ccs.neu.edu/groups/honors-program/freshsem/19951996/cloder/ [cited]

Magee and Friesen, 2001. CAREO Overview and Goals. Michael Magee and Norm Friesen. 2000, revised 2001. Campus Alberta Repository of Educational Objects. http://www.careo.org/documents/overview.html [cited]

Madison, 1997. Functional Requirements for Bibliographic Records: Final Report. Olivia Madison, et.al. International Federation of Library Associations and Institution, 1997. http://www.ifla.org/VII/s13/frbr/frbr.htm [cited]

McGee, 2003. Learning objects: Bloom's taxonomy and deeper learning principles. Patricia McGee. AACE E-Learn, November 18, 2003. http://educ3.utsa.edu/pmcgee/nlii/McGee_eLEARN_2003.doc

MERLOT, 2003. Peer Review of The Fugues of the Well-Tempered Clavier. MERLOT Music Review Panel. MERLOT, July 10, 2003. http://www.merlot.org/artifact/PeerReviewDetail.po?rOid=1010000000000136852 [cited]

Michigan, 2002. Mammography Machine Operator Performance Evaluation. Michigan Department of Consumer and Industry Services, January 10, 2002. http://http://www.michigan.gov/documents/cis_bhs_fhs_bhs_hfs_889_37201_7.pdf

Miller, 2001. RDF Calendar taskforce. Libby Miller. Institute of Learning and Research Technology, Bristol University, April 10, 2001. http://ilrt.org/discovery/2001/04/calendar/

Miller, 2003. RDF Annotations. Libby Miller. Institute of Learning and Research Technology, Bristol University, April 4, 2003. http://ilrt.org/discovery/2001/04/annotations/ [cited]

Monthienvi, 2001. Educational Metadata: Teacher's Friend or Foe? Rachada Monthienvichienchai, Angela Sasse and Richard Wheeldon. Euro-CSCL, January 17, 2001. http://malted.cs.ucl.ac.uk/dev/publications/Metadata_Paper.doc

Naraine, 2003. Is RSS the Answer to the Spam Crisis? Ryan Naraine. InternetNews.Com, September 1, 2003. http://www.internetnews.com/dev-news/article.php/3070851 [cited]

NEC, 2003. CiteSeer. Website. 2003. http://citeseer.nj.nec.com/cs

Nesbit, 2002. A Convergent Participation Model for Evaluation of Learning Objects. John Nesbit, Karen Belfer and John Vargo. Canadian Journal of Learning and Technology Volume 28(3) Fall / automne, 2002. http://www.cjlt.ca/content/vol28.3/nesbit_etal.html

Netscape, 1996. Inline Plug-Ins. Netscape Communications Corporation, 1996. Mirror, http://physics.hallym.ac.kr/resource/plugins/

Nilsson, 2003. RDF binding of LOM metadata. Mikael Nilsson. Centre for User Oriented IT Design, january 15, 2003. http://kmr.nada.kth.se/el/ims/metadata.html [cited]

NISO, 2000. ANSI/NISO Z39.84 -2000 Syntax for the Digital Object Identifier. National Information Standards Organization. http://www.niso.org/standards/standard_detail.cfm?std_id=480 [cited] [cited]

NISO, 2002. Data Dictionary?Technical Metadata for Digital Still Images. National Information Standards Organization and AIIM International, June 1, 2002. http://www.niso.org/standards/resources/Z39_87_trial_use.pdf

Norman, 2003. IMS LOM, Thumbnails, and Relations. D'Arcy Norman. D'Arcy Norman's Learning Commons Weblog, November 12, 2003. http://commons.ucalgary.ca/weblogs/dnorman/000234.html [cited]

Norman, 2003a. CanCore Metadata Guidelines Updated. D'Arcy Norman. D'Arcy Norman's Learning Commons Weblog, September 11, 2003. http://commons.ucalgary.ca/weblogs/dnorman/000209.html

OASIS, 2002. DIG35: Metadata Standard for Digital Images. OASIS Cover Pages, June 10, 2002. http://xml.coverpages.org/dig35.html

OASIS, 2003. Digital Object Identifier (DOI) System. OASIS Cover Pages, March 15, 2003. http://xml.coverpages.org/doi.html [cited]

Oliver, 2003. FRBR Functional Requirements for Bibliographic Records: What is FRBR and why is it important? Chris Oliver. Canadian Metadata Forum, September 19, 2003. http://www.nlc-bnc.ca/metaforum/n11-228-e.html [cited] [cited] [cited]

Paskin, 2003. DOI Handbook, version 3.3: Glossary. Norman Paskin. International DOI Foundation, November, 2003. http://www.doi.org/handbook_2000/glossary.html [cited]

PRISM, 2003. Publishing Requirements for Industry Standard Metadata. Website. http://prismstandard.org/

PURL, 2003. Persistent Uniform Resource Locator. http://www.purl.org/ [cited]

Recker and Wiley, 2001. A non-authoritative educational metadata ontology for filtering and recommending learning objects. Recker, M.M. and Wiley, D.A. Journal of Interactive Learning Environments, Swets and Zeitlinger, The Netherlands, 2001. Referenced in http://www.strath.ac.uk/Departments/CAP/reusing/intropart2.html [cited]

Rightscom, 2003. The MPEG-21 Rights Expression Language: A White Paper. Rightscom, July 14, 2003. http://www.rightscom.com/files/MPEG21_RELwhite_paper.pdf

Ruiz, 2003. Hierarchical Text Categorization Using Neural Networks. Miguel E. Euix and Padmini Srinivasn. Information Retrieval, 5, 87-118, 2002. http://mingo.info-science.uiowa.edu/padmini/Papers/hier.pdf

Ryan, 1998. Costner's "Postman" Stamped. Joal Ryan, E! Online, March 23, 1998. http://www.eonline.com/News/Items/0,1,2726,00.html [cited]

SAS, 2003. Diagrams for Relational Metadata Types. SAS 9 Open Metadata API Reference, 2003. http://support.sas.com/rnd/eai/openmeta/v9/reference/reldiags.html

Schulmeister, 2001. Taxonomy of Multimedia Component Interactivity A Contribution to the Current Metadata Debate. Rolf Schulmeister. Studies in Communication Sciences. Studi di scienze della communicazione. Special Issue (2003) - S. 61-80. http://www.izhd.uni-hamburg.de/pdfs/Interactivity.pdf [cited]

Senior Citizen's Guide, 2003. Resource Profiles. Senior Citizen's Guide, retrieved 2003. http://seniorcitizensguide.com/profiles/ [cited]

Shirkey, 2003. Otlet: Some ideas die because they are wrong. Clay Shirkey. Corante: Many-tp-Many, November 20, 2003. http://www.corante.com/many/archives/2003/11/20/otlet_some_ideas_die_because_they_are_wrong.php [cited]

Smith, 2003. Well-Tempered Clavier: Johann Sebastian Bach: Twenty-Seven Fugues and Select Preludes. Tim Smith and David Korevaar. Northern Arizona University. http://jan.ucc.nau.edu/~tas3/wtc.html [cited]

Sullivan, 2002. Death Of A Meta Tag. Danny Sullivan. Search Engine Watch, October 1, 2002. http://www.searchenginewatch.com/sereport/article.php/2165061

Sun, 2003. What Is Inheritance? Sun Microsystems. The Java Tutorial, 2003. http://java.sun.com/docs/books/tutorial/java/concepts/inheritance.html [cited]

Stufflebeam, 1971. The relevance of the CIPP evaluation model for educational accountability. Stufflebeam, D. L. Journal of Research and Development in Education. 5(1), 19-25., 1971. Cited in http://www.reusability.org/read/chapters/williams.doc

Sutton, 1999. IEEE 1484 LOM mappings to Dublin Core: Learning Object Metadata: Draft Document v3.6. Stuart A. Sutton. IEEE Learning Technology Standards Committee (LTSC), September 5, 1999. http://www.ischool.washington.edu/sasutton/IEEE1484.html [cited]

Swartz, 2000. RDF Site Summary (RSS) 1.0. Aaron Swartz. http://web.resource.org/rss/1.0/ [cited]

Technorati, 2000. Technorati. Website, 2003. http://www.technorati.com/

Tillett, 2002. The FRBR Model (Functional Requirements for Bibliographic Records). Barbara B. Tillett. Workshop on Authority Control among Chinese, Korean and Japanese Languages (CJK Authority 3), March, 2002. http://www.nii.ac.jp/publications/CJK-WS3/cjk3-07a.pdf

Trott, 2003. Comment Spam. Ben Trott. Six Log, October 13, 2003. http://www.sixapart.com/log/2003/10/comment_spam.shtml

Udell, 2003. Working with Bayesian Categorizers. Jon Udell. XML.Com, November 19, 2003. http://www.xml.com/pub/a/2003/11/19/udell.html

VMC, 2002. Multimedia Metadata Standards. Virtual Museum Canada. Canadian Heritage, April 27, 2002. http://www.chin.gc.ca/English/Standards/metadata_multimedia.html

W3C, 1999. Resource Description Framework (RDF) Model and Syntax Specification. World Wide Web Consortium, February 22, 1999. http://www.w3.org/TR/REC-rdf-syntax/ [cited]

W3C, 2001. Web Services Description Language (WSDL) 1.1 W3C Note 15 March 2001. World Wide Web Consortium. http://www.w3.org/TR/wsdl

W3C, 2002. Open Digital Rights Language (ODRL) Version 1.1. W3C Note 19 September 2002. World Wide Web Consortium. http://www.w3.org/TR/2002/NOTE-odrl-20020919/#eltId103 [cited]

W3C, 2003. RDF Vocabulary Description Language 1.0: RDF Schema. W3C Working Draft 10 October 2003. World Wide Web Consortium. http://www.w3.org/TR/rdf-schema/ [cited]

W3C, 2003a. Cascading Style Sheets home page. November. World Wide Web Consortium. http://www.w3.org/Style/CSS/ [cited]

WCHS, 1998. Siskel & Ebert. WCHS-TV News 8, 1998 (and not updated in five years). http://www.wchstv.com/synd_prog/siskelebert.html

Wikipedia, 2003. Reification. http://en.wikipedia.org/wiki/Reification [cited]

Williams, 2000. Evaluation of learning objects and instruction using learning objects. David D. Williams. The Instructional Use of Learning Objects, David A. Wiley, ed. http://www.reusability.org/read/chapters/williams.doc