CBHL Congrès 97 - James M. Turner, EBSI, Université de Montréal

Layers of meaning

Research in this area mostly centres around the model developed by Erwin Panofsky, which describes 3 levels of meaning: pre-iconographical, iconographical, and iconological.
The first two correspond to “ofness” and “aboutness” which we find in the library science literature in relation to subject access.
The pre-iconographical or “ofness” level, which has to do with identifiable objects, people, or events, “the class of primary or natural meanings” (Panofsky 1955, 27) [e.g., a picture of an apple].
The iconographical or “aboutness” level, which has to do with meaning discerned by interpretation. It involves acculturation to societal convention [e.g. the apple representing agriculture, children's nutrition, the juice industry].
The iconological or third level has to do with symbolic interpretations, and is predicated on knowledge and erudition, as well as substantial cultural education. For indexing purposes, we can forget this level, except in very specialised circumstances [e.g. Adam & Eve's adventures, or representing education because of the association with teachers].
Exceptions to this last: allegorical figures, personifications, pictures with texts which explain the artist's meaning (van Straten 1990, 1112).
Note: the concepts of “ofness” and “aboutness” are also referred to as “hard indexing” and “soft indexing” (Krause 1988).

Using these levels to provide subject access

Art collections have historically been indexed only at the “aboutness” level (Markey 1981).
Some collections of non-art pictures should use only the “ofness” level (Turner 1994, 165).
Both the “ofness” and “aboutness” levels can be used to find the same picture by different users (Shatford 1986, 53), or by the same user at different times.
Because of computer technology, we don't need to decide between the two anymore. We can have both. Do we want both?
Computers can easily manage all this information for us, if:
(1) we structure our databases carefully, and (2) we invest in the upfront indexing costs.

The “vocabulary problem”

Computer systems often use the “armchair method” of naming things (Furnas et al. 1987) which helps us understand the “vocabulary problem”.
Each person thinks of only a few names, thinks what is obvious to him or her is obvious to everyone else, and feels the possibilities are exhausted. However, over several people, a large pool of names for things is built up.
Of these, a few terms emerge as the most popular, so these are the obvious indexing terms.
With non-art pictures, maybe the possibilities are more obvious than with other types of information objects such as the names of computer systems, computer commands, employee functions, and so on (Turner 1994).

Separating “ofness” and “aboutness” terms

With non-art pictures, naming the people, objects, events in the pictures seems to be what's useful for retrieval later on.
We can nail down most of the indexing terms we want fairly quickly, but deciding which are the “ofness” and which are the “aboutness” terms isn't so easy.
This is partly because often the word symbol for the concept is the same at both levels (e.g. you're looking at a picture of horses, so the “ofness” descriptor is horses, but you can also say the picture is about horses, as well as maybe farm life, beasts of burden, cowboys, and so on).
Classification schemes and other controlled vocabularies purportedly at the “aboutness” level often contain “ofness” terms (Markey 1981, 25, 57, 65; 82; van Straten 1990, 12).
It seems useful to separate the vocabulary into the two categories so there can be two separate fields of descriptors in a database. This would allow more precise search strategies. However, in practice this is not easily accomplished because the line between the two types of descriptors is so fuzzy.
In the case of ordinary photographic images (as opposed to art images), the “aboutness” is often related to potential uses of the image (Krause 1988, 73-74, 81), e.g. “this picture might be good for expressing problems with fish stocks”.
However, with some picture collections the indexer needs to avoid making “aboutness” judgment calls. The user should be forced to ask “ofness” questions. For example, the National Archives might well want to index the “aboutness” of the Great Depression, but people creating a multimedia information system need to decide how they want to express the concept in their product.

Separation line

Access points particular to image collections

General

This section has to do with controlled vocabularies for expressing various aspects of visual documentation which we might well want to index. We say such avenues of access are particular to image collections because of their value in retrieving pictures. This does not mean that they are exclusive to image collections.
As a general rule, we will probably want to construct lookup tables as modules of a database. These can contain an entire system (e.g. the UDC geographical codes) or perhaps just those codes needed for indexing pictures actually found in the collection. For an example of a data structure for stock film, see Turner (1990, Appendices).
Take the time to build a solid data structure that responds to all present and foreseeable needs, and don't tie it to the technology, which will be different in a few years.
You may have to fight to convince your boss. Computer maxim: There's never enough time to do it right, but there's always time to do it again.

Camera angles

How the camera is looking at the object of interest is often important and sometimes critical to the user looking for pictures. For example, “I need shots of Chicago, but only aerial shots will do”. Or, “I need a picture of Hillary Clinton, preferably an extreme close-up”.
It is useful to adopt some standard terminology, e.g. the Glossary (1984) published by the National Film Board of Canada; however, you may have special needs related to your collection. If need be, make your own list or modify some other list.

Colours

Some collections might use this as an important access point, e.g. a collection of museum objects, a fashion designer's archives, a collection of pictures of flowers.
It might also be used to elicit emotion, e.g. “I need a shot of something blood-red that takes up most of the frame”.
When computers become smart enough to identify objects and index them, they will also be able to do searches for colours associated with objects. Already it's easy enough for them to calculate how much space some range of some colour takes up.

Time periods

In many situations this access point is critical. For example, “I need some street scenes of Rome in the 1940s”. Or, “I need some shots of automobiles, but they need to be models from within the last year or so”.
The tables in the Universal Decimal Classification are useful for expressing these, e.g. 19 = the twentieth century,. 194 = the 1940s, 1945 = 1945. Also, there are codes for the seasons, times of the day, times of war and peace, and so on. An important advantage is that time can be expressed down to a fraction of a second, but it can also be left general or vague.
For time periods, the design of the information system may need to take into account the real time (when the material was actually shot) and the represented time (e.g. a film shot in 1995 but set in the 1880s).

Geographical space

As with time periods, in many situations this access point is critical. For example, “I need some Toronto street scenes from the 1940s” means that the shots can only have been taken in Toronto.
Again, the UDC tables are very useful as a base vocabulary for expressing geographical space, e.g. 7 = North America, 71 = Canada, 713 = Ontario, 713.541 = metropolitan Toronto, 713.541*M5V 2S9 = the Globe and Mail Building. You can express geography very generally or very specifically.
Often geographical features cross political boundaries. The UDC tables provide for this, e.g. 23 = mountains, 234 = mountains of Europe, 234.2 = mountains of France and Belgium, 234.24 = Massif Central, 234.242 = mountains of the Auvergne region.
Other controlled vocabularies are available (e.g. the geographical subdivisions of the Library of Congress Subject Headings) for expressing geographical reality, but the advantages of a classification for this purpose are evident.
For geographical spaces, the design of the information system may need to take into account the real place (where the material was actually shot) and the represented place (e.g. a film shot in Costa Rica but set in Zimbabwe).

Existing vocabularies

Sometimes a thesaurus or a classification created for some other purpose can provide a base vocabulary to build on, sometimes (as with botanical classification) it can be used as is. Sometimes a combination of a few lists works best.

Some other possible access points

Expression of emotion: This is not useful in all collections, but in some collections it is very useful. Some kind of list or classification is desirable, at least as a base vocabulary. Indexing for emotion is dicey, because it is subject to the indexer's interpretation. Still, where this kind of access is needed, it is probably better to have imperfect access by emotion than none at all.
Lighting conditions: For a particular search, the user may want only indoor shots or only outdoor shots, or very bright shots or shots with very little light, or chiaroscuro effects, lots of shadow or none.
Texture: Users of a particular collection might want to be able to ask for only soft-focus shots, or very sharp images, or very grainy images, and so on.
Some other special conditions: Underwater shots, heat-detecting cameras, material shot from space, infrared film, particular clothing, weather, re-enactments, digitally-altered images, special effects, morphing.

Separation line

Conclusions

Issues related to picture indexing are numerous, and resolving them is a challenge.
Different approaches are needed for different kinds of collections, depending on their content and the needs of their users.
Standardise methods, data structures, and controlled vocabularies as much as possible to foster networked exchange of data. Use the Dublin Core.
However, don't let standards get in the way of good information management. Adopt the optimal methods, data structures, and controlled vocabularies you, your institution, and your users need in order to properly express the information you have.
Even if you can't afford to digitise your collection right away, you can still make a solid investment now, in the textual records (the metadata) that you need in any case to manage the picture information.
In designing a database for pictures, take all your present and foreseeable needs into consideration, and invest in the most solid data structure you can devise. The technology will catch up later.
Remember, the computers are supposed to be working for us, not the reverse.

Separation line

References & Bibliography

Discussions and mentions of problems with using the text model

Bearman, David. 1990.: Contexts of creation and dissemination as approaches to documents that move and speak: a paper presented at the National Archives of Canada Symposium on Documents that Move and Speak: Managing Moving Images and Recorded Sound Documents in Archives : Session on Intellectual Control: Theory and Concepts. Ottawa, 1990 04 3005 03.
Buckland, Michael K. 1991.: Information retrieval of more than text. Journal of the American Society for Information Science 42, no. 8 (September): 58688.
Goldsmith, Catherine. 1981.: Subject index related to the audio-visual collection of the Art Gallery of Ontario. In Computerized inventory standards for works of art, ed. Raymond Vézina, 10525. Montréal: Fides.
Jussim, Estelle. 1977.: The research uses of visual information. Library Trends 25, no. 4 (April): 76378.
O'Connor, Brian C. 1985.: Access to moving image documents: background concepts and proposals for surrogates for film and video works. Journal of Documentation 41, no. 4 (December): 20920.
Pierrard-Biniek, Maryse. 1981.: Analyse documentaire de l'image fixe: état de la question. Argus 10, no. 5 (septembreoctobre): 81-88.
Turner, James. 1994.: Determining the subject content of still and moving image documents for storage and retrieval: an experimental investigation. PhD thesis, University of Toronto.
Wright, Richard M. 1981.: Arrangement and indexing. In Picture Librarianship, ed. Helen P. Harrison, 13149. Phoenix, AZ: Oryx Press.

Layers of meaning, levels of indexing discussions

Egeter van Kuyk, R.H.J. 1981.: Historical film documentation at the Netherlands Information Service. UNESCO Journal of Information Science Librarianship and Archives Administration 3 (October-December): 22734.
Goodman, Nelson. 1972.: Problems and projects. New York: Bobbs-Merrill.
Krause, Michael G. 1988.: Intellectual problems of indexing picture collections. Audiovisual Librarian 14, no. 4 (November): 7381.
Markey, Karen. 1981.: Computer-assisted construction of a guide to themes and concepts in medieval art and their essential attributes. PhD diss., Syracuse University.
Markey, Karen. 1984a.: Interindexer consistency tests : a literature review and report of a test of consistency in indexing visual materials. Library and Information Science Research 6: 155-77.
Markey, Karen. 1988.: Access to iconographical research collections. Library Trends 37, no. 2 (fall): 15474.
Panofsky, Erwin. 1955.: Meaning in the visual arts : papers in and on art history. Garden City, N.Y.: Doubleday Anchor Books.
Shatford, Sara. 1984.: Describing a picture : a thousand words are seldom cost effective. Cataloging & Classification Quarterly 4, no. 4 (summer): 13-30.
Shatford, Sara. 1986.: Analyzing the subject of a picture : a theoretical approach. Cataloging & Classification Quarterly 6, no. 3 (spring): 39-62.
Turner, James M. 1993.: Subject access to pictures: considerations in the surrogation and indexing of visual documents for storage and retrieval. Visual Resources: an International Journal of Documentation 9, 24171.
van Straten, Roelof. 1990.: Indexing Italian prints with ICONCLASS. Visual Resources 7, no.1: 121.

Subject access

Barnett, Patricia, and Toni Petersen. 1989.: Subject analysis and AAT/MARC implementation. Art Documentation 8, no. 4 (December 1989): 17172.
Chenhall, Robert G., ed. 1988.: The revised nomenclature for museum cataloging: a revised and expanded version of Robert G. Chenhall's system for classifying man-made objects. Nashville: AASLH Press.
Kley, Ronald J. 1987.: Whatchamacallit: problems and potentials in nomenclature and classification. Curator 30, no. 2: 107112.
Markey, Karen. 1988.: Access to iconographical research collections. Library Trends 37, no. 2: 15474.
Markey, Karen. 1986.: Subject access to visual resources: a model for computer construction of thematic catalogs. New York: Greenwood Press.
Ostroff, Harriet. 1990.: Subject access to archival and manuscript material. American Archivist 53 (December): 100105.
Parker, Elisabeth Betz. 1982.: Graphic materials: rules for describing original items and historical collections. Washington: Library of Congress.
Parker, Elisabeth Betz. 1995.: LC thesaurus for graphic materials: topical terms for subject access. Washington: Library of Congress.
Roddy, Kevin. 1991.: Subject access to visual resources: what the 90s might portend. Library Hi Tech 9, no. 1: 4549.
Rorvig, Mark E. 1990.: Intellectual access to graphic information. Library Trends 38, no. 4 (March). Theme issue.
Shatford, Sara. 1986.: Analyzing the subject of a picture: a theoretial approach. Cataloging and Classification Quarterly 6, no. 3 (March): 3962.
Small, Jocelyn Penny. 1991.: Retrieving images verbally: no more key words and other heresies. Library Hi Tech 9, no. 1 (1991): 5160.

Structuring information and designing information systems

Bearman, David. 1988.: Considerations in the design of art scholarly databases. Library Trends 37, no. 2 (September): 206219.
Bearman, David. 1990.: Functional requirements for collections management systems. Archives and Museum Informatics Technical Report, no. 3. Pittsburgh: Archives & Museum Informatics.
Besser, Howard. 1989.: Adding analysis tools to image databases: facilitating research in geography and art history. In User-oriented content based text and image handling: RIAO (1988), March 2124, 1988. Cambridge, MA: MIT Press, 97290.
Besser, Howard. 1990.: Visual access to visual images: the UC Berkeley Image Database Project. Library Trends 38, no. 4 (March): 78798.
Evans, Linda J. 1988.: MARC for archival visual materials: a compendium of practice. Chicago: Chicago Historical Society.
Gecsei, Jan, and Daniel Martin. 1989.: Browsing access to visual information. Optical Information Systems (September): 23741.
Matters, Marion. 1990.: Reconciling sibling rivalry in the AACR2 “family”: the potential for agreement on rules for archival description of all types of materials. American Archivist 53 (December): 7693.
Olson, Nancy B. 1985.: Cataloging of audiovisual materials: a manual based on AACR2. Mankato, Minn.: Minnesota Scholarly Press.
Petersen, Toni. 1989.: Information on images: the Art and architecture thesaurus as a front end for retrieval. National Online Meeting Proceedings, 1989, compiled by Carol Nixon and Lauree Padgett. Medford, NJ: Learned Information: 33541.
Pisciotta, Henry. 1992.: The development of an indexing system using the AAT for Archpics. AAT Bulletin
Procedural guide to automating an art library, ed. Patricia J. Barnett and Amy E. Lucker. 1987.: Tucson, AZ: Art Libraries Society of North America, Occasional Papers, no. 7.
Rules for archival description = Règles pour la description des documents d'archives.: Ottawa: Bureau of Canadian Archivists, 1990+. Chapter 2: Multiple media fonds; Chapter 4: Graphic materials; Chapter 6: Architectural records; Chapter 7: Moving images; Chapter 8: Sound recordings.
Sarasan, Lenore. 1984.: Visual content access; an approach to the automated retrieval of visual information. Automatic Processing of Art History Data and Documents, 1. Pisa: Scuola Normale Superiore, 1984.
Schuller, Nancy S.: Standard abbreviations for image description for use in fine arts visual resource collections. Visual Resources Association, publications, No. 2 (US$10).
Shackleton, M.A. and W.J. Welsh. 1991.: Classification of facial features for recognition. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Lahaina, Maui, Hawaii, 3 June 1991. Los Alamitos, CA: IEEE Computer Society Press: 57379.
Stam, Deirdre C. 1989.: The quest for a code, or a brief history of the computerized cataloging of art objects. Art Documentation (March): 715.
Stone, Gerald, and Philip Sylvain. 1990.: ArchiVISTA: a new horizon in providing access to visual records of the National Archives of Canada. Library Trends 38, no. 4 (March): 73750.
Turner, James. 1990.: Representing and accessing information in the stockshot database at the National Film Board of Canada. Canadian Journal of Information Science 15, no. 4 (December): 122.
Van der Wateren, Jan Floris. 1988.: Achieving the link between art object and documentation: experiences in the British Architectural Library. Library Trends 37, no. 2: 24351.
White-Henson, Wendy. 1984.: Archival moving image materials: a cataloging manual. Washington: Library of Congress, Motion Picture, Broadcasting, and Recorded Sound Division.
Yee, Martha. 1988.: Moving image materials: genre terms. Washington: Cataloging Distribution Center, Library of Congress.
Zinkham, Helena, and Elisabeth Betz Parker. 1986.: Descriptive terms for graphic materials: genre and physical characteristic headings. Washington: Cataloging Distribution Service, Library of Congress.

Other references

Bakewell, Elizabeth, William O. Beeman, and Carol McMichael Reese. 1989.: Object, image, inquiry: the art historian at work. Santa Monica, CA: Getty Art History Information Program.
Bidd, Donald, Louise de Chevigny, and Margo Marshall. 1986.: PRECIS for subject access in a national audiovisual information system. Canadian Library Journal (June): 17784.
Cawkell, A.E. 1993.: An introduction to image processing and picture management. Journal of Document and Text Management 1, no. 1: 5363.
Furnas, G.W., T.K. Landauer, L.M. Gomez, and S.T. Dumais. 1987.: The vocabulary problem in human-system communication. Communications of the ACM 30, no. 11 (November): 96471.
Greenberg, Jane. 1993.: Intellectual control of visual archives: a comparison between the Art and architecture thesaurus and the Library of Congress thesaurus for graphic materials. Cataloging & Classification Quarterly 16, no. 1: 85117.
National Film Board of Canada = Office national du film du Canada.: Glossary = Glossaire. Research and Development = Recherche et développement. Montréal, 1984.

Separation line

General

Camera angles

Colours

Time periods

Geographical space

Existing vocabularies

Some other possible access points

Conclusions

References & Bibliography

Discussions and mentions of problems with using the text model

Layers of meaning, levels of indexing discussions

Subject access

Structuring information and designing information systems

Other references

Main menu | EBSI | Comments

Indexing pictures: some considerations

Table of contents

Introduction

Some kinds of visual collections

What do we mean by indexing?

Can indexing methods for text
work also for pictures?

Background

Some cataloguing issues

Some issues related to the electronic environment

So: can text indexing methods work with pictures?

Levels of description:
“ofness” and “aboutness”

Layers of meaning

Using these levels to provide subject access

The “vocabulary problem”

Separating “ofness” and “aboutness” terms

Access points particular to image collections

Indexing pictures: some considerations

Table of contents

Introduction

Some kinds of visual collections

What do we mean by indexing?

Can indexing methods for text work also for pictures?

Background

Some cataloguing issues

Some issues related to the electronic environment

So: can text indexing methods work with pictures?

Levels of description: “ofness” and “aboutness”

Layers of meaning

Using these levels to provide subject access

The “vocabulary problem”

Separating “ofness” and “aboutness” terms

Access points particular to image collections

General

Camera angles

Colours

Time periods

Geographical space

Existing vocabularies

Some other possible access points

Conclusions

References & Bibliography

Discussions and mentions of problems with using the text model

Layers of meaning, levels of indexing discussions

Subject access

Structuring information and designing information systems

Other references

Main menu | EBSI | Comments

Can indexing methods for text
work also for pictures?

Levels of description:
“ofness” and “aboutness”