Taxonomy of Thorendal

July 9, 2005

What is an information architect?

by Thorendal

I can hardly claim to be capable of giving a consice nor precise definition of what an information architect is. But I find it interesting that no one (that I have met at least) is able to.

In my world there excist two kinds of information architects: The very practical one and the more abstract one - so who is who, and who does what?

The practical ones
I regularly go to this Information Architect meetup in Copenhagen. Here I talk to quite a few information architects. What makes me wonder though is, that all the others in the group are working in a practial perspective with information architecture. In other words they very often work with the graphical stuf and how to structure interfaces, and functionalities on websites. A lot of method is practised in this area I know, but as I understand it, it is very much like testing-adjusting and the methods behind that kind of work which is used.
Definitely I discipline I have much respect for.

The abstract ones
My working title is information architect too, my education is a candidate degree in library and information science from the Royal School of Library and Information Science. I have focused my education on information and knowledge management, systems for information and knowledge management, communication of information and knowledge, and standards for handling information and knowledge.

So what does that make me able to actually do? Well data and information organization and structuring (my definition of knowledge makes it impossible to structure - one can only support handling knowledge).
That means that I know things about taxonomies, thesauri, and conctrolled vocabularies. I know about users seach patterns and relevance assesments, as it is tightly related to the organization of information. I know about metadata and standards for handling metadata.

Still none the wiser
I don’t know which kind of information architects are needed the most, and I don’t really understand either, that two so different groups of work areas are called the same. I have no suggestions to what the “practical” group should be called, as I find it logical that my domain is called information architecture - and the “pracical” ones would say the same about their area. Maybe they shoud both be called information architects - it just seems to confuse people as my expirience is that they think that all information architects are practical ones.

Last time we met in the meetup group we were actually talking about the situation that I was the only one out of ten people that night that were working with abstract information architecture. Another - for me bizarre - thing was, that I was the only girl present, and I usually am. What is that all about? Where are all the girls? (About 60% of all the students at the Royal School of Library and Information science are girls! And in genereal it is to be said that about 60% of the people who finish a long education are girls too!)

Anyways, I just don’t understand the situation and the confusion in the two rather different disciplines, and I have come across no one who has been able to explain to me me, why the domian has been split up in this way - if in fact it is one domain…

Filed under Taxonomy at 10:19 am
Add a comment »

June 22, 2005

Binary relevance meassuring in today’s information management systems

by Thorendal

Within the LIS sector (Library and Information Science) there excits two concepts that everyone knows: Precistion and recall. They are interesting to have a closer look at because they are very traditional meassurement within the domain and at least earlier on very used and respected perameters.

The Cranfield II Studies
These parameters were developed through some expiriments called the Cranfield II studies back in the early 50′ies in the UK. In these studies there was a distinction between two kinds of relevance: ‘Stated relevance’ and ‘User relevance’. It is obvious that the difference is that the former is predefined and the latter is based on the users assesement - and therefor subjective - of relevance (Ellis 1990, p. 15-16).

In the Cranfield II Studies outer paramenters such as context, user level e.g., have not been taken into account. This has of course been strongly criticized by all theory within this domain (Harter, 1996, p. 45).
The relevance in the Cranfield II studies looks at relevance as either a document is relevant and it is retrieved or it is not relevant and therefor it is not retrieved (Bruhns, 1998, p.17). Pretty basic.

The question is if they are adequate parameters for the meassurement of relevance in intellegent information retrieval systems nowadays? And are these perameters called precision and recall?

There is an ongoing discussion on how many aspects relevance as concept has. As the Cranfield II Studies operates with a binary concept of relevance it can be stated that it is way too narrow. On the other hand if the judgement relevant is rather broad it can cover all the grey areas of what is really relevant to the user or the search.
The most normalt thing nowadays is though to have more than two levels for the concept of relevance.

Precesion is defined as:
The ratio between the number of relevant and retrieved documents and the number og retrieved documents.

Recall is defines as:
The ration between the number of relevant and retrieved documents and the total number of relevant documents in the system - retrived or not. This meassure provides that the total number of relevant documents in the system for the specific search is known.

The somewhat conclusion must be then, that if a system can be trained sufficiently and is intelligent enough the precision and recall meassures should be accepetable for meassurering binary relevance - but then a new question emerges: Is anyone interested in a relative narrow or very broard definition of ‘relevant’?

References:
Bruhns, 1998,
Ellis, P. 1990,
Harter, 1996,

Filed under Taxonomy at 10:46 pm
2 comments »

June 16, 2005

FreePint

by Thorendal

For those of you within the information architecture domain who does not already know this site, I think you should have a look at FreePint. FreePint is an online network of information searchers. Members receive this free newsletter twice a month, which is packed with tips on finding quality and reliable business information on the Internet. I think a lot of their topics are pretty interesting, and they give a good impression about what is going on around the world with research, conferences, litterature and so on.
I don’t really look that much at the website, but I like their newsletter. But the website gives many opportunities for looking for and participating in forums, shopping for reports, networking and the like.
On the website I particularly like the forum in there where it is possible to see what kind of barriers people around the world are having in their work, and what kind of interests they have. And a lot of people use it, so it is a very active forum.

It might be very focused on librarians, but many other professionals would get a lot of good information from the site, as it is rather minded on research but still practical.

Freepint

Filed under Taxonomy at 9:34 am
Add a comment »

June 6, 2005

What is relevance?

by Thorendal

To begin with I have to say, that I will not be claiming to talk the truth about relevance. I do though find it very interesting - and puzzeling - that relevance can be view upon from so many different perspectives. And that is why I will write some posts about different aspects of relevance.

I have chosen to call this post:

What is relevance?

The concept of relevance differs depending on which article or book you read on the topic. Today it is more or less agree upon that relevance is a dynamic concept, in the sense that it is a mixture of classic algorithmes for relevance and the users own perception of the concept that constitutes the concept.
Therefor I will have a look at Saracevic’s (1996) types of relevance, as they do take into account that relevance is a dynamic concept. I like his way of differentiating types of relevance and mixing the objective and subjective types. That does not mean that others haven’t comitted good theory on the topic. I just happen to know and like Saracevic.

Saracevic divides relevance into five types:

1. Algorithmic relevance. This is an objective type of relevance where the outcome totally is depending on the match between query and the collection. The result is based on algorithmes and the degree of succes is reliant on the degree of consistency of query and reitrieved documents.

2. Topical relevance. This kind of relevance is very much depending on the indexing policy that the system is based on, as the relevance is compliant with ‘aboutness’. Relevance is only seen as topical matter and is therefor not necessarily reliant on the document. The succes of this kind of relevance is therefor very much dependent on the users ability to formulate their actually needs (the request) and on the retrieved documents.

3. Pertinence or cognitive relevance. This is a subjective kind of relevance. The relevance is to be judged in accordance to the actual information need. The succes is depending upon the indexing policy and methods and the users ability to understand it’s own needs and ability to formulate these needs up against the system. Pertinence or cognitive relevance is the relation between the user’s perception of the need and the retrived documents.

4. Situational relevance. This is a subjective and dynamic type of relevance, which is relying on the user’s perception and interpretation of the tasks and the social-cognitive context. That means that the concept situational relevance is the relation between the retrieved documents and the user task as the user interpret them. So the succes in situational relevance is very much depending on the users ability to use a piece of information to satisfy it’s need.

5. Motivational or affective relevance. This is a completely subjective type of relevance. It is a bit vague what this kind of relevance is besides it is the degree of satisfaction, succes or the like that can affect the user’s motivation and opinion.

It has to be remebered that every time the talk is on relevance, the context is very often ignored for the fairly static definitions of different types of relevance. The user’s inherent information is taken from one context and has to be used in another - the system. Relevance is multi-dimentional and complex, but if the user’s perception of relevance is taken into account, then relevance as concept can be useful.
In general the cognitive approach to information retrieval is gaining ground as systems are becoming more and more complex and the standards are raising. Earliere on the complexity didn’t excist to the same extent as today. More systems are coming into the everyday life and more users will be using these systems. Therefor it is important that the types of relevance that are used reflect this. So even though Saracevic’s types of relevance are fairly simple they are easy to use as meassurements for different kinds of more or less binary relevance.


References:

- Borland, P. and Ingwersen, P. (1998). Measures of relative relevance and ranked half life: Performance indicators for interactive IR. In: Croft, B.W., Moffat, A., van Rijsbergen, C.J., Wilkenson, R., and Zobel, J. eds. Proceedings of the 21st ACM Sigir Conference on Research and Development of Information Retrieval. Melbourne, 1998. Australia: ACM Press/York Press, p.324-331.
- Borland, P. (2000). Evaluation of interactive information retrieval systems. Åbo: Åbo Akademis förlag. 276 p.
- Saracevic, T. (1996). Relevance reconsidered ‘96. In: Ingwersen, P. and Pors, N.O. (eds). Informations Science: Integration in Perspectives. Proceeding of CoLIS 2, Second International Conference on Conceptions of Library and Information Science, Copenhagen, 1996. Copenhagen: Royal School of Librarianship, 1996, p.201-218.
- Schamber, L., Eisenberg, M.B. and Nilan, M.S. (1990). A re-examination of relevance: Toward a dynamic, situational definition. Information Processing & Management, (26), p. 755-775.

Filed under Taxonomy at 12:07 pm
Add a comment »