Re: weighted impact factors

Peter,

Thanks for keeping us up on this recent research.

The distinction Bollen et al. make between popularity and prestige is certainly evocative. I hope I am reading the abstract correctly in that a journal's prestige would be an emergent property of the citation statistics rather than a port from the
journal's offline reputation. If so, then I'd be very interested in seeing how they weight PageRank to accomplish this, as the Interarchive group has been exploring a similar tack (though without the "journal" layer).

If, on the other hand, the authors are merely weighting according to pre-Web factors instead of bootstrapping an emergent measure of prestige, that would be a lost opportunity. We've been there before, as when proponents of the "dot-museum" Web
suffix fell back on traditional definitions of a museum to justify imposing a class structure on the inherently democratic landscape of online art:

http://www.mediachannel.org/arts/perspectives/dotmuseum/index.shtml

Thankfully, the "dot-museum" movement seems to have completely flopped. I can't think of any "prestigious" museums who bought into it :)

jon

Peter Suber <peters@earlham.edu> on Wednesday, February 8, 2006 at 2:42 PM -0500 wrote:
>This is an interesting paper.
>
>Johan Bollen, Marko A. Rodriguez, and Herbert Van de Sompel, "Journal
>Status," a preprint. Self-archived in arXiv, January 9, 2006.
>
>Abstract: The status of an actor in a social context is commonly defined
>in terms of two factors: the total number of endorsements the actor
>receives from other actors and the prestige of the endorsing actors. These
>two factors indicate the distinction between popularity and expert
>appreciation of the actor, respectively. We refer to the former as
>popularity and to the latter as prestige. These notions of popularity and
>prestige also apply to the domain of scholarly assessment. The ISI Impact
>Factor (ISI IF) is defined as the mean number of citations a journal
>receives over a 2 year period. By merely counting the amount of citations
>and disregarding the prestige of the citing journals, the ISI IF is a
>metric of popularity, not of prestige. We demonstrate how a weighted
>version of the popular PageRank algorithm can be used to obtain a metric
>that reflects prestige. We contrast the rankings of journals according to
>their ISI IF and their weighted PageRank, and we provide an analysis that
>reveals both significant overlaps and differences. Furthermore, we
>introduce the Y-factor which is a simple combination of both the ISI IF and
>the weighted PageRank, and find that the resulting journal rankings
>correspond well to a general understanding of journal status.

Pickering and Noruzi articles on citation metrics (via Peter Suber)

Peter Suber forwarded these abstracts of recent articles on the limits (real or imagined) of Web impact factors.

jon

+++

Bobby Pickering, Thomson attacks Scopus Citation Tracker feature, Information World Review, February 17, 2006.
[ http://www.iwr.co.uk/2150515 ] http://www.iwr.co.uk/2150515

Thomson has three objections: (1) that Elsevier only tracks citation data back 10 years, so far, (2) that Elsevier focuses on the natural and social sciences instead of covering all disciplines, and (3) that Elsevier moves away from pre-defined
metrics like impact factors and lets users view citation data in many different ways, including citations per article and citations per author.

My take: Thomson is clearly construing every difference from its own model as a weakness. While the first two are real weaknesses, they're also side-effects of the newness of the service and will likely disappear over time. The third is actually
a strength of the Elsevier model. As citation data become available for many different kinds of processing, Thomson has decided to fight a losing battle: defending the impact factor as the single best perspective on the data, even one that is
necessary to make sense of every other perspective.

Peter

+++

Alireza Noruzi, "The Web Impact Factor : a critical review," The Electronic
Library 24 (2006). Self-archived February 9, 2006.
http://eprints.rclis.org/archive/00005543/

Abstract: We analyse the link-based web site impact measure known as the
Web Impact Factor (WIF). It is a quantitative tool for evaluating and
ranking web sites, top-level domains and sub-domains. We also discuss the
WIF's advantages and disadvantages, data collection problems, and validity
and reliability of WIF results. A key to webometric studies has been the
use of large-scale search engines, such as Yahoo and AltaVista that allow
measurements to be made of the total number of pages in a web site and the
total number of backlinks to the web site. These search engines provide
similar possibilities for the investigation of links between web
sites/pages to those provided by the academic journals citation databases
from the Institute of Scientific Information (ISI). But the content of the
Web is not of the same nature and quality as the databases maintained by
the ISI. This paper reviews how the WIF has been developed and applied. It
has been suggested that Web Impact Factors can be calculated as a way of
comparing the attractiveness of web sites or domains on the Web. It is
concluded that, while the WIF is arguably useful for quantitative
intra-country comparison, application beyond this (i.e., to inter-country
assessment) has little value. The paper attempts to make a critical review
over literature on the WIF and associated indicators.