Scholar Indices and Impact

Scholar indices are used to measure the contributions of scholars to their fields of research. Since the 2005 paper of Jorge E. Hirsch,^[1] the use of scholar indices has increased.

Definition

Sometimes called bibliometrics, scholar indices are mathematical and statistical tools that measure the significance of the contributions made by an academic to their field of research. Scholar indices may incorporate other assessments such as citation tracking and journal ranking.

Creation

Any aggregator of citations and references, could, given time, money and inclination, generate their own set of scholar indices. Publishers who are prominent in this field include Elsevier and Thomson Reuters.

Commercial software which use parsers and web search engines to generate sets of scholar indices or individual results are now available. Examples are: Publish or Perish;^[2] 'ScholarIndex';^[3] 'Scopus'^[4] and 'Google Scholar'.^[5]

Each software vendor primarily uses its own data as well as journals, publications, authority files, indexes, and subject categories to produce sets of scholar indices.

While some companies provide the data and the evaluated metrics as free downloads, others require subscriptions to cover costs of manufacture and upkeep of an efficient parser, search engine and document database.

Use

Scholar indices allow choice of journal collections, application of research funds, ranking of journals and the determination of significant contributors in a subject area.^{[6] [7]}

Advocates of scholar indices recommend their use in areas such as liaison services, references, instruction and collection management.^[8]

Critics of the use of scholar indices cite their limitations due to issues of accuracy, validity and applicability and debate their application to hiring, tenure, funding, award giving and membership decisions.

Although scholar indices may not completely describe the impact of an individual researcher's work, some academics will determine their own scholar indices to include in promotional material and curriculum vita for example. Others may study their scholar indices simply for their own sake.

Those interested in the field of scholar indices may find the results, for example data visualisation projects, exciting.

Types

To date, a number of scholar indices have been developed. One is the 'h-index' introduced by Jorge E. Hirsch in August 2005. Hirsch described the h-index as unbiased as it involved the relationship of an academic's volume of published papers and the number of citations for those papers creating less bias than either measure alone.

Another scholar index is the 'g-index' which measures citation counts of all well cited works over a period of time. The 'm-quotient' was developed to introduce a time limit to the h-index which was otherwise, an ever-increasing quantity.

Other variants of the h-index such as hI-index, e-index ^[9] and others are being developed and reviewed.

The Erdős number ^[10] was developed to measure the publication chain started by Paul Erdős.

All such scholar indices quantify the contribution of a researcher based on citations of their work only. Ideally, an assessment of a researcher's contribution to their field would include both scholar indices and an analysis of the quality of the work itself.

Hubs and Authority indices

Evaluation of the complete contribution of a scholar to their field of research can be assessed in two ways. One is by accounting for the number of citations received by the scholar. The other is by accounting for the quality of the references referred to by the scholar.

While being strongly cited makes a scholar a strong authority in his field, having strong references makes a scholar a strong hub, who knows all the significant work in that field. Calculation of the hub and authority indices requires the knowledge of the relationships between scholars being cited or referred to.

The Hubs and Authorities algorithm can be used for computing these indices. The algorithm performs a link analysis on a given network and assigns two scores to each node: a hub and an authority.;^[13][14]

A valuable and informative node in a network is usually pointed to by a large number of links, that is, it has a large indegree (see Fig. 1). Such a node is called an authority.^[15]

A node that points to many authority nodes is itself a useful resource and is called a hub. A hub usually has a large outdegree. In the context of literature citation, a hub is a review paper which cites many original papers, while an authority is an original seminal paper which is cited by many papers.^[16]

Fig. 1. While considering the connectivity of the green node S, the nodes y indicate the in-degree and nodes in x indicate the out-degree of node S.

A network can be constructed of nodes representing authors and links indicating references to published papers. Outgoing links indicate who the author cited and incoming links indicate who cited the author.

A researcher's hub score is the sum of authors' scores whose work is cited. A researcher's authority score is the sum of authors' hub scores who referenced the researcher's work.

The hub score increases if the author cites papers published by authors with high authority scores. The authority score increases when published papers are cited by authors with high hubs score.

The equations can be rewritten in a matrix-vector form. Let A be an adjacency matrix of the network and vectors h and a to contain all hubs and authorities scores, respectively. Then the scores can be calculated by the following formulas.^[17]

Hubs and authority index algorithm implementation

Hubs and authorities indices require the knowledge of the connectivity between scholars who refer and cite each other’s work. Since it is not always possible to accurately obtain these connectivity patterns, the adjacency matrix (A) regarding the scholar’s connections can be estimated.

For example, a scientist has an estimated local connectivity matrix. It is a combination of the work by which he is cited and works which he cites. Once the adjacency network is estimated, the hub and authority indices are determined by eigen-decomposition of (A.A’) and (A’.A) respectively. The steps followed for this specific implementation are as follows:

• Step 1: Let the scholar to be evaluated be S. This scholar is searched in Scopus and the total number of citations he has received= c, and the total number of references he has used=r.
• Step 2: A list of scholars who have cited the works of S are recorded as a set Y. The list of scholars who have been referred by S are recorded as X. The choice of members of X and Y are made in the order of decreasing citations in Scopus. Since an estimate of adjacency matrix A was sufficient, thus, we recorded members of X, and Y with citations greater than 100 in Scopus. Let the number of significant citations = c_s , and the number of significant references = r_s . For each significant member of X and Y, one evaluates the ratio of (the number of citations/number of references) as weights of matrix A.
• Step 3: Once the weights corresponding to the significant members of X and Y are calculated, the weights of not-so-significant members are estimated randomly. Thus, random weights are generated for c − c_s citations for members of Y and r − r_s references for members of X. The matrix A is thus generated as:

Where blocks of X and blocks of Y replace the ‘1’s. The connectivity follows the Fig 2.

Fig. 2. This image depicts the connections between S, X, Y for generating adjacency matrix A. Only S is connected to X and Y is connected to S, all other connections are 0. A is a unidirectional adjacency matrix.

• Step 4: The initial hub vector is calculated as primary eigen-vector (with the highest eigen-value) of A.A’ and normalized between [0,1]. The initial authority vector is the primary eigen-vector of A’.A and normalized too. Since these indices are interrelated, thus, the hub vector becomes A*initial authority vector, and the authority vector becomes A’*initial hub vector. Since the scientist ‘S’ was the first entry of matrix A, hence the first value of the hub and authority vectors are the hub-authority indices for scientist S. The pseudocode for the same is as follows is S is a scholar whose hub index is h and authority index is a.

Example 1: Scholar indices of well-known scientists

The table utilizes Scopus as a search engine only and the adjacency matrix is an estimation, thus the results tabulated above are extremely aggregated versions, and they must not be confused with absolute indices. A better estimate of adjacency matrix may produce variations in the indices. Also, Scopus keeps track of articles after 1995 only, so that is an additional constraint. (All the indices have been evaluated as of December 12, 2011.)

In this table, it is evident that different search engines yield different h-indices. It is possible that a scientist with high h-index may be a strong authority but not necessarily a strong hub. The validity of web-search engines is assessed as documents prior to 1995 are inaccessible. The number of publications of a particular author in a particular data base is responsible for affecting the hub-authority indices. Interdisciplinary work may be well assessed by the hub-authority index as opposed to other indices.

Example 2: Projected hub-authority indices with time (Toy example)

This is an example to understand the interplay of the various scholar indices.

A new scientist who began her academic career in 2009, has published 3 papers. Two papers have 2 citations each while the third paper has no citations. She has referenced 60 papers with 17 strong references among the 60. One of her co-authors has the lowest Erdős number 3. Her scholar indices as per December 2011 are:

• h-index= 2
• g-index=2
• m-quotient= 2/2=1
• Erdős number=3+1=4
• HUB index (per 10^2)= 11.702
• AUTHORITY index= 0.1061

In another year, she publishes another paper with 20 new references such that now she has a cumulative of 31 strong references, 4 papers with 2 citations for 2 papers and 0 citations for other 2 papers. Her hub-authority indices change:

HUB index=12.668

AUTHORITY index= 0.1061

In another year, her citations for the 3 papers increases to 10 and she continues to have 60 references with 17 strong references:

HUB index=11.568

AUTHORITY index=0.3241

In another year, her citations increase to 10 and she writes another paper such that the number of references goes up to 80 (with 31 strong ones):

HUB index=12.694

AUTHORITY index=0.3284

Thus to summarize, the following hub-authority indices are observed for this toy example.

The matlab code for the Example 1 and Toy example instances in Example 2 is attached as File 1.

File 1. Sample MATLAB code for hub-authority indices for famous scientists and toy example in Example 1 and Example 2.

These examples demonstrate the importance of the hub-authority indices in quantitative evaluation of the contributions of a scholar in a complete academic career.

Criticism

Scholar indices have limitations including lack of accuracy, validity and applicability. While the accuracy of scholar indices is questionable owing to the difference in spellings, difference in parser, search engines and document data bases maintained by various online sources, it might be possible to solve the accuracy issues if each author could be assigned to a unique ID instead of relying on the names for searches. Also each time these indices are reported, the method and search engines used must be mentioned to avoid ambiguity as much as possible.

The validity of scholar indices is a limitation because they do not weigh fields of varying importance. For example, John Pople, a theoretical chemist who has received a Nobel Prize, fares poorly in sets of h-indices.

Applicability of scholar indices has limitations when scholars emphasise practical advancement in an area of endeavour rather than the publishing of papers. It is also difficult to document works of an earlier decade as online documents, thereby decreasing their online impact factor. For example, the Scopus is an online database that calculates scholar indices for documents found after 1995 only. Any earlier works are not documented or evaluated.

Jorge E. Hirsch suggested the h-index should inform hiring, promotion, funding, tenure, award and societal committees to make judicious decisions. However, because of their limitations they are best viewed in a balanced way.

References