I. Introduction: This is where readers will get an idea of what your essay is about and what you will prove throughout. Do not give all of your information away here, but give readers a sample of what is to come. Don’t forget to review your writing plan to make sure you are hitting all of the points that you planned out while also stating your argument. A. Provide an overview of the issue you have selected, briefly describing main points and your argument. B. Compose an engaging thesis statement that explains the argument you will prove and support throughout your essay. This statement will give direction to your essay and should be well thought out. II. Body: The body is your opportunity to describe and support your argument in depth. Make sure your thoughts and evidence are clear and organized in a way that is easy for readers to follow and understand. A. Be sure that you write at least three paragraphs that support your key points and are focused, clearly state their intent, and move logically from one to the other, building the thesis argument as the essay progresses. B. Your body paragraphs should support your argument by combining thoughts and ideas with evidence from sources. There is no such thing as a right or wrong argument; the key is how it is supported and the quality of the evidence used. C. Address and refute any opposing viewpoints to your argument. This is your chance to discredit any opposing views, thus strengthening your own. III. Conclusion: Think of the conclusion as a review of your argument. Use this section to restate your argument and remind readers of your supporting evidence. Think of this as your last chance to persuade readers to agree with you.The Organizational Engine of Rankings: Connecting “New” and “Old”
Institutionalism.
Authors: Leopold Ringel, Jelena Brankovic and Tobias Werron
Date: July 2020
From: Politics and Governance(Vol. 8, Issue 2)
Publisher: Cogitatio Press
Document Type: Article
Length: 9,192 words
DOI: http://dx.doi.org/10.17645/pag.v8i2.2576
Abstract:
When explaining the ubiquity of rankings, researchers tend to emphasize macro or contextual phenomena, such
as the power of or the trust in numbers, neoliberal forces, or a general spirit of competition. Meanwhile, the
properties of rankers are rarely, if at all, taken into account. In contrast to the received wisdom, we argue that the
institutionalization of rankings in different fields is also contingent upon another, often-neglected factor: Overtime,
rankers have become increasingly more organized. To investigate the role of ranking organizations, we look into
the distinct properties of present-day rankings and highlight three dimensions along which rankings have evolved
over the course of the twentieth century, namely, publication frequency, handling complex tasks, and audience
engagement. On this basis, we argue that these dimensions have to a large extent been affected by formal
organization and we show how ranking organizations have over time developed capacities to: (a) publish
rankings on a continual basis; (b) handle the often complex production process by means of division of labor; and
(c) generate considerable degrees of attention by addressing large and diverse audiences. On a more general
note, we argue that accounting for the role of organization in the instutionalization of rankings requires a
combination of insights from both “old” and “new” strands of thinking in institutional theory.
Keywords
audiences; institutions; organizations; quantification; rankings
1. Introduction
Higher education, sports, business, human development, and the arts are only some among the growing number of areas
which are nowadays routinely subjected to novel forms of quantified comparisons and monitoring. Among these, rankings
stand out as particularly pervasive and impactful. Be it the Human Development Index, which the United Nations Development
Programme uses to measure the development of nations, TripAdvisor’s Popularity Index listing the best restaurants and hotels
in a city, or the Times Higher Education’s (THE) World University Rankings comparing universities on a worldwide scale–
rankings, it seems, play an increasingly important role in the contemporary world.
The extant literature, however, rarely addresses the problem of why rankings have become so pervasive. Instead, the
effectiveness of rankings is often taken for granted, while little is added to the explanation of the social processes undergirding
their institutionalization. Rather, the literature tends to refer to broader trends in society which rankings are part of, such as the
onset of digitalization (Mau, 2019), the aura of rationality surrounding numbers (Espeland & Stevens, 2008), the
institutionalized trust in numbers (Porter, 1996), the growing importance of performance measurement in governance
(Mehrpouya & Samiolo, 2016), neoliberalism (Munch, 2014), or even a general spirit of competition in mankind (de Rijcke,
Wallenburg, Wouters, & Bal, 2016).
Following our earlier work, we see the institutionalization of rankings as intimately connected to discursive processes unfolding
historically in different fields (Brankovic, Ringel, & Werron, 2018; Ringel & Werron, in press; Werron & Ringel, 2017). In plain
terms, as rankings become increasingly more present in public discourse, be it as objects of praise, criticism or simply “neutral”
reporting, they are more likely to become institutionalized. An integral part of this process is that rankings keep the ranked “on
their toes”: they create pressure for those who are subject to them to strive to not lose their position, maintain it, or improve it
(Espeland & Sauder, 2016; Esposito & Stark, 2019). However, we know only little about if and how rankers themselves add to
this, given that they are routinely overlooked in the extant literature (Rindova, Martins, Srinivas, & Chandler, 2018). So far, we
have learned only little about the properties, practices, and strategies devised by producers of rankings and how they intersect
with or add to the discursive institutionalization.
What is then to be said about those who produce rankings? While for a long time the process has predominantly been
undertaken by individuals who acted in their personal capacity as, for example, scientists or critics, and who typically had only
marginal organizational support, contemporary rankings are usually produced by organizations of various kinds, such as for
profit businesses, newspapers, international governmental and non-governmental organizations, or universities. A closer look
at some of the well-known inventories of rankings and similar devices (e.g., Bandura, 2011; Kelley & Simmons, 2014) reveals
that the most “successful”–that is, regularly published, well-known, and impactful–rankings today are the product of
organizational efforts and were first published usually between the 1980s and early-to-mid 2000s.
To address this, in this article we examine the (largely neglected) role of organizations in the rise and discursive
institutionalization of rankings. Broadly speaking, when it comes to the pervasiveness of organizations in modern society, two
explanations have dominated organizational scholarship throughout the twentieth century (Selznick, 1996; Stinchcombe, 1997).
One body of literature in organizational institutionalism, that is, the so-called “old” one, stresses the capabilities of formal
organizations to complete complex and resource-intensive tasks as opposed to other forms of social organization. In contrast,
the new institutionalist perspective, which emerged partly in response to the old institutionalist tradition, explains the explosion
in numbers of organizations populating the globe by emphasizing that formal organization itself has become a cultural
institution granting legitimacy to those embracing it (Meyer & Bromley, 2013; Meyer & Rowan, 1977). In simplified terms, formal
organizing thus abounds either because it has proven effective or because it is legitimate. Although, by and large, the
legitimacy argument is today considered a superior approach to this old question, we argue that revisiting the tenets of old
institutionalism holds promise for exposing some overlooked elements of the rankings phenomenon.
With these two potential trajectories of explanation in mind, in this article we review empirical studies focusing on the producers
of rankings in a diverse set of fields, particularly, in the arts, science and higher education, education, tourism, and business.
Specifically: in the first step we (a) scan the literature on rankings to identify potential relations between the organizational
expansion and the rise of rankings; and, in the second step, we (b) look for evidence of the effectiveness argument, rather than
that of cultural legitimacy, in the role organizations play in the institutionalization of rankings. By unveiling the properties,
practices, and strategies devised by key actors in discursive arenas, the producers of rankings, we wish to open a new avenue
in the broader research on their global institutionalization (Brankovic et al., 2018; Ringel & Werron, in press).
2. The Rise of Organizations: New Institutionalism and the “Old”
Since the nineteenth century and particularly since the end of World War II, we are witnessing a rampant spread of
organizations across sectors of society (Drori, Meyer, & Hwang, 2006; Meyer & Bromley, 2013). The new institutional theory, in
the tradition of Meyer, Rowan, Powell, DiMaggio, and others (Meyer & Rowan, 1977; Powell & DiMaggio, 1991), argues that
formal organization as such has become a cultural institution embodying and symbolizing the promise of our highest ideals of
rationality. The often described “rationalization” of society over the past decades, that is, the spread of universalized and
abstract cultural beliefs and templates about equality, education, human rights, gender and so forth, thus “creates a framework
that encourages organizing in a wide range of societies and domains” (Meyer & Bromley, 2013, p. 367) beyond actual
functional necessities. These beliefs and templates are to a large degree theorized and legitimized by science–a process that
new institutional theory refers to as “scientization” (Drori et al., 2006). Modern world society is therefore made up of a dense
latticework of scientized norms, values, standards, beliefs, and ideas, all of which are connected to specific visions of
organizational actorhood. Accordingly, studies in this tradition have shown that organizations are often more busy incorporating
such structures so as to adhere to so-called “rationalized myths” in order to be granted legitimacy by their environment.
How do scientized templates about formal organization and organizational governance find their way to the public and, by
extension, trigger the creation of, or change in existing organizations? How are they, as institutional theory would have it,
diffused? The literature suggests two sets of answers. One builds upon the work of DiMaggio and Powell (1983) which
conceives of society as comprising a multitude of organizational fields and outlines three isomorphic processes of diffusing
organizational templates within fields: coercive (organizational structures are diffused via laws or otherwise formally binding
rules based upon sanctions); normative (organizational structures are diffused based upon the belief that they are “right”); and
mimetic (organizational structures are diffused because they are taken for granted). The other draws on Strang and Meyer
(1993) who emphasize that cultural templates often do not travel directly through communicative networks, but are diffused
indirectly, for instance, through observation. They argue that cultural templates are more likely to be spread when they are
theorized: “By theorization we mean the self-conscious development and specification of abstract categories and the
formulation of patterned relationships such as chains of cause and effect” (Strang & Meyer, 1993, p. 492). Building upon these
insights, Werron (2014) suggests defining observers who create such theorizations as “universalized third parties.” Such third
parties, of which the producers of rankings are an example, specialize in publicly observing actors and addressing audiences,
thereby indirectly creating pressure among the said actors to follow the dominant models.
Capabilities of formal organizations, therefore, do not play a major role in the institutionalization of formal organization.
According to the new institutionalist line of argument and evidence, formal organizations, put simply, do not necessarily exist in
such great numbers because they are (more) effective (than other forms of social organization). If and under which
circumstances organizations are or need to be effective in carrying out certain tasks, is then a completely different and, from
the perspective of this tradition, secondary question, or at least one that it empirically rarely explores.
In the large body of literature which attributes the rise of formal organization to its superior capabilities to achieve certain tasks,
for instance when it comes to administering a region or a country, producing goods, and taking care of the sick, three classics
are worth revisiting here. Weber (1915/1947) famously defined organizations as beacons of western rationality due to their
unique properties: They produce written rules of conduct, outline specific spheres of competence, divide labor, have clear-cut
authority structures, employ trained personnel to fill positions, have some degree of discretion in resource allocation, and have
members who typically do not own the means of production but instead receive wages. Barnard (1938) argued that creating
formal organizations becomes a necessity when individuals face too many goals and tasks which are too complex for them to
accomplish on their own. He also stressed the importance of informality as a complementary mode of action and mechanism to
increase support among employees. Coase (1937) was also interested in question as to why individuals feel the necessity to
create formal organizations. He highlighted the transaction costs one has to carry when working towards achieving a goal and
emphasized that there are essentially two options: Services can either be bought on a market or made. In complex situations,
for example, when frequent discussions, deliberations, and negotiations are necessary, or, when products have to be
constantly refined and repaired, seeking vertical integration by means of creating a formal organization is more rational than
buying these services on a market.
In the following decades, a vast body of scholarship critically engaged with these contributions. Nevertheless, they in principle
maintained that formal organizations were a modern institution that, for all their flaws, arguably still did somethings more
effectively than any other form of social organizing and is for this reason an all-pervasive feature of modern society. In public
administration studies, for instance, Simon (1945) notably argued that due to anthropological cognitive constraints, individuals
exceed their analytical capabilities quickly and thus need to be able to cooperate with others in order to compensate for their
imperfections. As a result, organizations are characterized by “bounded rationality”: They do act rationally only to a certain
degree–and more so than any other type of collective actor (Simon, 1957). This, Simon explains, is because organizations
allow individuals to specialize (for example, by observing the environment only in legalistic terms as a company lawyer), which
allows them to put their limited cognitive abilities to better use than if they had to consider a multiplicity of aspects (we could, for
example, imagine a lawyer who also had to be a PR manager and an expert in the production of goods).
The first and second generation of American organizational sociologists, among which are Robert K. Merton, Philipp Selznick,
and Alvin Gouldner, were particularly interested in revisiting Weber’s claim of the superiority of organizations by studying intraorganizational
processes. They unveiled a variety of unintended and sometimes dysfunctional consequences. Nonetheless, as
Selznick (1996) and Stinchcombe (1997) argued, they as well acknowledge that organizations are, to some degree, more
efficient and effective at accomplishing the tasks at hand. They are able to do so by devising formal rules, which can broadly be
defined as “abstractions that govern” (Stinchcombe, 2001, p. 43). Furthermore, Stinchcombe makes the case that institutions
are not self-supporting, emergent structures with a life of their own but in need of support by actors on the ground. Institutions,
in short, can only survive if organizations deliver services promised by the institution and if the quality of these services is
considered acceptable.
Borrowing from the title of Stinchcombe’s book (2001), we might summarize the thrust of this body of literature as follows:
There are indeed times “when formality works.” There is thus a rich variety of contributions to organizational research arguing
for conceptualizing formal organizations as a modern phenomenon that has become institutionalized because it does–at least
some–things more effectively than any other form of social organization. Formal organizations divide labor, standardize
processes, and employ trained specialists (Weber); they coordinate human behavior while retaining autonomy in setting and
changing goals (Barnard); they deliver service which could not be bought as easily on a market (Coase); they compensate the
limited cognitive capacities of individuals (Simon); and they possess the ability to employ distinct mechanisms of governance
(Stinchcombe).
Summing up, according to new institutionalism, we would expect the institutionalization of rankings to lean heavily on the
cultural institutionalization of formal organization. The “old” institutionalism, on the other hand, would see the institutionalization
of rankings as primarily contingent upon the capabilities of formal organizations, which are in their effect superior to other forms
of social organizing. In the following, rather than seeing the two brands as mutually exclusive, we use their insights as
heuristics to analyze the role organizations play in the institutionalization of rankings. In the process, we wish to show that
institutionalism could in fact be practiced as a way of theorizing that incorporates insights from both the “old” and the “new”
tradition.
3. The Expansion of Rankings along Three Dimensions
To examine the role organizations play in the institutionalization of rankings, we reviewed the empirical literature on rankings
with a specific focus on studies examining their producers. Against this backdrop, we then selectively searched for first-hand
information on much discussed and long-lasting rankings in different fields, using publicly available data (published reports,
websites, and secondary sources). We systematically compared the cases and organized our insights into several themes.
Discussing these insights, this section specifies a variety of ways in which organizations matter in making rankings a regular
feature of many fields in modern society. The first dimension refers to the fact that rankings usually address and simultaneously
maintain a problem, thus inducing their regular and continuous publication as opposed to some of the earlier experiments with
rankings which tended to be one-off efforts. The second dimension refers to matters of handling complex tasks, that is, an
evolution from the production of relatively simple lists, simple indicators, and ranking a relatively small number of entities, to
more complex rankings, based on composite indicators and with a larger span. The third dimension, audience engagement,
refers to a transition from relatively narrow expert audiences to larger, more diverse, non-expert or lay ones. We take each of
the three dimensions in turn and selectively draw from examples of rankings that were produced before the ranking-frenzy of
the last thirty years, contrasting them with more recent rankings in which organizations play more significant roles.
3.1. Addressing, Producing, and Maintaining the Problem
Rankings neither appear nor exist in a social vacuum. They frequently address societal problems that are in many cases
perceived as major or even global challenges, such as education, corruption, or climate change. Even national rankings usually
refer to issues that are of universal nature and typically do not touch upon matters that would only make sense in a local
context. On the contrary, they normally draw from (global) templates offered by “universalized third parties” (Werron, 2014)–
particularly international (governmental or nongovernmental) organizations and universities. We might therefore suspect that,
following the widespread theorization of global challenges in the past decades, rankers should find it relatively easy to draw
from existing templates in order to define a problem to which their product should be an answer or a road to one.
However, rankings do not only address existing issues, but at times they also actively work towards creating the very problem
to which, they claim, their product is the remedy. In our earlier work on global university rankings, for example, we
demonstrated that, although such rankings nominally intend to measure something that is supposedly “out there,” in doing so
they discursively construct a distinct notion of scientific “excellence” (Brankovic et al., 2018). In many ways, rankers such as
the Shanghai Ranking or the THE World University Rankings do not just map the existing global field of universities, but
instead they importantly contribute to its emergence as a shared social space.
Whether rankings address existing challenges or actively create new ones, in order for them to survive as rankers, they have to
maintain the “problem.” This, we argue, is contingent upon their ability to keep the operation going. A crucial prerequisite of
doing so is, logically, to secure funding. However, as rankings are produced by a host of different types of organizations, there
is no one clear path for all of them to follow. For-profit organizations, such as the U.S. News or Mercer make rankings part of
their business strategy (for higher education check, for example, Stack, 2016). Many rankings, on the other hand, are produced
by public and non-profit organizations such the OECD, Transparency International, or the World Bank, and depend upon an
often complex and dense network of funding bodies, ranging from government agencies, prominent international organizations,
private foundations and philanthropies, to corporations.
Yet, however ripe the cultural and symbolic infrastructure for the birth of ever new rankings, for rankings to become effective
they also have to be published repeatedly and sometimes even regularly (Brankovic et al., 2018; Werron & Ringel, 2017).
Surprisingly, research often does not take note of the fact that present-day rankings are typically produced weekly, monthly,
annually, or biannually. To pinpoint the struggles such continual publication creates, it is illuminating to contrast modern-day
rankings with their predecessors (Brankovic, Ringel, & Werron, 2019). In fact, many of the early rankings and similar forms of
quantitative evaluation (that is, quantified comparisons sharing some, though not all properties of modern rankings), published
in the pre-1980s area, were one-off experiments by individual experts in their respective fields who had no or, at best, minor
organizational support.
Take the art field for instance: Even though efforts to establish numerical forms of evaluation in the arts can be traced back to
the eighteenth century, they only have become successful since the 1970s (when the periodic art ranking, the German
Kunstkompass, was first published) and particularly in the 2000s (Buckermann, 2019). Spoerhase’s (2018) study on art
rankings in the eighteenth and early nineteenth centuries illustrates this point. He traces quantified comparisons of (mostly)
deceased artists produced by other artists and art critics, starting with the publication of Roger de Piles’ Balance des Peintres
in 1708 and ending with Jean-Francois Sobry’s Balance de Peintres, Rectifee in 1810. In a similar exercise about a century
later, surrealist rankings published in the journal Litterature, edited by surrealist artists Andre Breton, Lois Aragon, and Philippe
Soupault, were also created as singular experiments (Schmidt-Burkhardt, 2005).
Science and higher education provide another instructive example. A review of higher education rankings between 1900 and
1980 reveals that almost all of them had only been published once (Webster, 1986). Among the first to create such league
tables was the psychologist James Cattell, professor at Columbia University and editor of the journal Science (Hammarfelt, de
Rijcke, & Wouters, 2017; Ringel & Werron, in press), who produced a variety of one-off rankings and whose method was
copied by many of his successors. Even in cases when a ranking exercise was repeated, such as those by Raymond Hughes,
published in 1925 and 1934, and the so-called Cartter Report, published in 1966 and 1970 (Webster, 1986), the periods
between the publication of the first and the second report–nine years and four years, respectively–were relatively long and
certainly cannot be considered regular.
What are the reasons for the absence of repetition in the production of these rankings, both in the arts and in the scientific
field? A tentative answer could be their purpose. Spoerhase (2018) suggests that the rankings and similar devices he studied
did not seem to aim at measuring current performances, to be re-assessed at a later stage, but rather resembled what we
today sometimes refer to as “best of all times” lists. After publishing such lists, individual rankers lacked interest in furthering
their assessments. In the scientific field, the previously mentioned Cattell produced rankings as part of his studies on the
origins of “eminence,” as a relatively stable category and he did not seem to show much interest in capturing change as such.
Organizations, on the other hand, have means to “force” their members to have a long-term interest in rankings: Within a “zone
of indifference” (Barnard, 1938), members of an organization can be expected to do as they are told irrespective of their
emotions, preferences, or (changing) interests.
A second reason for the one-off nature of many of these early rankings in different fields seems to be that repeating them
would have been not only costly, but also time-consuming: Producing anything on a regular basis instead of once obviously
has clear implications for the workload to carry and the money to spend–especially in the case of individuals for whom
producing rankings is not their sole or even their primary responsibility. Hammarfelt et al. (2017), for example, detail the
amount of time and energy Cattell had to dedicate to produce one ranking. He first had to scan encyclopedias, make a list of
“eminent” scientists in different fields, prepare a questionnaire, select those who should receive it, send out letters to them with
specific instructions, wait for the letter to return, weigh scores, calculate overall results, and, finally, publish his findings on the
“scientific strength” of universities in the journal Science, which he only could because he himself was the editor. The authors
suggest that Cattell did not repeat this exercise due to a “lack of data.” They further speculate that as the biographical directory
American Men of Science, which was the foundation of his sample, grew in size, Cattell on his own was not able to maintain
the “calculative power” necessary for compiling and reproducing the ranking. They conclude that “(i)t may have been too much
for Cattell” (Hammarfelt et al., 2017, p. 405).
All of this stands in stark contrast with how today’s rankers are able to handle their workload. Besides their superior ability to
retain financial resources mentioned earlier, they often have trained personnel specialized in and paid for the production of
rankings. In their study of rankings in the IT sector, for instance, Pollock and D’Adderio (2012) showcase how contemporary
ranking organizations such as Gartner have an “army” of employees who are responsible for the creation and maintenance of
so-called “magic quadrants.” This indicates that in the case of many contemporary rankings, the employment of those
responsible for the production process is contingent upon the ranking exercise to be continued. In turn, if individuals decide to
leave the organization, they can be replaced by others who continue their work. Take the Corruption Perception Index: Even
after its well-known founder, Johann Graf Lambsdorff, retired, and wanted to retire the Index (Global Integrity, 2009),
Transparency International continued the publication.
Not only employees, but also the ranking organizations themselves may have incentives to commit to continuing the
production. This is especially true when a ranking becomes so successful that the survival of the organization depends on it.
When it produced its first college rankings in the early 1980s, U.S. News, a media company, was in financial trouble. The
publication of its first successful–and still existing and widely read–ranking practically saved the business. In the coming years,
U.S. News became increasingly more invested in the production of rankings (Espeland & Sauder, 2016). In 1990, less than a
decade after its first publication, U.S. News expanded its ranking to include all four-year colleges in the U.S., alongside its
traditional top 25 lists (Jin & Whalley, 2007). Pigeon Paradise, an auction house producing league tables of pigeon races, is
another example of an organization rapidly expanding through rankings. They started out in the 2000s as a club ranking pigeon
races in Central Europe. This soon spurred interest in Asia, which attracted considerable financial resources as buyers sought
the most successful pigeons. The growth which ensued led to the Pigeon Paradise proclaiming itself the “engine of the pigeon
market” (Bahrami & Meyer, 2019). In these cases, two complementary characteristics of organizations seem to reinforce each
other: While the effectiveness of organizations contributes to the regular publication of rankings, once established, the rankings
use the “self-preserving tendencies” of organizations for their long-term institutionalization.
3.2. Handling Complex Tasks
Over time, rankings in different fields have arguably become more elaborate and more complex, which can add to their
perceived significance and legitimacy. Ranking organizations play a crucial role in this development as they can over time
become more efficient in managing the collection, quantification, evaluation, and aggregation of ever more information about
an increasing number of cases. To give some examples: The Programme for International Student Assessment started out as
a ranking of 43 OECD and non-OECD countries and in its last edition in 2015 included 72 countries; the Corruption Perception
Index by Transparency International went from 41 countries in 1995 to 180 countries and territories in 2018; and the website
The data upon which this paper is based were secured
from three different volumes of “Who’s Who,”
the only ones at the time available, in order that
I might profit by the very kind assistance of my friends,
Mr. and Mrs. Marion H. Hedges, without which I fear
I would not have carried out the investigation. (p. 317)
ArtFacts, founded in 2001, constantly increases its sample, currently holding at a staggering number of 601,331 artists.
However, organizations do not only increase the sample of individual rankings, they also often multiply the population. The U.S.
News, on the other hand, did not merely expand its original ranking by including more universities, but also began to “export”
rankings into other domains, such as hospitals, law firms, vacation destinations, and even cars.
In more abstract terms, contemporary rankers often act as what Latour (1987) refers to as “centres of calculation,” that is, they
mobilize, translate, standardize, quantify, and evaluate information by developing large-scale networks. Due to their calculative
power, formal organizations are able to include a large number of ranked entities. Such requirements greatly surpass the
capacities of single individuals, or even small teams. To produce a ranking of colleges in the U.S., Kunkel (1915), for example,
had to rely on his friends:
Such solutions are today, arguably, less common. Furthermore, producing complex rankings which include a large number of
entities usually requires a high degree of expertise and specialization. The personnel of ranking organizations is typically highly
educated, in possession of specialized knowledge, and sometimes receives additional extensive training–thus making up a
large labor force that serves as an important infrastructure for the emergence of ever more rankings. Organizations also rely on
their ability to mobilize the input and cooperation of experts and academics. The Freedom House’s Freedom in the World
Index, for example, assembles “a team of in-house and external analysts and expert advisers from the academic, think tank,
and human rights communities”; “[t]he 2020 edition involved more than 125 analysts, and 40 advisers” (Freedom House,
2020). The main task of the analysts is to evaluate their respective countries of expertise by using “a broad range of sources,
including news articles, academic analyses, reports from nongovernmental organizations, individual professional contacts, and
on-the-ground research” (Freedom House, 2020). The analysts then produce quantified scores, which “are discussed and
defended at a series of review meetings, organized by region and attended by Freedom House staff and a panel of expert
advisers” (Freedom House, 2020). Another example is the World Bank’s Ease of Doing Business Index, which relies on both
in-house and external expertise and in doing so consults thousands of experts who fill out the questionnaire, interacts with
these individuals, and then even does spot-checks in a variety of countries (World Bank Group, 2018). Arguably, only an
organization that is able to manufacture high levels of legitimacy, controls vast resources, and has extensive staffing and global
networks of expertise at its disposal could accomplish a task as complex and resource-intensive.
While centers of calculation still steer the processes and, to some degree, are able change the outcomes (see Pollock &
D’Adderio, 2012), other organizations create and maintain what Kornberger, Pflueger, and Mouritsen (2017) call “evaluative
infrastructures”–a mix of technologies, assemblages, institutional arrangements, cultural rules, norms, habits, and conventions,
allowing for the collection, creation, and procession of large amounts of data. Typical examples are platform organizations
which depend on users’ sharing their experience. In such cases, the organizations provide certain rules and framings (such as
algorithms), but the resulting evaluations, ratings, and rankings emerge in a more or less organic fashion. TripAdvisor, for
instance, has an algorithm that produces its popularity index which ranks hotels and restaurants in different areas, relying on
user review (Jeacle & Carter, 2011; Orlikowski & Scott, 2013). Even though the production of the Popularity Index proceeds
almost automatically as soon as the algorithm is in place, TripAdvisor still has to provide a great deal of maintenance work.
Moreover, operating at such a large scale in some sectors, such as tourism, involves setting up contracts with a variety of
auxiliary organizations, engaging in PR activities, providing help desks and counseling in cases of problems, and dealing with a
variety of legal questions and even lawsuits.
The increasingly complex evaluations and calculations, and not least their expansion, require a certain degree of
standardization, but also distance from the object of ranking. Just like any other, ranking organizations strive to improve their
management, as well as to develop strategies to socialize their members, allowing them to coordinate actions of often large,
diverse, and spatially dispersed groups of people (Ashforth, Sluss, & Harrison, 2007; Maanen & Schein, 1979). Orlikowski and
Scott (2013) detail the formally standardized learning process of newly hired staff at the Automobile Association which
publishes a yearly accommodation guide in Great Britain. The authors describe a well-honed, elaborate, and standardized
process of peer-learning at the end of which newcomers become experienced–and trusted–hotel evaluators. Similarly, in their
study on the Access to Medicine Index, Mehrpouya and Samiolo show how the Access to Medicine Foundation constantly
reminds its analysts to suppress their subjective impressions and feelings and analyze data as a “robot” (Mehrpouya &
Samiolo, 2016, p. 21). With such practices in place, ranking organizations are not only able to standardize the behavior of their
employees, but also to deal with its fluctuation. Their ability to handle ever more complex tasks, then, can be seen as both a
prerequisite and an effect of their involvement in the continual production of rankings.
3.3. Engaging Large and Diverse Audience
The aforementioned examples from the arts and sciences indicate that the early producers of rankings were typically experts in
the field, addressing a small circle of people and expert audiences, usually their professional peers (other artists and scientists,
respectively). The compilers of art (proto)rankings, for example, even deemphasized the validity of their findings (Spoerhase,
2018), signaling that they respected institutionalized structures of connoisseurship at the expense of promoting their easy-tounderstand
ranking tables to mass audiences. Similarly, in the field of science, “early attempts had in common that they
originated from the sciences themselves and, although they claimed to be of relevance for students, their audience mainly
consisted of fellow scholars” (Hammarfelt et al., 2017, p. 406).
Since the 1980s and 1990s, more and more rankings are explicitly produced for non-expert and in other ways diverse
audiences. In many cases, the very meaning of the ranking is based on the idea of transforming expert judgments into
information for broader publics. However, while many accounts emphasize the numerical authority of rankings, empirical
studies also highlight their aesthetic appeal. In their study on Gartner–an IT industry ranking organization–Pollock and
D’Adderio (2012) describe how the creators of Magic Quadrants always keep in mind that a ranking has to deliver a “beautiful
image”–a distribution that makes sense to clients, thus neither including too many nor too few cases, while offering a
meaningful spread. In a similar vein, Mehrpouya and Samiolo (2016) show how ranking organizations strive to create what their
informants call a “good distribution.” In Latoursian terms, ranking organizations invest a great deal of time and resources to
craft rankings as powerful and appealing “inscriptions” spanning and travelling a multiplicity of contexts (Latour, 1987).
In contrast to individual rankers, who seem to focus more on interacting with other experts, ranking organizations often strive to
maintain interest in their product across several audiences, some of which are more and some less competent when it comes
to understanding the complex and multifaceted calculations undergirding rankings. To address various audiences, ranking
organizations either develop hybrid practices, that is, they engage in activities spanning multiple fields (Furnari, 2014), or divide
labor processes, i.e., they set up different departments focusing on technical tasks and institutional environments amounting to
what is often referred to as decoupling (Meyer & Rowan, 1977). Crucially, all of the practices depend on the specific capacities
of formal organizations, particular division of labor in professional expertise, ranging from technical abilities (e.g., statistics,
accounting) and management to communication skills (e.g., scientific, marketing, public relations).
Rankers’ ability to span multiple audiences also allows them to have both the personnel specialized in adhering to a noble
cause (for instance, protecting the environment, or improving healthcare) and engaging in commercial activities to generate
revenue. Their complex structure enables organizations to even use their products to promote certain causes via rankings and
then provide remedial services. Kornberger and Carter (2010) detail how the Anholt City Brand Index ranks global cities to
promote the idea of a “city brand” and then establishes itself as a consultancy to help cities improve their brand. The
aforementioned Pigeon Paradise simultaneously addresses expert audiences (so-called “pigeon fanciers”) and uses its
rankings to increase the valor of pigeons, which it then auctions off to the highest bidder (wealthy investors in Asia). These
examples indicate that owing to their diverse and qualified workforce as well as their ability to coordinate the action of multiple
departments via managerial oversight, organizations are well equipped to expand into a multiplicity of fields. With modern
society comprising exponentially more fields over time (Christin, 2018; Fligstein & McAdam, 2012; Krause, 2018), such
capabilities are in great demand when it comes to the handling of devices such as rankings.
Another important factor for the engagement of large(r) and (more) diverse audiences is that rankers often end up attracting
attention by creating “news.” In contrast, early rankings were rarely set up with this goal in mind. It is therefore little surprising
that rankings which actually gained traction and were published regularly before the 1980s, such as the Kunstkompass, an
artist ranking by German journalist Willi Bongard, actually could rely on organizations to deliver certain tasks such as
marketing. In the case of the Kunstkompass, the journal in which it was published took care of printing, distributing, and
promoting the ranking over the years, allowing Bongard to use his time to engage in the public discourse surrounding his
ranking and to address criticism (Wilbers, 2019). Contemporary rankings, however, are often geared towards maximizing the
attention of a large, lay and in many cases also global audience (Brankovic et al., 2018; see also Kornberger & Carter, 2010).
Not only do they take great care of how they visualize their products, but they usually also employ communication or PR
experts responsible for monitoring, evaluating and engaging with stakeholders. Their tasks include, among others,
disseminating reports, communicating their product in a language which is more accessible to a broader audience, organizing
events, and devising elaborate social media strategies.
The production of rankings that reach beyond narrow circles of experts, especially those addressing global and diverse
audiences, increases the likelihood of criticism. Almost by definition, regularly published rankings are in a continuous battle for
legitimacy and are often fiercely debated. The Corruption Perception Index has been accused of furthering U.S.-American
interests (Gutterman, 2014), while university rankings regularly face pushback (Dorre, Lessenich, & Singe, 2013; Espeland &
Sauder, 2007). Criticism and efforts to avoid it can sometimes affect the way a ranking is made and push the ranker to be
“deliberately less bold” and even–as it was the case with the aforementioned Hughes’ and Cartters’ reports–to completely
refrain from presenting the findings in a rank order (Webster, 1992, p. 252) as a way to “de-emphasize the pecking-order
relationships” (Roose & Andersen, 1970, p. 2).
Some of the most prominent rankers today certainly do not shy away from controversy. To address critics and sceptics, they
often devise elaborate strategies. The earlier mentioned social media activity is one such example, while organizing events is
another. THE, for instance, organizes summits and launches, which they use to also get in touch with critics and/or experts,
“refine” their methodology, and see how they can “do it better” next time (Lim, 2018). Organizations are in general able to
counter criticism quite effectively as they have the means to orchestrate such collective responses by providing its members
who appear in public with “fronts, appearances, manner, routines” (Manning, 2008, p. 680), thus allowing them to promote a
favorable self-presentation (Ringel, 2018), even in spite of backlash.
Another strategy is to involve those who could potentially criticize and even de-legitimize the ranking in its production process.
A classification of journals by the Danish Agency for Science, Technology and Innovation, for instance, was created by
assigning the task of categorizing journals to scholars from the respective fields (Jensen, 2011). The agency thus acted as a
center of calculation by branching out its calculative processes, while at the same time creating barriers against criticism: The
evaluation criteria of the journals, if challenged, would be the responsibility of “the scientific community”–not of the Agency.
Arguably, the ranker, which in this case was a ministry, granted legitimacy and authority to the process–something which
would be far more difficult for an individual to achieve.
4. Concluding Discussion and Outlook
The expansion and effectiveness of rankings is certainly contingent upon their acceptance as a modern rationalized institution.
Research in the framework of new institutionalism has time and again demonstrated that collective actors who are perceived as
legitimate have easier access to resources than those who cannot or do not wish to incorporate institutionalized structures.
Even more fundamentally perhaps, we have to recognize that “organizations can best communicate with other organizations,”
that is, organizations consider other organizations “the only adequate points of contact” (Kuhl, 2015, p. 263). Hence, it is easier
for ranking organizations to acquire funding and establish cooperation compared to other types of actors, such as individuals,
simply because they are organizations. As a result, we might attribute the dominance of organizations among modern rankers
today, to some degree at least, to the symbolic quality of formal organizations as such.
In this article we have argued that the new institutionalist thesis on the global-cultural institutionalization of “organization” could
benefit from the (often forgotten) insights offered by the “old” institutionalism. We, however, do not wish to undermine cultural
rationalization as an important driving force behind global organizational expansion; organizational expansion as such is, after
all, not what we have tried to account for here. Rather, we call attention to the aspects of the expansion or “success” of
contemporary organizational rankers which, we argue, cannot be accounted for exclusively though the new institutionalist lens.
We neither wish to claim that rankings become institutionalized because they are produced by organizations; rather, we argue
that organizational capabilities are an important element in the larger mechanism of this complex process. Finally, we do not
claim that individuals cannot produce and reproduce popular rankings, but those are, to our knowledge, exceptions, rather than
a rule.
To elaborate this point further, and drawing on the empirical evidence mentioned thus far, we suggest that organizations
enable the ongoing production and promotion of rankings in at least three ways. First, plainly put, compared to individuals,
organizations are able to do more things at once. Transparency International, for instance, has a secretariat that is, among
other things, involved in the production of the Corruption Perception Index; it also has national chapters all over the world
engaging in different activities, such as advocacy, networking, engaging with the media, and lobbying. Extreme cases are large
and highly influential international organizations often employing hundreds of people, with the World Bank, the OECD, and the
United Nations being well-known examples. For such organizations, producing a ranking does not necessarily mean setting
other tasks aside. Take the World Bank: While producing and promoting Indexes, such as Doing Business Index and Human
Capital Index, it provides loans to countries, funds all kinds of projects, conducts research, and engages in advocacy.
Second, organizations can do more things at once–for a longer time. They often outlive individuals–even their founders–which
makes them more likely to sustain a long-term interest in the production of rankings. Individuals, on the other hand, not only
eventually pass away, but they also may lose interest in the production of rankings as they become dedicated to other
endeavors. The repeatedly mentioned James Cattell is a prime example: Albeit showing great interest in mapping scientific
“eminence,” he was also involved in the promotion of university reform and anti-war proclamations during World War I, which
eventually consumed much of his time and even put him in the position of losing his tenure at Columbia University. Put
differently, there are limits to what a single person, or even a group for that matter, can do in their lifetime–a restriction that
does not in principle apply to formal organizations.
Third, organizations can do more things at once, for a longer time–and can accumulate more resources, often from a greater
variety of sources. Producing and maintaining a ranking (or several different rankings) is in certain cases extremely resourceintensive.
Individual producers of rankings were therefore either forced to invest their own resources or wait for opportunities to
arise. Cattell, for instance, had the advantage of editing a leading journal in which he could publish more or less at his own will.
Kunkel, as we have seen, relied on friends. Individuals, as opposed to organizations, are also less likely to secure the funding
necessary to acquire expensive instruments and technologies needed to ensure the calculative power for the production of the
rankings they envision.
In conclusion, while in this article we acknowledge that the diffusion of rankings in contemporary society is largely a matter of
discursive institutionalization, we wish to draw attention to the properties, practices, and strategies devised by the actors
responsible for their ongoing production, which have largely remained ignored by rankings research to date. We therefore
argue that formal organization is, in a way, a vital cog in the engine of modern-day rankings. Having reviewed a large body of
empirical studies on rankings, we have identified three dimensions along which rankings have evolved decisively once being
produced by organizations: publication frequency; handling increasingly complex tasks; and audience engagement. We
illustrated how, in contrast to rankings produced by individuals, organizations are better equipped to publish rankings on a
continual basis, handle the increasingly complex production process, generate considerable degrees of attention by addressing
larger and more diverse audiences, and develop mechanisms to respond to their criticism. In short, there is reason to believe
that the organizational production of rankings provides an elementary and hitherto overlooked infrastructure undergirding the
discursive institutionalization of rankings. On a theoretical level, therefore, our analysis suggests that we are well-advised to
reconnect insights on the legitimacy of institutions as promoted by the “new” institutionalism with the “older” institutionalism’s
emphasis on the effectiveness and efficiency of organization in order to make sense of the pervasiveness of institutions such
as rankings.
Submitted: 22 October 2019 | Accepted: 19 December 2019 | Published: 9 April 2020
DOI: 10.17645/pag.v8i2.2576
Acknowledgments
We wish to thank the thematic issue editors and three anonymous reviewers for their comments and suggestions on earlier
versions of the article. All errors are our own.
Conflict of Interests
The authors declare no conflict of interests.
References
Ashforth, B., Sluss, D. M., & Harrison, S. H. (2007). Socialization in organizational contexts. International Review of Industrial
and Organizational Psychology, 2007, 1-70. https://doi.org/10.1002/ 9780470753378.ch1
Bahrami, R., & Meyer, M. (2019). Was motiviert die Rankingproduktion? Das Fallbeispiel der PIPA-Rankings [What motivates
the production of rankings? The example of PIPA]. In L. Ringel & T. Werron (Eds.), Rankings: Soziologische Fallstudien
[Rankings: Sociological case studies] (pp. 1-24). Wiesbaden: Springer VS.
Bandura, R. (2011). Composite indicators and rankings: Inventory 2011 (UNDP/ODS Working Paper). New York, NY: United
Nations Development Programme, Office of Development Studies.
Barnard, C. I. (1938). The functions of the executive. Cambridge, MA: Harvard University Press.
Brankovic, J., Ringel, L., & Werron, T. (2018). How rankings produce competition: The case of global university rankings.
Zeitschrift Fur Soziologie, 47(4), 270-288. https://doi.org/10.1515/zfsoz-2018-0118
Brankovic, J., Ringel, L., & Werron, T. (2019). Theorizing university rankings: A comparative research perspective (Working
Paper SFB 1288, No. 2). Bielefeld: Bielefeld University. Retrieved from https://pub.uni -bielefeld.de/record/2939616
Buckermann, P. (2019). Die Vermessung der Kunstwelt: Quantifizierende Beobachtungen und plurale Ordnungen der Kunst
[Measuring the art world: Quantified observations and the plurality of orders in the arts]. Weilerswist-Metternich: Velbruck
Wissenschaft.
Christin, A. (2018). Counting clicks: Quantification and variation in web journalism in the United States and France. American
Journal of Sociology, 123(5), 1382-1415. https://doi.org/10.1086/696137
Coase, R. H. (1937). The nature of the firm. Economica, 4(16), 386-405. https://doi.org/10.1111/j.1468-0335.1937.tb00002.x
de Rijcke, S., Wallenburg, I., Wouters, P., & Bal, R. (2016). Comparing comparisons: On rankings and accounting in hospitals
and universities. In J. Deville, M. Guggenheim, & Z. Hrdlickova (Eds.), Practising comparison: Logics, relations, collaborations
(pp. 251-280). Manchester: Mattering Press.
DiMaggio, P. J., & Powell, W. W. (1983). The iron cage revisited: Institutional isomorphism and collective rationality in
organizational fields. American Sociological Review, 48(2), 147-160.
Dorre, K., Lessenich, S., & Singe, I. (2013, April 27). German sociologists boycott academic ranking. Global Dialogue:
Magazine of the International Sociological Association, 3(3). Retrieved from http://globaldialogue.isa-sociology.org/german –
sociologists-boycott-academic-ranking
Drori, G. S., Meyer, J. W., & Hwang, H. (Eds.). (2006). Globalization and organization: World society and organizational
change. Oxford: Oxford University Press.
Espeland, W. N., & Sauder, M. (2007). Rankings and reactivity: How public measures recreate social worlds. American Journal
of Sociology, 113(1), 1-40.
Espeland, W. N., & Sauder, M. (2016). Engines of anxiety: Academic rankings, reputation, and accountability. New York, NY:
Russell Sage Foundation.
Espeland, W. N., & Stevens, M. L. (2008). A sociology of quantification. European Journal of Sociology/ Archives Europeennes
de Sociologie, 49(3), 401-436. https://doi.org/10.1017/S0003975609000150
Esposito, E., & Stark, D. (2019). What’s observed in a rating? Rankings as orientation in the face of uncertainty. Theory,
Culture & Society, 36(4), 3-26. https:// doi.org/10.1177/0263276419826276
Fligstein, N., & McAdam, D. (2012). A theory of fields. New York, NY: Oxford University Press.
Freedom House. (2020). Freedom in the world 2020 methodology. Freedom House. Retrieved from
https://freedomhouse.org/reports/freedom-world/ freedom-world-research-methodology
Furnari, S. (2014). Interstitial spaces: Microinteraction settings and the genesis of new practices between institutional fields.
Academy of Management Review, 39(4), 439-462. https://doi.org/10.5465/amr.2012. 0045
Global Integrity. (2009). Johann Lambsdorff retires the corruption perceptions ind gartnerex. Global Integrity. Retrieved from
https://www.globalintegrity. org/2009/09/18/post-452
Gutterman, E. (2014). The legitimacy of transnational NGOs: Lessons from the experience of Transparency International in
Germany and France. Review of International Studies, 40(2), 391-418. https://doi.org/ 10.1017/S0260210513000363
Hammarfelt, B., de Rijcke, S., & Wouters, P. (2017). From eminent men to excellent universities: University rankings as
calculative devices. Minerva, 55(4), 391-411. https://doi.org/10.1007/s11024-0179329-x
Jeacle, I., & Carter, C. (2011). In TripAdvisor we trust: Rankings, calculative regimes and abstract systems. Accounting,
Organizations and Society, 36(4/5), 293-309. https://doi.org/10.1016/ j.aos.2011.04.002
Jensen, C. B. (2011). Making lists, enlisting scientists: The bibliometric indicator, uncertainty and emergent agency. Science &
Technology Studies, 24(2), 64-84.
Jin, G. Z., & Whalley, A. (2007). The power of attention: Do rankings affect the financial resources of public colleges? (Working
Paper No. 12941). https:// doi.org/10.3386/w12941
Kelley, J., & Simmons, B. (2014). The power of performance indicators: Rankings, ratings and reactivity in international
relations (APSA 2014 Annual Meeting Paper). Washington, DC: American Political Science Association. Retrieved from
https://papers. ssrn.com/abstract=2451319
Kornberger, M., & Carter, C. (2010). Manufacturing competition: How accounting practices shape strategy making in cities.
Accounting, Auditing & Accountability Journal, 23(3), 325-349. https://doi.org/10.1108/ 09513571011034325
Kornberger, M., Pflueger, D., & Mouritsen, J. (2017). Evaluative infrastructures: Accounting for platform organization.
Accounting, Organizations and Society, 60, 79-95. https://doi.org/10.1016Zj.aos.2017.05.002
Krause, M. (2018). How fields vary. The British Journal of Sociology, 69(1), 3-22. https://doi.org/10.1111/ 1468-4446.12258
Kuhl, S. (2015). The diffusion of organizations: The role of foreign aid. In B. Holzer, F. Kastner, & T. Werron (Eds.), From
globalization to world society: Neo-institutional and systems-theoretical perspectives (pp. 258-278). New York, NY: Routledge.
Kunkel, B. W. (1915). The distinction of college graduates. School and Society, 2(35), 316-324.
Latour, B. (1987). Science in action: How to follow scientists and engineers through society. Cambridge, MA: Harvard
University Press.
Lim, M. A. (2018). The building of weak expertise: The work of global university rankers. Higher Education, 75(3), 415-430.
https://doi.org/10.1007/ s10734-017-0147-8
Maanen, J. V., & Schein, E. H. (1979). Toward a theory of organizational socialization. Research in Organizational Behavior, 1,
209-264.
Manning, P. K. (2008). Goffman on organizations. Organization Studies, 29(5), 677-699. https://doi.org/
10.1177/0170840608088767
Mau, S. (2019). The metric society: On the quantification of the social (1st ed.). Cambridge: Polity Press.
Mehrpouya, A., & Samiolo, R. (2016). Performance measurement in global governance: Ranking and the politics of variability.
Accounting, Organizations and Society, 55,12-31. https://doi.org/10.1016/j.aos.2016. 09.001
Meyer, J. W., & Bromley, P. (2013). The worldwide expansion of “organization.” Sociological Theory, 31(4), 366-389.
Meyer, J. W., & Rowan, B. (1977). Institutionalized organizations: Formal structure as myth and ceremony. American Journal of
Sociology, 83(2), 340-363.
Munch, R. (2014). Academic capitalism: Universities in the global struggle for excellence. London: Routledge.
Orlikowski, W. J., & Scott, S. V. (2013). What happens when evaluation goes online? Exploring apparatuses of valuation in the
travel sector. Organization Science, 25(3), 868-891. https://doi.org/10.1287/orsc. 2013.0877
Pollock, N., & D’Adderio, L. (2012). Give me a two-by-two matrix and I will create the market: Rankings, graphic visualisations
and sociomateriality. Accounting, Organizations and Society, 37(8), 565-586. https://doi. org/10.1016/j.aos.2012.06.004
Porter, T. M. (1996). Trust in numbers: The pursuit of objectivity in science and public life. Princeton, NJ: Princeton University
Press.
Powell, W. W., & DiMaggio, P. J. (1991). The new institutionalism in organizational analysis. Chicago, IL: University of Chicago
Press.
Rindova, V. P., Martins, L. L., Srinivas, S. B., & Chandler, D. (2018). The good, the bad, and the ugly of organizational
rankings: A multidisciplinary review of the literature and directions for future research. Journal of Management, 44(6),
2175-2208. https://doi.org/ 10.1177/0149206317741962
Ringel, L. (2018). Boundaries of visibility in the age of transparency: An integrative conceptualization. In L. Ringel, P. Hiller, &
C. Zietsma (Eds.), Toward permeable boundaries of organizations? (pp. 55-79). Bingley: Emerald Publishing.
Ringel, L., & Werron, T. (in press). Where do rankings come from? A historical-sociological perspective on the history of
modern rankings. In A. Epple, W. Erhart, & J. Grave (Eds.), Practices of comparing: Ordering and changing the worlds.
Bielefeld: Bielefeld University Press.
Roose, K. D., & Andersen, C. J. (1970). A rating of graduate programs. Washington, DC: American Council on Education.
Schmidt-Burkhardt, A. (2005). Stammbaume der Kunst, Zur Genealogie der Avantgarde [Familiy trees of the arts: The
genealogy of avantgarde]. Berlin and Boston, MA: De Gruyter.
Selznick, P. (1996). Institutionalism ‘old’ and ‘new.’ Administrative Science Quarterly, 41(2), 270-277.
Simon, H. A. (1945). Administrative behavior: A study of decision-making processes in administrative organization. New York,
NY: Free Press.
Simon, H. A. (1957). Models of Man. New York, NY: Wiley.
Spoerhase, C. (2018). Rankings: A pre-history. New Left Review, 114, 99-113.
Stack, M. (2016). Global university rankings and the mediatization of higher education. London: Palgrave Macmillan.
Stinchcombe, A. L. (1997). On the virtues of the old institutionalism. Annual Review of Sociology, 23(1), 1-18.
https://doi.org/10.1146/annurev.soc.23.1.1
Stinchcombe, A. L. (2001). When formality works: Authority and abstraction in law and organizations. Chicago, IL: Chicago
University Press. Retrieved from https://www.press.uchicago.edu/ucp/books/book/ chicago/W/bo3622920.html
Strang, D., & Meyer, J. W. (1993). Institutional conditions for diffusion. Theory and Society, 22(4), 487-511.
https://doi.org/10.1007/BF00993595
Weber, M. (1947). The theory of social and economic organization. New York, NY: Oxford University Press. (Original work
published ca. 1915)
Webster, D. S. (1986). Academic quality rankings of American colleges and universities. Springfield, IL: Charles C. Thomas.
Webster, D. S. (1992). Reputational rankings of colleges and universities, and individual disciplines and fields of study, from
their beginnings to the present. In J. C. Smart (Ed.), Higher education: Handbook of theory and research (Vol 8, pp. 234-304).
New York, NY: Agathon Press.
Werron, T. (2014). On public forms of competition. Cultural Studies-Critical Methodologies, 14(1), 62-76.
https://doi.org/10.1177/1532708613507891
Werron, T., & Ringel, L. (2017). Rankings in a comparative perspective: Conceptual remarks. In S. Lessenich (Ed.),
Geschlossene Gesellschaften: Verhandlungen des 38. Kongresses der Deutschen Gesellschaft fur Soziologie [Closed
societies: Proceeding of the 38th Conference of the German Sociological Association] (pp. 1-10). Essen: DGS.
Wilbers, S. (2019). Grenzarbeit im Kunstbetrieb: Zur Institutionalisierung des Rankings Kunstkompass [Boundary work in the
arts: The institutionalization of the ranking Kunstkompass]. In L. Ringel & T. Werron (Eds.), Rankings: Soziologische
Fallstudien [Rankings: Sociological case studies] (pp. 57-87). Wiesbaden: Springer VS.
World Bank Group. (2018). Doing business 2018: Reforming to create jobs–United States (Working Paper No. 121076).
Washington, DC: The World Bank. Retrieved from http://documents.worldbank.org/ curated/en/252431510213684171/DoingBusiness
-2018-reforming-to-create-jobs-United-States
Leopold Ringel is a Lecturer at the Faculty of Sociology of Bielefeld University, Germany. His current research focuses on the
unintended consequences of organizational transparency and quantification, rankings in particular. He has published in
Organization Studies and Research in the Sociology of Organizations. He holds a PhD from Heinrich-Heine-University
Duesseldorf, Germany.
Jelena Brankovic is a Postdoctoral Researcher at the Faculty of Sociology, Bielefeld University, Germany. Her current research
focuses on the role rankings and other forms of comparisons play in the institutional dynamics within and across sectors, with a
particular attention to the developments at the global level. She holds a PhD in Sociology from Ghent University, Belgium.
Tobias Werron is Professor of Sociological Theory at the Faculty of Sociology, Bielefeld University, Germany, and SFB 1288
Deputy Spokesperson since 2019. His research focuses on globalization, competition, rankings, and violent conflicts.
Leopold Ringel *, Jelena Brankovic and Tobias Werron
Faculty of Sociology, Bielefeld University, 33615 Bielefeld, Germany; E-Mails: leopold.ringel@uni-bielefeld.de (L.R.),
jelena.brankovic@uni-bielefeld.de (J.B.), tobias.werron@uni-bielefeld.de (T.W.)
* Corresponding author
Copyright: COPYRIGHT 2020 Cogitatio Press
http://www.cogitatiopress.com/ojs/index.php/politicsandgovernance
Source Citation
Ringel, Leopold, et al. “The Organizational Engine of Rankings: Connecting ‘New’ and ‘Old’ Institutionalism.” Politics and
Governance, vol. 8, no. 2, 2020, p. 36+. Gale In Context: Opposing Viewpoints,
https://link.gale.com/apps/doc/A628283090/OVIC?u=nhc_main&sid=OVIC&xid=a39e9e7c. Accessed 6 July 2020.
Gale Document Number: GALE|A628283090
Abstract
Background: Partnering with patients in healthcare research now benefits from a strong rationale and is
encouraged by funding agencies and research institutions. However, this new approach raises ethical issues for
patients, researchers, research professionals and administrators. The main objective of this review is to map the
literature related to the ethical issues associated with patient partnership in healthcare research, as well as the
recommendations to address them. Our global aim is to help researchers, patients, research institutions and
research ethics boards reflecting on and dealing with these issues.
Methods: We conducted a scoping review of the ethical issues and recommendations associated with partnering
with patients in healthcare research. After our search strategy, 31 peer reviewed articles published between 2007
and 2017 remained and were analyzed.
Results: We have identified 58 first-order ethical issues and challenges associated with patient partnership in
research, regrouped in 18 second-order ethical themes. Most of the issues are transversal to all phases and stages of
the research process and a lot of them could also apply to patient-partnership in other spheres of health, such as
governance, quality improvement, and education. We suggested that ethical issues and challenges of partnered
research can be related to four ethical frameworks: 1) Research ethics; 2) Research integrity; 3) Organizational ethics,
and 4) Relational ethics.
Conclusions: We have identified numerous ethical issues associated with the recent approach of patientpartnership
in research. These issues are more diverse than the issues associated with a more traditional research
approach. Indeed, the current discussion on how we address ethical issues in research is anchored in the
assumption that patients, as research participants, must be protected from risk. However, doing research with, and
not on, the patient involves changes in the way we reflect on the ethical issues associated with this approach to
research. We propose to broaden the ethical discussion on partnered research to not only rely on a research ethics
framework, but to also frame it within the areas of research integrity, organizational ethics and relational ethics.
Keywords: Scoping review, Ethical issues, Patient partnership, Patient engagement, Healthcare research
© The Author(s). 2020 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License,
which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give
appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if
changes were made. The images or other third party material in this article are included in the article’s Creative Commons
licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons
licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain
permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the
data made available in this article, unless otherwise stated in a credit line to the data.
* Correspondence: joe.trempe-martineau@hec.ca
1Department of Management, HEC Montreal, 3000 chemin de la
Cote-Ste-Catherine, Montreal, QC H3T2A7, Canada
Full list of author information is available at the end of the article
Martineau et al. BMC Medical Ethics (2020) 21:34
https://doi.org/10.1186/s12910-020-0460-0
Background
Patient engagement initiatives in different areas of
healthcare have proliferated in the last decade [1–3]. Patients
are now more commonly integrated in the governance
of healthcare institutions, in quality improvement
initiatives, education and policy making. In clinical settings,
patients are sometimes integrated in multidisciplinary
clinical teams, assisting professionals and
supporting patients in their journey of care, in rehabilitation,
and end-of-life care [3–5]. Lastly, patients are now
more often engaged in healthcare research initiatives,
not as research participants, but as research collaborators,
partners or even as co-researchers [5, 6]. However,
in these various initiatives, and despite all the good will
of the various actors involved, there is still more than
often a gap between the ideal conception of patient partnership
in meaningful and fruitful initiatives and the actual
practices of patient engagement in healthcare.
Patient engagement efforts can sometimes be tainted by
misconceptions, misunderstandings, and frustrations.
Carman et al. [1] describe patient engagement as “patients,
families, their representatives, and health professionals
working in active partnership at various levels
across the health care system […] to improve health and
health care” (p. 224). In the literature, patient engagement
usually refers to working and engaging with patients,
i.e. individuals with first-hand experience of
health issue, or with their caregivers. Carman et al. [1]
also describe different levels of patient participation, and
propose a continuum of patient engagement in healthcare,
ranging from patient consultation, to involvement,
to partnership and shared leadership. In the research
sphere, Marlett et al. [7] define patient engagement as
“collaborative research that is done by, with and for patients
to inform health care and health research decisions
and questions” (p. 1058). Macaulay et al. [8]
describe research partnership as “a mutually respectful
relationship based on sharing responsibilities, costs, and
benefits” (p. 775). Shippee et al. [6] distinguish between
a passive engagement of patient in research, in which
patients are research participants, and an active engagement,
which corresponds to the patients-as-partner view
proposed by Karazivan et al. [2]. In this latter view, patient
engagement initiatives aim at engaging patients
through an “equitable collaboration with individuals,
families and communities affected by a health topic at
all stages of the research process, from conception of the
study idea through dissemination of results/findings” (p.
314) [9].
Engaging with patients in healthcare research now benefits
from a strong ethical rationale, emphasizing that active
participation of patients in research has the potential
to empower patients, to ensure a more responsible research,
and to add value [10], quality, effectiveness, and
validity to the research process, among other potential
benefits [11–13]. However, the ideal or normative conception
of active patient engagement or partnership in research
is often confronted, in practice, with a more
pragmatic view. As mentioned before, full partnership is
intended, in theory, as engagement of patients and coleadership
in all stages of research, but the extent to
which this practice is really implemented in healthcare research
projects can be challenged. Also, authors recognise
that effective patient partnership in research requires additional
efforts when compared with a more traditional research
process [11]. Researchers often feel they are not
well equipped and trained to face these issues [14]. Indeed,
as interest, pressures and incentives to engage patients
in research increase, researchers, patients, members
of research ethics boards, research institutions, funding
agencies, health organizations managers and professionals
encounter more frequently questions and/or ethical dilemmas
related to patient engagement in health research.
Research with, and not on, the patient involves significant
changes in the governance of research projects, in the
composition of research teams, in the relationships between
team members, as well as in the design, conduct
and dissemination of research itself. In many cases, actors
are confronted with the various issues that emerge from
these changes, without any clear direction and support.
The main objective of this paper is thus to map the literature
related to the ethical issues associated with this emergent
practice of partnering with patients in healthcare
research, and to identify the proposed recommendations,
from the literature, to deal with these issues and challenges.
We focus here on patient partnership and shared leadership
in research, which corresponds to the highest level of patient
engagement on Carman et al.’s [1] continuum of patient
engagement in healthcare. A secondary objective is to
identify the “gaps” in existing literature, i.e. unaddressed issues
and missing recommendations requiring future research.
Globally, our aim is to help researchers, patients,
research institutions and research ethics boards, reflecting
on and dealing with these issues. This paper also aims to inform
policy making, and to improve the practice of patient
partnership in healthcare research.
Methods
To conduct our research, we adopted a scoping review
methodology. This methodology has the potential to
capture “key concept; gaps in the research and types and
sources of evidence to inform practice, policy making
and research” (p. 8) [15]. Scoping reviews “can be undertaken
as stand-alone projects in their own right, especially
where an area is complex or has not been
reviewed comprehensively before” (p. 194) [16].
According to Arksey and O’Malley [17], there are four
reasons to undertake a scoping review:
Martineau et al. BMC Medical Ethics (2020) 21:34 Page 2 of 20
1. To examine the extent, range and nature of
research activity in a particular area;
2. To determine the value of undertaking a full
systematic review;
3. To summarise and disseminate research findings; and
4. To identify research gaps in the existing literature.
Reasons 1, 3 and 4 corresponded to our research
objectives.
Our review was conducted following the scoping review
framework proposed by Arksey and O’Malley [17]. This
framework includes 6 stages: 1) Identifying the research
question; 2) Identifying relevant studies; 3) Selecting studies;
4) Charting the data; 5) Collating, summarising, and
reporting the results; and 6) Consulting with relevant
stakeholders. The six stages are reported hereafter.
Stage 1: identifying the research question
As mentioned, the main objectives of our research project
are to map the literature related to the ethical issues
associated with patient partnership in healthcare research,
and to identify the proposed recommendations
to deal with these issues and challenges. Thus, the research
question that guided our bibliographic search
strategy was “What are the ethical issues associated with
patient partnership in healthcare research discussed in
the published literature?”
Stage 2: identifying relevant studies
Two reviewers (AM and JTM) conducted the search and
review between December 2017 and December 2018. In
order to identify the studies relevant to our review, we
interrogated three electronic databases: (1) PubMed
(medical, biomedical and life sciences literature); (2)
CINAHL (nursing, allied health professions, biomedicine
and healthcare literature); (3) EMBASE (biomedical and
pharmacological literature). The research queries were
formulated according to the specific syntax for each
database. We interrogated the title and abstract fields
using keywords related to four concepts: “patient”, “engagement”,
“research”, and “ethics”.
In our preliminary analysis of the literature, we found out
that in some cases the term “community partnership” and
“patient partnership” were used interchangeably, and that
the term “community” was sometimes used to designate
stakeholders that are not usually part of traditional health
research teams, including patients, caregivers or user committee
members. We thus decided to included “community”
as a search term, along with “patient”, butmade sure to only
include in our study selection the studies addressing patient
partnership in research (see Stage 3 of our methodology).
The search terms used are summarized in Table 1. Research
results were saved using the bibliographic management
software EndNote.
Stage 3: selecting studies
Our research included peer reviewed studies published
in English from 2007 to 2017. We initially conducted the
search in English and in French (French is the first language
of the three authors of the paper), however, we
did not identify any article matching our inclusion criteria
in the French literature, so we focused only on the
English results. We focused on this 10-year window,
since our goal was to identify current ethical issues related
to the emergent practice of patients being engaged
as partners in health research initiatives. Patient engagement
in research is indeed a recent approach in healthcare
research, which has become more prominent over
the past 10 years, notably since research funding agencies
around the world have started to support this
approach.1
AM analyzed the title and abstract of each identified
article (n = 187) to determine if it addressed the ethical
issues associated with patient partnership in research.
After removing duplicates (n = 45), errata (n = 1), commentaries
(n = 10), clinical studies (n = 52), editorial (n =
4), thesis (n = 1), poster abstracts (n = 4), erroneously
identified references (n = 1), book chapter (n = 1), as well
as one article with no accessible full text (n = 1), AM analyzed
the full text of all remaining studies (n = 67) to
exclude those that did not address partnership with patients
in research (n = 38), such as in the case of patients
engaged as research participants, or in the case of patients
only engaged as advisors, which does not qualify
as patient partnership, according to Carman et al.
[1] continuum of patient engagement in healthcare
and the patient-as-partner view presented in the background
section [2, 5, 6]. We also added references
identified from hand search (n = 2). At the end of our
1For example, in the US, the PCORI Institute was founded in 2010,
and the National Center for Advancing Translational Science from the
NIH was founded in 2012, and they both supports since 2012 research
initiatives on patient and community engagement. The Strategy for
Patient-Oriented Research (SPOR) from the Canadian Institutes on
Health Research was created in 2011, and since then it supports, and
funds partnered healthcare research in Canada.
Table 1 Search terms
Patient Engagement Research Ethics
Patient* OR
Communit*
Engage* OR Partner* OR Collaborat* OR Commitment* OR Particip* OR Involv* OR
Represent* OR Implicat* OR Adher* OR Centered OR Guided OR Oriented OR Co*author*
OR Co*investigat*
Research* Ethic* OR Moral* OR
Integrit* OR value* OR
model*
Martineau et al. BMC Medical Ethics (2020) 21:34 Page 3 of 20
search strategy, 31 articles remained, which consisted
in our final study sample (final n = 31). A flow chart
of the search results is presented as Fig. 2 in the “Results”
section.
Stage 4: charting the data
After having identified all relevant articles to our review,
we collated bibliometric data to qualify our study sample,
as suggested by Peters et al. [18]. This data is presented
in Table 2, in the “Results” section.
We proceeded with the full text analysis of each article
in our study sample. AM conducted the initial review,
and JTM randomly selected 15 references for a second
independent review in order to verify intercoder agreement.
There was substantial agreement between both reviewers
on the identification of the issues in the
reviewed articles, and the few disagreements were resolved
during one research meeting between AM and
JTM.
All issues, barriers and challenges associated with partnering
with patients in every steps of the research
process were extracted from the selected articles, as well
as their associated recommendations for practice, when
addressed in the sampled articles. These first-order issues
and recommendations correspond to the initial set
of codes in our coding process. In line with our scoping
review approach, we decided at this point to include all
issues, difficulties, barriers and challenges mentioned in
our sampled articles, using a broad understanding of
what can be considered as an ethical issue. For example,
we encountered issues that were deemed at first more
practical, such as the absence of compensation offered
to patient-partners, but we could also see the underlying
ethical aspect to these issues, such as a search for justice
and equity between research team members in the case
of compensating patient-partners. We thus decided on
including all pertinent issues and challenges identified in
our sampled articles.
To organize our results, we used Shippee et al. [6]
framework of phases and stages of patient engagement
in research presented in Fig. 1. As patient partnership
aims to engage with patients in all steps of the research
process, we figured issues and recommendations about
this practice should also be addressed for every research
step.
Stage 5: collating, summarising, and reporting the results
During our review, it was first found that several issues
were transversal to all research phases and stages of research
and were identified as such by the authors of our
reviewed papers. We thus decided to create a separate
result section for those transversal issues (see Table 6 in
the Results section).
To synthetize our results in a conceptually pertinent
way, we regrouped the first-order issues (our initial
codes) in second-order themes of ethical issues (see
first column of Tables 3, 4, 5 and 6), through thematic
analysis [45]. The predominant themes were
elaborated using an inductive and iterative process
through multiple meetings between research team
members (JTM, AM and AB). During this iterative
process, all disagreements between the reviewers concerning
the classification of the first-order issues into
the second-order themes were resolved. In these
meetings, we worked on collating and summarizing
all identified first-order issues in these second-order
themes, organized in the three phases and eight
stages framework of patient engagement in research
proposed by Shippee et al. [6]. The labels of the
second-order themes of issues were also determined
collectively and inductively. Results are discussed and
presented in summary tables in the “Results” section.
We also used a data extraction chart to visually map the
literature, identify recurring issues and recommendations,
as well as to help us identify gaps and unaddressed challenges
and issues. In the chart, we indicated the presence
of all themes of issues in all sampled articles, regrouped in
stages of research. Note that the presence of a theme of
issue, as indicated by a “x” in the chart, does not qualify
the information about this issue, but only the presence of
the issue in the article reviewed, might it be discussed in
Table 2 Description of included studies
Included studies (n = 35)
n (%)
Study type
Research article 19 54%
Review 9 26%
Theoretical 1 3%
Practical application 2 6%
Research methodology
Qualitative 15 43%
Quantitative 0 0%
Mixed 5 14%
Countrya
USA 17 49%
Canada 5 14%
UK 5 14%
France 1 3%
Switzerland 1 3%
Netherlands 2 6%
Authorship
Patient as co-author 2 6%
Researcher(s) only as authors 29 83%
aAccording to first author affiliation
Martineau et al. BMC Medical Ethics (2020) 21:34 Page 4 of 20
depth, or only in passage. The extraction chart is provided
in Additional file 1.
Stage 6: consultation exercise
Although the consultation stage is optional in Arksey
and O’Malley’s [17] framework, we did integrate a
consultation exercise with a committee composed of
stakeholders and practitioners with expertise on patient
partnership in research. According to Arksey & O’Malley
[17], the consultation exercise is an “additional, parallel
element” of the scoping study framework that can
“inform and validate findings from the main scoping
Fig. 1 Framework of phases and stages of patient engagement in research, adapted from Shippee et al. 2015, p. 1157
Fig. 2 Flow chart of search strategy and search results
Martineau et al. BMC Medical Ethics (2020) 21:34 Page 5 of 20
Table 3 Issues associated with partnering with patients in the Preparatory phase of research
Themes Issues identified in the literature Recommendations identified in the literature
Research step: Agenda setting and funding
Lack of resources available to
support partnership in the
early phase of research
• Researchers may lack financial resources to support full
partnership with patients in the early phase of
research, such as when determining the research
question, before they obtain funding [19–22].
• New funding mechanisms that would recognise the
particularities of patient engagement in the phase of
research questions development should be created
[19].
• Future studies should consider including a full analysis
of the resources required for partnering with patients
in their research project [20].
• Researcher might not have enough time to engage in
full patient partnership at this early stage of research
[19, 21].
• In order to cope with this lack of time and resources,
researchers can turn to social media for
crowdsourcing, or to investigate community interest
for different avenues of research [23].
Disagreement on priorities • Confrontation may arise between researchers and
patients when prioritizing research questions or
selecting outcomes [24–26].
• Research partners should recognise the difference of
objectives between researchers and patients as an
opportunity to converge them into synergistic goals
[24].
• Research teams should make sure the research
question is relevant to all research partners equally,
including researchers and patient [27].
• Different patients may want different research
questions to be answered and different outcomes to
be measured [11].
• Researchers might select patient partners who are
connected with a greater patient community (e.g.,
through advocacy organizations) and who are able to
discuss broad concerns of interest for diverse patients,
not just for themselves or their special interests [11].
Tokenism • Challenges for researchers arise when they are faced
with a predetermined research topic by funding
organizations, while at the same time they are
required to implicate patients in the process of
research question prioritization. This situation may lead
to patient engagement tokenism and inhibit effective
implication of patient in the earlier step of research
[26].
• Engaging patient in the early stage of the research
question development is still uncommon. The
participation of patient in this stage of research is
commonly limited to consultation rather than full
partnership [19].
• Researchers believe that patients aren’t knowledgeable
enough to be able to choose research priorities [28].
• Researcher believe that patients lack the expertise to
be able to gain global understanding of the whole
study [21].
• Partners may face long periods of inactivity due to the
time required for review after protocol is submitted.
Partners who are not appropriately informed about
the delays may lose motivation and experience
frustration [22].
• Researchers could engage with patients only after
funding is secured [22].
Patients’ conflict of interest • During the step of topic solicitation and research
question development, challenges may occur when
patients focus on their own current health issues and
interest while discussing research priorities with
researchers [26].
• Patients can anticipate personal benefits as incentive
for participating in research, or research ethics
committee, which might make them overlook risks for
other participants and themselves in the ethical
evaluation of the project, in hope of these benefits
[21].
• Researchers can encourage patient-research-partners
to adopt a broad perspective when expressing their
opinion [11, 19, 24, 27].
Science is the pursuit and application of knowledge and understanding of the natural and social…
Clearly stating the definition, the values, the meaning of such values and the type of…
All answered must be typed using Times New Roman (size 12, double-spaced) font. No pictures…
All answered must be typed using Times New Roman (size 12, double-spaced) font. No pictures…
https://www.npr.org/sections/ed/2018/04/25/605092520/high-paying-trade-jobs-sit-empty-while-high-school-grads-line-up-for-university Click on the link above. Read the entire link and answer the questions below…
All answered must be typed using Times New Roman (size 12, double-spaced) font. No pictures…