How to measure the reputation of bureaucracies

The behavior of administrative authorities largely depends on the bureaucratic prestige that units and departments enjoy among different audiences. However, the current scholarship has not yet produced a coherent measure of bureaucratic reputation applicable to multiple agencies in different countries and over time. In his most recent essay, published in American Journal of Political ScienceBocconi Post-Doc and DONDENA researcher Luca Bellodi addressed this gap by developing a new strategy for calculating bureaucratic credibility using cutting-edge techniques in the field of natural language processing.

The paper uses a technique called word embedding, which calculates the probability that two or more groups of words will appear in the same linguistic context. In this case, the authors are interested in calculating the probability of an agency’s name being placed near words symbolizing high or low reputation and efficiency. This exercise continued using the names of 465 US and UK bureaucratic bodies mentioned in the parliamentary speeches of members of Congress and the UK Parliament for over 40 years.

The results of this strategy then passed additional validation tests and proved robust against other text corpora and alternative measures. In particular, the author was able to identify ups and downs in credibility that occur in the presence of a crisis such as that experienced by the UK Home Office in the wake of the Windrush scandal. The paper also notes that the US Environmental Protection Agency has suffered a steady decline over the past 30 years, while the UK Department of Health has improved its status over the same period.

The results suggest that members of the ruling party tend to praise bureaucratic institutions more than the opposition. At the same time, this difference decreases the more independent the agencies mentioned are.

In fact, it can be seen that the reputation index varies depending on the speaker’s political affiliation. Polarization can be observed in relation to how an MP talks about a particular authority depending on their party affiliation, and it appears to be consistently higher in the US than in the UK. Parties in power tend to speak better about government agencies overall, especially when the agency is under tighter government scrutiny. This indicates that political affiliation is an important component in forming beliefs about the work of the bureaucracy. Evidence of this is that more independent agencies come with significantly less polarization.

The development of a new measure of reputation, applicable to different political and institutional settings, can help answer several research questions and advance our understanding of bureaucracies in a comparative perspective. Agencies with different levels of credibility may have different patterns of external communication and outreach. Likewise, a positive reputation can allow them to generate more support, recruit more qualified personnel, and better protect themselves against political attacks. There is clearly a trade-off between nuance and coverage. Policies applicable to different environments inevitably lose the minute specifics of each context. However, by focusing on the references MEPs make to bureaucratic authorities in their speeches, the authors can draw on an informed and diverse audience, which should reflect the views of key stakeholders.

Provided by Bocconi University