Measurement

Concepts in the human sciences (humanities, social sciences, life sciences) are all what philosophers call "thick": they are descriptive and evaluative at the same time. As a result, they are "Ballung" concepts, with multiple, sometimes overlapping, meanings and fuzzy boundaries (the word is German and roughly means "clustered"). Most importantly, they are "essentially contested": there is no way to settle on a single, universally-accepted technical meaning for such concepts, because different meanings embody different valuations and different purposes. This makes measurement in the human sciences fraught.

But if the problem with such concepts is their value-ladenness, then what if we were to come up with measures of the values at stake in them? Could we organize their essential contests in a way that might guide the choice of multiple indicators? Because we do actually have experience, as a scientific community, with the measurement of values!

Click for a larger version.

The earliest value research happened under the label of "attitudes" and such work continues today.

In the 1950s, surveys began to ask questions about "ideology" in the context of vote choices and we have all been inundated with such talk every since. One spin-off of this is the discussion of so-called "media bias" - the actual or perceived political positioning of different media sources.

The graph to the left displays the position of various major Canadian news media, based on responses from 235 experts in that area. Media are positioned at their average rating on both economic and social issues, and are coloured based on their perceived closeness to Canada's federal political parties. The skew to the right will be obvious.

Major cross-national value surveys started in the 1970s, with the largest associated with the names of Geert Hofstede (succeeded by Michael Minkov), Shalom Schwartz, and Ronald Inglehart (succeeded by Christian Welzel).

Inglehart's World Values Survey and Schwartz's European Values Study ask respondents, amongst many other things, which qualities from a list of eleven it is especially important that children learn at home. The average rating of each quality differs across countries. What's more, qualities tend to bundle together. As a result, we can summarize most the variation between countries with only two dimensions: one contrasting independence with obedience and faith, one contrasting hard work with tolerance and respect for others.

Click for a larger version.

But how adequate are those eleven qualities? The surveys themselves have vacillated, trying additional items. Though there is reason to think that some large patterns may be fairly stable, there is also good reason to suspect that much more is going on. How much difference would it make if respondents were asked about 20 qualities? or 30? Or if they were asked about specific child or parenting behaviours instead? Or if they were asked to think about specific children - boys, girls, first-born, older, younger, other people's, children from stigmatized groups?

Surveys are not necessarily the best way to elicit people's values, even on relatively simple questions like whether their government should spend more, about the same, or less on health care, much less on questions like how children should be raised or how do you judge if your child is thriving. There are many alternatives. Computational text analysis allows the exploration of not just eleven words or phrases, but hundreds of thousands. Will it lead us to a richer picture than the analysis of survey results? Or will it only confirm the picture we already have? Time will tell!

Policy