In making legal judgements, Dutch courts regularly use so-called ‘societal views’. For example, within criminal law ‘socio-ethical acceptability’ is used as a measure of whether sexual contact between minors is a crime or not. Within private law ‘what is socially acceptable’ is used as a criterion in tort law, and at the intersection of liability law and corporate law, societal views play a role in the question of whether a natural person’s conduct can be imputed to the legal entity. However, although judges rely on views in society when making certain judgments, they rarely mention how or where they found those societal views, nor whether and how they investigated what exactly those societal views are. This raises the question of whether those societal views used for legal judgement correspond to the actual views held in society. This led a group of researchers, all associated with the Utrecht Centre for Accountability and Liability Law (UCALL) of Utrecht University the Netherlands, to carry out an empirical legal study to compare the societal views used in the judiciary with the views in society, using a survey. Based on this case example, this blog post aims to show the usefulness of survey research to test legal judgment. First, it describes the methodology and the rationale of using a survey, then provides the survey results and finally discusses the usefulness of this method for answering this and similar research questions.
Methods: A survey
To answer the empirical question of whether societal attitudes prevalent in the judiciary and referred to in judicial decisions correspond to actual attitudes in society, a survey was carried out under the general public. The rationale for this method is that a survey makes it possible to collect data within a large group of participants, which is necessary when trying to obtain a representative sample from society. To achieve this, the survey was conducted among a fixed opinion panel of 1,058 people forming a representative stratified sample of the Dutch population, with the help of an external commercial party.
Apart from general descriptive questions such as age and place of residence, the questionnaire addressed societal views on seven specific themes: a) criminal liability for consensual sex between juveniles; b) imputing a natural person’s conduct to a legal entity; c) the extent to which police should compensate for damages caused by raids; d) the mutual liability of co-owners of an animal or premises; e) the possibility of cancelling a contract under pressure of US economic sanctions; f) liability for the improper use of measurement instructions by estate agents; and, g) the liability of the healthcare provider for the use of an unsuitable medical device. For each of these topics, a brief introduction followed by different statements were presented to the opinion panel. The propositions presented were based on previous case law and/or literature on society’s views held by judges. Examples include: ‘A hospital must pay for the damage caused by the use of unsafe implants, even if the hospital is not insured for that damage’; ‘If a dog bites one of its owners, the other owners of the dog must also contribute to the costs incurred by the injured owner’; and ‘If a mechanic of a garage company pours motor oil into a ditch, polluting that ditch, the garage company will be responsible for it’.
For each statement, survey participants could indicate to what extent they (dis)agreed on a 7-point Likert scale ranging from ‘strongly disagree’ (1) to ‘strongly agree’ (7), plus the option I don’t understand (8). Questions on which 8 was scored were not included in the analysis. The data were analysed using SPSS, version 27. Per theme, a one-way repeated measures ANOVA (analysis of variance) was performed to investigate whether the questions within a given theme were answered differently from each other. This test can be used to compare participants’ answers to different questions on the same scale. If the ANOVA showed a significant difference in mean scores on the questions, post hoc testing was then performed to gain more insight into exactly which questions differed from each other. To reduce the likelihood of type-I errors (false positives), Bonferroni correction was applied here. Paired t-tests were then performed for the significant pairs. These tests compared the mean scores on 2 questions and provided more clarity on the nature of the differences than the post hoc test of the ANOVA. For some themes, mixed measures ANOVAs were then also conducted to see whether certain between-subject factors (such as gender or political affiliation) influenced the answers to the questions (within-subject factor).
Results of the survey
All responses were analysed to get a clearer picture of society’s views on each theme. The emphasis in analysing the data was on descriptive statistics (e.g., how were the statements answered and what is the opinion of respondents on the statements?). Findings were visualized and statistical differences made explicit in the text. See figure 1 for an example of a visualization of the distribution of answers on one of the questions within theme a) criminal liability for consensual sex between juveniles.
Figure 1. Results on question (theme a): sexual freedom of young people (aged 12 to 16) may be restricted by criminal courts.
Where relevant and possible, further analyses were also carried out, e.g. to investigate significant differences in opinion on certain statements within a theme or to be able to compare the opinion of certain groups (people who disagree with a certain statement versus people who agree with it, male versus female, old versus young, etc.). Within theme a) on consensual sex between minors there was a difference in public opinion on punishability of this behaviour based on initiative and consent (question 2), age difference (question 3), the presence of a relationship (question 4) and the nature of the sexual acts (question 5). What is more, men and women differed on three of the five questions, with women being more in favour of restriction of youth sexuality by criminal courts (see Figure 2, with * indicating significant differences).
Figure 2. Mean results on all 5 questions within theme a) criminal liability for consensual sex between juveniles, differentiated by gender.
These and other survey results were then compared with the views used by the court within the specific areas of the law. It was concluded that the judiciary often has a good grasp of societal views, and that many views are widely held within society which underpins their legal use. However, there are also discrepancies where the judiciary seems to deviate from the opinion of society. The debate was raised whether in such cases society’s view is the right criterion on which to base a judgment.
Limitations and strenghts
The chosen format entails some limitations, for instance, the introductions and questions have been stylised and adapted (simplified) in terms of language. And the operationalisation of the themes was rather basic, making the statements on which the respondents had to give their opinion much less informative than the real issues underlying them. Respondents received only little information. Moreover, we did not ask or receive any justification for the choices made, and these choices were always made rather quickly, at least not after much deliberation, which can obviously influence the results. A more qualitative approach, for example by conducting interviews or focus groups, could have remedied this. But this requires a much greater effort in terms of time especially. So, despite these limitations on the depth of the knowledge gained, we do know more than before conducting the survey.
The strength of this study and others alike lies in the large, representative number of respondents. Not a small group of experts, but a representative sample from society gave the results. And if a judge wants to know what society thinks to use precisely that view in their judgement, that is the domain where you should look for answers. So, in a sense, this study resulted in an opinion poll on seven issues which the judiciary can use to its advantage (by sticking to the chosen line or perhaps seeing an opening to choose a different route). A more general strength of this methodology is that it encourages interdisciplinary collaboration among different fields of law because it allows the questioning of different themes within one survey.
In this study, ELS contributed to the legal debate on the use of societal views in law by showing discrepancies between assumed and actual views.
This blog post is based on the book Opvattingen onderzoeken: Een verkennend juridisch-empirisch onderzoek naar het gebruik van maatschappelijke opvattingen in de Nederlandse rechtspraak (Examining views: An exploratory legal-empirical study of the use of societal views in Dutch jurisprudence), De Jong, Dalhuisen, Bouwman & Giesen (eds.), 2021, Den Haag: Boom Juridisch.
Distributed under the Creative Commons License CC BY 4.0
