Section 1: Background to these statistics
The Sentencing Council was set up in 2010 and produces guidelines for use by all criminal courts in England and Wales. The Sentencing Council has a statutory duty under the Coroners and Justice Act 2009 to monitor the operation and effect of its sentencing guidelines and to draw conclusions from this information. In order to help evaluate these guidelines, the Council conducts bespoke data collection exercises, ideally for several months pre and post guideline, in order to gather detailed information from sentencers about how they sentenced the offences covered by the guideline.
Since November 2017, this type of data collection exercise has involved asking sentencers to complete an online form, hosted on the Council’s website, for every adult offender they sentence for the offence in question, where it was the principal offence. The forms ask sentencers to give detailed sentence information which may vary slightly by collection and offence but typically include the culpability and harm factors relevant to their sentencing decision, the sentence starting point, any aggravating and mitigating factors (including whether there were considered to be relevant and recent previous convictions), information on the stage of any guilty plea entered and its impact on sentence and detailed information on the final sentence outcome. Sentencers are also given an opportunity to state the single most important factor they took into account when deciding on their sentence outcome.
Prior to the inception of these bespoke offence specific data collections, the Council ran the census style Crown Court Sentencing Survey (CCSS). This was a rich source of detailed sentencing data, providing a wealth of information on sentencing for a wide range of offences sentenced specifically within the Crown Court. However, following an external review, the CCSS was stopped at the end of March 2015, and the Council evolved its analytical approach to develop more focussed and targeted ‘guideline specific’ data collections in both magistrates’ courts and the Crown Court.
While there is often detailed analysis of the bespoke data collection exercises within the relevant guideline evaluation, it is hoped that publication of the raw underlying data will be useful, adding to the knowledge base to better understand sentencing factors in relation to outcomes. Publication of these data also falls within the Council’s strategic objectives for 2021 to 2026 to ensure that the Council’s work is evidence-based, and to work to enhance and strengthen the data and evidence that underpins it.
When considering the data, it is important to keep in mind that every case is unique and there are many factors, both relating to the offence and the offender’s personal circumstances that will be taken into account when deciding on the appropriate sentence. Therefore, there may be factors other than those collected on the form and detailed in the data that impact on the final sentence. Furthermore, while the same factors may be present in more than one case, the specific circumstances of each case may mean that the factors are not given the same importance in all cases which may, in turn, be reflected in the decision regarding an appropriate sentence for the offender in question.
This document is intended to be read alongside the raw data, so that its users can better understand the overall quality.
Section 2: Assessment of quality
i. Relevance
Relevance is about making sure that users of statistics and data are at the centre of statistical production: that their needs should be understood, their views sought and acted on, and their use of statistics supported. Relevance to the user is one of the key principles under the pillar of ‘Value’ in the Code of Practice for Statistics so the usefulness of these data has been considered from this user-perspective.
The datasets contain detailed information on the variety of sentencing factors sentencers were asked to consider when using the Harassment and stalking guideline to sentence harassment (s.2) and stalking (s.2A) offences. These factors may be relevant in determining the type of sentence handed down or the sentence length. The factors taken into account will vary depending upon the facts of each individual case.
The data also contain some basic demographic data about the offenders (their age group and gender), which could be used to examine how different groups are represented within the data and how factors and sentencing outcomes may vary from one group to another. It is intended that these data will be useful for any user who wants to better understand magistrates’ courts sentencing factors and outcomes for these offences. It was not possible to directly collect ethnicity data in this data collection.
Publishing these data contributes to fulfilling one of the Council’s responsibilities, of “promoting awareness amongst the public regarding the realities of sentencing and publishing information about sentencing practice in magistrates’ courts and the Crown Court” as well as one of the Council’s additional functions which says it must “promote understanding of, and public confidence in, sentencing and the criminal justice system”.
ii. Accuracy and reliability
Accuracy is the proximity between an estimate and the (unknown) true value. Reliability is the closeness of early estimates to subsequent estimated values. This section will provide users with an overview of how accurate and reliable the data are thought to be, by considering possible sources of error and bias.
Sources of error and bias
There are several types of error that can arise within data such as these, including coverage error, sampling error, non-response error and measurement error. Each of these, including how they may have occurred within the published data and how they have been dealt with (where possible), are described in detail below.
Coverage error
Coverage error occurs when the list used to select a sample (the ‘sampling frame’) does not have a one-to-one correspondence with the target population (the total group of units or people that we want to sample from). As this data collection covered all magistrates’ courts, rather than a sample of courts, the Council is confident that there should not be any coverage error within the data.
Sampling error
Sampling error is where there are differences between estimates generated using the sample and the true value for the population.
Offences of harassment (s.2) and stalking (s.2A) are summary only offences, meaning that they are primarily dealt with at the magistrates’ courts, where these data were collected from. However, the pre and post guideline data collections did not achieve a 100 per cent response rate from all courts and there are no comparable published sources of data from the same period on the key sentencing factors. As such, there is a risk of the data either being biased or not being representative of all harassment and stalking offences.
It has not been possible to make a comparison of these data with custodial sentence lengths for these offences from the Ministry of Justice’s Court Proceedings Database (CPD), an administrative database of court outcomes for both Crown Court and magistrates’ courts, in order to make an assessment of any potential sampling error. This is because the sentence lengths in this data collection were banded. However, as the distribution of sentence outcomes was very similar between the two sources, it is expected that the data are largely representative and still useful in identifying, for example, the most and least common factors taken into account and the sentences imposed.
Non-response error
There are two types of non-response: in the context of this data collection, ‘unit non-response’ is where a form was not filled in for an offender sentenced for these offences during the data collection period, and ‘item non-response’ is where a form was filled in, but a question or box that should have been completed was left blank, so the non-response was specific to a certain set of items on the form. Where these types of non-response occur, this can lead to error (or bias) in the data.
When the volume of forms returned was compared to the total number of adult offenders sentenced within the same dates as the data collection, this equated to an approximate response rate of 13 per cent for harassment and 14 per cent for stalking for the pre guideline data; post guideline there was a response rate of 21 per cent for harassment, and 27 per cent for stalking. If certain types of courts were more or less likely to respond, then this may have affected the data. For example, there is a chance that the resource of the sentencers to fill in the survey, related to how busy they were, could have impacted responses to the survey. Response rates may then have differed across courts, leading to biased estimates as a result of a form not being completed. This would produce unit non-response error.
Item non-response is another type of non-response which occurred across many of the variables, although it may affect some more than others. If the records with unknown or missing data are systematically different to those where clear data have been provided, this could lead to item non-response error.
Aside from the comparison with the CPD data discussed in the ‘Sampling error’ section earlier, there is no other source of evidence on the sentencing factors taken into account in magistrates’ courts for sentencing harassment (s.2) and stalking (s.2A) offences. It is therefore not possible to measure the extent to which these data may be affected by non-response error. However, there are some reasons why it is thought that non-response error may not be substantial within any analysis of the data:
- the sentencing outcomes were found to be fairly representative of all outcomes imposed for these offences at the time (as detailed earlier), so it could also be assumed that the factors indicated on the forms are also representative
- there is no explicit evidence of sentencers being more likely to fill in data collection forms for some types of cases more than for others, so it is assumed that this does not happen
Measurement error
We have assumed that sentencers have interpreted the form correctly and accurately recorded all the case details. However, there is always the chance of human error, and any differences between the true values related to the sentence imposed and the final published dataset are known as measurement error. Furthermore, given the wording of the instructions in the form, if a sentencer did not tick a particular factor then it has been assumed that this particular factor was not taken into account during sentencing. Similarly, if a factor was ticked then it has been assumed it was taken into account. However, this may not be the case and omission as a mistake may have been conflated with omission due to lack of relevance.
Most of the pre guideline data and all of the post guideline data were collected digitally (online), which gave the Council the opportunity to build in internal assurance processes and question routing. It is hoped this has improved the data quality over earlier data collections which were administered solely using paper forms.
While free text fields are useful for gathering detailed individualised comments, these take a lot of resource to process and are potentially more prone to misinterpretation, introducing error in the data. To minimise the effect of this, tick-box options were used for most questions and free text fields were only used where necessary.
iii. Timeliness and punctuality
The data collection was undertaken either side of the Harassment and stalking guideline coming into force in October 2018: between 1 November 2017 and 30 March 2018 for the pre guideline period, and between 23 April 2019 and 30 September 2019 for the post guideline period. Thus, with regards to the original intention for collecting the data (to monitor the impact and implementation of the Harassment and stalking guideline), it captured data in a timely way.
The Council recognises that the nature of harassment (s.2) and stalking (s.2A) offending and other external factors may have changed since the data collection exercise was undertaken and so the factors that sentencers considered in 2017-2019 may not be entirely representative of current sentencing practice. It is nevertheless hoped that publication of the raw underlying data collected will still be useful, adding to the knowledge base to better understand magistrates’ court sentencing factors in relation to outcomes.
iv. Accessibility and clarity
Publishing this information means that the data are free and equally available to all users. It is thought that these data might be of most interest to an expert user comfortable with processing and manipulating raw datasets. Alongside the raw datasets, we have also assembled a metadata document. This is intended to be read alongside the datasets to understand, for each variable in the data, what the range of values mean and if there are any limitations of using this variable to draw conclusions.
One of the challenges has been ensuring that the data are published at a sufficient level of detail to enable users to sufficiently delve into the individual factors behind magistrates’ court sentencing decisions, while still taking steps to reduce the risk of disclosure for the individual offenders as much as possible. There is a disclosure statement published alongside the data, and further details can be found in the metadata document.
For the user who still wants to understand the impact and implementation of the Harassment and stalking guideline but is not comfortable analysing data themselves, the Evaluation of the impact and implementation of the Sentencing Council’s intimidatory offences definitive guidelines fulfils this purpose, by utilising the same data source and providing additional narrative around findings from analysis of the data.
v. Coherence and comparability
Coherence and comparability are the degrees to which data derived from different sources or methods, but that refer to the same topic, are similar, and can be compared.
Comparability with existing analysis using the same data
The data being published were used as one of the sources for the intimidatory offences guideline evaluation. Therefore statistics drawn from the published data collection and those referenced within the guideline evaluation should be comparable.
The data published were also used to inform the Research review of the Overarching principles: domestic abuse sentencing guideline, in particular the data on the effect of the domestic abuse context on sentencing decisions. The data being published have undergone further cleaning and internal quality assurance since that piece of work in preparation ready for publication. As a result statistics from the data collection and those referenced in the research review may have very small differences, but overall, should be comparable.
Comparability with other data sources
As far as the Council is aware, there are no other data sources available on magistrates’ courts sentencing practice for these offences that contain both the factors taken into account by sentencers and details of the sentences imposed.
For further information about these data, please contact the Analysis and Research team at Research@sentencingcouncil.gov.uk.