Section 1: Background to the statistics
The Sentencing Council was set up in 2010 and produces guidelines for use by all criminal courts in England and Wales. The Robbery definitive guideline came into force in April 2016, replacing an earlier sentencing guideline produced by the Council’s predecessor body, the Sentencing Guidelines Council (SGC).
The Council has a statutory duty to monitor the impact of the sentencing guidelines it produces. The aim of the Robbery definitive guideline is to increase the consistency of the sentencing process, whilst ensuring that offences which cause serious harm to the victim and involve knives, firearms or imitation firearms result in the toughest sentences (for further information refer to the resource assessment published alongside the guideline). In order to evaluate the actual impact once the guideline was in use, an assessment of the Robbery definitive guideline was conducted and then published in February 2019.
To support this assessment, a data collection exercise was conducted post-guideline, in all locations of the Crown Court, in order to gather detailed information from sentencers in the Crown Court about how they sentenced offences using the Robbery definitive guideline. The survey was administered by the University of Leicester, and ran between 1 November 2016 and 28 April 2017. Over this period, a total of 642 valid forms were received, comprising of 492 records relating to street/less sophisticated commercial robbery, 47 relating to professionally planned commercial robbery, and 89 relating to dwelling robbery (for the remaining 14 records, the type of robbery was left blank and has been recorded in the dataset as ‘robbery’). The volume of cases in the published data is discussed later on in the context of overall robbery offending.
The data collection exercise involved asking sentencers in the Crown Court to complete a paper or electronic PDF form for every adult offender they sentenced for robbery (where this was the principal offence). The form asked sentencers to give detailed information on the: date of birth and gender of the offender; the sentencing date; the type of offence; the location of the offence; culpability and harm factors; the offence category; the sentence starting point; aggravating and mitigating factors (including previous convictions); information on whether there was a guilty plea and if so when it was entered and the reduction applied; and detailed information on the final sentence outcome. The specific court returning the survey was recorded against each form and sentencers were also given an opportunity to state the single most important factor they took into account when deciding on their final sentence.
This bespoke data collection was the third of its kind to provide the Council with detailed information on the sentencing factors taken into account by sentencers after the Crown Court Sentencing Survey (CCSS) finished. This was a rich source of detailed sentencing data, providing a wealth of information on sentencing for a wide range of offences sentenced specifically within the Crown Court. However, following an external review, the CCSS was stopped at the end of March 2015, and the Council evolved its analytical approach to develop more focussed and targeted ‘guideline-specific’ data collections in both magistrates’ courts and the Crown Court. This data collection release follows the publication of the theft from a shop or stall offence data in December 2020 and drug offences data in July 2022. The publication of these data falls within the Council’s strategic objectives for 2021 to 2026 to ensure that the Council’s work is evidence-based, and to work to enhance and strengthen the data and evidence that underpins it.
While there was some detailed analysis of the Crown Court data collection exercise undertaken specifically for the guideline assessment, it is hoped that publication of the raw underlying data will be useful, adding to the knowledge base to better understand Crown Court sentencing factors in relation to outcomes.
This document is intended to be read alongside the raw data, so that its users can better understand its overall quality.
When considering the data, it is important to keep in mind that every case is unique and there are many factors, both relating to the offence and the offender’s personal circumstances that will be taken into account when deciding on the appropriate sentence. Therefore, there may be factors other than those collected on the form and detailed in the data that impact on the final sentence. Furthermore, while the same factors may be present in more than one case, the specific circumstances of each case may mean that the factors are not given the same importance in all cases which may, in turn, be reflected in the decision regarding an appropriate sentence for the offender in question.
Section 2: Assessment of quality
i. Relevance
Relevance is about making sure that users of statistics and data are at the centre of statistical production: that their needs should be understood, their views sought and acted on, and their use of statistics supported. Relevance to the user is one of the key principles under the pillar of ‘Value’ in the Code of Practice for Statistics so the usefulness of these data has been considered from this user-perspective.
The datasets contain detailed information on the variety of sentencing factors sentencers were asked to consider when using the relevant robbery guideline. These factors may be relevant in determining the type of sentence handed down or the sentence length. The factors taken into account will vary depending upon the facts of each individual case. Sentencers were also asked to state the ‘Single most important factor’ that they took into account, with regards to the sentencing outcome. A coded and simplified version of the ‘Single most important factor’ has been published alongside the main dataset (see Annex A) and this will be only the third time that data like these will be available in the public domain. This should provide a new insight into the key factors affecting sentence outcomes.
The data also contain some basic demographic data about the offenders (their age group and gender), which could be used to examine how different groups are represented within the data and how factors and sentencing outcomes may vary from one group to another. It is intended that these data will be useful for any user who wants to better understand Crown Court sentencing factors and outcomes for these offences. It was not possible to directly collect ethnicity data in this data collection, but the Council is currently exploring options regarding linking this information into data collections in the future.
Publishing these data contributes to fulfilling one of the Council’s responsibilities, of “promoting awareness amongst the public regarding the realities of sentencing and publishing information about sentencing practice in magistrates’ courts and the Crown Court” as well as one of the Council’s additional functions which says it must “promote understanding of, and public confidence in, sentencing and the criminal justice system”.
ii. Accuracy and reliability
Accuracy is the proximity between an estimate and the (unknown) true value. Reliability is the closeness of early estimates to subsequent estimated values. This section will provide users with an overview of how accurate and reliable the data are thought to be, by considering possible sources of error and bias.
Sources of error and bias
There are several types of error that can arise within data such as these, including coverage error, sampling error, non-response error and measurement error. Each of these, including how they may have occurred within the published data and how they have been dealt with (where possible), are described in detail below.
- Coverage error
Coverage error occurs when the list used to select a sample (the ‘sampling frame’) does not have a one-to-one correspondence with the target population (the total group of units or people that we want to sample from). As this data collection covered all locations of the Crown Court, rather than a sample of courts, the Council is confident that there should not be any coverage error within the data.
- Sampling error
Sampling error occurs when a sample is taken, instead of observing the whole population, and where there are differences between estimates generated using the sample and the actual unknown true value for the population.
Robbery is an indictable only offence, meaning that it can be dealt with only at the Crown Court, and these data were collected from all locations of the Crown Court, rather than a sample of them. However, since the data collection did not achieve a 100 per cent response rate from all courts and there are no comparable published sources of data from the same period on the key factors used in the Crown Court to sentence these offences, there is a risk of the data either being biased or not being representative of robbery offences and offending.
To check whether analysis of these data could lead to sampling error, a comparison was made with data on robbery offences from the Court Proceedings Database (CPD), an administrative database of court outcomes for both Crown Court and magistrates’ courts held by the Ministry of Justice. Data from the equivalent time period were examined, and found to be broadly representative when compared with Crown Court sentencing outcomes. There was, however, a slight difference between the two sources when comparing the average custodial sentence length (ACSL), as noted in the robbery guideline assessment. In the CPD, the mean ACSL (after any reduction for guilty plea) was 4 years 1 month, whereas in the data collection, the overall ACSL was 4 years 8 months. This could be a result of the data collection capturing more serious cases, or could mean that the data collection captured a higher proportion of dwelling or professionally planned commercial robbery offences when compared with the CPD (as these offences attract higher sentence levels than street/less sophisticated commercial robbery). However, as the type of robbery is not available in the CPD, it is not possible to check this.
As a result of this difference, it is possible that the sentencing factors relevant in the published cases are not wholly representative of all offenders sentenced for robbery offences. However, as the distribution of sentence outcomes was very similar between the two sources, it is expected that the data are largely representative and still useful in identifying, for example, the most and least common factors taken into account and the sentences imposed.
An assessment was also made of how representative the demographics of the offenders contained in the data collection were of the total population of offenders sentenced for robbery offences. The proportion of offenders of each gender and age group were compared with the equivalent proportions from the CPD. The samples from the data collection were found to be broadly representative of the CPD data for both age and gender, which means that users can be confident when using these variables in examining how factors and sentencing outcomes may vary from one group to another.
- Non-response error
There are two types of non-response: in the context of this data collection, ‘unit non-response’ is where a form was not filled in for an offender sentenced for these offences during the data collection period, and ‘item non-response’ is where a form was filled in, but a question or box that should have been completed was left blank, so the non-response was specific to a certain set of items on the form. Where these types of non-response occur, this can lead to error (or bias) in the data.
When the volume of forms returned was compared to the total number of adult offenders sentenced within the same dates as the data collection, this equated to an approximate response rate of 48 per cent. If certain types of courts were more or less likely to respond, then this may have affected the data. For example, given that the survey was not supervised at the court level, there is a chance that the administration of the paper forms may have differed between courts, depending on the amount of resource available to distribute and collect the forms, or on the resource of the sentencers to fill in the survey which related to how busy they were. Response rates may then have differed across courts, leading to biased estimates as a result of a form not being completed. This would produce unit non-response error.
Item non-response is another type of non-response which occurred across many of the variables, although it may affect some more than others. If the records with unknown or missing data are systematically different to those where clear data have been provided, this could lead to item non-response error.
Aside from the comparison with the CPD data discussed in the ‘Coverage error and sampling error’ section earlier, there is no other source of evidence on the factors taken into account in the Crown Court for robbery offences post-guideline. It is therefore not possible to measure the extent to which these data may be affected by non-response error. However, there are several reasons why it is thought that non-response error may not be substantial within any analysis of the data:
-
- the sentencing outcomes were found to be fairly representative of all outcomes imposed for these offences at the time (as detailed earlier), so it could also be assumed that the factors indicated on the forms are also representative
- a reasonable volume of data were collected, so users do not need to rely on only a small number of offenders to conduct any analysis
- there is no explicit evidence of sentencers being more likely to fill in data collection forms for some types of cases more than for others, so it is assumed that this does not happen
- Measurement error
We have assumed that sentencers have interpreted the form correctly and accurately recorded all the case details, that these have then been accurately interpreted and inputted by the external contractors and accurately cleaned in preparation for publication. However, there is always the chance of human error at each of these stages, and any differences between the true values related to the sentence imposed and the final published dataset are known as measurement error. Furthermore, given the wording of the instructions in the form, if a sentencer did not tick a particular factor then it has been assumed that this particular factor was not taken into account during sentencing. Similarly, if a factor was ticked then it has been assumed it was taken into account. However, this may not be the case and omission as a mistake may have been conflated with omission due to lack of relevance.
There are two variables – ‘No relevant aggravating factors’ and ‘No relevant mitigating factors’ – that have been removed from the dataset due to concerns that the factors were misinterpreted. There were a number of instances of direct contradictions, either from sentencers ticking to say ‘no relevant mitigating factors’ were relevant while also indicating that a mitigating factor (e.g. ‘Age and/or lack of maturity where it affects the responsibility of the offender’) was relevant in their sentencing decision, or from sentencers not ticking any of the mitigating factors but also not ticking the box to indicate that ‘No relevant mitigating factors’ were considered. Similar contradictions were also seen with the variables relating to ‘No relevant aggravating factors’. Given that users will be able to re-create these factors themselves from the data using the absence of the other factors involved, it was decided that the removal of the potentially misleading data would be the appropriate approach with regards to the accuracy and quality of the overall data.
The style of questions and the format of the survey may also have contributed to the level of error. The data were collected using paper and electronic PDF forms as opposed to being collected digitally (online), so there was not the option to add in any internal assurance processes to flag inconsistent answering within the same form (for example, where a sentencer may have ticked the mitigating factor of the offender not having any previous convictions but then also provided a number for the volume of previous convictions taken into account during sentencing). To improve the data quality, we have applied these types of internal consistency checks prior to publication and amended some records where there were obvious discrepancies. For more details, please see the metadata file.
While free text fields are useful for gathering detailed individualised comments, these take a lot of resource to process and are potentially more prone to misinterpretation, introducing error in the data. To minimise the effect of this, tick-box options were used for most questions and free text fields were only used where necessary. As mentioned above, sentencers were asked to state the ‘Single most important factor’ influencing their sentence, and they were given a box to provide a free text answer. The answers to this question varied widely, both in the nature of the answers (the factors mentioned) and in how they were worded/structured. It was not possible to publish the raw answers as they included specific details about the offender, offence, location and other details that may have risked being disclosive. Cleaning and coding these data mitigates the risk of any offender being identifiable within the data but some assumptions have been made regarding interpretation of these answers.
Often, sentencers entered more than one factor in the ‘Single most important factor’ field; in these cases, the separate factors were individually coded and, as a result of this, multiple factors may be present for a single record. To make the data easier to analyse, the data have been provided in a different dataset to the rest of the data. See Annex A for more detail.
iii. Timeliness and punctuality
The data collection was undertaken between November 2016 and April 2017, several months after the definitive guideline came into force (April 2016). Thus, with regards to the original intention for collecting the data (to monitor the impact of the guideline on sentencing), it captured data in a timely way. The same guideline is still in place at the time of publishing, and the Council is not aware of any other policies, legislation or other changes that are likely to have had an impact on sentencing practice since the assessment of the impact and implementation of the guideline was published in February 2019. Thus, the data should still be wholly relevant and useful.
We recognise that the nature of robbery offending and other external factors may have changed since the data collection exercise was undertaken and so the factors that sentencers considered in 2016 and 2017 may not be entirely representative of current sentencing practice. It is nevertheless hoped that publication of the raw underlying data collected will still be useful, adding to the knowledge base to better understand Crown Court sentencing factors in relation to outcomes.
iv. Accessibility and clarity
Publishing this information means that the data are made free and equally available to all users. It is thought that these data might be of most interest to an expert user comfortable with processing and manipulating raw datasets. Alongside the raw datasets, we have also assembled a metadata document. This is intended to be read alongside the dataset to understand, for each variable in the data, what the range of values mean and if there are any limitations of using this variable to draw conclusions. An example of the form completed by sentencers has also been published, to aid users’ understanding of the way the questions were asked, the layout, etc, which may be useful in any analysis.
For the user who still wants to understand the impact of the Robbery definitive guideline but is not comfortable analysing data themselves, the robbery guideline assessment discussed previously fulfils this purpose, by utilising the same data source and providing additional narrative around findings from analysis of the data.
One of the challenges has been ensuring that the data are published at a sufficient level of detail to enable users to sufficiently delve into the individual factors behind Crown Court sentencing decisions, while still taking steps to reduce the risk of disclosure for the individual offenders as much as possible. There is a disclosure statement published alongside the data, and further details can be found in the metadata document.
v. Coherence and comparability
Coherence and comparability are the degrees to which data derived from different sources or methods, but that refer to the same topic, are similar, and the degrees to which data can be compared over time and domain.
Comparability with existing analysis using the same data
The data being published were used as one of the sources for the robbery guideline assessment; however, they have undergone further cleaning and internal quality assurance in preparation for publication. As a result, while we believe they should still be analogous, some very small differences may exist.
Comparability with other data sources
The Council collected comparable data on robbery offences sentenced in the Crown Court between October 2010 and the end of March 2015, as part of the CCSS. These data are published on the Council’s website. Although the types of factors taken into account may generally be comparable across the two data sources, there are several reasons why differences between the two would be expected. The CCSS data were collected during a different time period (before the current guideline was in force) and the forms themselves were different. Users who intend to compare the two sources should bear these differences in mind when interpreting any results.
For further information about these data, please contact the Analysis and Research team at Research@sentencingcouncil.gov.uk.
Annex A: Single factor analysis
As mentioned above, sentencers were asked to note the ‘Single most important’ factor considered while passing the sentence. This information was provided in a free text field which has been cleaned and coded internally within the OSC using thematic analysis. It has been provided in a separate dataset, with a unique identifier allowing it to be mapped onto the main dataset.
The data were examined and aligned with the main factors considered in the guideline. Each coded variable in the dataset is binary, with a value of 1 indicating the factor in question was mentioned and 0 indicating it was not.
There are 49 coded variables within the dataset. They are not an all-inclusive list but represent the main codes identified by the analyst; therefore, they should be interpreted with some caution. Missing data entries and unknown data entries have been coded separately as ‘missing’ and ‘other_unknown’ in the dataset respectively.
Around 93 per cent of records had the ‘Single most important’ factor variable populated (the remaining 7 per cent have been coded as ‘missing’). Of the records where an answer was provided, 90 per cent have been assigned to a variable in the dataset (excluding ‘other_unknown’). In some cases, sentencers highlighted more than one factor; where this was the case, the separate factors have been individually coded. As the data have been coded manually, it is possible that the accuracy of the analysis has been affected by the sometimes subjective nature of the free text responses; although a sample of records were manually checked to make sure that they were being coded appropriately, it was not possible to do this for every record. Any differences between the single most important factor as intended by the sentencer and the final published dataset is a source of measurement error.
Further details on the method used for assigning the variables and the possible issues associated with this method are given within the metadata document.
This analysis has been conducted as an iterative process, by engaging fully with the data, searching for relevant factors, reviewing the factors and logic behind the coding and conducting quality assurance. However, due to the subjective nature of thematic analysis, care should be taken when interpreting these data.