Reflecting Realities, the methodology behind the report

We publish a Reflecting Realities report every year and have done so since July 2018.

There are now six Reflecting Realities reports.These reports, identify and evaluate representation within picture books, fiction and non-fiction for ages 3–11 and provide a benchmark to track and understand progress. Each report provides guidance informed by what we observe in the titles submitted in each cycle. This guidance is devised to support both producers and consumers of children’s literature to be more critically reflective. This in turn enables publishers to produce better quality inclusive literature and teachers to be more considered in their curation of their book stock.

Since the first report we have received a great deal of interest in this work and many questions about the rationale, methodology, findings and guidance. We have been clear and open about that methodology since the publication of our first survey. We use the same method every year in order to compare the surveys year on year. This provides a benchmark and a useful set of figures for the publishing industry as well as allowing us to draw out trends and patterns.

From the first publication, we have shared the methodology we use and we have a steering group of eminent, people who work in this field who scrutinise our work. We want to continue to be transparent so we have summarised our methodology and our process in detail here for readers who are new to this work.

At the end of this article we have also listed the questions we have been asked - and the answers we have given - about the methodology and the report. 

1. We invite UK Publishers of Children’s Literature to identify, collate and submit every title in their stock catalogue that:

• is aimed at children aged 3-11
• was first published in the year on which the survey is focused
• is categorised as fiction, non-fiction or picture books
• features any Black, Asian, or minority ethnic characters at any point in the book

Our sample of publishers is based on those listed in the Writers and Artists Year book. This list is then cross-checked with the Publishers Association. We work with the majority of UK publishers of children’s books every day.  So once we have reviewed the Writers and Artists Year book and Publisher Association records, we send out an email to our established key contact for each publisher to ask for the information we require. We let the Heads of the Publishing Houses large and small know that we are doing this. In the years before the pandemic, we also invited all publishers to a meeting where we could explain the criteria again and talk to them about the process. We count the publishers by ‘house’, keeping all imprints under the publishing house they are listed as.  For example, we count Puffin and Ladybird as being part of the Penguin Random House output and Wayland as part of Hachette.

We give publishers a deadline date, once that date is past, we follow up with every publisher who hasn’t submitted anything. This is to make sure they don’t have anything to submit which might be the case.  We make sure we are inclusive of smaller publishers, allowing them to submit pdfs rather than hard copies.  During the pandemic all publishers submitted their books as pdfs.

2. When they send us their titles, the publishers also fill in a form where they submit figures for the total number of books they had published with humans as main cast characters, the total number of books published with animals as main cast characters and the total number of books published with inanimate objects as main cast characters. We introduced this form in the second year of the survey at the request of the publishers so that we can report an estimate of the proportions of these different types of books.  We then look at every single book submitted to apply the eligibility criteria to make sure that the books submitted qualify for processing.  Sometimes, and in good faith, publishers may have included (for example) a book not published in the survey year or a sticker book or a book that doesn’t meet the criteria in some other way.

3. Once we have all the submissions together our team read every single book.  They apply questions from an analysis framework which is structured to enable us to consider how many Black, Asian and Minority Ethnic characters are featured in each book, their position in the narrative, their degree of agency and the quality of the representation both in the text and in illustrations. This analysis framework was developed with our Steering Group before the first survey and is designed to ensure a consistency of approach across the review team each year. The framework helps us to effectively review the extent and quality of ethnic minority representation in each title.

4. For the first survey, the team of reviewers was made up of our Teaching Team and our Librarian with help from Dr Fen Coles who is a member of our Steering Group.  We realised there was an opportunity to develop the knowledge and understanding of future members of the publishing community and developed an internship programme to enable us to provide meaningful paid work experience with training to students studying publishing. Between 2019-2022 we have trained and worked with students from UCL, Bath Spa and City University. This initiative allows us to build expertise and capacity in the system by investing in future members of the publishing workforce.

5. Once every book has been reviewed, we collate and review the data generated from the review process.  This is a long and careful process which looks for trends and analyses the quality of portrayals of characters of colour as well as determining the headline numbers.

6. The figures for total output of children’s books are drawn from the Nielsen Book Database, which includes children’s fiction, non-fiction and picture books specifically aimed at 3-11 year olds but does not include comic strips, novelty books, annuals, early learning and reference books.  Nielsen generously provide us with this information, they do not charge or receive any remuneration from us. Nielsen give us the data for the entire category for the year of the survey.  For example, in 2019 this was 28,522 books. We then filter the data removing anything that is not a book for 3-11 (i.e. we do not count books aimed at the pre-school market), not a book (i.e. it could be on this list and be a sticker game, a calendar or a diary, in which case we wouldn’t count it) and first published outside the stated year we are considering.  This means we are using an overall figure for total output that meets the same criteria as that which we have given the publishers for submissions.  We also filter out multiple editions. This is because we would only count one edition of a book in the submissions. In 2019 the final figure once we had filtered the data was 6478 books.

7. Once we have done this work we conduct a review team briefing in which we share the findings and analysis with everyone who has been involved in reviewing the books. We invite questions, encourage critique and provide the opportunity for the reviewers to add anything they feel is important for consideration based on the sample of books they reviewed. We then take our findings to our steering group. We ask them to review, scrutinise and to question our work. Our Steering Group have a range of expertise across the literature field. They are all experts and their names are clearly published in each report. None of our Steering Group are paid for their time on the steering group they are all volunteers.

8. The Ethnic categories used in the Study were drawn from the UK Census categories with appropriate extensions to these definitions to accommodate broader representations of ethnicity in literature. Whilst acknowledging the limitations of the Census definitions of ethnicity, we chose to apply these to allow us to draw meaningful parallels between the characters in the English population versus the characters who populate the world of books.

9.The statistic that details the number of minority ethnic pupils of Primary school age in England is taken from the Department for Education Schools, pupils and their characteristics*.

This annual survey and the tremendous cross-sector and partnership work it has inspired would not be possible without the dedication and generosity of spirit of everyone involved. Our charity is truly grateful to the Arts Council for continuing to fund this work, the publishing industry for their participation and commitment to this research and for the diligence of every individual involved at every stage in this process.

Questions that we have been asked about the report – and our answers.

In the 2022 report the extent of presence of characters of colour in picturebooks is reported as being 61%. Does this mean that only 39% of picturebooks feature white characters?

No. The vast majority of picturebooks that make up the 61% feature multi-cultural casts of characters which include white characters. In 2017 very few books published included racially minorisited characters. In the 2022 report, we observed a shift towards the inclusion of more representative casts of characters in books published.

Why do the terms you use in the reports in the reports to describe ethnicity vary?

Language used to describe ethnicity has evolved over the time in which we have been publishing this report. In the reports published in 2018 and 2019 we adopted the acronym BAME, meaning Black, Asian and Minority Ethnic to encompass the spectrum of ethnicities observed as part of the review process. We used the term as it was a commonly recognised short hand and allowed us to succinctly relay the data. We used the acronym with a caveat, acknowledging its limitations as being reductive and problematic and recognising that such collective terms can diminish the heterogeneity of each community of individuals classified under this term. In the third report, in light of evolving discourse regarding the term, it felt inappropriate to continue to adopt the acronym for the sake of brevity at the expense of the offence that it has the potential to cause. In 2020, we chose to omit the use of the acronym for future reports, instead opting to use the terms ‘Black, Asian or Minority Ethnic, ‘Ethnic Minority’ and ‘Minority Ethnic’ or racialised minorities.

When you say only 10% of children’s books published have a character from a Black, Asian or minority ethnic background in them, this is misleading, because many characters in children’s books have no ethnicity, as they are not human.

This assertion, we assume is rooted in the hypothesis that a disproportionately large presence of animal protagonists over and above human protagonists would go some way towards qualifying the lack of ethnic minority presence.

Even though there are characters in books that are not human, that does not reduce the percentage of characters who are from a minority ethnic background; the number of animal characters does not reduce the number of characters from ethnic minority backgrounds. Given that we know that ethnic minority presence makes up such a small proportion of the human casts featured in children’s books, including statistics regarding the extent of animal casts only serves to highlight even more how marginalised characters of colour are. 10% of children’s books published in 2019 had a minority ethnic character.  This is a fact. At their request, we worked with publishers to include the proportion of non-human books into the survey and publish that figure in the report.  On page 9 of the 2020 report we clearly explain what we have done to answer the ‘animal and inanimate object’ question however, that doesn’t change the fact.

If you are a child standing in front of a bookshelf of 100 books, around 35 of them will have only animals or inanimate objects as their main cast and around 65 of them will have human characters. Only 10 of them will have characters of colour in them. If you are Black, SE Asian, Chinese, North African or from any other racialised minority you are looking in just those 10 books to find someone like you.  We have enormous experience of working with bookshops, libraries, and schools.  None of us have ever seen a collection of children’s books in any of these places organised into ‘animal’ ‘inanimate object’ ‘human’ collections.  Therefore, the child in front of the book shelf will be looking at the full range of books available.  The statistic describes the experience of that child. As our findings indicate, readers of colour must settle for two levels of marginalisation, the first in being under-represented in stories that centre human casts and the next being second to animals in the literary space.

The research directly contrasts the headline figures with the 34% figure of school population, and uses that comparison to conclude that characters from an ethnic minority background are significantly underrepresented. Why don’t you contrast the percentage of books that exclusively feature human characters in them to the pupil population figure?

10% of children’s books have a character in them that is from a Black or minority ethnic background. 34% of children in primary schools are from a Black or minority background. We highlight the contrast between those two figures to show how Black and minority ethnic children are underrepresented in children’s literature.  As we’ve explained in the answer above, to suggest that it would be better to contrast the percentage of pupils with the percentage of books with only human characters in them is a viewpoint which denies the lived experience of hundreds and thousands of children in bookshops, libraries and classrooms.

Why do you use the figure from the Nielsen database to calculate the total number of books published?

This is an industry standard set of statistics that enable us to make a year-on-year comparison and to see an overview of the commercial publishing output. The whole database does include all ISBN numbers but it is easy for us to filter out books that do not fit the criteria we have given publishers.  We’ve been clear about the fact that we use it and clear and open about how we use it and how we make sure that we don’t include duplicate books or books that are not eligible. It is a decision that we made at the very beginning of the survey and we want to continue to use it because we believe it provides us with an objective output sample figure. At their request we developed a way for publishers to self-report the ‘animal’ statistics, but for the overall output there is a long-standing and industry supported set of independent statistics.  

Have you contacted every publisher that has produced books in the year of the survey?

We focus our efforts on producing a picture of the commercial publishing output because those are the books that end up in bookshops, libraries, and classrooms. We do everything we can to make sure that we have contacted every publisher that publishes children’s fiction, non-fiction and picturebooks for the 3-11 age range (we have detailed that process above). Although we check each year against the most up-to-date lists and make a public call for submissions we don’t individually contact every self-published author.  Our understanding is that the self-published output in children’s books is a small proportion of the total output and a very small proportion of the books that get into bookshops, libraries and classrooms.  We’ve checked this out using the figures that we do have.  For example (and this isn’t a directly comparable figure, but gives an idea of the issue) in 2019, 540,395 non-fiction children’s books were sold in the UK, 4,868 of those were ‘independently published’ – less than 1% of total sales for children’s non-fiction books.  More than half of the self-publishers who were responsible for those 4868 books sold only one or two books in the year.  We know that the publishers we are able to contact and work with for the survey are responsible for producing the vast majority of books in our bookshops, libraries and classrooms and this makes us confident that the sample we work with gives us an accurate picture of the commercial publishing output.

Why don’t you do your research in the same way as the CCBC in America?

Prior to producing the first report in 2018, we arranged for a member of our Steering Group to visit the Cooperative Children’s Book Center. The CCBC have been producing an annual survey focused on ethnic representation in children’s books published in the USA for over three decades. Through the visit and our conversations with the CCBC, we were able to learn about their work and processes. This was an important fact-finding mission to support us in thinking about how to best shape our process to ensure that it was as methodical, rigorous, valid, and fair as possible and resulted in a report that would be helpful to all stakeholders in the UK.  We were grateful for the advice of the CCBC but we have never tried to replicate their process.

Our context in the UK in general, and in CLPE in particular is completely different to that of CCBC. In our day to day work we only ask publishers to send us the books that we would include on our programmes with schools. If we had used these books to base the survey on it would not have given a fair picture of publishing output. For example, we wouldn’t ask publishers to send us middle-grade series fiction but this is a big proportion of yearly output. The CCBC context is completely different. We were keen to ensure that we gave publishers the opportunity to scrutinise all their output and to send us everything they deemed eligible. We do everything we can to make sure that this happens because it is not in our interest or in the spirit of this work to do otherwise. Because we were starting the survey from scratch in 2018 we were able to develop a methodology that worked for the UK context and that could be replicated year on year.