by Nick Bailey

In July this year, the Department for Communities and Local Government (DCLG) published the Listening to Troubled Families report with a great fanfare. Its stated aim was ‘to inform [government] thinking and policy development’. At its launch, Secretary of State Eric Pickles hailed the report as providing ‘real insights into these families’ lives’ and as offering a ‘true understanding of the challenges local authorities face’. The government had set aside £448 million to tackle the problems identified. In short, the report was portrayed as a solid piece of research driving an ‘evidence-based’ approach to policy making. 

Reading the report, I became very concerned with how the sixteen case-study families had been recruited for the research and with how the information they had provided was being used. The report appeared to me to breach basic standards of ethical good practice. The government has clear official guidance on ethical standards for social research and appropriate procedures for ensuring that work done in its name complies with these. I was surprised, therefore, that the ethical issues had not been highlighted when the researcher applied for ethical approval for the work.

Under a Freedom of Information (FoI) request, I asked to see a copy of the ethics application form. The reply from the DCLG states that no ethical approval was required as the report ‘was not research’. To support this claim, it quotes a line in the report stating that it was ‘not formal research’. But the government guidelines do not distinguish between formal and informal research: if it is research, it has to comply with the ethical standards.

Unsatisfied with this response from the DCLG, I made a formal complaint setting out what I saw as the breaches of ethical standards. The response from the DCLG went slightly further this time. It stated that the report was not research because it was no more than a ‘dipstick’ operation, a piece of ‘informal information gathering’:

Having considered these guidance notes, their definitions of social research and the report in question, I can confirm that I do not consider ‘Listening to Troubled Families’ as being within the definition of Government social research and thus the scope of the guidance. My rationale for this is that this report falls more properly within the description ‘dipstick/informal information gathering’.

(Reply from Jane Todorovic, Head of Profession for the Government Social Research (GSR) service at DCLG, 3 October 2012)

If the report was not proper ‘social research’, this would raise a whole new set of questions about why it was given such extensive press and media prominence and why it was considered a reliable basis for policy. How many other ‘dipstick’ exercises have been given such coverage? Is it government policy to move from ‘evidence-based policy’ to ‘policy by dipstick’?

But these questions are a distraction from the main issue, which is about ethical failure. I think it is useful to look in detail at what appear to me to be the ethical issues with this report. The government’s standards set out five ethical principles, of which this report appears to have broken at least three.

The first principle seems to have been broken is about free and informed consent. A very basic requirement is that people who are asked to participate in social research should be freely able to decline and ‘should not in any way feel pressurised to take part’. But the families interviewed were recruited by local authority-run Family Intervention Projects and, as the report notes, these projects had a power of sanction over them. Can the families really be considered free to decline in this case?

Participants should have sufficient information to make an informed decision on the risks of participation. It is widely accepted good practice that written information is provided so participants have a record they can refer to. The FoI request I made to the DCLG specifically asked for copies of the written information given to participants. Since the DCLG did not provide any such information in its response, it appears that no written information was provided for participants.

Particular consideration needs to be given to the issue of informed consent in the case of children and ‘vulnerable’ adults. It is clear that some of those interviewed could be considered vulnerable; one respondent is described as having learning difficulties and ADHD, for example. Such issues would normally be identified in an application for ethical approval and appropriate protocols put in place to deal with them. As no such application was made, there is no evidence that the issues were considered, let alone appropriately handled.

Additional consent should be collected from participants after completion of fieldwork where very personal data will be used in the presentation of findings. Again, such issues would normally be discussed in the application for ethical approval. Since there was no application, there is no evidence such enhanced consent was considered here, although it was certainly warranted given the way material on families was presented.

The second principle that seems to have been ignored is about confidentiality and non-disclosure. Many of the sixteen families are large, which automatically increases disclosure risks, while the report further increases these by providing details on family composition including the number, age and gender of children. The report states that only names have been changed, with the implication that all other details are reported accurately.

In breaking these two principles, the report shows an apparent disregard for the rights and the welfare of these families, many of whom are highly vulnerable. Given that the stated aim of the Troubled Families Unit is to help families turn around their lives for the better, this bodes badly for how the Unit will go about its work in the future.

The third principle the report appears to ignore concerns sound and appropriate methods, and interpretation of findings. Here the report also demonstrates a disregard for the principles of sound policy making. The research cannot provide adequate evidence because it was based on a flawed sampling process. Just looking at family size, we know that the sample is highly unrepresentative of the wider population of supposedly ‘troubled families’, as Ruth Levitas pointed out in Still not listening.

The report does make passing acknowledgement of some of its limitations. As noted above, it does state that it was ‘not formal research’. But it stops short of saying that it was not research at all. The report also seeks to reassure us that it was a piece of sound research when it provides details of how families were recruited, and when it tells us that it was based on formal interviews that were recorded and transcribed.

It does also acknowledge that ‘the information [interviewees] gave us is not representative of the 120,000 families that are deemed as ‘troubled’’ (p. 5). But it ignores its own caveat when it claims that it provides a sound basis for policy making. These limitations were not highlighted in the Secretary of State’s endorsement of the work nor in the interviews given to promote the report.

No matter what label the government wishes to give it, the report clearly involved research activities carried out on human subjects. If these activities do not require ethical approval, there is something seriously wrong with the government’s guidelines. The guidelines exist to protect the public, in particular more vulnerable individuals, from harm or exploitation by those with power. It should not be possible for the powerful to simply by-pass such safeguards.

However flawed in design, the Listening to Troubled Families report is a piece of research. The ethical concerns identified in my complaint need to be addressed, not avoided by an attempt to reclassify the report. Satisfactory answers need to be provided or the report should be withdrawn. Unethical research has no place in public policy making.

Nick Bailey is a Senior Lecturer in Urban Studies at the University of Glasgow and a member of the PSE: UK research team.