SMILE trial data to be released

@Amw66 and @arewenearlythereyet - I agree that an average attendance collected over many weeks before baseline would have been a much better way of establishing a baseline figure for attendance, but unfortunately, that's not what they did.

Age would also have been useful, but I think it was deemed too much info and likely to lead to identification of participants (presumably because it will have been coded as DoB, and converting to something less identifiable would take work).

Other things that would have been useful to know would also be when the baseline, 3mo, 6mo and 12mo measures were made - I suspect some data (ie attendance) were missing simply because they occurred during the school holidays.

It also would be useful to know when each participant received therapy - particularly the LP course itself. There's an indication that this may have anywhere between baseline and 6 months into the study. The timepoints for data collection were also fairly 'flexi' (I think the paper has more details).

 

I'm still working back through the original documents, but originally they did intend to use school records, then ended up just asking the kids. Trish is more or less right about the measure: the possible responses were ½day, 1 day, 2, 3 , 4, and 5 days. What I haven't found yet is what constituted a "day". If they went in to school on three days for some specific lessons, did that count as 3 days or did they expect the kids to work out what fraction of a day was involved?

 

I get the feeling it’s a ‘where do you start’ when it comes to critiquing this. I suspect that this is part sloppy part deliberate. Thanks for your review so far. I still haven’t got round to re-reading the paper and associated docs but I think there will be plenty to go through from the looks of it.

 

You asked for:

"School attendance in the previous week, collected as a percentage
(10, 20, 40, 60, 80 and 100 %)."

You got a form of this, however in the SMILE projects initial protocol (http://www.bristol.ac.uk/media-library/sites/ccah/migrated/documents/smprotv6final.pdf) it said:

“The primary outcome measure for the interventions will be school attendance/home tuition at 6 months. Secondary outcome measures will be school attendance at 6 weeks, 3 months and 12 months; the SF36 (physical function) at 6 weeks, 3 months, 6 months and 12 months and pain visual analogue scale at 6 months.”

“Children and young people are asked about school attendance and home tuition in a two item inventory. We will ask for consent to check school attendance using school records and will do this at assessment, 3 months 6 months and 12 months.”

You don't seem to have the data from school records. They could argue that your request was a bit ambiguous, but I thought I remembered them arguing that the data should not be released because participants could be identified from school records, so it would be a bit of a stretch for them to treat your requests as if it included this useful data when they were fighting against release, then as if it did not once they'd lost.

I've got to admit that I've forgotten a lot of SMILE details, but I don't remember reading that they chose to not collect this data. The Buzzfeed article on SMILE includes quotes from Crawley that seem to indicate that the data were collected:

https://www.buzzfeed.com/tomchivers/inside-the-controversial-therapy-for-chronic-fatigue

 

@Esther12 - It's possible they just did a cross-check on a few records during the feasibility study. Once they are sure it all 'lines up', they then don't feel the need to keep asking the school for records. I'd say, fair enough. Having to gather attendance records from the school and then marry it up to each timepoint for each participant is far, far more work for them than simply relying on self-report at the time of the questionnaire - so if they could avoid it, I can understand why they would do that.

 

In Scotland all hell broke loose over illegal data sharing between schools and third parties ( including health , social services) a few years ago. There has been a significant court judgement regarding this and some parents are pushing for and enquiry.

Politically it is too sensitive and i doubt this will happen.

It may be that some initial data pre GDPR 2018 becoming fully enforceable , was obtained in the feasibility trial, but the mechanism to obtain data for full trial may have become illegal.

Note - i don' t know if this is applied differently in England and Wales.

 

That could all be true, but there was no mention of it in any of the SMILE papers. They said that they would check school attendance using school records, and in the media Crawley has claimed that they did so and that this validate the self-report outcome they reported result for, but we have no data on this. That just seems weird to me, especially considering all of the other problems around this trial. It also wouldn't surprise me if LP did lead to an increase in social desirability bias on self-reported school attendance. If they have data from school records why have they not reported anything from it?

 

So far I have read that they had problems with sixth-formers who were only required to attend appropriate lessons, which was why they proposed to change their primary measure from school attendance to sf-36. (page 10 of the feasibility study)

As an ex-secondary school teacher, I have no idea how they would collect accurate records of children's attendance if they were part-time, coming in for specific lessons. Registration takes place first thing in the morning and first thing in the afternoon, and a separate book would be kept in the office for pupils signing in and out (obviously, for fire checks, there has to be a check), but who would go through this to provide the figures, I have no idea, let alone whether it would be approved.

In the statistical analysis plan (page 7) they state that they "have obtained consent to check school attendance using school records at 3, 6 and 12 months" but do not say whose permission was obtained. I suspect it was from the parents, and then they found that they couldn't get it from the schools.

Can kids who have been put under pressure by the Lightning Process to deny their illness, be relied upon to give accurate answers to the questions on school attendance in the previous week? These are kids whose idea of a survey is often "How much TV do you watch? (a) not much (b) about average (c) lots.

 

H

Here there is a coded system. with specific codes for authorised and unauthorised attendance - separate codes for medical appointments etc.
It can break down if you are on a reduced timetable - there is a signing in and out process at the front desk and this data is entered onto the system. Sixth year have their own attendance taken.
If kids have anxiety, or are prone to sensory overload and the front desk is busy, they will avoid signing in and out - so they can be in but marked off -another problem.
If you go home after an hour does that still count as half a day?

Note - here schools cannot now share data without parent's written consent. So here it could have been the other way around.

The elephant in the room that is never addressed is - are these kids capable of doing much when they are at school.
Attendance doesn't count for much if you struggle doing anything much once there. Many are simply being set up to fail.

 

If you go in for three half days, what does that count as?

 

A penny ha'penny.

 

On the other hand unless you use objective information in studies its not acceptable to just say, "we couldn't get the objective info so we used subjective information".

Can you imagine if all parents had to do to insure their kids went to school whilst they were out at work was say, "did you go to school today".

In a situation wherby its not possible to design a study with objective info the outcomes are likely not able to stand by any conclusion.

The protocol promised school attendance records as a primary measure (regardless of the flaws in what that would show in terms of severity of disease, relapse etc), the records are about as objective as any study could wish for in terms of factual information, they should have delivered accordingly.

If attendance was to be used as a measure then the actual school records should have been used not just asking kids if they went to school or not.

 

Things may have changed but when I was 'at school' I wasn't, a lot of the time, and i was never asked about it unless I got in some form of trouble that drew attention to my not being at school. Do parents really ask their kids if they went to school today?

 

No, Esther Crawley does that.