1. Sign our petition calling on Cochrane to withdraw their review of Exercise Therapy for CFS here.
    Dismiss Notice
  2. Guest, the 'News in Brief' for the week beginning 18th March 2024 is here.
    Dismiss Notice
  3. Welcome! To read the Core Purpose and Values of our forum, click here.
    Dismiss Notice

SMILE trial data to be released

Discussion in 'Psychosomatic news - ME/CFS and Long Covid' started by JohnTheJack, Mar 12, 2019.

Tags:
  1. Adrian

    Adrian Administrator Staff Member

    Messages:
    6,479
    Location:
    UK
    There is data here in terms of things like work hours missed by parents that I don't think has been reported. Here are a few graphs showing the number of work hours missed by parents (carer 1 + carer2) at the various measurement points. But there is a lot of missing data

    WorkHoursMissed_Baseline.png
    WorkHoursMissed3m.png
    WorkHoursMissed_6M.png

    WorkHoursMissed_12m.png
     
  2. JohnTheJack

    JohnTheJack Moderator Staff Member

    Messages:
    4,350
    I don't. I think others are better placed than I am to comment on the data.

    You're welcome.
     
  3. Amw66

    Amw66 Senior Member (Voting Rights)

    Messages:
    6,263
    If there is a lot of missing data how robust can any conclusions be.

    I suspect data protection rules may have stymied procuring official attendance records. When it comes to children data sharing can be extremely strict in education.
    The study may have straddled timescale for recent data protection legislation being implemented?
     
  4. rvallee

    rvallee Senior Member (Voting Rights)

    Messages:
    12,299
    Location:
    Canada
    Ah, see, it's missing the secret unlocking code found in Tarot cards.

    Thinking out loud but it seems that relying on a copyrighted self-help "treatment" means that it's all that more difficult to get hold of the training material since it could be actionable if it was published without permission. I don't think I've seen anything beyond some participants' descriptions. It's impossible to judge the value of a treatment without revealing any part of it. They can't keep it secret while promoting it as valid science.

    Beyond the data, this is a self-help "treatment" created by a quack who claims he has healing hands and has people jumping on mats and doing affirmation rituals. Run in a trial, funded by famous universities. Published in a world-class medical journal. People jumping on mats and yelling at their symptoms.

    It just feels like there are too many layers of insane to work out here and I don't really understand why this obvious pseudoscience could worm its way into actual medical research without objections raised at every damn step in the process.
     
    Inara, MEMarge, JohnTheJack and 7 others like this.
  5. Wonko

    Wonko Senior Member (Voting Rights)

    Messages:
    6,674
    Location:
    UK
    Have you heard of money?

    Or one hand washing the other?

    There's a network of these 'people', all supporting each other, and universities these days, run by businessmen.

    They used to run 'carnivals', now they run universities, and fill them with whoever makes money, in the name of 'science'.
     
  6. Adrian

    Adrian Administrator Staff Member

    Messages:
    6,479
    Location:
    UK
    They used some imputation techniques to add in missing data but I think you need additional variables to be able to do this (things like age, sex, .... ) as you build a model of the relationships between the variables and use those to fill in missing values.

    In there paper they did look at the sensitivity to using these techniques and I think if the quote intention to treat figures they use imputation (but not sure). I suspect someone like @Lucibee or @Sid could say more.
     
    MEMarge, Lucibee, MSEsperanza and 2 others like this.
  7. Esther12

    Esther12 Senior Member (Voting Rights)

    Messages:
    4,393
    From what I remember of the ruling, Bristol was making arguments about reidentification via school records, so at that point they seemed to realise that you were requesting the data for the school attendance records. Might they have tried to omit that from the release? Could you just check with them (or would that risk being treated as an entirely new FOI?)

    edit: Actually, has anyone compared the school attendance results to those from the publish paper? Maybe these are the different figures? Sorry - I should have tried to do that before asking. I've forgotten most of the details of SMILE tbh.
     
    Andy, MEMarge and MSEsperanza like this.
  8. Lucibee

    Lucibee Senior Member (Voting Rights)

    Messages:
    1,483
    Location:
    Mid-Wales
    They do say that they tested the sensitivity with and without the imputed data, but they don't say what the models/parameters for those imputed data were. That's important. If the imputed data are based on the data provided (ie, assuming data are missing at random), then of course they won't change the results if they include them. However, if the data were not missing at random (which is likely to be the case - missing data are highly likely to indicate a worse outcome), you still have to make assumptions about how those missing data might be presenting differently. But the paper doesn't tell us what those assumptions were. And they haven't included any imputed (outcome) data in the dataset they have provided.
     
    Mithriel, Sean, Andy and 7 others like this.
  9. Lucibee

    Lucibee Senior Member (Voting Rights)

    Messages:
    1,483
    Location:
    Mid-Wales
    Like @Adrian I've had a look at the attendance data.

    Before I saw the data (back in 2017), I thought that they should have looked at mean difference and not difference in means (same applies to PACE data imo). That bears out, especially as the data are so skewed. Individual diffs show a much more normal distribution, but would still benefit from baseline adjustment (I don't have the facilities to do this at the moment, nor to test stat sig).

    I've shown graphs for the differences between baseline and 6 months and 12 months in SR school attendance, amalgamated to whole days (plus/minus 0.5-1, 1.5-2, 2.5-3 etc).

    6 months: Mean diff for SMC is 0.28 days. Mean diff for LP+SMC is 0.81 days. Diff of diffs = 0.54 days
    12 months: Mean diff for SMC is 0.79 days. Mean diff for LP+SMC is 1.57 days. Diff of diffs = 0.78 days

    School attendance at 6 months was supposed to be a primary outcome measure. The SMILE trial paper only reports attendance at 12 months as a secondary endpoint. Make of that what you will.

    Also note that 40% [correction, 25-30%] of the data are missing (in both groups) by the time we get to 12 months.

    SA_diff6mo_wholedays.png SA_diff12mo_wholedays.png
     
    Last edited: May 3, 2019
    Hutan, inox, Sean and 9 others like this.
  10. JohnTheJack

    JohnTheJack Moderator Staff Member

    Messages:
    4,350
    No it was decided on the old law and the argument re children was dismissed.
     
    ladycatlover, Amw66, Andy and 2 others like this.
  11. arewenearlythereyet

    arewenearlythereyet Senior Member (Voting Rights)

    Messages:
    2,092
    so attendance difference infers that this is compared to a base figure? Do we have these numbers? Surely if the base attendance was 50% (say 95 days out of a potential 190 in a year then one day difference wouldn’t be significant ...sorry I may be being a bit simplistic but shouldn’t they be looking at the context of the difference? I’m just wondering what the basis of that calculation is? I can’t yet look at the data since my iPad doesn’t unzip files ...need to crank open a laptop.
     
    Amw66, Andy and MEMarge like this.
  12. Trish

    Trish Moderator Staff Member

    Messages:
    51,872
    Location:
    UK
    I think the 'days attendance' is days per week, so the possible range in the attendance data is 0-5. The difference that lucibee has graphed is then the difference for each child between their self reported weekly attendance at baseline and 6 months which has a range from -5 to +5.
    And I realise I should have left that for Lucibee to answer. Sorry, fools rush in and all that.
     
    Hutan, ladycatlover, Lucibee and 4 others like this.
  13. Unable

    Unable Senior Member (Voting Rights)

    Messages:
    193
    Location:
    UK
    So -5 and +5 are only possible where initial attendance is either full, or zero respectively!

    Thus seeing only the change is weird because there are variable floor and ceiling effects for each individual!

    In reality this forces most of the outcomes to the centre of the range.
     
    Trish, MEMarge, Amw66 and 4 others like this.
  14. Adrian

    Adrian Administrator Staff Member

    Messages:
    6,479
    Location:
    UK
    I think that is the case there is also an issue that this is school attendance for a week in the reporting period which is flexible which I think can add additional biases in terms of which week someone reports. I'm less likely to do stuff and certainly fill out forms when my daughter is less well.
     
    MEMarge, Hutan, Unable and 6 others like this.
  15. arewenearlythereyet

    arewenearlythereyet Senior Member (Voting Rights)

    Messages:
    2,092
    No worries and thanks ...I’m sure she won’t mind. I’m still struggling to make head nor tail of what it says. I guess that laptop needs to come out and I’ll need to refresh my memory on her ‘methodology’ before trying to understand it?
     
    MEMarge, Lucibee, Trish and 2 others like this.
  16. Lucibee

    Lucibee Senior Member (Voting Rights)

    Messages:
    1,483
    Location:
    Mid-Wales
    The reason I looked at mean difference is because there are problems with the measures they chose to look at.

    One of their original primary outcome measures was mean school attendance at 6 months. This was subsequently changed to a secondary outcome measure (that's another issue), and was reported in Table 3 as being 2.6 days vs 3.2 days (SMC vs SMC+LP) at 6 months and 3.1 vs 4.1 at 12 months. However, this is not a good way to describe the data, which is very skewed, particularly at 12 months. [eta: It also tells you nothing about actual improvement, which is surely the most important thing.] Although the differences between the means are similar to what you would obtain by looking at mean differences, the implications are different, because you are *not* looking at improvements, which are small (less than a day for each group).

    Here are the graphs to show how skewed the data are at 6 and 12 months:

    SA_6mo.png SA_12mo.png

    I hope you can see why use of means is inappropriate to summarise (and compare) these data, whereas for mean *difference* it is more appropriate.

    Also, if you look closely at the data, particularly for those in the 5-day group at 12 months, most of those individuals had only increased their attendance by a day or so.
     
    Last edited: May 2, 2019
    MEMarge, Sly Saint, inox and 7 others like this.
  17. Lucibee

    Lucibee Senior Member (Voting Rights)

    Messages:
    1,483
    Location:
    Mid-Wales
    Baseline data certainly match (from Table 1).

    Just checked means from Table 3 - all present and correct.
     
  18. Amw66

    Amw66 Senior Member (Voting Rights)

    Messages:
    6,263
    I think this is a very valid point - what is the attendance datum ( say for the previous 3-6 months) from which you are measuring. The interventions have to be set oin context to what is " normal" for these individuals.
    This age group is also the one with the best prognosis for natural recovery (and if they are attending school on a semi regular basis then they are mildly affected)- so is there a method for adjusting for this too?
     
    MEMarge, Ebb Tide and Andy like this.
  19. Esther12

    Esther12 Senior Member (Voting Rights)

    Messages:
    4,393
    Thanks Lucibee - so that confirms that this is not the unpublished data from school records.

    @JohnTheJack - it could be worth checking if this data was omitted by Bristol in error?
     
    MEMarge and arewenearlythereyet like this.
  20. JohnTheJack

    JohnTheJack Moderator Staff Member

    Messages:
    4,350
    I'm still not clear that there is anything missing. I understood @Lucibee to be saying that all the data are there.

    Perhaps you could say exactly what in terms of my request you think is missing.

    I asked for:

     
    Hutan, Esther12 and Andy like this.

Share This Page