Cochrane Exercise Review Withdrawn - Individual Patient Data

They’ve been going on about this IPD meta analysis of exercise since we were still on the other forum. It’s going to suffer from the same methodological issues as conventional meta analysis based on summary data. IPD makes no difference. I’d say they’ll probably just submit it to a regular journal and forget about Cochrane. Another thing to look forward to. :rolleyes:
 
Sid said:
They’ve been going on about this IPD meta analysis of exercise since we were still on the other forum. It’s going to suffer from the same methodological issues as conventional meta analysis based on summary data. IPD makes no difference. I’d say they’ll probably just submit it to a regular journal and forget about Cochrane. Another thing to look forward to. :rolleyes:
Click to expand...
yes, they will do that. I wonder which journal they'll pick. hopefully there will be a pre-print which we can have a look at and critique!
 
Sid said:
They’ve been going on about this IPD meta analysis of exercise since we were still on the other forum. It’s going to suffer from the same methodological issues as conventional meta analysis based on summary data. IPD makes no difference. I’d say they’ll probably just submit it to a regular journal and forget about Cochrane. Another thing to look forward to. :rolleyes:
Click to expand...
Yea, I don't know what "IPD meta analysis" is but check out this*. Basic problem is the base studies are so poor [unblinded & lack of objective outcome indicators] that you cannot derive reliable data from them "garbage in, garbage out"*--- so why don't the authors just "own up" to that?
@Caroline Struthers
*
https://www.s4me.info/threads/germa...ort-out-now-may-2023.21266/page-7#post-448824
 
FMMM1 said:
Yea, I don't know what "IPD meta analysis" is but check out this*. Basic problem is the base studies are so poor [unblinded & lack of objective outcome indicators] that you cannot derive reliable data from them "garbage in, garbage out"*--- so why don't the authors just "own up" to that?
@Caroline Struthers
*
https://www.s4me.info/threads/germa...ort-out-now-may-2023.21266/page-7#post-448824
Click to expand...

IPD meta analysis just means analysing individual participant data from original studies (pooling them all together) instead of the usual meta-analytic approach of using summary data from published studies (eg means, standard deviations) to pool the results of many studies. It can’t overcome the methodological problems of original studies. It just compounds them into one big stew.
 
How does an IPD meta analysis work?

Does it focus on the probably few patients with good results and claim the improvement can be attributed to the intervention?
 
Hoopoe said:
How does an IPD meta analysis work?

Does it focus on the probably few patients with good results and claim the improvement can be attributed to the intervention?
Click to expand...
Their protocol describes what they intend to do with the individual patient data:
https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42018107473
They are dividing the studies into 3 batches according to what sort of comparitor/control group they had. Then, for each batch of studies, the data from fatigue scales used in each study is standardised to enable them to put all the data together and analyse it as if it was a single trial.
 
Sid said:
IPD meta analysis just means analysing individual participant data from original studies (pooling them all together) instead of the usual meta-analytic approach of using summary data from published studies (eg means, standard deviations) to pool the results of many studies. It can’t overcome the methodological problems of original studies. It just compounds them into one big stew.
Click to expand...

How do they deal with differences in the interventions? Do these get coded in some sense and linked in a model or do they just ignore the differences?
 
Why are they doing IPD analysis then? It sounds like IPD would make it easier to obtain statistically significant positive results. But didn't the previous review already obtain them? Maybe they're hoping to get positive results on wider range of outcomes?

Obviously it doesn't do anything to make the results credible because the data was obtained in a context of no blinding and probably inadequate control with an intervention that introduces bias.
 
Last edited:
I expect they will ignore any inconvenient differences in the type of CBT used in the different trials. Past experience with the out of date Cochrane reviews is they take everything at face value, and just crunch the numbers, despite them being meaningless.
 
Something tells me that such an analysis would be very bad for them, and probably explains why they tried making one that was so biased that it ended up being withdrawn.

There is no way it makes the trial looks good. But they did get away with overlapping entry and recovery thresholds so who even knows what else they could be allowed to do here.
 
Sid said:
They get combined if deemed similar enough.
Click to expand...

Then the question is how they determine similarity. From what I remember of the IPD protocol they don't talk about determining similarity at all - no notion of examination, measuring similarity or thresholds
 
Adrian said:
Then the question is how they determine similarity. From what I remember of the IPD protocol they don't talk about determining similarity at all - no notion of examination, measuring similarity or thresholds
Click to expand...

It apparently doesn't take much to decide they are similar.
In the 2014 systematic treatment evidence review conducted in the US, the authors pooled all Counseling and Behavior Therapies (including CBT), together and then concluded that, collectively, these therapies improved fatigue, function, quality of life, and global improvement.

It took many months and letters to get them to agree to reanalyze the evidence after separating out CBT from other forms of counseling and also separating out Oxford trials.

In the 2016 reanalysis addendum, the authors explained their rationale for originally combining the counseling and behavior trials as follows:
"Although there is enough variability amongst the other techniques [other than CBT] that a meta-analysis might be inappropriate, all involve supportive guidance aimed at improving coping strategies and reducing impact of one’s disease state on overall well-being justifying the decision to consider these together as a group. Although CBT is a unique approach with disputable underlying rationale regarding the fear avoidance theory contributing to the perpetuation of symptoms in ME/CFS, it has similar aims of improving coping strategies and improving overall well-being. We considered this as justification for our original approach of combining all interventions when determining the overall strength of evidence of this body of literature."


edited to correct spelling
 
Last edited:
Medfeb said:
"Although there is enough variability amongst the other techniques [other than CBT] that a meta-analysis might be inappropriate, all involve supportive guidance aimed at improving coping strategies and reducing impact of one’s disease state on overall well-being justifying the decision to consider these together as a group. Although CBT is a unique approach with disputable underlying rationale regarding the fear avoidance theory contributing to the perpetuation of symptoms in ME/CFS, it has similar aims of improving coping strategies and improving overall well-being. We considered this as justification for our original approach of combining all interventions when determining the overall strength of evidence of this body of literature."
Click to expand...

So what they are not saying is that the did a review of the actual CBT manuals to see what they were suggesting and looked to see if they are similar. Instead they seem to say that they have the same underlying theory thus they are the same. I wonder how such statements would translate to drugs (roughly the same chemicals - ok in different doses but its fine to combine).
 
If no sub-group analyses (with sufficient stats power) are reporting clinically significant benefit, as NICE showed, then how can the aggregate data. 0 + 0 = 0.

Although CBT is a unique approach with disputable underlying rationale regarding the fear avoidance theory contributing to the perpetuation of symptoms in ME/CFS, it has similar aims of improving coping strategies and improving overall well-being.

So is it directive or supportive? Or do those previously critical distinctions and their causal implications, as asserted long and loudly by you guys, not matter now? Just going to forget all those definite claims about cause and recovery and cure, and go with some vague, generic, clinically insignificant fluff about feeling a bit better?

Do I laugh or cry?
 
Adrian said:
Instead they seem to say that they have the same underlying theory thus they are the same.
Click to expand...
Or maybe enough to just be similar on intended outcomes - "improving coping strategies and improving overall well-being."
 
Adrian said:
So what they are not saying is that the did a review of the actual CBT manuals to see what they were suggesting and looked to see if they are similar. Instead they seem to say that they have the same underlying theory thus they are the same. I wonder how such statements would translate to drugs (roughly the same chemicals - ok in different doses but its fine to combine).
Click to expand...
but they arent even saying they have the same underlying theory. Far from containing 'roughly the same chemicals', differing psychotheraputic orientations often stem from significantly different underlying theories (eg cognitive behavioural, psychoanalytical & humanistic therapies are chalk & cheese is many ways). They are saying that they had the same aims.

Which really is very silly to combine treatments based on the same aims. I mean imagine combining results from 5 different painkillers simply because they were all aiming to reduce pain.
 
Medfeb said:
Or maybe enough to just be similar on intended outcomes - "improving coping strategies and improving overall well-being."
Click to expand...
It really doesn't make sense to combine things in that way.

What it really shows is the weakness of their method which seems to lack any process for looking at what they are combining.
 
You must log in or register to reply here.
Back
Top Bottom