O'Dowd-Crawley early intervention study

JohnTheJack said:
Thanks. Interesting. Does the dataset (such as it is) look complete to you, as in, they have given us all there is to be had?
Click to expand...
Doesn't appear to be. The protocol lists 4 primary outcomes:
  1. Feasibility (no data but conclusion is no)
  2. Suitability and acceptability of health economic measures (no data)
  3. Qualitative information about patients' views (no data)
  4. Information on the # of patients engaged and retained (?)
The spreadsheet only contains the secondary outcomes of the usual set of psychometric questionnaires. Aim was for 100 participants, only provided incomplete data for 44. It's also a comparison with "standard care" but we have no idea what that was, as usual not controlled for.

The "intervention" being a hodge-podge of the standard behavioral approach seems to indicate it serves no benefits to whatever this patient population is:
The EI has been adapted from the treatment approach delivered by the Frenchay CFS/ME service which follows NICE guidelines for CFS/ME (August 2007). It focuses on strategies to improve sleep (sleep hygiene) and balance activity using activity diaries. This includes making sleep and rest routine; monitoring the type and amount of activity undertaken every day; helping to develop consistent daily activity levels; and includes components of CBT, graded Exercise Therapy and activity management in a pragmatic approach delivered by the therapist. CBT will be used to explore barriers to progression, as fatigue and disability can be perpetuated by fearful cognitions, avoidance of perceived risky situations, all or nothing behaviour, inappropriate beliefs about rest and sleep, and focussing on symptoms only. A oneto-one model of delivery will best suit the individualised cognitive and behavioural responses identified.
Click to expand...
None of the data actually relate to any of this so basically the "intervention" and what is measured have about nothing to do with one another.

Looking at the protocol and noticed this interesting bit of... uh... "special" logic:
All primary and secondary outcomes are self rated to minimise observer bias.
Click to expand...
Another framing of the primary outcomes:
The primary outcome is recruitment to the trial, adherence and follow-up rates.
Click to expand...
There is no data on adherence of follow-up rates beyond missing data but there are no details about why data are missing. There is supposed to be a 6 months qualitative study including GP interviews, no data on this.
With informed consent of the participants, the interviews will be digitally recorded, transcribed, anonymised and analysed using standard thematic methods of building codes into themes and sub-themes using the process of constant comparison.
Click to expand...
Unclear whether the interviews even happened and, if not, why they did not happen.

There are various uses of generic fatigue, chronic fatigue and CFS. The data key defines this trial as being a CFS cohort:
These data are the results of a feasiblity randomised controlled trial in primary care to prevent Chronic Fatigue Syndrome (CFS) in adults (age ≥18 years).
Click to expand...
But the criteria are clearly not a valid CFS cohort:
1. Adult patients (over 18) presenting with an unexplained primary complaint of fatigue, as a new episode, lasting more than one month.
2. Patient has given written informed consent.
3. The participant has a Chalder Fatigue score >6
Click to expand...

The health economics analysis seems to have been dropped. Seems similar in thinking to the one done for PACE.

I think most of the relevant details would be in the TMG notes, just like with PACE:
The study will be managed by a Trial Management Group (TMG) which will meet every 4 to 6 weeks to monitor recruitment and data collection. The Trial Management Group will consist of: Dr Hazel O’Dowd (PI) a member of BRTC (trials unit) and coapplicants. A Trial Steering Committee (TSC) will also meet at the start and 6 monthly. The TSC will have an independent chair and will include experts in CFS/ME, General Practice and research methodology as well as two patient members. The group will monitor the progress of the trial, recruitment rate, the budget statements and any issues of clinical governance.
Click to expand...
 
One notable thing here is looking at the description, the intervention is basically the current behavioral paradigm, meaning it found the current paradigm to not be feasible. It's also very similar to the PACE model, which is a bit awkward.

Good reason not to publish. Not sure what they did different, however, to get people to tell them off. Maybe too loose of a cohort, not disabled enough to swallow the BS? Then again it seems that in most of those studies they carefully select their participants to maximize the outcomes they are seeking.
 
rvallee said:
Most of the variables are useless, basic psychometric questionnaires and basically the same set as in every other Chalder research. There are only two partially relevant data points: SF-36 and hours missed from work. The latter is hard to interpret as it's compared to what? Are they all people who used to work 40h normal weeks before? Few actual changes at 6 months. It's the only objective measurement and somehow it manages to be mostly subjective and open to interpretation. All but 5 rated as having missed fewer than 10h in the past 7 days, with most actually rating 0. There is an outlier that "missed" 44h of work at baseline and "missed" 56h of work at 6 months. So I guess someone working 56h week having dropped to 12 then 0? Really not sure how to interpret that.
Click to expand...

The raw results of scientific research of this nature is rarely clean, simply because participants can just answer whatever they want. All you can hope for is useful numbers in the aggregate...
 
rvallee said:
One notable thing here is looking at the description
Click to expand...
The EI has been adapted from the treatment approach delivered by the Frenchay CFS/ME service which follows NICE guidelines for CFS/ME (August 2007). It focuses on strategies to improve sleep (sleep hygiene) and balance activity using activity diaries. This includes making sleep and rest routine; monitoring the type and amount of activity undertaken every day; helping to develop consistent daily activity levels; and includes components of CBT, graded Exercise Therapy and activity management in a pragmatic approach delivered by the therapist. CBT will be used to explore barriers to progression, as fatigue and disability can be perpetuated by fearful cognitions, avoidance of perceived risky situations, all or nothing behaviour, inappropriate beliefs about rest and sleep, and focussing on symptoms only. A oneto-one model of delivery will best suit the individualised cognitive and behavioural responses identified.
Click to expand...
Guiding source of AfMEs pacing leaflet and other info/ literature.
 
rvallee said:
Doesn't appear to be. The protocol lists 4 primary outcomes:
  1. Feasibility (no data but conclusion is no)
  2. Suitability and acceptability of health economic measures (no data)
  3. Qualitative information about patients' views (no data)
  4. Information on the # of patients engaged and retained (?)
The spreadsheet only contains the secondary outcomes of the usual set of psychometric questionnaires. Aim was for 100 participants, only provided incomplete data for 44. It's also a comparison with "standard care" but we have no idea what that was, as usual not controlled for.

The "intervention" being a hodge-podge of the standard behavioral approach seems to indicate it serves no benefits to whatever this patient population is:

None of the data actually relate to any of this so basically the "intervention" and what is measured have about nothing to do with one another.

Looking at the protocol and noticed this interesting bit of... uh... "special" logic:

Another framing of the primary outcomes:

There is no data on adherence of follow-up rates beyond missing data but there are no details about why data are missing. There is supposed to be a 6 months qualitative study including GP interviews, no data on this.

Unclear whether the interviews even happened and, if not, why they did not happen.

There are various uses of generic fatigue, chronic fatigue and CFS. The data key defines this trial as being a CFS cohort:

But the criteria are clearly not a valid CFS cohort:


The health economics analysis seems to have been dropped. Seems similar in thinking to the one done for PACE.

I think most of the relevant details would be in the TMG notes, just like with PACE:
Click to expand...

OK, thanks. I don't think there is any chance of getting the TMG minutes. I think it's these data and that's it.
 
adambeyoncelowe said:
Oh, I forgot this is on adults! That would make sense.

Are there really no zeros (for unemployed)?
Click to expand...
I tried to find references from the protocol and the only thing that could fit looking at the WPAI is the question of whether you are currently employed. It's the only binary question so I guess the options 1-2 stand for 0-1. All the 2's at baseline have empty data for # of hours missed from work so it looks like 1=employed and 2=unemployed.

http://www.reillyassociates.net/WPAI_GH.html
 
rvallee said:
I tried to find references from the protocol and the only thing that could fit looking at the WPAI is the question of whether you are currently employed. It's the only binary question so I guess the options 1-2 stand for 0-1. All the 2's at baseline have empty data for # of hours missed from work so it looks like 1=employed and 2=unemployed.

http://www.reillyassociates.net/WPAI_GH.html
Click to expand...
Ah, so it's not part-time/full-time. It's unemployed/employed. Which is ridiculous, because someone doing 5 hours a week will be classed the same as someone doing 40.
 
JohnTheJack said:
'While the ICO do not make formal adjudications, except where a formal complaint is lodged, the ICO confirmed they were satisfied that NBT had applied the regulations appropriately and undertaken appropriate due diligence in reaching the decision to disclose the information.'

And here is the complete dataset, I think.

1. I would be grateful if someone could confirm that this is everything. (Maybe @Lucibee if you have the time?)

2. I shall then thank them for it and ask for the Information Booklet. Is there anything else?

3. I would be interested in writing this up in the hope of publication. Is there anyone who would be interested in working with me? I couldn't do the analysis. Would anyone want to do that bit in particular?
Click to expand...

Sorry - had no phone/internet for the past few days - fallen tree took out our line.

I've had a quick look. Better this time, but still missing WPAI qs 3 and 4. They may say that they only gave 6-month results for qs 2 and 5 in table 3, but they have provided data for qs 1 & 6 (and bl summary data for qs 3&4), which weren't reported, so I don't think that's a valid excuse (if they give it as one).

In particular, it is useful to know the answers to q4 (How many hours did you actually work in the past 7 days?), because at 6 months it would give an indication of whether pts were reducing their hours to better manage their condition - which wouldn't necessarily show up in q2 (Hours missed).

I presume that the answers to q1 (Are you in paid work?) are 1=Yes, 2=No - which is the opposite way round to some versions of the form.

The txt file explaining the data fields is very poor. It really should give min and max possible values for each variable (where relevant) - but hey, can't have everything, and most of them are guessable.

Analysis-wise - the main thing would be to confirm the data they have already published, which should be fairly straightforward.

eta: I can confirm that data does seem to match the summary data they provided. However, important caveat is that they have summarised baseline medians (IQRs) and then mean (95% CI) outcomes, which given the data characteristics, I would take with a hefty pinch of salt (makes it look much worse than it is): outcome data for hours missed (WPAI Q2) is based on a few pts with large values - medians would have been more representative.
 
Last edited:
Lucibee said:
Sorry - had no phone/internet for the past few days - fallen tree took out our line.

I've had a quick look. Better this time, but still missing WPAI qs 3 and 4. They may say that they only gave 6-month results for qs 2 and 5 in table 3, but they have provided data for qs 1 & 6 (and bl summary data for qs 3&4), which weren't reported, so I don't think that's a valid excuse (if they give it as one).

In particular, it is useful to know the answers to q4 (How many hours did you actually work in the past 7 days?), because at 6 months it would give an indication of whether pts were reducing their hours to better manage their condition - which wouldn't necessarily show up in q2 (Hours missed).

I presume that the answers to q1 (Are you in paid work?) are 1=Yes, 2=No - which is the opposite way round to some versions of the form.

The txt file explaining the data fields is very poor. It really should give min and max possible values for each variable (where relevant) - but hey, can't have everything, and most of them are guessable.

Analysis-wise - the main thing would be to confirm the data they have already published, which should be fairly straightforward.

eta: I can confirm that data does seem to match the summary data they provided. However, important caveat is that they have summarised baseline medians (IQRs) and then mean (95% CI) outcomes, which given the data characteristics, I would take with a hefty pinch of salt (makes it look much worse than it is): outcome data for hours missed (WPAI Q2) is based on a few pts with large values - medians would have been more representative.
Click to expand...

Thanks very much for that.

I saw about your connection. Hope everything is OK and no other damage and it's all repaired. The wilds of Wales.

So I should ask for the information booklet and the results for WPAI questions 3 and 4. Is that correct?

Namely:
'Median hours missed from work due to other reasons in the past seven days' and 'Median hours worked in the past seven days'.
 
JohnTheJack said:
Thanks very much for that.

I saw about your connection. Hope everything is OK and no other damage and it's all repaired. The wilds of Wales.

So I should ask for the information booklet and the results for WPAI questions 3 and 4. Is that correct?

Namely:
'Median hours missed from work due to other reasons in the past seven days' and 'Median hours worked in the past seven days'.
Click to expand...

The individual data would be preferable to just the results, if poss (don't mention "medians" unless you just want the summary data).

Q3 should be: "During the past seven days, how many hours did you miss from work because of any other reason, such as vacation, holidays, time off to participate in this study?"
Q4 should be: "During the past 7 days, how many hours did you actually work?"

So, best to ask for "Hours missed from work due to other reasons in the past seven days" and "Hours worked in the past seven days" to be safe. [eta]

It's been a fun few days! Several neighbours had their phones and routers fried by a lightning strike a few hours before the tree fell, so Openreach had their work cut out!
 
Last edited:
Lucibee said:
The individual data would be preferable to just the results, if poss (don't mention "medians" unless you just want the summary data).

Q3 should be: "During the past seven days, how many hours did you miss from work because of any other reason, such as vacation, holidays, time off to participate in this study?"
Q4 should be: "During the past 7 days, how many hours did you actually work?"

It's been a fun few days! Several neighbours had their phones and routers fried by a lightning strike a few hours before the tree fell, so Openreach had their work cut out!
Click to expand...

Ah, yes, I was using the format from the data summary. I'll ask for that then.

Yeah, I can imagine. They have been busy. Hope it's all good now.
 
Lucibee said:
eta: I can confirm that data does seem to match the summary data they provided. However, important caveat is that they have summarised baseline medians (IQRs) and then mean (95% CI) outcomes, which given the data characteristics, I would take with a hefty pinch of salt (makes it look much worse than it is): outcome data for hours missed (WPAI Q2) is based on a few pts with large values - medians would have been more representative.
Click to expand...

IQRs is usually a sign of heavily skewed data. medians would indeed be helpful, or indeed some way of visualising the distribution of results.

Still disappointing that the data set is still missing some of the most interesting questions.
 
Snow Leopard said:
Still disappointing that the data set is still missing some of the most interesting questions.
Click to expand...
Which happen to be, as usual, the primary outcomes. Which makes it at least the 5th or so time this happened with a PACE researcher? And still they continue to get funding where they submit primary outcomes and a methodology, essentially a contract, and bury them when they can't massage the data into something mildly convincing for the usual outcome they have pre-determined for 3 decades by now.

Because there was a conclusion, which is that PACE-style trials (and the current guidelines) are not feasible, based on non-acceptance by the participants, which was the primary outcome. Surely if they concluded it is not feasible they did do their interviews and ask those primary outcome questions, which were described in protocol as being produced as anonymized data.

So where are the data, Lebowski? Because they sure are somewhere, paid for by public money, having found that the current paradigm is so absurdly bad people who aren't desperate enough, whoever this cohort may be, find it useless. Or maybe? Who knows what they said? Those are just the primary outcomes submitted as part of a publicly-funded feasibility study so it's not as if there was any way of knowing what people think of the current, highly controversial, "medical" paradigm, which somehow does not include much that is actually relevant to medicine and rather is closer to The Secret and life coaching fluff where alternative medicine practicioners also report that their clients feel better but somehow this is different because reasons.
 
Snow Leopard said:
IQRs is usually a sign of heavily skewed data. medians would indeed be helpful, or indeed some way of visualising the distribution of results.
Click to expand...

Here's the WPAI Q2 data:

WPAI Q2: Hours missed due to illness in past 7 days
Controls
Baseline: 5 of 13 missed any hours (3, 4, 12, 16, 19). [median 0, IQR 0-4]
6 months: 3 of 11 missed any hours (2, 9, 9). [median 0, IQR 0-0]

EI group
Baseline: 7 of 18 missed hours (0.5, 1*, 2, 4, 8*, 10*, 44). [*subsequently dropped out] [median 0, IQR 0-2]
6 months: 4 of 15 missed hours (2, 5, 16, 56). [median 0, IQR 0-0]

Looking for a difference on the basis of means at 6 months is extremely unsafe, and would give a false impression that the EI group have done statistically worse, when there isn't enough information to say that.

However, Q4 would give useful info on the hours actually worked, and whether pts had reduced their hours during the 6 months to compensate.
The outlier (44 and 56 hours in one pt) needs investigation (typo?), particularly as the EQ and SF36 data for this pt were low.

tbh, with data like these, I'm not surprised they stopped the study. I suspect there will be something in qualitative data that confirms that. But if there was any indication that early intervention might pose harm to pts, they really should be writing it up. Unfortunately, there is probably not enough data from controls to enable a proper comparison.
 
Last edited:
You must log in or register to reply here.
Back
Top Bottom