Putting the PASC Score to the Test: Clinical vs. Statistical Accuracy in Long COVID Diagnosis, 2025, Azola et al

forestglip

forestglip

Administrator
Staff member
Now published, see post #9
---------

919 – Putting the PASC Score to the Test: Clinical vs Statistical Accuracy in Long COVID

Alba M. Azola, Leah H. Rubin, Rebecca E. Easter, Rebecca Veenhuis, Hannah Parker, Christina Della Penna, Holly Schultz, Isabel Santiuste

[Line breaks added]

Background
Long COVID (LC) is a mass disabling event affecting millions worldwide. Given the broad definitions and lack of biomarkers, there is an urgent need for diagnostic tools to identify those affected.

Here we aimed to validate the RECOVER Post-Acute Sequelae of SARS-CoV-2 Infection (PASC) Score in a cohort of SARS-CoV-2 infected patients with LC and fully recovered individuals while iteratively improving the tool’s sensitivity and specificity.

Methods
Participants included 100 LC patients from LC clinics in Baltimore, MD between August 2023 and July 2024 who met the National Academy of Medicine(NAM) 2024 LC definition, and 18 SARS-CoV-2 infected but fully recovered individuals.

LC participants were required to have least one neuropsychiatric symptom (e.g., brain fog). Exclusion criteria included history of psychosis, recent substance misuse(nicotine, cannabis excluded), and lack of English proficiency.

Participants completed comprehensive surveys and questionnaires assessing symptoms based on the methods of the PASC score publication.

Using the NAM 2024 LC definition as the ‘true’ condition, we compared evaluation metrics for the REVOVER PASC score cutoff(PASC Total >12) as well as comparing the presence of individual, pairs, and triplets of symptoms. Evaluation metrics(e.g., sensitivity, specificity, F1) were calculated based on these classifications for the overall PASC score and symptom combinations.

Results
The LC cohort(n=100) had a mean age of 47.7 years, was predominantly female(73%), White(78%), and well-educated(76% >16 years). Controls(n=18) had similar demographic characteristics.

LC diagnosis and PASC scores were significantly associated(χ2=44.72, P<0.001). The PASC score showed excellent specificity(100%) and positive predictive value(PPV; 100%) albeit limited sensitivity(80%), missing approximately 20% of the patients with LC. The negative predictive value(NPV) was 47.37%, indicating that only 47% of those who tested negatively via PASC score did not have LC.

When examining whether combinations of symptoms performed better than the total PASC score cutoff, we found that the presence of loss of smell/taste, post-exertional malaise, or brain fog demonstrated 93% sensitivity, 100% specificity, and PPV, 72% NPV, and an F1 score of 0.964.

Conclusions
Validation of the RECOVER PASC supports its utility and highlights the need for ongoing refinement of the LC definition. We call for national efforts to create and validate a readily implementable clinical tool for LC diagnosis.

Link (Conference on Retroviruses and Opportunistic Infections) [Abstract Only]
 
Last edited:
Just to illustrate its real world relevance for anyone that might not know, if you're a patient that gets told "No you don't have long COVID based on this score", negative predictive value (NPV) tells you the chance that you really don't have long COVID.

But the percent only makes sense if its based on the real prevalence of long COVID in the population, which this study's NPV is not.

As Wikipedia says:
Note that the positive and negative predictive values can only be estimated using data from a cross-sectional study or other population-based study in which valid prevalence estimates may be obtained. In contrast, the sensitivity and specificity can be estimated from case-control studies.
Click to expand...
 
Putting the PASC Score to the Test: Clinical vs. Statistical Accuracy in Long COVID Diagnosis

Azola, Alba; Dastgheyb, Raha M.; Easter, Rebecca; Parker, Hannah; Della Penna, Christina; Santiuste, Isabel; Schultz, Holly; Ehrenspeck, Ana; Veenhuis, Rebecca; Rubin, Leah H.

Objective
To validate the RECOVER Post-Acute Sequelae of SARS-CoV-2 infection (PASC) score in a cohort of patients who develop long COVID (LC) or fully recover while iteratively improving the tool’s sensitivity and specificity.

Methods
A cross-sectional study in 130 LC patients followed at LC clinics in Baltimore, MD, USA, who met the National Academies of Sciences, Engineering, and Medicine (NASEM) 2024 LC definition, and 60 SARS-CoV-2 exposed but fully recovered individuals.

LC participants were required to have at least one neuropsychiatric symptom. Participants completed comprehensive surveys and questionnaires assessing symptoms based on published methods to determine PASC score.

Using the NASEM 2024 LC definition as the “true” condition, we compared evaluation metrics for the RECOVER PASC score cutoff ( PASC > 12) and the presence of individual/multiple symptoms. Evaluation metrics (e.g., sensitivity, specificity, F1) were calculated based on these classifications for the overall PASC score and symptom combinations.

Results
The LC cohort ( n = 130) had a mean age of 47.2 years and was predominantly female (72%), White (79%), and well-educated (77% > 16 years). Controls ( n = 60) were similar demographically.

LC diagnosis and PASC scores were significantly associated ( χ 2 = 102.99, P < 0.001). The PASC score showed excellent specificity (100%) and positive predictive value (PPV; 100%) albeit limited sensitivity (80%), missing 20% of participants with LC.

We found that loss of smell/taste, post-exertional malaise, or lack of sexual desire or capacity demonstrated 94% sensitivity, 92% specificity, and 96% PPV, 87% NPV, and an F1 score of 0.949.

Conclusion
Validation of the RECOVER PASC supports its utility and highlights the need for ongoing refinement of the LC definition. We call for national efforts to develop readily implementable clinical tools for LC diagnosis.

Web | DOI | PDF | Journal of General Internal Medicine
 
What do you expect to happen when you recruit two different groups, one based on positively having symptoms and one based on positively not having symptoms?

Of course a questionnaire asking about symptoms will do a pretty good job at differentiating them. That doesn’t mean the questionnaire is any good or has any practical value.
 
I can't find the letter by Goldman and Martin that the authors are responding to here:

Letter to Editor External Clinical Validation of the RECOVER Research Index: A Response to Goldman and Martin, 2026, Azola et al

Azola, Alba; Veenhuis, Rebecca T.; Rubin, Leah H.

Dear Editor:

We thank Drs. Goldman and Martin for their thoughtful comments and careful consideration of our study.1 Their letter highlights important distinctions regarding the interpretation of diagnostic performance studies, and we appreciate the opportunity to clarify our study objective and refine the interpretation of our findings.

The central premise of their letter is that our study was intended as a population-based validation study of the RECOVER Long COVID research index (RECOVER PASC score). That was not the objective of our investigation.

Rather, our study addressed a different question: How well does the RECOVER PASC score classify individuals who received a clinical diagnosis of Long COVID after comprehensive evaluation in specialty clinics, compared with individuals who had documented SARS-CoV-2 infection and recovered without persistent symptoms? Our goal was to evaluate the performance of the RECOVER PASC score in a rigorously phenotyped clinical cohort of individuals evaluated for suspected Long COVID and to contribute to the ongoing independent evaluation and iterative refinement of the RECOVER PASC score.

The subsequent 2024 update of the RECOVER-Adult Long COVID research index, which incorporated additional participant data, expanded symptom ascertainment informed by patient-community input, and a revised symptom-weighting model and threshold, further demonstrates that the index is intended to evolve as new evidence becomes available.2Our study should be interpreted as an external clinical validation of the RECOVER PASC score within a well-characterized specialty referral cohort rather than as an assessment of its performance in the general population.

The use of a specialty referral cohort defines the population to which the findings apply; it does not diminish the value of understanding how the RECOVER PASC score performs in the clinical setting in which patients with persistent post-COVID symptoms are evaluated. Our study was not designed to establish the diagnostic performance of the RECOVER PASC score in the general population or among patients with alternative symptom-overlapping conditions.
...
Click to expand...

Web | DOI | PDF | Journal of General Internal Medicine
 
Chandelier said:
I can't find the letter by Goldman and Martin that the authors are responding to here:

Letter to Editor External Clinical Validation of the RECOVER Research Index: A Response to Goldman and Martin, 2026, Azola et al

Azola, Alba; Veenhuis, Rebecca T.; Rubin, Leah H.



Web | DOI | PDF | Journal of General Internal Medicine
Click to expand...
link.springer.com

RE: Putting the PASC Score to the Test: Clinical vs. Statistical Accuracy in Long COVID Diagnosis (Azola et al. JGIM) - Journal of General Internal Medicine

link.springer.com link.springer.com

RE: Putting the PASC Score to the Test: Clinical vs. Statistical Accuracy in Long COVID Diagnosis (Azola et al.​


Download PDF

Dear Editor:

We commend Azola et al.1 for studying the Long COVID (LC) research classification2 developed by the Researching COVID to Enhance Recovery (RECOVER) Initiative (referred to as “PASC score”). We especially praise the authors’ willingness to consider any symptom as a potential manifestation of LC, as described by the National Academies of Science, Engineering and Medicine (NASEM).3,4 We note, however, several conceptual problems with Azola’s approach.

Regarding sensitivity of the PASC score, the optimal test
group would be a representative sample of persons with LC
(i.e., from the general population). Instead, Azola et al. drew
their sample from their LC Clinic, a population almost certainly
enriched for patients with multiple symptoms. Yet, a
PASC score indicative of LC requires multiple symptoms.2
If patients with just one symptom or several low-point value
LC symptoms had been included in the case group, sensitivity
would be considerably lower. This is an example of
spectrum bias.5 Furthermore, Azola et al. required the cases
have at least one neuropsychiatric symptom, and, indeed,
62% had brain fog (worth 3 PASC score points). Requiring
that case group participants possess a portion of the case
definition will bias towards higher sensitivity.

Regarding specificity, the optimal approach would be
to assemble a representative group of patients who have
symptoms overlapping with LC but whose etiology does not
include earlier SARS-CoV-2 infection. This assemblage is
needed to serve the relevant purpose of an LC classifier. That
is, a classifier is needed to distinguish patients who present
with any single or combination of symptoms for which the
clinician wonders if the etiology is LC versus some other
condition. In contrast, the patient group assembled by Azola
et al. to validate specificity is stated to be “fully recovered”
and “healthy controls,” thus unsurprising that no one met the
PASC score threshold for LC, yielding inflated specificity.

Lastly, test validation studies which use a case-control
design cannot assess positive and negative predictive value
because this study design does not represent the prevalence of
the underlying disease in any population of interest. Azola et al.
are correct to assess different combinations of fewer symptoms
(see supplement), yet their estimates of test characteristics for
each combination of symptoms are subject to the same biases.

In summary, Azola et al. have estimated the fraction of
patients cared for in their LC Clinic (with at least one neuropsychiatric
symptom) who meet the PASC score threshold
for LC. They have not, however, validated the PASC score
due to bias in estimates of sensitivity, specificity, and positive
and negative predictive value.

Author Contribution Drafting of the manuscript: Goldman and Martin

Funding No funding was provided for this work.

Jason D. Goldman, MD, MPH1
Jeffrey N. Martin, MD, MPH2
1Division of Allergy & Infectious Diseases,
University of Washington, Seattle, WA, USA;
2Department of Epidemiology and Biostatistics,
University of California, San Francisco,
San Francisco, USA

Corresponding Author: Jeffrey N. Martin, MD, MPH; Department
of Epidemiology and Biostatistics, University of California, San Francisco,
San Francisco, USA (e-mail: jeffrey.martin@ucsf.edu).

REFERENCES

1. Azola A, Dastgheyb RM, Easter R, et al. Putting the PASC score to the
test: Clinical vs. statistical accuracy in Long COVID diagnosis. J Gen
Int Med. 2025: https:// doi. org/ 10. 1007/ s11606- 025- 10042-6.

2. Geng LN, Erlandson KM, Hornig M, et al. 2024 Update of
the RECOVER-Adult Long COVID Research Index. JAMA.
2025;333(8):694-700.

3. National Academies of Sciences, Engineering, and Medicine. A Long
COVID Definition: A Chronic, Systemic Disease State with Profound
Consequences. Washington, DC: The National Academies
Press. 2024. https:// doi. org/ 10. 17226/ 27768.

4. Ely EW, Brown LM, Fineberg HV. National Academies of Sciences,
Engineering, and Medicine Committee on Examining the Working
Definition for Long Covid. Long Covid Defined. New Engl J Med.
2024;391(18):1746–1753.

5. Newman TB, Kohn MA. Evidence-based diagnosis: An introduction to
clinical epidemiology. Chapter 4: Critical appraisal of studies of diagnostic
test accuracy. Pages 91–93. Second edition. Cambridge University
Press, New York; 2019.
 
You must log in or register to reply here.
Back
Top Bottom