GPT for ME/CFS Questions

Initial posts have been moved from another thread.

I just noticed that now GPT-models pre trained on data are available for free users too. Is someone up to the task of training one on Me-pedia, trial by error, and science for ME (if members consent). It would be so much more useful for us than vanilla chatgpt. I’m only half joking with the proposal.

edit: forestglip knows more about this than me, please refer to the message below mine.

 

By pre-trained, do you mean fine tuned, or are you talking about the custom GPTs, which is basically just giving it default prompts and attachments. The former would be ideal as you can give it unlimited data, and I've thought a lot about that possibility, but it's still very expensive, as far as I know.

The second is good if the amount of data isn't too big. And it returns much more accurate answers. But all of MEpedia might be too much.

 

hmmm, maybe using the mayo clinic primer and a couple of solid reviews added with some select me-pedia pages for context?

 

That is a really good idea. I didn't know it was free now, but if so I'll see what I can do later.

 

Thinking about the first option, yeah, it would only be affordable with GPT-2 or GPT-3, even GPT-3.5 via the API key would be stretching it

 

amazing! if it looks feasible, maybe we can create a thread to discuss it

 

The science library in this forum also has a couple of shortish posts that would be relevant, I’m specifically thinking about the naming one, but others are good too.

 

It's not the only option, but OpenAI has a system to train specialist models. I don't know what it involves but it should be possible to do something like this soon enough. So the idea already exists, I just don't know how much work would be involved and how much it would cost.

And if it's complicated for now, one rule with AI models is that whatever limitations and complications is involved in using them is likely to be solved not much later than the time it'd take to work around those. So it's best to wait until it's easy to do it. Which kind of works for us, as if it's too much work, we can't do it anyway.

The next big thing is like to be what's called agents. It mostly amounts to being more autonomous, capable of solving intermediary steps and doing more than simply responding to prompts. So at this point a capable agent could simply be given the task of building a specialist model using the best resources out there.

 

(note - earlier posts in this thread have been moved from the linked thread)

Continuing on with the conversation from another thread, I set up a custom GPT, named MEGPT for now, to be able to give it some default relevant documents so that it can answer questions about ME/CFS effectively.

Accessible at this link, though you'll need a ChatGPT account: https://chatgpt.com/g/g-AvMmt4r3f-megpt

So far it only has the Mayo Clinic Proceedings Diagnosis and Management webpage.

I'll put all the files that it has access to on Google Drive: https://drive.google.com/drive/folders/1FaBFOiUYZCXmmnM1RcS8lcvqyg0rfDaE?usp=drive_link

The data limit is actually larger than I had thought, because it's using something called embeddings, which means it stores a bunch of chunks of the documents and then only searches for and uses the relevant parts for answering questions. It says up to 2 million tokens per file, and up to 20 files. 2 million tokens is somewhere around 1.5 million words. So it might actually be able to use all of MEpedia. Though I'm not quite sure how I'd turn it into text files or if I'd even be allowed to.


Here are the custom instructions I gave it:

Although it seems to be ignoring the "include citations" part.

Testing difference in answers between this GPT and regular ChatGPT:

"What treatments exist for ME/CFS?"

MEGPT:

ChatGPT:

 

Though it's free for others to use, I had to make a Plus account for $20 per month to create it. I'm not a big fan of paying OpenAI, and I would prefer to do this with Anthropic Claude, as they seem to care more about AI safety, but as yet, Claude doesn't have anything similar to custom GPTs, but hopefully in the future.

 

Okay, this is really cool. Instead of downloading all of MEpedia, which would have its advantages, but would be pretty difficult, I wouldn't get updates, and I'm not sure I'm allowed, I instead made the GPT able to access the live MEpedia API. So basically it can browse the wiki to answer questions. When you ask it a question, first it uses the search to find the most relevant pages, then it goes to those pages to get the answer.

New instructions:

Example conversation:

For the three responses, here were the searches and pages it automatically used to get the information:

1.
Search: "ME Action founder"
Read: "The MEAction Network"

2.
Search: "dextro naltrexone"
Read: "Dextro-naltrexone"

3.
Search: "Incline Village outbreak"
Read: "1984 Incline Village chronic fatigue syndrome outbreak"

Honestly, kind of scared at how well AI already works and how easily I could put it together.

Edit: It looks to not be able to get pages that are too long, such as the Pace Trial page. Which lol even has a note on the page that it needs to be shortened.

 

@forestglip this is so cool!

I gave your model a badly written article about ME, the kind you often see on health websites, and asked it to make bullet points detailing errors, it was pretty well done, to the point that with some fine tuning and proofreading it would be emailable to article creators.

 

I’m wondering if you could either add this document for the chatgpt model: https://www.s4me.info/threads/the-name-question.332/ and/or ask it in instructions to prioritise the naming ME/CFS (or ME)

 

@Jenny TipsforME tried to do this

Members only - Meet Grace the new ME advocacy chatbot | Page 3 | Science for ME (s4me.info)

 

I haven't the faintest idea how any of this works, but can you also ask it to prioritise other fairly reliable sources? Like the 2021 NICE guideline?

If so, maybe it could even learn about the Larun review problem...

 

If a moderator could chime in to let me know if I can use posts from the library, then sure. For now, I added the Wikipedia page for ME/CFS which talks about naming, and added a bit in the instructions to use ME/CFS over CFS.

The three files it has now are Mayo Clinic Proceedings, the Wikipedia page, and the NICE guidelines, plus access to all of MEpedia.

Current instructions:

Hmm not sure what this is.

 

See the threads on the Cochrance Exercise Review…

I wonder how we could give the chatmodel some much needed scepticism.

 

Oh yeah, very similar! Looks like she put a lot of work into it. I think since then the GPT Actions have been released which allows direct access to other websites' APIs (like MEpedia) which makes things a lot easier than downloading lots of MEpedia, which it looks like she was doing. I haven't looked at all the posts in detail, but there's probably some good ideas there to add to this.

Also the file limit has been increased by a lot since then.

 

Looks like there's the issue of many MEpedia articles being out of date. I wonder if MEAction could hire people to write for the wiki. I think there are paid writers for certain topics on Wikipedia. (Paid by third parties, not by Wikimedia.)

"please tell me about decodeme timelines"