Researchers should think twice about whether the future understanding of human biology should be outsourced to private companies or whether one should set up a different system that guarantees a higher chance of benefits for all, first.
Achievements of Artificial Intelligence (AI)
- Thread starter ME/CFS Science Blog
- Start date
From the website..
“We’ll work together to publish these outputs and open-source the accompanying data and code. Guest investigators retain the rights to their data at every step during their project. They will also have the rights to any independent follow-on research that might come out of our collaboration.
We aren’t asking for guest investigators to give us their data, either in the application or during the collaboration, and we’re requesting that applicants include only publicly available information in their application materials.“
Even if that is a nice message by Anthropic, I don't see how this deals with the core issue of the problem at all. The problem remains the same.
This is just one initiative — it’s not the entire future of human biology, or asking for private data or closing off the results.
They don’t say in the description which models they will give access to for the projects. But, if they gave access to the frontier models and their own teams, it would be presumably very valuable for some scientists and some problems (maybe especially genetic data). Do you propose using local hosted models? I don’t understand exactly who designs and builds those — are they Chinese open source?
Yes, Anthropic is just doing what all the other companies are also doing. It isn't clear to me what role closing off results really plays in the long run. It seems you can happily release results and still destroy disciplines whilst doing so. Maybe for mathematics that doesn't matter in the end, but perhaps now is the right time to pay some attention to such problems in the field of medicine.
It most certainly will be valuable to solving problems and most certainly in the field of genetics. I suspect that progress in the genetics of ME/CFS would already be possible today. Various genetic questions have been asked in various threads and I suspect that many of those can be resolved with much greater ease than one may be anticipating and there are people at the AI companies that are keen on doing exactly that. I believe this will happen and it seems inevitable to me. I believe by next year we'll be discussing such results right here on S4ME.
At the same time larger questions could be at play too. If solving problems is at the cost the discipline itself for the benefit of private companies, long term outcomes become less clear.
One hope is that the open weight models are always just a bit behind and perhaps this is the case and one has to do nothing and should cooperate with everything asap, but perhaps now is also the time to ensure that things actually go in the right direction instead of just hoping they do.
Recent posts and this thread more widely are I think good examples of two areas which make AI such a thorny and contentious topic.
It’s a great example of the messiness of the world.
The jaggedness of intelligence also leads to some of the biggest risks. As I wrote here
More reading on the jagged egde of AI
https://mitsloan.mit.edu/ideas-made-to-matter/working-definitions/what-is-jagged-ai-frontier
https://yalereview.org/article/melanie-mitchell-jagged-intelligence
- There is potential for both significant good and for significant harms
- The capability of these tools is very uneven, that is they can be very good at some things and still bad at others (this is called the jagged frontier)
It’s a great example of the messiness of the world.
The jaggedness of intelligence also leads to some of the biggest risks. As I wrote here
This all clashes with our intuition but also how we build trust and extrapolate our ideas of intelligence.
More reading on the jagged egde of AI
https://mitsloan.mit.edu/ideas-made-to-matter/working-definitions/what-is-jagged-ai-frontier
https://yalereview.org/article/melanie-mitchell-jagged-intelligence
If we could build such a system. Alas, we cannot.
It's a lot like how most of the criticism over the AI bubble misses out on the point that humans simply don't know how to do these things "the right way". There is no such way. The convoluted imperfect process is necessary because we have no idea how to do it right before it's done, and even after it's done people get it all wrong anyway.
I've also seen some criticism over the coming wave of AI reviews of the entire published literature, and what it means for the peer review system to be shown where all the errors are, and how much of published literature is crap. I've seen some academics saying that everyone knows the academic system is broken, and this is just forcing the issue, so it should be done right but without AI. Except they've had decades to do that, during which "everyone knew" the system was broken, and yet nothing competent is being done about it. In fact things are getting worse, as we painfully know.
We are clothed barbarians. It's easy to forget how we are barely civilized, and frankly don't know how to do anything beyond what's right in front of us, and barely at that.
Last edited:
mariovitali
Senior Member (Voting Rights)
According to the following, AI is now better than humans in reviewing studies :
Large language models detect methodological errors that human peer review misses in clinical trials
Large language models detect methodological errors that human peer review misses in clinical trials
Peter T
Senior Member (Voting Rights)
For many of the papers reported here, spotting errors missed by peer review is hardly an achievement. Indeed on some topics in some journals the presence of methodological errors is a given.
The study says that AI is better than humans in spotting inserted errors, which is maybe a bit different. In reality I believe the much stronger statement you made also holds: That AI is better at reviewing studies in general and I think most people that have recently made submissions will tell you this as well.
But what that means is completely unclear. Mathematical journals are closing. It seems inevitable that journals in all other sciences will follow within the year. But what comes next? Nobody seems to think that it's useful to have journals where AI's review AI work, because that's unnecessarily convoluted, so what's the next step? There exists now "talk journals", but that hardly seems like it will make up a large proportion of the upcoming full picture.
That is a great question. It does remind me of what happened with news, particularly local news but then more widely, with the internet and social media. Industries which were already a bit iffy but then were pushed over the edge. And what resulted, although sold as democratisation, has not been good for democracy.
fst
Senior Member (Voting Rights)
Maybe slightly tangential, but the whole peer-review and publishing system is deeply broken.
Publishers once had a useful function, that is, actually publishing (printing) and distributing research. They have become little more than glorified PDF hosters. They get their content for free (from researchers, funded by public money), and their peer-reviews for free (from researchers) and then have the audacity to charge hundreds of Euros/Dollars for subscriptions or, if some researchers are really committed to open science, charge them hundreds of Euros to make their papers open access.
In fact, some predatory publishers made the latter their sole business model. Send us your low-quality research and a fee, and we'll organize a "peer-review" and publish it. Of course the publish-or-perish culture in academia fuels such developments.
And as you can imagine, even in reputable journals, the researchers that do peer-
review are always highly motivated to do this unpaid work and totally not overworked anyway (/sarcasm), which leads to the high quality reviews that we know and cherish.
The average peer review is about the same quality as the average paper (or worse). It's not hard to imagine that AI can improve this. Out of experience, however, there's still the occasional hallucination/sloppy suggestion that makes no sense at all and I think there needs to be a human-in-the-loop to catch these things.
The larger problem in my opinion is how to make it happen in a way that is actually beneficial for science. Most publishers, especially the low quality ones, have absolutely no incentive to provide high quality peer-reviews, it kills their business. Neither have many researchers. So I am confident, that sloppy research will keep getting produced and published.
Publishers once had a useful function, that is, actually publishing (printing) and distributing research. They have become little more than glorified PDF hosters. They get their content for free (from researchers, funded by public money), and their peer-reviews for free (from researchers) and then have the audacity to charge hundreds of Euros/Dollars for subscriptions or, if some researchers are really committed to open science, charge them hundreds of Euros to make their papers open access.
In fact, some predatory publishers made the latter their sole business model. Send us your low-quality research and a fee, and we'll organize a "peer-review" and publish it. Of course the publish-or-perish culture in academia fuels such developments.
And as you can imagine, even in reputable journals, the researchers that do peer-
review are always highly motivated to do this unpaid work and totally not overworked anyway (/sarcasm), which leads to the high quality reviews that we know and cherish.
The average peer review is about the same quality as the average paper (or worse). It's not hard to imagine that AI can improve this. Out of experience, however, there's still the occasional hallucination/sloppy suggestion that makes no sense at all and I think there needs to be a human-in-the-loop to catch these things.
The larger problem in my opinion is how to make it happen in a way that is actually beneficial for science. Most publishers, especially the low quality ones, have absolutely no incentive to provide high quality peer-reviews, it kills their business. Neither have many researchers. So I am confident, that sloppy research will keep getting produced and published.
fst
Senior Member (Voting Rights)
Yes. I think a reasonable approach would be to have one or two AI reviews, and then some human reviewers as usual. Reviewers don't see the other reviews until they submitted their own to avoid bias (this is already standard practice).
Then reviewers discuss and reach a consensus. That way, the points that the AI raised are brought into the discussion, but reviewers are free to disregard them if they are bogus. The discussion amongst reviewers is maybe more of a conference-thing, not sure how much discussion exists in journals.
And then you can maybe also reduce the number of human reviewers per paper by 1-2, reducing the workload on reviewers.
Realistically, I think some reviewers probably copy the paper into an LLM already and ask for a review, event though they are usually told not to (for reasons of copyright and intellectual property).
“
Today, in collaboration with Anthropic, we’re excited to share rapid initial progress towards an antibody treatment for food allergy, which affects over 200 million people worldwide, discovered with the help of Claude.
At Divergence Labs, we believe that AI can unlock the next generation of antibody therapeutics with more effective mechanisms than before. Such mechanisms are urgently needed in food allergy, which involve overcoming one of the tightest affinity protein pairs in immunology, IgE and its receptor FcεRI, on mast cells.
Treatment with Xolair—the only approved antibody for food allergy—sequesters circulating IgE, but mast cells remain sensitized for weeks to months due to the extraordinarily slow dissociation of IgE from FcεRI.
We envision a treatment that actively pulls IgE off of FcεRI, without waiting for natural dissociation. Like picking a lock, this requires the right combination of geometry and force.
Our research team, working with Claude, ideated on mechanisms to accomplish this disruption, and converged on an antibody format with two arms with different functions and epitopes. Within one month, we designed, synthesized, and experimentally confirmed the biophysical function of the two arms, which we are now assembling into full therapeutic formats.”
The Anthropic article about the above:
Towards a best-in-class IgE disruptor for food allergywith Claude-enabled antibody design
Early progress on accelerating drug development with Claude
The PDF preprint from the company:Towards a best-in-class IgE disruptor for food allergywith Claude-enabled antibody design
ME/CFS Science Blog
Senior Member (Voting Rights)
1) We’ve written a new article about the spectacular progress in artificial intelligence (AI), the billions that AI companies are investing to transform drug discovery, and why this is a reason for hope for ME/CFS patients.
https://mecfsscience.org/the-ai-revolution/
Here's a brief summary
2) We start by highlighting how AI models have recently solved some of the hardest mathematical problems, finding insights that eluded the greatest minds of our generation. They have also become better at coding than almost any human professional.
3) We use quotes from experts in math and computer science because they are the canary in the coal mine for what is likely going to happen to many other fields as well, including science and medicine.
4) We suspect that this important development hasn’t really gotten through to most people for several reasons. First, there’s so much hype and overstatement about AI that it’s difficult to look beyond the slop.
5) Most people encounter AI through fake images or bullshit-text from spammers and crypto-bros.
Second, people might have tried an AI model in the past and never updated their assessment (they’ve truly gotten a lot better)
6) Third, most people only use AI superficially, for example, to create an image or do a quick internet search using a free version. Running multiple agents using the top-tier models gives much more spectacular results.
7) Fourth, there are big commercial interests at play, so news articles about discoveries are often seen as a strategy to increase the hype. Unless you’re in coding or math, it’s likely that you haven’t seen the full power of AI with your own eyes.
8) AI companies such as Anthropic, OpenAI, and Isomorphic Labs are investing billions into drug discovery. The time and money needed to develop drugs could be cut dramatically if AI models can accurately predict what effect they’ll have in the human body.
9) It’s easy to be cynical about such grand ambitions, but at least the greatest minds of our generation are focused on solving disease instead of designing credit default swaps or advertising algorithms for social media.
10) Progress in medicine will likely not be as fast as in coding or math. Biology is not a closed system: whatever an AI predicts still has to be tested in the lab and in patients.
11) While it might take longer than in math and coding, we suspect that AI will also transform science and medicine. The goal of this article is to raise awareness of this development in the ME/CFS community so we can take advantage of it and be ahead of the curve.
12) We also think these developments are a reason for cautious hope for ME/CFS patients. The decade-long timeline for new drug discovery may be drastically shortened in the future.
13) In the nearer future, AI models can help identify already-approved drugs that may be relevant to ME/CFS pathology and unravel the meaning of genetic findings such as DecodeME.
14) None of this is meant to overlook the many downsides and dangers of AI, but ME/CFS patients are among the wretched of this earth, and they have the most to gain from an AI revolution.
https://mecfsscience.org/the-ai-revolution/
Here's a brief summary
2) We start by highlighting how AI models have recently solved some of the hardest mathematical problems, finding insights that eluded the greatest minds of our generation. They have also become better at coding than almost any human professional.
3) We use quotes from experts in math and computer science because they are the canary in the coal mine for what is likely going to happen to many other fields as well, including science and medicine.
4) We suspect that this important development hasn’t really gotten through to most people for several reasons. First, there’s so much hype and overstatement about AI that it’s difficult to look beyond the slop.
5) Most people encounter AI through fake images or bullshit-text from spammers and crypto-bros.
Second, people might have tried an AI model in the past and never updated their assessment (they’ve truly gotten a lot better)
6) Third, most people only use AI superficially, for example, to create an image or do a quick internet search using a free version. Running multiple agents using the top-tier models gives much more spectacular results.
7) Fourth, there are big commercial interests at play, so news articles about discoveries are often seen as a strategy to increase the hype. Unless you’re in coding or math, it’s likely that you haven’t seen the full power of AI with your own eyes.
8) AI companies such as Anthropic, OpenAI, and Isomorphic Labs are investing billions into drug discovery. The time and money needed to develop drugs could be cut dramatically if AI models can accurately predict what effect they’ll have in the human body.
9) It’s easy to be cynical about such grand ambitions, but at least the greatest minds of our generation are focused on solving disease instead of designing credit default swaps or advertising algorithms for social media.
10) Progress in medicine will likely not be as fast as in coding or math. Biology is not a closed system: whatever an AI predicts still has to be tested in the lab and in patients.
11) While it might take longer than in math and coding, we suspect that AI will also transform science and medicine. The goal of this article is to raise awareness of this development in the ME/CFS community so we can take advantage of it and be ahead of the curve.
12) We also think these developments are a reason for cautious hope for ME/CFS patients. The decade-long timeline for new drug discovery may be drastically shortened in the future.
13) In the nearer future, AI models can help identify already-approved drugs that may be relevant to ME/CFS pathology and unravel the meaning of genetic findings such as DecodeME.
14) None of this is meant to overlook the many downsides and dangers of AI, but ME/CFS patients are among the wretched of this earth, and they have the most to gain from an AI revolution.
chillier
Senior Member (Voting Rights)
@EndME you posted a summary of a critique of a paper at some point recently done by I think gpt astra and I thought it seemed quite impressive. There's been talk maybe on this thread or elsewhere of posting reviews and hypotheses from AI models and how to handle that - I know @mariovitali you are keen on that. I know these things are becoming extremely powerful, though when I ask free models like gemini for example about hypotheses I get a version of the currently popular ideas/memes around MECFS aka some constellation of energy metabolism, mitochondria, autoantibodies, neuroinflammation, viral reactivation and so forth.
I'm curious when you ask the frontier models this question do you get a different/more plausible output than that? I'm curious about the capacity for frontier models to do a literature review. If you instructed it to only focus on the clinical temporal symptoms of the disease, primary literature of experiments and epidemiology done in ME, CFS, chronic fatigue, and maybe also the primary literature and clinical manifestations of other diseases. Could it pick up on useful things do you think?
On this forum lately there's been a lot of picking up of old, potentially very relevant and reasonably carried out research like the buspirone stuff forestglip has been picking out. Would an AI agent trawling through this material be able to find things?
I'm curious when you ask the frontier models this question do you get a different/more plausible output than that? I'm curious about the capacity for frontier models to do a literature review. If you instructed it to only focus on the clinical temporal symptoms of the disease, primary literature of experiments and epidemiology done in ME, CFS, chronic fatigue, and maybe also the primary literature and clinical manifestations of other diseases. Could it pick up on useful things do you think?
On this forum lately there's been a lot of picking up of old, potentially very relevant and reasonably carried out research like the buspirone stuff forestglip has been picking out. Would an AI agent trawling through this material be able to find things?
Last edited:
I think as ever a lot depends upon how they are prompted and used. You’ve been able to get useful summaries from models for a long time but also rubbish ones. The same applies to frontier models now.
We’ve built the ultimate machine to give us the ultimate answer (or answers). Now we just need to ask the right questions. Douglas Adams was right again.