Achievements of Artificial Intelligence (AI)
- Thread starter ME/CFS Science Blog
- Start date
fst
Senior Member (Voting Rights)
Wouldn't it be funny if the AI craze led to more open-source software and disensh*tiffication of everything?
Given the amount of AI slop we have to endure these days, it would only be fair.
tralfamadorian97
Established Member (Voting Rights)
Great article. I've been reading/ listening to the Substack blog "Owl Posting". The author has worked in various machine learning roles in the biotech industry and has a large number of relevant connections. I've found his articles useful for understanding where AI/ML for science may be heading. For instance, I enjoyed this one from earlier in the year on lab robotics.
Last edited:
ME/CFS Science Blog
Senior Member (Voting Rights)
Anyone has any insights on this one. My first thought is that it might be a big deal if LLM can improve itself by tinkering with the weights directly.
Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights
Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights
Obviously the project I reported in my previous post was way too successful and threatening to certain people, so this sort of thing had to be stopped.
It looks huge, though opinions are divided as it just happened. It's not self-improvement yet, this is larger models doing the equivalent of pruning or optimizing neurons in much smaller models, but this is the sort of stuff that could lead to models self-improving, as it was just discovered so the possibilities are still open. It bypasses most of the training, is more of a knowledge transfer kind of thing, where one model that knows how to do something implants into another.
From what I understand the larger model took the smaller model as input, so there would need to be a way of doing that in chunks as the larger models are far too big for current context windows, but it's really impressive how it can read weights and figure out what they do. This is the real "woah" moment. It can interpret parts of those models and knows what they do, well enough to be able to fix issues in them. And an interesting part is that fixing some specialized parts generalized to improving the model overall.
It's been replicated so people will tinker with this heavily. The sheer speed at which it happens is incredible. Hours after it was reported people had already replicated parts of it and are trying to figure out where to take it from there.
Yeah there’s a lot of debate. I’ve been following what I can. I think a lot is overblown to be honest and certainly nothing near self improvement or even as significant as some claim. This looks like another case of something interesting but nothing that a human couldn’t do. Or indeed a smaller model
This is worth a read from those interested https://simonwillison.net/2026/Nov/3/qwen-can-interpret-model-weights/
I’m excited about where things are going but it is sometimes odd to see some of the scepticism we apply to other areas not applied to the wild claims in the world of AI. It may end being the case that only one of these models can really understand one of these models enough to make the sorts of leaps prophesied. But we’ll need strong evidence of that and serious debate on if it’s a path we want to take how we manage it. That seems kind of important.
This is worth a read from those interested https://simonwillison.net/2026/Nov/3/qwen-can-interpret-model-weights/
I’m excited about where things are going but it is sometimes odd to see some of the scepticism we apply to other areas not applied to the wild claims in the world of AI. It may end being the case that only one of these models can really understand one of these models enough to make the sorts of leaps prophesied. But we’ll need strong evidence of that and serious debate on if it’s a path we want to take how we manage it. That seems kind of important.
So you’re telling me this is going to kill the surveillance state, break the advertising revenue Google and Facebook depend upon and Anthropic and OpenAI are going to go bankrupt?
Utsikt
Senior Member (Voting Rights)
I had not thought that far, but yes, if digital terrorism becomes that easy people might adapt. Although they have a lot of inertia in their favour.
Oh noes!
I have never understood why things like fridges, etc, need to be connected to the internet. That, as we have just seen, is asking for trouble.
Air gaps in electronic and electrical devices, including for data, are there for a reason.
It is weird. I get why having systems monitoring control is useful but not the connection to the wider internet part. I think sometimes it now comes down to that just being seen as the easiest way to do it. But it really isn’t hard or expensive to setup a private network as you say. Maybe some started that way but then someone in head office decided they wanted to see all the data or it would be more efficient to have everything centrally managed and didn’t consider the hidden costs of doing so.Oh noes!
I have never understood why things like fridges, etc, need to be connected to the internet. That, as we have just seen, is asking for trouble.
Air gaps in electronic and electrical devices, including for data, are there for a reason.
More widely many of these stories seem to be those hidden costs becoming more apparent after years of lack of investment in security or bug fixes and general technical debt accrued with the push for ‘more features’.
What I’ve seen from many of these AI companies is exactly the same approach with even less understanding of what they’re building. So I will not be surprised when that comes back to bite too. Some of this has already been seen with many of the ‘surprises’ from ‘rogue agents’. Most of it has just been not taking building good, reliable, tested systems seriously and instead focusing on ‘more features’.
Last edited:
fst
Senior Member (Voting Rights)
The problem is the fridge usually does not report to your (the fridge-owners') office but to the cloud-backend of the fridge manufacturer.
There's multiple reasons for this, ultimately it boils down to convenience (or laziness?) and in some cases the fridge manufacturer thinking it is good business to provide a "platform" to do "big data". And probably also years of hyperscalers pushing this architecture via consultants.