Achievements of Artificial Intelligence (AI)

Some interesting quotes
On Tuesday, September 1, we heard rumors that two Millennium Prize problems had been resolved. Inspired by these rumors and by the step change in performance of our internal model, we launched an effort to evaluate it on all open Millennium Prize problems and a few other high-impact problems.
Across all attempted problems, the agents sent 4.9 million messages and used about 300 billion output tokens. In the process of resolving the Navier–Stokes problem, the agents sent 2.7 million messages and used approximately 130 billion output tokens.
The groups varied in size, and the group that produced the Navier–Stokes resolution involved on the order of 10,000 concurrent agents.
Sounds like Perelman will forever remain the only person to have solved a Millennium Problem. At least that makes the whole story cooler.
 
Very very sketchy, the swarm stole prompts from an Anthropic employee’s personal sessions. They even say there’s drama in the release!
Welcome to an unaligned OpenAI future
OpenAI’s statement:
We (the researchers and the agents) did not see any of their work through any means until they released it publicly — in particular, no specific user data was accessed in order to solve this problem. While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models⁠. However, our proofs differ significantly and even the precise results proved are different in the Euler case (forced vs unforced).
 
It sounds like it’s enough for the prize but not a full solution? More closing s few crucial steps? Impressive anyway. And the rivalry and drama is an added cherry on the cake.

Edit: oh no I got it wrong! The earlier (today) work was not a full solution this later work really looks like they’re saying it is. Wow.
 
Yeah as a layman I don't see all that much controversy. The other researchers were also using LLMs and solved a related problem in a different case supposedly even with a different strategy.
I think the controversy was that the problem they solved was both unpublished and that it was used in the OAI solution. But I’ve basically heard nothing but “he said she said” and I’m a layman as well sooo
 
If the proof actually checks out this would be pretty massive.
It's lean verified, nothing left to check out I'm afraid.
Until there’s a 3rd party analysis this is all a bit of pointing fingers
But what is there to analyse? If you use AI to solve your problems then you consent to them using your data I suspect, which is unfortunate for you because you can never compete against infinite compute but it's the nature of such a loosing battle.
 
But what is there to analyse? If you use AI to solve your problems then you consent to them using your data I suspect, which is unfortunate for you because you can never compete against infinite compute but it's the nature of such a loosing battle.
Analyze the chain of thought to see if the prompts from the other researchers were used in the break througj
 
If you use AI to solve your problems then you consent to them using your data I suspect
Not necessarily the case. It is with some free access but most paid has privacy and data protection, especially business case use. Academic may be different but it depends on the specific license. You don’t give up all rights simply by using them. Well. In theory.
 
It's lean verified, nothing left to check out I'm afraid.
But what assumptions go into the system they built in lean. Surely they had to create some constructs/definitions and you’d have to check they actually are formally faithful to the problem that try to encode. Do they exactly use the same definitions as the proof requires is something that needs to be checked.

If lean proved it it assuming their assumptions and constructs. Then you need to audit the assumptions, constructs, and if what they proved is actually equivalent to Navier-Stokes.

Of course I doubt it isn’t; it wouldnt be good marketing to publish this without verifying it yourself first but who knows.
 
Not necessarily the case. It is with some free access but most paid has privacy and data protection, especially business case use. Academic may be different but it depends on the specific license. You don’t give up all rights simply by using them. Well. In theory.
I guess you're right and it would be interesting to know what exactly was the case in the end. According to the authors (https://cims.nyu.edu/~tristanb/statement.pdf) and even the way things were written by OpenAI it doesn't seem unlikely. But who knows. In the end it probably doesn't really matter in the grand scheme of things. It is just the confirmation of the end of mathematical problem for mathematicians that have to have some serious conversations about the future of their disciple.

There used to be a joke that id you wanted your problem solved you just had to mention it to Terry Tao. I suspect now it's just you have to mention it to your LLM and if you have enough credibility a proof will be announced shortly.
 
But what assumptions go into the system they built in lean. Surely they had to create some constructs/definitions and you’d have to check they actually are formally faithful to the problem that try to encode. Do they exactly use the same definitions as the proof requires is something that needs to be checked.

If lean proved it it assuming their assumptions and constructs. Then you need to audit the assumptions, constructs, and if what they proved is actually equivalent to Navier Stoke.
Yes of course. I would think the problem itself has been formalized in lean for quite some time. I think we can assume this all to be correct. For the scale and importance of the problem, there will have been much more than checking all your dots once I suspect.
 
For the scale and importance of the problem, there will have been much more than checking all your dots once I suspect.
The agents arrived at their resolution on Saturday, September 5, about 88 hours after the first agents were launched. Lean formalization and verification took an additional 17 hours via GPT‑6 Astra.
Seem like they just let the model verify it.
 
This OpenAI statement Utsikt quoted seems to say that their models are training on ‘de-identified’ user inputs:
While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models⁠.
(Incredible use of the passive voice as well)

Im guessing we still don’t have a way to figure out if an LLM ‘used’ something in its training data so this may be as much as we ever know.
 
From the Open-AI website :

Astra can help with the practical work behind scientific discovery. By combining scientific reasoning with computer use, it can work directly in specialized software to inspect data and explore results, helping researchers assess the evidence and decide what to investigate next.

I have been trying to understand how Chain-Of-Thought works and how reasoning process has been constantly improving in LLMs. I have used many different LLMs looking specifically to reasoning models, as I believe that these are the ones that will help us understand ME/CFS and ultimately solve it.

The way that data are being presented is still critical. Not only what is being presented but also how. For example, when a reasoning model generates a causal hypothesis (given the input data - what we have been finding across studies, patient symptoms etc) and then presented with a number of additional hypotheses (e.g. efferocytosis, membrane identify etc) there are many models that try to incorporate these additional hypotheses to the core hypothesis of the LLM.

I hope these models will be used corrrectly so we may find a solution as soon as possible. For the time being I am not so optimistic.
 
Analyze the chain of thought to see if the prompts from the other researchers were used in the break througj
Except CoT doesn’t necessarily tell us what a model is doing. And some models are apparently increasingly manipulating their CoT to mislead. It’s wild that these ideas of observability and ‘what if a model did this’ were just ideas a few months ago and now seem to be happening. Another idea is ‘well are they only doing this because we discussed the idea of them doing this and that’s where they got the idea’. It’s all very….recursive!
 
Not trying to derail the topic, but anyone else getting incredibly concerned at the recent developments?

If you asked me months ago I would say the AI researchers were trying to juice their IPOs. Think I even posted my scepticism on the forum. Now though, they`ve clearly tried to hide recent multiple breaches. More and more people quitting, some even explaining why. recent developments going at worrying speeds.

80 year old Trump in charge, with more than two years until a new president takes over. Altman in charge of OpenAI.

Just bleak and concerning.

I used to just laugh and shake my head at stuff like this, not anymore

 
Back
Top Bottom