Promising but too early to claim action-driven age. As a research idea, maybe bringing in Adversarial Critics component to make LLMs better at delivering actionable insights.
I agree with your findings and the fact that the technology as it is is prime to unlock intelligent looking agents for specific tasks. I believe one of the major challenges we still have to address is the right part of your diagram, the iterative/incremental/continuous learning loop – today we still do this fairly manually, and learning typically can lead to memory loss, etc. Ideally the "AGI" / the agent should be able to drive that loop itself as well, and not lose memory.
Yeah I'm super curious about how to actually drive that loop effectively. I'd be super curious what approaches or teams you're most interested in recently.
I'm building https://text-generator.io which crawls input links and analyses images to generate better text, looking into some new features like fact checking and factual text generation are now possible with this style of think step by step, e.g. "to fact check this in going to Google..." Etc fixing up the factual errors is possible with crawling/distance metrics/rewriting with LLM s as well so we are getting really close now to AGI with the actions that can happen behind the scenes.
The system also does OCR so it can understand and generate about receipts, docs and screenshots which is pretty cool, eventually AGI with have to be able to do everything humans do and along the way there will be a lot of bitter lessons where people keep thinking systems don't need to be trained/differentiable, AGI is going to be about computer learning and search and hard coding things from our own minds are helpful but not going to get us to AGI by themselves
Very cool! We have some action driven behavior in our product, Consensus, which is a search engine that uses LLMs to fins answers in scientific research papers. Check it out: https://consensus.app/
If you read the ReAct paper even they are doing instruction tuning using the method from Zelikman et al that effectively has an RL component, it's not just prompt hacking.
It seems to me that getting everyone involved in advancing AI capabilities educated on the risks from AI that researchers have been talking about for years is one of the most tractable ways we can reduce AI risk.
From the interview: "We just want more researchers to acknowledge and be aware of potential misuse. When you start working in the chemistry space, you do get informed about misuse of chemistry, and you’re sort of responsible for making sure you avoid that as much as possible. In machine learning, there’s nothing of the sort. There’s no guidance on misuse of the technology."
(I think the risk of not solving the alignment problem is a far greater threat than the risk of misusing ML to create biochemical weapons, but the biochemical weapons example does a great job to illustrate the concern with ML as a field lacking the same safety culture as e.g. chemistry.)
In AGI G is for general, hence an Action-driven AI which aims at AGI can't be limited to a mere digital world. Like humans an AGI should be able to act efficiently in the real world, therefore AGI is for robotics or it is an illusion.
Well, you can apply your Natural GI in a digital world but fortunately you live in a real world with a real body and that's why you have a GI. Robotics is the path to true AGI. Period
I would like to collaborate with ai directly within my existing work environment To tell her to do things like try 50 different layer blend mode variations of a layered psd file
Things like that will really come in handy for project such as design and visual effects with her a lot more complicated than just creating an image because they have functional requirements
Promising but too early to claim action-driven age. As a research idea, maybe bringing in Adversarial Critics component to make LLMs better at delivering actionable insights.
Really great write up John.
I agree with your findings and the fact that the technology as it is is prime to unlock intelligent looking agents for specific tasks. I believe one of the major challenges we still have to address is the right part of your diagram, the iterative/incremental/continuous learning loop – today we still do this fairly manually, and learning typically can lead to memory loss, etc. Ideally the "AGI" / the agent should be able to drive that loop itself as well, and not lose memory.
Nice to hear from you Clement!!
Yeah I'm super curious about how to actually drive that loop effectively. I'd be super curious what approaches or teams you're most interested in recently.
I'm building https://text-generator.io which crawls input links and analyses images to generate better text, looking into some new features like fact checking and factual text generation are now possible with this style of think step by step, e.g. "to fact check this in going to Google..." Etc fixing up the factual errors is possible with crawling/distance metrics/rewriting with LLM s as well so we are getting really close now to AGI with the actions that can happen behind the scenes.
The system also does OCR so it can understand and generate about receipts, docs and screenshots which is pretty cool, eventually AGI with have to be able to do everything humans do and along the way there will be a lot of bitter lessons where people keep thinking systems don't need to be trained/differentiable, AGI is going to be about computer learning and search and hard coding things from our own minds are helpful but not going to get us to AGI by themselves
Very cool! We have some action driven behavior in our product, Consensus, which is a search engine that uses LLMs to fins answers in scientific research papers. Check it out: https://consensus.app/
Great piece. I'd be curious to read more about why you think RL is the way forward rather than the in-context tricks like chain-of-though with ReAct.
If you read the ReAct paper even they are doing instruction tuning using the method from Zelikman et al that effectively has an RL component, it's not just prompt hacking.
based
Thanks for the post John. Really enjoyed it.
Please write more.
Thanks for the "PS: A note on alignment"!
It seems to me that getting everyone involved in advancing AI capabilities educated on the risks from AI that researchers have been talking about for years is one of the most tractable ways we can reduce AI risk.
I said as much here two days ago in response to a researcher who **never previously realized** "how AI technologies for drug discovery could potentially be misused" in an interview titled "AI suggested 40,000 new possible chemical weapons in just six hours": https://www.lesswrong.com/posts/3TCYqur9YzuZ4qhtq/meta-ai-announces-cicero-human-level-diplomacy-play-with?commentId=aaBsxm2KPizLLnpmy#aaBsxm2KPizLLnpmy
From the interview: "We just want more researchers to acknowledge and be aware of potential misuse. When you start working in the chemistry space, you do get informed about misuse of chemistry, and you’re sort of responsible for making sure you avoid that as much as possible. In machine learning, there’s nothing of the sort. There’s no guidance on misuse of the technology."
(I think the risk of not solving the alignment problem is a far greater threat than the risk of misusing ML to create biochemical weapons, but the biochemical weapons example does a great job to illustrate the concern with ML as a field lacking the same safety culture as e.g. chemistry.)
In AGI G is for general, hence an Action-driven AI which aims at AGI can't be limited to a mere digital world. Like humans an AGI should be able to act efficiently in the real world, therefore AGI is for robotics or it is an illusion.
Meh, I do all my work on a computer am I an illusion?
Well, you can apply your Natural GI in a digital world but fortunately you live in a real world with a real body and that's why you have a GI. Robotics is the path to true AGI. Period
I actually find it at least a bit unfortunate that I live in a real world with a real body since if I didn't I would be much more powerful.
Can I trade this to use Photoshop to create a picture? As opposed to generate a picture directly with stable diffusion.
Hm could you share why you would want to do that?
For the same reason I would want to make a humanoid robot that can drive a car
Stable diffusion can already make you an image so what is it you want that it doesn't do?
I would like to collaborate with ai directly within my existing work environment To tell her to do things like try 50 different layer blend mode variations of a layered psd file
Ooh yeah well I think that should be pretty doable. One way of thinking about this is people should be able to code in natural language.
Things like that will really come in handy for project such as design and visual effects with her a lot more complicated than just creating an image because they have functional requirements