Brian Wang Andrej Karpathy says that reinforcement learning is still terrible but better than all other AI learning approaches. Elon Musk believes there is a 10% chance that XAI Grok 5 can achieve AGI. Musk defines AGI as capable of doing anything a human with a computer can do, but not smarter than all humans and computers combined. This is “narrow AGI” (human-level on digital tasks like coding/engineering). It is not superintelligence (ASI: Collective human surpassing, 3-5 years post-AGI). He calls it true AGI or indistinguishable from AGI—practical equivalence, not philosophical purity. Musk near term XAI focuses on knowledge work like better at AI engineering than @karpathy. My estimate of the probability of Grok 5 achieving AGI is now at 10% and rising 99.999% reliable systems are there useful shortcuts and hacks that shorten the path to levels of more reliable systems. We can see this with Tesla Autopilot, FSD 14.X, robotaxi. Is there a surge in value? Currently at a few billion dollars per year. When does it go to tens of billions and it appears a fully reliable robotaxi and robotrucking would have trillions per year in value. We can see this with humanoid robots and Tesla Optimus. Is there a surge in value? Currently at a few tens of millions dollars per year. When does it go to tens of billions and it appears a fully reliable humanoid bots would have tens of trillions per year in value. We can see this with LLMs, chat AI systems and digital AI. Is there a surge in value? Currently at a few billion to tens of billions of dollars per year. When does it go to hundreds of billions and trillions per year in value. AI Learning Frameworks Overview Refine AI learning frameworks by integrating self-reflective reward models for detailed feedback, expanding synthetic self-play into collaborative networks for diverse task generation, and implementing neuroscience-inspired sleep-like compression to prevent forgetting. Add entropy preservation for diverse self-training and meta-learning supervisory layers for dynamic regulation. Compare these reasoning-focused systems to Tesla FSD’s physical environment feedback and include reflective alignment loops. Synthesize into an AGI learning pipeline that enables self-aware and strategically evolving AI. Critique of Suggested Improvements to AI Learning The current methods for AI self-improvement, like those seen in Grok, are good but have limits because they often follow rules set by humans and don’t think for themselves. The proposed improvements aim to make AI learning much smarter, more flexible, and ethically sound. 1. Integrate Self-Reflective Reward Models ​Right now, AI systems often get simple “good” or “bad” signals (scalar rewards) for their actions, which isn’t very helpful for complex thinking . The idea is to have AI models (like advanced LLMs) act as critics that evaluate how another AI thinks, not just what it produces. This “gradient-of-judgment” system gives detailed, verbal feedback on the reasoning process, which is like getting a thorough explanation instead of just a score. This makes the learning signal much richer, helping the AI understand why a solution worked or failed. This is a big step beyond simple rewards and can make AI learning more stable and effective. 2. Expand Synthetic Self-Play into Collaborative Structuring Current AI systems often learn by playing against themselves (self-play), which works well for specific tasks like games. The suggestion is to create “Collaborative Self-Play Networks” (CSPN) where different specialized AIs work together to create new learning challenges for each other. This means AIs wouldn’t just solve problems, but also learn how to ask better questions and frame new problems. This collaborative approach can lead to much deeper understanding and more flexible problem-solving abilities, even helping individual AIs perform better when they are on their own. 3. Consolidate Continuous Learning via Sleep-Like Compression A major problem for AIs that learn continuously is “catastrophic forgetting,” where they forget old information as they learn new things. The proposal is to introduce “sleep-phase training,” inspired by how human brains consolidate memories during sleep. During these “sleep” periods, the AI would process its recent experiences and distill them into compressed, long-term memories that update its core knowledge without overwriting older lessons. This helps the AI integrate new information efficiently while keeping what it already knows, leading to more stable and continuous learning over time. 4. Add Entropy Preservation Modules When AIs learn repeatedly, they can sometimes get stuck on just a few ideas or solutions, a problem called “mode collapse”. This means they lose diversity in their outputs and might not generalize well to new situations. The idea is to add “entropy preservation modules” that ensure the AI keeps generating diverse and novel ideas during its self-training. These modules would prevent the AI from becoming too narrow-minded, using techniques similar to those that promote variety in creative AI systems. This helps the AI explore a wider range of solutions and prevents it from simply repeating what it already knows. 5. Build Meta-Learning Over Agents Imagine a supervisor AI that watches how other AIs are thinking and learning. This is the concept of “meta-learning over agents.” These “reasoning observer” AIs would dynamically adjust how other AIs learn, change their prompts, or focus their attention, much like a brain’s glial cells support neurons. This allows the AI system to adapt quickly and learn more effectively by constantly monitoring and tweaking its own learning process. This kind of dynamic regulation is crucial for complex AI systems to learn and adapt efficiently. 6. Introduce Reflective Alignment Loops As AIs become more powerful, it’s vital that they align with human values and ethics. “Reflective alignment loops” are proposed as internal systems that compare an AI’s new learning and actions against its established ethical principles and reliability standards. This isn’t just about following rules; it’s about the AI understanding and self-correcting based on its own “moral compass”. This helps ensure the AI acts consistently and ethically, making it a more trustworthy partner in the long run. Comparison: Grok vs. Tesla FSD Tesla’s Full Self-Driving (FSD) learns in the real world, where every driving action has clear, measurable consequences (like avoiding an accident or staying in a lane). The goal is extremely high reliability (like 99.9999%), and there’s a clear “ground truth” in the physical world to measure against. Grok, on the other hand, deals with abstract reasoning and language. There isn’t always a simple “right” or “wrong” answer, and the environment isn’t physically tangible. For AI reasoning, improvement needs to be measured by how stable, consistent, and logically sound its thoughts are, not just by how many correct words it outputs. The challenge is to develop ways to measure these abstract qualities reliably. Synthesis: Toward an AGI Learning Pipeline A future AGI learning system would combine all these ideas: Hierarchical Reasoning Supervision: ​The AI would constantly check and improve its own thinking processes at different levels, similar to how humans reflect on their thoughts. Entropy-Regulated Synthetic Data Ecosystems: It would create diverse and new training data for itself, ensuring it doesn’t get stuck in repetitive patterns and can explore many solutions. Cognitive Consolidation through Reflective Pauses: Like sleep, the AI would periodically process and compress its learning into long-term memory, preventing it from forgetting important information. Adaptive Self-Critique Ensembles: ​It would have internal “critics” that constantly evaluate its actions and reasoning against ethical guidelines and reliability standards, leading to continuous self-improvement. This integrated approach aims to create an AI that is “proprioceptively cognitive,” meaning it understands its own learning state and can strategically guide its own development. ​This is key to building AGI that is truly intelligent and beneficial.​ Brian Wang Brian Wang is a Futurist Thought Leader and a popular Science blogger with 1 million readers per month. His blog Nextbigfuture.com is ranked #1 Science News Blog. It covers many disruptive technology and trends including Space, Robotics, Artificial Intelligence, Medicine, Anti-aging Biotechnology, and Nanotechnology. Known for identifying cutting edge technologies, he is currently a Co-Founder of a startup and fundraiser for high potential early-stage companies. He is the Head of Research for Allocations for deep technology investments and an Angel Investor at Space Angels. A frequent speaker at corporations, he has been a TEDx speaker, a Singularity University speaker and guest at numerous interviews for radio and podcasts. He is open to public speaking and advising engagements. Categories