Artificial Intelligence

Traditional autoregressive language models generate text sequentially, one token at a time, leading to slower outputs with limited coherence and quality.

Diffusion models are an alternative approach. Instead of direct prediction, they iteratively refine noise, enabling faster generation, dynamic error correction, and greater control. This makes them particularly effective for editing tasks, including in math and code.

https://github.com/ML-GSAI/LLaDA

MiMo-7B, a series of reasoning-focused language models trained from scratch, demonstrating that small models can achieve exceptional mathematical and code reasoning capabilities, even outperforming larger 32B models. Key innovations include:

• Pre-training optimizations: Enhanced data pipelines, multi-dimensional filtering, and a three-stage data mixture (25T tokens) with Multiple-Token Prediction for improved reasoning.
• Post-training techniques: Curated 130K math/code problems with rule-based rewards, a difficulty-driven code reward for sparse tasks, and data re-sampling to stabilize RL training.
• RL infrastructure: A Seamless Rollout Engine accelerates training/validation by 2.29×/1.96×, paired with robust inference support. MiMo-7B-RL matches OpenAI’s o1-mini on reasoning tasks, with all models (base, SFT, RL) open-sourced to advance the community’s development of powerful reasoning LLMs.

an in-depth discussion of mimo-7b >https://www.youtube.com/watch?v=y6mSdLgJYQY

It's being implemented now

Crossposted from https://lemm.ee/post/63661379

I am not sure we are in a Human VS AI world, but rather we are adding another "Brain Layer" as seen here. We all have an ancient reptilian brain, that is wrapped by our limbic or animal brain, that is wrapped by our human brain, and now we have wrapped a new AI brian over the set. I certainly feel my brain has expanded and is now capable of doing things not possible for me before. I foresee some competition with AI, but I anticipate there will be a human in the mix on the other end. What do you think?

Ironically, I could not get AI to make the bottom image so forgive my amatuer GIMP skills.

• GitHub https://github.com/ace-step/ACE-Step
• HF: https://huggingface.co/ACE-Step/ACE-Step-v1-3.5B