Franz Srambical
============
Hi! I'm Franz. I started and scaled p(doom), a Discord-based research community, from zero >200 members.

We work on addressing core blockers towards general intelligence that cannot be solved by scaling up compute. We work on everything from kernel-level optimizations to large-scale distributed systems for pre-training and reinforcement learning. A lot of our current work involves finding, investigating and exploiting novel data troves [1] [2], and building open infrastructure/codebases. I am ex-distributed systems software engineer at Celonis, dropped out of uni (Informatics at TUM), minored in Computational Neuroscience (courses at LMU Graduate School of Systemic Neurosciences), ex-linux kernel developer, ex-'alignment researcher' in my high school years, ex-RA at Bastian Rieck's lab, ex-'BCI researcher' & founding member at neuroTUM.

Our current research interests include lengthening model's task horizons using crowd-code, a dataset of dense IDE interactions of months-long research engineering. We also recently released Jasmine, a simple, performant and scalable JAX-based world modeling codebase, and common-craft, five thousand hours of curated Minecraft footage.

• A blog post on performance-degradation free value assertions in JAX using a very recent and still private JAX API.
• A blog post on how a lot of PPO implementations are technically wrong (including Deepmind's reference implementation in RLax).
• A blog post on how practically everyone in LLM post-training is currently implicitly using REINFORCE with baseline, clipping and a likelihood ratio, and not PPO (in the classical sense).
• A blog post on a simple mental model that permits straightforward contextualization of the current research frontier and extrapolation of what the most important future research directions are going to be.
• A blog post on why the Transformer's causal mask is its ultimate feature, not a bug
• A 30-page paper on AGI alignment written in 2018/19 during my high school years (unfortunately in German)
• A review paper on superhuman poker bots
• A 2-page writeup on the necessity of learning rate warmup under muParametrization (with code)
• A poster on why the Transformer's causal mask is its ultimate feature, not a bug
• A poster on the 99-year leasehold system in Singapore as a means to mitigate generational wealth (unfortunately in German)
• A crappy 'it's just x but with y'-type computer vision paper on 2% better (and 3% worse) panoptic 3D reconstruction (which included one-shot finetuning a 1B parameter diffusion model hours before the deadline)
• A 2-page doc outlining my thoughts on whether machines can think

• Invited talk at Cohere Labs on recipes for reasoning, adaptive compute, expanding a model's task horizon, automated verification signal mining, our open-source world modeling codebase, and crowd-sourcing data to teach humans to code like humans (for months).
• Talk on the necessity of learning rate warmup under muParametrization (and a real-time case study on the absurd speed of modern AI research)
• Talk on AlphaFold 3, motivating its architectural design through the lens of the Transformer architecture and its modern variants
• Talk at MunichNLP on p(doom), adaptive compute at inference time and predicting text-based protein function descriptions directly from sequence, bypassing structure
• Talk on the 'translation gap' between core and applied machine learning research, scaling protein function prediction as neural machine translation, ESM-3, AlphaFold 3, the causal mask, muTransfer and ARC-AGI
• Talk on beating modern dynamic graph classification baselines using a 1-layer LSTM
• On neurons that count and ultrasonic communication in frogs (I have catastrophically forgotten the contents of this talk)