Just know stuff (or, how to achieve success in a machine learning PhD) (2023) (kidger.site)

Oxford math PhD graduate publishes a practical manifesto for succeeding in an ML PhD: “Just Know Stuff.” Drawing on his thesis (a textbook on Neural Differential Equations) and fast academic/industry traction, he argues the central lesson is deep, hands‑on technical knowledge rather than surface-level familiarity. The post functions as a concrete curriculum for early-stage PhD students, emphasizing implementation, cross‑disciplinary breadth, and production-ready software skills—making it a useful roadmap for researchers who want to generate sound ideas, spot bad ones, and transition to industry. The recommended toolkit mixes core ML theory, scientific computing, and software engineering: mastery of forward/reverse autodiff (and writing custom gradients in PyTorch/JAX), Hutchinson/Hutch++ trace estimators, optimal Jacobian accumulation, fast linear algebra (Strassen/Winograd), building convolutions/attention, universal approximation results, GNNs/Transformers/U‑Nets, continuous‑time views (ODEs/SGDs/GRUs), generative models (flows, VAEs, WGANs, score‑based diffusion), distributed training (e.g., jax.pmap), Bayesian hyperparameter tuning (Ax), and optimizer math. He also stresses numerical methods (QR/SVD/Cholesky, ODE solvers, Monte‑Carlo vs QMC, floating‑point pitfalls, sparse solvers) and professional tooling: Git, testing/CI, performance languages (C++/Rust/Triton), bindings, design patterns, and reproducible packages. The post’s value is pragmatic: implement key algorithms from scratch, understand tradeoffs and numerics, and write clean, fast, maintainable code—skills that materially improve research quality and employability.