Software Engineer (ML) Intern · Arm · Jun 2025 to Aug 2025
ML performance engineering at Arm
Integrated KleidiAI™ microkernels into Microsoft's ONNX Runtime to accelerate ML inference on Arm CPUs, and built the C++ profiling tooling to prove the gains.
- C++
- Python
- ONNX Runtime
- KleidiAI
- Linux
Context
At Arm in Galway I worked on ML inference performance: making models run faster on Arm CPUs by getting KleidiAI™ (Arm's library of optimised microkernels) into the hands of every framework that matters. My summer focused on Microsoft's open-source ONNX Runtime.
What I did
- Integrated KleidiAI microkernels into ONNX Runtime, accelerating ML model inference on Arm CPUs in one of the most widely deployed inference engines in the world.
- Built a C++ profiling tool to identify intra-operator overheads in ML models and quantify exactly where KleidiAI's performance gains landed.
- Built Python performance visualisations with Pandas and Plotly, cutting graph generation time from 10 seconds to under 1.
- Automated CI testing for cross-compiled Android builds via ADB and SSH, and worked in Agile sprints with the team.
Impact
Upstream work in an open-source project used across the industry: faster inference for anyone running ONNX models on Arm hardware, and profiling evidence to back the numbers.
What I learned
How to work properly inside an Agile team: tracking progress, staying resilient, and (the underrated one) asking for help in a way that actually unblocks you.
It was also my first exposure to strict development procedure at scale: testing standards, documentation, the formalities of pull request submission, CI testing, and cross-environment compilation.
On the technical side I got to see the operators behind neural network architectures at the CPU level. That turned the linear algebra from my coursework, and a deep learning and CNN course I'd taken, into something concrete, because that level is where ML performance actually comes from.