We recently held our 2nd annual Engineering Conference at our offices in London. It brought together 100 members of our Product & Engineering organisation with a further 150 watching from home. Since the pandemic, events where we bring our organisation together have become even more important – an opportunity to share ideas and learn from both internal and guest speakers from our tech partners, and to engage with people outside of our normal day-to-day team interactions.
Making it relevant for those attending was essential, and judging by the initial feedback, it looks like we nailed it:
“Such an amazing event with some really inspiring speakers. I learnt so much!”
“Definitely ranks highly as one of the most insightful and engaging days in my time at 10x”
Antony Jenkins, our founder, chair, and CEO, opened the day emphasising that 10x is a technology company servicing financial services companies as opposed to a fintech, stressing that technology and engineering are our business.
Mark Holt, our Chief Product and Engineering Officer, followed with the day’s keynote highlighting the need for continuous improvement, making everything we do something to be proud of, and introducing the theme for the day “Move fast, don’t break things” with the day's sessions focussing on scaling, reliability, security, architectural simplicity, and continuous delivery.
Starting with scaling, Jenny Farrell delved into how her Performance Engineering team models production traffic, how it is kept representative and several times ahead of actual production volumes. Jenny and the team have helped identify significant improvements in our already highly scalable platform, so gaining an insight into how she thinks about latency and statistics will help our teams think even more deeply about how their services perform in production.
Keeping with the theme of scalability, Darren Ward from our SuperCore team took us on a deep dive of optimisations used to establish substantial performance gains for our high-performance ledger; sharing techniques that other engineering teams can adopt. As usual any interaction with the database is your enemy...
Testing at high volumes can surface bottlenecks in a system and we love NewRelic for helping us observe how our platform performs across all our client environments. Wojtek Oledzki from our SRE team retold the story of a production incident where he worked his way through several layers of AWS limits before discovering the true bottleneck: poor design of a logging subsystem that caused us to trip an AWS limit deep down in their stack.
The 10x platform utilises Kafka to decouple systems and help processes scale. Steve Murkin outlined options that his team have considered and implemented for improving resilience when consuming from Kafka topics and avoiding poison pills. Steve’s talk was followed-up by Antony Stubbs from Confluent who went into depth on how to improve performance using Parallel Consumers, an approach that several of our engineering teams are currently trialling.
As a system grows, it is inevitable that it will need to be refactored, but this must be done safely. MJ and Wayne Pounder from our infrastructure engineering team shared how they have been evolving several aspects of our platform architecture. Simplifying it to improve its resilience, improving the security posture and managing the roll out with zero downtime, ensuring no impact for our customers.
Eilon Elhadad from Aqua Security joined us to share his insights into securing the software supply chain with a focus on recent high-profile vulnerabilities and tips on how to protect against them. And Luke Vear went into detail of one tool 10x has developed to protect against Advanced Persistent Threats in all our environments which forces all infrastructure to be considered ephemeral.
As Mark Holt stressed in his keynote, continuous improvement and chipping away at issues is key. Jason Costa reflected on his experiences of software delivery at both a legacy bank and neo bank and shared his top tips on how teams can improve their ways of working to make continuous delivery safe and effective.
And Jason Barto, EMEA lead for resilience at AWS, shared techniques that can be introduced at different stages of the SDLC for improving resilience including System Theoretic Process Analysis during requirements analysis and design, Failure Mode & Effect Analysis (FMEA) during design and development, and Chaos Engineering from development to production.
10x has long been a practitioner of both FMEA and chaos engineering (we have our own chaos monkey tool called Logan’s Run that runs around killing nodes that are more than 5 days old), and this talk helped connect the practices.
Optimising the SDLC is not however sufficient, Daniel Bryant from Ambassador highlighted the importance of reducing cognitive load for engineers and utilising paved roads to make doing things the right way, the easy way. Which echoes our engineering enablement team’s primary goal of avoiding engineers having to do any undifferentiated heavy lifting. Daniel was followed by a fantastic talk on change from Kevlin Henney which closed out the day.
In the spirit of continuous improvement, we are now gathering feedback from our colleagues to make next year’s conference 10x better.
Thank you to all our guest speakers - Daniel Bryant from Ambassador Labs, Jason Barto from AWS, Eilon Elhadad from Aqua Security, Antony Stubbs from Confluent, and Kevlin Henney.