The Climate Risk Group: Batch API development

Product Overview

The Climate Risk Group is a group of companies committed to quantifying and communicating the costs of climate change. Our mission is to influence decision makers – large and small – towards action that prevents catastrophic climate change using innovation, empathy and logic.

The Climate Risk Group harnesses the science and technology of the award-winning Climate Risk Engines, the global platform of XDI, and the household level insights of Climate Valuation. Our work leverages physical risk data, influence with global players and grass roots momentum to create a compelling case for change towards a resilient and zero carbon world.

Responsibilities

I served as a Software Developer with a primary focus on leveraging the Rust programming language to enhance the company's technological capabilities. My core responsibilities revolved around the development and maintenance of a mission-critical API that interfaced with our sophisticated risk analysis engine.

The core of the analysis engine was crafted in Python, which is renowned for its versatility and ease of use. However, as each analysis task demanded a significant amount of time to complete, I contributed to optimizing the system by using Rust as a powerful tool for performance enhancement. Rust's efficiency, speed, and memory safety features made it the perfect choice for this task.

One of the key challenges we faced was efficiently managing batch API calls from the Rust API to the risk analysis engine. To address this, we deployed our Rust application alongside NATS on a Kubernetes cluster. This architecture allowed for high-performance communication and seamless data transfer between the two software packages while ensuring reliability and scalability.

To further optimize the system's scalability and resource utilization, Kubernetes horizontal pod auto-scaling was used. This dynamic scaling mechanism ensured that we allocated resources as efficiently as possible, enabling us to handle both peak loads and periods of lower demand with equal ease.

Throughout my tenure at the company, my primary focus was on the development and continuous improvement of this Rust package. This involved designing and implementing efficient algorithms, optimizing code for performance, and ensuring robust error handling to maintain the highest level of reliability.