MapReduce Infrastructure

Overview A distributed, fault-tolerant MapReduce implementation built from scratch in C++ with gRPC and multithreading capabilities. Architecture Master-Worker Architecture: Centralized job scheduling and coordination Fault Tolerance: Automatic task reassignment and recovery mechanisms Load Balancing: Dynamic task distribution across worker nodes Network Communication: gRPC-based inter-node communication Key Features Distributed Processing: Handles large-scale data processing across multiple nodes Fault Recovery: Automatic detection and recovery from node failures Scalability: Supports dynamic addition/removal of worker nodes Performance Monitoring: Real-time metrics and progress tracking Technical Implementation Language: C++ for high performance Communication: gRPC for efficient network communication Concurrency: Multithreading for parallel task execution Storage: Efficient data serialization and storage management Use Cases Large-scale data processing Distributed computing research Educational purposes for understanding MapReduce concepts View on GitHub →

1 min · 123 words · Raj Shah

Quant Developer Intern

Overview Quant Developer Intern at Barclays Investment Bank in the Statistical Modelling and Development team, focusing on high-frequency trading systems and performance optimization. Key Achievements 23% p99 latency improvement by profiling the Java quoting pipeline with JMH, diagnosing cross-socket NUMA traffic in a JNI-backed C++ library, and optimizing synchronization 2.6× throughput increase and 50% message size reduction by migrating inter-process communication from string-encoded messages to Protocol Buffers Real-time event stream design to convert yield-space quotes to price-space valuations for unified fair pricing Technical Contributions Performance Optimization Profiled Java-C++ quoting pipeline using JMH (Java Microbenchmark Harness) Diagnosed and resolved cross-socket NUMA traffic issues in JNI-backed C++ libraries Optimized synchronization mechanisms for reduced latency System Architecture Migrated IPC from string-encoded messages to Protocol Buffers Designed real-time event streams for yield-to-price space conversion Implemented unified fair pricing mechanisms Technologies Languages: Java, C++ Performance: JMH benchmarking, NUMA optimization Communication: Protocol Buffers, JNI Systems: High-frequency trading infrastructure

1 min · 152 words · Raj Shah