Skip to content

Md. Saidur Rahman Pulok

Machine Learning Engineer at Optimizely · Dhaka, Bangladesh

I work on making large language models cheaper and faster to run: how to serve them, which model to send a request to, and how much compute to spend on reasoning.

At Optimizely I work on LLM evaluation, model routing, fine-tuning and the production ML pipelines behind them. Before that I shipped AgroGPT and an on-premise face-recognition system at ACI, and built backend services for a US clinical trial platform.

Outside work I build and measure LLM systems from the ground up, on hardware I own, and publish the code and the numbers. I studied Mechanical Engineering at BUET, where my undergraduate research applied machine learning to gas pipeline leak detection.

Photo of Md. Saidur Rahman Pulok

Research interests

  • Efficient inference and serving

    Batching, scheduling, and KV-cache and memory management for LLM serving, under the tight budgets of real hardware.

    See AxiomServe →
  • Cost-aware model routing

    Sending each request to the cheapest model likely to answer it well, and measuring honestly when that beats a single model.

    See routewise →
  • Inference-time compute

    When extra sampling and verification are worth their tokens, and what verifiers actually learn.

    See VeriSearch →

Selected work

A cost–quality benchmark for LLM routing

Can a learned router beat every single model on both cost and accuracy?

Benchmarked learned routers on public HELM results covering 8.5k prompts and 22 models, with 5-fold cross-validation and bootstrap confidence intervals.

At equal accuracy, routing cost 48% less than the best cheap model (DeepSeek-V3) on 2024 models and 25% less than the best model (Qwen3) on 2025 models, and beat every single model by 3 to 4.5 points.

Project pageCode
  • Python
  • scikit-learn
  • HELM
  • Bootstrap CIs
Accuracy against cost per 1,000 prompts on HELM Lite. The learned routers' curves sit above every single model and above the best mix of two models.
Click to enlarge

An LLM serving runtime built from scratch

What does it take to serve an LLM well, without generate() doing the work?

Batched prefill and token-by-token decode, KV-cache reuse, continuous batching, admission control and scheduling, with streaming and cancellation, benchmarked on a laptop GPU.

At 8 concurrent requests, continuous batching cut p95 time-to-first-token to 119 ms, versus 503 ms for static batching, on an RTX 5070 Laptop GPU.

Project pageCode
  • Python
  • PyTorch
  • Transformers
  • FastAPI
  • asyncio
Median time to first token against client concurrency. Sequential execution grows to about 2.8 seconds at 8 clients, while static and continuous batching stay near zero.
Click to enlarge

Verifier-guided reasoning under token budgets

Can a small learned verifier pick better answers than majority voting when the token budget is fixed?

Sampled GSM8K solutions from Qwen2.5-Math-1.5B, trained a DistilRoBERTa verifier on them and compared selection strategies under hard token caps, end to end on an 8 GB laptop GPU.

Early stopping used 35% fewer tokens for about one point of accuracy. The verifier tied with voting, and its high AUROC came from an answer-format shortcut (0.47 on extracted answers).

Project pageCode
  • Python
  • PyTorch
  • Transformers
  • Qwen2.5-Math
  • DistilRoBERTa
Answer accuracy against mean generated tokens for three token caps. Early stopping moves each cap's point left with almost no drop in accuracy.
Click to enlarge

Publications

  1. Integrating Machine Learning for Enhanced Detection and Localization of Gas Pipeline Leaks

    M. S. Rahman, F. Khandoker, S. S. Arnov, M. A. Islam

    IUT International Conference on Core Engineering and Technology (ICCET), 2024

Experience and education

  1. Apr 2026 – Present

    Optimizely

    Machine Learning Engineer I

    Designed a 46-task, 13-category benchmark to evaluate Optimizely's AI desktop app, Mark (formerly Opal), against third-party AI tools. Built an on-device LiteLLM routing proxy that sends each request to a model sized for the task, used daily by the team. Designed an Argo Workflows ML pipeline on EKS, productionized 2 ML pipelines on Snowflake data, and ran QLoRA fine-tuning R&D on Qwen2.5 for customer-data recall.

  2. Aug 2025 – Apr 2026

    ACI PLC

    Machine Learning Engineer

    Productionized an on-premise face-recognition attendance system that replaced fingerprint check-in for 50+ employees (FastAPI, Redis Streams, RTSP cameras, InsightFace on ONNX/CUDA, Qdrant). Built and launched AgroGPT (agrogpt.app), a RAG platform for precision agriculture, with Airflow pipelines for weather and market data and Prometheus/Grafana monitoring.

  3. Mar 2025 – Jul 2025

    Brick Line Technology Ltd.

    Jr. Software Engineer

    Built Python backend services for Zeda (zeda.ai), a clinical trial management system used by several US clinics, on FastAPI and AWS (RDS, DynamoDB, S3, Lambda). Implemented MFA with AWS Cognito and Twilio, and shipped several generative AI features.

  4. 2019 – 2024

    Bangladesh University of Engineering and Technology (BUET)

    B.Sc. in Mechanical Engineering

    Undergraduate research on machine learning for gas pipeline leak detection and localization.

News

Contact

I'm happy to talk about research collaborations, graduate opportunities, roles in ML systems, or anything on this page. Email is the fastest way to reach me.

saidur.pulok@gmail.com