Hi, my name is
Mayank Deshpande.
I love to build Software
I'm a Perception Engineer at Contoro Robotics, where I own the stack that lets our robots actually see what they're picking up—cameras, LiDAR, segmentation models, and a whole lot of 3D geometry.
Outside of work my head is mostly in Multimodal and Embodied AI, and I build RAG and AI engineering projects on weekends just to see how far I can push them. I also read about markets and finance almost every day.
Always looking to work on exciting projects—feel free to reach out if you'd like to build something amazing together!
About Me
I absolutely love talking Football, Finance, and Startups—and honestly, I think the life would've been way more exciting if I was already working in one of these (but hey, I'm getting there!).
Something else I find really cool? Robots. I did my Master's in Robotics at Maryland, where I spent countless hours diving into Controls, Perception, and way too much C++. These days I'm a Perception Engineer at Contoro Robotics, owning the perception stack for a fleet of autonomous unloading robots—RGB-D cameras, LiDAR, segmentation models, calibration, and the 3D geometry that ties it all together.
What actually got me here was a course on Multimodal Foundation Models. I got completely hooked and couldn't stop exploring them, and that's still where my head goes outside of work: Embodied AI, plus RAG and AI engineering projects I build on weekends just to see how far I can push them. One thing I've realized along the way: it's incredibly valuable to master one area deeply before branching out—being able to claim mastery over a domain has so much cross-applicability (Thank You, Kyle!)
I'm excited for all the great conversations and adventures ahead as I keep exploring this fascinating journey!
- Languages: Python, C++ (11/14/17), CUDA, MATLAB, Bash
- 3D Perception: Open3D, PCL, RANSAC & plane fitting, ICP, camera calibration, TF2, occupancy mapping, 6-DoF pose estimation
- Deep Learning: PyTorch, Detectron2, Mask R-CNN, PointRend, DETR, SAM, DINOv2, ViT, ONNX, TensorRT
- Robotics & Tooling: ROS1/ROS2, RGB-D cameras, 2D/3D LiDAR, MoveIt, Gazebo, CARLA, Docker, Git, pytest, GoogleTest
Where I’ve Worked
Robotics Perception Engineer @ Contoro Robotics
Oct 2025 - Present
- Own the perception stack for the entire fleet of autonomous box unloading robots. RGB-D cameras and 2D/3D LiDAR feeding segmentation, point-cloud reconstruction, 6-DoF grasp poses, and obstacle detection.
- Raised segmentation recall 15 points on unseen sites and cut merged/split boxes 40%. Rebuilt the dataset pipeline around visual clustering and negative mining based on field heuristics.
- Train and ship the segmentation and detection models the robots run: PointRend, DETR, SAM and ViT backbones in PyTorch and Detectron2. Redesigned the complete ML development architecture to be scalable and maintainable, allowing developers to systematically evaluate and improve model performance.
- Cut inference time on the robot by 35% and got two models sharing one GPU, by fixing how frames are preprocessed and batched and when weights are held in memory. Exported through ONNX/TensorRT and quantized where accuracy allowed.
- Developed a per-site adaptive learning loop that learns from operator corrections while the robot keeps unloading, then swaps the new weights asynchronously.
- Took box dimension error from about 4 cm down to under 5 mm by measuring faces from mask pixels projected onto the fitted plane, improving the major pipeline algorithms and point cloud manipulation.
- Own calibration for the sensor rig: intrinsics, extrinsics, hand-eye. Solved as rigid-body transforms in SE(3), checked against fiducial targets, and developed an auto calibration pipeline.
- Wrote the point-cloud and 3D LiDAR layer: plane fitting, clustering, outlier rejection, occupancy mapping to construct a high accuracy World Model of the scene at a tight latency budget.
- Developed strategies to tackle the common depth issues like dark surfaces drop out, depth edges throw flying pixels, and multipath interference. That work drives sensor choice and mounting geometry for the future hardware decisions.
Some Things I’ve Built
Featured Project
Humanoid Robot Gait Generation via Imitation Learning
Developed a pipeline to enable a humanoid robot to learn human-like walking gaits using imitation learning techniques. Leveraged the Human3.6M motion capture dataset, processed via motion retargeting, to serve as expert demonstrations within the PyBullet physics simulator. Implemented and trained Actor-Critic agents using Proximal Policy Optimization for Imitation Learning (PPO-IL), demonstrating successful motion imitation of stable walking behaviors.
- Python
- PyBullet
- Reinforcement Learning (PPO-IL)
- Imitation Learning
- Stable-Baselines3
![Humanoid Robot Gait Generation via Imitation Learning]()
Featured Project
Adaptive RL-MPC for Autonomous Lane-Changing
A hybrid RL-MPC framework for autonomous vehicle lane-changing, integrating Soft Actor-Critic/PPO/TD3 to dynamically adjust risk-aware weights based on traffic conditions.Designed and trained the RL agent using a curriculum-based protocol in the SUMO simulation environment, achieving 30% higher success rates, 25% lower collision rates, and 20% faster lane-change completion compared to traditional MPC systems.
- Python
- SUMO
- Reinforcement Learning (SAC/PPO/TD3)
- Model Predictive Control
- TraCI
![Adaptive RL-MPC for Autonomous Lane-Changing]()
Featured Project
Temporal Coherence Evaluation in Video-Language Models
Developed a framework to evaluate temporal coherence in multimodal foundation models for video understanding tasks, introducing metrics like CLIPGain for temporal consistency in video captioning and leveraging BERTScore for semantic analysis in video QA. The project was tested on benchmarks such as TOMATO and MSR-VTT, providing nuanced insights into temporal reasoning. Open-sourced the methodology to guide advancements in time-aware AI research.
- Python
- PyTorch
- CLIP
- BERT
- HuggingFace
![Temporal Coherence Evaluation in Video-Language Models]()
Featured Project
VisualOdom-Particle-Filter
A vision-based localization and visual odometry system for drones utilizing particle filters and advanced image encoding techniques. This project integrates CNN and VecKM (Inner product sum) based encoders, and Histogram of Features encoders to accurately estimate similarity score to update the weights of particle filter and effeciently localize the drone.We were able to validate the performance through simulation in Gazebo and ROS2.
- Python
- ROS2
- OpenCV
- PyTorch
- Particle Filter
- CNN Encoders
![VisualOdom-Particle-Filter]()
Featured Project
Structure from Motion
A robust Structure from Motion (SfM) pipeline implemented in Python, utilizing OpenCV and Open3D to reconstruct accurate 3D structures from sequential 2D images. This project processes image pairs to detect and match features, recover camera poses, perform triangulation, and visualize the resulting 3D point cloud, demonstrating effective 3D reconstruction and visualization capabilities.
- Python
- OpenCV
- Open3D
- NumPy
- Matplotlib
- SciPy
![Structure from Motion]()
Featured Project
MultiRobot Search and Rescue
A swarm robotic system for Search and Rescue operations, developed with C++ and ROS2. It features autonomous navigation, real-time object detection using YOLOv5, and seamless multi-robot coordination. The system supports dynamic robot spawning via configuration files or in-code settings, integrates with Nav2 for advanced path planning, and utilizes Gazebo for realistic simulations.
- C++
- ROS2
- OpenCV
- GoogleTest
- Doxygen
- YOLOv5
![MultiRobot Search and Rescue]()
Featured Project
Right Invariant Extended Kalman Filter (RIEKF) for Object-Based SLAM
A classic Python implementation of the Right Invariant Extended Kalman Filter (RIEKF) based on the seminal paper "A Right Invariant Extended Kalman Filter for Object-Based SLAM." This project provides a robust framework for simultaneous localization and mapping (SLAM) using object-based observations. I have tried to do detailed documentation and keep a modular code structure to facilitate easy integration in practical applications.
- Python
- NumPy
- SciPy
- Matplotlib
![Right Invariant Extended Kalman Filter (RIEKF) for Object-Based SLAM]()
Featured Project
Quadruped Gait Simulation
A simulation of a quadruped robot developed using C++ and ROS2 Humble. This project includes a detailed URDF model exported from SolidWorks, ROS2 packages for simulation and teleoperation. It enables autonomous navigation and real-time control through Joypad commands, offering a flexible environment for testing and refining quadruped gaits. Currently, the implementation supports trotting and crawling gaits, with plans to incorporate additional gait patterns in the future.
- C++
- ROS2
- Gazebo
- Python3
- URDF
- Solidworks
![Quadruped Gait Simulation]()
Other Noteworthy Projects
view the archiveAutoPano
Developed an automatic panorama stitching solution using traditional techniques and deep learning models (HomographyNet), achieving high-quality results with supervised and unsupervised learning, validated on synthetic and real-world image sets.
Human detection and Tracking
A C++ module for detecting and tracking human obstacles using Monocular Camera with ResNet & OpenCV integration within the robot's reference frame.
Stereo Vision
A Python-based stereo vision pipeline to accurately generate disparity and depth maps from stereo image pairs with OpenCV and NumPy.
Pb-Lite and DL for Edge Detection
An edge detection implementation that integrates Probability-based methods (Pb-Lite) with deep learning models like DenseNet, ResNext and a cutom architecture to improve accuracy and robustness.
AutoCalib
A Python implementation of Zhang's camera calibration method, automating checkerboard detection and accurately estimating camera parameters using OpenCV and NumPy.
LQR and LQG for stabilizing Two Pendulum cart
Modeled and controlled a two-pendulum crane system using cutom LQR and LQG controllers implementation in MATLAB.
Publications
This paper evaluates the integration of the Robotics Middleware Framework (RMF) with Free Fleet, analyzing the performance of different path planning algorithms in multi-robot scenarios to enhance autonomous mobile robot fleet management.
What’s Next?
Get In Touch
I'd love to connect! Feel free to email me, check out my resume above, or visit my LinkedIn page to learn more.
Say Hello