Jarvis
0.904 ROC-AUC on face verification, vs 0.837 for a ResNet18 baseline
- Team project
- 2026
- Multimodal
A multimodal assistant that recognises its user, then plans and delegates work through tool-calling agents.
Project information
- Team project
- 2026
- PyTorch, FastMCP, FastAPI, Metric learning, Multi-agent systems
- 0.904, ROC-AUC on face verification, vs 0.837 for a ResNet18 baseline
Overview
Jarvis is a multimodal assistant built by a five-person team. It recognises who it is talking to, then gives them conversational access to vision tools, scheduling and delegated tasks from a phone.
My part
Face verification. I built a PyTorch Siamese network trained with metric learning and triplet loss. It reached 0.9044 ROC-AUC across 8,805 validation image pairs, ahead of the ResNet18 classification baseline at 0.8368.
Enrolment without retraining. New users register live through a webcam. Their reference images update the identity embeddings, so the recognition model never needs retraining. Anti-spoofing checks guard the enrolment flow.
The agent core. I extended the system into an assistant: a supervisor agent that tool-calls sub-agents (a planning agent and a work agent) through a FastMCP loop, served to a FastAPI mobile web interface.
0.904ROC-AUC on face verification, vs 0.837 for a ResNet18 baseline