Google Summer of Code '25 — Keploy

Open-source contributor · AI-powered code review · Go + JavaScript Jun – Sep 2025 · Remote

Keploy is an open-source no-code testing platform. It captures real API calls and replays them as tests — keeping actual production behavior as the ground truth, eliminating the work of writing test suites by hand. Over the past three years it's been selected as a GSoC organization and has grown a community of contributors around the idea that testing infrastructure shouldn't require manual scaffolding.

For GSoC 2025, I picked the hardest project on their list: build an AI-powered code review agent for Golang and JavaScript that runs on CI, requires no paid APIs, and works for any open-source project with zero configuration.

No vendor lock-in. No "works on my machine." A GitHub Action that reviews every PR and tells you what's wrong — free, accurate, and fast.

What I built

The KeployAI Code Review Agent is a Go-based GitHub Action that performs inline static analysis on every pull request — security vulnerabilities, performance issues, best practices, style. It generates inline comments directly on the diff, in the same place reviewers look.

The core constraint I designed around: it had to be free for open-source projects. GitHub provides free hosted runners. GitHub-hosted Models are free for public repos. I built around that constraint first, then added a self-hosted Ollama fallback for teams who want to run it on their own infrastructure.

The technical work

LLM benchmarking and selection

I didn't start with a model. I started with a benchmark. Evaluated multiple open-source LLMs against real Go and JavaScript codebases — measuring accuracy, latency, context window utilization, and false positive rates. The right model wasn't the biggest one; it was the one that gave the best precision per token on actual code review tasks.

This process produced the model selection that shipped in the agent — not a vendor recommendation, but a measured decision.

Quantized GGUF deployment

Deployed quantized GGUF models for the self-hosted Ollama path. The result: memory usage halved, inference time cut in half — accuracy maintained. Without quantization, self-hosted inference was either too slow for CI or too expensive to run for most teams.

Chunking and latency optimization

Large PRs break naive LLM integrations. A 40-file diff doesn't fit in any context window, and splitting it blindly loses the context the model needs to reason. I designed chunking algorithms that split diffs intelligently — preserving enough surrounding context for coherent analysis, discarding irrelevant noise. The result: 300% reduction in response latency and reliable multi-file handling at scale.

Smart file prioritization

Not all files in a PR are equally risky. I built a prioritization layer that identifies critical files — core logic, authentication surfaces, security-sensitive paths — and reviews them first. This optimizes token budget and review quality simultaneously: the things most likely to matter get reviewed most carefully.

Multi-provider fallback

The agent routes to GitHub-hosted Models by default. If unavailable, it falls back to a local Ollama instance automatically. No configuration required for the default path; full control available for teams that want it.

CI/CD pipeline

Built the GitHub Actions workflows that power the fully automated review cycle — every PR, every commit, zero manual steps. Standard GitHub token only. Setup time for a new repo: minutes.

Impact

• Code review time on CI: hours → minutes
• Response latency: down 300% vs. naive LLM integration
• Self-hosted inference: memory and time halved via quantized models
• Free to run on any open-source project — no API keys required
• Available on the GitHub Marketplace

What it took

Four months, ~350 hours. The gap between "this works in a notebook" and "this ships in CI for thousands of repos" is large, and closing it is an operational problem as much as a technical one.

It taught me to think about latency budgets before features, cost constraints before capabilities, fallback paths before happy paths. The same instincts I'd carry into production systems work later.