CarbonForge

Built for teams fine-tuning and serving open-weight models, CarbonForge ships optimized inference containers for open-source models, unlocking more performance from the GPUs you already have.

Every container bundles: 1) the model, 2) vLLM tuned for your workload, 3) our optimization layer on top.

You can drop it into your existing stack with same API and weights, we only change the serving configuration.

–

How it works: CarbonForge plugs into the existing compile/deploy path and searches for Pareto-optimal operating points that reduce joules per token without degrading model quality. The output is a reproducible before/after benchmark and a set of deployment configurations that meet SLA with lower energy: compiler and kernel choices, precision, batching, scheduling, and hardware-specific knobs.

The goal: Make efficient AI the default, so the next generation of inference can scale without power becoming the tax on progress.

The ecosystem: CarbonForge is proud to be a Mila-backed startup.