Stop Counting Tokens: Measure AI Coding Cost by Merged Pull Requests

What happened

The article argues that companies should measure AI coding costs by the cost per merged pull request, alongside rework and defects, instead of relying on token counts or usage caps. It describes routing routine coding tasks to open-weight models and reserving frontier models for harder work. At HPE, a hybrid platform routes workloads from more than 150 internal applications to on-premises open-weight models, with reported token cost reductions of 30 to 50%.

Read next

• More changes in today’s briefingTypeSafe AI and Treg release open-source people search tool powered by Jev

The story so far

An MIT Technology Review article recommends measuring AI coding costs per merged pull request rather than relying on token counts or usage caps, and reports that HPE has reduced token costs by 30 to 50% through a hybrid model platform. The proposed measure divides total AI-assisted development costs by the number of changes that pass review and are merged, with rework and defects tracked alongside it. The article describes routing routine coding tasks to open-weight models and reserving frontier models for harder work. At HPE, the platform routes workloads from more than 150 internal applications to on-premises open-weight models; the reported savings apply to token costs.