Stargazer v2
Stargazer v2 estimates what it costs to run a language model on a device (memory, latency, energy and heat) before the model ships to it.
The problem
The cost of running a model on a device is discovered only after the model reaches the device.
A model that runs comfortably on a developer's machine can run out of memory, slow down under sustained use or drain the battery on the phones and boards it is shipped to. Each runtime, backend and device changes that cost, and nothing in a typical release checks it beforehand. Stargazer v2 studies how often this happens, measures it, and works towards estimating it before release.
From v1 to v2
Estimated the performance and resource use of clustering applications on an edge cluster of Raspberry Pi devices, from two static inputs (code complexity and input size), with regressions and a deep neural network.
Asks the same question of language models on end-user devices: the memory, latency, energy and heat of a model, estimated before it runs on the target. v1's method (measure, fit, validate) carries over; its input feature, hardware and task do not.
Where to go
Paper
The article in progress: problem, contributions, research questions and structure.
— contributions drafted · — Read about the paper →Literature
A close reading of the papers behind the argument, and the gap they leave open.
— read from PDF · — See the literature →Experiments
What was measured on real devices: the v1 reproduction and a laptop and phone testbed.
— devices · — models · pilot See the measurements →MSR Study
What practitioners report when the cost of running a model on their device surprises them, mined from GitHub issues.
Pilot labelling done · human labelling round pending Open the study →Team
Links
- Repository · code, data and these pages
- Issues · where the work is tracked
- Stargazer v1 fork · the reproducibility package the v1 reproduction starts from