The question buried in your next vendor contract
There is a single line item that determines whether your AI vendor is a FERPA problem or not, and it almost never appears in the sales deck. It is not model quality. It is not token pricing. It is this: where does inference actually happen?
If the answer is "our cloud," then every prompt a twelve-year-old types (their essay draft, their math struggle, the personal detail they accidentally paste in) leaves your building, crosses a network, and lands on someone else's servers. If the answer is "on the device," that data never leaves. Same feature, radically different legal exposure. And most procurement processes treat this as a footnote when it should be the first question on the call.
By mid-2026 this stopped being a theoretical distinction. Google, Apple, and Microsoft have all shipped sub-4B-parameter models that run locally: Gemini Nano on Android and Chrome, Apple's on-device foundation models exposed through the Foundation Models framework, and Microsoft's Phi line running comfortably on a Copilot+ NPU. These are not toys. A well-tuned 3B model handles summarization, feedback, rubric-based grading, and Socratic questioning at a quality that would have required a cloud round-trip eighteen months ago. "Run it locally" graduated from aspiration to deployment option this year.
Why edtech coverage gets this wrong
Every write-up I read frames on-device AI as a cost or latency win. Both are real. You dodge per-token API bills, and you kill the 400ms to 2s network round-trip that makes an AI tutor feel laggy in front of a distracted kid. We covered the latency and infrastructure side of this argument in Rethinking Tech Strategies Amid Rising Hardware Costs, and it holds.
But for K-12 specifically, cost and speed are the least interesting part. The real story is data residency.
Here is the mechanism that matters. FERPA and COPPA compliance friction comes almost entirely from data leaving your control. The moment student prompts hit a vendor's servers, your legal team has to answer a cascade of questions: What is retained? For how long? Is it used for training? Where are the servers physically located? Who at the vendor can read it? What is the sub-processor list? Is there a signed data processing agreement that survives the vendor getting acquired? These reviews routinely stall deployments for two or three months, and sometimes they kill them outright.
Local inference does not answer those questions. It deletes them. If the prompt is processed on the student's Chromebook or iPad and the response is generated there, there is no data flow to govern, no retention policy to audit, no sub-processor to vet. You cannot leak data you never transmitted. The compliance surface collapses to nearly nothing.
That is a qualitatively different pitch. "Our AI is smarter" is a claim you cannot verify in a procurement cycle and that decays every time a competitor ships a new model. "Your students' homework never touches our data center" is an architectural fact you can write into a contract and hand to your general counsel.
What this means in a real evaluation
Make "where does inference happen?" a first-order question, and watch how fast it separates vendors. Concretely, take this into your next call:
- Ask for the data flow diagram, not the privacy policy. A privacy policy tells you what a vendor promises to do with data they receive. A data flow diagram tells you whether they receive it at all. Those are not the same document, and the second one is the one that matters.
- Distinguish on-device from "edge" and "private cloud." Vendors will blur these. On-device means the model runs on the student's hardware. "Edge" often means a regional data center, which is still their infrastructure and still a data transfer. "Private cloud" means your data lives in an isolated tenant on someone else's servers, which is better than shared multi-tenant but still a transfer with a DPA attached. Only true on-device inference removes the data flow entirely.
- Ask what breaks offline. This is a cheap diagnostic. If the AI features still work with the network cable pulled, inference is genuinely local. If they degrade or die, prompts are going somewhere.
- Ask about the hybrid fallback. Many realistic deployments run small models locally and escalate hard queries to the cloud. That is fine, but you need to know exactly which queries escalate and whether student PII rides along when they do. The honest answer is a routing rule you can inspect, not a shrug.
None of this requires you to believe on-device is always the right choice. A local 3B model will not match a frontier cloud model on the hardest reasoning tasks, and some workloads legitimately need the big model. The point is that you should be making that tradeoff deliberately, with the data-governance cost priced in, rather than defaulting to cloud because the sales engineer never raised the alternative.
The decision window is now
This is not evergreen advice, and that is the part worth sitting with. Contracts signed this quarter lock in an architecture for years. If you commit to a cloud-only vendor in 2026, you have committed your student data flows to their servers through the length of that agreement, right at the moment local inference became a genuine option. Waiting a year does not cost you a feature; it costs you the chance to pick the architecture before it hardens.
We build Omega Copilot to keep inference as close to the student as the task allows, precisely so the FERPA and COPPA conversation is short rather than a three-month review. That is an architecture decision we made on purpose, not a marketing line.
If you are evaluating any AI-in-education tool this year, put one question at the top of the list before pricing, before model benchmarks, before the demo: where does inference happen? Everything downstream, including whether your legal team says yes in a week or in a quarter, follows from the answer. For more on how we think about these architecture-level tradeoffs, the running thread starts at our README.