Choose a batching boundary: 1000 tiny requests at 5 ms each vs one batch at 400 ms. Which wins at high load, and why?

prerequisite: NN-009

Context fixture

{
  "serial": "1000 × 5ms",
  "batch": "400ms"
}

Hints · 0/3

Your answer

Opening…