Lessons / Part 6. Running a model on your own computer
32 Write the jobs down before buying hardware
List the jobs and what counts as working, and the memory each needs tells you whether any hardware is needed at all.
What people usually ask
What GPU should I buy to run AI locally in my lab? Budget is about
2000 dollars.
What comes back
A graphics card recommendation with a benchmark table, chosen before anyone said what the card would be asked to do.
Ask this instead
Do not recommend hardware yet. Privacy rule, one line: lab data
never leaves the building, so every job below runs on the machine
in the attached specification.
Jobs:
1. Summarize supplier datasheets, PDF, 5 to 40 pages. Working means
the summary names every limit value in the sheet.
2. Draft procedures from my bullet notes. Working means a draft I
edit for ten minutes rather than rewrite.
3. Fit yield against four inputs on a table of 60 runs. Working
means a held out score beside a mean predictor.
For each job: the smallest model class that does it on this machine,
its memory need in GB beside the card memory and the system memory
listed, and whether it needs new hardware at all. Say plainly if
job 3 wants processor cores rather than a card. One table, then a
yes or no on buying anything, then stop.
Attach: The job list with its working test for each, and the machine’s specification (processor, system memory, card and card memory).
Why it works: The hardware follows from the jobs and their memory needs, and a tabular machine learning job on sixty rows needs no card at all.
Drafted with Fable 5.1 from my notes; chosen and edited by me.