Your own agents
on your own hardware
Demystified
The power of technology has never been more accessible. The gap between ordinary people and systems of their own has never been smaller. We’re here to bridge that gap, connecting agency to capabilities.
Sizing your needs
The right setup depends on what your business needs to do, how much work happens at once, and how quickly the work needs to finish.
1. Where does AI fit?
Not everything needs AI. Many business tasks can be handled by ordinary software. The first step is understanding what your business needs done and where AI would be useful.
The AI used to build and improve your software is a separate choice from the AI used during normal business operations. Providers such as OpenAI and Anthropic offer powerful frontier models without requiring you to own expensive hardware. However, using frontier models during normal business operations is expensive. That’s when running your own models on your own hardware becomes viable.
2. When does the work happen?
A system supporting your real-time operations needs enough capacity to handle all the work across your entire system all at the same time. Overnight jobs can take longer, spreading the work out over more hours and requiring less capacity, and cheaper hardware.
3. How busy does it get?
Twenty connected devices do not mean twenty jobs running at once. What matters is how much and what type of work each needs done. Existing usage records can reveal those patterns, giving us a starting point for estimating hardware needs.1
4. How critical?
Can an outage affect your bottom line? Then redundancy is a no-brainer.2 Can your business keep running while the system is restored? Then a cheaper hardware setup may be a viable option.
5. What computer do you need?
- CPU: runs business software and databases, same as the computers you already use.
- GPU: the AI part of a computer.3 Your work defines the models you run, and the models define your GPU.
- Memory: smarter AI, longer conversations, and more work running at once require more memory.
- Storage: more software, larger files, and longer backup history require more storage.
- Network: more locations and more data moving at once require more network capacity.
6. How can you be sure?
New worlds are hard to navigate. We’ll show you what’s possible and help you get there. Contact us to take the first step.
A worked example: twenty restaurants
Twenty restaurants, one shared system.
1. Where does AI fit?
| Restaurant operations | |
|---|---|
| Taking, routing, displaying, and printing orders | Software / CPU |
| Tracking inventory and ordering supplies | Software / CPU |
| Scanning labels to check in deliveries | Software / CPU |
| Managing and updating menu boards | Software / CPU |
| Staff scheduling | |
| Interpreting availability and scheduling preferences | Local AI / GPU |
| Tracking availability | Software / CPU |
| Sending notifications | Software / CPU |
| Checking PTO balances | Software / CPU |
| Recording call-outs and missed clock-ins | Software / CPU |
| Generating schedules | Local AI / GPU |
| Confirming availability | Local AI / GPU |
| Handling shift changes | Local AI / GPU |
| Finding cover for no-shows | Local AI / GPU |
| Staff assistant | |
| Questions and instructions | Local AI / GPU |
| Records, permissions, and approved actions | Software / CPU |
| Work between restaurants | |
| Equipment, capacity, and transfers | Software / CPU |
| Interpreting staff requests | Local AI / GPU |
| Making improvements to any of the software above | Cloud AI |
2. When does the work happen?
Orders, kitchen updates, staff questions, and last-minute staffing changes need quick responses during the dinner rush. Schedule planning, document processing, and software development can wait until after it, spreading the work over more hours.
3. How busy does it get?
Assume each restaurant takes 300 orders an hour. Each order sends twenty requests—pieces of work—to the shared system for routing, stock updates, and records.
Add seven kitchen displays, twenty active staff devices, and three menu displays per restaurant. The heavier case doubles orders and staff activity.
| Work | Calculation | Dinner rush · requests/second | Heavier demand · requests/second |
|---|---|---|---|
| Orders | 20 restaurants × 300 or 600 orders/hour × 20 operations ÷ 3,600 | 33.33 | 66.67 |
| Kitchen displays | 140 displays checking every 2 seconds | 70 | 70 |
| Staff apps | 400 devices making one request every 10 or 5 seconds | 40 | 80 |
| Menu displays | 60 displays checking for updates every 60 seconds | 1 | 1 |
| Total | Sum of the four workloads | 144.33 | 217.67 |
Menu displays store their media locally and check for updates. Stock and record updates are already counted with orders. The heavier case explores demand, not a five-year forecast.
AI answers
For AI, size for the answers needed at once and how quickly they must arrive. Reading a question and generating an answer place different demands on the computer.4
A token is a small piece of text. These targets describe how many answers arrive together and how quickly they appear:
| Target | Dinner rush | Heavier demand |
|---|---|---|
| Simultaneous responses | 10 | 20 |
| Input per response | 2,000 tokens | |
| Output per response | 300 tokens | |
| Generation rate per response | 20 tokens/second | |
| Combined generation rate | 200 tokens/second | 400 tokens/second |
| Input if all requests arrive together | 20,000 tokens | 40,000 tokens |
| Time to first token | Within 3 seconds | |
A 300-token answer at that rate takes about 15 seconds after it starts. Longer conversations, reasoning, and tool use add work.
4. How critical?
Essential work must continue if a computer fails. That calls for two computers, each sized to carry the essential heavier workload alone. Splitting a model across both would defeat that protection. The second computer also needs current data, a way to take over, and safeguards against repeating actions.2
5. What computer do you need?
The equipment
The budget uses two ASUS GX10 computers, each with a 20-core Arm CPU, NVIDIA Blackwell GPU, 128 GB of shared memory, and a 1 TB SSD.5 Existing screens, staff devices, network, and backup power are reused.
| Equipment | Quantity × unit price | Total |
|---|---|---|
| ASUS GX10 · 128 GB / 1 TB | 2 × $5,999 | $11,998.00 |
| Toshiba N300 · 12 TB | 4 × $699 | $2,796.00 |
| OWC dual-drive enclosure | 2 × $199.99 | $399.98 |
| Central hardware | Before tax | $15,193.98 |
The GX10 combines ordinary computing, AI processing, and model memory. Software compatibility with Arm/Linux and combined performance still need testing.
CPU and network
CPU demand depends on how much work arrives together and how much processing each request needs. Assume 5–20 milliseconds per request, with 40% spare capacity:
CPU capacity = requests/second × CPU seconds/request ÷ 0.60
| Estimate | Dinner rush | Heavier demand |
|---|---|---|
| CPU capacity with headroom | 1.20–4.81 core-equivalents | 1.81–7.26 core-equivalents |
| Network data before overhead | 23.09 Mbps | 34.83 Mbps |
These figures estimate processing demand. Actual CPUs differ, and AI uses the CPU too. The spare capacity is assumed, not measured.
For the network, assume 20,000 bytes sent and received per request:
Mbps = requests/second × 20,000 bytes × 8 ÷ 1,000,000
Allow more network capacity for retries, AI, backups, video, and keeping the computers in sync. Check the restaurant links, central connection, and local network.
Memory
Memory = model weights + conversation cache + runtime workspace + applications/database + operating headroom
A model’s stored values—its weights—are only the starting point. A 32-billion-parameter model at 4 bits needs about 16 GB for raw weights: parameters × bits ÷ 8. Conversations, software, and operating overhead need more. Enough memory to load the model does not prove it will answer quickly.4
The model and its storage format remain to be selected and tested for quality and speed.
Storage and backups
Each enclosure holds two mirrored 12 TB drives: 12 TB usable at each of two locations, not 48 TB of unique storage. Allow room for files and backup history. Mirroring protects against drive failure; restoring deleted files needs backup history too.6
Running costs
The electricity budget assumes continuous operation: two 240 W computer adapters, two 72 W enclosure supplies, 90% conversion efficiency, and 100 W of added load on reused servers. At $0.1447/kWh:
((2 × 240 + 2 × 72) ÷ 0.90 + 100) ÷ 1,000 × 8,760 × $0.1447 = $1,005.61/year
These are adapter ratings, not measured consumption. Enclosure power includes the disks; reused servers add no purchase cost. Specifications: ASUS and OWC enclosure.
Reserve $15,193.98 ÷ 5 = $3,038.80/year for equipment replacement over five years. IT support is budgeted separately in the restaurant example. That example also includes an illustrative $200 a month across the group for cloud AI used in software improvements. Cloud usage is separate from the local-computer capacity estimates.
6. How can you be sure?
Run the software, database, and model together at dinner-rush demand. Then test the heavier load with one computer unavailable. Check that:
- at least 95% of order-to-kitchen updates arrive within one second, stock stays correct, and actions are not duplicated;
- AI answers meet the agreed quality, speed, and simultaneous-response targets;
- memory and storage cover real conversations, files, and backup history;
- essential work recovers after a computer or network failure, and background jobs finish.
The dinner rush sets the capacity. The cost of an outage determines the backup plan. Testing the complete system checks that both are covered.
Sources
- Google SRE: Monitoring Distributed Systems — demand, response times, errors, resource saturation, and capacity planning.
- Google SRE: Reliable Product Launches at Scale — capacity, redundancy, and failure planning. The restaurant inputs are our assumptions.
- Intel: AI Accelerators — how GPUs accelerate AI workloads.
- NVIDIA: Inference Optimization — prompt processing, generation, model weights, conversation cache, and concurrency.
- ASUS GX10 specifications — processor, memory options, interfaces, and power supply. Retailer links identify the selected configurations and reference prices.
- OWC: RAID vs. Backup — drive mirroring and protection against deletion. The linked enclosure specifications list its bays, RAID modes, and 72 W power supply.