The short answer on Ceph hardware recommendations for a small cluster: four identical nodes (three is the floor), enterprise SSDs with power-loss protection as OSDs, six CPU threads and at least 8 GiB of RAM per NVMe OSD, a separate boot SSD, and a dedicated 10 Gb/s network for Ceph traffic. Skip RAID controllers, bargain consumer SSDs and N100-class mini-PCs.
Below that line Ceph still runs. It runs badly at the worst moment, which is recovery, when a drive or node has died and the cluster is copying data to get back to full redundancy. Most of what follows exists to shorten that window. For daemon-by-daemon minimums rather than a shopping list, see the Ceph minimum hardware requirements breakdown.
Who should skip Ceph entirely
If your setup is one mini-PC and a NAS, you do not need Ceph. One box running ZFS, with snapshots replicated to the NAS, protects data better per dollar and per weekend than a three-node cluster you half understand, and an offsite cloud bucket is a perfectly good second copy.
Ceph earns its keep when storage has to survive an entire host going away: Proxmox HA with VM disks on RBD, Kubernetes persistent volumes, or CephFS shared across machines. Learning is a fair reason too, as long as you admit that is what you are doing, because a lab and the home of the only copy of the family photos have different hardware lists. Still choosing a platform? Read Ceph vs Gluster first.
What hardware does Ceph recommend per node?
Per node, Ceph’s documentation points to six CPU threads per NVMe OSD (three per HDD OSD), total RAM above twice the OSD count times the 4 GiB memory target, one enterprise drive of at least 1 TiB per OSD, a boot SSD of 256 GB minimum and 960 GB recommended, and 10 Gb/s or faster networking.
Those figures come from the Ceph hardware recommendations page for the Tentacle release, with Proxmox’s hyper-converged Ceph guide adding the parts that matter for a homelab:
| Component | Recommendation | Notes |
|---|---|---|
| CPU | 6 threads per NVMe OSD (4 minimum), 3 per HDD OSD (1 minimum) | Threads count when hyperthreading is on; a monitor wants 2 cores |
| RAM | More than OSDs × 4 GiB × 2, plus at least 20% extra | Proxmox: at least 8 GiB per OSD for good performance |
| OSD drives | Enterprise SSD with PLP, 1+ DWPD, 1 TiB or larger | One OSD per drive, capacity spread evenly across nodes |
| Boot and monitor | SSD, 256 GB minimum, 960 GB recommended | Monitor data wants about 100 GB per daemon |
| Controller | HBA in IT/JBOD mode, or none for NVMe | No RAID controllers |
| Network | 10 Gb/s dedicated to Ceph, 25 Gb/s for substantial workloads | Separate 1 Gb/s for management and corosync |
Run that for a node with three NVMe OSDs: 18 threads recommended, 12 as a floor, and 24 GiB of RAM for Ceph alone before the 20% buffer and before a single VM boots. On a hyper-converged Proxmox node that also runs your services, 64 GB is where it stops feeling tight.
Erasure coding pushes the CPU side up, since parity has to be computed on every write and every degraded read. That makes erasure coding versus replication partly a CPU budget decision, not only a capacity one.
Can you run Ceph on an N100 mini-PC?
You can, but only as a lab. Intel’s N100 spec sheet lists four cores and four threads, a 16 GB memory ceiling on a single channel, no ECC support and nine PCIe lanes. Ceph’s minimum for one NVMe OSD is four threads, so a single OSD claims the whole CPU before the monitor, manager or any VM gets a cycle.
Memory is no kinder. One OSD under the doubling rule wants 8 GiB and Ceph’s table asks for 5 GB or more per monitor, leaving very little of 16 GB for the OS and whatever you bought the box to run. ServeTheHome notes that most mini PCs are limited to one or two M.2 2280 SSDs, and PLP drives at that size are uncommon; Micron positions its M.2 2280 7450 with power-loss protection as a server boot drive. Onboard Ethernet is typically 1 or 2.5 GbE.
Almost everyone’s first Ceph cluster is three small boxes on a gigabit switch, and it is a good classroom. You will learn cephadm, watch CRUSH move data, and meet Ceph HEALTH_WARN when you pull a power cable on purpose. Do not put VM disks you care about on it. With one N100 and a NAS, the sane design is services on the N100, data on the NAS, and no Ceph.
Which drives should you buy for Ceph OSDs?
Buy enterprise SSDs with power-loss protection, rated at 1 DWPD or better and at least 1 TiB each, one OSD per drive. Ceph’s docs call bargain client-class or off-brand SSDs a false economy because they cliff: after an initial burst, sustained performance falls off considerably once a limited cache fills.
Endurance matters less than people assume. The docs say most OSD deployments do not need more than 1 DWPD, the read-optimized class, which is good news if you are shopping used. A used enterprise SATA or U.2 drive with healthy wear figures in smartctl makes a better OSD than a new consumer NVMe drive with bigger numbers on the box. Keep sizes even across nodes; Proxmox’s example is that four 500 GB disks per node beat one 1 TB disk plus three 250 GB disks.
If bulk capacity has to be spinning disks:
- Put WAL+DB on SSD, at four to five HDD OSDs per SATA SSD or up to fifteen per NVMe. Lose that SSD and every OSD behind it goes with it.
- Treat drives over 8 TB as capacity for large files and objects that are not performance-sensitive, which is how the docs frame them.
- Benchmark with the volatile write cache on and off; the docs say disabling it can dramatically raise IOPS and cut commit latency.
- Read Proxmox’s advice to prefer SSDs over HDDs in small setups as a recovery-time argument, not a speed one.
Skip the RAID controller. Proxmox says RAID controllers are not designed for the Ceph workload, and Ceph’s docs note RAID-mode HBAs may show higher latency than IT-mode ones. Use a plain HBA in IT mode for SATA or SAS, nothing at all for NVMe, and a ZFS or MD mirror for the boot drive if the chassis has two slots.
How fast does the Ceph network need to be?
A dedicated 10 Gb/s link is the realistic floor, and 25 Gb/s is the recommendation for substantial workloads. Ceph’s docs put the cost in recovery time: about three hours to replicate 1 TiB over 1 Gb/s versus twenty minutes over 10 Gb/s, and thirty hours versus three for 10 TiB. Slow recovery is a durability problem.
Proxmox’s Ceph benchmark paper from December 2023 is vendor-published and run on Proxmox’s own hardware, but its finding is blunt: a 10 Gbit/s network “can be easily overwhelmed,” with one very fast disk enough to make the network the bottleneck, and 25 Gbit/s can bottleneck too. For a homelab, 10 GbE is still the right buy. Your NVMe OSDs will outrun it, and that is fine. 2.5 GbE beats gigabit and is still a learning network.
Three nodes is where you can skip the switch. Proxmox’s full mesh guide cables three nodes directly; the simple routed setup gives the best performance, and routed-with-fallback survives losing one link by going through the third node. The same page recommends switches above three nodes or for clusters you expect to grow, so decide now.
Keep corosync and management on a separate 1 GbE network, as Proxmox recommends. Ceph’s docs also recommend bonding across two switches, because a single switch is a failure domain. Most homelabs accept that risk. Accept it on purpose.
Three builds that make sense
| Tier | Nodes | Per node | Ceph network | Good for |
|---|---|---|---|---|
| Lab | 3 × N100-class mini-PC | 1 SSD OSD, 16 GB RAM | 1 or 2.5 GbE | Learning cephadm, CRUSH and failure drills |
| Homelab with real data | 4 × 10 GbE mini-server (MS-01 class) | Boot SSD plus 2 PLP NVMe OSDs, 64 GB RAM | 10 GbE through a switch, or a mesh if you stop at 3 | Proxmox HA, RBD VM disks, CephFS |
| Small office | 4 to 5 used 1U/2U servers | IT-mode HBA, 8 to 10 OSDs, 128 GB RAM | 25 GbE bonded across two switches | Data someone would be fired for losing |
The middle row is where most readers should land. ServeTheHome’s MS-01 review describes dual SFP+ 10 GbE on an Intel X710, three M.2 slots taking 2280 or 22110 drives (one can take U.2, behind a voltage switch that can damage an M.2 drive if set wrong), and 64 GB maximum supported memory. That gives each node a boot drive, two PLP NVMe OSDs, a real Ceph network and RAM for VMs. Four nodes with two OSDs each is eight, short of the 12 Proxmox recommends, which is one more reason not to stop at three.
The bottom row follows the docs’ own example of a 1U server with 8 to 10 OSDs being well provisioned at 128 GB. Whichever tier you pick, grow one drive at a time with the cephadm OSD guide, keep the CRUSH failure domain at host level as covered in Ceph pools, placement groups and CRUSH rules, and remember none of this is a backup. Replication faithfully copies an accidental delete to every replica. Keep a copy on the NAS or offsite.
FAQ
how many nodes do you need for ceph
Three nodes is the minimum for a cluster that survives losing one, and four is the better recommendation. With three-way replication and a host failure domain, a three-node cluster has nowhere to rebuild after a host dies, so it stays degraded until that host returns. Proxmox recommends at least three nodes and 12 OSDs.
how much ram does ceph need per osd
Plan on 8 GiB of RAM per OSD. BlueStore’s default osd_memory_target is 4 GiB, and Ceph’s docs recommend total server RAM above the OSD count times that target times two, plus at least 20% extra. Proxmox also recommends at least 8 GiB per OSD. Below 2 GB is not recommended at all.
can i use consumer ssds for ceph
You can for a lab, but Ceph’s documentation calls bargain client-class SSDs a false economy. Client models can cliff, with sustained performance dropping considerably once a limited cache fills, and they lack the power-loss protection enterprise drives carry. For data you care about, buy enterprise drives with PLP rated at 1 DWPD or more, even used.
is 1gb ethernet enough for ceph
For a lab, yes; for data you rely on, no. Ceph lists 1 Gb/s as the bare minimum but recommends 10 Gb/s or more, because recovery traffic rides the network: the docs estimate three hours to replicate 1 TiB at 1 Gb/s versus twenty minutes at 10 Gb/s. Slower recovery means a longer window for a second failure.
