Multiple SATA Controller Question

[zfrogz]

So, I recently picked up an Asrock C2550D4I board to replace my aging non ECC board. I've got 6 drives in my zpool which matched the number of ports on my old mobo but this new one has a whopping 12 sata ports across 3 different controllers on a mini ITX board. My question is this... Would it be better to balance the drives across the different controllers to spread out the workload and prevent a bottleneck or would it be more efficient to cram them all onto a single controller?


- Intel® C2550 : 2 x SATA3 6.0 Gb/s, 4 x SATA2 3.0 Gb/s
- Marvell SE9172: 2 x SATA3 6.0 Gb/s
- Marvell SE9230: 4 x SATA3 6.0 Gb/s

 

[solarisguy]

Standard recommendation would be to attach two disks to each of the three controllers, and on Intel using SATA3 ports.

However, you may find that one or both Marvells are marginally compatible with FreeNAS (FreeBSD). Test under heavy load before committing your data to them.

 

[Ericloewe]

I'd stick with Intel, as the Marvell may be slightly more potentially problematic. For what it's worth, the FreeNAS Mini uses the two Intel 6Gb/s ports and the rest from Marvell Controllers, so they aren't exactly bad.

 

[zfrogz]

So far both Marvell controllers seem to be working great (although haven't tested hot swapping a drive). Maybe I'll run some benchmarks and report back.

 

[mav@]

Marvell controllers may have two kinds of firmware: RAID and HBA. RAID one can be identified by reporting 8 ports without port multiplier support, while HBA report real ports and support multipliers. RAID firmware is known for its hot-plug problems.

 

[zfrogz]

Benchmarks are in. Intel takes the crown.

dd if=/dev/zero of=/mnt/vol0/home/media/tmp.dat bs=2048k count=50k
dd if=/mnt/vol0/home/media/tmp.dat of=/dev/null bs=2048k count=50k

(2x SATA3-Intel C2550)(2-Marvell SE9172)(2-Marvell SE9230)

Write

51200+0 records in

51200+0 records out

107374182400 bytes transferred in 491.332557 secs (218536673 bytes/sec)

Read

51200+0 records in

51200+0 records out

107374182400 bytes transferred in 308.266564 secs (348316019 bytes/sec)


(2xSATA3/4xSATA2-Intel C2550)

Write

51200+0 records in

51200+0 records out

107374182400 bytes transferred in 486.216661 secs (220836082 bytes/sec)

Read

51200+0 records in

51200+0 records out

107374182400 bytes transferred in 275.368329 secs (389929310 bytes/sec)


(2-Marvell SE9172)(4-Marvell SE9230)

Write

51200+0 records in

51200+0 records out

107374182400 bytes transferred in 525.181007 secs (204451762 bytes/sec)

Read

51200+0 records in

51200+0 records out

107374182400 bytes transferred in 290.809246 secs (369225476 bytes/sec)


(2xSATA3-Intel C2550)(4-Marvell SE9230)

Write

51200+0 records in

51200+0 records out

107374182400 bytes transferred in 612.323706 secs (175355260 bytes/sec)

Read

51200+0 records in

51200+0 records out

107374182400 bytes transferred in 328.914876 secs (326449760 bytes/sec)

 

[Ericloewe]

Benchmarks are in. Intel takes the crown.

dd if=/dev/zero of=/mnt/vol0/home/media/tmp.dat bs=2048k count=50k
dd if=/mnt/vol0/home/media/tmp.dat of=/dev/null bs=2048k count=50k

(2x SATA3-Intel C2550)(2-Marvell SE9172)(2-Marvell SE9230)

Write

51200+0 records in

51200+0 records out

107374182400 bytes transferred in 491.332557 secs (218536673 bytes/sec)

Read

51200+0 records in

51200+0 records out

107374182400 bytes transferred in 308.266564 secs (348316019 bytes/sec)


(2xSATA3/4xSATA2-Intel C2550)

Write

51200+0 records in

51200+0 records out

107374182400 bytes transferred in 486.216661 secs (220836082 bytes/sec)

Read

51200+0 records in

51200+0 records out

107374182400 bytes transferred in 275.368329 secs (389929310 bytes/sec)


(2-Marvell SE9172)(4-Marvell SE9230)

Write

51200+0 records in

51200+0 records out

107374182400 bytes transferred in 525.181007 secs (204451762 bytes/sec)

Read

51200+0 records in

51200+0 records out

107374182400 bytes transferred in 290.809246 secs (369225476 bytes/sec)


(2xSATA3-Intel C2550)(4-Marvell SE9230)

Write

51200+0 records in

51200+0 records out

107374182400 bytes transferred in 612.323706 secs (175355260 bytes/sec)

Read

51200+0 records in

51200+0 records out

107374182400 bytes transferred in 328.914876 secs (326449760 bytes/sec)

tl;dr Intel 3Gb/s is faster than Marvell 6Gb/s. Sounds like my Gigabyte X58A-UD3R.

 

[mav@]

From my tests, SATA controllers in Intel chipsets have total bandwidth limit of about 1GB/s. Theoretically that is not enough to saturate all 6 SATA ports same time, even for 3Gbps. But practically that is enough in most cases. And it is much more then many external SATA chips can provide. External chips often use only one PCIe x1 link, that is just a joke for 2-4 6Gbps ports.

The fact that Intel SATA alone overpowered three controllers working together once more proves that Intel's SATAs are the best on market.