friolator
Explorer
- Joined
- Jun 9, 2016
- Messages
- 80
We're rebuilding our FreeNAS system from the ground up in the next couple of weeks. Here's the current hardware configuration I'm thinking of:
* ASRock X99 Extreme 4 motherboard
* Xeon E5 2603 v3 6-core
* 32GB ECC RAM
* 64GB SSD System Drive
* Chelsio T580-SO-CR 40GbE NIC
* LSI 9305 24i
* LSI 9201-16e
* GTX 710 (PCIe x1) GPU
The system will be in an enclosure with 20 hot-swap bays, which we already have. These will be connected to the LSI 9305. The second LSI card is for our existing FreeNAS system, which will be stripped and turned into an external enclosure with hot-swap bays. This will give us a total of 36 x 2TB drives. Future expansion would probably happen by replacing the drives with larger ones, rather than adding another enclosure.
I chose the ASRock motherboard because it's a solid performer. We use it in several of our high end PCs in-house for color correction, restoration work, etc. It's been a very reliable board and you can find them for under $200.
The big question at this point is the 40GbE setup. Looking through old posts here, this seems to be the one to get, but if anyone has other suggestions, I'm open. We have an IBM 16-port 40GbE switch. There will be at least 4 machines connected to this network: Two of them will require pretty good performance. One probably won't need as much speed. So some of the lesser machines might be connected at 10G instead of 40. Though I doubt we'd need it, the Chelsio card is dual-port, so we could take advantage of that as well, if we need more bandwidth. I suspect drive speed is going to be the bottleneck here, though.
The files we're working with are primarily 4k and lower resolution image sequences. Hundreds of thousands of files, one for each frame of film. I believe the 36 drives will give us the performance we need to be writing and reading files from two different machines simultaneously, but that's not a common scenario. More often than not, only a single machine will be accessing the files at any time.
We'll do everything through iSCSI.
Any thoughts?
* ASRock X99 Extreme 4 motherboard
* Xeon E5 2603 v3 6-core
* 32GB ECC RAM
* 64GB SSD System Drive
* Chelsio T580-SO-CR 40GbE NIC
* LSI 9305 24i
* LSI 9201-16e
* GTX 710 (PCIe x1) GPU
The system will be in an enclosure with 20 hot-swap bays, which we already have. These will be connected to the LSI 9305. The second LSI card is for our existing FreeNAS system, which will be stripped and turned into an external enclosure with hot-swap bays. This will give us a total of 36 x 2TB drives. Future expansion would probably happen by replacing the drives with larger ones, rather than adding another enclosure.
I chose the ASRock motherboard because it's a solid performer. We use it in several of our high end PCs in-house for color correction, restoration work, etc. It's been a very reliable board and you can find them for under $200.
The big question at this point is the 40GbE setup. Looking through old posts here, this seems to be the one to get, but if anyone has other suggestions, I'm open. We have an IBM 16-port 40GbE switch. There will be at least 4 machines connected to this network: Two of them will require pretty good performance. One probably won't need as much speed. So some of the lesser machines might be connected at 10G instead of 40. Though I doubt we'd need it, the Chelsio card is dual-port, so we could take advantage of that as well, if we need more bandwidth. I suspect drive speed is going to be the bottleneck here, though.
The files we're working with are primarily 4k and lower resolution image sequences. Hundreds of thousands of files, one for each frame of film. I believe the 36 drives will give us the performance we need to be writing and reading files from two different machines simultaneously, but that's not a common scenario. More often than not, only a single machine will be accessing the files at any time.
We'll do everything through iSCSI.
Any thoughts?