taylorjonl
Dabbler
- Joined
- Dec 14, 2013
- Messages
- 11
I posted in off-topic because I will not be using FreeNAS but this forum seems to have the most knowledge of ZFS for hobbyists. I currently have a box that has these parts:
So I went on eBay and bought IBM M1015 x3 and I will reflash them with the LSI firmware to make them pure HBAs.
So here is how I am thinking of configuring this box:
X X X X
X X X X
X X X X
X X X X
A + O O
* * * *
X = storage
A = l2arc
+ = zil
O = spare
* = fast
The storage drives will be my Seagate 2tb drives setup in two RAIDZ2 arrays that will be striped, then the fast drives will be some 1tb WD Red drives setup in RAIDZ. The l2arc will probably be my OCZ Vertex 2 at least at first. The zil I was looking to find a cheap and small SLC SSD, around 20GB. I am open for suggestion but would like to use as much hardware that I currently have as possible.
The motherboard is dual processor(only one is populated right now) with triple channel memory and has 18 DIMM slots. It supports up to 144GB, if I keep going with 12GB triple channel kits I max out at 72GB. These 12GB kits are around $100 each, if I go with the 24GB kits they are around $300. Also each processor has 9 DIMMs, so to use all the DIMMs I have to get a new $400 CPU.
So here are the two options:
144GB = $300x6 + $400 = $2200
72GB = $100x5 + $400 = $900
So ZFS loves RAM, I have read the rule is 1GB per 1TB of raw storage and 1GB per 50GB of l2arc. Firstly does this sound accurate?
If I do the math, I have 32tb in Seagate drives, then I have 4tb in WD Red drives and I have 120GB of l2arc. So if I start to calculate my RAM requirements from the above I need around 38GB of RAM just for ZFS? If that math is correct I would probably just round it up to 40GB. Now I want this box to be my NAS along with hosting some VMs(using KVM).
I plan on using SmartOS which is based on Illumos which is based on OpenSolaris. This has a concept similar to FreeBSD jails, so I *think* I can build OpenIndiana in a "zone" since it is also based on Illumos. I would also host some other zones for maybe a web server, ftp, sshd, etc. As much as possible I would use zones over KVM, but I do need(or at least want) to host a Window VM for my HTPC(I get my cable through a Ceton tuner).
My main question is, I know that ZFS will consume as much RAM as it is allowed and I thought I had read that it will give it back if the system needs the RAM, so if I were to have the 72GB of RAM with the above configuration, then I start reserving RAM for VMs and zones, what happens as I take more RAM from ZFS?
Does ZFS just start losing performance down to a floor or will the universe implode? For example, if I decide to reserve 40GB for VMs and zones, this leaves 32GB for ZFS, or even reserve 48GB for VMs and zones, leaving 24GB for ZFS, will it still work it just won't work as well as if it had more RAM?
I think 32GB for VMs and zones will probably be enough for me but I really want to understand how the system will behave if I need to push the system harder.
- Tyan S7012 Motherboard
- Intel Xeon E5620 Processor 2.4
- Kingston ValueRAM 12GB DDR3
- Areca ARC-1680IX-24 PCIe x8 SAS RAID Card
- Seagate Barracuda LP 2 TB 5900RPM SATA 3 GB/s x20
- OCZ Vertex 2 120 GB SSD
So I went on eBay and bought IBM M1015 x3 and I will reflash them with the LSI firmware to make them pure HBAs.
So here is how I am thinking of configuring this box:
X X X X
X X X X
X X X X
X X X X
A + O O
* * * *
X = storage
A = l2arc
+ = zil
O = spare
* = fast
The storage drives will be my Seagate 2tb drives setup in two RAIDZ2 arrays that will be striped, then the fast drives will be some 1tb WD Red drives setup in RAIDZ. The l2arc will probably be my OCZ Vertex 2 at least at first. The zil I was looking to find a cheap and small SLC SSD, around 20GB. I am open for suggestion but would like to use as much hardware that I currently have as possible.
The motherboard is dual processor(only one is populated right now) with triple channel memory and has 18 DIMM slots. It supports up to 144GB, if I keep going with 12GB triple channel kits I max out at 72GB. These 12GB kits are around $100 each, if I go with the 24GB kits they are around $300. Also each processor has 9 DIMMs, so to use all the DIMMs I have to get a new $400 CPU.
So here are the two options:
144GB = $300x6 + $400 = $2200
72GB = $100x5 + $400 = $900
So ZFS loves RAM, I have read the rule is 1GB per 1TB of raw storage and 1GB per 50GB of l2arc. Firstly does this sound accurate?
If I do the math, I have 32tb in Seagate drives, then I have 4tb in WD Red drives and I have 120GB of l2arc. So if I start to calculate my RAM requirements from the above I need around 38GB of RAM just for ZFS? If that math is correct I would probably just round it up to 40GB. Now I want this box to be my NAS along with hosting some VMs(using KVM).
I plan on using SmartOS which is based on Illumos which is based on OpenSolaris. This has a concept similar to FreeBSD jails, so I *think* I can build OpenIndiana in a "zone" since it is also based on Illumos. I would also host some other zones for maybe a web server, ftp, sshd, etc. As much as possible I would use zones over KVM, but I do need(or at least want) to host a Window VM for my HTPC(I get my cable through a Ceton tuner).
My main question is, I know that ZFS will consume as much RAM as it is allowed and I thought I had read that it will give it back if the system needs the RAM, so if I were to have the 72GB of RAM with the above configuration, then I start reserving RAM for VMs and zones, what happens as I take more RAM from ZFS?
Does ZFS just start losing performance down to a floor or will the universe implode? For example, if I decide to reserve 40GB for VMs and zones, this leaves 32GB for ZFS, or even reserve 48GB for VMs and zones, leaving 24GB for ZFS, will it still work it just won't work as well as if it had more RAM?
I think 32GB for VMs and zones will probably be enough for me but I really want to understand how the system will behave if I need to push the system harder.