Early in May our old storage system running Nexenta 3 Community Edition began to go haywire, luckily I had a new storage system in my budget this year, so I started researching what I wanted for the new system.
I recently refurbished another server from openfiler to Freenas, but this is an easy backupserver with no load. So I decided to try Freenas on this system.
I opted for a Dell R730 server with E5-2620 v3, 64 GB RAM, and PERC H330 with no disks from Dell, but with two sd cards in mirror for freenas os. I got 7 Samsung 845DC Pros for a quite good price, so I went for those. I have set them up as tre sets of mirrors within one volume. This system will be the nfs storage for our vmware servers. At the moment it is around 20 virtual servers. On the old server I did the error to set up 6 drives as a zraid1, not good for performance for the usage we got.
The migration of the virtual servers started on monday, and it went really smooth, until the last one. The performance to copy the server files was abysmal (around 40 Mbps). I really can't understand why it was that slow, but the old server was struggeling quite a lot, so I blame that!
Every server is now up and running smoothly and the systems are quite a lot snappier than before. I repurposed the old nexenta server today with Freenas for a replication server. This server is a Dell R515 with 2x Opteron 4130 with 32 GB RAM, and 6 Seagate Cheetah 15k7 600 GB drives. I also got a 400 GB 845DC Pro that I set up as a ZIL for that server.
One thing I hope that will be improved is the setup of LACP, it is not possible to get it running without rebooting the system after I set it up in the console. And on the R730 it was a bit annoying that the network adaptors did not correspond to the correct numbers in FreeNAS. For example the 1 adapter on the server is bge2 and not bge0 as it should.
This server is now also set up as tre sets of mirrors. Immidiately Freenas was complaining about two of the drives that was on the way of failing because of the SMART module. This is something Nexenta should have! I have ran scrubs weekly on the old system, and had replaced 2 drives in that system the last two years. I had a spare and replaced one of the drives that freenas was complaining about. It is so much easier to change and replace the drives in freenas than in nexenta. Really happy about that!
One thing I would like to have more of in Freenas that Nexenta got is more drive statistics, awaiting read, write precentage busy, how many writes and reads each drives does and so on. Most of this is available trough CLI, but I would like some of it in GUI, this was one of the main places I looked in the nexenta setup when looking for problems.
It could also be nice to have more zfs options so that I could tune the recordsize and other nifty options in nexenta.
Other than that I do like that FreeNAS is easier to update, more updates, easier to get running, better information about the disks and smart errors, easier to replace drives. But the most important one for us is that the replication feature is way better than the tier feature in nexenta to have a "backup" of the running servers! I haven't testet that feature out a lot, but as far as it have worked today while setting it up and replicated every snapshot made on the new server to the repurposed old server in a breeze it looks fabulous for our needs. So I have set up the snapshot task to take a snapshot every hour during working hours tonight, it will be interesting to see how it is working tomorrow.
This setup is quite a overkill for our needs at the moment, but the old setup lacked a lot in performance, so I wanted this one to be a lot faster, and the price was actually really good when I sourced the SSDs myself instead of letting Dell put them in for me.
I will be able to run some performance tests if someone is interested, let me know what commands I should run to test the system.
I recently refurbished another server from openfiler to Freenas, but this is an easy backupserver with no load. So I decided to try Freenas on this system.
I opted for a Dell R730 server with E5-2620 v3, 64 GB RAM, and PERC H330 with no disks from Dell, but with two sd cards in mirror for freenas os. I got 7 Samsung 845DC Pros for a quite good price, so I went for those. I have set them up as tre sets of mirrors within one volume. This system will be the nfs storage for our vmware servers. At the moment it is around 20 virtual servers. On the old server I did the error to set up 6 drives as a zraid1, not good for performance for the usage we got.
The migration of the virtual servers started on monday, and it went really smooth, until the last one. The performance to copy the server files was abysmal (around 40 Mbps). I really can't understand why it was that slow, but the old server was struggeling quite a lot, so I blame that!
Every server is now up and running smoothly and the systems are quite a lot snappier than before. I repurposed the old nexenta server today with Freenas for a replication server. This server is a Dell R515 with 2x Opteron 4130 with 32 GB RAM, and 6 Seagate Cheetah 15k7 600 GB drives. I also got a 400 GB 845DC Pro that I set up as a ZIL for that server.
One thing I hope that will be improved is the setup of LACP, it is not possible to get it running without rebooting the system after I set it up in the console. And on the R730 it was a bit annoying that the network adaptors did not correspond to the correct numbers in FreeNAS. For example the 1 adapter on the server is bge2 and not bge0 as it should.
This server is now also set up as tre sets of mirrors. Immidiately Freenas was complaining about two of the drives that was on the way of failing because of the SMART module. This is something Nexenta should have! I have ran scrubs weekly on the old system, and had replaced 2 drives in that system the last two years. I had a spare and replaced one of the drives that freenas was complaining about. It is so much easier to change and replace the drives in freenas than in nexenta. Really happy about that!
One thing I would like to have more of in Freenas that Nexenta got is more drive statistics, awaiting read, write precentage busy, how many writes and reads each drives does and so on. Most of this is available trough CLI, but I would like some of it in GUI, this was one of the main places I looked in the nexenta setup when looking for problems.
It could also be nice to have more zfs options so that I could tune the recordsize and other nifty options in nexenta.
Other than that I do like that FreeNAS is easier to update, more updates, easier to get running, better information about the disks and smart errors, easier to replace drives. But the most important one for us is that the replication feature is way better than the tier feature in nexenta to have a "backup" of the running servers! I haven't testet that feature out a lot, but as far as it have worked today while setting it up and replicated every snapshot made on the new server to the repurposed old server in a breeze it looks fabulous for our needs. So I have set up the snapshot task to take a snapshot every hour during working hours tonight, it will be interesting to see how it is working tomorrow.
This setup is quite a overkill for our needs at the moment, but the old setup lacked a lot in performance, so I wanted this one to be a lot faster, and the price was actually really good when I sourced the SSDs myself instead of letting Dell put them in for me.
I will be able to run some performance tests if someone is interested, let me know what commands I should run to test the system.
