SkogNisse
Cadet
- Joined
- Dec 15, 2016
- Messages
- 8
Hello There!
First to say, thanks for all the great info and knowledge that is daily shared here in this forum. I have been able, from being a normal windows guy, been able to setup my home FreeNAS 9.10-stable server, with plex/plexpy , sickrage, btsync etc in separate jails, and all is running and tugging along just nicely.
However, there is one big drawback , i haven't been able to kink out. We're talking about CIFS write performance over Ethernet.
From what it seems, the smb/cifs service, seems somewhat bottlenecked compared to the performance of FreeNAS itself.
I have a fairly powerfull server, for a home user's point of view, Consisting of :
Supermicro CSE847A-R1400 36bay 4U Chassis , Dual 1400W Ratet PSUs,
Dual Xeon X5550 (2x 4core/8thread) , and 48GB PC3-10600 ECC-REG Dimms
5x Dell Perc H310 Cross Flashed to LSI-9208 IT mode F20 firmware (Ebay from pulled servers, not china copies)
18x WD Red 3TB NAS for STORAGE pool 8x Hitachi Ultrastar 10K SAS 450GB drives for INCOMING pool.
(Two Mellanox 10GB ethernet cards,. was used for testing, they are removed now since the performance of cifs made it useless anyway)
And yes, it runs VmWare ESXi 6.5 on the bare metal. i have other windows services i need to run also, on a separate vm.
I have locked 32GB ram to the FreeNAS guest, and running all 5 perc cards as true passtrough, vt-d enabled.
Haven't had any sort of compatibility or stability issue.
My previous setup was Windows 2008 server, with Areca hardware raid controller.And that has always been stable, and blazing fast. the Areca ARC-1882-IX24 4GB cache easily gave me 650-750 MB/sec over SMB shares. (on 10GB ethernet)
My problem is this :
CIFS seems to be running a bottleneck at around 80 MB/sec write. (testing from one pc, my workstation).
When i download from the NAS , i get 112 MB/sec stable, so download seems a lot faster. (1GB Ethernet)
Upload to NAS is mostly 80 MB/s. (1GB Ethernet)
If i try to browse smb folders, delete files, and do other operations simultaneously, it feels ultra slow, like the commands are "in que" waiting for free time in between the ongoing file transfer.
so i can see delete file operations at 1file/s. if i pause the transfer, it speeds up instantly.
Is this really supposed to behave this poorly ?
To rule out if it was just the hardware, software or anythning, i did a fresh install of FreeNAS 10 nightly , on my old fileserver yesterday. Bare metal, straight to the box.
(i5 3330s, 16GB ddr3, asus z77-i desktop motherboard) but performance was similar there, around 85-90 MB/s to it, and 110 MB/s from it. behaved exactly the same.
FTP is a lot faster, i have no other way of testing speed for now. iSCSI i have no knowledge of , but it seems a bit more complicated then needed to be.
The sad part is i have already spent countless hours to setup everything running smoothly, and i really love the Resilver only existing data, snapshots, and most of all rehashing of "bit rot" data . since i do store 30+ TiB of data, that is rarely used or refreshed, the bit rot capability alone was one of the major reasons i switched to zfs from the 1300$ areca controller alone.
But now the performance is really making me bummed out.
I have tried to search and do some tweaks , but not any that have had any great changes.
I have also tried to enable the "autotune" feature, but that over time just made it go even slower. Periods my writes was down to 60-65 MB/s, and that's just rubbish.
a 250$ Qnap nas box can do that.
Hoping to hear back from people with similar experience, and how they managed to solve it. or if it's just that this setup is not what suits my application.
Basicly i would like it to max out my current 1gb Ethernet trough smb shares, with no problems, as i would expect of such powerful hardware.
I am certain that this is somewhat fixable with proper tuning and setup, i just don't know where to start.
best regards,
Tommy
First to say, thanks for all the great info and knowledge that is daily shared here in this forum. I have been able, from being a normal windows guy, been able to setup my home FreeNAS 9.10-stable server, with plex/plexpy , sickrage, btsync etc in separate jails, and all is running and tugging along just nicely.
However, there is one big drawback , i haven't been able to kink out. We're talking about CIFS write performance over Ethernet.
From what it seems, the smb/cifs service, seems somewhat bottlenecked compared to the performance of FreeNAS itself.
I have a fairly powerfull server, for a home user's point of view, Consisting of :
Supermicro CSE847A-R1400 36bay 4U Chassis , Dual 1400W Ratet PSUs,
Dual Xeon X5550 (2x 4core/8thread) , and 48GB PC3-10600 ECC-REG Dimms
5x Dell Perc H310 Cross Flashed to LSI-9208 IT mode F20 firmware (Ebay from pulled servers, not china copies)
18x WD Red 3TB NAS for STORAGE pool 8x Hitachi Ultrastar 10K SAS 450GB drives for INCOMING pool.
(Two Mellanox 10GB ethernet cards,. was used for testing, they are removed now since the performance of cifs made it useless anyway)
And yes, it runs VmWare ESXi 6.5 on the bare metal. i have other windows services i need to run also, on a separate vm.
I have locked 32GB ram to the FreeNAS guest, and running all 5 perc cards as true passtrough, vt-d enabled.
Haven't had any sort of compatibility or stability issue.
My previous setup was Windows 2008 server, with Areca hardware raid controller.And that has always been stable, and blazing fast. the Areca ARC-1882-IX24 4GB cache easily gave me 650-750 MB/sec over SMB shares. (on 10GB ethernet)
My problem is this :
CIFS seems to be running a bottleneck at around 80 MB/sec write. (testing from one pc, my workstation).
When i download from the NAS , i get 112 MB/sec stable, so download seems a lot faster. (1GB Ethernet)
Upload to NAS is mostly 80 MB/s. (1GB Ethernet)
If i try to browse smb folders, delete files, and do other operations simultaneously, it feels ultra slow, like the commands are "in que" waiting for free time in between the ongoing file transfer.
so i can see delete file operations at 1file/s. if i pause the transfer, it speeds up instantly.
Is this really supposed to behave this poorly ?
To rule out if it was just the hardware, software or anythning, i did a fresh install of FreeNAS 10 nightly , on my old fileserver yesterday. Bare metal, straight to the box.
(i5 3330s, 16GB ddr3, asus z77-i desktop motherboard) but performance was similar there, around 85-90 MB/s to it, and 110 MB/s from it. behaved exactly the same.
FTP is a lot faster, i have no other way of testing speed for now. iSCSI i have no knowledge of , but it seems a bit more complicated then needed to be.
The sad part is i have already spent countless hours to setup everything running smoothly, and i really love the Resilver only existing data, snapshots, and most of all rehashing of "bit rot" data . since i do store 30+ TiB of data, that is rarely used or refreshed, the bit rot capability alone was one of the major reasons i switched to zfs from the 1300$ areca controller alone.
But now the performance is really making me bummed out.
I have tried to search and do some tweaks , but not any that have had any great changes.
I have also tried to enable the "autotune" feature, but that over time just made it go even slower. Periods my writes was down to 60-65 MB/s, and that's just rubbish.
a 250$ Qnap nas box can do that.
Hoping to hear back from people with similar experience, and how they managed to solve it. or if it's just that this setup is not what suits my application.
Basicly i would like it to max out my current 1gb Ethernet trough smb shares, with no problems, as i would expect of such powerful hardware.
I am certain that this is somewhat fixable with proper tuning and setup, i just don't know where to start.
best regards,
Tommy
Last edited: