Noctris
Contributor
- Joined
- Jul 3, 2013
- Messages
- 163
HI,
Let me introduce myself first since that seems to be the polite thing to do ;-) I'm Noctris. Long time reader, first time poster. I've got several medium to high-end freenas boxes running for work, had one at home which was getting a bit light for my needs ( 8GB DDR2, intel duo something) so i started on a new build.
Hardware
I tryed to reuse some of the stuff i still had and purchased some extra's to meet spec. so currently the new build ( nebula) is running on:
Asus P8Z77-M ( https://www.asus.com/Motherboards/P8Z77M/)
Intel(R) Core(TM) i3-3220 CPU @ 3.30GHz
16 GB ( 2 X 8gb corsair )16043MB ( noticed that this goes up and down in the freenas interface. A previous time i booted, it gave 15538 Mb ?)
M1015 in IT mode
Intel RES2SV240 6Gbps SATA SAS RAID Expander 24port LSI
Intel PCIX Dual Gb NIC
6 X 3Tb red in RaidZ2
2 X 3TB Red in Mirror
8 X 1TB seagate in raidZ2 ( these are old drives that served as "spare" disks and where never plugged in so i consider them new)
4 X Seagate 2TB in Mirrored stripe ( same as above, 3 to 4 years old but straight out of the sealed box)
4 X 2TB wd greens -> for playing and testing
24 Bay Logic Case (http://www.logic-case.com/products/...s-minisas-550m-deep-with-fan-wall-sc-4324s-f/)
8 Gb usb for boot ( X 2 in Mirror) Don't know the brand, these are thumb drives we had printed for our own company. These where tested and considered good ( and during the course of my "adventure the last couple of days, i switched these aswell , just to be sure)
Installed freenas 9.2.1 image, then upgraded to 9.3 Stable ( last stable train update)
the 8 X 1TB raidZ2 was made on my previous system and contains a bunch of data ( about 63% filled)
I think that about sums it up.
The issue:
I started setting up the freenas , made my pools , datasets and zvol's ( esx iscsi) , installed plugin jails, created iscsi zvol and all was well. In fact, performance exceeded my expectations. however, whenever i put some real load on it, it goes down. No kernel panic, no error messages on drives or anything, just.. down. for example: i'm rsyncing the data from the 8 X 1TB RaidZ2 to the new 6 X 3TB RaidZ2 internally. Speeds are in the 300 MB/Sec range which is more then acceptable to me. It does this between 5 and 40 minutes and then it reboots ( sometimes even sort of a shutdown as the mobo does not seem to start booting again) This also happens when i copy data from our office servers to here ( which is through a wireless link that goes across the street ) so when only using 7 to 8 MB/sec
I've done memtests, changed boot devices, checked logs, looked for core dumps, pushed my logs to external syslog but helas: No joy. There does not seem to be a real dump or panic. Currently i'm running a debug version of the kernel. I'm probably missing something but i could really use some help where to look for even the smallest indication of what is happening ( or maybe people that are more knowledgeable on hardware recognize this kind of stuff ?)
At this point, i'm ready to just buy a supermicro mobo with xeon and ecc ( as i do for my pro builds) , but on the other hand, the gear i'm using now might not be top notch server grade, i've ran freenas on ( A LOT) less then this and it ran fine. So before spending an additional 500 to 600 euro, I would just like to make sure i dotted the i's and crossed the T's.
Any ideas ?
Let me introduce myself first since that seems to be the polite thing to do ;-) I'm Noctris. Long time reader, first time poster. I've got several medium to high-end freenas boxes running for work, had one at home which was getting a bit light for my needs ( 8GB DDR2, intel duo something) so i started on a new build.
Hardware
I tryed to reuse some of the stuff i still had and purchased some extra's to meet spec. so currently the new build ( nebula) is running on:
Asus P8Z77-M ( https://www.asus.com/Motherboards/P8Z77M/)
Intel(R) Core(TM) i3-3220 CPU @ 3.30GHz
16 GB ( 2 X 8gb corsair )16043MB ( noticed that this goes up and down in the freenas interface. A previous time i booted, it gave 15538 Mb ?)
M1015 in IT mode
Intel RES2SV240 6Gbps SATA SAS RAID Expander 24port LSI
Intel PCIX Dual Gb NIC
6 X 3Tb red in RaidZ2
2 X 3TB Red in Mirror
8 X 1TB seagate in raidZ2 ( these are old drives that served as "spare" disks and where never plugged in so i consider them new)
4 X Seagate 2TB in Mirrored stripe ( same as above, 3 to 4 years old but straight out of the sealed box)
4 X 2TB wd greens -> for playing and testing
24 Bay Logic Case (http://www.logic-case.com/products/...s-minisas-550m-deep-with-fan-wall-sc-4324s-f/)
8 Gb usb for boot ( X 2 in Mirror) Don't know the brand, these are thumb drives we had printed for our own company. These where tested and considered good ( and during the course of my "adventure the last couple of days, i switched these aswell , just to be sure)
Installed freenas 9.2.1 image, then upgraded to 9.3 Stable ( last stable train update)
the 8 X 1TB raidZ2 was made on my previous system and contains a bunch of data ( about 63% filled)
I think that about sums it up.
The issue:
I started setting up the freenas , made my pools , datasets and zvol's ( esx iscsi) , installed plugin jails, created iscsi zvol and all was well. In fact, performance exceeded my expectations. however, whenever i put some real load on it, it goes down. No kernel panic, no error messages on drives or anything, just.. down. for example: i'm rsyncing the data from the 8 X 1TB RaidZ2 to the new 6 X 3TB RaidZ2 internally. Speeds are in the 300 MB/Sec range which is more then acceptable to me. It does this between 5 and 40 minutes and then it reboots ( sometimes even sort of a shutdown as the mobo does not seem to start booting again) This also happens when i copy data from our office servers to here ( which is through a wireless link that goes across the street ) so when only using 7 to 8 MB/sec
I've done memtests, changed boot devices, checked logs, looked for core dumps, pushed my logs to external syslog but helas: No joy. There does not seem to be a real dump or panic. Currently i'm running a debug version of the kernel. I'm probably missing something but i could really use some help where to look for even the smallest indication of what is happening ( or maybe people that are more knowledgeable on hardware recognize this kind of stuff ?)
At this point, i'm ready to just buy a supermicro mobo with xeon and ecc ( as i do for my pro builds) , but on the other hand, the gear i'm using now might not be top notch server grade, i've ran freenas on ( A LOT) less then this and it ran fine. So before spending an additional 500 to 600 euro, I would just like to make sure i dotted the i's and crossed the T's.
Any ideas ?