A few questions regarding my first FreeNAS build and ZFS and AFP

[RamGuy]

I've finally moved my 7x Western Digital RE4 2TB array from my desktop computer over to a dedicated file server running FreeNAS 8.0.4, as this is my first attempt at FreeBSD and FreeNAS in the first place and I have only had experience running and configuring Windows and Mac's in the past I was curious about how this was all going to turn out.

But after one DOA motherboard and a pain in the arse rack-chase to build in it's all finally up and running. My hardware is as follows:

- Asus P8B-E/4L Intel C204 motherboard
- Intel Xeon E3-1270 CPU
- 16GB (4x 4GB) 1333MHz ECC RAM
- Gigabyte GeForce GT 520 (as I was stupid enough to get the E3-1270 instead of the E3-1275)
- LSI MegaRAID 9260-8i RAID controller
- FireWire 800 controller (but I guess FreeNAS does not support pushing backup towards external hard-drivers? nor grabbing data from external hard drives)
- 1x Intel 520-series SSD 60GB for FreeNAS 8.0.4 installation
- 7x Western Digital RE4 2TB drives for data storage


Installation went smoothly, network interface have been configured to run a 2x gigabit LACP link to my Cisco SRW2024 managed switch with fixed IP-address and the web-ui of FreeNAS is fairly easy to use even though I don't understand half of it's features.

Unfortunately the LSI MegaRAID 9260-8i does not seem to feature the possibility to just passthrough my hard drives without any RAID-configuration, and there doesn't seem to be any JBOD option so I have to configure my drives in a hardware RAID on the controller before I can get them up in FreeNAS resulting in no ZFS RAID possibilities. Honestly I have no idea how the whole ZFS filesystem work, but after what I have gathered the ideal solution would to be to have every disk running for themselves, formatting them with ZFS and then setting them up in either a RAIDZ or RAIDZ2 configuration. But that won't be a possibility because of the 9260-8i RAID controller.

So currently my seven drivers are configured in a RAID6 array on the controller, and this array has been formatted as ZFS in FreeNAS. Should I be doing something else here, or is this simply the best way for me to do it considering the RAID controller?



Besides this the setup has been fairly easy and straightforward, the only thing I can't seem to get my head around is getting the Apple File Sharing up and running. I have configured my share user, I have configured the AFP service, I have configured so my network share should be shared over AFP to my Mac's and they are all able to locate the FreeNAS share but I'm not able to connect? I keep retrieving connection failed every time I try. The moment I try connecting using CIFS / Samba it works though, but I would like to use AFP for my Mac's as I tend to get better performance that way.

Am I missing something obvious here? As the CIFS / Samba share seems to be working perfectly there shouldn't be anything other than my AFP configuration messing up things.

 

[ProtoSD]

Unfortunately the LSI MegaRAID 9260-8i does not seem to feature the possibility to just passthrough my hard drives without any RAID-configuration, and there doesn't seem to be any JBOD option so I have to configure my drives in a hardware RAID on the controller before I can get them up in FreeNAS resulting in no ZFS RAID possibilities. Honestly I have no idea how the whole ZFS filesystem work, but after what I have gathered the ideal solution would to be to have every disk running for themselves, formatting them with ZFS and then setting them up in either a RAIDZ or RAIDZ2 configuration. But that won't be a possibility because of the 9260-8i RAID controller.

So currently my seven drivers are configured in a RAID6 array on the controller, and this array has been formatted as ZFS in FreeNAS. Should I be doing something else here, or is this simply the best way for me to do it considering the RAID controller?

I'm not an expert on this because I don't use an LSI RAID controller, but having digested a large number of posts here it sounds like you need to flash your controller to "IT" mode which will let you see your individual disks, which is what you want for using ZFS.

I think there's a link about it in my FAQ (signature below), but let me double check and if not I'll update this post with a link.

If you haven't loaded your system up with files, I'd wait until you do this, otherwise you're not getting the full benefit of ZFS.

Congrats on your setup and welcome to the forums!


EDIT: I didn't find it in the FAQ and there are several threads that discuss the topic, but this link should help you find what you need.

http://www.servethehome.com/ibm-serveraid-br10i-lsi-sas3082e-r-pciexpress-sas-raid-controller/

http://forums.servethehome.com/showthread.php?19-LSI-RAID-Controller-HBA-Equivalency-Mapping


Can anyone else here provide a better link for flashing his controller to IT mode?

 

[ProtoSD]

RamGuy,

Here's some more info I found:

MegaRAID SAS 9260-4i
MegaRAID SAS 9260-8i
MegaRAID SAS 9260DE-8i
MegaRAID SAS 9261-8i
MegaRAID SAS 9280-4i4e
MegaRAID SAS 9280-8e
MegaRAID SAS 9280DE-8e

Firmware: 12.11.0-0016 (APP-2.110.03-0967)
Date: 27-SEP-10
Filename: 12.11.0-0016_SAS_2108_FW_Image_APP-2.110.03-0967_u1.zip

http://www.lsi.com/lookup/License.a...RAID SAS 9280DE-8e&subType=Firmware&locale=EN or non-direct http://www.lsi.com/storage_home/pro...b_s_value_line/sas9260-4i/index.html#Firmware

Changes: http://www.lsi.com/DistributionSyst...16_SAS_2108_FW_Image_APP-2.110.03-0967_u1.txt

 

[RamGuy]

Doesn't seem like any of your links work? Is it supposed to be a link for a IT-firmware for the 9260-8i? I was under the impression that such a thing did not exsist.

 

[ProtoSD]

RamGuy,

I just clicked on all of the links posted above and each of them works.

Yes they are about the IT firmware for the 9260-8i. Like I said, I'm not an expert, but the links I posted and posts I've seen here make me think that the IT firmware does exist.

 

[louisk]

If you don't want to flash the firmware, you can always create X RAID0 arrays in the controller, one for each drive. I did this with a PERC because FreeBSD wouldn't see it as pass-through. It works fine, and drive replacement is normal. new drive -> new RAID0 in raid controller -> replace in ZFS. 1 extra step. not too bad.

I would say you don't need firewire. You can setup AFP to present time machine disks and backup your macs over the network. I don't know about windows, but I would expect there is some kind of backup solution for them as well.

 

[RichR]

Besides this the setup has been fairly easy and straightforward, the only thing I can't seem to get my head around is getting the Apple File Sharing up and running. I have configured my share user, I have configured the AFP service, I have configured so my network share should be shared over AFP to my Mac's and they are all able to locate the FreeNAS share but I'm not able to connect? I keep retrieving connection failed every time I try. The moment I try connecting using CIFS / Samba it works though, but I would like to use AFP for my Mac's as I tend to get better performance that way.

Am I missing something obvious here? As the CIFS / Samba share seems to be working perfectly there shouldn't be anything other than my AFP configuration messing up things.

Hi and welcome.

Make sure you put a user in the "Allow List" of the share in FreeNAS, along with the full path. Make sure that user has the proper permissions to read/write in the Volume, and it's that user that is accessing everything from your Mac. If Finder on your Mac is giving you grief, try Go>Connect to server> then put afp://your.ip.address

Good luck

Rich

 

[RamGuy]

RamGuy,

I just clicked on all of the links posted above and each of them works.

Yes they are about the IT firmware for the 9260-8i. Like I said, I'm not an expert, but the links I posted and posts I've seen here make me think that the IT firmware does exist.

That's awkward as the first link just redirects me to LSI frontpage on both my Mac's and my Windows computers and the second one just takes me to a page with links to different products pages for different MegaRAID cards (9260-8i being one of them) and there I can not locate the firmware you are referring to, just the last six releases from LSI (non being IT for what I can see) and the txt file one requires login.


I could always go the single drive in RAID0 route as mentioned above, but after what I've gathered that's not ideal either? So a IT firmware would be the best choice I guess.

 

[RamGuy]

Then I've configured all drives in single-drive RAID0 arrays, wasn't entirely sure which settings to configure them with so I just went with the default 64K strip size, write ahead with BBU and direct I/O for each array.

In FreeNAS I select all arrays / drives and created a 10TB RAIDZ ZFS-array and all seems to be working, but the AFP Share still wont authenticate me when trying to access it through Finder on Mac OS X Lion.


I might be considering adding an SSD for caching later on, as I have one slot left on my 9260-8i and still have four SATA ports on the on-board C204 controller as well. Would I be able to add SSD for caching later on, or would I have to initialise the whole setup once again when adding a SSD for caching?

 

[RamGuy]

There seems to be an issue of some sort, I was moving all my data over to the file server from my Windows 7 system (samba / cifs) and after about 1 out 3tb of data was moved I lost connection with the server entirely. I ran and checked the server and it had rebooted for some reason, it didn't seem hot so I didn't really understand why it would restart in the middle of a file transfer.

Not only that but it didn't seem to get back up after the reboot either, I didn't have a monitor hooked up to it at that time so I couldn't really check what was going on but even after half an hour after it had rebooted it was still not visible on my network and the webui was not accessible through the fixed IP-address. So I just shut it down for the night.

The next day I hooked up a monitor and fired it back up again, then it booted normally and everything seemed to just fine. After being up and running for several hours I started my file transfer once again and after about another 1tb of data was copied it dropped out once again, so I ran to check out if I could figure anything out but as this happens instantly out of nowhere I wasn't able to see what might have caused the server to reboot but I could see that it was doing some ZFS file checking of some sort upon rebooting which just seems to get stuck and keeping on forever forcing me to shut the computer down, wait for a while (hours) and then boot it back up and again whereas everything seems to run perfectly again. If I reboot it without waiting for a while it will just keep on with all those lines you can see on the second picture.

I took some photos with my iPhone of the things going on after the crash / reboot for you to see.




Do you have any idea why it keeps rebooting out of the blue? Things seems to be working perfectly up until it just shuts and reboots and start with all this checking which never ends for some reason.

 

[louisk]

Looks like the controller is having timeouts.

Have you tried updating the firmware?

 

[RamGuy]

It's on the second latest firmware from October 2011, there has been one release after this so I highly doubt that would change much? Might it be something awkward going on as I've configured each drive in single RAID0 configuration with the backup battery unit enabled? I wasn't sure whether that was going to be such a good idea or not.

 

[louisk]

I'm running that kind of configuration on my FreeNAS with a PERC. Been very reliable.

 

[RamGuy]

I'll try the firmware update then, it actually mentions something about better support for the battery backup unit which I' am using so why the heck not. I won't be going anywhere while it's looping anyway.

 

[louisk]

I would try the firmware update. I would not expect any difference with the behavior of the BBU. Mine splatters notes on the console about charging. If the firmware update doesn't work, you could also try flashing the alternate BIOS. I don't know that it would help, but at this point, its worth a try as well.

 

[RamGuy]

Firmware update didn't change anything. Which alternate firmware? As I said earlier the link posted don't take me anywhere other than to the front page of LSI.

 

[RamGuy]

It seems to be a bug with LSI MegaRAID firmware and FreeBSD actually, I will contact our LSI partner at work tomorrow and probably get a RMA on it.

If I wore to replace my LSI MegaRAID 9260-8i with something else, what would you recommend me to get? Perhaps something that actually manages to passthrough all the drivers so I don't need to configure them in RAID0 on the RAID controller and all that nonsense so FreeNAS, ZFS and my Intel Xeon E3-1270 can do all the handling and all the work like it's intended in the first place.

 

[louisk]

The alternate firmware that will allow the card to be used as a JBOD. I don't have a link, I don't know anything about these cards.

 

[RamGuy]

Then I've got a IBM M1015 on the way which I'm going to flash with LSI MegaRAID 9211-8i IT firmware which should so the trick.

 

[Vegard Kolbeinsen]

Did you get this to work? I found a good card on eBay and want to know if this works without to much hassle?