Hello! Studying FreeNAS for possible installation here

[twelve12pm]

Hi All!

I'm currently reading through FreeNAS documentation and I've joined the forums to (you guessed it!) ask lots of questions!

We are a small organization hoping to grow.

I find FreeNAS intriguing because it offers the possibility of starting small with virtually endless room to grow in the future, as our needs and hopefully our budget will allow.

Currently I'm studying how to set up a central file server for our organization. In addition to simply storing files, we need it to host our Subversion server which implies storing our repositories. FreeNAS may fit the bill since it is based on FreeBSD and therefore any software can be installed. I think the larger FreeNAS Mini might fit the bill, and that's the possibility I'm studying right now. One little caveat: I've been away from FreeBSD and system administration since FreeBSD 4.0, so I'm a *little* rusty.

I'm sure the forums will hear from me quite a bit as questions arise...

Have a great day,
12

 

[m0nkey_]

Welcome to the forums! :)

To get a FreeNAS server up and running, you don't really need any FreeBSD experience, however it does help.

Follow the documentation at https://doc.freenas.org. Also check out the resources section on how-to and more.

 

[Redcoat]

I think the larger FreeNAS Mini might fit the bill, and that's the possibility I'm studying right now.

Welcome to the forums.

If you search here for Asrock C2750D4l (the mobo of the two Mini versions) you'll find that many members here (including me) have experienced issues with that board, mainly related to temperature and/or the chipset bugs related to the watchdog and the Atom processor.
The chipset issues could only be dealt with by claims through the supply chain. From my own experience and reports here I can say that iXsystems stepped up wonderfully on board failures with an extended warranty program, and there are many reports of Asrock doing the same. You'll find a thread or two detailing these experiences when you search. Here's one https://forums.freenas.org/index.php?threads/how-long-your-asrock-c2550d4i-or-c2750d4i-lasted.45445/ Hopefully the chipset issues are now a matter of the past for current supplies.

Now, the Mini's compact chassis doesn't do much to enhance the cooling possibilities for the installed HDD's or the passively-cooled processor. The same mobo in a different case offers more integrity-enhancing cooling possibilities for both the board and any HDD's. So, if I were doing this all again, and certainly if I were doing it for a small business where the server was a key piece of the operational structure, I would not buy the Mini despite the stellar support that iXsystems has provided me, because I would want a piece of hardware with greater natural integrity and the cramped chassis is really limited in cooling capability, even with modifications, in my personal experience with the Mini.

The Mini XL is of course in a larger chassis and may have better cooling capability, but there's at least one forum user who did "fan work" on his XL to enhance it.

 

[twelve12pm]

Welcome to the forums! :)

To get a FreeNAS server up and running, you don't really need any FreeBSD experience, however it does help.

Follow the documentation at https://doc.freenas.org. Also check out the resources section on how-to and more.

Thank you for your reply. I am sure that getting the server up and running as a NAS will be relatively simple. The FreeBSD experience will be needed when I set up a jail (or more than one?) to get our other services such as Subversion going. But I'm studying the FreeNAS documentation as well as the FreeBSD Handbook to get back into it.

 

[twelve12pm]

Welcome to the forums.

If you search here for Asrock C2750D4l (the mobo of the two Mini versions) you'll find that many members here (including me) have experienced issues with that board, mainly related to temperature and/or the chipset bugs related to the watchdog and the Atom processor.
The chipset issues could only be dealt with by claims through the supply chain. From my own experience and reports here I can say that iXsystems stepped up wonderfully on board failures with an extended warranty program, and there are many reports of Asrock doing the same. You'll find a thread or two detailing these experiences when you search. Here's one https://forums.freenas.org/index.php?threads/how-long-your-asrock-c2550d4i-or-c2750d4i-lasted.45445/ Hopefully the chipset issues are now a matter of the past for current supplies.

Now, the Mini's compact chassis doesn't do much to enhance the cooling possibilities for the installed HDD's or the passively-cooled processor. The same mobo in a different case offers more integrity-enhancing cooling possibilities for both the board and any HDD's. So, if I were doing this all again, and certainly if I were doing it for a small business where the server was a key piece of the operational structure, I would not buy the Mini despite the stellar support that iXsystems has provided me, because I would want a piece of hardware with greater natural integrity and the cramped chassis is really limited in cooling capability, even with modifications, in my personal experience with the Mini.

The Mini XL is of course in a larger chassis and may have better cooling capability, but there's at least one forum user who did "fan work" on his XL to enhance it.

Dear Redcoat,

Thank you so much for telling me about the cooling and chipset issues. It is for discovering things like this about other users' experiences that I am here in the first place. That is very helpful. I was thinking of specifying the XL to have more drive bays available and to leave one or two empty to allow for growing the pool by gradually replacing drives with bigger ones over time, without losing redundancy at the time of the rebuild (which would occur if there is no spare bay if I understand correctly). I will look further into the cooling and chipset issues you mentioned and if I do end up specifying the Mini then we will definitely take care, either through "fan work" or whatever enhancements, to keep it cool. I might even have it installed in a cooled cabinet of some sort. Either way, thank you for mentioning that. By the way, if you had it to do all over again, what box (if any) would you consider this time around?

Thanks for your input,
12

 

[Redcoat]

By the way, if you had it to do all over again, what box (if any) would you consider this time around?

Well, @twelve12pm, for starters I probably should tell you where my head was when I did it the first time, then tell you what I would do today.
I bought the Mini after a year of lurking and reading here because I concluded that I was not familiar enough with servers (as opposed to workstations and PC's of which I have spec'd, built and maintained many over 35 years) and I had enough else going on at the time that I would just be better off buying one rather than building.
If I today had a strong disincentive to building myself, and/or did not have the capability to maintain a server and related networking equipment, I would pick the same tested design that I would build myself and commission whatever computer tech outfit was going to be my maintenance arm to build and test it, and supply it working with all the parts manufacturers' warranties, and give me a decent price for a maintenance contract.
The "tested design" I would most likely build myself today to satisfy the need I had at the time I bought the Mini would be that of one of the well-known contributors here, @DrKK, or something very close to it. There is a three-part post series here on the forum that describes his recent build - start here https://forums.freenas.org/index.ph...romised-of-drkks-new-build.54332/#post-376616 for some pics and use the references in its beginning to access the first two parts of the series.
I would neither choose the white case (black) nor the LED's (none) personally, but nether of those reflect on the inherent quality of the assemblage - you will see that @DrKK makes light reference to those items himself.
He chose a high quality mobo that many here have used with success. Note that he makes a point of the cooling aspect - choice of spacious case, fans, etc. His case has plenty of drive bays. He upped the memory to 32GB from his originally noted 16GB - see the major items spec in his post's signature.

All that said, there are lots of other build reports and discussions here in the forum. There are lots of requests for comments on proposed builds. What you will see repeatedly are requests from the more experienced here for the inquirer to provide some detail on the intended duty and requirements of the server - what is it storing, how much is it storing, how many people are accessing it and how often and with what (PC's, Unix workstations, Macs, etc.), how is it to be backed up, +++
I personally can't relate to a Subversion server and its repositories but perhaps others here can provide objective comment based on that (level of) detail alone.

If you haven't already done so, head for the Resources tab here as @monkey recommended, especially the hardware guide. Look forward to seeing how your project develops. Good luck!

 

[joeschmuck]

I'm going to jump in here and add this comment. FreeNAS is not a setup and forget about NAS. It requires you to learn about it and how to maintain it. It was never created for that end user.

I do not want to tell you that you shoudn't run FreeNAS, I just want to caution you that you need to learn something is all. Since you have previous FreeBSD knowledge then you should do fine. Read the User Guide at least two times from cover to cover (I actually need to read it again, many changes in the past few years). I would also recommend that you run FreeNAS in a VM such as VirtualBox or VMWare Player/Workstation to test it out. Configure the system, create and destroy pools/datasets, jails, etc... You will get a good feel for it this way and figure out if this is what you desire.

FreeNAS is the most Bang for your Buck for a NAS.

As for the ASRock motherboard, there was a fault and iXsystems had no idea but it created a lot of pain for many people. iXsystems stepped up to the plate and replaced the motherboard even after the warranty had expired. I would not recommend this motherboard at all.

If you desire a hands off NAS, buy a Synology or Qnap but if youa re running Subversion then I can't see you wanting to have just a simple NAS, the throughput would be much slower for the same money.

Also, while you play with FreeNAS in a VM, figure out what is the capacity and redundancy you thing you need for the next 3 to 5 years? You can use Subversion in a FreeNAS jail. You can also test that out in a VM.

The main issue with building a system right now is the cost of DDR4 ECC RAM, that stuff is not cheap right now and I have no idea when the prices will start to drop. Otherwise the other prices are not too bad.

The system @DrKK created runs on the same motherboard I've been using, the Supermicro X11SSM-F. I might have been the secord or third person to use this motherboard as a FreeNAS machine. It had it's initial problems but then the initial BIOS was updated and fixed the minor issues we had. Yo don't need to use a third party fan, the one that comes with the CPU works fine and is quiet.

Good luck and hopefully you will be joining our group as a FreeNAS user.

 

[gpsguy]

If you prefer a PDF/ePub version over HTML, see the link in my signature.

I'm studying the FreeNAS documentation

 

[twelve12pm]

@Redcoat, @joeschmuck, @gpsguy, thank you all for so much helpful input...

@Redcoat, I'm not afraid of hardware or to construct a system myself, though my thought process shared some themes with yours when you started -- I figured that specifying and assembling a box myself would most likely not save any money. If anything, it might cost more because I wouldn't benefit from the economies of scale that iXsystems gets when they build them in quantity. My bigger concern was that if my custom build were different than anything else out there, I might run into problems and have a difficult time getting help. So far, my experience in this forum is helping to alleviate that concern. I'm going to study @DrKK's builds.

@joeschmuck, thanks for jumping in. I am aware that nothing server-related is setup-and-forget. Everything requires maintenance, especially in this day and age of having to plug security holes on an almost daily basis. There is also monitoring the health of the disks, etc. Eventually we will need someone on staff who does system administration all day, but until then, it's me.

So... I am actually in the process of reading the FreeNAS User Guide from cover to cover. I'm up to chapter 8 right now and looking forward to the chapter on jails, since that will be an important part of this setup. Next up on my reading list is the FreeBSD Handbook to refresh my memory (since I haven't used it since the FreeBSD 4.0 days) and catch up on all the new stuff it can do. I have to say that the FreeBSD Project and Foundation have grown quite a bit since the ol' Walnut Creek CD-ROM days (ah, memories).

I downloaded FreeNAS-11.1 earlier and am running it in VMware right now. Even though the virtual machine is installed on a USB 2.0 external drive, and even though the installation media is on the same drive (this is just a test after all) the installation time from acceptance through initial reboot was under four minutes! Quite impressive. The ability to test drive the software in this manner, and the fact that it's based on FreeBSD, are two very attractive aspects of FreeNAS for our use case. The boss will like the page with all the fancy graphs in the web interface.

I'm going to test Subversion on the virtual machine. Also my colleagues are getting excited and there's talk of putting some additional internal web-based services on it. So we'll see where this goes. I am going to continue studying documentation and @DrKK's build now. I'll be back with questions... lots of questions.

 

[joeschmuck]

since the ol' Walnut Creek CD-ROM days (ah, memories).

Wow, that was a blast from the past!

Also my colleagues are getting excited and there's talk of putting some additional internal web-based services on it. So we'll see where this goes.

I am pleased that you are testing out the system on VMWare, it really is the best way to see if this product will meet your needs. However when you start talking about web-based services, just be careful. While FreeNAS is fairly full featured it still is the newest kid on the block for VM offerings. If you can use VMWare ESXi for your server base and then build other VMs on that, I think you would have the best of both worlds, a nice and fast NAS and a VM base that is an industry hardened platform. So you could run your web services on a Linux VM, have all your storage on FreeNAS, and if you need something else, just spin up another VM. I know you can run Subversion in a FreeNAS jail so I would think that would be fine, but if you need to have an OS that more people of more familiar with like some Linux varriant the a VM of that might be better.

Here is another thing we see periodically on this forum, an Admin who inherited a FreeNAS system and no one at his/her company has any idea how it works and they have a failure. Do let it happen where you are the only person who has a clue what if going one.

 

[twelve12pm]

Wow, that was a blast from the past!
I am pleased that you are testing out the system on VMWare, it really is the best way to see if this product will meet your needs. However when you start talking about web-based services, just be careful. While FreeNAS is fairly full featured it still is the newest kid on the block for VM offerings. If you can use VMWare ESXi for your server base and then build other VMs on that, I think you would have the best of both worlds, a nice and fast NAS and a VM base that is an industry hardened platform. So you could run your web services on a Linux VM, have all your storage on FreeNAS, and if you need something else, just spin up another VM. I know you can run Subversion in a FreeNAS jail so I would think that would be fine, but if you need to have an OS that more people of more familiar with like some Linux varriant the a VM of that might be better.

Here is another thing we see periodically on this forum, an Admin who inherited a FreeNAS system and no one at his/her company has any idea how it works and they have a failure. Do let it happen where you are the only person who has a clue what if going one.

@joeschmuck, thanks again for your input. Yes, I always test, test, test before committing to something, particularly when (as you say) it will be a big pain if/when it breaks down. To help mitigate the headaches for the next admin after me, and also to help me reproduce setups I've done in the past, I tend to document everything quite clearly. It's a lot of work and I never feel like doing it, but it saves so much time later on.

When talking about web-based services, I'd like to emphasize that we're talking about stuff to be used internally in our organization, not to serve a website to the world. For example, we have an inventory control system called PartKeepr that runs on a pretty standard LAMP stack or whatever. Right now it is running on a Debian virtual machine on some workstation; obviously both the workstation and the vm must be on for it to work. That's an annoying problem we'd like to eliminate. I'm sure it will run fine on FreeBSD.

In terms of FreeNAS being the newest kid on the block to borrow your phrasing, it is my understanding that it is basically FreeBSD with a particular configuration/tuning and some additional software like the web interface and glue scripts to make it all work. Am I right or wrong about that? In my mind, if it is essentially FreeBSD, then there ought to be plenty of skilled people out there who know this system, or who at least can be told that it's basically FreeBSD with some extra goodies. Hopefully I am not mistaken on that point.

I don't know much (okay, anything) about VMWare ESXi right now. (That should be next on my study list, huh?) I am only familiar with VMWare Workstation, Player, and Fusion (which is basically Workstation for Mac). So I'm not sure what you mean when you say that I could use ESXi as a server base with other VMs on that. Do you mean that this stuff runs on the FreeNAS box? Or is it supposed to run on a separate server and mount storage located in the FreeNAS box via iSCSI or something? (I'm guessing it's the latter...)

 

[joeschmuck]

FreeNAS is based on FreeBSD but it does not include all of FreeBSD, however in a jail you can pretty much make a full featured FreeBSD.

When I'm talking about new kid on the block, I'm referring to VMs specifically. Your PartKeepr application would need to be installed into a VM on FreeNAS. I'm not saying it will not work, it may work perfectly well. I'm just saying VMWare has a better following and if I were to use VMs in a work environment then I'd use VMWare.

Yes, you should check out VMWare ESXi. You should do a Google search on Type 1 hypervisor. I'm not trying to force you in a specific direction, I really want you to do the research and figure out what is best your your situation.

There are quite a few network admins here, I'm not one of them, that use this product daily in a work environment. They would honestly be better to give you advice in this topic.

Do you mean that this stuff runs on the FreeNAS box? Or is it supposed to run on a separate server and mount storage located in the FreeNAS box via iSCSI or something? (I'm guessing it's the latter...)

Stuff running on the FreeNAS box. With ESXi you can have a FreeNAS VM, Debain VM, Windows VM, etc... on the same computer and they all appear to be individual computers, but you can also do the same thing with FreeNAS but the difference is VMWare has years of experience and support, FreeBSD Bhyve is new. The good thing here is you can have very fast data throughput between the VMs because it's all internal. There are a lot of benefits to VMWare but you need to really figure this out yourself. Maybe all you need is a few small VMs and FreeNAS will server those well.

Also note that FreeNAS will be making a significant change in version 11.2 with respect to jails and VM support. You will still be able to use jails via CLI from what I'm told but GUI support will be gone. Bhyve will be the VM future for FreeNAS.

I hope I didn't confuse you any.

And just a little more food for thought... Maybe you do go with FreeNAS for now, you can later take your system and easily convert it over to VMWare, meaning you can place your current FreeNAS on top of VMWare ESXi and it will run. You would need to make a single configuration change with respect to the Network NIC but that would be it within FreeNAS. Setting up VMWare would take a few hours for someone who has never done it before, but I'd use a dummy system to play with it of course before touching the real system. So you can change your mind at a later date if that made sense for your business plan.

Back to FreeNAS itself... If the data is really important then you need to design the pool properly. If you are going to use iSCSI then you need to ensure you do not exceed 50% storage capacity, you will need lots of RAM, and again, design the pool configuration properly so you can achieve the high throughput you desire.

 

[twelve12pm]

FreeNAS is based on FreeBSD but it does not include all of FreeBSD, however in a jail you can pretty much make a full featured FreeBSD.

When I'm talking about new kid on the block, I'm referring to VMs specifically. Your PartKeepr application would need to be installed into a VM on FreeNAS. I'm not saying it will not work, it may work perfectly well. I'm just saying VMWare has a better following and if I were to use VMs in a work environment then I'd use VMWare.

Yes, you should check out VMWare ESXi. You should do a Google search on Type 1 hypervisor. I'm not trying to force you in a specific direction, I really want you to do the research and figure out what is best your your situation.

There are quite a few network admins here, I'm not one of them, that use this product daily in a work environment. They would honestly be better to give you advice in this topic.


Stuff running on the FreeNAS box. With ESXi you can have a FreeNAS VM, Debain VM, Windows VM, etc... on the same computer and they all appear to be individual computers, but you can also do the same thing with FreeNAS but the difference is VMWare has years of experience and support, FreeBSD Bhyve is new. The good thing here is you can have very fast data throughput between the VMs because it's all internal. There are a lot of benefits to VMWare but you need to really figure this out yourself. Maybe all you need is a few small VMs and FreeNAS will server those well.

Also note that FreeNAS will be making a significant change in version 11.2 with respect to jails and VM support. You will still be able to use jails via CLI from what I'm told but GUI support will be gone. Bhyve will be the VM future for FreeNAS.

I hope I didn't confuse you any.

And just a little more food for thought... Maybe you do go with FreeNAS for now, you can later take your system and easily convert it over to VMWare, meaning you can place your current FreeNAS on top of VMWare ESXi and it will run. You would need to make a single configuration change with respect to the Network NIC but that would be it within FreeNAS. Setting up VMWare would take a few hours for someone who has never done it before, but I'd use a dummy system to play with it of course before touching the real system. So you can change your mind at a later date if that made sense for your business plan.

Back to FreeNAS itself... If the data is really important then you need to design the pool properly. If you are going to use iSCSI then you need to ensure you do not exceed 50% storage capacity, you will need lots of RAM, and again, design the pool configuration properly so you can achieve the high throughput you desire.

Dear @joeschmuck, thank you again for such detailed information. I am *very* glad that you told me about the FreeBSD jails soon becoming obsolete in FreeNAS, to be replaced by Bhyve. It would suck to invest a lot of time into configuring jails and whatnot only to find out that the next release requires a whole new configuration. Then again, change is the only constant and that would probably happen anyway at some point, which goes along with what you said before, that FreeNAS isn't a fire-and-forget NAS, it's something that requires knowledge and ongoing maintenance.

I looked up Bhyve in the FreeBSD Handbook. It seems interesting for experimenting with -- for example the virtual machines can use ZFS volumes rather than files as virtual disks -- but as far as our server goes, I agree that we probably want to use stuff that has been around longer. I will look into ESXi. (I just noticed the four ESXi-related buttons in your sig.) Can't wait to see the price tag. I have plenty of experience with other VMware products over the past dozen years or so, so if ESXi is anything like, say, Workstation, I should be able to succeed with it. If I can move other VMware virtual machines directly into ESXi the way I can move them between Workstation, Player, and Fusion, that would be a big bonus.

Thanks again for all your help. I'll be back with more questions, I'm sure.

 

[joeschmuck]

If I can move other VMware virtual machines directly into ESXi the way I can move them between Workstation, Player, and Fusion, that would be a big bonus.

Of course you can but you will figure this out when you start reading about it.

 

[danb35]

You will still be able to use jails via CLI from what I'm told but GUI support will be gone.

No, this is not correct. Jails aren't going anywhere. jkh was trying to do that with the abomination that was FN10, but it died a well-deserved death, and with it that way of thinking. What is happening, though, is that jail management is changing from warden to iocage.

 

[wblock]

Also note that FreeNAS will be making a significant change in version 11.2 with respect to jails and VM support. You will still be able to use jails via CLI from what I'm told but GUI support will be gone. Bhyve will be the VM future for FreeNAS.

This does not sound correct. Jails are awesome, and last I knew, GUI support for iocage jails was planned. This is in addition to VM support.

 

[gpsguy]

After moving them onto ESXi, you won't go back.

While many users use the VMware converter for Physical to Virtual (P2V) migrations, you can also do Virtual to Virtual (V2V) conversions from your current environment to ESXi.

If I can move other VMware virtual machines directly into ESXi

 

[joeschmuck]

What is happening, though, is that jail management is changing from warden to iocage.

This does not sound correct. Jails are awesome, and last I knew, GUI support for iocage jails was planned. This is in addition to VM support.

Let me se if I have this correct... The current Warden jails are going away completely? I guess I was under the impression that the Warden jails would remain via CLI support and iocage would be the new jails and supported via the GUI.

Also I was under the impression that the Warden jails would not be compatable with the iocage jails and there would be no automatic conversion during the upgrade to 11.2.

Please educate me on this. I know I don't read all the threads in the forums so I do miss a lot of information going around.

 

[danb35]

The current Warden jails are going away completely? I guess I was under the impression that the Warden jails would remain via CLI support and iocage would be the new jails and supported via the GUI.

I don't know that I can address this issue one way or the other. At a minimum, as I understand it, iocage will become the new standard.

there would be no automatic conversion during the upgrade to 11.2.

My understanding is that a conversion script is still planned for 11.2.

 

[joeschmuck]

I don't know that I can address this issue one way or the other. At a minimum, as I understand it, iocage will become the new standard.

My understanding is that a conversion script is still planned for 11.2.

Appreciate that info. Guess we will see what happens. By the time 11.2 comes out I will have my backup FreeNAS machine operational and replication in full swing, then I can take the plunge and try the upgrade with minimal risk.