multiple FreeNAS boxes on a network

[JoeB]

Hi Guys-

My FreeNAS discs are now full ( 93% or 5TB ), I need to build another box, soon!

I am running 9.10.1 (the latest software) and the following plugins in seperate jails, installed from the plugins tab:
- Plex Media server
- SickRage
- Sabnzbd

The plugins all work perfectly together, fully automaticly.

I'd like to keep the plugins running in freenas, if possible.
I would ideally like to build another box which would be only file storage, and move some files to that box to free up the existing one.
I'd then add the new box to plex, well as the existing box; however I cannot see a way to map network drives, e.g. the new box would be a networked drive.

The only thing I can think of (as i'm not a super-techy person), is to move plex, sickrage and sabnzbd to a windows server (i have a space PC doing nothing, but with only a small HDD). Then map the 2 freenas boxes as network drives.

What are my options? Please don't bash me as this is quite technicality challenging for me as it is.
I was never expecting to run out of space, but I buy tons of movies and now I buy bluerays; they are 25G each!

 

[joeschmuck]

Do you need to build a second machine or just want to build a second machine? If you are looking at possible easy options, list your hardware specs and we can give you some good options.

 

[JoeB]

It's a HP ProLiant MicroServer Gen7 with 12G EEC RAM and 4 x 3TB WD Red HDDs, mirrored, so 5.1TB usable space. I have 190GB free !

 

[joeschmuck]

I have 190GB free !

You have a lot of space, not sure why you are looking for more. ;)

Well you have one easy option if you are really only looking to expand your storage and are happy with the performance of your current server and unfortunately you are limited by your case size:
1) Replace each one of your hard drives one by one with a larger drive size, so if you replaced each drive with a 6TB drive, your capacity would double.

Now I do have some advice for you and it's based on your stated usage... If you are only using the FreeNAS storage for video content and it is replaceable (you have the original media), and you backup any additional files to a safe place periodically (preferably automatically), then you might want to destroy your mirrors and create a RAIDZ1. If you do this and install four 6TB drives, you will have approx. 16.4TB of storage. The downsides are you have to either backup all your data to somewhere or loose most of it (I'd save what you plan to watch a lot of) and should two drives fail, all your data is gone. You can tolerate a single drive failure but with 6TB drives, if they are full, it is possible to loose all your data during that second failed drive.

Also, if you still want 2 drive protection, I'd reformat the drives for RAIDZ2 vice mirrors as it allows for any 2 drives to fail without data loss.

I read that you could convert the DVD drive bay into a Hard Drive bay, this would give you 5 hard drives to use and increase your capacity. I wanted to see if you could use the eSATA connection but I don't have the info on your system to see if you can tap off on the inside of if you need to bring the cable from the outside and to the inside. Of course it it's just screwed on the back, you could relocate it inside the case. When you replace your hard drives, see if you can find a pair of slim drive to fit in the DVD bay. It's worth looking into. If you can buy one 3TB drive and place it into your DVD drive bay and created a RAIDZ2, you have added ~3TB to your pool.

Another option is to reduce the file size of your videos. There are tools out there to make your BluRay videos a respectable size like 3GB for the main movie vice super large and still retain great quality. The tool I use isn't free, DVDFab, but it works fantastic. There are free tools out there which do work great but some require a steep learning curve.

Lastly, the only other good option is like you said, to build a new system, but this time build one which has more drive bays.

So I hope I've given you a lot to think about. If you want to buy/build another server, I recommend a large case and you can take the drives you presently have and combine those with four more drives if you like into a single system, and you could run all 3TB drive if you like, it keeps the costs down. Again, I'd recommend backing up your data and creating a new pool, RAIDZ2 if you build a new system.

 

[nojohnny101]

@joeschmuck has laid out some good advice.

Only thing I will add: It would make more sense to me to keep one FreeNAS box unless your circumstances explicitly call for two FreeNAS boxes. No reason to complicate your management and backup solutions.

If I were you, I would build a new box (if budget allows) and this time spend some good time researching components that will set you up well into the future. This doesn't mean buy expensive, this just means buy a case that has more drive slots then your current setup, this means make sure your board can support 64gb of ECC ram, etc.

Lots of good veterans on here that are willing to help, but do your research then ask questions!

 

[gpsguy]

Some users stack 2 drives in the DVD bay using an adapter like the two listed below.

http://www.frozencpu.com/products/1...pter_Noise_Reducer_-_Black_AV00-0100-AKS.html

https://eshop.macsales.com/item/OWC/MM52T35/

To use either the ODD and/or eSATA port, you'd need to run one of the hacked BIOS' to unlock AHCI on these ports. Why HP didn't do it, is beyond me. To use eSATA, you'd need to route the cable in from the backside of the case.

I read that you could convert the DVD drive bay into a Hard Drive bay, this would give you 5 hard drives to use and increase your capacity. I wanted to see if you could use the eSATA connection but I don't have the info on your system to see if you can tap off on the inside of if you need to bring the cable from the outside and to the inside.

 

[JoeB]

Thanks all, that's quite a bit to think about. I'm happy with the hardware, it's only running plex to 3 TVs and only ever 2 at a time. The TVs seem to request the raw video mkv's so there is no transcoding happening, so the limit is the network speed, which is gigabit to the switch, then the TVs are 100mbps i believe.

Anyway, if i understood the above, my options are:

1) Keep existing box - flash modded bios - add 5th drive to ODD and maybe a 6th to the external(rear) sata port - Use Z1 - This gives 9T (4hdds), 12T(5hdds) 15T(6hdds)

2) Build a new box. get a 8U case - add a motherboard that has 8 sata ports

Option 1 will take me probably 2 years into the future, so i'm happy to do that. A few questions i have:

With Z1, can i add the 5th disk, and then the 6th disk at later dates; e.g. start with 4 disks, then add the 5th a few months time etc?

Can Z1 be upgraded to z2 without having to move all the data off and back on again?

How do I move all my data (currently 5T) to another server with assurances that its actually copied to all? I know FTP can stop half way through a file without notifiying the user :-(

Also:

Another option is to reduce the file size of your videos. There are tools out there to make your BluRay videos a respectable size like 3GB for the main movie vice super large and still retain great quality. The tool I use isn't free, DVDFab, but it works fantastic.

I also use DVDFab in COPY mode. What settings are you using to reduce a BD to 3G ? I tried to copy a BD BD50 to BD9 and it took almost 3hrs to copy the disk!


Sorry for the long post, it's a lot of data and I really don't want to have to copy all the DVDs again as it took forever when I did it the 1st time!

 

[joeschmuck]

2) Build a new box. get a 8U case - add a motherboard that has 8 sata ports

8U would be BIG! You probably don't understand the "U" term but I get your point, a case which could house 8 drives.

What I don't know is if you could install the motherboard from the current system into a new larger case. Someone who has seen one of those boards might be able to comment.

With Z1, can i add the 5th disk, and then the 6th disk at later dates; e.g. start with 4 disks, then add the 5th a few months time etc?

Nope, a pool must be created with the number of drives it will need to live with, for the purposes of this story. You could start out with smaller drives and replace each drive with a larger one in the future but that is about the only option.

Can Z1 be upgraded to z2 without having to move all the data off and back on again?

Nope, you need to destroy the pool and then recreate it which means all your data is destroyed.

How do I move all my data (currently 5T) to another server with assurances that its actually copied to all? I know FTP can stop half way through a file without notifiying the user :-(

That is typically a manual process for me but I have used simple tools like SyncToy from Microsoft which will allow you to "Contribute" (that is the term in the software that I'd use) or copy your data from one place to another. There are other free tools out there as well.

One other option is... (I like to think outside the box at times)
1) Buy all the new hard drives you need to build a new larger pool.
2) Install these drives in a different physical computer that has 8GB RAM or more and the SATA ports you need, maybe the one you are typing on now.
3) Boot new computer using FreeNAS on a USB Flash drive.
4) Create your new pool and a new CIFS share and start copying your data over.
5) On old system remove the pool (do not destory it).
6) Transfer the new drives into your server and Auto-Import your new pool.
7) Configure as required.

I also use DVDFab in COPY mode. What settings are you using to reduce a BD to 3G ? I tried to copy a BD BD50 to BD9 and it took almost 3hrs to copy the disk!

I use it to rip the Main Movie and create it as an MP4, I use high quality single pass, my personal experience was that with two pass I just couldn't see the difference and the file size didn't really change. And yes, it does take a long time to even do just the main movie. I'm pretty sure you could take your present content that you have copied and set it up to batch convert all your movies and since they are already on a hard drive, it may be faster. But you need a place to store them. So I'd take and try one movie, make sure it works fine, then delete the original to free up space. As you free up space you can then start caching several jobs and walk away for several hours. It will take time since you will be doing the conversion over an Ethernet connection but that is okay, the goal is to free up some space.

So my advice to you is to see if you can reduce the file size, but maybe you really want all the BD extras. It's your call but if you can fit 5+ movies into the same space as 1 movie, well that is a big gain.

 

[JoeB]

How to large conpanies add more storage if you cant add another disk?

Sent from my GT-I9195 using Tapatalk

 

[pirateghost]

How to large conpanies add more storage if you cant add another disk?

Sent from my GT-I9195 using Tapatalk

You appear to be confused.

A pool is comprised of one or more vdevs.

A vdev is comprised of one or more disks (a single disk is a disaster waiting to happen).

You can expand pools with more vdevs.

You cannot add disks to a vdev once it's created.

 

[danb35]

How to large conpanies add more storage if you cant add another disk?

So, to get to the conclusion of @pirateghost's post, you add storage by adding more redundant groups of disks. To use an example very close to reality, suppose my server has 6 x 2 TB disks in RAIDZ2 (one vdev), and I want to add capacity. I could replace each of those 2 TB disks with larger ones, but then I'm throwing out perfectly good disks. So instead, I can add a second group (another vdev) of 6 x 4 TB disks in RAIDZ2 to the pool, which will expand it by about 16 TB. When I want to expand again, I can add another group (yet another vdev) of six disks (it helps that my chassis has room for 36 disks), and repeat as needed.

 

[joeschmuck]

@JoeB So while someone can add more storage to a server, in your current situation you cannot take advantage of this without some hardware changes which is possible, you could add a new HBA and add an external box to hold more hard drives. I was trying to keep my recommendations to what you have right now.

 

[adrianwi]

I've got 5 drives in my HP N54L and it's as simple as adding a drive into the ODD bay. Think I used an adapter to convert power from molex but there is already a spare SATA port on the motherboard. Not sure how you'd squeeze in another one though :D

I was in a similar position about 12 months ago and decided to play the long game. I built a much bigger box with the capacity to hold twice as many disks, that will hopefully meet my requirements for the next 4-5 years. The N54L is still in use as a nightly backup. Not the cheapest option short-term, but I suspect it will be overall.

 

[JoeB]

OK guys thanks for all the help! I've ordered a 5th 3T drive and will be flashing the bios as required. I managed to talk a friend into allowing me to dump 6T onto their server which has raid etc for a week or so until i can get Freenas running again.

When i destroy the set, and recreate as z1, will i lose my jails? Took ages to set them all up and would like to retain them, if possible.

 

[adrianwi]

Yes, destroying the pool and recreating with more disks will remove everything on it, so your datasets including the jail one. If you copy everything off the pool and then copy everything back, you should be able configure the jail config and they'll all work again. Do a search on the forum and you find a few threads with some suggested steps, generally using replication to backup and restore your pool.

 

[Stux]

Yes, destroying the pool and recreating with more disks will remove everything on it, so your datasets including the jail one. If you copy everything off the pool and then copy everything back, you should be able configure the jail config and they'll all work again. Do a search on the forum and you find a few threads with some suggested steps, generally using replication to backup and restore your pool.

Forgive me where I must of missed it, but since you're planning on running Z1, you must have a backup in place right? So you just refresh your backup, blow away your old pool, and then restore from backup.

Right? I mean, you're running planning on running 5 x 3TB in RaidZ1... you'll have a backup right?

Unless the only content is bluray rips, and you're really happy to re-rip them?

 

[JoeB]

It's currently setup as mirror at the moment, i'm changing to Z1. I backup some important files like photos etc to the cloud every now and then, but not the media, too large.

 

[Stux]

It's currently setup as mirror at the moment, i'm changing to Z1. I backup some important files like photos etc to the cloud every now and then, but not the media, too large.

So how do you feel about the near certainty of losing your array eventually?

 

[joeschmuck]

@JoeB Have you considered shrinking the size of the videos at all?

 

[JoeB]

So how do you feel about the near certainty of losing your array eventually?

I thought Freenas was supposed to be a mature product, why would it lose the array?