Advise On Building A Storage Solution

[Joseph Dinsdale]

Hi all,

So i have reached a point now where I need to rethink my storage solution. So to begin with let me tell you what my current storage solution is and why I believe I need to change it.

So i have 3 small home NAS's, the enclosure are D-Link ShareCenter Pulse DNS-320. they have 2 hard drive bays each. 2 of these has 2x 4TB hard drives in striped raid. So this gives me 8TB on each one. The last one has 2x 2TB in striped raid, giving me 4TB on that one.

Currently the 4TB one is used for me to place backups of my Software installers along with Backup images of my computers. This drive is currently a little over half full.

My 8TB drive is used to keep backups of all my HD media I have purchased from places like iTunes, I also use it to backup any photos that I take using my camera, I keep the all the photos in a RAW format so they take up a fire bit of room. This drive currently has around 500GB until it is full.

My second 8TB drive is a direct backup of the first. I use a program called SyncToy, its a free tool made by Microsoft to Sync 2 folders. So I use that to Sync both my 8TB NAS's.

So my problem with this is when i need to upgrade to more storage i have to buy an extra enclosure + the hard drives to go with them. this is costly close to £400 ($605) to get one ready to use with 8TB storage. Also i need space to put them and they all need power and a network cable.

As I eat up a lot of storage I have decided to build my self a Storage Server. I have speced up a Server for about £500 ($756) that has 16 hot swappable hard drive bay. This will solve the problem of needing to keep buying enclosure when I need more space, all I will need to do is buy the hard drives. Also this only needs 1 power and 1 network cable.

So my current plan is to buy the server next month when I get paid. But then I come to the issue of the best way to set it up.

I will not be buying any hard drives to go with it, that will be something I buy the following month. So on the server FreeNAS will be installed on a 32GB USB 3.0 that will be plugged in to the back of the server. The I am planning on taking the 2 4TB drives from my old Backup NAS and putting them in to the server.

I will then make this in to a Pool called Media. the pool will have both the 2 4TB hard drive set-up in strip raid, to give me back the 8TB storage. Then from my old Media NAS i will transfer the data from that to this new Media Pool.

Following that i will take the other 2 4TB hard drives from the Media NAS, add them to the server and make a new Pool called Media backup, and set that up as a raid as well to give me back that 8TB storage.

Then i will look up away to make the Media Pool backup to the Media Backup Pool. The in the future when i need more storage i will buy hard drives add them to the server and extend the Pools.

So is this the best way to set-up a new Storage Solution? i will keep adding new hard drives and extending the Pools until the 16 bays are all in use.

Also what would be the best way to make the Media Pool backup to the Media Backup Pool, is there a tool in FreeNAS to do this? or should i just keep using the SyncToy from Microsoft?

 

[jgreco]

For the participants contemplating banging their heads on this one, already handled

https://forums.freenas.org/index.php?threads/raid-not-working-correctly.30633/

 

[DifferentStrokes]

This will solve the problem of needing to keep buying enclosure when I need more space, all I will need to do is buy the hard drives. Also this only needs 1 power and 1 network cable.

I'd suggest a motherboard with a IPMI port. If you've never had it you didn't know what your missing, but once you've used it you never what to be without it. This will mean 2 network cables though.

So on the server FreeNAS will be installed on a 32GB USB 3.0 that will be plugged in to the back of the server.

While 32GB drives are fine they are also a bit overkill. You can easily run 16GB drives if you'd like to save a few dollars. Or better yet, use the money you save to buy a second USB drive. You can make a mirrored boot devices. Very cheap insurance.

I will then make this in to a Pool called Media. the pool will have both the 2 4TB hard drive set-up in strip raid, to give me back the 8TB storage. Then from my old Media NAS i will transfer the data from that to this new Media Pool.Following that i will take the other 2 4TB hard drives from the Media NAS, add them to the server and make a new Pool called Media backup, and set that up as a raid as well to give me back that 8TB storage.

You may want to rethink this a little. If you have a setup like that you will not have any redundancy to your data and I'd think you would what that. Check out @cyberjock's ZFS primer. This maybe some good reading as well.

The in the future when i need more storage i will buy hard drives add them to the server and extend the Pools.

Here again, you may what to learn some more about vdevs and pools before you start. You can't add drives to vdevs and make them grow, but you can add vdevs to a pool and make it grow. You can also replace all the drives in an array with bigger drives and make it grow. These are different ends to a means and you should learn as much as you can about there pros and cons before you start throwing data on your NAS.

Also what would be the best way to make the Media Pool backup to the Media Backup Pool, is there a tool in FreeNAS to do this? or should i just keep using the SyncToy from Microsoft?

I think you'll be better off doing this locally on the FreeNAS box. You can go different routes here but the first 2 that come to mind is rsync and ZFS replication.

Feel free to ask more questions.

 

[Joseph Dinsdale]

For the participants contemplating banging their heads on this one, already handled

https://forums.freenas.org/index.php?threads/raid-not-working-correctly.30633/

No offence you are not been helpful here. The post you have linked to was a question I had about a stripped raid not working as I was expecting while running at test on a virtual machine. All you did in that post was call my current set-up rubbish and stupid with out really saying why and how to improve it, and you more or less called me a lunatic my saying my set-up is lunacy. It took you a couple of reply's before you gave me any advice. Then you come to this post about a new project im about to start and claim people are banning her heads as if I have asked something stupid and you say that this has already been handled on a different post. Which is nothing to do with this post about this project.

I don't appreciate the way you have currently talked to me by referencing that I am a lunatic because of my set-up or that I am asking stupid questions. I am not an expert about NAS's or setting up Raid's, so I do need help and advice. So unless you have anything positive to say or any good advice please don't post on my topics any more.

 

[jgreco]

No offence you are not been helpful here. The post you have linked to was a question I had about a stripped raid not working as I was expecting while running at test on a virtual machine. All you did in that post was call my current set-up rubbish and stupid with out really saying why and how to improve it, and you more or less called me a lunatic my saying my set-up is lunacy. It took you a couple of reply's before you gave me any advice. Then you come to this post about a new project im about to start and claim people are banning her heads as if I have asked something stupid and you say that this has already been handled on a different post. Which is nothing to do with this post about this project.

I don't appreciate the way you have currently talked to me by referencing that I am a lunatic because of my set-up or that I am asking stupid questions. I am not an expert about NAS's or setting up Raid's, so I do need help and advice. So unless you have anything positive to say or any good advice please don't post on my topics any more.

And I don't appreciate you coming in, totally ignoring forum etiquette, totally not bothering to read the readily available reference materials, and then taking offense at the answers you get-that-you-don't-like.

So, I will give you some pointers.

Up at the top of each page is a link, conveniently in red, labeled "Forum Rules," which provides some basic guidelines.

Before posting a new thread, always search the forums using popular internet search engines, such as Google, first. It is very likely that your question has already been asked and that a solution has been posted.

You failed to do this, for both RAID0 and for the missing disk space issue.

The community is more eager to help someone who has tried to help themselves and is trying to do a proper setup of FreeNAS.

You failed to do this, since you obviously missed Cyberjock's pool guide and my virtualization warning.

Proper Location for New Thread: Prior to creating a new thread, review the sections of the forum so you can place your posting in the proper location. For instance, the “How-To Guides” section is not for asking questions, it is for someone to post a well thought-out guide that others can use for some specific topic. Questions asking for help should be posted to the “Help & Support” area, and if specific to a subtopic (ex. “Storage”), post there.

You failed to do this. Posting this thread in the Development forum is ... total WTF.

Which is nothing to do with this post about this project.

In my opinion, that is a ridiculous statement. You're trying to take your existing disaster-of-a-setup and migrate it into the same exact paradigm on FreeNAS. I consider that sufficiently equivalent that the reasons it is very bad for your current setup are also completely valid reasons it fails on FreeNAS. Except that on FreeNAS, it's actually worse, because you've given ZFS nowhere to recover from if a checksum fails. ZFS "needs" the redundancy.

So, as a moderator, I'm going to admonish you to read the materials we've made available for new users, especially Cyberjock's slideshow, and to read up and get yourself up to speed a little. What you've done to this point suggests that you think it is fine to wander on in and that you'll be spoonfed answers to your questions. As noted above, we're generally very happy to get someone who's tried to help themselves straightened out, no matter how much of a train wreck they might be, but the flip side of that coin is that we're not really that enthusiastic about the people who can't be arsed to even do the basics.

Thread closed.

 

[jgreco]

And thread moved from Development to new user forum.