FreeNAS 9.10.1 moving data files

[muzik]

It has taken me weeks and lots of reading to decide between the few good servers out there; FreeNAS, MediaVault, OpenNAS, etc.

I settled now for FreeNAS. During the research, which includes an almost complete read of the 306 page documentation, I've tried as well to find an answer to this question below.

I have installed, far from configured fully, but getting there. And part of this configuration involves the transfer of data.

Data: music, pictures, movies, each that I'd like a folder for (I'm sure in FreeNAS that word, folder, does not exist). Anyway, the pressing matter now is how do I copy this data from an existing server that has services such as ftp, ssh, SAMBA, to my FreeNAS box. I finally accepted that I could not simply plug the drives in the FreeNAS server, sad, but OK. But I don't know how to move all this data to the new FreeNAS server.

Perhaps a relevant question, since those three categories will span several hard drives (another old term) it would be good to know how to configure Plex or emby, or any of the music server applications there, to scan multiple hard drives. I don't know how, perhaps you do.

Perhaps while you're reading this, you may have an idea of a plugin that will serve pictures to a TV. There's none such currently in the official plugin on the WEBUI. I don't need a loaded plugin to do this, something small and simple to all me enjoy pictures I take on TV.

I'm looking forward to this long-term relationship and change. It's my first post; please be nice.

 

[muzik]

I forgot to say; many thanks for your time reading.

 

[nojohnny101]

Welcome to the forums! You will find the FreeNAS community very friendly and helpful. I should also say that you are off to a great start with doing proper research and homework before posting, we like that! :)

Make sure, when posting, to post your hardware and software specs as per forum rules. So you have already built and installed FreeNAS? What is your:
- motherboard?
- RAM?
- CPU?
- How many drives, what size, and how are they configured (raidz1, raidz2, mirrors, etc)?
- PSU
- what is the FreeNAS OS installed on?

So I'm assuming, from what I can understand from your post, that you have already installed FreeNAS and that is populated with drives? Do you plan to install the hard drives you have your data into FreeNAS once you get your data off of them? I'm assuming your current data is not on a NAS that uses ZFS. If that is true, then problem rsync is your best bet. See here in the manual

Data: music, pictures, movies, each that I'd like a folder for (I'm sure in FreeNAS that word, folder, does not exist)

Correct, these are referred to as "datasets" but are not quite the same as folders.

Here is the guide that I used to setup plex: Install Plex in a Jail
There is no such thing as any plugin you install (or any additional software in FreeNAS) having to "scan multiple hard drives". Or rather, you don't have to tell it to do this or configure it that way. FreeNAS groups all storage into pools and vdevs, and those are automatically handled by FreeNAS. The data is automatically striped across the array. Here is a great presentation we recommend to new comers which lays out all the relevant terms very clearly:
Slideshow for Noobs

Perhaps while you're reading this, you may have an idea of a plugin that will serve pictures to a TV. There's none such currently in the official plugin on the WEBUI. I don't need a loaded plugin to do this, something small and simple to all me enjoy pictures I take on TV.

It is not the most elegant solution, but plex can handle photos. Doesn't mean it is good at it (very little control or organizational tools for photos) but it can do it. You could then just mirror your screen on a tablet or laptop or phone or whatever to some type of Appletv or chromecast. I have not set it up, but I assume Owncloud could be appropriate for this as well although it is more in the lines of a personal NAS "dropbox" then a dedicated photo organizer/viewer.

Hope this helps!

 

[muzik]

Welcome to the forums! You will find the FreeNAS community very friendly and helpful. I should also say that you are off to a great start with doing proper research and homework before posting, we like that!:)

Thanks for your very warm welcome!:)

Make sure, when posting, to post your hardware and software specs as per forum rules. So you have already built and installed FreeNAS? What is your:
- motherboard?
- RAM?
- CPU?
- How many drives, what size, and how are they configured (raidz1, raidz2, mirrors, etc)?
- PSU
- what is the FreeNAS OS installed on?

It's a HP Proliant N40L with 8GB RAM. FreeNAS is installed on an 8GB USB stick. By preference was to install it to a 60GB SATA hard drive, but at install, I was informed by the installer that the preferred destination is a flash drive. I had spent no fewer than 8 hours, looking for such definitive solution, during which I condemned a laptop so I can use its hd. That's OK.

Right now, I have a single 3TB HD in the server. Once I have moved data, I will have three of the same. I think I elected RAID0 during install. My datasets, the once I care about much, will be backed up to an external USB hard-drive formatted as ext4, weekly. I hope this won't be a problem. I need to lookup setting up cron jobs on FreeNAS. Perhaps someone has a better way and can share. But that's my plan for now.

So I'm assuming, from what I can understand from your post, that you have already installed FreeNAS and that is populated with drives?

Yes.

Do you plan to install the hard drives you have your data into FreeNAS once you get your data off of them?

Yes.

I'm assuming your current data is not on a NAS that uses ZFS.

If that is true, then problem rsync is your best bet. See here in the manual
Correct assumption, which technically means, yes. Thanks for the manual.


Correct, these are referred to as "datasets" but are not quite the same as folders.

Here is the guide that I used to setup plex: Install Plex in a Jail

There is no such thing as any plugin you install (or any additional software in FreeNAS) having to "scan multiple hard drives". Or rather, you don't have to tell it to do this or configure it that way. FreeNAS groups all storage into pools and vdevs, and those are automatically handled by FreeNAS. The data is automatically striped across the array. Here is a great presentation we recommend to new comers which lays out all the relevant terms very clearly:
Slideshow for Noobs

Based on what I'd read, I should have known this. But I'm getting used to the idea of jail and that tripped that question. For example, it was amazing to read and see that each plugin would demand off the DHCP server its own IP.

While on networking, if I remove the server from a router and join it on another, would all these IPs auto update based on the DHCP of the new router? Or, would this need manual configuration?

It is not the most elegant solution, but plex can handle photos. Doesn't mean it is good at it (very little control or organizational tools for photos) but it can do it. You could then just mirror your screen on a tablet or laptop or phone or whatever to some type of Appletv or chromecast. I have not set it up, but I assume Owncloud could be appropriate for this as well although it is more in the lines of a personal NAS "dropbox" then a dedicated photo organizer/viewer.

Hope this helps!

Thanks this has helped. I will keep looking for something other than Plex for this. I just need something simple.

What do you mean, "mirror my screen"? If I used this, would those devices not simply connect to the Plex server and be able to access the pictures?

Can I ask, I recall reading somewhere, of the possibility of installing other plugins that are not in the WEBUI. Where can I find a list of those and how can they be installed?
Many thanks!

 

[depasseg]

But I don't know how to move all this data to the new FreeNAS server.

A SMB/CIFS share is probably the easiest way to present the dataset to the client that has the source data for your copy task. Some use a tool like Robocopy for windows to make sure all the files and folders get copied properly.

If you are running a linux then look into rsync.

 

[nojohnny101]

What do you mean, "mirror my screen"? If I used this, would those devices not simply connect to the Plex server and be able to access the pictures?

Yes. The term mirroring is specifically used by Apple to showever is on an iPad or iPhone on a tv hooked up to an appletv. Google calls it "casting". I thought you wanted to get the photos on the TV and if you did put them in plex, "casting" or "mirroring" would be one way to get it from a phone/tablet to a TV.

There are of course tons of ways to get things to a TV and I'm not sure exactly what devices you have or your setup.

 

[muzik]

A SMB/CIFS share is probably the easiest way to present the dataset to the client that has the source data for your copy task. Some use a tool like Robocopy for windows to make sure all the files and folders get copied properly.

If you are running a linux then look into rsync.

Can I ask if the details in this post, particularly, the portion that allows for preformatted disks with data to be used is current, as I don't see that option in the version I'm using:
https://www.linux.com/news/configuring-storage-freenas

 

[depasseg]

I'm not exactly sure what you mean, but if you have single drives, then yes, they can be imported one at a time:
http://doc.freenas.org/9.10/storage.html#import-disk

 

[muzik]

I'm not exactly sure what you mean, but if you have single drives, then yes, they can be imported one at a time:
http://doc.freenas.org/9.10/storage.html#import-disk

Awesome. This saves a ton of time, messing about with rsync, which I'd use later for backup. But for now, that I can simply bring my disks, one at a time, each of which has data into the NAS is wonderful. The link you sent is great. It says "The import is meant to be a temporary measure to copy the data from a disk to an existing ZFS dataset." What's the problem of leaving things that way, not copying but using the disks that are brought in, formatted as ext4, stay that way? Any ideas, please.

 

[depasseg]

Awesome. This saves a ton of time, messing about with rsync, which I'd use later for backup. But for now, that I can simply bring my disks, one at a time, each of which has data into the NAS is wonderful. The link you sent is great. It says "The import is meant to be a temporary measure to copy the data from a disk to an existing ZFS dataset." What's the problem of leaving things that way, not copying but using the disks that are brought in, formatted as ext4, stay that way? Any ideas, please.

You won't be able to share them or add data to them. All you can do is import.

 

[muzik]

depasseg, you're just a great person. I thank you; a lot.

We're off to a good start, I'm now importing a 2TB and already I can see the shares via CIFS. Thanks. Is there a "status" page that can show you the different activities; serving files, to whom? importing discs; progress, etc? That would be nice, gives an idea as to what's happening currently.

Is their a way to browse datasets locally on the server, aside using services such as NFS, CIFS?

I am slowly liking this system, the way volume, dataset, etc work. I've been confused for a while. But old habits die hard. And so I ask, how can I recover from a hard drive crash, when I feel this much loss of control as to what's exactly happening to my data in a volume?

Earlier times, you could pull a HD from a machine and use any of the disc recovery tools. Is that possible with this. If so, how? And I ask because here we have the volume growing as discs are added. And one does not need to creat a new folder/dataset if a HD/vol gets filled up. We simply keep adding to a data set and the actual space from the volume is used to accomodate new data. That actual space is spread across several HDs but is presented to the user as a single volume.

If I got all of that right, can you please answer how one goes about recovering data if say, one of the HDs stops working?

 

[depasseg]

You need to read cyberjocks newcomer guide. I fear you are seeing yourself up for complete data loss by expanding a drive at a time.

There aren't data recovery tools. You need to configure the vdevs with the appropriate level of redundancy. Most people do something like install 6 disks and configure then as raid z2. This allows any 2 of the 6 drives to fail and not lose data. Any failed disk can be replaced while the system is still running.

I'm not aware of anything that lets you see the status of everything going on.

 

[muzik]

You need to read cyberjocks newcomer guide. I fear you are seeing yourself up for complete data loss by expanding a drive at a time.

There aren't data recovery tools. You need to configure the vdevs with the appropriate level of redundancy. Most people do something like install 6 disks and configure then as raid z2. This allows any 2 of the 6 drives to fail and not lose data. Any failed disk can be replaced while the system is still running.

Thanks for this. The problem is that all the discs I want to use 3 x 3TB have data, except one. So the plan is to import to that one, clean/format via FreeNAS after the import and make that disc available to the NAS. And then move onto the next disc. I don't have a ton of blank HDs here. I'm moving from one server to FreeNAS. That's my trouble. Any suggestion you might have to help me with this plan? Or is it all doomed?

I'm not aware of anything that lets you see the status of everything going on.

That's unfortunate. A status page of all services, plugins, network activity, sure would be useful for a NAS.

I will now go print and read cyberjocks newcomer guide.
Thanks so much. You're the one keeping me hanging on this project. The first import seem to have taken most of the day, but it's done now, as I don't see disc activity light on and I have been able to click and see the contents of folders, which was not possible earlier. Many thanks!

 

[muzik]

You need to read cyberjocks newcomer guide.

I've just now been looking, but can't find it. Perhaps if you know of the link, would you please share.

 

[SweetAndLow]

In my signature there is lots of reading.

Sent from my Nexus 5X using Tapatalk

 

[Stux]

I'm not aware of anything that lets you see the status of everything going on.

top

Is about the closest

 

[Stux]

ZFS doesn't believe in data recovery tools ;), and rather it plans to obviate it them through advanced cow and data redundancy.

You need to have enough redundancy to withstand projected drive failures.

And a backup.

 

[snaptec]

Maybe lend some drives?
You wont get anything except striped single hdd vdevs by adding just a disk ...

How full are the hdds?
You could copy everything to one, make a mirror out of the other 2 and Import from the third.

And you have a backup, right?


Gesendet von iPhone mit Tapatalk