So This New Client Has This FreeNAS Box...

[El_Man]

Hello, FreeNASers.

I just got this new client, a photography business, who are running a 15Tb FreeNAS machine for their main storage over the network. And here's me, a poor little FreeNAS virgin. I just got a look inside it today, and here's what I found (NB, the machine was on at the time, and they were really busy so I couldn't turn it off and move it out of the cramped space it was in for a proper looksee, so some details are unknown) :

1. Motherboard/CPU/RAM - an Asus H87M-E, with an Intel i3-4130, and 8Gb of RAM (non-ECC).
2. Disks - eight Seagate 3Tb SATA disks, couldn't tell the model number. Most were plugged into the MB, but two were plugged into an extra SATA card in the PCI Express slot.
   
3. PSU - couldn't see, either brand or wattage. :(
   
4. No idea how the vdevs and zpools are set up. (where can I find that out?)
5. No backups are happening to two smaller 4Tb NAS boxes they have onsite.

Am slowly working my way through the doco (and Cyberjock's noob notes, thank you, sir ;) ), and to reiterate, I did not build this machine. But from what I've read so far, this could be a problem in the making, as far as their data goes. What can I recommend for these people, apart from running away screaming?

Sincerely, El-Man.

 

[Ericloewe]

The processor is salvageable, the disks should be too. Depending on the pool configuration, getting this fixed could be as easy as moving the disks to new hardware.

To figure that part out, open an SSH session (details on turning on the SSH service are in the manual) and give us the output (in CODE tags) of

Code:

zpool status

and we'll help you interpret the data.

In the meantime, hardware recommendations... The brand-new version of our hardware recommendations sticky was posted yesterday (today?) and has updated recommendations for a few scenarios.

I'm assuming the current hardware is good enough, performance-wise. If it's not, there's not much that can be used (except maybe the HDDs). If it is enough, I'd do the following:

• Get a new motherboard (easiest option is an X10SL7-F, which already has an LSI SAS2308 controller, so it can handle a total of 14 disks)
• Get ECC RAM (from the motherboard's tested memory list)
• Move CPU to new motherboard
  
• Flash controller to P16 IT firmware
  
• [If zpool checks out] move disks over to new motherboard
• [If not] Back stuff up, make new pool on new hardware
• Move boot disk to new hardware

You'll have to take a look at the PSU somehow, God knows what they have in there...

While we're at it, let's examine the SMART data for all drives.

Back in the SSH session, type

Code:

camcontrol devlist

and take note of the drive identifiers (they should be similar to "da2" and/or "ada4").

Then, for each drive, run

Code:

smartctl -a /dev/*drive id*

stick the output in Pastebin and give us the link(s).

Finally, tell us which FreeNAS version we're dealing with here.;)

 

[no_connection]

Get a quote from iX for a suitable NAS.

Or if they are to build something suitable, don't touch the current NAS, just leave it as is until all data is transferred, then use it for second backup. First backup being something else.

Problem with "why not use that as backup" is that you could become dependent on using it for retrieving your data.
Kind of like seat belts, not really "in use" for daily travel but you will be glad you didn't use rope when you crash.

 

[AlainD]

Hello, FreeNASers.

I just got this new client, a photography business, who are running a 15Tb FreeNAS machine for their main storage over the network.

....

Sincerely, El-Man.

Do they keep photographs for a long time or are they just handling them (for example printing them; or cheap "booth" photography). Just ask if it would matter if they lose all pictures from before 2014 ;-)

If they value there photographs for more than a few months : get a off-site backup for their archive with verification. Either a tape solution (very reliable, but high investment) or multiple disk stored off-line and off-site and with the contents verified from time to time (for. example with hashdeep). For disk I would at least make 3 copies (on separate disks) and store them on 3 locations. If done with some thought the archives won't change and they are easy to check (at least yearly). If not needed don't use the largest disks ;-)

Replace the FreeNAS machine with a server class machine (see hardware recommendations or trueNAS.) Afterwards the SATA disks from the current setup can be reused for either backup (a second new low spec'ed server) or off-line archive backup.

 

[El_Man]

Thank you for the suggestions, folks. Will try to get an SSL session with the box to obtain some more info as soon as I can.

I have looked at tape backup, as the LTO-6 drives using LTFS look promising - lots of space and reliable, but they're still kind of pricey (NZ$3500 for the drive alone), and client doesn't have the money to spare. Then again, how much is their data worth to them, and they need suitable offsite backup...

 

[wintermute000]

Explain to them without ECC they are running the (small) risk of going boom at any time. The whole lot.
Even if its run down to the local electronics retailed and drop 1k on USB hard drives, make a backup. Stat.

15tb of reliable storage and cheap does not fit in the same sentence. The old adage still applies: they only get to pick two.... I agree they need a proper enterprise class NAS/SAN solution.

 

[Mynorx]

Don't forget the most important thing, get some kind of document signed releasing you from any liability from data loss etc...

 

[El_Man]

Okay, here we go... there was a communication error with these folks - they gave me the wrong email, which is one reason this took so long.

To figure that part out, open an SSH session (details on turning on the SSH service are in the manual) and give us the output (in CODE tags) of
zpool status and we'll help you interpret the data.

Code:

[root@Fileserver ~]# zpool status  
  pool: Raidz2  
state: ONLINE  
  scan: scrub repaired 0 in 14h6m with 0 errors on Sun Sep 21 14:06:19 2014  
config:  
  
  NAME  STATE  READ WRITE CKSUM  
  Raidz2  ONLINE  0  0  0  
  raidz2-0  ONLINE  0  0  0  
  gptid/2e7a9c8d-bdf4-11e3-92ba-bcee7b878417  ONLINE  0  0  0  
  gptid/2ebd45b5-bdf4-11e3-92ba-bcee7b878417  ONLINE  0  0  0  
  gptid/2f04c30c-bdf4-11e3-92ba-bcee7b878417  ONLINE  0  0  0  
  gptid/2f486a4a-bdf4-11e3-92ba-bcee7b878417  ONLINE  0  0  0  
  gptid/2f8ed5ec-bdf4-11e3-92ba-bcee7b878417  ONLINE  0  0  0  
  gptid/2fd2ae24-bdf4-11e3-92ba-bcee7b878417  ONLINE  0  0  0  
  gptid/3019aedb-bdf4-11e3-92ba-bcee7b878417  ONLINE  0  0  0  
  gptid/30639f06-bdf4-11e3-92ba-bcee7b878417  ONLINE  0  0  0  
  
errors: No known data errors 

While we're at it, let's examine the SMART data for all drives.
<snip>
stick the output in Pastebin and give us the link(s).

http://pastebin.com/JBMmLLHD

Finally, tell us which FreeNAS version we're dealing with here.;)

Code:

Hostname    Fileserver.local
Build    FreeNAS-9.2.1.4.1-RELEASE-x64 (fec915c)
Platform    Intel(R) Core(TM) i3-4130 CPU @ 3.40GHz
Memory    7852MB
System Time    Thu Oct 02 17:07:54 PDT 2014
Uptime    5:07PM up 23:08, 0 users
Load Average    0.00, 0.00, 0.00

Hmm, 'surveillance' class hard disks, they seem good enough... (?)

Still couldn't get the machine out to look at the PSU, but at least the lady in charge seemed receptive to getting better hardware.

Oh, one other question - as I first mentioned, they have two other NAS boxes there, a Seagate and a Netgear ReadyNAS. Is there a way I can make the FreeNAS send copies of its files to these machines? Or some sort of backup that way, until a tape backup is installed.

 

[Ericloewe]

OK, a few thoughts:

The output of spool status got messed up. Are all 8 drives on the same intent? If not, post the output directly from the session to pastebin - you're going to have some trouble with that pool. If they are, all drives are in a RAIDZ2 vdev, which means all should be ok.

There's good and bad news about the drives proper:
Good news is that they look fine. Bad news is that they've never run any SMART tests, so there could be trouble hiding around the corner. They've also run for less than half a year, so that sounds positive.

As for secondary backups - that's easy if they support rsync. If they don't you'll have to hack something together. I believe that one of the popular plugins supports something like that. You could also use a client machine to copy stuff around on a schedule.

 

[El_Man]

The output of spool status got messed up. Are all 8 drives on the same intent?

I was confused until I realised you meant indent. But yes, they are, so are all the same vdev. Phew.

Good news is that they look fine. Bad news is that they've never run any SMART tests, so there could be trouble hiding around the corner. They've also run for less than half a year, so that sounds positive.

I'll read up on how to schedule/run a SMART test.

As for secondary backups - that's easy if they support rsync. If they don't you'll have to hack something together. I believe that one of the popular plugins supports something like that. You could also use a client machine to copy stuff around on a schedule.

I shall browse the plugins and the doco for the other NAS. Thank you for your advice, my good sir. :)

 

[Ericloewe]

I was confused until I realised you meant indent. But yes, they are, so are all the same vdev. Phew.

Oops, sorry about that. That's what I get for not double-checking Windows 8's spellchecker.

I'd say your priority is to move the pool to server-grade hardware, so new motherboard and RAM, plus anything else that looks like it's too crummy for the job. Hopefully the disks will pass the SMART tests without a problem, making it a simple matter of moving everything over.

 

[El_Man]

Small threadbump here, but it's about the same machine...

Scheduled and ran some SMART tests, and there were no errors reported. And I discovered that there are indeed some rsync backups taking place to one of the other onsite NAS boxes, so that is good.

I had thought about getting a tape backup for them, but a couple of things put me off, firstly, the high cost of the tape drive itself. LTO-6 drives are really pricey. Secondly, the FreeNAS does not directly talk to a tape drive, any backups would have to be done manually over the network by another PC. Am thinking of a bunch of WD 4Tb Greens in external cases, connected using eSATA, to rysnc to. Pity that USB drives have such shortcomings for reliability.

For now, two more questions, apologies if it's in the manual somewhere, but...

Moving to new hardware - is it simply a matter of moving the disks to a new case with a server-class motherboard and RAM, removing the bootable USB stick from the old machine, inserting it into the new, and booting?

Is there anything special stored on the USB stick, or will it simply be an unpacked FreeNAS image? So if the stick dies, can I simply unpack the same image to a new one and boot without problem?

 

[danb35]

Is there anything special stored on the USB stick, or will it simply be an unpacked FreeNAS image? So if the stick dies, can I simply unpack the same image to a new one and boot without problem?

The config database is stored on the USB stick, so you'd need to download that from the working system, and then upload to the new install. Everything else should Just Work (tm).