Replacing Hard Disk - swap partition question

[bkaufx]

I am replacing a hard disk that had some SMART errors. I did it with the zpool command through ssh instead of through the GUI because searching online it just seems like people complain about how the GUI doesn't work for this.

Anyway, The old drive was ada2p2, and the new one is just ada2. I just realized this might be because when I first created the zpool through the GUI it partitioned the drive to create a swap partition. Do I need to redo this drive replacement so that I can have a swap partition on the new drive? What is the benefit of this and how do I do it? Just use the GUI to replace the drive?

Here are the steps I have taken so far:
1. zpool offline pool-name disk-name
2. turned off computer and replaced disk
3. zpool replace pool-name old-disk-name new-disk-name ( old disk name was ada2p2, new disk name was ada2 )

That is all i have done. it is now 70% through resilvering. My plan is that when it is done i will detach ada2p2 and be done with the process. any thoughts on this? Searching the message board it seems like there is no consensus on how to replace a failing drive which kind of scares me.

I am running the 8.0.4 release version. I originally installed the 8.0.3 release then GUI upgraded to 8.0.4-RC and then 8.0.4-Release. The zpool was created through the GUI in 8.0.3 by doing create volume, selecting my four disks, and picking zfs / raidz.

 

[ProtoSD]

bkaufx,

Sorry this is quick post. If you do a gpart show from the command line you'll see how all of your other disks are partitioned, including the swap partition at the beginning of each disk BEFORE the adaXp2. The swap partition is striped across your pool, so since you didn't properly recreate it on the disk you're replacing it may have unintended consequences.

If you take a look at this post about a bug I reported with replacing disks, the output in the first CODE window, near the end of the output has the commands you would use to manually partition the disk with the correct swap partition. You may need to reference the output from the 'gpart show' to get the correct values to match your other disks.

http://forums.freenas.org/showthread.php?5810-Replace-Disk-fails-from-GUI-in-8-0-3-p1

 

[bkaufx]

Resilvering finished, here's some info. The new disk doesn't show up in gpart show, which is weird to me but I guess maybe it is because I never partitioned it manually...

Based on the output of the swapinfo command, the system seems well aware that I no longer have that swap partition there. Does this just mean that my swap went down from 8gb to 6gb? I don't think I really need the extra two gigs. I have 8 gigs of ram, and only four drives. I am not doing any highly intensive tasks, just using as a media server.

GPART SHOW:

Code:

=>     63  8069040  da0  MBR  (3.8G)
       63  1930257    1  freebsd  [active]  (943M)
  1930320       63       - free -  (32K)
  1930383  1930257    2  freebsd  (943M)
  3860640     3024    3  freebsd  (1.5M)
  3863664    41328    4  freebsd  (20M)
  3904992  4164111       - free -  (2.0G)

=>      0  1930257  da0s1  BSD  (943M)
        0       16         - free -  (8.0K)
       16  1930241      1  !0  (943M)

=>      0  1930257  da0s2  BSD  (943M)
        0       16         - free -  (8.0K)
       16  1930241      1  !0  (943M)

=>        34  1953525101  ada0  GPT  (932G)
          34          94        - free -  (47K)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  1949330703     2  freebsd-zfs  (930G)

=>        34  1953525101  ada1  GPT  (932G)
          34          94        - free -  (47K)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  1949330703     2  freebsd-zfs  (930G)

=>        34  3907029101  ada3  GPT  (1.8T)
          34          94        - free -  (47K)
         128     4194304     1  freebsd-swap  (2.0G)
     4194432  3902834703     2  freebsd-zfs  (1.8T)

ZPOOL STATUS

Code:

  pool: data
 state: ONLINE
 scrub: none requested
config:

	NAME        STATE     READ WRITE CKSUM
	data        ONLINE       0     0     0
	  raidz1    ONLINE       0     0     0
	    ada0p2  ONLINE       0     0     0
	    ada1p2  ONLINE       0     0     0
	    ada2    ONLINE       0     0     0
	    ada3p2  ONLINE       0     0     0

errors: No known data errors

SWAPINFO

Code:

Device          1K-blocks     Used    Avail Capacity
/dev/ada0p1       2097152        0  2097152     0%
/dev/ada1p1       2097152        0  2097152     0%
/dev/ada3p1       2097152        0  2097152     0%
Total             6291456        0  6291456     0%

 

[Durkatlon]

Something I've been wondering about which is related to this. So when a disk goes bad and let's say it drops off the machine (no longer visible). Does that mean the system will imminently crash because the swap space is now broken, or is FreeBSD smart about this?

I gather the swap is not used in a redundant manner, so if the swap on the broken disk was in use at the time and had some memory paged out to it, it seems like the OS would crash when it found out those pages are no longer available.

 

[peterh]

If a disc fails AND it contains swapped-out pages that will be requested the system will fail.
Thus reducing the number of swapdevices as few as possible is beneficial. Only if swapping
is violent will swapping to ( up to 4 discs ) be of value.

A single disk will usually serve all swap needs on most systems. A pure fileserver with enough memory will
manage without any swap at all.

 

[bkaufx]

Okay, I guess I don't care about the swap space not being on the new drive. But can someone explain why the new drive doesn't show up on gpart? I never formatted the drive. When I put it in the computer I think it was formatted ext3. Is it safe to assume that zpool formatted it in zfs correctly? Any way to verify this?

 

[ProtoSD]

Is it safe to assume that zpool formatted it in zfs correctly? Any way to verify this?

The reason gpart show doesn't show anything is because it wasn't partitioned. ZFS will just write to it regardless, but without the partition table FreeNAS/GUI will probably choke on it at some point.

Here's a tutorial about disks commands in FreeNAS. There are probably a few more commands & examples that need to be added when I get time.

http://protosd.blogspot.com/2011/12/useful-commands-for-diagnosingmanaging.html

 

[peterh]

One might use filebased swap : the freebsd handbook says :
http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/adding-swap-space.html

There a swapfile could be made on a suitable zfs-filesystem
The same filesystem could be used to store parts of system info ( logfiles ?) Just an idea...

 

[joeschmuck]

@bkaufx
I don't run a swap file on my drives but then again I have 8GB of RAM. I didn't run a swap file when I had 4GB of RAM either and never had a problem but I upgraded to the 8 as a safety measure. I don't know if anyone out here will tell you that you don't need a swap file but I can say with certainty that for my applications that I do not need one. In my mind this simplifies the problem of a potential system crash due to a hard drive failure and the SWAP partition being in use. I'd rather have a way to add a separate swap drive (say flash drive) vice using a striped swap file across all the drives.

Anyway, my point is, if you have enough RAM, you might consider doing away with the swap partition all together whenever you get to a point where you can reformat all your drives but please do not let me talk you out of using a swap partition if that's what you want to do. I'm just providing my experience.

 

[ProtoSD]

I think the abandonment of swap is discouraged because it can be used as a crash dump analysis tool. I guess if you have enough memory and don't think that's important, go fer it...

 

[peterh]

The current problem is that swap is done to partitions on all storage disks. If swap was
dedicated to a specific drive OR done to a file on top of a mirrored/raid-ed filesystem then
we would have no dependencies and could for instance replace a drive on-the-fly.

Turning off swap gives the same benefits although we miss swap and crashdump.

 

[Durkatlon]

Swap on all data drives seems like a really weird design. I wish I could easily get rid of these partitions on my FN8 systems. I have hot-swap enclosures for my NASes but these are of course completely useless if the OS will just crash when a drive goes missing. Grr....

 

[joeschmuck]

I think the abandonment of swap is discouraged because it can be used as a crash dump analysis tool. I guess if you have enough memory and don't think that's important, go fer it...

I didn't realize the crash dump was pushed to the swap partition. Well I'd still like a separate swap drive in-lue of spreading it across all the drives. A 4GB USB Flash drive would be enough and they are cheap. Maybe it's just wishful thinking.

 

[peterh]

If you need swap as crashdump device it has to be at least as large as your physical memory
In addition you need to set dumpdev="/dev/ad4s1b" ( or wherever your swap is located) to have savecore
actually save your crashdump somewhere. And yes, savecore needs at least as much space
in savecores dumpdir as you have memory. ( /var/crash default )

 

[joeschmuck]

If you need swap as crashdump device it has to be at least as large as your physical memory
In addition you need to set dumpdev="/dev/ad4s1b" ( or wherever your swap is located) to have savecore
actually save your crashdump somewhere. And yes, savecore needs at least as much space
in savecores dumpdir as you have memory. ( /var/crash default )

Very good to know and makes sense to me.

So the default swap file size of 2GB per drive might not be enough. Say I have 8 GB of RAM and three 2TB drives in a single pool. This gives me 6GB of swap space, not enough for the 8GB of RAM. Sound about right?

 

[peterh]

Yes.
You also need space at /var/crash to collect the coredump(s), this vill not be possible on a 2G USB-stick. So in essence,
you don't get coredumps on freenas-es.
That said, freebsd is fairly well debugged, and there is no real need to always have a working crashdump in freenas.
Should problems happen on a specific hardware one cat install a full freebsd on a disk and use that when debugging.