Install takes 20 minutes

[Void_]

Hi, encountered this issue and I think it could help some others.

Hardware: SuperMicro X10SL7-F, Xeon E3-1240, Crucial ECC memory 16gb (2x8)
Usb: Sandisk Cruzer fit 64gb, Duracell 8gb

After booting through the grub menu, selecting the source and setting the root password.
The setup starts and seems to do nothing at the 'version number' line right after the 'set da0 active' line.
After about 20 minutes the setup will resume, where it will hang for another 5 minutes.

Maybe it's related to the 64gb stick, maybe the 8gb one I don't know.
If you ever catch the setup seemingly doing nothing just let it run for a while.

 

[DifferentStrokes]

I sense a disturbance in the force. ;)

 

[joeschmuck]

I positive its the 64 GB stick because during the installation the entire boot drive is formatted which would mean 64GB. It does take considerable time.

In the future I wouldn't use such a large USB flash drive, 16GB would be the largest I'd ever consider unless you are using a high quality USB 3.0 flash drive and on a USB 3.0 enabled port.

 

[danb35]

Creating a ZFS pool doesn't involve traversing the entire device as does formatting some other filesystems. Think about when you created your storage pool--it takes less than a minute, and doesn't vary much with disk capacity or number of disks. Consequently, I'm not at all sure that the size of the 64 GB device has anything to do with this. Of course, if it's a crappy or defective device, that could cause all kinds of problems.

 

[DrKK]

The man has a point Joe.

Unless, of course, Joe is saying that the **BOOT** device in particular, because of something peculiar about the boot device being a boot device, gets a full surface format.

 

[joeschmuck]

It has been my personal experience that the USB Flash boot device takes longer to format depending on its size ever since we have started to use the ZFS (GPT) format on the Boot Device and the only answer I have is something which sticks in the back of my head which says that the drive is performing a scrub but hell if I can recall where I read that. So either it's a scrub or formatting or both which is causing the delay. I'm certain that if the OP looks at the Flash drive activity light during this installation delay, that light will be flashing the entire time. And I know this concern has been brought up before in these forums and addressed, or maybe it was a bug ticket, I don't recall.

I think the only way to know the true answer is to look at the installation code or have one of the developers chime in.

And you cannot compare a high speed internal hard drive directly to a low speed USB device when comparing capacities. Most internal hard drives have some sort of built in cache and they are running at SATA speeds. Poor USB is so slow and a cache only exists on expensive USB devices (I do have one but I'd never use it in my FreeNAS machine, it's for financial data).

Lastly the Sandisk Cruzer fit 64gb has a slow write speed but a fast read speed (~8MB/sec write, ~20MB/sec read) based on results posted at this web site. I have a Fit 32GB and my tests connected to a USB 3.0 port on my main computing system are: Seq Read = 36MB/sec, Seq Write = 25MB/sec, Random Read = 28MB/sec, Ramdom Write = 2.3MB/sec, Random Read (4k blocks) = 4.8MB/sec, Random Write (4k blocks) = .9MB/sec. So it's not a bad little device but you must take this into perspective and if the OP used a USB 3.0 enabled port or not.

My Conclusion... This is not a problem in the sense that because it is slow that something wrong/bad is occurring, it isn't.

At the OP, for your peace of mind, run a scrub on your boot drive and see that it passes and how long a scrub takes. Or you could use a different device. 64GB is a bit large in my opinion, not worth the cost for a FreeNAS project. Also, are you using both USB sticks you listed as a mirrored boot system or just the 64GB fit? I've never used a mirrored boot system but that too would add time into the formatting/scrub process.

If anyone has any first hand knowledge using a 64GB USB flash drive for the boot device and either had the same issue or didn't experience it at all, I'd like to hear about it. I'd say most of us are running either 8 or 16 GB USB sticks though and I'd think the 16GB stick users would have seen an installation time of several minutes.

 

[cyberjock]

I'm using a 16GB USB drive, and it is *very* slow relative to using the 16GB SATA DOM I have in my Mini. The short answer is that the file data, the transactions and transaction groups (the way that metadata is created and stored) is similar to a "random write" workload, which I've said over and over is extremely poor performing when using USB drives.

So yeah, USB is gonna be slow, its not gonna get faster unless USB drives start making serious advances in the "random write" workload, and there isn't much anyone can do about it except not use USB.