Network Upgrade Questions

Status
Not open for further replies.

panther320

Cadet
Joined
Oct 25, 2017
Messages
7
Hi FreeNAS users,
So I've owned a FreeNAS nas for a few months now, with the following specs:

Code:
i5-760
8GB RAM
1x, yes, you heard me right, just one WD Caviar SE 750GB HDD
Biostar TH55HD
Integrated Gigabit LAN on my home network


As you can probably imagine from these specs, I am itching for an upgrade, since I'm tired of transferring files at less than 5 Mb/s. I've decided to look into major makeovers for my NAS setup, but I have a lot of questions that I haven't gotten satisfactory answers from Google for. The major upgrades I'm looking at are to move to 10GbE networking, switch to compact rackmount servers, and a storage upgrade (though I'm considering doing that later, I just want to focus on the networking for the time being).

I've been planning my network setup a little and you can see how I'm envisioning it here:

jcbjqr0.jpg


So here are my questions (there are many, so don't feel obligated to answer all of them. If there's anything you want to add to the discussion, or something you feel might be helpful, just reply to the post with it.) Also, not all of these will be networking, since I'm not sure where to put this post.

1. For 10GbE networking, is it better to use CX4 or SFP+ gear? I'm finding 10GbE CX4 switches to be a lot cheaper per port than SFP+ ones (http://www.ebay.com/itm/202146182153 vs http://www.ebay.com/itm/323032634858), but I'm also finding that CX4 ones typically lack normal RJ45 ports alongside them. While I have a gigabit switch right now, being able to manage all my networking in one switch would be nice. The other thing I'm noticing is that SFP+/SFP NICs and cables are a lot cheaper than their CX4 counterparts. So which one is probably better?

2. Also on the topic of switches, I was initially going to go with a 2-port card hooked up P2P with my 2 workstations in the diagram, but I decided to go with a switch for scalability. My initial concern was the price, since I had heard that they were exorbitantly expensive, so I didn't even check. But I'm finding switches like the ones I linked for under $100. Is there a catch or are these legit switches? Also, can my network be laid out like they would be if I went with P2P where I have my 10Gbe stuff on a different subnet, communicating on their own or will it interfere with the gigabit internet stuff?

3. Last question about switches. Do I need uplinks? Are they necessary?

4. My current NAS is in a Micro ATX box that is really suited better for an HTPC than a NAS, since it only has two drive bays. I've decided to move to rackmount stuff for greater storage expansion and more compactness (though noise might be an issue for me). I don't want to buy a prebuilt server, since I'm a little disappointed with the prices and options. Instead, I'm thinking of getting a Chenbro or similar 1u or 2u case with 4 or more drive bays and buying a used Supermicro 1156 motherboard, and hook up an LSI HBA to the backplane, since it'd suit my needs a lot better (most rackmount servers aren't designed for NAS use from what I see, and the ones that do are way out of my budget). Is this a good idea or should I buy prebuilt servers?

5. When I do replace my 750GB HDD with a bigger array, how should I safely transfer the ZFS datasets over? Does FreeNAS have a good way to do this?

6. On most servers, what's the purpose of the management port? How does one make use of it? Do you need specialized management consoles? People on the internet seem to be suggesting that you do. How does usage differ between RS232 and Ethernet management ports? Is IPMI related, and how do you take advantage of that? On ethernet management ports, normal and IPMI, do you just hook it up to a switch? Will that work?

7. Is SMB3 required to do 10Gbe? Is throughput reduced when SMB3 is not used? Is there a way within FreeNAS for the non-SMB3 clients I have to connect too, even if the speed isn't great, while still not limiting the throughput of modern clients?

8. Building on the SMB3 thing, in the diagram, you might see a server called "iSCSI to SMB bridge server". This is an idea I came up with to address the SMB3 thing, if it even is an issue, but I'm realizing it might not be as good of an idea as I initially thought. The idea was simply to set up iSCSI targets on the main NAS, and to mount those iSCSI shares on a separate server. Then, I would have the iSCSI shares shared on an SMB network with my retro machines using P2P gigabit connections and dirt cheap nics. But after doing some research on iSCSI, it seems like it wouldn't be possible to use my previously existing ZFS datasets (shares) as iSCSI targets without overwriting them. This obviously can't work, I need to have something available on SMB and iSCSI and I need the retro and modern clients to be able to access and write to the same stuff. Besides, it seemed a little overcomplicated and adds extra cost. Does anyone know of a different way to do this?

I think that's everything. I still have yet to budget everything out, since I'm not buying until later this summer. Everything should fall within my max restriction of about $1000 for everything excluding the hard drives if I buy used gear on ebay. I might edit the post if I come up with other questions.

Thanks in advance!
 

garm

Wizard
Joined
Aug 19, 2017
Messages
1,555
Building a small enterprise class server closet is all well and fine.. but start with getting decent hardware for your data..

The motherboard you have doesn’t support ECC memory and you have a single drive pool..

If the data on that pool is of any value, please start with adding a mirror to it. You can add a larger drive then what you have now and then replace the original drive with another one of the same capacity, expand your storage. No need to “move” any datasets.

Also, 750 GB / 10 GbE is 10 minutes..
 

panther320

Cadet
Joined
Oct 25, 2017
Messages
7
Building a small enterprise class server closet is all well and fine.. but start with getting decent hardware for your data..

The motherboard you have doesn’t support ECC memory and you have a single drive pool..

If the data on that pool is of any value, please start with adding a mirror to it. You can add a larger drive then what you have now and then replace the original drive with another one of the same capacity, expand your storage. No need to “move” any datasets.

Also, 750 GB / 10 GbE is 10 minutes..
That's sort of what I'm trying to do. I want to focus on getting everything else before I go out and buy a bunch of hard drives only to find out my current box has no way of supporting them. And the data on that one 750GB HDD is just stuff like operating system ISOs. It's not incredibly important, but it would be annoying to lose.

I'm well aware of the shitty state of my current NAS. That's exactly why I'm upgrading it.
 
Joined
Jan 18, 2017
Messages
524
1. I have CX4 and sfp+ stuff don't bother with the CX4 just get SFP+ more versatile and readily available
2. The switch you linked is capable of VLAN's which can give you a lot of room to play
3. Networking is not my thing but my switches don't give a damn I have my FreeNAS on the CX4 and my desktop on an XFP 10GBe module.
4. I moved from an IstarUSA 4U which served me well until my current Supermicro case was presented to me, frankly I'm thinking of putting hot swap enclosures in it and putting it back into service. I grabbed a used X7 board and parts and rammed it in the case and all was well, I would certainly do it again of course with newer gear though.
5. I personally would add a mirror to the single drive say a 6TB and wait for the resilver to finish (should be quick) then use the replace function to replace the 750GB with another 6TB and you'll have a 6TB pool once the resilver finishes again (assuming auto expand is enabled)
6. The management ethernet port allows you to control/monitor the computer entirely independent from the OS as if you were physically in front of the computer (I'll never setup another server for personal use without it again) Supermicro's you can use their program to connect to it or simply use a web browser that supports JAVA to connect to the IPMI's IP address.
7. SMB3 is not required to use 10GBe, clients that are 1GBe or less will still be able to connect to your FreeNAS if it is connected via 10GBe. Some clients may require you to enable SMB1 but well I haven't encountered that problem yet.
8. iSCSI I know nothing about but I have to ask..... why do you require it? depending what you're doing SMB can likely handle it.
 
Last edited:
Joined
Apr 9, 2015
Messages
1,258
1. Personally I would say SFP+ especially if you will use it with multiple devices. I honestly had to look up the other one and my guess is it will get harder to find stuff to work with it as time goes on. Bite the bullet and do it right the first time rather than doing it twice and a greater cost.

2. The switch you linked is SFP not SFP+ that is the problem with searching ebay. SFP is something like a 4Gbps max so no you don't want that. I ended up getting https://www.unixplus.com/products/quanta-lb6m-24-port-10gbe-sfp-4x-1gbe-l2-l3-switch as I know eventually I will make most of my network 10G but I can still use a regular switch and with a firmware change I can use SFP gigabit copper. Most of the SFP+ stuff I have come across can do a lot more than the average user will even think about. Mine was just plugged in and hooked up and it worked. Haven't even scrapped the surface.

3. Most of the "uplinks" I have seen are either crossover ports or faster ports dedicated to connecting switches. If you are only going from copper gigabit to 10G any port that will allow you to go from one to the other will be fine.

4. HUH.... How are most rack chassis not built for NAS use? A board goes in the same as any other case and they have power. IMHO they are better suited for NAS use as a NAS needs drives to function well and with FreeNAS they work great. It depends more on your budget and end goal than anything. The problem with rack servers is they are not as home friendly. That is it. You can however either build or buy a cabinet or case. And with a 1U or 2U easily get a bracket to wall hand. The 48 drive Chenbro to me is the best thing I could have bought. I will use it till I die or drives that are supported go the way of the dodo.

5. You can snapshot and replicate or copy. Kinda up to how you want to do it. It will also depend on what your pool ends up being in the end. I went from a single drive to a raidZ3 when I went from my test system to my production system. I just copied everything over for my data and set the rest up (jails, vm's, etc) fresh.

6. To manage remotely. And plain and simple you want IPMI unless you want to futz with another funky cable.

7. No it does not have to be SMB3. Will it be a little slower.... Maybe but going from 1GB to 10GB will be more than fast enough of a difference for you not to really even need to care. For older clients you will just have to find a solution that works and use it. Will depend on what you are dealing with.

8. Don't mess with simplicity. iSCSI requires a lot more work to setup and will come at a performance loss as the system has to deal with blocks rather than data. Just make it possible to connect with the version of SMB that the XP boxes can use. You are able to set a minimum and maximum version anyway. And it may be useful to look into dumping SMB and just use NFS network wide. It is possible to find clients that will work and not have a need to worry about the security aspects.
 
Joined
Jan 18, 2017
Messages
524
2. The switch you linked is SFP not SFP+ that is the problem with searching ebay. SFP is something like a 4Gbps max so no you don't want that. I ended up getting https://www.unixplus.com/products/quanta-lb6m-24-port-10gbe-sfp-4x-1gbe-l2-l3-switch as I know eventually I will make most of my network 10G but I can still use a regular switch and with a firmware change I can use SFP gigabit copper. Most of the SFP+ stuff I have come across can do a lot more than the average user will even think about. Mine was just plugged in and hooked up and it worked. Haven't even scrapped the surface.

Good catch I didn't notice that.
 
Joined
Apr 9, 2015
Messages
1,258
Good catch I didn't notice that.

Went through a lot of that when trying to find something cheap that would work for me. I don't do anything fancy with my network and have used unmanaged switches so far and with the ability to plug in 24 SFP modules the one I found should work perfect. Flashing it and being able to use 1Gb copper sfp's makes it even better. Right now I have an 8 port switch that the 24 port will end up replacing along with another 8 port that sits with my TV that provides ethernet for the electronics.

Honestly for what I paid for it the switch was pure gold. I think I ended up paying 300 shipped. They sell on ebay as well so if the OP pitched an offer around 225 to 250 plus shipping I bet they would bite. Brand new is hard to beat that price. They are also running a 50 dollar discount directly on their site right now if you just want to buy it outright but that dies on April 1st https://forums.servethehome.com/ind...nta-lb6m-deal-thread.17824/page-2#post-175374
 
Status
Not open for further replies.
Top