My Dream System (I think)

[Rand]

Congrats:)

Not sure why you had issues with importing your Sophos Config though?
Have done that a couple of times back and forth already and no issues at all.
Any particular issues occuring?

 

[gpsguy]

I wondered the same thing.

As long as one had the same number of NIC's or more in the new machine, it shouldn't be too much of an issue. One of the problems I've run into in the past with the software version on physical hardware has been a re-arrangement of the NIC's. On that box with 5 NIC's, first I had the find the one with the management interface and then map the old interfaces to the new ones.

Not sure why you had issues with importing your Sophos Config though?

 

[joeschmuck]

You both nailed it on the head, it was a NIC issue. My original box had three NICs a mixture of Intel and RealTek but when trying to import to the VM, now I have two NICs, a VM (E1000) and a pass-thru Intel. I will try it again to see what happens, I did only give it a shot one time and when it failed I just assumed it was not possible. I didn't fight too hard with it.

 

[jgreco]

Another reason I like the Intel ethernets, and the older em-based ones... that's actually the hardware VMware emulates.

 

[joeschmuck]

Got it figured out... I had to change my LAN port (originally eth2) to eth0 and then when I imported the backup file, I had access to Sophos and could reconfigure the WAN port to eth1. Yes, my original Sophos machine had some odd port arrangements because of my Intel NIC add-on card but all is good now. And I'm saving about 48 watts an hour by turning off the standalone Sophos machine, not that spending all that money buying a new system means I'm saving in the long run because I doubt I'll recoup those costs in electrical savings alone.

I will still retain my backup router so if I need to take the server offline, I'll still have network connectivity for the house, don't want to see all hell break loose when folks can't get to the internet.

 

[jgreco]

Yeah, the X9 stuff wasn't a problem, the PCH ports on the X9 Cougar Point and Patsburg just magically worked for passthrough, and I was very disappointed that the X10 Wellsburg stuff didn't.

Also I should point out that I meant to go get the recipe for fixing this. It's basically adding appropriate entries for the Wellsburg AHCI controller to /etc/vmware/passthru.map ... if anyone ever needs any help with this, contact me for details.

 

[joeschmuck]

There has been another ESXi update, 6.0U2. I need to really read over the changes to see if I should upgrade. Just putting it out there.

 

[jgreco]

Yeah, get used to the ~monthly ESXi update merry-go-round. A practical bit of advice: don't get too uptight about it. If a patch doesn't seem to address anything that's causing you operational grief, then it may not be worth the trouble. Kind of like FreeNAS upgrades when you need the 100% uptime... try to avoid it unless it is important.

I busted one of the hypervisors this morning by bringing it up to current patch level. We've got a precarious balance of hypervisors and control plane here, if I bring vSphere up to "current" then it can't manage the older hypervisors, and now it seems I can't bring the newer hypervisors up to current patch or vSphere can't manage /them/. Bleh. I had to roll it back, thank heavens for Shift-R.

 

[Black Ninja]

Still not particularly worrying. Here at home, we're on the far end of a utility run that basically goes through Tarzan's jungle, about two miles through backyards with fifty or sixty year old trees. We lose power once or twice a year, the A rail UPS (~10 min runtime) always discharge fully before transferring onto the B rail UPS's which are generator-backed. Since replacing them about 4 years ago they've probably been through at least 8 full cycles and they're still showing as very healthy.

I'm actually quite pleased with the new APC SMX units, though I don't know how "FreeNAS-compatible" they might be. Unlike the old APC Smart-UPS units, I haven't caught any of these doing any battery-cooking, and in fact we managed to hit the five year mark before we finally got any sort of battery warning. I had purchased a desktop SMT 750 to experiment with back in ~2010, and it finally gave it up a few months ago and needed a new battery pack. We had one other SMT unit arrive DOA but I think that may be the total extent of the bad things I can say about the new models. I was always griping about replacing batteries in the older Smart-UPS's... with several dozen units there was always one needing replacing, and usually the batteries had cooked and bulged without any alarm or warning.

If you are using generator, wouldn't you think that online ups will be more proper, like SURTA line up, instead line interactive SMX ?

 

[jgreco]

If you are using generator, wouldn't you think that online ups will be more proper, like SURTA line up, instead line interactive SMX ?

Perhaps, but that can turn into a bit of a rabbit hole. If you look at the things a double conversion power supply protects against, the ones that it handles that a line interactive unit *doesn't*:

Harmonic distortion
Electrical line noise
Frequency instability
Switching transient

In general, none of this is really a problem for properly sized gear. If you want to continue to worry about it, then the problem becomes that it would also make sense to worry about it for the primary load UPS's as well. This gets expensive quickly.

 

[Black Ninja]

I personally like the idea of a real "uninterrupted" power supply which actually is only the online ups, for the fact it won't switch to battery and back , so no switching no risk of load get dropped and the output will be always the same stable power, BUT
in your case with generator involved, I just wanted to mention that read that online ups is recommended for the following reason:
Online ups can work with much wide range of input voltages, unlike line interactive which has a smaller input range and will switch to battery back and forward when outside the range. Generators (smaller are worst) has higher voltage fluctuations and can trigger line interactive ups to keep switching back and forward, while online can take much wider range on inputs voltage.

P.S.Just thought you should know. I don't have generator yet, and mine are also line interactive, for now.

 

[jgreco]

Get a high quality generator. Gillette makes some well-regarded stuff. They use heavy duty diodes for which they provide a lifetime warranty, and heavy duty capacitors as part of their CLC voltage regulation design. They used to have a nice explanation of their "kleen-power" technology on their web site but I don't see it right now.

 

[Black Ninja]

Get a high quality generator. Gillette makes some well-regarded stuff. They use heavy duty diodes for which they provide a lifetime warranty, and heavy duty capacitors as part of their CLC voltage regulation design. They used to have a nice explanation of their "kleen-power" technology on their web site but I don't see it right now.

I wouldn't consider anything less than the best , it s felony
I took a peak at these guys you mention and their patriot statement:"All generator sets are USA wound, built, and thoroughly tested. Complete production models are USA factory built." and as long as it says honda on the pictures I could trust them ))))))

Anyways I just wanted to make sure my green friend is aware of it. And and honestly speaking I was hoping you would said that you already have online ups, cause I am considering that option but it seems we are not together on this one.

 

[jgreco]

I wouldn't consider anything less than the best , it s felony:)
I took a peak at these guys you mention and their patriot statement:"All generator sets are USA wound, built, and thoroughly tested. Complete production models are USA factory built." and as long as it says honda on the pictures I could trust them :)))))))

I believe they've always favored the Hondas because they're known as such a reliable engine. I know you used to be able to go down to Elkhart and see their manufacturing floor where they'd be happy to give a personalized tour; they're very proud of the business. I'm told that one of the owners recently bought out the remaining owners, so I don't know what that signals for production values or quality.

Anyways I just wanted to make sure my green friend is aware of it. And and honestly speaking I was hoping you would said that you already have online ups, cause I am considering that option but it seems we are not together on this one.

Just because something is "better" doesn't make it better. You don't see us recommending those "better" RAID controller cards for ZFS.

 

[Black Ninja]

Well it's different: Any raid controller is not good for zfs, but online UPS is better cause that's actually the only real "uninterrupted" power supply. Everything else just carries the name UPS but actually is directly connected to the power. Let me remind you about line interactive UPS that every time when switches to battery mode is actually cutting the power for several ms , and there is a risk the power supply will drop the load from having shorter "hold time" that the ups can make the switch.

I never had that problem (as some may say) , cause I carefully compare any PSU I buy hold time with UPS specs for switch time, but the "click" always made me nervous to this day

P.S.I use to disable a UPS self-test for that reason.

 

[jgreco]

Well it's different: Any raid controller is not good for zfs, but online UPS is better cause that's actually the only real "uninterrupted" power supply. Everything else just carries the name UPS but actually is directly connected to the power. Let me remind you about line interactive UPS that every time when switches to battery mode is actually cutting the power for several ms , and there is a risk the power supply will drop the load from having shorter "hold time" that the ups can make the switch.

I'd rather be aware that there's an extremely marginal PSU in play. If you look at the way I advise people to size a UPS, you might notice that there are very large safety margins in there. A PSU should be able to hold its own for several milliseconds.

P.S.I use to disable a UPS self-test for that reason.

That's the equivalent of closing your eyes, covering your ears, and yelling "la la la" at the top of your voice.

Some of us actually expect our gear to work in the real world. So when you run a generator test every two weeks and let the automatic transfer switch cut from A to B, there's this very interesting several seconds where it is transiting from A to B and not actually powered. Good test for all the gear.

 

[Black Ninja]

A PSU should be able to hold its own for several milliseconds.

That's right. And it's even part of atx spec (15ms) , but consumer gear not always comply with it. Even from the same manufacture lower end products ( like this guy)may not comply with atx specs while higher line product does. I even have few of those low end PSU in question purchase before I was aware, they are not in use anymore of course.

Some of us actually expect our gear to work in the real world. So when you run a generator test every two weeks and let the automatic transfer switch cut from A to B, there's this very interesting several seconds where it is transiting from A to B and not actually powered. Good test for all the gear.

I see what you are saying. I don't have a generator but I already have a automatic transfer switch (APC model) which I am going to get when get generator one day. This switching gap between A and B on transfer switch I would worry because it will not cause your servers to drop their load, since you'll have a ups in between to pick up that interruption.

By the way I did enable the UPS test recently feeling little bit more comfortable knowing that I have dual PSU on most of my servers so if one drop the load hopeful other wont.

I would assume in enterprise environment they would use online ups , this is something you would know better I am only assuming ?

P.S. By the way Linus tech tips got a new shiny ups's and they are online type

 

[jgreco]

I see what you are saying. I don't have a generator but I already have a automatic transfer switch (APC model) which I am going to get when get generator one day. This switching gap between A and B on transfer switch I would worry because it will not cause your servers to drop their load, since you'll have a ups in between to pick up that interruption.

Yeah, I'm not talking that little stuff. I'm talking something a little bit bigger...

http://www.aaronequipment.com/usede...switches/cummins-onan-oncu3000-c-23h-42105002

That's actually even bigger than what I'm talking about, but same order of magnitude. The thing is, when this thing flips from A to B, you're talking contacts rated in the thousand amp range.

By the way I did enable the UPS test recently feeling little bit more comfortable knowing that I have dual PSU on most of my servers so if one drop the load hopeful other wont.

I would assume in enterprise environment they would use online ups , this is something you would know better I am only assuming ?

Assume what you want.

In a large data center, it is incredibly inefficient to implement something that's the equivalent of an "online" or "double conversion" UPS, because you're talking massive energy, massive heat, and massive costs thrown into the double conversion. You have to remember that a data center is often dealing with megawatts of power. The data center we're in has 14 megawatts of backup generator capacity. You can roast a lot of hot dogs with that.

Most major data centers use a combination of static transfer switches (silicon based) and very careful monitoring of power quality in order to be able to very quickly move load from raw utility power to some form of redundancy for the UPS's.

This document talks a little bit about the various topologies. Note that it is *ONLY* discussing the section of the problem between the UPS and the load.

This document talks about automatic transfer switches versus static transfer switches.

This document talks about UPS design strategies to achieve efficiency.

I particularly love this quote: "Typically incorporated in the double-conversion UPSs that are the first choice for almost all critical data center applications, this functionality works by, in effect, feeding the load direct from the mains supply when this supply is within tolerance and problem free, but switching to normal double-conversion within two milliseconds of a mains supply problem occurring"

That's literally the definition of a standby UPS.

So what's really going on is that a large data center quite likely feeds your PDU from utility power under normal conditions, and steps in with UPS-provided power as needed.

P.S. By the way Linus tech tips got a new shiny ups's and they are online type :)

Good, now you know what stupid does.

 

[Black Ninja]

Well I don't know why so big. My impression was that you were talking about small residential generator for a home in 200amp range. Unless you have a something very big that take 3000amps, but that's out of my area of knowledge. I was thinking really small like this which can feed the whole house of course but can add ups to it which can provide a buffer till generator kicks in and stabilizes.

Assume what you want.

I don't like to assume , but I am excellent assumer.


I am gonna have to check check the links you provide, very interesting stuff by the way. I guess by datacenter I didn't mean a actual datacenter, but in something smaller like business environment with several racks in a server room. Perhaps that's what's more accurate of what I had in mind.

"Typically incorporated in the double-conversion UPSs that are the first choice for almost all critical data center applications, this functionality works by, in effect, feeding the load direct from the mains supply when this supply is within tolerance and problem free, but switching to normal double-conversion within two milliseconds of a mains supply problem occurring"

I am not sure where is this quote from ,but this won't be the online UPS I understood: The idea is that it run from batteries the whole the time and AC from outside is charging the batteries in a mean time , so it will never have to switch , or even change it's output voltage regardless if AC source is removed or not. Otherwise what is the point of the double conversion ? It will be no different than stand by UPS (line interactive) like you said if this quote is true?! If is not feed by the batteries all the time and therefore avoid any switching transfers, there is not much a advantage over line interactive UPS.

P.S. Main disadvantage I see in online ups is the lower efficiency because of the double conversion - around 82% compared to line interactive which is easily get around 97-98%

 

[jgreco]

I am gonna have to check check the links you provide, very interesting stuff by the way. I guess by datacenter I didn't mean a actual datacenter, but in something smaller like business environment with several racks in a server room. Perhaps that's what's more accurate of what I had in mind.

The small units seem to be online double conversion without any sort of "eco" mode, but you can see "eco" modes pop up at moderate sizes:
http://www.apc.com/shop/us/en/produ...ance-Bypass-and-Distribution/P-SY200K250DR-PD
100kW

100kW is suitable for maybe a 16 rack deployment, depending on power density.

I am not sure where is this quote from ,but this won't be the online UPS I understood: The idea is that it run from batteries the whole the time and AC from outside is charging the batteries in a mean time , so it will never have to switch , or even change it's output voltage regardless if AC source is removed or not. Otherwise what is the point of the double conversion ? It will be no different than stand by UPS (line interactive) like you said if this quote is true?! If is not feed by the batteries all the time and therefore avoid any switching transfers, there is not much a advantage over line interactive UPS.

The quote's from one of the linked documents. You appear to understand the irony. That was my *point*. Double conversion is so ridiculously inefficient that it eventually gets to a point where it is kinda just a lie.

P.S. Main disadvantage I see in online ups is the lower efficiency because of the double conversion - around 82% compared to line interactive which is easily get around 97-98%

As you say, the whole problem with the double-conversion stuff is the relatively low efficiency of the process. That's a huge financial incentive when you're dealing with megawatt sized deployments because that ~20% inefficiency is both electric cost and also mostly converted to heat, which then incurs additional cost for cooling. Its like turning on an air conditioner to deal with the space heater you refuse to turn off.