My Dream System (I think)

[Black Ninja]

Here is a screen shot of how I track the power usage on my networking system. The only reason I purchased this mFi was because I could remotely turn power on and off to an outlet, the measurements are a bonus. I haven;t changed the labels yet but the FreeNAS is actually my ESXi server which runs FreeNAS and Sophos (2 instances) and Ubuntu, etc... I have turned off Ubuntu and tried to mitigate any changes to ESXi, meaning running any additional VMs, just to see what the power difference is having Ubuntu off for the entire month. I run BOINC on Ubuntu.
View attachment 14269

This is little bit more advanced compared to kill a watt. I think it looks cool , and most important very easy to deploy and move around. I may have to get one to play with.

 

[Black Ninja]

By the way, is this a stand alone device and connect wireless to any wifi network , or it's part of Ubiquity system or Ubiquity AP you must have to use it ?

 

[Dice]

@joeschmuck thnx. I found similar products (though also including temperature measuring and graphs) selling at a local power provider for the US equivalent of about $120.
I'll admit you've got me tempted :D

Well ups can monitor the total usage if we are talking about smart UPS here , but it's not really usefully for that.

For what reason is the power readings off a smart UPS not useful?

 

[Black Ninja]

For what reason is the power readings off a smart UPS not useful?

Because UPS can show you a moment values and even that is not very accurate , but it doesn't have to be since it's not it's purpose. Yes it can log every 5 mins or 5 hours, but this is just a moment reading so you can see where your load is as far as not overloading the ups. It can't show you accumulate power consumption for a minute or an hour or a day.

 

[joeschmuck]

The mFi I have is WiFi connected but because my system sits in the basement, the signal isn't 100% reliable. There is one that is hard wired which is what I was hoping to get but that didn't happen. But it works and I believe @cyberjock turned me on to this product.

 

[joeschmuck]

Update on my system configuration... I removed the PERC H310 RAID card, it was giving me some grief. I am now running a single SSD for ESXi vice the mirrored SSDs. I have temporarily moved my Sophos Firewall to another ESXi machine while I'm reworking my main ESXi server. So within another week I will likely have this setup normally again. I need to find out if one of the M4 SSDs will be transplanted into my daughters notebook. I need a T5 torx bit and of course I have screwdrivers in T6 and up, just not the T5. I may need to buy a thinner SSD for her but we will see.

I'm not sure if I will replace the PERC card with another card, it may have to wait until after Christmas. In the meantime I am looking for a small computer for Sophos. I do actually like running it on a separate piece of hardware just to keep the internet up and running when I'm tinkering around.

 

[Black Ninja]

I'm not sure if I will replace the PERC card with another card, it may have to wait until after Christmas.

If you want redundancy that's the only way. I would.

I do actually like running it on a separate piece of hardware just to keep the internet up and running when I'm tinkering around.

Look at the hardware that netgate provides for pfsense. You don't have to buy from them , but you can find great super low power tested hardware for firewals/routers. And then you can buy your own perhaps cheaper. I build my pfsense box based on something they offer but I made some improvements.

 

[joeschmuck]

I build my pfsense box based on something they offer but I made some improvements.

I may look into pfsense for the heck of it. I can always play around. Sophos isn't the easiest to configure.

 

[Black Ninja]

I may look into pfsense for the heck of it. I can always play around. Sophos isn't the easiest to configure.

Pfsense is better and more powerful, but my point is the hardware part. Whatever is good for pfsense as router/firewall will work for your sophos too , cause they serve very similar purpose. Use the info to get a nice low power hardware that was already tested and you'll know will work and you'll know what speeds (WAN/VPN) is capable of for your needs.


P.S.I only have 75 Mbits DOWN /10Mbit UP internet , but I know my hardware will be capable of 1Gb internet when it comes, in fact it should be able to do 2-3Gb.

 

[Dice]

I'm not sure if I will replace the PERC card with another card, it may have to wait until after Christmas. In the meantime I am looking for a small computer for Sophos. I do actually like running it on a separate piece of hardware just to keep the internet up and running when I'm tinkering around.

I was also into the idea of an AiO, but as soon as internet would die anytime the server was <tinkered with>..
I ended up getting another box for bare metal pfsense purposes. Quickly it got converted successfully to ESXi, with pfSense on top. With internet-critical components it is a huge relief and very comforting to have a separated machine, in particular on ESXi so that no worries occur with different "exported configs" (I don't trust those on pfsense half as much as I do on freenas - it might be something I've read a while ago, might not be valid. But it is the basis on how I ended up on ESXi).

At the moment I've structured a few layers of backup solutions to my pfSense.
I've saved a functioning pfsense VM that is basic without any extras going, loaded onto ESXi as a sort of "get me online NOW" backup to tinkering mishaps (during which you'd no longer be able to google :P). This one is not operational typically. It is not a "last known working configuration" but a rather downscaled version in terms of what I need to get my stuff up and working. A fail safe mode if you like.
On top of that safety net I've a running version with some configurations, firewall, port forwardings etc to suit the chosen functionality of the appliances on the network.
The last and most experimental of the VM's is still very much a work in progress. It is aimed to boast all sorts of fancy functionalities I can get out of pfSense (sort of).. including two way VPN's, squid etc..
I'm not there yet. I'm in no hurry...

Hope you'll find some ideas on an approach to structure the 'safety net' around your router solution software and ESXi. Let me hear your thoughts.

Cheers.

 

[joeschmuck]

I like your approach using multiple VMs on ESXi. I never thought of it that way but now I do. I may just retain the AMD system I have for that ESXi and firewall solution and buy a new small case for it all to fit into. Right now it's just my test rig with a single 120GB SSD which is more than enough to handle it. Also I may purchase a more efficient power supply.

 

[Black Ninja]

Decisions, decisions..

 

[bashywash]

I've been on this thread for about an hour and was really looking to build a similar system. I have one question I still have not been able to figure out the answer to.

Instead of going with the Supermicro X11SSM-F and SI-PEX40062 port SATA PCI-E Cards with pass through, I was looking at the Supermicro X11SSL-CF and passing the LSI 3008 in IT mode. However this is a C232 board and lacks some virtualization vt- features of the C236 board. Any opinions on running into issues with the C232 board?

thanks

 

[joeschmuck]

I've been on this thread for about an hour and was really looking to build a similar system. I have one question I still have not been able to figure out the answer to.

Instead of going with the Supermicro X11SSM-F and SI-PEX40062 port SATA PCI-E Cards with pass through, I was looking at the Supermicro X11SSL-CF and passing the LSI 3008 in IT mode. However this is a C232 board and lacks some virtualization vt- features of the C236 board. Any opinions on running into issues with the C232 board?

thanks

Well I hope that this thread was a good read for you, it was fun writing it.

Have you read this thread about the Supermicro X11 boards?

I know that if I could do it over again, I'd get the X11SSL-CF just so I could do exactly what you are suggesting. I would honestly do a search on the internet for "X11SSL-CF esxi sas3 passthough" or something similar. Find out what can be done.

If I were to purchase the SATA PCI-E cards over again, I'd rather have a single 8 port card than two 4 port cards but I already had one of the cards before I started so what's a man to do. What I do like about these cards is they don't produce much heat like a RAID card does, and they do not require any firmware changes, they just work.

As for the differences in vt-d features, I didn't see that but I have been on a computer all day and my eyes are bugging out. Both support vt-d. There are some other management features like Intel vPro Technology but that shouldn't be an issue for a home use ESXi box. I doub't it would be an issue for an office either as I would imagine you would have to find products which take advantage of those features.

So my advice stands, Google it, find out if you can make the X11SSL-CF LSI-3008 controller into IT mode. Just remember that this is SAS3 and you will need to use the proper software (sas3flash) to reprogram it into IT mode, you can't use SAS2 (sas2flash).

Hope that helps some.

-Mark

 

[Sander Jansen]

I dont know if this is the right place to ask this but can someone help me with my setup, i have ESXi running and i have freenas in a VM but all im getting (over time of a few days) is mpt0 request timeouts (post here: https://forums.freenas.org/index.php?threads/mpt0-request-xxxx-timed-out-for-ccb.49771/ ) and so far im not abele to fix it.

Sadly my board does not have support for vt-d so direct passthrough is not possible but it should be possible to get things running? it runs fine for 24+ hours before things go south... if someone has experience with freenas in a vm and can help me that would be great... i have tried a lot of things but so far not successful...

 

[joeschmuck]

Sadly my board does not have support for vt-d so direct passthrough is not possible but it should be possible to get things running?

I don't think so.

Now please understand that I'm not an ESXi expert here but I'll toss you what I know.

First is that the drives need to be passed directly to the VM, ESXi cannot be part of the path. If your system does not support vt-d then I suspect you are runing hardware which will not properly support what you desire and you are putting your data at high risk. While I understand what you are attempting to do, I'm not 100% certain the RAW Disk Mapping works perfectly. Also, are all your drives hooked up to the motherboard directly or going through a RAID controller? I would ensure that the six drives you have identified for FreeNAS are on the motherboard controller. Also have you upgraded to the latest firmware for the Marvel controllers? I already posted in the other thread that since you have 15 drives and only 12 SATA ports, I suspect you are using an additional RAID card and it might not be flashed into IT mode.

Right now I would try to pass through both Marvel controllers (that would be 6 ports) and if that works then you might be able to do this. I have yet to see an RDM pass through work properly for FreeNAS.

 

[Black Ninja]

It's been a year since the last word was said here. I hope joeshmuck is still fine and kicking.

Should I write here , or should I start new post when I want to discuss something related to this discussion ?!

 

[Mirfster]

It's been a year since the last word was said here. I hope joeshmuck is still fine and kicking. :)

Should I write here , or should I start new post when I want to discuss something related to this discussion ?!

I'd vote for a new thread, this one is huuuuge... ;)

 

[Black Ninja]

Perhaps you are right.

You've been right before

 

[joeschmuck]

Yea, this is a pretty large thread and there has been a lot of other discussion on similar topics. I can say that I'm very pleased with my final outcome and while I have made some changes recently, overall everything remains the same.

So if you start a new thread then you will likely be able to obtain some good fast information. Right now I'm enjoying the snow in Atlantic City. Not what we expected for a vacation but it will definitely help us remember the trip.