pfSense vs. OPNSense?

[ornias]

I find it most enlightening how the PFsense fanclub, tries to blame things on the FreeBSD project while:
A. They totally botched their own review too
B. They clearly didn't contact anyone with the wireguard team
C. Released it before FreeBSD (which also means not all pre-release testing/review has been completed)
D. It was their damn contractor in the freaking first place!?

I don't get how the PFsense folks think FreeBSD was wrong here, it never made a release and was patched-up and removed before the release. That seems like fine open-source peer-review process to me.
Yes they merged it into their dev-tree too soon which was quickly corrected.

It's not unusual in opensource for something to get merged into the dev-tree and removed before release. It's not best-practice, but it happens.

 

[HolyK]

@ornias Yea i (partially) agree. But I don't think either of the party did anything really *that* wrong at the beginning. Yes pfSense released their custom code (contracted or not, does not matter), if it was crap or not does not matter much either. Then they pulled it back (for whatever reason, again not that important).
In my eyes all of that is perfectly fine! I am working with SAP for more than a decade and witnessed pulled-back releases of kernels/components/etc several times because of various reasons (and they have wayyyyyy more complex lifecycle/QA than pfSense). On the other hand i never saw a community shi*storm like the one around pfSense. Reason is simple, there is no "free/community" SAP release for productive usage.

What went wrong is the whole public back and forth accusation of who did something wrong and who is attacking who. I will not repeat of what someone else said on reddit dozen times ... it would be my personal opinion anyway and i don't want to pick sides.

I was shaking my head about how much of the unnecessary hate/craziness the (mostly) Reddit community is capable to generate. I saw a comment somewhere stating that the most of the complains are coming from "free" users. Enerprise/paying customers aren't part of this and they're just sitting back and chilling. The guy said that the technical support quality and response is good.

I am (still) running pfsense 2.4.5 and i have no need to rush for upgrade. I know there are (were) some OVPN issues and with the apparently degraded PPPoE speeds. Wireguerd would be interesting thing to try but i can live w/o that. I am happy with my current stable setup and i have no reason to migrate to something else (OPNSense or other platforms) just because someone did a very bad PR job (eg: andgy dev taking the "bait" in maillist and releasing official blogpost)...

//EDIT: one more thing ... bunch of the ppl complained that they upgraded pfS, switched to WG and now they have to re-do the OVPN setup because WG was removed, so they're angry at pfS .... well all of this is on THEM, not on pfsense! Rule number one says Never ever jump on .0 release on prod env! NEVER! (otherwise expect issues, rollbacks, data corruption/loss so restores form backup, etc...). I hope they learned their lesson here...

 

[ornias]

The contrast is chilling though:
The FreeBSD community took their responsibility and started fixing and retracting code.

Whereas the pfSense community (and management, as is the usual with those folks) started a flamewar when confronted with their failure.

 

[danb35]

bunch of the ppl complained that they upgraded pfS, switched to WG and now they have to re-do the OVPN setup because WG was removed, so they're angry at pfS

Well, it sure doesn't do much for my argument up-thread that less-frequent releases tend to result in more stable code...

 

[HolyK]

Well, it sure doesn't do much for my argument up-thread that less-frequent releases tend to result in more stable code...

Well i would say your argument (i agree with you) is still valid if you add the condition to NOT install point-zero releases. They released 2.5.0 (feature release) - skip that one and wait till 2.5.1. You will essentially skip most of the issues related to new features.

I would say iXsystems did it right with the proactive statement that 12.0.U1 is the "Mission critical" release. If you want to minimize the risk just skip the .0 release and go with U1. Same rule applied here. I am glad iX is aware about this (yes, they learned their lesson as well) so we're happy users :]

 

[danb35]

I would say iXsystems did it right with the proactive statement that 12.0.U1 is the "Mission critical" release.

...except that U1 had a major data integrity bug. "Don't use x.0" really isn't helpful advice, in that (1) it tends to let vendors off the hook for releasing crap code; (2) if followed, there would never be a x.1 release (or x.0.1, or whatever other versioning scheme they had); and (3) it breeds excessive confidence that x.1 (or whatever) will be "safe".

 

[HolyK]

Thanks for the links, especially to the two summary posts at the top. I'm not on reddit, so much appreciated.

Well i just copy/pasted few links, the heavy lifting was done by the authors :D
I am not a heavy "redditor", I usually just scroll by and read few posts which looks interesting but this pfS / WG thing was all over the place. Also i found that sometimes it is very easy way to get a quick answer on something specific. Some subredits are quite interresting and have knowledgeable ppl in their communities. Since we're on NAS forum I'll just mention /r/DataHoarder ... :]

@danb35
I was expecting such comment :] . Yes, there was a bug but it was a very specific bug under a very specific circumstances not a pile of serious bugs in a matter of "we just broke things which worked in .0" (or at least i did not had that feeling) on the other hand looking at the Jira they fixed several of issues affecting broad audience in U1.
Also I never said "to eliminate all issues" but "to minimize" , that's a huge difference!. I am more like to wait few weeks/months to get 0.1 (or even 0.2 if i don't like the 0.1 list of open bugs) and handle few bugs (affecting my use case) rather than charging towards .0 a minute after it is released and then complaining that half of my functions are not working ... some people just thinks the "RELEASE" is rock-solid and nothing can go wrong. They skip proper testing, backups, they don't read bugreports/open issues, they update and then go berserk ... some of them learn their lesson, some will do that again next time ...

I will not argue with you because you might have very different experiences in completely another area than i do so we both could be right just because "we've been there, done that". I will just stand by my statement that the feature-rich versions (.0 releases in general but it depends on the versioning schema ofc...) tends to bring much more issues then the "bugfixing" releases. That is what i learned over the years within my area of work. Anyway that does not mean that some new issues slips by in later releases of SW (been there as well, it ended up with complete system restore)

So back to the topic ... i see you had pfSense but moved to OPNSense (and had issues). Are you staying with OPNS or thinking about move back to pfS ?

 

[Patrick M. Hausen]

Just found a thorough and well balanced article on Ars:

Buffer overruns, license violations, and bad code: FreeBSD 13’s close call

40,000 lines of flawed code almost made it into FreeBSD's kernel—we examine how.

arstechnica.com

 

[danb35]

The more I read about this, the worse everyone looks--the WG team probably least bad, but still...

 

[danb35]

So back to the topic ... i see you had pfSense but moved to OPNSense (and had issues). Are you staying with OPNS or thinking about move back to pfS ?

I had moved back to pfSense when an OPNsense update semi-bricked my router ("semi-"bricked in that it was fully recoverable, but until I reloaded the OS--which was pfSense at that point; it was the last straw for OPNsense--it was completely non-functional). I'd also found OPNsense's support the worst I've encountered to date in F/OSS, as described in a bit more detail up-thread. But this and other recent developments (also discussed here) don't make staying with pfSense look too attractive either.

No firm decisions right now--as I say, staying with pfSense in the long term doesn't seem too attractive, for a variety of reasons (most of which are discussed here). I'm kind of soured on OPNsense, but might give it a try with just configuring it, from scratch, by hand (when I'd tried it previously, I'd tried importing portions of my pfSense config; maybe that messed things up in a non-obvious way). As time goes on, though, my requirements increase--I now need multi-WAN with failover, reverse proxy, multiple isolated LAN networks, etc., that I wasn't using when I first looked into this. The reverse proxy could be handled by port forwarding to an internal VM or jail, I guess, but it's nice having that on the router.

 

[Newfoundland.Republic]

pfSense's (and FreeBSD's, for that matter) response and approach does cause me some pause. That said, I do not see this as hopeless. Given the publicity of the, err..., situation, it is likely to cause a change in approach for the better.

Of course, it is more than possible to upgefukct the outcome.

 

[danb35]

pfSense's (and FreeBSD's, for that matter) response and approach does cause me some pause.

FreeBSD's response is promising, but vague--they agree there's a problem, they promise they'll do better, but pretty light on details of what they'll do to make it better (which, come to think of it, reminds me of iX after the disaster of The Release That Must Not Be Named--with which response I was far from satisfied, though they seem to have made improvements since then). They accepted bad code, but they weren't the origin of that code. Netgate's response, IMO, is much more troublesome, mainly in that they seem to be blaming everyone else for failing to stop their bad code, while being unwilling to accept the responsibility of being the source of that bad code.

 

[Jailer]

Netgate's response, IMO, is much more troublesome, mainly in that they seem to be blaming everyone else for failing to stop their bad code, while being unwilling to accept the responsibility of being the source of that bad code.

That type of flippant attitude has been pervasive at netgate for a while. They don't even try to hide it most times. It's really making me want to bite the bullet and go through the trouble of switching to OPNsense. I don't have an extremely complex setup but it would take some time to reproduce it.

 

[Patrick M. Hausen]

Well, I've made my bed:

GitHub - punktDe/vagrant-opnsense: Bootstrap an OPNsense development environment in Vagrant

Bootstrap an OPNsense development environment in Vagrant - punktDe/vagrant-opnsense

github.com

Development environment ready, finally working on the BIND plugin.

 

[ornias]

reminds me of iX after the disaster of The Release That Must Not Be Named--with which response I was far from satisfied, though they seem to have made improvements since then).

The difference being: iX made a severe design mistake and had the balls to trash it based on community feedback, though that said release would've been possible and technically solid with a little more time and effort.

FreeBSD just butched a review so terribly that we need to ask ourselves how many backdoors got leaked in the past decade.

 

[Constantin]

That's exactly it. The issue is not that a allegedly "half-hearted" programmer was hired to deliver 40,000 lines of code to the FreeBSD 13 kernel, it's that no rigorous process / review team existed to review it before allowing said code to be merged into the kernel. That this happened with critical crypto-related software is even more puzzling given FreeBSDs alleged focus on security.

Worse, the article / timeline suggests that the discovery was by pure happenstance, that Netgate had already started using this crummy code in production hardware, etc. suggesting that Netgate was also sorely deficient re: code review. Perhaps netgate reacted this strongly because it is so darn embarrassing to have hired the wrong resource, accepted crummy code, and then incorporated said code into current product - especially for a company that allegedly specializes in this stuff.

I hope it leads to a change at FreeBSD re: contributions to the code base. But that will take time and money, something that the more permissive licensing at FreeBSD likely doesn't produce in abundance. As for Netgate, well, I wouldn't use their stuff unless they come clean, apologize to their user base, and do a internal review of their code to look for the kinds of backdoor-opportunities that allegedly riddled this part of their production software.

 

[Etorix]

Just found a thorough and well balanced article on Ars:

Buffer overruns, license violations, and bad code: FreeBSD 13’s close call

40,000 lines of flawed code almost made it into FreeBSD's kernel—we examine how.

arstechnica.com

Appalling, but very interesting reading. Thanks!

 

[Oriann]

I have set up OPNsense from scratch half a year ago and till now without any error, updates are rolling fine. I have no interest in getting back to pfSense since their update policy is slow as hell. And as discussed early wireguard implementations problems and incomprehension.

 

[danb35]

I have no interest in getting back to pfSense since their update policy is slow as hell.

So what? I've heard this complaint before, but I really don't understand it. Why do you care that your firewall doesn't have updates very often for the core system? Packages (which includes pretty much everything public-facing) update much more frequently, of course, but how often does the core of the firewall need to update?

I have plenty of concerns with pfSense (most, if not all, of which are discussed in this thread), but "infrequent updates" isn't even on my radar.

 

[Constantin]

Odd then that if the update policy is slow (potentially suggesting a careful review process) that all that bad code made it into their main code base.

that’s the real issue for me. Netgate sponsored this work, they accepted it, etc. 40,000 lines of code without competent review. If true, this level of incompetence makes Netscape navigator look good.