Comments on: FreeNAS QA – Our Processes https://www.truenas.com/blog/freenas-qa-our-processes/ Mon, 07 Apr 2025 16:28:29 +0000 hourly 1 https://wordpress.org/?v=6.7.2 By: Joon Lee https://www.truenas.com/blog/freenas-qa-our-processes/#comment-5382 Thu, 09 Nov 2017 00:39:16 +0000 http://web.freenas.org/?p=3771#comment-5382 In reply to John Brown.

We understand your concern. However, automated QA has been extremely helpful with a myriad of issues. It’s even allowed us to fix issues before they became a problem.

]]> By: John Brown https://www.truenas.com/blog/freenas-qa-our-processes/#comment-5381 Thu, 25 May 2017 14:12:34 +0000 http://web.freenas.org/?p=3771#comment-5381 While perhaps “neat”, I’m skeptical that automated QA testing is a good approach. Are there any numbers indicating how many (if any) bugs were caught by an automated test after it was set up?
It’s true that an update in one place can create a bug somewhere else. However, the vast majority of those cases are predictable based on the context of the update. Setting up automated testing to be run into perpetuity creates over-head that could be spent on other efforts. Worse, it creates a false sense of security that because a certain area of the system has some automated tests that there won’t be some bug there that isn’t detected by any test.
I don’t see where it has been established that this approach is more effective than old fashioned, direct QA. You identify bugs, check twice based on update context, fix them and move on. What is broken about that? How does doing extra work to automate a test over and over in an area that you can see working help?

]]>