[TRENCHES] Tuesday, December 4, 2012 - Black Box
LegionPerseus VeilRegistered User, Moderator, Penny Arcade Staff, Vanilla Staff vanilla
Black Box
http://trenchescomic.com/comic/post/black-box
Imaginary authority
AnonymousI was part of a team that started out with five people and quickly grew to twelve. We built and maintained a casino-like site (meaning we handled peoples actual money) which had some domestic success and
subscribers started rising.Eventually we needed a new level of QA, bugs were slipping through that we simply couldn’t allow so the team was re-organized to meet the new plan.
The more hardcore programmers were set to do only programming, we hired dedicated server-personnel instead of relying on a hosting-service, and I was put in charge of the newly formed QA-team.
I was given new responsibilities and new orders, I was told that I had the final sign-off before anything was to be published. Nothing was leaving the production environment until my signature was on a document stating that the new release had been tested and the remaining bugs had been accepted to be fixed in the next release. We also set up a schedule which stated that a new release had to be signed of at a certain date and uploaded no earlier than 24 hours after said deadline.
The day before I was supposed to either sign off or reject the first scheduled release I already saw that there was no way this version was finished, we couldn’t allow it to go live. So I called my superiors to inform them what was going on and tell them that we needed to reschedule.
“Oh, that release? I had the programmers push that version live last night.”
Later on the bugs came back to bite us in the ass and I got the blame, naturally.
Posts
As I (and the anonymous contributor) have learned, you need to make sure your release management process is followed. If someone breaks the process, you raise it up the chain as a huge deal.
Sounds like a small operation, so "up the chain" probably means "to the guy in charge", who may have been the one who did this thing.... But at the least if you raise a huge stink you have a bigger "I told you so" card to play when they go to assign blame.
You mean development environment, Anonymous?
... That's an odd thing to pick at. Chalk it up to semantics.
Yeah, this.
QA needs to be in a sign-off position. That's the whole point of having QA.
They don't reach the conclusions in a vacuum, and need to have inputs from basically everyone so that they can make a proper decision- If this bug goes live, what is the impact? Are there workarounds? Who is impacted? If there are business users, are they aware and are they okay with the bug going live?
It's not the job of programmers to decide whether code goes into production- their frame of reference doesn't allow for them to make a clear and unbiased decision. They do of course have a part to play in the process with coding practices and unit testing.
It is not the job of project managers to decide if code goes into production- they are concerned with timelines and costs instead of the impact on end users.
You want to have someone impartial whose ONLY job is to find any issues with the release and to make sure that those issues are identified and can be worked around. That someone is always QA, assuming you are large enough to have a separate QA entity. It's accepted best practice, and it's like that for a very good reason- doing it other ways is risky, and you don't want to get burned.
Twitch: KoopahTroopah - Steam: Koopah
Most folks indeed, but clearly not all. I work for one of the big players in the videogame industry, and most triple-A games in the studio, while they are in production, are referred to as being in Prod and not Dev. Yes, this is far from being standard... And yes, this is causing confusion among the newcomers.... But no, not all software development companies follow the exact guideline of "Dev" VS "Prod".
My two cents!
And I couldn't disagree with you more. QA's job title of "quality assurance" is unfortunate, because our real job is Quality Investigators. We find issues with the product and report them. Whether or not they should be fixed is a business decision that should be made by business stakeholders with input from testers, developers, support, etc.
The input of test is valuable in determining whether a bug should be fixed. This is why bug triage teams I have been part of include test, dev and product management representation at minimum.
It is the role of business stakeholders to decide when to ship software, at least in organizations that are expected to make money. Software is a business, and releasing software is a business decision. The fact that testers are not truthfully in a position to stop it (nor should we be) is what makes "sign-off" absurd. Our role is to provide our objective assessment on the quality of the product based on the evidence we have gathered and explain what issues users will face if the software were to ship. That needs to be taken into context alongside the business needs to make a call about whether to release or not.
As for "best practice", none of the testing industry groups I have been involved with would consider tester ownership of the release process to be positive. If you want to control the release timeline, be a product manager.
@nwfn
I'm...not sure where you think I said that QA is in charge of the release process, or that that would be a best practice.
I also stated very clearly that QA works with basically everyone to get the facts together.
If a defect is accepted for release into production (usually by a change approval authority), great. If it is not, QA puts up a big red sign that says "don't release this". There will always be situations where QA doesn't have time to be involved in a meaningful way ("the site is down and we need to get it up now! Do whatever it takes!"), but if you are saying from a best practices perspective that the Release Management process should not listen to QA and release something without their approval outside of very rare and extreme circumstances, I don't want to be anywhere near your testing industry groups :-/