Tag Archives: QA

Webinar: Mastering Interactive Development Lingo

Due to unfamiliar and sometimes confusing  language, interactive projects can end up costing more or take longer to complete than they were supposed to. One of the primary contributors to project delay is a result of what we like to call the “Miscommunication Tax.” In this webinar we took the opportunity to talk through some of the lingo that consistently lends itself to miscommunications during a development project. If you are selling interactive projects, working on interactive projects, or would like to hear what a user experience designer, a developer, and quality assurance engineer have to say about confusing terms and how to create clarity around them, you’ll want to tune in.

Bonus Content:

Filed under Nerdery Webinar

QA Myths: “I don’t need to test because QA will catch it.”

Having a dedicated group of testers can be an invaluable resource when you’re a developer. But having testers (or maybe even a full-blown quality assurance department) doesn’t mean that a developer should avoid executing tests. Developers should understand how to most effectively use their own time, as well as the time of the testers.

Unit tests are useful and so are “sanity checks”

There are plenty of other people discussing how – from a developer perspective – unit testing is as important as “going to the gym” so I won’t belabour the point. But as a quality assurance (QA) professional I have experienced a marked difference between code that has extensive unit tests and code that does not. That said, I’ve also noticed improved quality amongst developers that do functional sanity checks. These checks became such a good indication of a project’s readiness for QA that we instituted a series of quality gateways here at The Nerdery to encourage this behavior.

So, what’s a “functional sanity check?” Short answer: Use your application/code/system as if you were a user. This can be as simple as loading up a website in Internet Explorer 8 to see if it looks remotely close to the way it does in Chrome. This could also be a meeting of the development team where someone drives through the application and everyone else has the chance to catch, triage, and assign defects as the team discovers them. Performing the most basic functional integration tests eliminates the easy-to-find issues.

“Why bother?”

As a project prepares to enter its QA phase, I’ve heard some developers say “why should I bother finding all these tickets when QA is just going to test everything?” This attitude is dangerous because at first blush it seems to make sense. No one wants to duplicate effort or make a project drag on longer than necessary. But in reality, projects are neither faster nor cheaper when all test activities are left to QA.

I’ve seen these projects slow to a crawl in their QA cycle. Their issue databases become quickly bloated with issues that are low-hanging fruit and – with the deadline quickly approaching – the time that could have been spent on final polish is instead spent on making sure the whole project doesn’t fall apart. The team inevitably faces the unenviable choice of either prolonging development to add polish, or “finishing” the project without a full QA cycle.

From a QA Engineer’s perspective, these projects are a source of frustration. Spending your days writing defect tickets for “Site does not load in Chrome” or “User cannot login” or “Order form does not validate submissions” is like being a home inspector and being asked to sign off on a house that has no walls and is on fire.

Let’s all do what we do best

On the opposite end of the spectrum, projects that have passed all of our internal quality gateways and were built by developers that are passionate about quality tend to breeze through their QA cycle. They hit their deadlines and satisfy clients and users. These successes are the result of developer/QA partnerships where both groups can leverage their unique strengths.

In this scenario, developers have helped mop up the obvious issues by performing unit tests during development and basic sanity tests as integration occurs. From there, QA Engineers can focus on helping the development team double-check each and every requirement in each and every environment, and test the boundaries of the application. Good QA Engineers will vigorously push the application in ways the developers didn’t consider. And that is where the partnership flourishes.

Developers, remember, are focused on making the application work. That’s their job. And so, they are the most likely people to see their code work. In counterpoint, QA Engineers are focused on verifying that the application works. That may seem a small difference, but it’s why QA Engineers are good at finding unexpected variations and results outside the “happy path” that developers have worked hard to create.

This push and pull is invaluable to developing high-quality software but is only possible when QA Engineers are forced to be creative. And they can’t be creative if all their time is spent documenting “fish in a barrel” defects. When a developer has tested enough to feel confident that “QA won’t find anything,” a QA Engineer must leverage all her/his testing skills to verify everything works. This translates into a highly motivated and engaged QA Engineer who will work very hard to help a development team meet the goals of the client and/or users.

And thus, instead of QA being a safety net that developers use to avoid testing their own work, QA helps push developers – and their the applications – toward the best possible final result.

Filed under Quality Assurance

You Got Your UX in my QA

UXinQA2There is a fundamental relationship between user experience designers and quality assurance engineers, and I think it’s time to take that relationship to the next level. User Experience (UX) Designers and Quality Assurance (QA) Engineers are both deeply concerned about the user, and everyone wins when they talk to each other early and often.

My QA counterparts and I both want the websites we deliver to be usable. We want choices to be relevant. We want things to work. We want apps to make sense. We don’t want modal windows to prevent completion of integral tasks. In short, UX and QA both want to deliver a product that won’t leave the user cursing the heavens, having to force herself into a Zen-like contemplation of kittens in Christmas sweaters in order to quell the slow-burning rage that threatens to destroy her as she tries to perform simple tasks like transferring money to a savings account, buying tickets to a ballgame, or monitoring a sick parent’s medication schedule. UX designers are the experience architects and QA engineers are the building inspectors of the development lifecycle. UX figures out who we’re building for and why, analyzes user and business requirements, and incorporates it all with stakeholder vision and known constraints.  We then strive to design an experience that does no more and no less than what it needs to be. QA ensures that I, as a UX Designer, don’t make stupid blunders that will undermine my stakeholders’ goals and interfere with my users’ experience when all is said and done. Do developers help with this, too? Of course. And perhaps unsurprisingly, I also advocate for early and regular consultation between UX and development. But the type of value that QA can bring to a project when consulted early on goes beyond the type of specialized guidance and knowledge that even developers bring to the table.

To illustrate — if I’m designing a deck for my next-door neighbor who just moved his aging parents in, a QA engineer is the inspector that can inform me (before I’ve poured my footings) that even though I wouldn’t know by looking at it, the soil in my area is actually classified as a mixture of clay and sand, which means that diameter of my footings needs to be wider than it would in normal soil. This inspector might also point out that if my neighbor’s aging parents currently use power wheelchairs, or if they might need them in the future, it would be in my best interests to increase the diameter of the footings even further to accommodate the extra weight. And the inspector might lastly point out that with wheelchairs on the deck, I’ll need a ramp, and that the ramp ought to have a slope no greater than 9.5 degrees.

The most obvious software development analogue to that example is accessibility. Call it ADA compliance, 508 compliance or what have you; the point is that there are standards that you’re almost certainly not aware of for making sure that your application or website is usable for those with disabilities. But a good QA team has engineers that are steeped in those standards, and they know how those standards relate to every device you’ve targeted. Making changes to accommodate accessibility late in the game – or after your site is done – can be very costly, so why not have the conversation early on? Ditto that argument for data and application security. And ditto that argument for just plain old usability. Bring QA in for a quick consultation on your wireframes, and chances are you’re likely to save yourself some revisions.

As a designer, is it my job to think through use cases? Yes. Am I good at it? Yes. Am I able to consider every contingency of every scenario my project entails and implies? No. This is why a strong relationship between UX and QA is indispensable. UX Designers and QA Engineers have a similar mandate, but they fulfill it with different toolkits and from different perspectives. As one of my friends in QA puts it, “UX creates and QA destroys.” But we both do it for the same reason — we respect and love the user. QA engineers bring their extensive experience with devices, platforms, browsers, protocols and standards to bear on every project they touch. They are the brilliant generalists that throw our researched, prototyped, revised, and user-tested designs into the fiery furnace of the real world to see what burns and what doesn’t. Loop them in earlier, and you can avoid putting out fires later. So what does this collaboration look like? Here at The Nerdery we offer Embedded QA, in which QA is along for the entire ride on a project. We’ve seen tremendous success with this approach, but it carries some additional cost. The type of collaboration I’m talking about can be much leaner. It can be a conversation, or maybe two hours with an accessibility expert before you get into visual design to make sure your navigation scheme and color palette won’t set off red flags down the road. Build strong relationships up front with the folks that are responsible for signing off on your project down the road, and you’re guaranteed to save yourself time and headaches.

NerdCast #86: Making the Case for UX and QA Partnerships

2014.02-Christopher-Stephan-Alex-HoferIn this episode of the NerdCast we discuss the crazy notion to bring Quality Assurance engineers into the usability testing process.  It’s not that much of a crazy notion the more Christopher Stephan and QA Engineer Alex Hofer talk throughout this discussion about bridging the gap between these two project activities concerned with quality control.

Host: Ryan Carlson

Guests: Christopher Stephan (UX) and Alex Hofer (QA)

Listen Now: Running Time: 0:22:11 / Subscribe on iTunes

Filed under NerdCast

NerdCast #85: Targeted Cyber Crime – Discussing BlackPOS

NerdCast Album ArtOn this episode of the NerdCast we interview security experts Chris Wade and Jason Herbst from the Nerdery QA team. We look at the malware that was used to target high profile retail companies in a massive case of stolen data. The software called BlackPOS is a brilliant piece of software and in another context is genius in its design. Hear more about how the malware works, what it can reportedly do based on security research firms, and what Jason and Chris think of our current state of security.

Host: Ryan Carlson (Tech Evangelist)

Guests: Chris Wade and Jason Herbst (QA Department)

Listen Now: Running Time: 0:23:13 / Subscribe on iTunes


Reclaiming Usability – Part 4: UX and QA, an Evaluative Partnership

When we can Test, we can ImproveUsability is nothing new to IT, software, or design. It is one of the fundamental building blocks of the UX practice, and is the grandfather of evaluative UX methods which help us to judge the effectiveness of the solution we are constructing. Despite its history, recent usage has given it a layer of tarnish, making it a hard to sell and hard to include where it could do some good.

In previous installments I have re-introduced you to usability as a part of the user experience process and practice, dusted off the value and offered it to you both plated in gold and on the cheap. Also I have explored how poor use and understanding blackened its name, and told you what lies in wait if you leave it out. Last time I told you not only how you can use it but also when. Finally, in part four, I’ll show you how it could change what we presently think of as QA.

When you want to get everything right

I’ve said previously that usability sometimes coincides with and is then presumed to be a QA function. I have also said that this perception is primarily the result of the timing with which it is frequently applied, and the time and budget available to fix things. Based on what I’ve said in previous posts, you might expect me to repeat what I’ve said about leaving usability until too late in the project. To those I say; Don’t leave your usability until too late in the project. If you want to know why, you may want to revisit parts two and three of this series. If you find yourself taking offense that any UX method could possibly be seen as a QA measure out of a sense of injured professional pride, stop. Before you warm up even one more indignant neuron, let me turn that on its head.

Hey! You got your QA in my UX

Much of what we do in UX is, when you boil it down far enough, is a QA measure. Which is to say that its purpose is to improve the quality of the product and the experience of the user interacting with it. We do research  with users to ensure that the we design the right thing to solve the right problem. Then we test with users to determine how successful we have been in our designs.

Usability, and evaluative UX methods in particular, are the only way that we can objectively evaluate design by engaging with real users to evaluate our work. This is not a test of “did it work as it was intended to”, but instead a test of “is this intended to do the right thing.” It is in this regard, as a tool for ensuring design quality in particular, that usability stands shoulder to shoulder with established QA practices when they test the designed functionality of developed code. So yes, UX is QA.

Nuh-Uh! You got your UX all over my QA

QA, as a practice, has it’s own problems. The greatest of which is the fact that they are consulted at the eleventh hour, when their expertise and findings are more difficult to act upon. Does this sound familiar? I think QA should engage early and often, in much the same way I think evaluative UX measures should be taken early and often. Now, with that idea tickling your synapses, it should seem only natural that QA professionals should become UX professionals. From the ranks of testers, technicians and troubleshooters, we can grow usability analysts. We can build partnerships that surround a project – testing and improving it from early discovery to post-launch management, not just from ideation to launch.

We can arm QA departments with valuable UX tools and methods that they can apply either on their own and in partnership. They can continue to actively support the success of the project by doing what they do best – poking holes and breaking things. They can also engage with the project at more points in its lifecycle. Finally, they can do formally what they’ve always done quietly; evaluate with UX in mind. And in this, QA is UX.

The Efficiencies of Partnership

Not every organization will have the ability to have UX and QA work closely and cooperatively. Not every organization has both departments in-house. However, if those of us in UX, and our colleagues in QA both start viewing our roles as complementary, and start working from the same basic play books, the opportunity to collaborate improves regardless of whether it’s within or across companies.

By linking UX and QA, UX maintains project visibility for more of the lifecycle, beyond the inception of the development phase, and in return QA gains formative input and can raise concerns when they arise throughout the project. We also double our resources when it comes to staffing our evaluative efforts. Methodologies, insights and resources can be shared to ensure that projects get the care and attention they need throughout their lifecycle.

I hinted a bit at what QA staff can do as UX Analysts, but let me be more concrete. There are many tasks and services that can be  accomplished in partnership with QA, or offered whole and complete by a QA department.

  • Heuristic Review (Investigation, done in a Discovery phase)

    • Format defined by UX

    • Conducted by: QA or UX

    • Synthesis: QA, UX or Collaborative

  • Usability/Task Analysis (Evaluation, done in all phases)

    • Format defined by UX

    • Test authoring: UX or Collaborative

    • Test execution: Collaborative

    • Synthesis: Collaborative

  • Accessibility Audits (Evaluation, done in all phases)

    • Format defined by QA

    • Conducted by: QA or UX

    • Synthesis: QA, UX or Collaborative

This allows QA departments to be more proactive, more engaged with the project’s health and success. Also it opens new revenue streams and engagement models that allow QA to show it’s value either in standalone engagements or as part of a larger project. In working with UX, we multiply the reach of both practices to positively affect project outcomes and finished products.

To sum up

UX has similar aims as QA. QA has the same overall goals as UX. There is a lot to be gained by building a common vocabulary, toolset and partnership between the two. Each can further enable the other, and together the combined resources and expertise can transform our expectations of project success. So, UX and QA professionals…get on that.

A final thought on usability

If you come away from here with one thought I would prefer it be this: The thought that you should be doing more usability. That you should want your designs to be vetted as thoroughly as your functioning code. That you should care about quality enough to apportion the time and budget to get it right, rather than eat the time and cost of fixing it when its wrong.

It doesn’t have to be expensive. It doesn’t have to be elaborate. But it should be in your project process. You should have space to size and plan and implement evaluations at the right times and in effective ways. You should include evaluative tasks in your projects. You should test your stuff.

Filed under The UX Files

The Moment I Realized Testing and QA Are Entirely Different Things

I’ve been a web developer for many years and I didn’t really know what “QA” was before I came to the Nerdery. I did a little browser testing sometimes during the later stages of development or after my project was live. More often than not I either didn’t have the time or wasn’t being paid enough to do specific “Quality Assurance” on the sites I built. In all honesty, the result was obvious.

When I sent my first project to QA at the Nerdery I was nervous and excited. I felt very thorough clicking through the site trying to see what they would find; I’d checked the site on a bunch of browsers; I read the Test Plan – we call them “TPS” reports – and felt like my bases were covered. By that time I’d heard tales of the tenacity of our Quality Assurance department but I was confident I’d done my diligence. Then the first “Ticket” came in…

113 Tickets Later and my notion of professional web development was changed forever. It was humbling and exhilarating to read the “Tickets” that were submitted. They were insightful, comprehensive, user-focused insights into the project I was sure I’d nailed. They saw things I didn’t and brought them to my attention with detailed precision. They evaluated every part of the website whether I thought it was relevant or not and reported exactly what someone – not me – would experience. They took the time to test and retest until everything – EVERYTHING – was correct.

Through this process it was evident that our QA Engineers had a deep understanding and enduring passion for “assuring” extraordinary user experiences on the web. Their patience, attention, and creativity were a perfect reflection of the planning and design effort put forth at the beginning of the project. The result was obvious.

In our industry we elevate the designers, user experience teams, and the developers as the artists and engineers in the interactive space, and we should because we are those things. But after design and development wind down a whole new group is there to make sure everything is done, done well, and done right. 113 tickets later and I was a raving fan of our Nerdery QA team. They are the ones that are the truly indispensable artists and engineers in the interactive space.

Filed under Tech Tips, Technology

NerdCast #74: Webinar Q&A – Debugging the Benefits of Quality Assurance

2014.01-NerdCast-Matthew-Erickson-Justin-HolmanWe get to address all of the great questions we had during the webinar all about Quality Assurance, custom engagements, dealing with documentation, and typical timelines. This discussion is going to be applicable for anybody whether you attended the webinar or not. Although be warned, thar be dragons – we do reference content from the webinar that might be helpful. We are joined again by Matthew Erickson and Justin Holman two veterans of The Nerdery QA department.

If you missed the webinar, check out the full recording.

Host: Ryan Carlson - Tech Evangelist

Guests: Matthew Erickson (QA Operations Manager), and Justin Holman (QA Controller)

Listen Now: Running Time: 0:36:35 / Subscribe on iTunes

Filed under NerdCast

NerdCast #67: Webinar Q&A – Front-End Standards at The Nerdery

2013.12-Nerdcast-Ted-Flynn-Brian-BellIn this episode of the NerdCast we are joined by our webinar panelists Brian Bell and Ted Flynn to cover the great Q&A questions submitted before, during, and after the live webinar event that we did not have time to address. Join us to get into the weeds on front-end development standards and what we do about them here at The Nerdery.

Host: Ryan Carlson - Tech Evangelist

Guests: Brian Bell and Ted Flynn, Senior Front End Developers at The Nerdery

Listen Now: Running Time: 0:25:38 / Subscribe on iTunes


NerdCast #54: Confessions of a QA Engineer – The Web Accessibility Edition

2013.10-Bree-ComptonThis episode of The NerdCast we talk with Bree Compton, a Quality Assurance engineer who shares her passion for the role of a QA engineer, what they do, and her work on accessibility testing within the department. Hear how she got her start in QA, her Nerdery origin story, and her advice to others looking to break into this critical development role.

Host: Ryan Carlson

Guests: Bree Compton, Quality Assurance Engineer at The Nerdery

Listen Now:

Running Time: 0:36:44 / Subscribe on iTunes

Filed under NerdCast