Tag Archives: QA

What Owners Need to Know Before Rebuilding a Website or App


“The pressure is on and you need to decide whether you can move forward with [your current software] or if it’s more efficient to start over.”

bad-app-reviewsIt’s never fun to spend money and get less than you expected. Consider a business owner with a website or app that either needs emergency updates or long-due feature enhancements. Perhaps it was built without much concern for scalability or future feature-needs. Maybe its developer moved to Timbuktu before completion. Or worse, the app was live and users were peppering the App Store with 1 star reviews. Whatever the reason, the business owners and stakeholders are left wondering: “What are my options? How do I evaluate those options? Do I have to start from scratch? Will all my previous effort/time/money just go to waste?” Read more

Filed under Articles

Your Money is Buying More Than Just Software Testing, It’s Quality Assurance

NQA-screens-galoreIf we were only concerned with testing software we would need to think of a different name for our Quality Assurance department. There isn’t anybody applying stickers on widgets from an assembly line stamped “Inspected by No. 2177.” We do things a bit differently here at The Nerdery and it’s earned us the respect of our partners by putting the right engineers on the right projects. The investment our partners make on Quality Assurance for their project ensures that they have smart people providing real solutions throughout their projects to save them time, reduce overall effort, and ensure project success.

Read more

Filed under Quality Assurance

Minimum Viable Product and Quality Assurance

On a perfect project, everything would go exactly as planned. The client would clearly indicate exactly what they want and never change their mind. There would be no complications along the way to interfere with the schedule. My department, Quality Assurance (QA), would receive the completed project on time, all features would be clearly documented and only a handful of edge-case bugs would be found.

Perfect projects don’t exist. Fact of the matter is, regardless of the team’s skill, there’s never a guarantee of success. There may be scope creep, clients may change their minds, development complications may arise, deadlines may be missed and projects may fall behind. There are plenty of counters to all of these – more than I could ever hope to cover here. However, we’re here to talk about how QA’s minimum viable product can help get the project back on track.

Read more

Filed under Quality Assurance

In an ocean of browsers and gadgets, support standards for web & mobile platforms are ephemeral

Programming, interactive media, and the web have come a long, long way. It’s humbling to realize nine years have passed since the Arduino introduced an affordable microcontroller to the public, or seven years since the original iPhone redefined our expectations of what a cell phone can be.

2014…another year of the internet, social media, home automation, video games, wearable tech, smart TVs, and multiple ecosystems of mobile applications running on a growing variety of phones and tablets. Maybe you’re developing an app or website or you know you need one…In an ocean of computers, browsers, and gadgets, how do you choose what to support? Support means testing and there’s simply too many options to test on every single phone, tablet, and browser.

Whether it be a website, native mobile application or web app, the environment you support defines the reach and intent of your presence online. Choosing the right platforms to support demonstrates a strong product vision, general technological awareness and long-term plan for the work. So, what is the right direction to aim? Read more

Webinar: Mastering Interactive Development Lingo

Due to unfamiliar and sometimes confusing  language, interactive projects can end up costing more or take longer to complete than they were supposed to. One of the primary contributors to project delay is a result of what we like to call the “Miscommunication Tax.” In this webinar we took the opportunity to talk through some of the lingo that consistently lends itself to miscommunications during a development project. If you are selling interactive projects, working on interactive projects, or would like to hear what a user experience designer, a developer, and quality assurance engineer have to say about confusing terms and how to create clarity around them, you’ll want to tune in.

Bonus Content:

Filed under Nerdery Webinar

QA Myths: “I don’t need to test because QA will catch it.”

Having a dedicated group of testers can be an invaluable resource when you’re a developer. But having testers (or maybe even a full-blown quality assurance department) doesn’t mean that a developer should avoid executing tests. Developers should understand how to most effectively use their own time, as well as the time of the testers.

Unit tests are useful and so are “sanity checks”

There are plenty of other people discussing how – from a developer perspective – unit testing is as important as “going to the gym” so I won’t belabour the point. But as a quality assurance (QA) professional I have experienced a marked difference between code that has extensive unit tests and code that does not. That said, I’ve also noticed improved quality amongst developers that do functional sanity checks. These checks became such a good indication of a project’s readiness for QA that we instituted a series of quality gateways here at The Nerdery to encourage this behavior.

So, what’s a “functional sanity check?” Short answer: Use your application/code/system as if you were a user. This can be as simple as loading up a website in Internet Explorer 8 to see if it looks remotely close to the way it does in Chrome. This could also be a meeting of the development team where someone drives through the application and everyone else has the chance to catch, triage, and assign defects as the team discovers them. Performing the most basic functional integration tests eliminates the easy-to-find issues.

“Why bother?”

As a project prepares to enter its QA phase, I’ve heard some developers say “why should I bother finding all these tickets when QA is just going to test everything?” This attitude is dangerous because at first blush it seems to make sense. No one wants to duplicate effort or make a project drag on longer than necessary. But in reality, projects are neither faster nor cheaper when all test activities are left to QA.

I’ve seen these projects slow to a crawl in their QA cycle. Their issue databases become quickly bloated with issues that are low-hanging fruit and – with the deadline quickly approaching – the time that could have been spent on final polish is instead spent on making sure the whole project doesn’t fall apart. The team inevitably faces the unenviable choice of either prolonging development to add polish, or “finishing” the project without a full QA cycle.

From a QA Engineer’s perspective, these projects are a source of frustration. Spending your days writing defect tickets for “Site does not load in Chrome” or “User cannot login” or “Order form does not validate submissions” is like being a home inspector and being asked to sign off on a house that has no walls and is on fire.

Let’s all do what we do best

On the opposite end of the spectrum, projects that have passed all of our internal quality gateways and were built by developers that are passionate about quality tend to breeze through their QA cycle. They hit their deadlines and satisfy clients and users. These successes are the result of developer/QA partnerships where both groups can leverage their unique strengths.

In this scenario, developers have helped mop up the obvious issues by performing unit tests during development and basic sanity tests as integration occurs. From there, QA Engineers can focus on helping the development team double-check each and every requirement in each and every environment, and test the boundaries of the application. Good QA Engineers will vigorously push the application in ways the developers didn’t consider. And that is where the partnership flourishes.

Developers, remember, are focused on making the application work. That’s their job. And so, they are the most likely people to see their code work. In counterpoint, QA Engineers are focused on verifying that the application works. That may seem a small difference, but it’s why QA Engineers are good at finding unexpected variations and results outside the “happy path” that developers have worked hard to create.

This push and pull is invaluable to developing high-quality software but is only possible when QA Engineers are forced to be creative. And they can’t be creative if all their time is spent documenting “fish in a barrel” defects. When a developer has tested enough to feel confident that “QA won’t find anything,” a QA Engineer must leverage all her/his testing skills to verify everything works. This translates into a highly motivated and engaged QA Engineer who will work very hard to help a development team meet the goals of the client and/or users.

And thus, instead of QA being a safety net that developers use to avoid testing their own work, QA helps push developers – and their the applications – toward the best possible final result.

Filed under Quality Assurance

You Got Your UX in my QA

UXinQA2There is a fundamental relationship between user experience designers and quality assurance engineers, and I think it’s time to take that relationship to the next level. User Experience (UX) Designers and Quality Assurance (QA) Engineers are both deeply concerned about the user, and everyone wins when they talk to each other early and often.

My QA counterparts and I both want the websites we deliver to be usable. We want choices to be relevant. We want things to work. We want apps to make sense. We don’t want modal windows to prevent completion of integral tasks. In short, UX and QA both want to deliver a product that won’t leave the user cursing the heavens, having to force herself into a Zen-like contemplation of kittens in Christmas sweaters in order to quell the slow-burning rage that threatens to destroy her as she tries to perform simple tasks like transferring money to a savings account, buying tickets to a ballgame, or monitoring a sick parent’s medication schedule. UX designers are the experience architects and QA engineers are the building inspectors of the development lifecycle. UX figures out who we’re building for and why, analyzes user and business requirements, and incorporates it all with stakeholder vision and known constraints.  We then strive to design an experience that does no more and no less than what it needs to be. QA ensures that I, as a UX Designer, don’t make stupid blunders that will undermine my stakeholders’ goals and interfere with my users’ experience when all is said and done. Do developers help with this, too? Of course. And perhaps unsurprisingly, I also advocate for early and regular consultation between UX and development. But the type of value that QA can bring to a project when consulted early on goes beyond the type of specialized guidance and knowledge that even developers bring to the table.

To illustrate — if I’m designing a deck for my next-door neighbor who just moved his aging parents in, a QA engineer is the inspector that can inform me (before I’ve poured my footings) that even though I wouldn’t know by looking at it, the soil in my area is actually classified as a mixture of clay and sand, which means that diameter of my footings needs to be wider than it would in normal soil. This inspector might also point out that if my neighbor’s aging parents currently use power wheelchairs, or if they might need them in the future, it would be in my best interests to increase the diameter of the footings even further to accommodate the extra weight. And the inspector might lastly point out that with wheelchairs on the deck, I’ll need a ramp, and that the ramp ought to have a slope no greater than 9.5 degrees.

The most obvious software development analogue to that example is accessibility. Call it ADA compliance, 508 compliance or what have you; the point is that there are standards that you’re almost certainly not aware of for making sure that your application or website is usable for those with disabilities. But a good QA team has engineers that are steeped in those standards, and they know how those standards relate to every device you’ve targeted. Making changes to accommodate accessibility late in the game – or after your site is done – can be very costly, so why not have the conversation early on? Ditto that argument for data and application security. And ditto that argument for just plain old usability. Bring QA in for a quick consultation on your wireframes, and chances are you’re likely to save yourself some revisions.

As a designer, is it my job to think through use cases? Yes. Am I good at it? Yes. Am I able to consider every contingency of every scenario my project entails and implies? No. This is why a strong relationship between UX and QA is indispensable. UX Designers and QA Engineers have a similar mandate, but they fulfill it with different toolkits and from different perspectives. As one of my friends in QA puts it, “UX creates and QA destroys.” But we both do it for the same reason — we respect and love the user. QA engineers bring their extensive experience with devices, platforms, browsers, protocols and standards to bear on every project they touch. They are the brilliant generalists that throw our researched, prototyped, revised, and user-tested designs into the fiery furnace of the real world to see what burns and what doesn’t. Loop them in earlier, and you can avoid putting out fires later. So what does this collaboration look like? Here at The Nerdery we offer Embedded QA, in which QA is along for the entire ride on a project. We’ve seen tremendous success with this approach, but it carries some additional cost. The type of collaboration I’m talking about can be much leaner. It can be a conversation, or maybe two hours with an accessibility expert before you get into visual design to make sure your navigation scheme and color palette won’t set off red flags down the road. Build strong relationships up front with the folks that are responsible for signing off on your project down the road, and you’re guaranteed to save yourself time and headaches.

NerdCast #86: Making the Case for UX and QA Partnerships

2014.02-Christopher-Stephan-Alex-HoferIn this episode of the NerdCast we discuss the crazy notion to bring Quality Assurance engineers into the usability testing process.  It’s not that much of a crazy notion the more Christopher Stephan and QA Engineer Alex Hofer talk throughout this discussion about bridging the gap between these two project activities concerned with quality control.

Host: Ryan Carlson

Guests: Christopher Stephan (UX) and Alex Hofer (QA)

Listen Now: Running Time: 0:22:11 / Subscribe on iTunes

Filed under NerdCast

NerdCast #85: Targeted Cyber Crime – Discussing BlackPOS

NerdCast Album ArtOn this episode of the NerdCast we interview security experts Chris Wade and Jason Herbst from the Nerdery QA team. We look at the malware that was used to target high profile retail companies in a massive case of stolen data. The software called BlackPOS is a brilliant piece of software and in another context is genius in its design. Hear more about how the malware works, what it can reportedly do based on security research firms, and what Jason and Chris think of our current state of security.

Host: Ryan Carlson (Tech Evangelist)

Guests: Chris Wade and Jason Herbst (QA Department)

Listen Now: Running Time: 0:23:13 / Subscribe on iTunes


Reclaiming Usability – Part 4: UX and QA, an Evaluative Partnership

When we can Test, we can ImproveUsability is nothing new to IT, software, or design. It is one of the fundamental building blocks of the UX practice, and is the grandfather of evaluative UX methods which help us to judge the effectiveness of the solution we are constructing. Despite its history, recent usage has given it a layer of tarnish, making it a hard to sell and hard to include where it could do some good.

In previous installments I have re-introduced you to usability as a part of the user experience process and practice, dusted off the value and offered it to you both plated in gold and on the cheap. Also I have explored how poor use and understanding blackened its name, and told you what lies in wait if you leave it out. Last time I told you not only how you can use it but also when. Finally, in part four, I’ll show you how it could change what we presently think of as QA.

When you want to get everything right

I’ve said previously that usability sometimes coincides with and is then presumed to be a QA function. I have also said that this perception is primarily the result of the timing with which it is frequently applied, and the time and budget available to fix things. Based on what I’ve said in previous posts, you might expect me to repeat what I’ve said about leaving usability until too late in the project. To those I say; Don’t leave your usability until too late in the project. If you want to know why, you may want to revisit parts two and three of this series. If you find yourself taking offense that any UX method could possibly be seen as a QA measure out of a sense of injured professional pride, stop. Before you warm up even one more indignant neuron, let me turn that on its head.

Hey! You got your QA in my UX

Much of what we do in UX is, when you boil it down far enough, is a QA measure. Which is to say that its purpose is to improve the quality of the product and the experience of the user interacting with it. We do research  with users to ensure that the we design the right thing to solve the right problem. Then we test with users to determine how successful we have been in our designs.

Usability, and evaluative UX methods in particular, are the only way that we can objectively evaluate design by engaging with real users to evaluate our work. This is not a test of “did it work as it was intended to”, but instead a test of “is this intended to do the right thing.” It is in this regard, as a tool for ensuring design quality in particular, that usability stands shoulder to shoulder with established QA practices when they test the designed functionality of developed code. So yes, UX is QA.

Nuh-Uh! You got your UX all over my QA

QA, as a practice, has it’s own problems. The greatest of which is the fact that they are consulted at the eleventh hour, when their expertise and findings are more difficult to act upon. Does this sound familiar? I think QA should engage early and often, in much the same way I think evaluative UX measures should be taken early and often. Now, with that idea tickling your synapses, it should seem only natural that QA professionals should become UX professionals. From the ranks of testers, technicians and troubleshooters, we can grow usability analysts. We can build partnerships that surround a project – testing and improving it from early discovery to post-launch management, not just from ideation to launch.

We can arm QA departments with valuable UX tools and methods that they can apply either on their own and in partnership. They can continue to actively support the success of the project by doing what they do best – poking holes and breaking things. They can also engage with the project at more points in its lifecycle. Finally, they can do formally what they’ve always done quietly; evaluate with UX in mind. And in this, QA is UX.

The Efficiencies of Partnership

Not every organization will have the ability to have UX and QA work closely and cooperatively. Not every organization has both departments in-house. However, if those of us in UX, and our colleagues in QA both start viewing our roles as complementary, and start working from the same basic play books, the opportunity to collaborate improves regardless of whether it’s within or across companies.

By linking UX and QA, UX maintains project visibility for more of the lifecycle, beyond the inception of the development phase, and in return QA gains formative input and can raise concerns when they arise throughout the project. We also double our resources when it comes to staffing our evaluative efforts. Methodologies, insights and resources can be shared to ensure that projects get the care and attention they need throughout their lifecycle.

I hinted a bit at what QA staff can do as UX Analysts, but let me be more concrete. There are many tasks and services that can be  accomplished in partnership with QA, or offered whole and complete by a QA department.

  • Heuristic Review (Investigation, done in a Discovery phase)

    • Format defined by UX

    • Conducted by: QA or UX

    • Synthesis: QA, UX or Collaborative

  • Usability/Task Analysis (Evaluation, done in all phases)

    • Format defined by UX

    • Test authoring: UX or Collaborative

    • Test execution: Collaborative

    • Synthesis: Collaborative

  • Accessibility Audits (Evaluation, done in all phases)

    • Format defined by QA

    • Conducted by: QA or UX

    • Synthesis: QA, UX or Collaborative

This allows QA departments to be more proactive, more engaged with the project’s health and success. Also it opens new revenue streams and engagement models that allow QA to show it’s value either in standalone engagements or as part of a larger project. In working with UX, we multiply the reach of both practices to positively affect project outcomes and finished products.

To sum up

UX has similar aims as QA. QA has the same overall goals as UX. There is a lot to be gained by building a common vocabulary, toolset and partnership between the two. Each can further enable the other, and together the combined resources and expertise can transform our expectations of project success. So, UX and QA professionals…get on that.

A final thought on usability

If you come away from here with one thought I would prefer it be this: The thought that you should be doing more usability. That you should want your designs to be vetted as thoroughly as your functioning code. That you should care about quality enough to apportion the time and budget to get it right, rather than eat the time and cost of fixing it when its wrong.

It doesn’t have to be expensive. It doesn’t have to be elaborate. But it should be in your project process. You should have space to size and plan and implement evaluations at the right times and in effective ways. You should include evaluative tasks in your projects. You should test your stuff.

Filed under The UX Files