May 14

A Skilled Software Tester

A few months ago an acquaintance of mine asked me what I did for a living, and so I replied “I’m a Software Tester … A Skilled Software Tester”.  There was a noticeable pause between the 2 parts of my answer – mostly because I feel that there’s sometimes a false belief amongst people that don’t know any better that being a Software Tester doesn’t require an individual to be very skilled (this is something I believe that’s changing as the community of skilled software testers continues to grow, learn, promote & learn skills, challenge old ways to doing things, and focus on testing that has value).

I continued my answer to the question with a brief explanation of what I meant, without going on for an extended period of time making it seem like a speech or a lunch & learn.  I explained that I don’t spend my testing time doing things “the old way” (I actually found a better term for it thanks to Keith Klain – more on this in a separate post), of filling out templates, spending hours writing and executing heavily scripted test cases, or writing 30 page test plan documents that nobody will actually read.  I spend my time testing, what & how I test will depend on the application and the different situations surrounding the application. I don’t test everything the same way because I don’t believe in best practices, I believe in good practices in certain contexts. I work with developers, project managers and other testers to provide information about the quality of the software. That was my explanation.

In the weeks that followed, I spent some time thinking about that explanation and how I could explain it better in the future. The content of my explanation was composed of conversations I’ve had with many people (testers and non-testers), explaining to them that there was more to testing than templates, documents and scripted test cases – that skilled software testing and testers did indeed exist and were really good at what they did. I had spoken about testing skills and shifting focus to testing that added value to the project to quite a few people and was looking into ways to do it better. I came across a post written by Keith Klain The skilled testing revolution … which explains that things are changing, the old ways of doing things are on their last legs, and that skilled software testing is starting to gain momentum and recognition. I’ve send the post to some people as it highlights, and better explains the message I’ve been aiming to get across.

As I continue to learn, apply what I’m learning, and talk to other testers – all of which contributes to me becoming a better and more skilled software tester everyday –  I’m glad to say that I’m part of the skilled testing revolution.

Apr 25

A Day Full of Errors & Crashes

Yesterday was an extremely productive day for me – I got a lot done. A lot of work involving a lot of effort, considerable time and a good amount of thinking (most of the things I put my energy and effort towards require thinking).  It was a good day in terms of productivity.  I also encountered a lot of errors & crashes – but I wasn’t testing.

Now let me explain – as a Software Tester I encounter (and generate) a lot of errors, crashes, undesirable/inconsistent/unpredictable behaviours during my investigation of an application.  The errors & crashes I encountered yesterday happened while I wasn’t testing.  They happend while I was using different software applications to complete or perform some tasks – this type of thing isn’t rare for me. What is rare is that the errors & crashes I encountered yesterday happened while I was using different software applications to complete or perform tasks on all 3 of my devices.

A software application I had been using on my laptop as a project and test management tool generated an error and crashed 3 different times – first generating an error message and then ending abruptly seconds later. The error message generated was displayed on the screen for barely a second – I had no time to even read a word it said.  The first time it happend the processes “crippled” my entire machine and I had to restart it.

A few hours later after I had completed my “priority tasks” for the day, I was in relaxation mode. I thought I’d search for an action movie to rent or possibly purchase on my tablet. For some reason every time I filtered the movie selection by action movies the app would crash on my tablet.  After 3 times I gave up – wasn’t my night to watch a movie.

A few hours after that I wasn’t asleep as I should’ve been (I guess the 80km/h winds outside had something to do with it) I decided to reactivate the iMessage service on my phone. I had turned it off earlier in the day for a particular reason.  I was prompted to entered my password to activate it and for some reason I just wasn’t able to activate it. After a few tries (I think it was about 3) I decided to give up and leave it alone until the following morning.  This morning I entered the password once and I was able to reactivate it within seconds.

I guess there will be days like that – I just didn’t foresee it happening while I wasn’t officially testing, and on all 3 devices.

Apr 24

Lesson Learned: Applying a Testing Rule to a Car Scenario

Throughout my Testing career one of the things I’ve learned (and learned early on in my career) was to never report or identify possible causes of a problem without proper knowledge based on investigation and facts to back up my claim.  This “rule” can be based on a “finding” during testing or discovering a defect during testing.  Part of my job (and approach to testing) is to explore, discover, learn about the application I’m testing and investigate it – this includes investigating behaviours and defects I come across so that I can provide knowledgable information about what I’m reporting. I’d never identify a defect and then list possible causes with uneducated guesses – without any investigation, or facts of some kind to explain why I believe something would be the cause of the problem.

While I’d never do this during any type of testing – this is exactly what I did in a scenario involving my car recently.  About a week ago I had identified a burning smell coming from my car after I had driven it.  I’ve had a few cars and encountered my fair share of different problems with them but I had never encountered this type of burning smell before.  I had just changed my wheels and tires but I knew this wasn’t the issue behind the problem – the tires were the correct size and the rims weren’t rubbing against the shocks nor the callipers in the front or back.  The days went by and the burning smell was still present.  For some reason I was convinced that the burning smell was a result of either: 1 – oil burning somewhere in the engine or 2 – an electrical wire burning somewhere.

I’m not sure what I based the possible causes on – I don’t have a mechanical lift or any other type of equipment to diagnose the cause of such problems.  My reasons were based on uneducated guesses. I decided to visit a buddy of mines who’s a mechanic with his own shop and told him the problem and what I believed the causes to be.  We took a road test, came back to the garage and he smelled the burning smell and decided to put the car onto the lift right away.  Took him less than 30 seconds to identify the cause of the issue (burning smell) – there was a plastic bag or some type of plastic that had gotten stuck under the car and melted onto the exhaust pipe towards the front of the car.

So much for what I thought and was almost sure the causes of the burning smell were. It was a good reminder for me to perhaps apply what I apply in my testing to other parts of life.

Lesson learned.

Apr 10

Is “Testing” a Bad Word?

There have been a few instances where I’d be working on a task related to my testing of a feature of application and I’d overhear something that would make me ask myself “Is Testing a bad word?”

Things like “this will have to be qa’d” or “we’re going to have to quality control this application” or “we’ll send this off to qa”. Once I even heard a software tester say “hey we’re not quality assurance”,  I smiled for an instant until I heard the second part of the statement “we’re quality control”.  While I do realize these statements are made due to miseducation or a misunderstanding of what these “software testers” actually do and the purpose they serve, it doesn’t stop me from asking myself if testing is a bad word.

There are some individuals with whom I’ve had conversations with to try to help them distinguish between the two and maybe even enlighten them to the fact that they aren’t assuring quality or controlling quality as Software Testers. I’ve even used some examples I’ve learned studying Michael Bolton’s work to illustrate my point “The role for us is not quality assurance; we don’t have control over the schedule, the budget, programmer staffing, product scope, the development model, customer relationships, and so forth.”  I’m often told “well here we are quality assurance”.

But on the other side, there are other individuals I’ve spoken to – smart, enthusiastic Software Testers who want to think when doing their job, who want to do meaningful work and have the results of their efforts serve a good purpose, who are enlightened by some of the content I’m sharing with them – I like to refer to this as the bright side.

So is Testing a Bad Word?  Hmm I guess that depends on who you ask.

Mar 19

Testing shouldn’t take more than …

There have been a few times in my career where I’ve had somebody (Test Manager, Test Coordinator, Development Manager etc)  tell me something along the lines of “testing this shouldn’t take more than n hours or n days.”  This can be conveyed (and interpreted) in two ways; we have n hours or n days to test, or it can mean that you should be able to complete your testing in n hours or n days.  More often than not, it means the latter. The statement conveyed in that manner completely disregards the fact that complete testing is not possible (I won’t be going into this detail in this post).  Furthermore, the statement is often made without a real understanding of the feature to be tested. Sometimes the test manager or coordinator can make the statement without knowledge of the technical details, the risks, and without any consideration given to the details for the specific feature or application.  Other times the statement is made based on what somebody else, for example a developer or business analyst has said.

In my opinion this is similar to somebody making a personal recommendation to try a restaurant based on what somebody else has said without ever having eaten at the restaurant themselves.

Either way it leads to unrealistic expectations, inaccurate estimates, and a lot of misunderstandings & confusion (not to mention a poor understanding of Software Testing) – unless the Software Tester takes action to prevent that. This can lead to a lot of explanations, disagreements, meetings and more which ultimately cuts into testing time.  The first time I was in this situation, and every subsequent time I’ve always made it a point to speak up and explain to the individual making the statement what the testing for the particular situation actually entails and why the statement may be inaccurate – those with a stake in the project (Product Owners, Business Owners, Project Managers) should be aware of what was developed, what was tested, what wasn’t tested, along with relevant and valuable information discovered during testing so that they can make the appropriate decision regarding the feature or application.

A good understanding of what Software Testing is and what it should set out to accomplish – the mission, goals, and purpose is something those with an influence on testing activities within an organization should be aware of – as it may dictate how valuable testing time is spent.

Feb 18

Testing is not Repetitive – Part 2

In Part 1 of this topic I wrote about the experiences that made me realize that Software Testing is not repetitive.  These are my experiences from over 7 years ago when I first started my career in Software Testing.

When I began to realize that skilled Software Testing was not a repetitive job, but one that required and made use of different types of skills & knowledge and was a thinking persons job – I immediately knew I wanted to become a skilled Software Tester.  I didn’t yet exactly know what that entailed, or how I would start my journey on becoming one, but that didn’t matter – I knew where I wanted to go so to speak and now I would work on getting there.

I saw two types of Testers (there are a lot more but for the scope of this post I’ll stick with two); those who wrote and ran the same test cases over and over & those who used different testing skills, knowledge, experience in their testing to find some great bugs and report valuable information.  The latter were the ones other individuals (testers, developers, product owners) would go to for input on how & what to test. They were the ones who would themselves go to ask developers for information, find information using different channels that were available to them.  They didn’t tell other testers what to do, but made themselves available to them as a good source of knowledge.

I was assigned to and chosen to test more “complex” projects – this is what led me to start working on testing different types of applications. By “complex” I mean testing applications for which there were no pre-existing test cases. Applications that nobody within the Testing Team knew much about in terms of functionality and technicality. I would talk to (ask questions) and work with developers and data analysts to determine what the application did, how it would be used and most importantly (and a learning experience for me) how I was going to test it.  I wasn’t just testing the application from the UI, I was now testing different components that made up the application.  I enjoyed being assigned and chosen for these projects.  Not only did I work hard to be the one the test manager chose for these projects but I showed that I wanted to learn and was both willing to and able to do so.

From this point on; about 7-8 months into my Testing career I knew that I wanted to learn more, get better at what I did and not just sit and execute the same tests over and over – a job that a robot would be able to do or even somebody with less knowledge & skill than me could do.

Feb 17

Testing is not Repetitive – Part 1

When I first started my career in Software Testing (about 7 and a half years ago now) one of the most common phrases I would hear is that “testing is a repetitive job”.  It wasn’t uncommon to read job descriptions with the statement “the candidate doesn’t mind doing repetitive work”.  At that time I didn’t know any better – I mean here were people with tons of experience and skills, who knew a lot more about the profession than me, so I believed it (at that point I didn’t have any real experience or knowledge about the profession).

This false belief/statement didn’t hold true in my mind for long.  At my second job (about 6-7 months into my career) I had picked up fast and started to work on many different types of applications.  I hadn’t yet read much content or books about Software Testing but I quickly realized that the uses of each type of application were different from each other and that the user would interact differently depending on the type of application, and that different applications may be subject to different rules and regulations. These were the applications I was now testing.  I was working at a company developing and distributing mobile content – games, ringtones, wallpapers, mobile web pages.  I was testing all of these applications and the features within them on the different projects I was working on and I realized it didn’t make sense to test the same test cases for each type of application because the use cases for one type of application wasn’t really applicable in a different type of application.  It didn’t even make sense for me to execute the same test cases repeatedly for the same application – while I understood the need for regression testing, I wasn’t sure how much and when it should be done and if spending numerous hours and days executing the same tests over and over was actually increasing the quality of the software.

At this point I wasn’t yet aware of the context-driven school of testing and that it was an extremely skilled approach to testing to which some great & knowledgable Software Testers have contributed and that there was community of skilled context-driven Software Testers. I was aware that while some testing may be repetitive (this may actually Checking and not Testing – I won’t go into this difference in this post); skilled testing that required one to think and use their brain to be creative and learn about the technology behind the application in order to better test software was anything but repetitive.

Even after I came to this realization I would still hear that “testing is repetitive” and would often see this statement in job offers. I was even asked in a few job interviews during those times if I was willing to do repetitive work – even though I knew I was a tester who did just more than execute tests over and over based on test scripts I wrote, I would still answer yes – at that time I didn’t yet have the confidence, skill set, experience and knowledge to say otherwise; to actually have a viewpoint of software testing that was different from somebody who had more experience than me who believed in that statement.

This was over 7 years ago now, a lot has changed – including my confidence level, skill set, experience and knowledge.  While I don’t read through as many job offer descriptions as I used to, I’ll still read over a select few from the ones I do receive – and one thing I don’t see often (if ever) is a statement along the lines of “repetitive work”.  I think a lot more people (although maybe not enough) within the Software/IT industry realize that Testing is actually a skilled & knowledgeable profession.

In Part 2 I’ll write about the things that made me realize early on in my career that I didn’t want to be a Tester that would execute test scrips over and over and define that as “Testing”.

Jan 29

Test Tools

I’ve used & learned how to use quite a few test tools over the course of my career thus far (and continue to do so) and one thing I’ve always believed in was that a test tool should aid you in your testing but shouldn’t define or limit the way you test or what you test.  About a week ago a fellow Software Tester I follow on Twitter, Benjamin Yaroch tweeted “A tool should help you accomplish a task, not dictate how the task is done.” I couldn’t agree more – but I will keep the scope and focus of this post on test tools (as opposed to tools in general) & bug/defect tracking tools in particular. I think my next post will be on SOA Test tools I’ve used.

I’ve been fortunate to have worked at companies and with teams that chose test tools that aided in what we did and what we were aiming to accomplish. I’ve also seen instances where because a particular test tool was available, it was chosen to be used within teams or projects where it wasn’t the best fit, which resulted in the way things were done to be built around the capabilities & features of the tool.

I’ve been asked “what is the best bug/defect tracking tool to invest in and use?”  My answer is that it depends on different things. Who will use it? How will they use it? Who will read the bugs? Who will update the bugs & the information in the bugs? Does the tool need to do more than track bugs?

A few years ago I was the principle software tester in a Software R&D team (one of the best teams I’ve ever had the opportunity to work with), we worked using an Agile SCRUM methodology and one of the tools we used was Rally Project Manager (which does a lot more than just serve as a bug/defect logging tool). For how we used it (user stories; associated bugs/defects; work effort estimates; time estimates; task breakdowns; acceptance criteria; sprint planning, burn down charts; measuring velocity etc …) I can’t think of a better tool I have used for the purpose we needed it for. The tool aided us in what we wanted & needed to accomplish – we didn’t modify the way we worked or the way we did things around the tool.

Currently I’m working on a project where I’m using HP Quality Center. The tool is a great fit for what we need it for (tracking new features & associated info; screen captures, bugs/defects). In this instance Rally Project Manager would not be the best tool – it would be overkill to use it.

I’ve worked on some independent projects with a small team where we used Microsoft Excel to keep track of bugs/defects.  It was easy for every member of the team to view, read, update information. It was great for how we intended to work and use it.

All this to iterate, test tools should serve a purpose – aid in what you’re doing, not define how you’re doing it.

 

Jan 15

Learning with colleagues

Depending on certain variables it can sometimes be rare or popular (at the opposite end of the spectrum) to meet somebody at work, on your team who is as passionate about the craft (and getting better at it) as you are. These dependent variables (among many) can be company, company culture, team, team structure, projects, communication methods, technology, and of course colleagues themselves.

Working as a consultant you meet many people. I recently met a fellow Software Tester very interested in learning more about certain approaches to testing and to become a better, skilled (building on current skill level) Software Tester – very similar to myself.

One of the areas of testing he’s worked with and is working on becoming better at is Performance Testing. Of course Scott Barber’s name came up as we’ve both read a lot of his work and apply what we’ve learned from the content where applicable. My colleague at work has more knowledge and skill than me in the area of Performance Testing – it’s actually an area of testing where I want to improve my skill level (and working towards that goal).

One technology I’ve worked with considerably and improved my knowledge & skill level (and continuing to improve) over the last 4-5 years is testing applications built on SOA. I’ve worked with both SoapUI Pro and SOAPSonar as my primary testing tools. I’ve created and executed operational + inter-operational tests based on what I was testing and the way in which the back-end service would be used (or be called) in a production environment by actual users. I have more knowledge and skill than my colleague testing applications built on SOA – testing applications built on SOA is something he’s very interested in and wants to improve his skill level.

We’ve agreed to share and exchange knowledge, test approaches, technical knowledge regarding different tools & how to use them, the different types of applications we’ve tested, and documentation we’ve created to aid in test setup & execution over the next few weeks. Of course there is the intersection where SOA based tools may be used to do performance tests.

Looking forward to the learning curve ahead!

 

 

Dec 30

Testers Signing Off

Not too long ago while working on a project at a company a certain practice that was common and expected of testers caught my attention, not because it was interesting but because it made no sense to me.  This practice was Software Testers “signing off” on a project or fixes.  Why would I sign off on a product or fixes for defects? I’m not a Product Owner, I’m a Software Tester.  I test (explore, discover, investigate) in order to provide the product owners/stakeholders valuable information about the state of the product so that they can make informed decisions about how to proceed with product delivery.  A blog post I had read about a year ago by Michael Bolton immediately came to mind (http://www.developsense.com/blog/2010/05/testers-get-out-of-the-quality-assurance-business/)

As I dug deeper to understand why this was being done and where it all started, I had a few conversations with Michael and he was kind enough to help me out and lead me down the path to find some of the answers.  He suggested that instead of viewing and presenting the task of tester sign off to management as wrong, figure out the goal and alternatives will arise.

This is exactly what I did and soon discovered that what management believed in regards to software testing – its goal, purpose, and role was quite different from what I believed and practiced. Management believed that the QA Team (or Test Team as I prefer to call it) was responsible for assuring the quality of the product; that we were the gatekeepers of quality; that testers could and should assure the quality of work done by others.

At that moment I understood (understood being different from agreeing) why testers at the company were required and expected to sign off on product and defect fixes.  As Michael states in his post “it’s time for our craft to grow up.”