Software Metrics: What are they and how to track them

SoftwareMetrics

Software Metrics is a very controversial topic. A quick search on Google for good software metrics to track comes with lots of results. Ask any tester, manager or ask this question on online forums or raise it in tester gatherings and you will get a mix of answers. Some of them are mentioned below:

  • Test cases passed & Failed
  • Execution Productivity
  • Number of Defects Raised
  • Number of test cycles
  • Number of defects rejected by root cause
  • Number of duplicated defects
  • Cost of quality
  • Defect rejection ratio
  • Manual test case execution days
  • Number of code builds
  • Number of successful code builds
  • Duration of code builds
  • % availability of test environments
  • Average number of testers
  • Average manual test runs per tester per test execution day
  • Average manual runs per tester per effective test execution day
  • Total runs per tester per test execution day
  • Runs per manual test case
  • Number of defects by severity and total defects
  • Number of defects at end of UAT by severity
  • Automated test runs
  • Manual test runs
  • Total test runs
  • Effective test execution days
  • Total test cases for project
  • Number of manual test cases prepared during project
  • Number of test case writers
  • Effective test case preparation days
  • Defect Leakage
  • Return on Investment
  • Downtime log
  • Test Script Execution Time
  • Code Coverage
  • Requirement Volatility
  • Test case effectiveness
  • Defect Detection rate

Metrics can help guide your team and help you measure your team’s progress towards its goals. Metrics, on the other hand can also be a pit of wasted effort, numbers recorded for the sake of numbers.

Why do you need Metrics?

There are some really good and bad reasons to track metrics. Lisa Crispin and Janet Gregory write in their book Agile Testing

When Metrics are used as guideposts, telling the team when it’s getting off track or providing feedback that it’s on the right track, they’re worth gathering. Is our number of unit tests going up every day? Why did code coverage take a dive from 75% to 65%? It might have been a good reason – maybe we got rid of unused code that was covered by tests. Metrics can alert us to problems, but in isolation they don’t usually provide value.

Before you try to decide what metrics to collect, you should ask yourself this question. What am I going to do with this information? Some reasons why you might need to collect metrics are as follows:

  • To help estimate for future projects based on past experience.
  • To monitor progress within a programme or project.
  • To identify training needs of individuals or groups.
  • To estimate cost to completion.
  • To monitor progress of staff.

Identify the reason first and then decide what metrics you need to collect. Lastly, decide how to collect this information.

If your goals are measurable, the measurements you need to gather to track the metrics will be obvious.

Lean Measurements and ROI:

In their book, Implementing Lean Software Development: From Concept to Cash,Mary and Tom Poppendieck, go into detail about how one can apply lean initiatives to testing and developments. They explain that fundamental lean measurement is the times it takes to go “from concept to cash”, from a customer’s feature request to delivered software. They call this measurement “cycle time”. The focus then is on the team’s ability to “repeatedly and reliably” deliver new business value. The team tries to continuously improve their process and reduce cycle time. Another lean measurement used is financial return. If a team is developing a profitable product, it needs to understand how it can achieve the most profit. Even if there is no profit involved, a team should look into the Return on Investment to make sure its delivering best value. (1)

 This also echoes my point made above about knowing how to delight your customer and knowing what the ROI of this activity is. Don’t forget there is a cost overhead in collecting metrics:

  • Tooling and code manipulation to collect metrics.
  • Reporting of metrics to parties.
  • Analysis of metrics – to understand the impact points.

 Your activity of capturing metrics should pass the test of economics, wherein the benefits derived are far greater than the cost expended.

What Not to do with Metrics:

Metrics that involve the whole team are more likely to drive you towards success than are measures confined to isolated roles or groups. Metrics can also be used in a terrible way. One of these is to use them as a basis for an individual team member’s performance evaluation. One of my colleagues who works for a test consultancy in India was telling me about one of her previous work experiences where the company judged testers by the number of defects they found.

How does this help them do their jobs better? Bug counting results in the generation of meaningless metrics. I’ve raised 40 bugs in a half day of testing once; the code was still working, it just needed lots of tidying up before being released. On a different project, I raised two bugs in a single day of testing, each one of which on its own would have been sufficient to halt testing and forced developers back to rework things. When Microsoft released Windows 2000, there were over 65,000 known issues (bugs), yet is was a very successful product. IBM released an ALGOL compiler in the 1960s with one bug: it wouldn’t load. In short, not all bugs are equal.

Similarly, is it safe to say that a development team that produces a higher number of defects per lines of code is doing a bad job? Or a team that finds more defects is doing a good job? Even if the thought holds up, how motivating is it for a team to be whacked over the head with numbers? (2)

Some popular Metrics to Track:

 

  • Cyclomatic Complexity:

Fools ignore complexity. Pragmatists suffer it. Some can avoid it. Geniuses remove it.
– Alan Perlis, American Scientist

McCabe’s cyclomatic complexity is a software quality metric that quantifies the complexity of a software program. Complexity is inferred by measuring the number of linearly independent paths through the program. The higher the number, the more complex the code. Measurement of McCabe’s cyclomatic complexity metric ensures that developers are sensitive to the fact that programs with high McCabe numbers are likely to be difficult to understand and therefore have a higher probability of containing defects. The cyclomatic complexity number also indicates the number of test cases that would have to be written to execute all paths in a program.

  •  Code Coverage:

Another traditional metric to see how much of the code is exercised by tests. A lot of commercial and open source code coverage tools available which can be integrated into the build pipeline so you will immediately know if the coverage has gone up or down. One thing to note here is that these tools only measure the coverage of the code you have written. If you have missed some functionality, then this will not be spotted by these tools. You can have 90% of code coverage but you can also be missing 10% of functionality. TDD can help with that. (3)

  •  Defect Metrics:

Lots of metric combination can be found in this category. An important one for me is the number of defects found in production. This will tell you how your team did in the last release but it won’t tell you how good you are progressing in the current one. On the other hand it will give you an indication on what processes to change to reduce the number of defects.

  •  Iteration Metrics:

  1. How much work has the team completed
  2. How much is left to do
  3. Iteration burndown
  4. Estimated vs Actual time taken for tasks.
  • Automation and Manual Test Pass Rate:

Track the number of tests at each level. Unit, Functional, Story Level, GUI level, Performance etc. Although we get happy seeing the number of tests go up, but the numbers on their own are not enough. If you have a 1000 tests but they only give you 10% coverage, then this is not a good position to be in.

Further Reading:

Cem Kaner has provided a framework for understanding and rethinking software metrics. Read the following two short papers to know almost everything about software metrics.

  1. Software Engineering Metrics: What do they measure and how do we know
  2. The Darker Side of Metrics

References:

(1) – Implementing Lean Software Development: From Concept to Cash

(2,3) – Agile Testing

 

Read More

Coding

Should Testers know how to code?

I was actively involved in recruitment of testers for Trainline.com in 2012 and came across an age old discussion once again which made me write about this topic. Should Testers know how to code?

Adding to this, there was also a statement made by Adam Goucher at CAST2012..

This is the last generation of testers that don’t know how to code

In my opinion, it totally depends on the situation. For example, in companies like Microsoft and Google, virtually all testers must know some form of coding. There are still companies out there which only require manual testers. If coding know how was not a requirement, I would also prefer to hire a tester who has critical thinking, analytical, investigative skills, good communication, understanding of risk and knowledge of common areas where bugs tend to hide as compared to someone who can just write C#.

 I also agree with Cem Kraner’s view below:

Testing within a business involves at least two types of knowledge: knowledge about testing and knowledge about the type of application under test. Much of the PR about the need for “software development engineers in test” has come from software companies. For them, software is the subject matter. For them, expecting someone to understand code is like a bank expecting someone to understand financial statements. It is unfortunate that the software publishers and software service providers have been speaking with a louder megaphone than the other industries: the result has been that some non-software companies are changing recruiting standards to look for people with stronger software knowledge instead of stronger industry (e.g. banking) knowledge.

Vacancies for “Developers in Test” or “QA Developers” are coming out more and more, especially in Agile environments. We need to remember, automation testing can never replace the need to do manual testing. There has to be a balance between the two. Skilled testers are adaptable people and should not be threatened by test automation. It’s a valuable skill to have and it’s not like learning how to code will make you forget manual testing. Secondly, the more the tester knows about coding, they’ll be able to do their jobs better, and the more career opportunities they will have (more on this later – I have some stats to prove this). Additionally, talking to developers in a language they can understand, writing clear bug reports which put your message across correctly and quickly has to be a good thing, right?

As a recruiter, you also need to use a common sense approach. I have seen companies that seem to use a standard list of demands, that they once created, and keep on using because they have been told it is a good list by the consultant they hired to create it, or they see other companies ask for the same skills, so they assume they are vital. You should look at the job spec, the role of a tester in your organisation and your team structure. What does a tester’s role involve, instead of what others testers in your company do/can do. Sometime you may find that the ability to write code is desirable instead of a requirement. If it is a requirement, state the level of familiarity with coding that you require. This way you will avoid putting off good candidates who will just be overwhelmed by the long list of requirements. Here at the Trainline, I have amended our requirements in job specs to be inline with the role of a QA Developer.

Lastly, what all programming skills should a tester have? This totally depends upon the job requirement and differs from one company to the other. Some positions are for automated testers and for this you require a good set of programming skills to create test frameworks. A recommendation for a manual tester or a tester thinking of improving their programming skills would be to:

  • Learn enough in any language (look at your target market or see research below) so you can write some test automation helper code, create classes, make asserts etc.
  • Learn enough of general programming concepts so that you can write code for typical easy programming exercises available on the internet.
  •  Once you have mastered this, drill deeper into object oriented design skills/concepts and create a mock test framework testing a website or an application.
  •  Also attend tester gatherings and talk to other testers/managers and see what skills they seek in a tester.

Now, let’s have a look at testing job specs taken from 2012. This research has been done by a UK based recruitment company.

Percentage of jobs that require Good/Exceptional coding skills:

Q1/Q2 2012 – 63%

Q3/Q4 2012 – 84%

Breakdown of languages required:

  Q1/Q2 2012  –  28% – Java, 26% – C#, 23% – Ruby, 16% – C, 7% – PHP

Q3/Q4 2012  –  39% – Java, 35% – C#,  8% – Ruby, 12% – C, 6% – PHP

The remaining  37% in Q1/2 2012 and 16% in Q3/4 2012 were a mixture of “we don’t need someone with any coding experience”, or “we purely need a Manual tester.”

Research also shows that salaries for testers with exceptional coding abilities have risen by £3500 per annum in the past 6 months.

What this means:

The research above suggests there are 21% more people being hired in the past 6 months alone with good to exceptional coding knowledge than without, however, there is clearly still a requirement for Manual testers.

Java is a still the main scripting requirement but Ruby has become increasingly popular.

Top Test Automation Technologies are:

  •   Selenium, including SeleniumRC and Webdriver
  •   QTP
  •   XUnit frameworks such as JUnit, NUnit, TestNG, etc
  •   BDD tools like Specflow/Cucumber
  •   Watir or Watin.

The above results are for the UK. A similar study was done in the US not long ago and you can read the details about that here.

Final Words:

Adam Goucher is probably right. But for now, manual testing and manual testers are here to stay. As more and more companies move towards the “Developer in Test” mentality, I see future testers taking up coding, especially if they want to work for the best companies and not to forget, these roles pay considerably more as compared to manual testing roles.

Read More

Handling pop up windows in Webdriver

In WebDriver, testing popup windows involve switching the driver to the popup window and then running the corresponding actions. Steps involved are:

  1. Identify the popup or new windows
  2. Switch to the new window
  3. Carry out some tasks
  4. Switch back to parent window

You can use the “WebDriver.SwitchTo().Window()” method to switch to a window with a known name. If the name is not known, you can use “WebDriver.WindowHandles()” to obtain a list of known windows. You may pass the handle to “SwitchTo().Window()”

You can usually get the window name by looking at the page source and checking for the value of the <title> tag. This name can then be passed to “SwitchTo().Window()”. For example:

// This method take in a name of a window as a parameter, and then switches to that window.
 
public void SwitchToWindow(string windowName)
{
 
_driver.SwitchTo().Window(windowName);
 
}

If you do not know the name of the window, you can use a different approach. For example.

 ReadOnlyCollection&lt;string&gt; handles = _driver.WindowHandles;
 
            foreach (string windowHandle in handles)
            {
               string newWindowName = _driver.SwitchTo().Window(windowHandle).Title;
               Console.WriteLine("Window Hadle is: " + windowHandle + " and Window Name is:
               " + newWindowName);
}

The above will print out the window handles and window titles of all open windows. You can then use the newly discovered window name in the SwitchToWindow function described above or use a new function like below:

 public void SwitchToWindow(string windowName)
        {
           string parentWindowHandle = _driver.CurrentWindowHandle;
 
            IWebDriver newWindow = null;
 
            ReadOnlyCollection&lt;string&gt; handles = _driver.WindowHandles;
 
            foreach (string windowHandle in handles)
            {
                newWindow = _driver.SwitchTo().Window(windowHandle);
 
                if (newWindow.Title.Equals(windowName))
                {
                    Console.WriteLine("Switching to window with handle " + windowHandle +
                    " and window name" + windowName );
                    break;
 
                }
            }
        }

See the first line of code in the above function. It captures the current window handle. In case you want to switch to the parent window, you can simply do this:

 _driver.SwitchTo().Window(parentWindowHandle);

Read More

Do not expect everyone to understand testing!

As a tester, it is your job to tell others what you need in order to do your job properly. A tester is largely affected by the choices made by management and programmers. If other stakeholders are unclear about their plans or design a product which is hard to test, then this will affect your test plan. I’ts not that they do not understand or care about testing and software quality. They probably just do not understand what impact their actions can have on the test process. An important part of testing is to explain testing to other stakeholders.

I read somewhere that your explanations about testing are like a flu vaccine, healthy and not painful, but the effect wears off, so you have to do it over and over again.

Read More

Never dream or wish to have absolute control over the release of a product.

When testers control a release, they must also bear full responsibility of the quality of a product. What I have seen happen is that if any bugs are missed by the testers and make their way to production, the rest of the team (in most cases) will blame the testers. Why did the tester release such a buggy product? On the other hand, if a tester delays the release then they will come under pressure for being perfectionists or quality fanatics.

The people who control the project are the best suited to hold the responsibility to release the product.  So if you are ever given the authority to control a release, insist on sharing that authority with others in the team.

Read More

Webdriver – How to Write Automated Tests Using Page Object Design Patterns

Look at this code

Site.Click("crt1h-yth");
Site.Click("chk-formb1");
Site.Click("pay_02");

and compare it with this..

Shop.AddToCart();
Shop.Checkout();
Shop.MakePayment();

The functionality is the same, but the way to present this is so much different. This is where page objects come into the fold and this blogpost is just about that.

So what are page objects ?

Page Object is a Design Pattern which has become popular in test automation for enhancing test maintenance and reducing code duplication. Page object is a language neutral pattern for representing a complete page or a part of a page in an objected oriented manner. We use them to model the application’s user interface. Page objects expose methods that reflect things that a user can see and do on a page. It also hides the details of telling the browser how to do those things. In short, page object encapsulates behaviour of a page.

Your tests then use the methods of this page object class whenever they need to interact with that page of the UI. The benefit is that if the UI changes for the page, the tests themselves don’t need to change, only the code within the page object needs to change. Subsequently all changes to support that new UI are located in one place.

So why do page objects? The main reasons are

  • Maintenance
  • Readability of scripts
  • Reduced or eliminated duplication
  • Reusability

Consider the following example, which does not use page object design.

namespace SeleniumTests
{
[TestFixture]
public class LinkedInTest
{
private IWebDriver driver;
private string baseURL;
 
[SetUp]
public void SetupTest()
{
driver = new FirefoxDriver();
baseURL = "http://www.linkedin.com/";
 
}
 
[TearDown]
public void TeardownTest()
{
driver.Quit();
}
 
[Test]
public void LoginAndLogoutOfLinkedIn()
{
driver.Navigate().GoToUrl(baseURL + "/");
driver.FindElement(By.Id("session_key-login")).Clear();
driver.FindElement(By.Id("session_key-login")).SendKeys("test@123.com");
driver.FindElement(By.Id("session_password-login")).Clear();
driver.FindElement(By.Id("session_password-login")).SendKeys("awesomePswd");
driver.FindElement(By.Id("btn-login")).Click();
Assert.assertTrue(driver.isElementPresent("inbox button"),
                                "Login was unsuccessful");
driver.FindElement(By.LinkText("Sign Out")).Click();
}
 
}
}

There are a few problems with this approach:

  1. There is no clear separation between test methods and the locators of the application (Ids or LinkText locators that I’ve used above). They are all in a single method. If the application changes its identifiers or layout, then the tests must change.
  2. Imagine a scenario of multiple tests which require the use of this login functionality (There can be various scenarios to do with testing login functionality). The same login code will be repeated again and again in each test. Any change in UI will mean that all tests will have to be modified.
  3. The above code is not very readable, or maintainable, has duplication and is not reusable. (Exactly the issues addressed by Page Object Design).

 

By applying the Page Object design technique, I can rewrite the above test in the following way.

Firstly here is the flow that I will be automating using C#, NUnit and Webdriver:

  1. Navigate to the LinkedIn Homepage
  2. Enter Username
  3. Enter Password
  4. Click Login
  5. Assert Some Text is appearing on the HomePage after login
  6. Logout
  7. Assert that I have signed out

 

Below is the Page Object for the Login-In Page:

using System;
using OpenQA.Selenium;
 
namespace SeleniumTests
{
    public class LoginPage
    {
        protected readonly IWebDriver WebDriver;
 
        public LoginPage(IWebDriver webdriver)
        {
            this.WebDriver = webdriver;
 
            string title = WebDriver.Title;
 
            if (!title.Equals("World's Largest Professional Network | LinkedIn"))
            {
                throw new InvalidOperationException("This is not the Login Page. Current page is: "
                                                    + title);
            }
        }
 
        /*
        A property to enter username on login page
     */
        public string EnterUsername
        {
            set
            {
                IWebElement usernameField = WebDriver.FindElement(By.Id("session_key-login"));
                usernameField.Clear();
                usernameField.SendKeys(value);
            }
        }
 
        /*
        A property to enter password on login page
     */
        public string EnterPassword
        {
            set
            {
                IWebElement passwordField = WebDriver.FindElement(By.Id("session_password-login"));
                passwordField.Clear();
                passwordField.SendKeys(value);
            }
        }
 
        /*
        A method to click the login button on the page
     */
        public void ClickLogin()
        {
            IWebElement loginButton = WebDriver.FindElement(By.Id("btn-login")); // find the login button
            loginButton.Click(); // click on the login button
        }
    }
}

Here is how the Page Object for the HomePage looks like:

using System;
using OpenQA.Selenium;
 
namespace SeleniumTests
{
    public class HomePage
    {
        protected readonly IWebDriver WebDriver;
 
        public HomePage(IWebDriver webdriver)
        {
            this.WebDriver = webdriver;
 
            string title = WebDriver.Title;
 
            if (!title.Equals("Welcome, Hasan! | LinkedIn"))
            {
                throw new InvalidOperationException("This is not the HomePage. Current page is: "
                                                    + title);
            }
        }
 
        /*
A method to logout of LinkedIn
*/
 public void Logout()
        {
            driver.FindElement(By.LinkText("Sign Out")).Click();
        }
    }
}

Finally, here is my LinkedIn login Test that consumes the two page objects:

using NUnit.Framework;
using OpenQA.Selenium;
 
namespace SeleniumTests
{
    [TestFixture]
    public class LinkedInTest
    {
        private IWebDriver _driver;
 
        [SetUp]
        public void Setup()
        {
            _driver = SeleniumHelper.GetSelenium();
        }
 
        [TearDown]
        public void TearDown()
        {
            SeleniumHelper.TearDownCurrentSelenium();
        }
 
        [Test]
        public void VerifyThatYouCanLoginAndLogoutOfLinkedIn()
        {
           // Create an instance of the Login Page
            LoginPage login = new LoginPage(_driver);
 
            //Enter Username
            login.EnterUsername = "HasanAziz@test.com";
 
            //Enter Password
            login.EnterPassword = "XYZ123";
 
            // Click Login
            login.ClickLogin();
 
            //Create an instance of the HomePage
            HomePage home = new HomePage(_driver);
 
            //Assert that some text is displayed on the Home Page
            Assert.IsTrue(_driver.PageSource.Contains("LinkedIn Today:"));
 
            //LogOut from LinkedIn
            home.Logout();
 
            // Assert
            Assert.IsTrue(_driver.Title.Equals("Signed Out | LinkedIn"));
        }
    }
}

As you can see from above, this is much more clearer now. The comments in the tests are really not needed as they are just repeating what the code is doing (Something I mentioned in my blogpost “Comments & Bad Comments” earlier, but I have added these here just so you could understand whats going on.

There is alot of flexibility when it comes to designing page objects. I could also use techniques like inheritance, function overloading or any other Object Oriented programming concept, and  instead of having the browser name and LinkedIn URL set as hardcoded values, I could have picked them up from a config file etc. Lets take another example of the eBay homepage. You can automate it using 1 Page Object if you wish, but ideally you can have 5-6 Page objects for the home page only.

Although the flexibility is present, there are a few basic rules that you should adhere to for mainiting your code:

  1. Do not create page objects at once, do only what you need at this given time. You can spend weeks in trying to create page objects for your whole application and this would be a waste of time. Your page objects will grow when new requirements come in which inturn will require new test scripts.
  2. Asserts do not belong in page objects, they belong in test scripts. Page objects only interact with the page, they do not make the decision of whether something is passing or failing.
  3. An exception to the above rule is that there should be one single verification within the page object and that is to verify that the page and any important elements within the page were loaded correctly. This verification should be done while instantiating the page object. In my example above, both the LogInPage and HomePage constructors check that the expected page is available and ready for requests from the test.

Further Reading:

  • There are other Design Patterns, some use Page Factory for instantiating objects. More on this here 
  • Some Information on Page objects can be found on code.google.com
  • Very Useful test design considerations can be found on the Selenium HQ site

Books on this topic:

  • I recommend xUnit Test Patterns: Refactoring Test Code by Gerard Meszaros – Almost 900 pages of patterns that can be used for automation and unit testing.
  • Refactoring: Improving the Design of Existing Code by Martin Fowler and Kent Beck. Its developer centric but a very good read.
  • Design Patterns: Elements of Reusable Object-Oriented Software – This is on my reading list but I have heard very good reviews about it.

Read More

Webdriver – Drop Down boxes, Check Boxes & Filling in forms

Here is how you can select values in a drop down box: The drop Down has an ID called “Domain” and values Yahoo, Virgin, AOL.

SelectElement select = new SelectElement(_webdriver.FindElement(By.Id(“Domain”)));

Select.DeselectAll();

select.SelectByText(“Yahoo”);

This will deselect all options, and then select the option withthe displayed text of “Yahoo”.

You can select Radio Boxes and Check Boxes by the following command:

driver.FindElement(By.Id(“SingleJourneyCheckBox”)).Click();

You can find out whether this checkbox/radiobox is alrady selected by this command.

IWebElement checkbox = driver.FindElement(By.Id(“SingleJourneyCheckBox”));

if(checkbox.Selected)

{

//do something here.

}

If you were filling out a form and  once you’ve finished, you probably want to submit it. One way to do this would be to find the “submit” button and click it:

driver.findElement(By.id(“submit”)).Click(); // Assume the button has the ID “submit”

Alternatively, WebDriver has the convenience method “submit” on every element. If you call this on an element within a form, WebDriver will crawl up the DOM until it finds the enclosing form and then calls submit on that. If the element isn’t in a form, then the NoSuchElementException will be thrown:

element.submit();

Read More

What is your Quality Philosophy?

I recently went to a Tester Gathering where this question was brought up. There were many answers to it. Some of these were:

  • Software should be stable.
  • The software does what it’s supposed to do from a functional perspective.
  • To deliver defect free code as best possible, accounting for schedule, risk and resources.
  • Quality software is reasonably bug-free, delivered on time and within budget, meets requirements and is maintainable.
  • To deliver what is needed, in a quality that is accepted by the client.
  • Test early and test often

This turned out to be a very controversial topic and we had a very good debate about it. We talked about Six Sigma, Total Quality Management and even discussed Deming’s 14 Point Philosophy ( I recommend reading his work, more on him can be found here. A brief introduction is below).

Dr. W. Edwards Deming was a statistician who went to Japan to help with the census after World War II. Deming also taught statistical process control to leaders of prominent Japanese businesses. His message was this: By improving quality, companies will decrease expenses as well as increase productivity and market share. After applying Deming’s techniques, Japanese businesses like Toyota, Fuji, and Sony saw great success. Their quality was far superior to that of their global competitors, and their costs were lower. The demand for Japanese products soared – and by the 1970s, many of these companies dominated the global market.

I used to often ask this question when conducting interviews. My purpose for asking this question was to know how the candidate thinks. I often had candidates recite texts memorised from books which lead me to ask clarifying questions to satisfy my curiosity.  Many used to get caught out as they did not think about the answers and just memorised some text. This allowed me to initially separate good candidates from the average ones.

Quality philosophy is also dictated by the circumstances. Sure there are basic ideas of what quality means but if you’re looking for a quality philosophy, then it also depends on the domain, the company management, the company culture, the teams and the project.  It can also include quality in:

  • How a project is planned, designed, implemented and reviewed.
  • Reporting statuses among departments and department heads.
  • How requirements are written so Developers can produce the design with ease
  • Documentation Standards
  • Testing
  • Release Management etc.

Churchill said: “They say no one is perfect. Than they say that practice makes perfect. I wish they made up their damn mind!”

Talking about perfection or getting things absolutely right – As much as we would like our software to be perfect, it is a futile attempt to make it so. There comes a point where the time and expense to make a program perfect gives diminishing returns. In my experience so far, it is impossible to deliver ‘perfect quality’. There is always this or that little bit that could be improved, simplified, re-factored or documented better etc.

I also strongly believe in the fact that the client/customer has final say over quality – if the client is satisfied, the quality is ‘ok’. If the client is happy, the quality is ‘good’. If I ask this question in an interview I expect to hear “customer” somewhere in the answer. You must seek to identify the customer, whether external or internal to the organization, then advocate for that customer with your QA skill set. The QA Engineer should be happy when they think the customer will be happy. We try to measure that “happiness” with metrics and quality gateways before release, but it is the customer feedback after release that counts. The quality of project management, documentation standards, requirements acquisition and testing, all must be in sync with the ultimate customer requirements regarding quality. The cost of producing the end result, including all interim steps, must reflect end user/customer requirements to quality.

As I said earlier, it is a very controversial question but one which has no correct answer. Like your favourite colour, or car, or your love for Justin Bieber, your quality philosophy is what makes you a unique individual. There is no “right” answer to this, but it might indicate as to what ways you think about such things.

So what is my philosophy I hear you say? Well it’s this:

  1. Know that nothing is perfect and never will be.
  2. Be a professional and passionate about what you do. Give it your best shot and try to do better every day.

This is valid whether it is about testing, development,doing business, life in general or just being human.

Read More

Webdriver Commands & Operations

 1. Opening a New page:

 driver.get(“http://www.google.com”);

WebDriver will wait until the page has fully loaded (that is, the “onload” event has fired) before returningcontrol to your test or script. It’s worth noting that if your page uses a lot of AJAX on load then WebDriver may not know when it has completely loaded. If you need to ensure such pages are fullyloaded then you can use “waits”. More on this later.

2. Locating UI Elements & Interactions with the page:

Imagine that you have an element displayed on a page with the following code

<input type=”text” name=”Password” id=”Password-ID”/>

You can find this element by using Any of the following commands:

Locate By ID:

This is the most efficient and preferred way to locate an element.

IWebElement element;

element=driver.findElement(By.id(“Password-ID”));

Locate By Name:

Find the input element with matching name attribute.

IWebElement element;

element=driver.findElement(By.name(“Password”));

Locate By XPath:

WebDriver uses a browser’s native XPath capabilities wherever possible. On browsers that don’t have native XPath support, webdriver provides its own implementation. This can lead to some unexpected behaviour unless you are aware of the differences in the various xpath engines.

 IWebElement element;

element=driver.findElement(By.xpath(“//input[@id=’Password-ID’]”));

 Locate By Class Name:

“Class” in this case refers to the attribute on the DOM element. Often in practical use there are many DOM elements with the same class name, so be very carefull when finding elements using this option. For example, lets say that you have the following html code in your browser:

<div class="cheese"><span>Cheddar</span></div><div class="cheese"><span>Brie
</span></div>

If you use the below code, it will only target the first element it finds, i.e Cheddar.

IWebElement element;

element=driver.findElement(By.ClassName(“cheese”));

But if you want to find multiple elements, you can use the command below:

IList<IWebElement> cheeses = driver.FindElements(By.ClassName("cheese"));

 Locate by Tag Name:

You can also locate an element by its DOM tag name.

For example, say you want to find an iframe which is displayed on the page with the following code:

<iframe src="..."></iframe>

The command you will need is this:

IWebElement frame = driver.FindElement(By.TagName(“iframe”));

 Locate By Link Text:

You can find an element by matching its visible text.

Say you have a link displayed with the following code.

<a href="http://www.thetrainline.com/Hotels">Hotels</a>>

The command you will need is this:

IWebElement cheese = driver.FindElement(By.LinkText(“Hotels”));

 Locate By Partial Link Text:

You can find an element by matching its partial visible text.

Say you have a link displayed with the following code.

<a href="http://www.thetrainline.com/BestFareFinder">Best Fare Finder</a>> 

The command you will need is this:

IWebElement cheese = driver.FindElement(By.LinkText(“Fare”));

When using all of the above commands to find elements, if no element can be found then a NoSuchElementException will be thrown.

3. What to do after finding an element:

So you have found an element, what would you like to do with it?

You can enter some text in the text field:

element.SendKeys(“some text”);

 Keep in mind that typing something into a text field will not clear it. Instead, what you type will be appended to what’s already there.  You can easily clear the contents of the text field or text area:

element.Clear();

You can click an element:

After finding the element using its link text or partial link text, you can click on it.

element.Click();

Read More