6. Test design techniques ←

Test design is a significant step in the Software Development Life Cycle (SDLC), also known as creating test suites or testing a program. In other words, its primary purpose is to create a set of inputs that can provide a set of expected outputs, to address these concerns:

What to test and what not to test
How to stimulate the system and with what data values
How the system should react and respond to the stimuli

Therefore, various techniques exist for test design and execution. It is understandably crucial to utilize some effective test design techniques since software development is getting more complicated. Generally, software testing design techniques help you write better test cases and optimize testing processes. It also helps reduce the time of executing test cases while escalating test coverage. In this post, I will discuss the overview of software testing design techniques.

The importance of test design techniques

Test design techniques are applied to satisfy the goals of every individual in software development projects, including testers. Although the main purpose is to ensure that the products meet the expectations of clients and their businesses, these techniques allow testers to execute the test effortlessly based on various risk factors. Here is a checklist of standards that a smooth testing process meets:

Gather information to understand users’ requirements
Derive all important business scenarios
Design test scenarios for every derived critical business scenarios
Assign all planned test scenarios to different test cases

Then, you will have to choose a test design technique for each requirement. At this point, if things are correctly implemented, you can make significant changes that affect your ROI extraordinarily.

Two of main advantages of techniques lie on their consistency and repeatability:

Possibility to reproduce a test:
At several testing phases, testing techniques are considered as a set of rules help ensure a minimum level of consistency. Testers, indeed, would work much more efficiently with a base, thereby reducing a significant amount of effort in later fixing.
Increasing of the found bugs:
Test design techniques can also be used as analytical tools. When applying techniques to elements, we often see problems in the definition of those elements (Overview of Test Design Techniques in Software Development).

Test design techniques are methods used to create effective and efficient test cases. These techniques help ensure thorough test coverage and increase the likelihood of finding defects in the software. Here are some common test design techniques:

Divides the input domain into groups (equivalence classes) and tests a representative value from each group. This helps ensure that if one value in an equivalence class works, the others are likely to work as well.

Equivalence Partitioning:

Step-by-Step Process of Equivalence Partitioning:

1. Identify the input field: Determine the input field to be tested, such as the username field in our example.

2. Define equivalence classes: Categorize the possible input values into distinct equivalence classes. Each class represents a set of inputs with similar behaviour or characteristics.

3. Determine representative values: Select representative values from each equivalence class. These values should cover the boundary conditions and critical scenarios within each class.

4. Create test cases: Generate test cases that cover each equivalence class. For example, one test case would include a valid username, another would include an invalid username, and a third test case would cover an empty username.

5. Execute the test cases: Run the test cases using the defined equivalence classes to validate the behavior of the system. Observe and record the results for each test case.

By following this step-by-step process, testers can effectively apply Equivalence Partitioning to streamline test case design and achieve comprehensive test coverage while minimizing redundancy (Equivalence Partitioning: Step-by-Step Process for Efficient Test).

Example of Equivalence Partitioning Technique.

Boundary Value Analysis:

Focuses on testing values on the edges or boundaries of input domains. Test cases are designed for the minimum, maximum, and values just inside/outside these boundaries. This is based on the assumption that errors often occur at boundaries.

Boundary value analysis is one of the best testing techniques in the software industry for testing software functions. It tests and detects if the software functions perfectly with the boundary values. The above technique tests values on, inside and outside particular boundaries. Testers detect major issues by analyzing the boundary conditions and ensuring the software works perfectly.

The benefits of boundary value analysis

Testers enjoy countless benefits by employing effective boundary value analysis, such as.

Wider test coverage: Since the test focuses on boundary values, a wider test scope is achieved by exploring values at the edges of input and output ranges. With effective boundary value analysis, testers ensure critical values are systematically tested, and defects are identified.
Better software quality: By testing the boundary conditions thoroughly, the overall quality of the software is enhanced. Since the test helps identify vulnerabilities and defects at critical points, proper software functioning is assured. The test paves the way for software to function reliably, correctly and consistently across different boundaries.
Identification of errors early: The best part of boundary value analysis is the early identification of errors in the development life cycle. Errors are more likely to occur at the boundaries, and testers test values at the boundaries. Such errors are addressed instantly before it progresses in the system.
Reduce risk: Boundary value analysis lessens the risk connected with boundary-related flaws. Concentrating on edge cases and critical scenarios makes the possibility of issues or failures associated with the boundary values less risky.
More reliable: By testing boundary values, the tester verifies if the software can handle edge or extreme cases without unpredicted behavior or errors. This way, the system’s reliability is increased and authenticated according to real-world scenarios.
Agreement with standards: Many organizations and industries mandate agreement with regulations and specific standards. By employing effective boundary value analysis, software testing is defined against the boundaries of such standards, making sure the organizations meet the agreed standards.
Cost and time efficiency: Testers arrange testing efforts based on boundary values, and testers allocate resources also perfectly. With effective boundary value analysis, testers can reduce unwanted tests, optimize the testing process, and concentrate on a targeted approach for detecting issues.
Customer satisfaction: By testing the boundary conditions thoroughly, the testers ensure that the software works perfectly across various outputs and inputs. Such a scenario leads to a better user experience and fewer errors as the user inputs values. Hence, overall customer satisfaction is achieved with effective boundary value analysis.

How to perform boundary value analysis

Sequences of steps are involved in effective boundary value analysis, as mentioned below.

Finding boundaries: The first step is to understand the software’s specifications or requirements and determine boundaries as input and output values. The boundaries may possess values of any type, such as maximum or minimum values that are above or below boundaries and values accurately at boundaries.
Partitioning input space: The next step is partitioning the input values into meaningful ranges. For example, if the values range from 1 to 100, then portioning them as values less than 1, values between 1 and 100 and values greater than 100 is necessary.
Picking the test cases: Test cases that exercise the boundaries of every partition are picked. Concentrate on boundaries such as lower and upper, which often leads to issues. To ensure comprehensive testing always include invalid and valid test cases.
Execution of test cases: The next step is the execution of test cases by inputting the value at boundaries and analyzing the system’s behavior. Make a note of the results of all test cases.
Validation of expected results: Based on defined boundaries, determine the predictable results for each test case. Compare the actual and expected results to know the issues or defects.
Document and report the issues: Document any anomalies during the test with a clear description, including the input values, actual results, expected results etc. Report the issues to the development team for further processing.
Refine: Look at the results of boundary value analysis to see if there are any lags or areas to improve. Refine the cases and repeat the process to achieve comprehensive coverage (What is Boundary Value Analysis?).

Example of Boundary Value Analysis Technique.

Decision Table Testing:

Represents combinations of inputs and conditions with corresponding actions in a table. It helps ensure that all possible combinations of conditions and inputs are tested.

Let's consider an example.

Consider this from a vehicle insurance Functional Spec:

If the age of the driver is greater than 25 and the driver is female, offer a discount of 10%.

In Equivalence Partitioning and Boundary Value Analysis, the focus was on single or specific inputs and probably more focused on UI. The techniques help in designing test cases for the individual fields – for example, Age individually and Gender individually in the above example.

However, the combination also matters in the above example – i.e. Age and Gender together. There are 4 combinations that arise – Female driver aged 26(>25), Female driver aged 24 (<25), Male Driver aged 24(<25), Male driver aged 26(>25). The first is a positive test case with an expected result of 10% discount and the rest are negative test cases with a discount of 0%.

Consider the following from another specification – For a Female driver aged >25, no accidents, SUV, with children – offer a 20% discount. Here the number of combinations comes to 16, with 4 fields that can be evaluated as True or False.

When the number of decisions in a specification increases, it is difficult to keep track of all combinations. The Decision Table technique comes in handy for such situations.

A Condition is an input field that can be evaluated as True or False , eg: Age < 25 – True or False. A decision is the expected result for a combination of conditions. Specifications are full of business rules which are nothing but decisions based on a combination of inputs. Individual conditions are usually simple to understand but the combinations can turn out to be complex. A decision table ensures every combination has been captured systematically.

A decision table lists all the input conditions that can occur and all the actions that can arise from them. These are structured into a table as rows, with the conditions at the top of the table and the possible actions at the bottom.

Let us look at a Car Rental Example:

The specification says: If Age is over 23 and the person has a Clean driving record, supply rental car, else reject.

There are 2 conditions: Age and Clean driving record

And 2 Actions: supply rental car, do not supply rental car

2 conditions = 4 rules.

We list the conditions in the first column, followed by the Actions.

Then, we start evaluating the conditions to be True or False and for the combinations that arise, the appropriate action.

With 2 conditions, there are 4 combinations (two to the power of the number of things to be combined), so if three conditions – the number of combinations is 8(2^3), and for 4 conditions, the number of combinations is 16 (2 ^ 4).

Decision Table technique is used after Equivalence Partitioning and Boundary Value Analysis have been applied (ISTQB – What is Decision Table Testing in Software Testing).

State Transition Testing:

Applicable for systems with different states. Test cases are designed to verify transitions between states, ensuring that the system behaves correctly during state changes.

Example of Boundary Value Analysis Technique.

Let’s explore a real-world example to understand state transition testing better. We’ll explore how state transition testing can be applied to an Automated Teller Machine (ATM) system.

The ATM system undergoes various states and transitions based on user interactions:

This technique helps us uncover defects, verify correct transitions, and enhance the overall reliability of the software.

We can create effective state transition diagrams and derive meaningful test cases by understanding a system’s states, events, and actions. Incorporating state transition testing into our testing process contributes to delivering high-quality software that meets user expectations (Software Testing: State Transition).

Use Case Testing:

Tests the system's functionality based on its use cases. Test cases are derived from real-world scenarios to validate that the software meets user requirements.

Example of a Use Case.

Assuming the use case of an online shopping system between a customer and seller through an online shopping application. Here’s what the Use Case Diagram would be like.

The above use case diagram depicts an Online shopping system consisting of two main actors: Customer and Seller. Six use cases are identified to explain the high-level functionality of the online shopping system.

A customer can view products, add products to cart, checkout, and place orders. A seller can view products, update existing product details and its price, and add/remove products from inventory.

Here, considering an online shopping scenario initiated by a customer, the following are the details of the elements involved.

Actor: A Customer.
Trigger: The customer wishes to purchase the items via an online shopping app.
Precondition: Network should be available, and the app should be logged in.
System: Online Shopping App.
Goal: Complete purchase successfully.
Stakeholder: App owner.

Best Practices to Create a Use Case.

Here are some of the best practices to follow for creating a use case:

In a use case, each step should exhibit an action.
The number of use cases should be somewhere between five to ten for better understanding and less complexity.
Avoid using if statements. if statement comes with an else statement, and employing this would create complexity and confusion. Instead, use verbs like validate and verify. (The system verifies the login is valid)
Use case steps should be simple, short, and easy to understand.
Do not include UI while creating use cases for any system. UI is bound to change in the Agile framework, and we would not be interested in updating the use case for any UI update.
Use cases should show a high level of representation and should not include minor things of the system.
Use cases should only explain the functional part of the system and should not include the design.

Advantages of Use Case.

Creating a use case has many benefits, some of them are:

Use case diagrams fall under the functional requirement documentation technique. It is used for defining and organizing functional requirements in a system.
Use cases help ensure that the correct system is developed by capturing the requirements from the end user’s point of view.
It is used to model the basic flow of events in a use case.
It is used to visualize software design at a high level. Large complex applications can be split into small modules.
It is used to capture the core functionalities of a system.
It is presented in a simple and non-technical language so that it is easily understood by stakeholders, customers, developers, and testers.
As Use cases focus on the users of the system, not the system itself, real system needs can be identified earlier in the design process.
Use cases could serve in later stages as a base for test case creation.

(Use Case vs Test Case: Core Differences)

Pairwise Testing (Combinatorial Testing):

Generates test cases that cover all possible combinations of input parameters while keeping the number of test cases manageable. It is particularly useful when dealing with a large number of input combinations.

This method is useful for creating tests for applications that include multiple parameters (Pairwise Testing Guide: How To Perform Pairwise Testing).

For example, if there is a software app with various inputs and various possible settings for each input, it is not practical to check all possible combinations for all of these factors. Thus, instead, a subset of combinations is created that satisfies all factors. As a result, a permutation and combination method simplifies system testing.

Pairwise testing is efficient in the later stages of development. It is usually fulfilled by general functional testing. Before using pairwise testing, it is important to ensure that the main functionality works with default settings properly.

It should be used only in the stable work of the app when the current tests are already losing their effectiveness (Pairwise Testing).

Let’s follow simple steps to create test cases using the Pairwise testing technique.

Example of Pairwise testing technique.

The following examples are concluded the method; in this example, we have three parameters which can have the following values as A, B, C; 1, 2; X, Y which means that the number of combinations 3 x 2 x 2 = 12. If we want to test just 2-pairs or pairwise testing which covers every possible combination of two parameters, there should be 6 combinations. The highlighted test cases TC2, TC3, TC5, TC8, TC10, TC11 are not necessary (Combinatorial Testing).

There are many TOOLS that adopted and integrated pairwise testing as one of the test generation methods.

In today’s complex system of multiple devices, browsers, Operating systems and so many other variables, it may take years to test an application with a single input parameter. Pairwise can help us create better test cases with optimum effort without exhausting the system.

There are many TOOLS that adopted and integrated pairwise testing as one of the test generation methods.

Orthogonal Array Testing:

Orthogonal Array Testing (OAT) is software testing technique that uses orthogonal arrays to create test cases. It is statistical testing approach especially useful when system to be tested has huge data inputs. Orthogonal array testing helps to maximize test coverage by pairing and combining the inputs and testing the system with comparatively less number of test cases for time saving.

Similar to pairwise testing, it selects a subset of combinations from a larger set, aiming to achieve maximum test coverage with a smaller number of test cases.

For example, when a train ticket has to be verified, factors such as – the number of passengers, ticket number, seat numbers, and train numbers have to be tested. One by one testing of each factor/input is cumbersome. It is more efficient when the QA engineer combines more inputs together and does testing. In such cases, we can use the Orthogonal Array testing method.

Example of OAT technique.

A Web page has three distinct sections (Top, Middle, Bottom) that can be individually shown or hidden from a user

No of Factors = 3 (Top, Middle, Bottom)
No of Levels (Visibility) = 2 (Hidden or Shown)
Array Type = L4(23)

(4 is the number of runs arrived after creating the OAT array)

If we go for the Conventional testing technique, we need test cases like 2 X 3 = 6 Test Cases

If we go for OAT Testing we need 4 Test cases as shown below:

(What is Orthogonal Array Testing? (Example))

Orthogonal Array Testing (OAT) stands out as a powerful and efficient technique in the realm of software testing and beyond. Its ability to optimize test case selection while maintaining comprehensive coverage makes it a valuable addition to any quality assurance toolkit. OAT’s advantages in terms of efficiency, early defect detection, and resource optimization provide clear benefits in cost reduction and enhanced product quality.

However, it’s crucial to acknowledge the challenges and limitations it presents, such as the initial setup overhead and the potential for missing certain interactions. Nevertheless, when applied judiciously, OAT can significantly improve testing processes and contribute to the development of reliable, high-quality software (What is Orthogonal Array Testing? | Why & How to Perform?).

Random Testing:

Involves selecting random inputs for testing. While it may not be exhaustive, it can help uncover unexpected issues and is particularly useful in exploratory testing.

Working Random Testing:

Step-1: Identify Input domain

Step-2: Select test inputs independently/randomly from the input domain

Step-3: Test the system on these inputs and form a random test set

Step-4: Compare the result with system specification

Step-5: If the Report fails then take necessary action.

The below image represents the working of Random Testing more clearly.

Types of Random Testing

Random input sequence generation: It is also known as Random Number Generator (RNG) in which a random sequential number or symbols is being generated which cannot be assumed during the random selection.
Random sequence of data inputs: In this, all the data are selected randomly for the inputs which can be used during the testing.
Random data selection from an existing database: The record where all the data are available from that record only the data can be selected for testing afterward no additional data cannot be added which are not available in the record (Random Testing in Software Testing).

Random testing is a valuable and intriguing testing technique that brings a fresh perspective to traditional testing techniques. It helps identify unexpected defects, vulnerabilities, and edge cases that other testing approaches might miss.

This will definitely not replace the structured testing methods.

Instead, it serves as a complementary strategy to improve test coverage and increase the chances of discovering hidden bugs. Its unpredictable nature allows testers to simulate real-world scenarios and challenge the software in ways that may not have been expected during the other testing methods (Random Testing In Software Testing – An Overview).

Error Guessing:

Relies on the tester's experience and intuition to identify potential error-prone areas in the software. Test cases are designed based on the tester's expectation of where defects might occur.

The error-guessing technique is an intuitive and creative approach used in software testing to uncover potential defects or errors.

Testers use their expertise, domain knowledge, and intuition to foresee and detect faults in software systems. This goes beyond formal test cases and planned scenarios, allowing testers to think like users and predict how the product will act. This testing adds a human element to testing and stimulates critical thinking, improving system quality by detecting problems that could otherwise go undetected.

Using the error-guessing approach in software testing is a creative procedure that allows testers to rely on their intuition and knowledge to resolve errors that could be found while testing your software. Here’s a step-by-step guide to implementing this technique:

Step 1: Understand the System

Familiarise yourself with the software system being tested, its features, user interactions, and documentation to ensure a successful testing procedure. This knowledge will act as the foundation for your testing procedure.

Step 2: Tap into your Experience

Use your previous testing experience and domain expertise to guide your intuition during the testing procedure. Consider similar systems or projects and any faults or flaws you’ve experienced. This experience will help guide your intuition.

Step 3: Put Yourself in the User’s Shoes

The most important idea is to consider the end user’s point of view and identify probable scenarios that could lead to errors. Consider edge situations, unexpected behaviours, or unusual use circumstances that could lead to issues.

Step 4: Think Outside the Box

Out-of-the-box thinking can help identify hidden flaws in software, such as if a user performs a specific action in a particular sequence or if any unusual combinations of inputs could trigger errors. It can also reveal hidden flaws that formal test cases may not address.

Step 5: Document and Communicate Findings

Document your judgments and keep a record of your theories. Communicate your findings to the development team, testers, or stakeholders to ensure they know of potential hazards.

Step 6: Execute the Testing Approach

Design and run tests based on assumptions to implement your error-guessing strategy. Run the software system and document any faults or deviations from intended behaviour.

Step 7: Analyze Results and Iterate

Examine the outcomes of mistake-guessing tests to refine your strategy. Learn from each testing cycle and apply fresh ideas and observations to subsequent error-guessing sessions (What is Error Guessing Technique in Software Testing?).

Example of Error Guessing technique.

Error Guessing is one of the popular techniques of testing, even if it is not an accurate approach of performing testing still it makes the testing work simple and saves a lots of time. But when it is combined with other testing techniques we get better results. In this testing, it is essential to have skilled and experienced testers (Error Guessing in Software Testing).

User Story Testing:

User story testing is a brief, concise explanation of a product or capability from the end user’s perspective. In an Agile process, a user story is a small workflow unit. It allows the development team to understand the user’s wants and needs and ensures that the program fits those demands. It is often used in software development.

User stories are typically written in a specific format;

User
Functionality
Goal

User stories are essential for directing the development process and ensuring the final product meets the consumer’s demands. As a User, I seek Functionality so that the Goal is structured. Examples include wanting to follow their purchase status online and knowing when to anticipate delivery. Testers must understand and turn user stories into appealing test cases to ensure the program functions as intended.

User stories are a great component for any development team, but there are several crucial characteristics to consider while designing them, mentioned below:

A quick and concise explanation of a software feature or need.
Written from the standpoint of a user or stakeholder.
Features acceptance criteria for defining the feature’s expected behaviour. It may be divided into smaller, more manageable chunks (epics, themes, or tasks).
Often written in “user, functionality, goal” – format, the development team and stakeholders reach a common understanding.
Priorities are assigned based on business value and influence on end-user experience (User Stories In Testing: How To Convert it Into Test Cases?).

User Story Testing acts like a guide, showing the way to make users really happy. When you add Crowd Testing to it, this approach gets even better, going beyond the usual limits and uncovering all kinds of helpful insights from people all around the world.

Testing User Stories helps make sure that software is built with real people’s needs in mind, making it a strong way to ensure good quality. It’s about trying to understand the world from the user’s point of view (The Power of User Story Testing: Amplifying Your Confidence in Product Quality).

Scenario-based Testing:

Tests are designed based on specific scenarios or user journeys, simulating real-world usage to uncover potential issues.

Test Scenarios are created to ensure that every functionality a website or app offers is working as expected. It is best to gather input from clients, stakeholders, and developers to create real/accurate test scenarios. This helps effectively cover all possible user scenarios and enables comprehensive testing of all business flows of the software in question.

Test Scenarios are required to verify the entire system’s performance from the users’ perspective. When creating them, testers need to place themselves in the users’ shoes to clarify what real-world scenarios the software will have to handle when made public.

How to create a Test Scenario?

Carefully study the Requirement Document – Business Requirement Specification (BRS), Software Requirement Specification (SRS), and Functional Requirement Specification (FRS) about the System Under Test (SUT).
Isolate every requirement, and identify what possible user actions need to be tested.
Figure out the technical issues associated with the requirement.
Also, remember to analyze and frame possible system abuse scenarios by evaluating the software with a hacker’s eyes.
Enumerate test scenarios that cover every possible feature of the software.
Ensure that these scenarios cover every user flow and business flow involved in the operation of the website or app.
After listing the test scenarios, create a Traceability Matrix to ensure every requirement is mapped to a test scenario.
Get the scenarios reviewed by a supervisor, and then push them to be reviewed by other stakeholders involved in the project (How to create Test Scenarios? (with Examples)).

Example of User Story Testing technique.

Test Scenario 1

Check the Login functionality.

Ask the following questions in this scenario:

Can the user log in with a valid email id and password?
What happens when an invalid email id and valid password?
What happens when a valid email id and an invalid password are entered?
What happens when an invalid email id and invalid password are entered?
What happens when the email id and password fields are blank and the Login button is clicked?
Is the Forgot password? Is the option working as expected?
Is every link on the page functioning as expected?

Model-Based Testing:

Uses models (e.g., state diagrams, flowcharts) to generate test cases. Changes in the model can automatically generate new test scenarios.

Model-based testing is a software testing technique that uses a model to generate tests. A model is a description of a system’s behavior. In model-based testing, the run-time software’s behavior is evaluated according to the tests generated by the model.

Some examples of a model include the following:

The model-based testing technique describes how a system should behave in response to an action that is determined by a model. This concept is illustrated in the diagram below:

Advantages

Model-based testing is a crucial part of the software testing process due to its many benefits. Some of these benefits include:

Allowing exhaustive testing.
Both time and cost-effective.
Ensures early detection of defects/bugs in the software.
Can be done on different machines.
If the model is modified, the changes can easily be tested.

Disadvantages

Despite the numerous benefits of model-based testing, there are a few challenges associated with it. These include the following:

If the model is modified, it can result in a new set of test cases.
The testing team must be skilled to perform this testing and hence, a learning curve is introduced.
The model needs to be formally specified before testing can take place (What is model-based testing?).

Part 6. Test design techniques