I. Definitions
1. What is Exploratory Testing?
Exploratory testing is a form of software testing in which the design and execution of the test will be deployed concurrently. Therefore, it will depend on the knowledge and capabilities of the test performers instead of the previously generated Test case, checklist.
2. Characteristics
a. Characteristics
Exploratory testing is different from other types of scripting tests (it requires a higher level of thinking, digging, and discovery)
- Scripted testing can be considered as a train, it will run on an available track (Test case).
- With exploration test, it is like a car, also from point A to B, but will not necessarily follow the same path, but can run different routes to get to B.
Exploratory testing is commonly used in the Agile model because:
- It gives quick system feedback to the development team
- Highlight the difference in test views of all roles, members of the project team
- Advanced discovery, investigation, and insight into the software under test.
b. Difference between test scenario and probe test
| Scripted testing | Exploratory testing |
|---|---|
| Made based on the project’s required documentation | Made against the system requirements documentation and explored throughout the test |
| Define the test cases first (Test case) | Test cases will be identified during test execution |
| Test confirmation by specification documentation | Analysis, investigation of systems, applications |
| Emphasis is placed on prediction and decision-making | Emphasize adaptability and learning |
| Requires the testing to be confirmed by the steps taken | A discovery is required throughout implementation |
| It’s about controlled testing | There is always a need to improve the way testing is performed |
| Do the same test as when you speak – a prepared presentation already exists | It was like a spontaneous and unfettered conversation |
| The testing process is controlled in the available scenarios | The testing process is controlled and carried out by the tester’s mind |
II. Advantages and disadvantages
1. Advantages
- Useful when project specification documents are unavailable or only partially, rather sketchy.
- It involves the investigation process to find more errors than normal testing.
- Discovers and finds bugs ignored by other testing techniques (as it approaches the system by digging and analyzing into the smallest part of the application).
- Combine as many test types as possible, including different scenarios and scenarios.
- Quick feedback to the system for the development team.
- Encourage the creativity and intuition of testers
- Generate new ideas during testing.
2. Restrictions
- Since test cases are generated and executed randomly, it is difficult to fully list which cases must be performed.
- Test results depend a lot on the knowledge, skills and experience of testers.
- It is possible to omit the cases if the tester has little knowledge about the system.
- Not suitable for long-term projects.
III. Apply test exploration to the project
1. How should test and exploration be applied in the project?
- In the early stage of the project, when there are many changes in the requirements or development process, the use of test exploration will be highly effective.
- Developers can also apply tentative testing to unit testing, it will have optimum efficiency while familiarizing the system.
- In Agile model, there is a short period and limited time to write Testcase.
- Performing probe testing saves time, having important output outputs from old cycles, applied to subsequent scrums.
2. Steps to apply probe test
Exploratory testing is performed in 5 steps, it is also known as session-based management (SBTM cycle).
a. Bug classification
Classification of errors commonly encountered in previous projects
Analyze the root cause of the problem or error
Find out the risks and develop an application test idea.
b. Test charter
As a statement of test objectives, it is possible to test ideas on how to test.
Test regulations include:
- Test something
- How to test
- Need to consider, pay attention to what during the test
- Test ideas are the starting point of exploration testing
- Test rules will determine whether or not the end users can use the system.
c. Limited time
Time for a pair of testers to work together is not less than 90 minutes
There shouldn’t be any interruptions for 90 minutes of doing it.
The time frame can be stretched or reduced by 45 minutes.
Each session should be tested for the response status of the system and correctly prepared for the output.
d. Result evaluation
Evaluate the bugs
Learn from the test
Coverage analysis
e. Discuss the results
Synthesize the results after the test
Compare test results with the charter (test charter)
Check if additional testing is required (judging by the amount of system bug, a feature may have clustering bug).
f. Sample session of a sample session
3. Methods to carry out exploration testing effectively.
Guess the error:
Based on your knowledge, experience from past projects and overview of the project, you can make predictions about some possible system features that may have bugs.
- In a real project, there are very simple functions and extra effort to probe it is unnecessary like login / logout / forgot password / list displayed. Instead, we will probe into the main functions of the system that are related to many other functions. For example, with the customer management system we will have functions like add / edit / delete / authorize the account – that’s where we need to focus on exploring.
Create test diagrams and use cases
Should create interactive diagrams of interactions and relationships between functions in the system will help cover more cases. And create user cases that help to understand product usage from an end-user perspective.
- With projects, most likely we will have a user story or mindmap about the linkage between features, take advantage of that document to probe the main threads and also help you detect different test cases of threading. present.
The previous errors
Study the bugs that have occurred in previous builds so that you can focus on features that may have the same or more bugs.
- With each working sprint, pay attention to the bugs that have occurred, we will gain knowledge for the guesswork of features that may have bugs in new builds. It may be necessary to re-test the function area with too many previous bugs or new features with similar, similar handling.
Error handling
Probe testing can be done by testing against a variety of scenarios, so that you can test for error handling in the code well.
Discuss
The understanding of the system through the process of communication, discussion and validation at work, meetings, Q&A, … will be very helpful for planning test.
- Test probing is not fixed by an existing scenario, but it will likely have more and more cases that you will discover during the test. Discuss with the other team members for more cases they will provide you, from their perspective. That really makes sense for the team.
Questions and checklist
Asking questions like “what, why, how, where, when, …” can provide clues during exploration testing.
- When performing the test, please ask yourself the questions “what, why, how, where, when, …” about the feature being tested or when detect defect. It will help you to go deeper and quickly find root cause.
- For systems that are a bit more complex, make sure you have a checklist of features to make sure you haven’t missed any. Should apply both new-test, re-test and especially regression testing.
4. Some notes when testing probes
Focus on goals:
The goal of probe testing is that you act as an end user to proactively find and identify bugs.
- For example, when the system is put into use, too detailed abnormal cases are not necessary, such as when you rarely enter a name of more than 255 characters (must have test, don’t go deep). But make sure you have tested all login cases with account types (active / Inactive), role (admin / user / Accountant …) and check both the permissions of the accounts in the system.
Don’t create a script, but need a plan for testing:
Planning helps clarify the specific aspects of the system you want to test, including specific system requirements or expectations.
Don’t try to check too much
The purpose of tentative testing is not the quality assurance scope but rather to find defects that cannot be detected by other types of testing. Its essence is to focus on parts of the system that are beyond common interactions, use, and are less likely to be thoroughly tested.
Keep a clear record of what is done:
Please keep a clear record of the tasks performed, problems discovered in a document file.
Defining scope (time, process)
Make sure that your testing is responsive to the project timeline and progress. Care is deeply absorbed and wasted too much time on a certain function, affecting the overall progress.
Define a specific timebox for each function and remember it.
Select exploration techniques that meet the needs of the project
There are main types such as:
- Freestyle (adhoc testing)
- Based on scenarios
- Based on strategy (decision table, cause-and-effect graph, error guessing).
Understanding the techniques to use and share it with project stakeholders such as product owner, developers is required. Define the scope (time, process).
References