5. Testing process
Testing progresses through different stages, from planning to reporting. The process described in this section should be adapted to the scope of testing, the development model, and the needs of the application or project.
5.1 Testing Process Stages
5.1.1 Test Plan
The test plan is typically prepared by the Product Owner in cooperation with developers, the Project Manager, and other relevant stakeholders. Responsibility may also be assigned to a designated testing specialist or developer if such a role exists.
When preparing the test plan, consider the following:
- Testing objectives: What should be verified through testing?
- Testing scope: What will be tested and what is excluded from testing?
- Testing methods: Manual testing, automated testing, and other approaches.
- Testing levels: For example, unit, integration, and system testing.
- Schedule: When testing will be performed and how it fits into other development activities.
- Resources: Who will participate in testing and what skills are required?
- Test environments: Where testing will be performed, including required devices, software, and environments.
- Test data: What data will be used and how it will be managed?
- Risks: Potential problems and how they may affect testing.
- Acceptance criteria: The conditions for determining when testing is sufficient or complete.
- Reporting: How test results will be documented and communicated.
- Tools: Which tools will be used for testing.
5.1.2 User Stories
User stories are typically created by the Product Owner, often in cooperation with the development team, stakeholders, and, when appropriate, users.
A user story should include or consider:
- Description from the user's perspective: What does the user want to do and why?
- User role: Who will use the functionality?
- Goal: What does the user want to achieve?
- Benefit: Why is the functionality important to the user?
- Acceptance criteria: The conditions that must be met for the story to be considered complete.
- Priority: How important the story is compared with other stories.
- Estimate: The expected effort or complexity, for example in story points.
- Dependencies: Whether the story depends on other functionality.
- Additional information: Images, examples, specifications, or other information as needed.
Well-defined user stories and acceptance criteria also provide a good basis for creating both manual and automated tests.
5.1.3 Creating Test Cases
Test cases are typically created by a designated team member. The team should agree on who is responsible for creating and maintaining test cases and who else participates in the work.
A test case should include at least:
- Test steps: The steps required to perform the test.
- Test data: The input data required for the test.
- Expected result: What should happen if the functionality works correctly.
Test cases should be clear enough that another person can execute the test based on the documented information.
5.1.4 Test Execution
Tests are executed and the results are recorded using the agreed testing or documentation tool. Excel can be used for documenting and executing test cases when a dedicated test management tool is not required.
A typical test execution process includes:
- Preparation: Ensure that the test environment, test data, and required tools are ready.
- Check preconditions: Verify that all prerequisites for executing the test are met.
- Execute the test steps: Perform the documented test steps in the planned order.
- Observe the results: Monitor the behaviour of the application during the test.
- Compare the results: Compare the actual results with the expected results.
- Record the result: Document the outcome of the test, for example as passed or failed.
- Report deviations: Record defects or other deviations that require further investigation.
5.1.5 Defect Reporting
The purpose of defect reporting is to document identified problems clearly and consistently so that they can be analysed, corrected, and retested. A well-documented defect also makes communication between testers, developers, and other stakeholders easier.
When reporting a defect, include the following information where applicable:
- Clear description: What happened and under what circumstances did the problem occur?
- Steps to reproduce: The steps required to reproduce the problem.
- Expected and actual results: What should have happened and what actually happened?
- Environment: Where the defect was identified, for example the browser, device, operating system, application version, or test environment.
- Severity and priority: How significant the defect is and how urgently it should be addressed.
- Supporting information: Screenshots, logs, error messages, or other information that helps with investigation.
- Unique identifier: An identifier that allows the defect to be found and tracked.
- Date and reporter: When the defect was reported and by whom.
5.1.6 Fixing and Retesting Defects
After a defect has been fixed, it should be retested to verify that the fix works as expected and that the original problem has been resolved.
Where appropriate, regression testing should also be performed to ensure that the change has not caused problems in related or previously working functionality.
5.1.7 Test Monitoring and Final Report
Testing progress should be monitored using an approach appropriate to the scope and complexity of the testing activities.
The status of testing should be communicated to relevant stakeholders when needed. Depending on the project or application, reporting may include:
- completed, passed, and failed tests
- outstanding defects and their severity
- areas that have not yet been tested
- identified risks or limitations
- overall testing status
- whether the agreed acceptance or completion criteria have been met
The required level of reporting should be agreed separately based on the scope and needs of the testing work.
Main changes from the Finnish version
I kept the structure and meaning close to your original, but made a few small improvements:
- 5.1.4: I changed the Excel statement slightly. Saying simply “Excel is a good tool” sounds like the guideline specifically recommends Excel. The revised wording leaves it as a lightweight option.
- 5.1.6: I added regression testing, because verifying that a fix has not broken related functionality is an important part of this stage.
- 5.1.7: I added examples of what monitoring and final reporting can contain. The original was quite brief compared with the other process stages.
- 5.1.3: I added that a test case should be understandable enough for another person to execute it. This makes the practical purpose of documenting test cases clearer.
- 5.1.2: I connected good user stories and acceptance criteria to manual and automated testing, which fits well with the automation guidance you are already developing.