Testing is the most crucial and underrated part of digital delivery. It requires input from everyone involved in the project. The best digital products have been tested thoroughly by all stakeholders before being launched.

Types of testing

DE&S coordinates and performs multiple forms of testing:

  • Functional testing (DE&S)
  • Accessibility testing (DE&S)
  • User Acceptance Testing (business)
    • Acceptance testing 
    • CMS author testing

Functional testing

Functional testing checks that an application, website or system it is doing exactly what it is meant to across different browsers and devices.

It may include:

  • Smoke testing — This is done after the release of each build to ensure that software stability is intact and there are no anomalies.
  • Regression testing — This test ensures that changes to the codebase (new code, debugging strategies, etc.) do not disrupt the already existing functions or trigger any instability.
  • Integration testing — If a system requires multiple functional modules to work effectively, integration testing is done to ensure that individual modules work as expected when operating in combination with each other. It validates that the end-to-end outcome of the system meets these necessary standards.

Accessibility testing

Accessibility testing is performed to ensure the solution complies with the WCAG 2.2 AA guidelines. This ensures people who access the solution in different ways (such as users of assistive technologies) can access the same content.

Accessibility testers use a mix of technologies and techniques, such as screen reader software, keyboard-only interaction, colour contrast analysis, and code analysis.

Any issues found at this stage will be flagged with the developer to resolve.

Read more about Digital NSW accessibility and inclusivity guidelines.

User acceptance testing (UAT)

User acceptance testing (UAT) is done by the product owner in a pre-production environment. It is performed to ensure that the website or product performs as expected without any errors.

The objectives of UAT are to:

  • confirm that the solution performs business functions according to business requirements
  • ensure the solution matches the approved designs
  • ensure the solution is compliant with regulatory and/or legal requirements
  • certify that the solution is deemed ready to be moved into production.

As extensive testing has been conducted prior to User Acceptance Testing, UAT is focussed on business process validation rather than cosmetic or system issues. 

User Acceptance Testing considers the end user experience and the experience of content editors.

There are limits to what can be tested. Sometimes the content (especially text and images) in a solution will be placeholder content at this stage — often the Latin words ‘lorem ipsum’. This will be changed later when content is uploaded. 

When errors are found in a test report, DE&S will verify the bug by first identifying whether it is part of the solution, and will then try to replicate it.. Issues will be assessed as bugs, change requests, or content changes.

  • Bugs which are verified by DE&S testers and are considered in scope will be resolved by developers during the project. Depending on the available time, an agreement between DE&S and the business may be sought to resolve the issue after the initial launch of the project.
  • Change Requests (CRs) - if the issue identified is outside of the original scope of the project but is deemed critical by the business, a Change Request can be issued to add it to scope. This will need to be agreed with DE&S and may impact the cost and timeline of the project. 
  • Content changes will be the responsibility of the business team involved in the project.

Once bugs are resolved, the reporter will be notified and asked to retest the issue to verify that it is resolved. This is an important step and should not be missed.

How to ensure a successful test phase

  • Identify testers early on and ensure they have capacity during the test windows.
  • Identify CMS users for CMS author testing and ensure they have capacity, CMS access and are briefed on the project.
  • Set aside time to test and retest the solution.
  • Ensure you and the other testers are aware of what and how to test (including what is in scope). 
  • Focus on the core functionality rather than look and feel — content can be added at a later stage but this is your opportunity to test that things work properly.

Once all bugs are resolved and verified by the reporter, we will ask for formal approval to deploy to the production environment.