Introduction
In the world of visual testing, achieving precision and accuracy is paramount. The concept of "match setting" is integral to this process, determining how images are compared during testing. This comprehensive guide delves into the various match levels, their applications, and best practices for setting match levels to ensure thorough and effective visual testing.
What is Match Setting?
Understanding Match Setting
Match setting refers to the configuration of criteria that determine how images are compared in visual tests. This process ensures that any deviations between the current and baseline images are accurately identified, facilitating robust regression testing and quality assurance.
Importance of Match Setting
Proper match setting is crucial for:
Detecting visual changes that could affect user experience.
Ensuring consistency across different environments and platforms.
Streamlining the testing process by filtering out irrelevant differences.
Available Match Levels
Strict Match Level
The strict match level is the default and most commonly recommended setting. It verifies that the page content matches the baseline image closely enough that the human eye would not see any difference. This level detects changes in:
Text
Font
Color
Graphics
Element positioning
Best Used For: Regression tests on a particular browser/OS with mostly static content.
Layout Match Level
The layout match level focuses on validating the consistency of the page layout. It identifies various page elements such as text, images, buttons, and columns, ensuring their relative positions are consistent. This level ignores differences in:
Text content
Graphics
Colors
Styles
Best Used For: Validating pages with dynamic content, language localization, and cross-environment testing.
Ignore Colors Match Level
Similar to the strict match level but with an emphasis on ignoring color changes. This level is effective for content that remains static while colors may vary.
Best Used For: Pages with buttons or elements that appear in different colors.
Exact Match Level
The exact match level conducts a pixel-to-pixel comparison of the checkpoint and baseline images. It is sensitive to even the smallest differences, such as rendering anomalies.
Best Used For: Scenarios where precise rendering is crucial, though not recommended for ordinary verification due to its sensitivity to minor, non-visual differences.
How to Specify a Match Level When Running Tests
Setting a Match Level for Multiple Tests
Use the setMatchLevel method within a configuration object, followed by eyes.setConfiguration on each Eyes instance before calling the open method. This sets the default match level for all subsequent checkpoints in tests executed by that Eyes instance.
Setting a Match Level for a Series of Checkpoints
Invoke the eyes.setMatchLevel method at any point during a test to change the default match level of all subsequent checkpoints on that Eyes instance.
Setting the Match Level in a Region of a Checkpoint
Define match levels for specific regions within a checkpoint using the following Fluent API methods:
strict(regions)
layout(regions)
ignorecolors(regions)
exact(regions)
Outside these regions, the page is matched using the default match level for that checkpoint or the one set for the Eyes instance.
Using Eyes Test Manager
Set match levels within particular steps using the Eyes Test Manager. This setting will apply to all future test runs of the checkpoint for that step.
Previewing the Effect of a Match Level
Using Eyes Test Manager
The Eyes Test Manager provides the capability to preview the impact of different match levels on test results. This feature allows you to:
View the results of a step at one match level.
See potential results with a different match level.
This functionality aids in fine-tuning match settings to ensure optimal detection of relevant changes.
Practical Applications and Examples
Example 1: Strict Match Level for Static Content
When testing a webpage with static content such as a landing page, the strict match level ensures that any minor text or graphical changes are detected, maintaining the integrity of the visual presentation.
Example 2: Layout Match Level for Dynamic Content
For a news website with frequently updated articles, the layout match level ensures that the structure and placement of elements remain consistent across updates, regardless of content changes.
Example 3: Ignore Colors Match Level for Thematic Variations
In an e-commerce site where buttons and banners change colors based on promotions, the ignore colors match level ensures functionality without flagging irrelevant color changes.
Example 4: Exact Match Level for Rendering Precision
For a design-centric platform where pixel-perfect rendering is crucial, the exact match level detects any slight rendering anomalies, ensuring visual fidelity.
Best Practices for Match Setting
Regularly Update Baselines
Keep baseline images up-to-date to reflect the current design and content of your application. This reduces false positives and ensures accurate comparisons.
Tailor Match Levels to Test Scenarios
Choose match levels based on the specific requirements of each test scenario. For example, use strict match for detailed regression tests and layout match for broader cross-environment checks.
Combine Match Levels Strategically
For comprehensive testing, combine different match levels. Use strict match for critical areas and layout match for less critical, dynamic sections.
Automate Match Level Configuration
Incorporate match level settings into your automated test scripts to ensure consistency and reduce manual intervention.
Leverage Previewing Tools
Utilize previewing tools in the Eyes Test Manager to understand the impact of different match levels and fine-tune settings for optimal results.
Case Study: Implementing Match Setting in a Complex Testing Environment
Scenario
A financial services company needed to ensure visual consistency across their web application, which included dynamic dashboards and static reports.
Implementation
Baseline Creation: Established baseline images for all critical pages and components.
Match Level Configuration:
Used strict match level for static report pages.
Applied layout match level for dynamic dashboard pages.
Automated Testing: Integrated match level settings into their CI/CD pipeline to automate visual testing.
Regular Updates: Updated baseline images periodically to align with design changes.
Results
The company achieved consistent visual quality, with reduced false positives and increased confidence in their visual testing process.
Conclusion
Understanding and effectively implementing match settings is crucial for robust visual testing. By selecting appropriate match levels and leveraging tools like the Eyes Test Manager, organizations can ensure visual consistency and quality across their applications. This guide provides the knowledge and strategies needed to master match settings and enhance your visual testing efforts.
Key Takeaways
Match setting is crucial for precise visual testing.
Different match levels (strict, layout, ignore colors, exact) serve various purposes.
Properly configured match settings enhance testing accuracy and efficiency.
Previewing match levels helps fine-tune visual tests.
Regular updates to baselines and strategic use of match levels ensure optimal results.
FAQs
What is match setting in visual testing?
Match setting refers to configuring the criteria for comparing images in visual tests, ensuring accurate identification of differences between the current and baseline images.
What are the different match levels available?
The main match levels are strict, layout, ignore colors, and exact, each serving different purposes based on testing requirements.
How do I set a match level for multiple tests?
Use the setMatchLevel method in a configuration object and apply it using eyes.setConfiguration before opening the Eyes instance.
Can I specify different match levels for different regions in a checkpoint?
Yes, you can define match levels for specific regions using Fluent API methods like strict(regions), layout(regions), ignorecolors(regions), and exact(regions).
How can I preview the effect of different match levels?
Use the Eyes Test Manager to preview the impact of different match levels on test results, helping you fine-tune settings for optimal outcomes.
Why is the strict match level recommended by default?
The strict match level closely matches the baseline image to detect differences visible to the human eye, making it ideal for detailed regression tests.
When should I use the layout match level?
The layout match level is suitable for validating the structural consistency of pages with dynamic content or across different environments.
What is the purpose of the exact match level?
The exact match level performs a pixel-to-pixel comparison, useful for scenarios requiring precise rendering verification, though it is not recommended for general testing due to its sensitivity.
Comments