mirror of
https://codeberg.org/forgejo/forgejo.git
synced 2025-09-30 19:22:08 +00:00
6ac0ab3549
In the helper method `accessibilityCheck`, accessibility checks are performed on the page including measurements of contrast between elements, and the theme is transitioned from the `light` to `dark` theme and back between two checks (with a hard-coded 100ms wait). Experimentally this is proven to result in false-positive failures of the accessibilityCheck.
I believe the cause of these failures are CSS transitions that are occurring either between the theme transition, or from operations that have occurred before the `accessibilityCheck` method is called. For example, one test `Markdown insert link` pops open the `Add a link` dialog box which has a CSS transition applied as the background is greyed out.
To fix this, I've wrapped both accessibility scans within `accessibilityCheck` in a `expect(...).toPass({ timeout: 1000 })` to allow Playwright to retry the assertions over the next second. 1 second exceeds the time that I've observed of any CSS transition in the project (max found was 0.3 seconds).
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9051
Reviewed-by: Antonin Delpeuch <wetneb@noreply.codeberg.org>
Reviewed-by: Beowulf <beowulf@beocode.eu>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
40 lines
1.8 KiB
TypeScript
40 lines
1.8 KiB
TypeScript
import {expect, type Page} from '@playwright/test';
|
|
import {AxeBuilder} from '@axe-core/playwright';
|
|
|
|
export async function accessibilityCheck({page}: {page: Page}, includes: string[], excludes: string[], disabledRules: string[]) {
|
|
// contrast of inline links is still a global issue in Forgejo
|
|
disabledRules.push('link-in-text-block');
|
|
|
|
let accessibilityScanner = new AxeBuilder({page})
|
|
.disableRules(disabledRules);
|
|
// passing the whole array seems to be not supported,
|
|
// iterating has the nice side-effectof skipping this if the array is empty
|
|
for (const incl of includes) {
|
|
// passing the whole array seems to be not supported
|
|
accessibilityScanner = accessibilityScanner.include(incl);
|
|
}
|
|
for (const excl of excludes) {
|
|
accessibilityScanner = accessibilityScanner.exclude(excl);
|
|
}
|
|
|
|
// Scan the page both in dark and light theme.
|
|
//
|
|
// Have observed failures during this scanning which are understood to be caused by CSS transitions, either applied to
|
|
// whatever last action occurred on the page before `accessibilityCheck` was called, or during the transition from
|
|
// dark to light. As there are a variety of transitions in Forgejo's CSS files (primarily in fomantic) with ease
|
|
// elements between 0.1 and 0.3 seconds, we give the accessibility scanner up to 1s to settle into success for each
|
|
// scan.
|
|
await expect(async () => {
|
|
const accessibilityScanResults = await accessibilityScanner.analyze();
|
|
expect(accessibilityScanResults.violations).toEqual([]);
|
|
}).toPass({timeout: 1000});
|
|
|
|
await page.emulateMedia({colorScheme: 'dark'});
|
|
|
|
await expect(async () => {
|
|
const accessibilityScanResults = await accessibilityScanner.analyze();
|
|
expect(accessibilityScanResults.violations).toEqual([]);
|
|
}).toPass({timeout: 1000});
|
|
|
|
await page.emulateMedia({colorScheme: 'light'});
|
|
}
|