oss/forgejo
oss/forgejo
1
0
Fork 0
mirror of https://codeberg.org/forgejo/forgejo.git synced 2025-10-15 19:42:04 +00:00
Code Wiki Activity

test(e2e): improve resiliency of accessibilityCheck (#9051)

Browse source 
In the helper method `accessibilityCheck`, accessibility checks are performed on the page including measurements of contrast between elements, and the theme is transitioned from the `light` to `dark` theme and back between two checks (with a hard-coded 100ms wait).  Experimentally this is proven to result in false-positive failures of the accessibilityCheck.

I believe the cause of these failures are CSS transitions that are occurring either between the theme transition, or from operations that have occurred before the `accessibilityCheck` method is called.  For example, one test `Markdown insert link` pops open the `Add a link` dialog box which has a CSS transition applied as the background is greyed out.

To fix this, I've wrapped both accessibility scans within `accessibilityCheck` in a `expect(...).toPass({ timeout: 1000 })` to allow Playwright to retry the assertions over the next second.  1 second exceeds the time that I've observed of any CSS transition in the project (max found was 0.3 seconds).

Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/9051
Reviewed-by: Antonin Delpeuch <wetneb@noreply.codeberg.org>
Reviewed-by: Beowulf <beowulf@beocode.eu>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
This commit is contained in:
Mathieu Fenniak 2025-08-29 07:33:59 +02:00 • committed by Beowulf
parent 9d896028bd
commit 6ac0ab3549
1 changed files with 18 additions and 13 deletions
Download patch file Download diff file Expand all files Collapse all files

31
tests/e2e/shared/accessibility.ts
View file

@ -17,19 +17,24 @@ export async function accessibilityCheck({page}: {page: Page}, includes: string[
accessibilityScanner = accessibilityScanner.exclude(excl);
}
// scan the page both in dark and light theme
let accessibilityScanResults = await accessibilityScanner.analyze();
expect(accessibilityScanResults.violations).toEqual([]);
// Scan the page both in dark and light theme.
//
// Have observed failures during this scanning which are understood to be caused by CSS transitions, either applied to
// whatever last action occurred on the page before `accessibilityCheck` was called, or during the transition from
// dark to light. As there are a variety of transitions in Forgejo's CSS files (primarily in fomantic) with ease
// elements between 0.1 and 0.3 seconds, we give the accessibility scanner up to 1s to settle into success for each
// scan.
await expect(async () => {
const accessibilityScanResults = await accessibilityScanner.analyze();
expect(accessibilityScanResults.violations).toEqual([]);
}).toPass({timeout: 1000});
await page.emulateMedia({colorScheme: 'dark'});
// in https://codeberg.org/forgejo/forgejo/pulls/5899 there have been
// some weird failures related to contrast scanning,
// reporting for colours that haven't been used and no trace in the
// screenshots.
// Since this was only happening with some browsers and not always,
// my bet is on a transition effect on dark/light mode switch.
// Waiting a little seems to work around this.
await page.waitForTimeout(100); // eslint-disable-line playwright/no-wait-for-timeout
accessibilityScanResults = await accessibilityScanner.analyze();
expect(accessibilityScanResults.violations).toEqual([]);
await expect(async () => {
const accessibilityScanResults = await accessibilityScanner.analyze();
expect(accessibilityScanResults.violations).toEqual([]);
}).toPass({timeout: 1000});
await page.emulateMedia({colorScheme: 'light'});
}