In reviewing the output of issues and PRs created by copilot, unless someone is already familiar with the axe-core rules, it can be hard to tell when an issue is about an actual WCAG failure, vs when a best practice issue is being called out.
As just one example, when scanning pages with valid nested documents, the best practice rule for "landmark-main-is-top-level" was flagged.
The initial issue text was benign enough, though lacking an overt statement that this is a best practice issue:
An accessibility scan flagged the element <div role="main" tabindex="-1"> on [ name of file here ] because main landmark should not be contained in another landmark. Learn more about why this was flagged by visiting https://dequeuniversity.com/rules/axe/4.10/landmark-main-is-top-level?application=playwright.
But the PR created fibs, stating (emphasis mine):
The page contained a <div role="main"> element nested inside the actual <main> element, which violates WCAG 2.1 guidelines.
There were other best practice issues that were filed in my testing, where Copilot then fabricated rational in the auto-generated PR to state it was fixing WCAG violations, when they were not.
In reviewing the output of issues and PRs created by copilot, unless someone is already familiar with the axe-core rules, it can be hard to tell when an issue is about an actual WCAG failure, vs when a best practice issue is being called out.
As just one example, when scanning pages with valid nested documents, the best practice rule for "landmark-main-is-top-level" was flagged.
The initial issue text was benign enough, though lacking an overt statement that this is a best practice issue:
But the PR created fibs, stating (emphasis mine):
There were other best practice issues that were filed in my testing, where Copilot then fabricated rational in the auto-generated PR to state it was fixing WCAG violations, when they were not.