Siddhant Deval
Siddhant Deval
Siddhant Deval
frontend5 min read

Production Accessibility Engineering: Axe-Core, Headless Audits & Continuous Quality Gates

Static linting catches less than 25% of accessibility defects. This article implements the complete 4-tier Accessibility Testing Pyramid: unit testing with vitest-axe, headless full-page and component audits in Playwright, synthetic keyboard traversal tests, APCA/WCAG contrast mathematical validation, and CI pull-request quality gates.

Series·Part 6 of 6

Frontend Accessibility Architecture: From Semantic DOM Foundations to Resilient Accessible State Engines

Previous ChapterLive Regions, Streaming State & Async Announcers: Managing Dynamic Content & Toast Stacks

Automated Accessibility Testing & CI Pipeline: Building a Zero-Regression AXTree Gate

Parts 1–5 built the full accessibility architecture of the Enterprise Accessible Design System & Portal Canvas: AT runtime awareness, AXTree compilation, focus state machines, APG-compliant compound widgets, and a production live region system. The remaining architectural gap is the one that makes the entire investment permanent rather than temporary: automated testing that prevents regressions from being merged.

Accessibility regressions are the silent counterpart to visual regressions. They do not produce JavaScript errors. They do not break builds. A developer removes aria-label from a navigation landmark in a refactor, the site ships, and NVDA users navigate to a region announced only as "navigation" — indistinguishable from the three other navigation regions on the page. No error was thrown. No unit test failed. No CI gate caught it.

This article builds the complete CI accessibility testing pyramid for the Portal Canvas: unit tests with vitest-axe that validate ARIA roles and names at the component level, integration tests with @testing-library/user-event that simulate real keyboard navigation sequences, and Playwright end-to-end tests with axe-playwright that run the full AXTree analysis on every build. The pyramid includes a violation policy system for managing accepted violations with documented justification and owner accountability.

1. The Accessibility Testing Pyramid

Layer Tool Stack What It Catches What It Misses
Unit vitest + vitest-axe Missing ARIA roles, empty names, invalid attribute values, duplicate IDs Keyboard interaction sequences, focus order, dynamic state
Integration RTL + user-event Keyboard navigation sequences, roving tabindex movement, focus restoration Full AXTree, platform AT behavior, visual contrast
E2E Playwright + axe-playwright Full AXTree violations, contrast failures, landmark structure Browser-specific AT quirks, braille output
Crucial Requirement

The testing pyramid is an additive coverage model — each layer catches different failure categories. No single layer is sufficient. A component can pass all axe unit tests (valid ARIA) while still failing keyboard navigation integration tests (incorrect key handling). Both failures are accessibility regressions.

2. Unit Testing with vitest-axe

2.1 Setup

BASH
npm install --save-dev vitest-axe axe-core @testing-library/react @testing-library/jest-dom
TYPESCRIPT
// vitest.setup.ts
import '@testing-library/jest-dom';
import { toHaveNoViolations } from 'vitest-axe';
import { expect } from 'vitest';

expect.extend({ toHaveNoViolations });

2.2 Component Unit Tests

TYPESCRIPT
// tests/unit/components/WorkspaceSearch.test.tsx
import { describe, it, expect } from 'vitest';
import { render } from '@testing-library/react';
import { axe } from 'vitest-axe';
import { WorkspaceSearch } from '@/components/WorkspaceSearch';

const mockOptions = [
  { id: '1', label: 'Dashboard', href: '/dashboard' },
  { id: '2', label: 'Workspace Settings', href: '/settings' },
  { id: '3', label: 'Analytics', href: '/analytics' },
];

describe('WorkspaceSearch accessibility', () => {
  it('has no axe violations in closed state', async () => {
    const { container } = render(
      <WorkspaceSearch allOptions={mockOptions} />
    );
    const results = await axe(container);
    expect(results).toHaveNoViolations();
  });

  it('has no axe violations with listbox open', async () => {
    const { container, getByRole } = render(
      <WorkspaceSearch allOptions={mockOptions} />
    );
    const combobox = getByRole('combobox', { name: /search workspace/i });
    // Simulate typing to open the listbox
    await userEvent.type(combobox, 'work');

    const results = await axe(container);
    expect(results).toHaveNoViolations();
  });

  it('combobox has accessible name', () => {
    const { getByRole } = render(
      <WorkspaceSearch allOptions={mockOptions} />
    );
    expect(getByRole('combobox', { name: /search workspace/i })).toBeInTheDocument();
  });

  it('options have required roles when popup is open', async () => {
    const { getByRole, getAllByRole } = render(
      <WorkspaceSearch allOptions={mockOptions} />
    );
    const combobox = getByRole('combobox');
    await userEvent.type(combobox, 'a');
    // All visible options must have role="option"
    const options = getAllByRole('option');
    expect(options.length).toBeGreaterThan(0);
  });
});
TYPESCRIPT
// tests/unit/components/PortalShell.test.tsx
import { describe, it, expect } from 'vitest';
import { render } from '@testing-library/react';
import { axe } from 'vitest-axe';
import { PortalShell } from '@/components/PortalShell';

describe('PortalShell accessibility', () => {
  it('has no axe violations', async () => {
    const { container } = render(
      <PortalShell>
        <h1>Dashboard</h1>
        <p>Content</p>
      </PortalShell>
    );
    const results = await axe(container);
    expect(results).toHaveNoViolations();
  });

  it('has exactly one main landmark', () => {
    const { getAllByRole } = render(
      <PortalShell><h1>Test</h1></PortalShell>
    );
    expect(getAllByRole('main')).toHaveLength(1);
  });

  it('navigation landmarks have accessible names', () => {
    const { getAllByRole } = render(
      <PortalShell><h1>Test</h1></PortalShell>
    );
    const navs = getAllByRole('navigation');
    navs.forEach((nav) => {
      expect(nav).toHaveAttribute('aria-label');
      expect(nav.getAttribute('aria-label')).not.toBe('');
    });
  });

  it('skip link targets main content', () => {
    const { getByText, getByRole } = render(
      <PortalShell><h1>Test</h1></PortalShell>
    );
    const skipLink = getByText(/skip to main content/i);
    const main = getByRole('main');
    expect(skipLink).toHaveAttribute('href', '#main-content');
    expect(main).toHaveAttribute('id', 'main-content');
  });
});

2.3 Axe Rule Configuration

TYPESCRIPT
// tests/a11y-config.ts — shared axe configuration
import type { RunOptions } from 'axe-core';

export const baseAxeConfig: RunOptions = {
  rules: {
    // Colour contrast is handled by design system tokens — verified manually
    // and covered in Playwright E2E contrast tests. Disable in unit tests
    // to avoid false positives from test environment (no computed styles).
    'color-contrast': { enabled: false },
    // Region rule: every section of content must be in a landmark
    // — enforced by architectural convention; individual component tests
    // render fragments, not full pages
    'region': { enabled: false },
  },
};
Pro Tip & Optimization

Disable color-contrast in unit and integration tests — jsdom does not compute CSS styles, so computed contrast ratios are always wrong in the test environment. Run contrast tests only in Playwright E2E tests against a real Chromium instance with computed styles.

3. Integration Testing — Keyboard Navigation Sequences

3.1 Combobox Keyboard Navigation Test

TYPESCRIPT
// tests/integration/keyboard/ComboboxKeyboard.test.tsx
import { describe, it, expect } from 'vitest';
import { render, screen } from '@testing-library/react';
import userEvent from '@testing-library/user-event';
import { WorkspaceSearch } from '@/components/WorkspaceSearch';

const options = [
  { id: '1', label: 'Dashboard', href: '/dashboard' },
  { id: '2', label: 'Workspace Settings', href: '/settings' },
];

describe('Combobox keyboard navigation', () => {
  it('opens listbox on ArrowDown', async () => {
    const user = userEvent.setup();
    render(<WorkspaceSearch allOptions={options} />);

    const combobox = screen.getByRole('combobox');
    await user.click(combobox);
    await user.keyboard('{ArrowDown}');

    expect(combobox).toHaveAttribute('aria-expanded', 'true');
    expect(screen.getByRole('listbox')).toBeInTheDocument();
  });

  it('moves virtual focus through options with ArrowDown', async () => {
    const user = userEvent.setup();
    render(<WorkspaceSearch allOptions={options} />);

    const combobox = screen.getByRole('combobox');
    await user.type(combobox, 'a');
    await user.keyboard('{ArrowDown}');

    // aria-activedescendant must point to the first option
    const activeId = combobox.getAttribute('aria-activedescendant');
    expect(activeId).toBeTruthy();
    const activeOption = document.getElementById(activeId!);
    expect(activeOption).toHaveAttribute('aria-selected', 'true');
  });

  it('selects option on Enter', async () => {
    const user = userEvent.setup();
    render(<WorkspaceSearch allOptions={options} />);

    const combobox = screen.getByRole('combobox');
    await user.type(combobox, 'Dash');
    await user.keyboard('{ArrowDown}');
    await user.keyboard('{Enter}');

    expect(combobox).toHaveValue('Dashboard');
    expect(combobox).toHaveAttribute('aria-expanded', 'false');
  });

  it('closes listbox on Escape', async () => {
    const user = userEvent.setup();
    render(<WorkspaceSearch allOptions={options} />);

    const combobox = screen.getByRole('combobox');
    await user.type(combobox, 'a');
    await user.keyboard('{Escape}');

    expect(combobox).toHaveAttribute('aria-expanded', 'false');
  });

  it('wraps from last option to first on ArrowDown', async () => {
    const user = userEvent.setup();
    render(<WorkspaceSearch allOptions={options} />);

    const combobox = screen.getByRole('combobox');
    await user.type(combobox, 'a');
    // Arrow down twice to reach last option (2 options)
    await user.keyboard('{ArrowDown}');
    await user.keyboard('{ArrowDown}');
    // Arrow down once more — should wrap to first option
    await user.keyboard('{ArrowDown}');

    const activeId = combobox.getAttribute('aria-activedescendant');
    const activeOption = document.getElementById(activeId!);
    const allOptions = screen.getAllByRole('option');
    expect(activeOption).toBe(allOptions[0]);
  });
});

3.2 Tablist Keyboard Navigation Test

TYPESCRIPT
// tests/integration/keyboard/SettingsTablist.test.tsx
import { describe, it, expect } from 'vitest';
import { render, screen } from '@testing-library/react';
import userEvent from '@testing-library/user-event';
import { SettingsTablist } from '@/components/SettingsTablist';

const tabs = [
  { id: 'profile', label: 'Profile', content: <p>Profile content</p> },
  { id: 'security', label: 'Security', content: <p>Security content</p> },
  { id: 'billing', label: 'Billing', content: <p>Billing content</p> },
];

describe('SettingsTablist keyboard navigation', () => {
  it('first tab is selected and has tabindex=0 initially', () => {
    render(<SettingsTablist tabs={tabs} />);
    const firstTab = screen.getByRole('tab', { name: 'Profile' });
    expect(firstTab).toHaveAttribute('aria-selected', 'true');
    expect(firstTab).toHaveAttribute('tabindex', '0');
  });

  it('other tabs have tabindex=-1 initially', () => {
    render(<SettingsTablist tabs={tabs} />);
    const secondTab = screen.getByRole('tab', { name: 'Security' });
    expect(secondTab).toHaveAttribute('tabindex', '-1');
  });

  it('ArrowRight moves focus to next tab (manual activation: does not select)', async () => {
    const user = userEvent.setup();
    render(<SettingsTablist tabs={tabs} />);

    const firstTab = screen.getByRole('tab', { name: 'Profile' });
    await user.click(firstTab);
    await user.keyboard('{ArrowRight}');

    const secondTab = screen.getByRole('tab', { name: 'Security' });
    expect(secondTab).toHaveFocus();
    // Manual activation: focus moved but tab not yet selected
    expect(secondTab).toHaveAttribute('aria-selected', 'false');
    expect(firstTab).toHaveAttribute('aria-selected', 'true');
  });

  it('Enter activates the focused tab', async () => {
    const user = userEvent.setup();
    render(<SettingsTablist tabs={tabs} />);

    const firstTab = screen.getByRole('tab', { name: 'Profile' });
    await user.click(firstTab);
    await user.keyboard('{ArrowRight}');
    await user.keyboard('{Enter}');

    const secondTab = screen.getByRole('tab', { name: 'Security' });
    expect(secondTab).toHaveAttribute('aria-selected', 'true');
    // Profile panel hidden; Security panel visible
    expect(screen.getByText('Security content')).not.toHaveAttribute('hidden');
    expect(screen.queryByText('Profile content')?.closest('[role="tabpanel"]'))
      .toHaveAttribute('hidden');
  });

  it('ArrowRight wraps from last tab to first', async () => {
    const user = userEvent.setup();
    render(<SettingsTablist tabs={tabs} />);

    const lastTab = screen.getByRole('tab', { name: 'Billing' });
    await user.click(lastTab);
    await user.keyboard('{ArrowRight}');

    const firstTab = screen.getByRole('tab', { name: 'Profile' });
    expect(firstTab).toHaveFocus();
  });
});

3.3 Focus Restoration Test

TYPESCRIPT
// tests/integration/focus/DialogFocusRestoration.test.tsx
import { describe, it, expect } from 'vitest';
import { render, screen } from '@testing-library/react';
import userEvent from '@testing-library/user-event';
import { AccessibleDialog } from '@/components/AccessibleDialog';

describe('Dialog focus management', () => {
  it('moves focus to dialog on open', async () => {
    const user = userEvent.setup();
    const { rerender } = render(
      <AccessibleDialog isOpen={false} triggerId="open-btn" onClose={() => {}}>
        <p>Dialog content</p>
        <button id="dialog-close">Close</button>
      </AccessibleDialog>
    );

    rerender(
      <AccessibleDialog isOpen={true} triggerId="open-btn" onClose={() => {}}>
        <p>Dialog content</p>
        <button id="dialog-close">Close</button>
      </AccessibleDialog>
    );

    // First focusable element inside dialog should receive focus
    expect(screen.getByRole('button', { name: /close/i })).toHaveFocus();
  });

  it('restores focus to trigger element on close', async () => {
    const user = userEvent.setup();
    const triggerButton = document.createElement('button');
    triggerButton.id = 'open-btn';
    triggerButton.textContent = 'Open settings';
    document.body.appendChild(triggerButton);

    const onClose = vi.fn();
    const { rerender } = render(
      <AccessibleDialog isOpen={true} triggerId="open-btn" onClose={onClose}>
        <button>Dialog action</button>
      </AccessibleDialog>
    );

    rerender(
      <AccessibleDialog isOpen={false} triggerId="open-btn" onClose={onClose}>
        <button>Dialog action</button>
      </AccessibleDialog>
    );

    expect(triggerButton).toHaveFocus();
    document.body.removeChild(triggerButton);
  });
});

4. End-to-End Testing with Playwright + axe-playwright

4.1 Setup

BASH
npm install --save-dev @playwright/test @axe-core/playwright
npx playwright install chromium
TYPESCRIPT
// playwright.config.ts
import { defineConfig } from '@playwright/test';

export default defineConfig({
  testDir: './tests/e2e',
  use: {
    baseURL: 'http://localhost:3000',
    // Run with accessibility features enabled
    contextOptions: {
      reducedMotion: 'no-preference',
    },
  },
  webServer: {
    command: 'npm run dev',
    port: 3000,
    reuseExistingServer: !process.env.CI,
  },
});

4.2 Full-Page AXTree Scan

TYPESCRIPT
// tests/e2e/accessibility/portal-axe.spec.ts
import { test, expect } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';

test.describe('Portal Canvas accessibility — full-page axe scan', () => {
  test('dashboard page has no critical or serious violations', async ({ page }) => {
    await page.goto('/dashboard');

    const accessibilityScanResults = await new AxeBuilder({ page })
      .withTags(['wcag2a', 'wcag2aa', 'wcag21aa', 'wcag22aa'])
      .exclude('.third-party-embed')  // Explicitly exclude un-fixable third-party content
      .analyze();

    // Filter to critical and serious — ignore minor/moderate for now
    const blocking = accessibilityScanResults.violations.filter(
      (v) => v.impact === 'critical' || v.impact === 'serious'
    );

    expect(blocking, formatViolations(blocking)).toHaveLength(0);
  });

  test('settings page has no critical or serious violations', async ({ page }) => {
    await page.goto('/settings');
    const results = await new AxeBuilder({ page })
      .withTags(['wcag2a', 'wcag2aa', 'wcag21aa'])
      .analyze();

    const blocking = results.violations.filter(
      (v) => v.impact === 'critical' || v.impact === 'serious'
    );
    expect(blocking, formatViolations(blocking)).toHaveLength(0);
  });
});

function formatViolations(violations: typeof [] & { id: string; description: string; nodes: unknown[] }[]) {
  if (violations.length === 0) return '';
  return violations
    .map(
      (v) =>
        `\n  [${v.impact}] ${v.id}: ${v.description}\n  Affected elements: ${(v.nodes as unknown[]).length}`
    )
    .join('\n');
}

4.3 Accessibility Snapshot Testing

Playwright's accessibility snapshot captures the AXTree as a structured object, enabling regression detection on the AT-visible structure without a real screen reader:

TYPESCRIPT
// tests/e2e/accessibility/navigation-snapshot.spec.ts
import { test, expect } from '@playwright/test';

test('global navigation AXTree snapshot', async ({ page }) => {
  await page.goto('/dashboard');

  const nav = page.getByRole('navigation', { name: 'Global navigation' });
  await expect(nav).toMatchAriaSnapshot(`
    - navigation "Global navigation":
      - list:
        - listitem:
          - link "Dashboard" [current=page]
        - listitem:
          - link "Workspace"
        - listitem:
          - link "Analytics"
  `);
});

test('settings tablist AXTree snapshot', async ({ page }) => {
  await page.goto('/settings');

  const tablist = page.getByRole('tablist', { name: 'Settings categories' });
  await expect(tablist).toMatchAriaSnapshot(`
    - tablist "Settings categories":
      - tab "Profile" [selected=true]
      - tab "Security" [selected=false]
      - tab "Billing" [selected=false]
  `);
});

5. The Violation Policy System

Not every accessibility violation is immediately fixable. Third-party embeds, legacy content, and known browser-AT incompatibilities may produce violations that require documented exceptions. The violation policy system provides structure for managing these exceptions with accountability.

TYPESCRIPT
// tests/a11y-violations-policy.ts
// Every accepted violation must have:
// - A unique ID
// - The axe rule ID it suppresses
// - An owner (GitHub handle)
// - An expiry date (no longer than 90 days)
// - A documented justification

export interface AcceptedViolation {
  id: string;
  ruleId: string;
  owner: string;
  expiresAt: string; // ISO 8601
  justification: string;
  affectedSelectors: string[];
}

export const acceptedViolations: AcceptedViolation[] = [
  {
    id: 'AV-001',
    ruleId: 'color-contrast',
    owner: '@design-system-team',
    expiresAt: '2027-01-15',
    justification:
      'Legacy chart library color palette requires upstream fix. Tracked in issue #1234. ' +
      'Chart data is available in accessible data table form via the "View data" toggle.',
    affectedSelectors: ['.recharts-legend-item-text'],
  },
];

6. CI Pipeline Integration

YAML
# .github/workflows/accessibility.yml
name: Accessibility CI

on:
  pull_request:
    branches: [main]
  push:
    branches: [main]

jobs:
  a11y-unit:
    name: Unit Accessibility Tests
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: '20' }
      - run: npm ci
      - name: Run vitest-axe unit tests
        run: npx vitest run tests/unit tests/integration --reporter=verbose

  a11y-e2e:
    name: E2E Accessibility Scan
    runs-on: ubuntu-latest
    needs: a11y-unit
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: '20' }
      - run: npm ci
      - run: npx playwright install chromium --with-deps
      - name: Build application
        run: npm run build
      - name: Run Playwright accessibility tests
        run: npx playwright test tests/e2e/accessibility/
      - name: Upload violation report on failure
        if: failure()
        uses: actions/upload-artifact@v4
        with:
          name: axe-violation-report
          path: test-results/

Summary

Concept Rule
Accessibility regressions produce no JavaScript errors — they must be caught by an automated testing pyramid that covers ARIA roles/names (unit), keyboard interaction sequences (integration), and full AXTree correctness (E2E). Every PR that touches interactive components must pass all three layers of the accessibility testing pyramid.
axe unit tests should disable color-contrast — jsdom computes no CSS, making contrast results meaningless; run contrast checks only in Playwright E2E against a real Chromium instance. Configure baseAxeConfig with contrast disabled for unit/integration; enable it in Playwright only.
Playwright's toMatchAriaSnapshot() catches structural AXTree regressions — an engineer renaming an aria-label will break the snapshot test before the change is merged. Add AXTree snapshot tests for all landmark regions and compound widgets.
The violation policy system provides structured, time-boxed exceptions for accepted violations; entries without a documented justification, owner, and expiry date are never valid. Every accepted violation in the policy file must have an owner, a 90-day max expiry, and a tracked issue link.

What's Next

This is the final article in the Frontend Web Accessibility & Inclusive Architecture series. The complete series has built a production-grade accessible design system from the AT runtime model (Part 1) through AXTree compilation (Part 2), focus state machines (Part 3), APG-compliant compound widgets (Part 4), live region architecture (Part 5), and the full automated testing pyramid (Part 6). The Portal Canvas now has every layer of its accessibility contract enforced — architecturally, at runtime, and in CI.

Research & Synthesis Note

This article was developed with AI-assisted deep search, specification cross-referencing, and technical research synthesis.

#Accessibility#a11y#axe-core#Vitest#Playwright#CI/CD#Testing
Siddhant Deval

Written by Siddhant Deval

Senior Full-Stack Engineer building high-scale architectures, browser performance engineering systems, and SaaS platforms.

LinkedIn·GitHub·Twitter
Series·Part 6 of 6

Frontend Accessibility Architecture: From Semantic DOM Foundations to Resilient Accessible State Engines

Curriculum Roadmap
01Accessibility Foundations: Assistive Technology Runtimes, WCAG POUR & Semantic Document Architecture
02Semantic HTML & The Accessibility Tree: How Browsers Transform DOM into AXTree Nodes
03Deterministic Keyboard Navigation & Focus State Engines: Traps, Restoration & Roving TabIndex
04WAI-ARIA Compound Widgets: Building Accessible Comboboxes, Menus, and Dialogs from Scratch
05Live Regions, Streaming State & Async Announcers: Managing Dynamic Content & Toast Stacks
06Production Accessibility Engineering: Axe-Core, Headless Audits & Continuous Quality Gates
Reading
Previous ChapterLive Regions, Streaming State & Async Announcers: Managing Dynamic Content & Toast Stacks
Previous Article
Accessibility Foundations: Assistive Technology Runtimes, WCAG POUR & Semantic Document Architecture
Next Article
Mental Model Shift: Thinking in Domain Objects Instead of Re-Renders