Document Version: 1.0 Date: 2026-04-08 Status: ACTIVE - Defines quality requirements for engine-sim-cli project Purpose: Clear requirements for code quality, testing, and development practices

This document defines the quality standards that all team members must follow. It establishes clear expectations, protocols for handling test failures, consequences for violations, and provides training materials for the quality safeguards.

1. Test-Driven Development (TDD): Write tests BEFORE implementing features
2. Red-Green-Refactor Cycle: Tests must compile before implementation (RED phase)
3. Evidence-Based Decision Making: Gather evidence, don't speculate unless asked
4. Critical Thinking: Challenge assumptions, verify facts before proceeding
5. SOLID Principles: Single Responsibility, Open-Closed, Liskov Substitution, Interface Segregation, Dependency Inversion
6. Test Value: Tests must add real business value, not just coverage vanity
7. Determinism: Tests must be deterministic and repeatable

When a test fails, follow this protocol:

1. STOP: Immediately stop current work
2. ANALYZE: Read test failure message carefully
3. REPRODUCE: Attempt to reproduce the failure locally
4. ROOT CAUSE: Identify the root cause, not just surface symptoms
5. FIX: Implement minimal fix that addresses root cause
6. VERIFY: Run the specific failing test to verify the fix
7. REGRESSION: Run related tests to ensure no regressions
8. DOCUMENT: Document the fix with explanation

A "critical" test failure is one that:

• Blocks team members from continuing work
• Prevents builds from succeeding
• Causes production functionality to break
• Indicates architectural or design problem

Protocol for Critical Failures:

1. Immediate Notification to team-lead and product-owner
2. Stop All Related Work until failure is resolved
3. Assign Priority - Critical failures are P0, must be addressed immediately
4. Timebox Investigation - Maximum 4 hours for initial root cause analysis
5. Require Approval - Critical fixes require team approval before committing

When tests produce different results on repeated runs, this indicates loss of determinism.

Protocol:

1. Verify Test Setup - Check for hidden state, timing dependencies
2. Isolate Component - Test components in isolation
3. Check Float Precision - Floating point calculations can cause small variations
4. Review State Management - Ensure complete cleanup between test runs
5. Add Logging - Add diagnostic logging to understand behavior differences

Integration test failures indicate problems with component interaction.

Protocol:

1. Check Component Initialization - Verify all components properly initialized
2. Check Data Flow - Verify data flows correctly between components
3. Check Configuration - Verify test configuration is correct
4. Check Environment - Verify test environment is properly set up
5. Verify Mock Behavior - Ensure mocks behave as expected

Definition: Committing code without running the full test suite.

Consequences: CRITICAL

• Immediate rollback of violating commit required
• All subsequent work blocked until tests are green
• Team meeting required to discuss violation

Example: "Skipping failing test X to unblock work Y" is a violation.

Definition: Writing tests that don't compile or that test non-existent behavior.

Consequences: CRITICAL

• Test must be rewritten or removed
• Cannot commit feature until test is fixed
• Team code review required

Example: "Test for null pointer crash" that dereferences null is invalid.

Definition: Code produces different results for same inputs, making debugging difficult.

Consequences: HIGH

• Must be addressed before feature is considered complete
• Requires investigation and root cause fix
• May require refactoring

Example: Using time-based random number generation without seeding.

Definition: Deliberately choosing not to follow quality standards despite awareness.

Consequences: CRITICAL

• Immediate team intervention required
• Code review cannot approve work
• Possible disciplinary action

Example: "Skipping TDD because it takes too long" is a violation.

Definition: Tests that break easily due to implementation details not intended behavior.

Consequences: MEDIUM

• Tests must be refactored to test intent, not implementation
• Test coverage should not be primary metric
• Tests must add real business value

Example: Tests that check exact error message strings instead of exception types.

Location: verification_system.sh

Purpose: Comprehensive verification of all fixes and functionality.

How to Use:

# Run full verification
./verification_system.sh

# Run specific verification mode
./verification_system.sh --mode audio
./verification_system.sh --mode input
./verification_system.sh --mode engine

What It Tests:

• Audio output with frequency measurement
• Input response timing
• Engine startup behavior
• Binary verification
• Configuration validation

Interpreting Results:

• All PASSING: Fix is verified working
• PARTIAL PASSING: Fix works but has edge cases
• FAILING: Fix does not work, root cause analysis needed

Document: TESTING_GUIDE.md

Purpose: Guidelines for writing effective tests.

Key Principles:

1. Test Real Production Code: Don't test mocks or external libraries
2. Test Business Scenarios: Test actual usage patterns, not edge cases
3. Test Intent, Not Implementation: Test what should happen, not how it happens
4. Use Mocks to Control Scenarios: Mocks enable testing specific conditions
5. Avoid Fragile Tests: Don't depend on exact error messages or implementation details
6. Prioritize Happy Path: Test main success scenarios first, reasonable exception cases second

Test Value Criteria:

• Does this test catch real bugs?
• Does this test prevent regressions?
• Does this test verify important user requirements?
• Would this test's failure block production release?

The project enforces quality checks before commits:

Checks Performed:

1. Test Suite Status: All tests must be passing
2. Code Review: At least one team member approval required
3. SOLID Compliance: Code review must verify SOLID principles
4. Documentation: New features must be documented
5. TDD Compliance: Tests must exist for new features

When Checks Fail:

• Commit is blocked
• Root cause analysis required
• Team meeting may be required

Continuous Integration Requirements:

1. All Tests Must Pass: Build fails if any test fails
2. No Code Compilation Warnings: Treat warnings as errors
3. Code Coverage Minimum: Maintain 80%+ coverage for new code
4. Static Analysis: No violations from linters/static analyzers
5. Documentation Build: Documentation must build without errors

Purpose: Automatically verify quality standards before commits.

How It Works:

1. Test Check: Runs full test suite
2. Code Quality Check: Runs static analysis and linting
3. Documentation Check: Validates documentation completeness
4. Commit Blocking: Blocks commit if any check fails

File: .git/hooks/pre-commit

Required Checks:

# Run all tests
make test

# Check exit code
if [ $? -ne 0 ]; then
    echo "ERROR: Tests are failing. Cannot commit."
    exit 1
fi

# Run static analysis
make static-analysis

# Check for SOLID violations
make solid-check

Purpose: Track all test failures and their resolutions.

Components:

1. Failure Database: Records each test failure with details
2. Root Cause Analysis: Documents investigation and resolution
3. Prevention: Documents preventive measures

File: test/TEST_FAILURE_LOG.md

Example Entry:

## Failure #42: Deterministic Test Failure

**Date**: 2026-04-08
**Test**: SineWave_SyncPull_DeterministicRepeatability
**Severity**: CRITICAL

**Symptoms**:
- 76% difference between runs
- Loss of determinism
- Inconsistent output for same inputs

**Root Cause**:
- State accumulation between test runs
- Floating point precision issues
- Incomplete cleanup of simulator state

**Resolution**:
- Added explicit cleanup in test teardown
- Fixed floating point precision issues
- Added diagnostic logging

**Prevention**:
- All tests now include explicit cleanup
- State management utility created for test isolation
- Floating point usage guidelines added

**Verification**:
- Test now passes with <5% variance
- Verified with 10 consecutive runs

Current Components:

Audio Module
├── IAudioStrategy (Interface)
│   ├── ThreadedStrategy (Implementation)
│   └── SyncPullStrategy (Implementation)
├── IAudioHardwareProvider (Interface)
│   └── CoreAudioHardwareProvider (Implementation)
├── State Management
│   ├── AudioState
│   ├── BufferState
│   ├── Diagnostics
│   └── StrategyContext (Composer)
└── Common Utilities
    ├── CircularBuffer
    └── AudioUtils

Quality Safeguards:

• All interfaces are minimal and focused
• Each component has single responsibility
• Dependencies are inverted (depend on abstractions)
• Strategy pattern enables swappable behaviors
• Factory pattern for object creation

Components:

Verification System
├── verification_system.sh (Main Script)
├── Audio Verification
│   ├── Frequency measurement
│   ├── Amplitude analysis
│   └── Output validation
├── Input Verification
│   ├── Keypress timing
│   └── Control response
├── Engine Verification
│   ├── Startup monitoring
│   ├── RPM tracking
│   └── Hang detection
└── Binary Verification
    ├── Dependency checking
    └── Integrity validation

Quality Safeguards:

• Each verification script is independent
• Clear pass/fail criteria
• Diagnostic output for debugging
• Automated for consistency

When Tests Fail:

1. Immediate Communication: Notify team in appropriate channel
2. Stop Work: Halt related work until failure is understood
3. Root Cause Analysis: Don't just fix symptoms
4. Timebox: 4 hours maximum for initial investigation
5. Ask for Help: If stuck, escalate immediately

When Standards Violated:

1. Identify Violation: Reference this document for definition
2. Determine Severity: Use severity levels from Section 3.1
3. Follow Consequences: Apply consequences from Section 3.2
4. Learn: Document lessons learned for prevention

During Code Review:

1. Quality Checklist: Use checklist from this document
2. Test Coverage Review: Verify tests add real value
3. SOLID Review: Verify each principle for compliance
4. Documentation Review: Ensure code is self-documenting
5. Ask Questions: Challenge assumptions, verify decisions

• VERIFICATION_README.md - Comprehensive verification system documentation
• TESTING_GUIDE.md - Testing best practices and guidelines
• ARCHITECTURE_TODO.md - Architecture task tracking
• ARCHITECTURE_AUDIT.md - Architecture audit findings
• AUDIO_MODULE_ARCHITECTURE.md - Audio module architecture documentation

Historical investigation and audit documents are archived in docs/archive/:

• ARCHITECTURE_COMPARISON_REPORT.md - Architecture comparison analysis
• ACTION_PLAN.md - Original action plan
• ANALYSIS_CORRECTIONS.md - Analysis corrections
• AUDIO_PIPELINE_VERIFICATION.md - Audio pipeline verification

The quality system consists of:

• Pre-commit hooks for test and quality enforcement
• CI pipeline requirements for automated checks
• Test failure tracking and prevention
• Comprehensive standards documentation (this document)
• Training materials for team onboarding

Document End