Context

At the moment, ScanAPI uses both DeepSource and Ruff for code quality checks. While both tools are valuable, their responsibilities overlap, which has introduced inconsistencies in how code is validated locally versus in CI.

A concrete example is line length enforcement:

• Ruff is configured with line-length = 80, but E501 (line too long) is ignored
• DeepSource enforces max_line_length = 80

As a result:

• Code may pass locally (Ruff does not report issues)
• The same code may fail or raise warnings in CI (DeepSource)

Ex: https://github.com/scanapi/scanapi/pull/866/checks

This creates friction for contributors and reduces confidence in local tooling.

More broadly, this setup increases cognitive load, as contributors must understand multiple tools with overlapping responsibilities.

Additionally, DeepSource provides automated PR comments and suggestions, which are not currently replicated by the existing setup.

At the same time, the repository already includes:

• A well-defined Makefile for local execution
• GitHub Actions for CI (tests, coverage, lint, mypy)
• GitHub Advanced Security (CodeQL, Dependabot, secret scanning)

This means most of DeepSource’s responsibilities are already covered elsewhere.

Problem

• Duplicate responsibility between Ruff and DeepSource (e.g. line length, linting rules)
• Inconsistent feedback between local development and CI
• Contributors cannot fully rely on local checks before opening a PR
• Additional complexity in maintaining multiple configurations
• DeepSource introduces a non-standard tool in the Python OSS ecosystem
• Quality checks are partially implicit (DeepSource) and partially explicit (Makefile/CI)
• Loss of clarity on how PR feedback is generated (DeepSource vs CI vs local tools)

Constraints to keep in mind

• Contributors should be able to run all checks locally via Makefile
• CI should mirror local behavior
• The setup should follow widely adopted Python tooling standards
• The solution must remain easy to maintain
• Security and quality checks should be explicit and modular
• PR feedback should remain clear and actionable

DeepSource Coverage Mapping

Observations

• The project already has a strong and modern setup:
  • Ruff for linting
  • mypy enforced in CI
  • pytest + coverage + Codecov
• Security is already covered via:
  • CodeQL
  • Dependabot
  • Secret scanning
• Bandit exists but is not enforced in CI yet
• Docstring coverage is currently enforced via DeepSource only
• DeepSource is not adding unique value beyond:
  • docstring coverage
  • PR feedback/comments
  • some additional heuristics (e.g. performance insights)

Gap Summary (After Removing DeepSource)

Remaining actions needed:

• Replace docstring coverage with interrogate
• Replace PR comments/feedback with:
  • reviewdog OR
  • Ruff --output-format=github (simpler alternative)
• Optionally:
  • Add make bandit to CI
  • Add pip-audit for explicit dependency scanning
• Decide on line-length enforcement:
  • Enable E501 in Ruff OR ignore it everywhere

Possible Directions

Proposal

Remove DeepSource and rely on existing tooling + small additions.

Proposed stack:

• Ruff → lint, formatting, anti-patterns
• pytest + pytest-cov → testing and coverage
• mypy → type checking
• interrogate → docstring coverage
• reviewdog (optional) → PR comments and feedback
• Bandit → security linting (optional CI step)
• GitHub Advanced Security → code security
• Dependabot → dependency security
• Secret scanning → secrets protection

Consequences

Positive:

• Full alignment between local (make) and CI
• No duplication of rules
• Lower contributor friction
• Explicit, maintainable tooling
• No loss in security coverage
• Faster feedback loop
• Flexible PR feedback (reviewdog)

Negative:

• Loss of DeepSource’s centralized dashboard
• Need to explicitly configure PR feedback (reviewdog or GitHub)
• Need to replace docstring coverage tool
• Need to optionally enforce Bandit in CI

We’d love to hear your thoughts on this proposal. In particular:

• Do you see any gaps in coverage compared to what DeepSource provides today?
• Are there concerns about contributor experience or onboarding with this change?
• Are there other tools or approaches we haven’t considered that could better replace DeepSource?

This decision impacts both maintainability and contributor workflow, so feedback from the community is especially valuable before moving forward. 🙌