Skip to content

Latest commit

 

History

History
448 lines (301 loc) · 21.2 KB

File metadata and controls

448 lines (301 loc) · 21.2 KB

Provar MCP

Beta: Provar MCP is currently in Beta. This is offered to all Provar users at no additional cost, and is an open source project hosted on GitHub here. General Availability is coming soon. We welcome feedback via GitHub Issues.


What is Provar MCP?

Provar MCP is an AI-assisted quality layer built directly into the Provar DX CLI. It implements the Model Context Protocol (MCP) — an open standard that lets AI assistants call tools on your behalf — and exposes a rich set of Provar project operations to AI clients such as Claude Desktop, Claude Code, and Cursor.

Once connected, your AI assistant can:

  • Inspect your Provar Automation project and surface coverage gaps
  • Generate Java Page Objects and XML test case skeletons
  • Validate every level of the test hierarchy (test cases, suites, plans, and the full project) against 30+ quality rules
  • Set up and manage your provardx-properties.json run configuration
  • Trigger Provar Automation test runs and Provar Quality Hub managed runs — all from inside a chat session

The MCP server runs entirely on your local machine. No project files, test code, or credentials are transmitted to Provar servers.


Prerequisites

Before you can use Provar MCP, ensure the following are in place:

Requirement Version Notes
Provar Automation ≥ 2.18.2 or ≥ 3.0.6 Must be installed with an activated license on the same machine. The MCP server reads license state from ~/Provar/.licenses/.
Salesforce CLI (sf) ≥ 2.x Install with npm install -g @salesforce/cli
Provar DX CLI plugin ≥ 1.5.0-beta Install with sf plugins install @provartesting/provardx-cli
Node.js ≥ 18 Installed automatically with the Salesforce CLI
An MCP-compatible AI client Claude Desktop, Claude Code (VS Code / CLI), or Cursor
An existing Provar Automation project The MCP server works best when pointed at a real project directory. Project context (connections, environments, Page Objects, test cases) is what the AI reads and reasons over.

License requirements

Provar MCP requires an active Provar Automation license on the machine where the server runs. Validation is automatic:

  1. The server reads ~/Provar/.licenses/*.properties — the same files written by the Provar Automation IDE — and checks that a license is activated and was last verified online within 48 hours.
  2. Successful validations are cached for 2 hours, so frequent server restarts do not cause repeated disk reads.
  3. If no valid license is found, the server exits immediately with a clear error message. Open Provar Automation IDE and ensure your license is activated, then retry.

There is no separate MCP license. Your existing Provar Automation license covers MCP usage.


Installation

Step 1 — Install the Salesforce CLI

npm install -g @salesforce/cli
sf --version

Step 2 — Install the Provar DX CLI plugin

sf plugins install @provartesting/provardx-cli
sf provar mcp start --help

Step 3 — Authenticate with Quality Hub (optional, recommended)

Run sf provar auth login to connect your Provar account and unlock full Quality Hub API validation (170+ rules, quality scoring). Without this, the MCP server runs in local-only mode using structural rules.

sf provar auth login

This opens a browser to the Provar login page. After you authenticate, your API key is stored at ~/.provar/credentials.json and picked up automatically by the MCP server on every subsequent tool call.

For CI/CD pipelines (GitHub Actions, Jenkins, etc.) where a browser cannot open: run sf provar auth login once on your local machine, copy the api_key value from ~/.provar/credentials.json, and store it as the PROVAR_API_KEY environment variable or secret in your pipeline. The key is valid for approximately 90 days — rotate the secret when it expires by running sf provar auth login again locally.

Step 4 — Configure your AI client

Claude Desktop

Edit the Claude Desktop MCP configuration file:

  • macOS / Linux: ~/Library/Application Support/Claude/claude_desktop_config.json
  • Windows: %APPDATA%\Claude\claude_desktop_config.json
{
  "mcpServers": {
    "provar": {
      "command": "sf",
      "args": ["provar", "mcp", "start", "--allowed-paths", "/path/to/your/provar/project"]
    }
  }
}

Restart Claude Desktop after saving. The Provar tools will appear in the tool list automatically.

Claude Code (VS Code / CLI)

Claude Code can be configured via the claude CLI command or by editing a JSON config file. Both approaches work in the terminal, the VS Code extension, and the Claude Code Desktop app.

Via terminal (recommended):

# User-scoped — works across all your projects
claude mcp add provar -s user -- sf provar mcp start --allowed-paths /path/to/your/provar/project

# Project-scoped, shared — creates .mcp.json at project root; commit to source control
claude mcp add provar -s project -- sf provar mcp start --allowed-paths /path/to/your/provar/project

Via config file — create .mcp.json at your project root:

{
  "mcpServers": {
    "provar": {
      "command": "sf",
      "args": ["provar", "mcp", "start", "--allowed-paths", "/path/to/your/provar/project"]
    }
  }
}

sf not found? GUI environments (VS Code, Claude Code Desktop) often launch with a restricted PATH that doesn't include sf. Use npx as the command instead:

# Terminal
claude mcp add provar -s user -- npx -y @salesforce/cli provar mcp start --allowed-paths /path/to/your/provar/project
{
  "mcpServers": {
    "provar": {
      "command": "npx",
      "args": ["-y", "@salesforce/cli", "provar", "mcp", "start", "--allowed-paths", "/path/to/your/provar/project"]
    }
  }
}

GitHub Copilot (VS Code)

Create .vscode/mcp.json in your workspace root (commit to share with your team):

{
  "servers": {
    "provar": {
      "type": "stdio",
      "command": "sf",
      "args": ["provar", "mcp", "start", "--allowed-paths", "${workspaceFolder}"]
    }
  }
}

Open the GitHub Copilot Chat panel and switch to Agent mode. The Provar tools will appear in the tool list.

sf not found? Replace "command": "sf" with "command": "npx" and prepend "-y", "@salesforce/cli" to the args array.

Cursor

Add to .cursor/mcp.json in your workspace root (project-level) or ~/.cursor/mcp.json (global):

{
  "mcpServers": {
    "provar": {
      "command": "sf",
      "args": ["provar", "mcp", "start", "--allowed-paths", "/path/to/your/provar/project"]
    }
  }
}

Restart Cursor after saving. The Provar tools will appear under Settings → MCP.

sf not found? Replace "command": "sf" with "command": "npx" and prepend "-y", "@salesforce/cli" to the args array.

Important: Set --allowed-paths to the root of your Provar Automation project directory (the folder containing your .testproject file). The server will only read and write files within this boundary.


Verify the connection

Once your AI client is configured, ask it:

"Call provardx_ping with message 'hello'"

Expected response:

{ "pong": "hello", "ts": "2026-04-07T...", "server": "provar-mcp@1.0.0" }

If this fails, see the Troubleshooting section.


Use cases

Inspect your project

Get an instant inventory of your Provar project — file counts, coverage gaps, and missing configurations.

Prompt:

"Use provar_project_inspect on my project at /workspace/MyProvarProject and tell me what you find — how many test cases are there, and which ones aren't covered by any test plan?"

What you get back:

  • Total test case count, suite structure, Page Object count
  • A list of test cases not referenced by any test plan (coverage gaps)
  • Whether a provardx-properties.json config file exists

Validate a test case

Score an existing test case for schema compliance and best-practice quality issues.

Prompt:

"Validate the test case at /workspace/MyProvarProject/tests/regression/LoginTest.testcase and explain any issues."

What you get back:

  • validity_score (schema compliance, 0–100) and quality_score (best practices, 0–100)
  • Specific rule violations with IDs, severities, and descriptions
  • Actionable suggestions (e.g. "Add a missing XML declaration", "Test case ID is not a valid UUID")
  • validation_source"quality_hub" if authenticated, "local" if no API key is configured

Get more: Run sf provar auth login once to unlock Quality Hub API validation (170+ rules). Without a key the tool still returns useful results using local structural rules.


Generate a Page Object

Have the AI scaffold a new Java Page Object for a Salesforce page with correct annotations and @FindBy stubs.

Prompt:

"Generate a Salesforce Page Object for the Account Detail page. Include fields for Account Name (input), Industry (select), and a Save button. Write it to /workspace/MyProvarProject/src/pageobjects/accounts/AccountDetailPage.java."

What you get back:

  • A valid Java file with @SalesforcePage annotation
  • @FindBy annotations for each field using sensible locator strategies
  • File written to disk (use dry_run: true in the tool call to preview without writing)

Generate a test case

Scaffold a new XML test case with a proper UUID, sequential step IDs, and a clean structure ready for Provar Automation.

Prompt:

"Generate a test case called 'Verify Account Creation' with steps for navigating to the Accounts page, clicking New, filling in Account Name, and saving. Write it to /workspace/MyProvarProject/tests/smoke/VerifyAccountCreation.testcase."


Set up your run configuration

Let the AI create and validate a provardx-properties.json — the properties file that tells the Provar DX CLI how to run your tests.

Prompt:

"Generate a provardx-properties.json at /workspace/MyProvarProject/provardx-properties.json with projectPath set to /workspace/MyProvarProject and provarHome set to /Applications/Provar. Then validate it and tell me if anything is missing."


Validate the full project hierarchy

Get a single quality score for your entire project — test cases, suites, plans, connections, environments, and cross-cutting rules all evaluated together.

Prompt:

"Validate the full test project at /workspace/MyProvarProject. The project has connections named SandboxOrg and ProdOrg, and environments QA and UAT. Give me a quality report."

What you get back:

  • Overall project quality score (0–100)
  • Test plan coverage percentage
  • Breakdown of violations by rule ID
  • Per-plan quality scores

Trigger a Provar Automation test run

Ask the AI to run your local Provar Automation test suite and report results.

Prompt:

"Load the properties file at /workspace/MyProvarProject/provardx-properties.json, compile the project, then run the tests and tell me the results."

The AI will chain:

  1. provar_automation_config_load — registers the properties file
  2. provar_automation_compile — compiles Page Objects
  3. provar_automation_testrun — executes the test run
  4. provar_testrun_report_locate — finds the JUnit/HTML report paths

Trigger a Quality Hub managed test run

Kick off a managed test run via Provar Quality Hub and poll until it completes.

Pre-requisite: Authenticate the Salesforce CLI against your Quality Hub org first:

sf org login web -a MyQHOrg
sf provar quality-hub connect -o MyQHOrg

Prompt:

"Connect to the Quality Hub org MyQHOrg, start a test run using config file config/smoke-run.json, and poll every 30 seconds until it completes or fails."

The AI will chain:

  1. provar_qualityhub_connect — connects to the org
  2. provar_qualityhub_testrun — triggers the run
  3. provar_qualityhub_testrun_report — polls status in a loop
  4. Reports final pass/fail status and a summary of results

Root cause analysis after a test run failure

After a failed run, ask the AI to classify failures and identify patterns.

Prompt:

"My test run just finished. Analyse the results at /workspace/MyProvarProject/Results/ and classify any failures — tell me which are pre-existing issues and which look like new regressions."

What you get back:

  • Classified failure categories (environment issue, assertion failure, locator issue, etc.)
  • Identification of Page Objects involved in failures
  • Suggested next steps

Create a Quality Hub defect from a failed test

Turn a failed test execution directly into a Quality Hub defect, without leaving your AI chat.

Prompt:

"The test 'LoginTest' failed in the last run. Create a defect in Quality Hub for it."


Available tools (reference)

Tool What it does
provardx_ping Sanity check — verifies the server is running
provar_project_inspect Inventory project artefacts and surface coverage gaps
provar_project_validate Full project quality validation from disk
provar_connection_list List connections and named environments from the project
provar_pageobject_generate Generate a Java Page Object skeleton
provar_pageobject_validate Validate Page Object quality (30+ rules)
provar_testcase_generate Generate an XML test case skeleton
provar_testcase_validate Validate test case XML (schema + best-practices scores)
provar_testcase_step_edit Atomically add or remove a single step in a test case
provar_testsuite_validate Validate a test suite hierarchy
provar_testplan_validate Validate a test plan with metadata completeness checks
provar_testplan_create Create a new test plan
provar_testplan_add-instance Wire a test case into a plan suite
provar_testplan_create-suite Create a new test suite inside a plan
provar_testplan_remove-instance Remove a test instance from a plan suite
provar_properties_generate Generate a provardx-properties.json from the standard template
provar_properties_read Read and parse a provardx-properties.json
provar_properties_set Update fields in a provardx-properties.json
provar_properties_validate Validate a provardx-properties.json against the schema
provar_ant_generate Generate an ANT build.xml for CI/CD pipeline execution
provar_ant_validate Validate an ANT build.xml
provar_automation_setup Detect or download/install Provar Automation binaries
provar_automation_config_load Register a properties file as the active config
provar_automation_compile Compile Page Objects after changes
provar_automation_metadata_download Download Salesforce metadata into the project
provar_automation_testrun Trigger a local Provar Automation test run
provar_qualityhub_connect Connect to a Quality Hub org
provar_qualityhub_display Display connected Quality Hub org info
provar_qualityhub_testrun Trigger a Quality Hub managed test run
provar_qualityhub_testrun_report Poll test run status
provar_qualityhub_testrun_abort Abort an in-progress test run
provar_qualityhub_testcase_retrieve Retrieve test cases by user story or component
provar_qualityhub_defect_create Create Quality Hub defects from failed executions
provar_qualityhub_examples_retrieve Retrieve corpus examples to ground test generation
provar_testrun_report_locate Resolve JUnit/HTML report paths after a run
provar_testrun_rca Classify failures and detect regressions
provar_nitrox_discover Discover NitroX component metadata
provar_nitrox_generate Generate a NitroX component
provar_nitrox_patch Patch a NitroX component definition
provar_nitrox_read Read a NitroX component definition
provar_nitrox_validate Validate a NitroX component

Security

  • Local only. The MCP server communicates via stdio — no TCP port is opened, no network listener is started.
  • Path-scoped. All file operations are restricted to the directories you specify via --allowed-paths. Path traversal (../) is blocked.
  • No data exfiltration. Project files, test code, and credentials are never transmitted to Provar servers.
  • Credential safety. Quality Hub and Automation tools invoke the Salesforce CLI as a subprocess. Org credentials stay in the SF CLI's own credential store and are never read or logged by the MCP server.
  • Audit log. Every tool invocation is logged to stderr with a unique request ID in structured JSON format. Capture stderr to maintain an audit trail.

Troubleshooting

"No activated Provar license found" / LICENSE_NOT_FOUND Open Provar Automation IDE → Help → Manage License → ensure the license is Activated. Then restart the MCP server.

"Warning: license validated from offline cache" (on stderr) The server started successfully but the license cache is over 2 hours old. This is a warning only. If the cache exceeds 48 hours without a successful online re-validation, the next startup will fail. Restart the server while Provar Automation IDE is connected to the internet to refresh the cache.

SF_NOT_FOUND error from Quality Hub / Automation tools The sf CLI binary is not on the PATH that the MCP server sees (common with macOS and Windows GUI apps). Use npx as the command in your MCP config — it resolves @salesforce/cli from your npm cache without needing sf on PATH:

{ "command": "npx", "args": ["-y", "@salesforce/cli", "provar", "mcp", "start", "--allowed-paths", "..."] }

Alternatively, use the full path to the sf binary (e.g. /usr/local/bin/sf on macOS).

PATH_NOT_ALLOWED error The path passed to a tool is outside the --allowed-paths root. Update --allowed-paths in your client config and restart the server.

Tools not appearing in Claude Desktop After editing claude_desktop_config.json, fully quit and reopen Claude Desktop (Cmd+Q on macOS, not just close the window).

Server starts then immediately exits Check the plugin is installed: sf plugins | grep provardx. If missing: sf plugins install @provartesting/provardx-cli.


Support