Skip to content

feat(language-service): implement pull-based diagnostics (LSP 3.17)#66700

Open
kbrilla wants to merge 2 commits intoangular:mainfrom
kbrilla:feat/pull-based-diagnostics
Open

feat(language-service): implement pull-based diagnostics (LSP 3.17)#66700
kbrilla wants to merge 2 commits intoangular:mainfrom
kbrilla:feat/pull-based-diagnostics

Conversation

@kbrilla
Copy link
Copy Markdown
Contributor

@kbrilla kbrilla commented Jan 21, 2026

PR: feat(language-service): implement pull-based diagnostics (LSP 3.17)

Description

Implements Pull-Based Diagnostics (LSP 3.17) for the Angular Language Service, allowing the VS Code client to request diagnostics on-demand rather than having the server push them on every change.

This feature works together with #66668 (Client-Side File Watching) to provide comprehensive performance improvements across the entire development workflow:

Features

LSP 3.17 Pull Diagnostics Support

  • textDocument/diagnostic handler for document-level diagnostics
  • workspace/diagnostic handler for workspace-wide diagnostics
  • Result caching via resultId to skip unchanged documents
  • workspace/diagnostic/refresh for server-initiated refresh
  • Automatic fallback to push-based diagnostics for older clients

Result Caching

When a file hasn't changed and the client provides the previousResultId, the server returns an Unchanged report in ~0.2ms instead of recomputing diagnostics:

Client                          Server
  |                                |
  |-- textDocument/diagnostic ---->|
  |      (with previousResultId)   |
  |                                |
  |<-- Unchanged (resultId match)--|  <- Cache hit: ~0.2ms
  |         OR                     |
  |<-- Full (new diagnostics) ----|  <- Cache miss: compute

How They Work Together

Phase Optimization Benefit
Project Open #66668 File Watching Fast initialization, no ENOSPC errors
File Editing Pull Diagnostics Cached results, client-controlled timing
Tab Switching Pull Diagnostics Instant cached responses
Multi-file Changes Pull Diagnostics Batch workspace requests

Testing

  • 6 new LSP integration tests covering:
    • Basic pull diagnostics request
    • Unchanged report caching
    • Full report on document change
    • Valid template diagnostics
    • Related information in diagnostics
    • Workspace diagnostics

Measured Performance (Integration Tests)

Scenario Result Notes
Cached (unchanged) request 0.2ms Returns Unchanged report
Switch between files (cached) <1ms Both files return cached
Workspace diagnostics ~2000ms Batched request for all files
First request (cold) ~200-500ms Initial computation

Breaking Changes

None

Related Issues

Closes #66699

Complements #66668 (Client-Side File Watching)

Dependencies

This PR is built on top of the LSP library upgrade PR and should be merged after it:


AI Disclosure

This PR was developed using Claude Opus 4.5 AI assistant under human orchestration and review by @kbrilla.

@pullapprove pullapprove Bot requested a review from devversion January 21, 2026 22:23
@angular-robot angular-robot Bot added detected: feature PR contains a feature commit area: build & ci Related the build and CI infrastructure of the project area: language-service Issues related to Angular's VS Code language service labels Jan 21, 2026
@ngbot ngbot Bot modified the milestone: Backlog Jan 21, 2026
@JeanMeche JeanMeche requested a review from atscott January 21, 2026 22:29
@alan-agius4 alan-agius4 added area: vscode-extension Issues related to the Angular Language Service VsCode extension and removed area: language-service Issues related to Angular's VS Code language service labels Jan 22, 2026
@atscott
Copy link
Copy Markdown
Contributor

atscott commented Jan 23, 2026

/gemini review

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This is an excellent pull request that implements a significant performance improvement with pull-based diagnostics. The code is well-structured, particularly the new diagnostics handler and the clean integration into the session management. The dependency upgrades and necessary code adaptations for the new LSP library version are handled correctly. The addition of a comprehensive test suite is also a great step to ensure the feature's stability. I have one suggestion to improve the robustness of the new tests by replacing fixed timeouts with a more reliable synchronization mechanism.

});

// Wait a bit for the server to process
await setTimeout(500);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using a fixed setTimeout can lead to flaky tests if the server takes longer than expected to process. A more robust approach would be to wait for the server to signal that it's ready for diagnostics to be pulled. The server does this by sending a workspace/diagnostic/refresh request.

You can create a promise that resolves when this request is received and await it instead of using setTimeout. This would make the test more reliable and potentially faster.

This pattern should be applied to the other new tests in this suite that use setTimeout for waiting.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tests now use an event-driven approach instead of fixed setTimeout(500). They wait for the server's workspace/diagnostic/refresh notification via waitForDiagnosticRefresh(), making the tests more robust and faster.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. All tests now use waitForDiagnosticRefresh() which awaits the server's workspace/diagnostic/refresh notification instead of a fixed setTimeout(500). This is event-driven and resolves immediately when the server signals readiness.

* Extract the numeric version from TypeScript's ScriptInfo version string.
* Version format is like "ScriptInfo-0-123" - we extract the final numeric part.
*/
function extractVersionNumber(versionString: string): number {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Risk: You are relying on the internal format of ScriptInfo version strings (e.g., "SVC-1-123").

  • If the format changes (e.g., to a hash or UUID), this function returns 0.
  • Bug: If it returns 0 for changed content (because parsing failed), the cache check currentVersionNum === cached.version (0 === 0) will falsely return DocumentDiagnosticReportKind.Unchanged.
  • Fix: Store the full versionString in the cache and compare strings. The LSP version field in the report can be null or valid integer if available, but for our internal change detection, exact string equality is safer and more robust.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of parsing the internal TypeScript ScriptInfo version string format (which could change), the code now stores and compares the full version string directly. This prevents false "unchanged" reports if parsing fails.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. The extractVersionNumber function is removed. The cache now stores and compares the full version string directly via scriptInfo.getLatestVersion(), using exact string equality. No parsing, no risk of false cache hits.

// Get all open files from the session
const openFiles = session.getOpenFiles();

for (const filePath of openFiles) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Observation: onWorkspaceDiagnostic iterates synchronously over all open files and calls getSemanticDiagnostics.

  • Impact: On a large project with many open files, this could freeze the Node.js process for a significant time, blocking other LSP requests (like completion or hover).
  • Fix: Consider awaiting setImmediate or a small delay between file processing to yield to the event loop, allowing other requests to be handled.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The onWorkspaceDiagnostic function now yields to the event loop between processing files using await yieldToEventLoop() (which uses setImmediate), preventing the server from freezing on large workspaces with many open files.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. onWorkspaceDiagnostic now awaits yieldToEventLoop() (which uses setImmediate) between processing each file, allowing hover/completion/etc. requests to be handled during workspace diagnostic computation.

@kbrilla kbrilla force-pushed the feat/pull-based-diagnostics branch from 7059f72 to f6f6663 Compare January 25, 2026 21:18
@angular-robot angular-robot Bot added the area: language-service Issues related to Angular's VS Code language service label Jan 25, 2026
@kbrilla kbrilla force-pushed the feat/pull-based-diagnostics branch from d508f15 to 01f44fc Compare February 4, 2026 21:56
@kbrilla kbrilla requested a review from atscott February 4, 2026 21:57
Comment on lines +87 to +94
if (isDebugMode) {
console.timeEnd(label);
}

const suggestionLabel = `Pull diagnostics - getSuggestionDiagnostics for ${fileName}`;
if (isDebugMode) {
console.time(suggestionLabel);
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (isDebugMode) {
console.timeEnd(label);
}
const suggestionLabel = `Pull diagnostics - getSuggestionDiagnostics for ${fileName}`;
if (isDebugMode) {
console.time(suggestionLabel);
}
const suggestionLabel = `Pull diagnostics - getSuggestionDiagnostics for ${fileName}`;
if (isDebugMode) {
console.timeEnd(label);
console.time(suggestionLabel);
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applied. Both console.timeEnd(semanticLabel) and console.time(suggestionLabel) are now in a single if (isDebugMode) block. Also renamed labelsemanticLabel for consistency with suggestionLabel.

* Clear all diagnostic caches.
* Should be called on workspace reset or server restart.
*/
export function clearAllDiagnosticCaches(): void {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this appears unused. It should likely be hooked into onDidChangeConfiguration or shutdown, or removed if unnecessary.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed. Only clearDiagnosticCache(uri) remains — called when individual documents are closed. Since the cache is module-scoped and the server process terminates on shutdown, a full-cache clear is not needed.

* Uses setImmediate between files to yield to the event loop, preventing
* the server from blocking on large workspaces with many open files.
*/
export async function onWorkspaceDiagnostic(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like there could/should be more code sharing between this and the onDocumentDiagnostic function?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refactored. Extracted a shared computeDiagnosticsForFile(session, uri, previousResultId) function that handles caching, version checking, and diagnostic computation. Both onDocumentDiagnostic and onWorkspaceDiagnostic now delegate to it — onDocumentDiagnostic is just a one-liner wrapper.

/**
* Handle the workspace/diagnostic request (LSP 3.17 Pull Diagnostics).
*
* This handler provides diagnostics for all open files in the workspace.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From what I gather, the workspace/diagnostic LSP method is explicitly designed to provide diagnostics for the entire project (workspace), not just open files.

Introduced in LSP 3.17, this "pull" model allows the client to actively ask the server for diagnostics across the entire project.
Scope: Specifically designed to handle closed files and provide a project-wide overview.
Workflow: The client can request a "workspace" report. This is what allows you to see errors in a file you haven't even opened yet, similar to how a full build command would behave.

Some servers (like pyright or rust-analyzer) have specific settings to toggle "check all files" vs. "check open files only" because full-workspace scanning is resource-heavy.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. onWorkspaceDiagnostic now calls session.getProjectFileNames() which enumerates all source files across all configured Angular projects (via project.getFileNames(true, true) — excludes .d.ts library files and config files). Open files are processed first for priority, followed by remaining project files. This provides full project-wide diagnostic coverage.

* Generate a unique result ID for caching purposes.
*/
function generateResultId(): string {
return `${Date.now()}-${Math.random().toString(36).substring(2, 9)}`;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just use a counter that's incremented?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed. Now uses a simple monotonically increasing counter: let nextResultId = 1; function generateResultId(): string { return String(nextResultId++); }

@kbrilla kbrilla force-pushed the feat/pull-based-diagnostics branch 2 times, most recently from 9577b5b to bba9fb2 Compare February 8, 2026 14:53
@kbrilla kbrilla requested a review from atscott February 8, 2026 14:57
@kbrilla kbrilla force-pushed the feat/pull-based-diagnostics branch from bba9fb2 to 8a4bb40 Compare February 8, 2026 18:41
kbrilla added a commit to kbrilla/angular that referenced this pull request Feb 8, 2026
…agnostics

When pull-based diagnostics (LSP 3.17) is active, progressive project init
now sends a workspace/diagnostic/refresh notification instead of computing
and pushing diagnostics for each open file. This avoids wasted computation
since the pull-based client would discard pushed diagnostics anyway.

Falls back to the chunked push-based approach when pull diagnostics is not
supported by the client.

Depends on:
- PR angular#66700 (Pull-Based Diagnostics)
- PR angular#66966 (Progressive Project Initialization)
@atscott
Copy link
Copy Markdown
Contributor

atscott commented Feb 18, 2026

@kbrilla is this ready for review again?

@kbrilla
Copy link
Copy Markdown
Contributor Author

kbrilla commented Feb 19, 2026

@atscott - yes, it is ready for review.
On project startup, pull mode now does workspace/diagnostic/refresh, which was the only piece taken from #66967, which was closed.

if (isDebugMode) {
console.time(semanticLabel);
}
const diagnostics = languageService.getSemanticDiagnostics(fileName);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a shared helper that does the time logging and the two types of diagnostic fetching and share it here and with sendPendingDiagnostics

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implemented. We now use a shared helper for diagnostics computation (getLspDiagnosticsForFile) and a shared file-iteration helper (forEachDiagnosticsFile) used by both pull and push (sendPendingDiagnostics).

const currentVersion = scriptInfo.getLatestVersion();

// Check if we can return an unchanged report
if (previousResultId && cached && cached.resultId === previousResultId) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Issue: scriptInfo.getLatestVersion() only reflects changes to the file content. It does not reflect changes to the project configuration (e.g. enabling strictTemplates or changing angularCompilerOptions). If the project configuration changes, the server triggers a refreshDiagnostics(). The client will then request diagnostics again (potentially sending the previousResultId). Since the file content hasn't changed, currentVersion === cached.version will be true, and the server will return Unchanged. However, the diagnostics should be recomputed because the compilation settings changed.

Fix: The diagnosticResultCache should be cleared (or version invalidated) whenever a global diagnostic refresh occurs (i.e. inside Session.refreshDiagnostics or when the project updates). You might need to expose a clearAllDiagnosticCache() function from handlers/diagnostics.ts and call it in Session.refreshDiagnostics().

Copy link
Copy Markdown
Contributor Author

@kbrilla kbrilla Feb 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed, but via epochs (not full cache clear). We track project/global diagnostic epochs in cache entries and bump them on project/global invalidation. So config/project updates force Full recomputation even when file text version is unchanged. It is not exacly what You asked for so if I should use simpler approach for invalidating cache please say.

@angular-robot angular-robot Bot added the area: performance Issues related to performance label Feb 19, 2026
@kbrilla kbrilla requested a review from atscott February 20, 2026 15:29
Copy link
Copy Markdown
Contributor

@atscott atscott left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need 15 commits for this change. Please clean up the commit history.

if (!project || project.isClosed()) {
continue;
}
impactedProjects.add(project.getProjectName());
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the comment above, it states:

// For open/change triggers, use lightweight tiered invalidation:
// - external template edits: invalidate template + owning component files
// - TS edits: invalidate project diagnostics epoch

But this adds the project unconditionally, even when the changed file is a template file. Should this instead go in the else block below? Otherwise edits to template files invalidate the project diagnostics when changes there cannot affect additional files in the project.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

impactedProjects.add() is now in the else block — template edits only clear cache for the template + owning component files via impactedFiles, and do NOT bump the project epoch.

this.openFiles.delete(filePath);
this.projectService.closeClientFile(filePath);
// Clear the diagnostic cache for this file
clearDiagnosticCache(textDocument.uri);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The diagnosticResultCache Map in handlers/diagnostics.ts only purges entries when a document is explicitly closed (textDocument/didClose). If files in the workspace are deleted or renamed outside the boundaries of the VS Code editor buffers, their URIs persist in the cache indefinitely for the lifetime of the process. While the objects are tiny, it essentially constitutes a small memory leak in scenarios involving mass file moving/deleting. Consider implementing it as an LRU map or adding a mechanism to evict non-existent paths during project epoch bumps or watched-files events.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aded stale entry eviction in onWorkspaceDiagnostic. After each workspace diagnostic sweep, entries for URIs no longer in any active project are removed. Combined with the didChangeWatchedFiles handler clearing entries on FileChangeType.Deleted, deleted/renamed files are evicted through multiple paths. The only scenario where the sweep doesn't run is if the client doesn't support workspace/diagnostic at all — but then there's nothing to cache because onWorkspaceDiagnostic is never called in the first place.

If You prefer LRU, please say so and will add it today.

@kbrilla kbrilla force-pushed the feat/pull-based-diagnostics branch from a6f28fe to 806219c Compare March 3, 2026 10:55
@pullapprove pullapprove Bot requested a review from ENAML March 3, 2026 10:55
@angular-robot angular-robot Bot added area: compiler Issues related to `ngc`, Angular's template compiler requires: TGP This PR requires a passing TGP before merging is allowed labels Mar 3, 2026
kbrilla added 2 commits March 3, 2026 12:00
Implement support for LSP 3.17 Pull-Based Diagnostics, allowing
clients to request diagnostics on-demand rather than receiving them
pushed from the server.

Key changes:
- Add diagnostics handler with onDocumentDiagnostic and
  onWorkspaceDiagnostic implementations
- Register diagnosticProvider capability in server initialization
- Detect client support for pull diagnostics and workspace refresh
- Implement result caching with resultId to return unchanged reports
  when documents haven't been modified
- Support fallback to push-based diagnostics for older clients
- Tiered invalidation: template edits invalidate only template +
  owning component; TS edits bump project epoch for dependents
- Evict diagnostic cache on file delete from watched-files events
- Add integration tests for pull-based diagnostics

Closes angular#66699
@kbrilla kbrilla force-pushed the feat/pull-based-diagnostics branch from 806219c to bf2697d Compare March 3, 2026 11:24
@kbrilla kbrilla requested a review from atscott March 3, 2026 12:50
@kbrilla
Copy link
Copy Markdown
Contributor Author

kbrilla commented Mar 3, 2026

Changes since last review:

  • Cache eviction: Added stale entry cleanup in onWorkspaceDiagnostic — evicts cache entries for files no longer in any active project after each workspace sweep
  • Commit history: Squashed to 2 commits (feat + test) as requested
  • E2e tests: Added pull diagnostics e2e spec (diagnostics_spec.ts) — verifies diagnostics show in Problems panel and update on file changes
  • E2e config: Added --password-store=basic and --use-mock-keychain flags to prevent macOS Keychain dialogs from freezing the Extension Host; added XQuartz PATH detection for local macOS Xvfb runs (same infra fix as the markdown fences branch)
  • LSP integration tests: Added ivy_spec.ts tests covering textDocument/diagnostic, workspace/diagnostic, result caching (Unchanged reports), and cache invalidation
  • Test infrastructure: Added angular.getServerCapabilities and angular.getPushDiagnosticsCount commands for test observability — verifies pull mode is active and push diagnostics aren't sent
  • Test fixture: Added diagnostics-test.component.ts/html for exercising diagnostic flows

Copy link
Copy Markdown
Contributor

@atscott atscott left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the thorough work on this PR and for addressing the edge cases around cache invalidation and memory leaks.

As I'm evaluating this, I'm trying to balance the performance benefits against the long-term maintenance cost of the added complexity.

Specifically, I'm wondering if the resultId caching mechanism is worth the complexity it introduces (epochs, eviction logic, stateful map). Since the Angular compiler and the underlying TypeScript program are already highly incremental and cache diagnostics internally, we aren't saving core compilation work here. The saved work is primarily the TCB-to-template mapping and JSON serialization.

I'm wondering if a simpler middle ground would be a better fit for maintainability: What if we support the LSP textDocument/diagnostic request (the pull model) but without the resultId caching?

In this model, when the client pulls diagnostics for a file, we would always compute and return the full diagnostic report for that file, rather than checking a cache to return Unchanged.

This would allow us to:

  1. Keep the main benefit of pull diagnostics: The client only asks for what it needs (e.g., the active file), saving work by not processing the other hundreds of files in the project.
  2. Eliminate the complexity: We could remove the diagnosticResultCache, the project epochs, and the eviction logic entirely, making the implementation stateless and much easier to maintain.

This simpler approach also seems to be what other reference implementations do (for example, typescript-go handles the pull request but doesn't implement resultId caching - https://github.com/microsoft/typescript-go/blob/main/internal/ls/diagnostics.go, https://github.com/microsoft/typescript-go/blob/fdea8102676c0f3f5027b026a9bd4f289c1c471c/internal/lsp/server.go#L1269-L1271).

What are your thoughts on this alternative?

Comment on lines +499 to +503
// Template edit: invalidate template + owning component files only.
// Templates cannot affect other files in the project.
for (const component of languageService.getComponentLocationsForTemplate(changedFile)) {
impactedFiles.add(component.fileName);
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure diagnostics of the component file are affected by template changes unless this is actually due to the template being deleted?

Comment on lines +505 to +506
// TS edit: needs project-wide epoch bump since exports may affect dependents.
impactedProjects.add(project.getProjectName());
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this just because we don't have access to the dependency graph in the language service and you're just invalidating the entire project when a TS file changes as a result?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: build & ci Related the build and CI infrastructure of the project area: compiler Issues related to `ngc`, Angular's template compiler area: language-service Issues related to Angular's VS Code language service area: performance Issues related to performance area: vscode-extension Issues related to the Angular Language Service VsCode extension detected: feature PR contains a feature commit requires: TGP This PR requires a passing TGP before merging is allowed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(language-service): Pull-Based Diagnostics (LSP 3.17)

3 participants