Add image support to conversations and PDF side panel - #1073

Walkthrough

The change adds image attachments with validation, previews, persistence, multimodal OpenAI-compatible requests, and localized UI text. It also updates PDF side-panel handling, source packaging, generated-output ignores, documentation, and automated coverage.

Changes

Image support

Sidebar and release support

Priority: ➖ Normal

Estimated code review effort: 5 (Critical) | ~90 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant User
  participant InputBox
  participant ConversationCard
  participant ImageValidation
  participant OpenAICompatibleAPI
  User->>InputBox: select, paste, or drop images
  InputBox->>ImageValidation: validate and read data URLs
  InputBox->>ConversationCard: submit text and images
  ConversationCard->>OpenAICompatibleAPI: send validated session
  OpenAICompatibleAPI-->>ConversationCard: stream response and preserve image record

Suggested reviewers: peterdavehello

❌ Failed checks (1 warning)

• X
• Mastodon
• Reddit
• LinkedIn

• Add image selection, paste, drop, preview, validation, and OpenAI-compatible multimodal requests.
• Preserve images across conversation history, follow-ups, retries, clearing, and local storage.
• Reliably open PDF side panels and cover behavior with unit and Edge tests.

graph TD
  Menu["PDF Menu"] -->|current window| Panel["Native Side Panel"] --> Input["Image Input"] --> Validate["Image Validation"] --> Session["Conversation Session"] --> Payload["Multimodal Payload"] --> API["Vision API"]
  Session --> History["History Preview"]

Loading

The following are alternative approaches to this PR:

1. Externalize images in IndexedDB

• ➕ Keeps large base64 values out of serialized conversation records
• ➕ Could support larger or longer-lived image histories
• ➖ Requires attachment identifiers, migrations, cleanup, and export handling
• ➖ Adds asynchronous storage reads before requests and history rendering

2. Provider-specific upload adapters

• ➕ Could use native file-upload capabilities and avoid inline base64 payloads
• ➕ May support providers outside the OpenAI-compatible protocol
• ➖ Introduces provider-specific lifecycle and authentication behavior
• ➖ Substantially expands scope beyond standard Chat Completions image content

Recommendation: Use the PR's inline data-URL approach for this scope: it maps directly to OpenAI-compatible image_url content, preserves existing session storage semantics, and is bounded by strict message and session limits. Retain the synchronous WINDOW_ID_CURRENT fallback because asynchronous tab lookup can lose the required browser gesture. Consider IndexedDB only if future requirements materially increase attachment size or retention.

Enhancement (16) +886 / -31

main.jsonAdd English image workflow messages +16/-0

Add English image workflow messages

• Adds labels, guidance, progress text, and validation errors for image attachments.

src/_locales/en/main.json

main.jsonAdd Simplified Chinese image messages +16/-0

Add Simplified Chinese image messages

• Localizes attachment controls, image guidance, reading state, and validation errors.

src/_locales/zh-hans/main.json

main.jsonAdd Traditional Chinese image messages +16/-0

Add Traditional Chinese image messages

• Localizes attachment controls, image guidance, reading state, and validation errors.

src/_locales/zh-hant/main.json

index.jsxIntegrate images into conversation lifecycle +35/-6

Integrate images into conversation lifecycle

• Passes image drafts into submissions, validates endpoint compatibility and session limits, renders historical images, restores retry attachments, and clears drafts with conversations.

src/components/ConversationCard/index.jsx

session.mjsRestore images when retrying messages +1/-0

Restore images when retrying messages

• Copies the retried record's images into the retry session so interrupted and completed retries retain their attachments.

src/components/ConversationCard/session.mjs

index.jsxRender images in user messages +17/-1

Render images in user messages

• Displays validated image data URLs below expanded user questions and defines the corresponding component property type.

src/components/ConversationItem/index.jsx

images.mjsValidate and encode selected image files +118/-0

Validate and encode selected image files

• Defines supported MIME types and attachment limits, filters invalid batches, and reads accepted files into typed base64 data URLs.

src/components/InputBox/images.mjs

index.jsxAdd image attachment controls and draft handling +371/-18

Add image attachment controls and draft handling

• Supports file selection, paste, drop, previews, removal, image-only submission, localized errors, and asynchronous read protection. It also prevents stale submissions from clearing newer drafts and disables images for unsupported modes.

src/components/InputBox/index.jsx

App.jsxIdentify the active image-support panel revision +8/-0

Identify the active image-support panel revision

• Shows a compact revision marker that distinguishes native sidebar surfaces from standalone panel usage.

src/pages/IndependentPanel/App.jsx

images.mjsCentralize multimodal image validation and serialization +238/-0

Centralize multimodal image validation and serialization

• Validates image data URLs and per-message/session limits, determines endpoint eligibility, and builds standard OpenAI text and image_url content. Native Ollama chat and unsupported endpoint types are explicitly rejected.

src/services/apis/images.mjs

openai-api.mjsGuard OpenAI-compatible requests containing images +5/-0

Guard OpenAI-compatible requests containing images

• Validates all current and historical attachments before resolving and sending an OpenAI-compatible request.

src/services/apis/openai-api.mjs

openai-compatible-core.mjsSend multimodal OpenAI chat messages +18/-1

Send multimodal OpenAI chat messages

• Rejects images on incompatible endpoints and serializes current user attachments into standard multimodal Chat Completions content.

src/services/apis/openai-compatible-core.mjs

shared.mjsPersist images with conversation records +13/-3

Persist images with conversation records

• Copies current images into newly created records and preserves them when an existing response is replaced during retry.

src/services/apis/shared.mjs

init-session.mjsInitialize image-aware sessions +2/-0

Initialize image-aware sessions

• Adds an images collection to the common session shape for current-turn attachments.

src/services/init-session.mjs

wrappers.mjsReject unsupported images before request execution +6/-1

Reject unsupported images before request execution

• Validates session attachments and endpoint compatibility before posting session state or invoking provider executors.

src/services/wrappers.mjs

get-conversation-pairs.mjsInclude historical images in chat context +6/-1

Include historical images in chat context

• Builds multimodal user content for stored conversation records while retaining the existing text-only completion path.

src/utils/get-conversation-pairs.mjs

Bug fix (6) +51 / -118

index.mjsRemove per-tab side-panel path updates +0/-71

Remove per-tab side-panel path updates

• Removes the tabs.onUpdated handler that configured tab-specific side-panel options, allowing the manifest-defined global panel route to remain authoritative.

src/background/index.mjs

menus.mjsRegister context-menu clicks before asynchronous setup +37/-33

Register context-menu clicks before asynchronous setup

• Installs the click listener before menu removal, storage, or localization can yield, preventing the first service-worker-waking click from being missed. Refresh failures are also logged without duplicating listeners.

src/background/menus.mjs

redact.mjsRedact image payloads from diagnostics +1/-0

Redact image payloads from diagnostics

• Treats image fields as sensitive so base64 attachment data is removed from logged or reported structures.

src/background/redact.mjs

index.mjsOpen PDF side panels with current-window fallback +7/-11

Open PDF side panels with current-window fallback

• Uses a valid callback window ID when present and synchronously falls back to WINDOW_ID_CURRENT when PDF callbacks omit it. The global panel route avoids Edge's unreliable tab-specific PDF behavior.

src/content-script/menu-tools/index.mjs

index.htmlUse standards mode and full-height panel roots +4/-3

Use standards mode and full-height panel roots

• Adds an HTML doctype and propagates full-height styling through the document, body, and application root.

src/pages/IndependentPanel/index.html

styles.scssFill the native side-panel viewport +2/-0

Fill the native side-panel viewport

• Makes the IndependentPanel root occupy the available height for stable narrow-panel layout.

src/pages/IndependentPanel/styles.scss

Tests (8) +1471 / -11

images-smoke.cjsAdd Edge image conversation smoke coverage +397/-0

Add Edge image conversation smoke coverage

• Exercises selection, removal, paste, drop, image-only sending, API payloads, retries, history retention, clearing, unsupported modes, and narrow-width rendering against a mock server.

tests/manual/images-smoke.cjs

sidepanel-smoke.cjsAdd native Edge PDF side-panel smoke coverage +741/-0

Add native Edge PDF side-panel smoke coverage

• Launches an instrumented extension against a local PDF and verifies missing-window callbacks, native panel creation, image controls, clipping, previews, tab stability, and absence of PDF refetches.

tests/manual/sidepanel-smoke.cjs

menus.test.mjsTest service-worker menu wake-up ordering +43/-0

Test service-worker menu wake-up ordering

• Confirms the click listener exists before asynchronous menu removal finishes and remains single-registered across refreshes.

tests/unit/background/menus.test.mjs

redact.test.mjsTest image payload redaction +1/-0

Test image payload redaction

• Adds base64 image data to the sensitive-field redaction fixture.

tests/unit/background/redact.test.mjs

conversation-card-session.test.mjsTest retry image retention +11/-0

Test retry image retention

• Verifies that retries restore target attachments and interrupted retry history keeps images without mutating the source session.

tests/unit/components/conversation-card-session.test.mjs

input-box-images.test.mjsTest input image validation and encoding +45/-0

Test input image validation and encoding

• Covers supported types, empty and oversized files, count and total limits, and the non-FileReader data-URL fallback.

tests/unit/components/input-box-images.test.mjs

menu-tools.test.mjsTest current-window side-panel fallback +12/-11

Test current-window side-panel fallback

• Updates expectations for global side-panel opening and verifies synchronous WINDOW_ID_CURRENT use for missing or invalid callback window IDs.

tests/unit/content-script/menu-tools.test.mjs

images.test.mjsTest multimodal API validation and persistence +221/-0

Test multimodal API validation and persistence

• Covers data-URL validation, size limits, endpoint eligibility, current and historical payload serialization, unsupported completions, record persistence, and retry updates.

tests/unit/services/apis/images.test.mjs

Documentation (4) +123 / -0

IMAGE-SUPPORT.zh-CN.mdDocument image-support fork behavior +73/-0

Document image-support fork behavior

• Provides Chinese installation, usage, privacy, limits, rebuilding, PDF side-panel troubleshooting, and validation guidance for the fork.

IMAGE-SUPPORT.zh-CN.md

README.mdDescribe image and PDF side-panel support +25/-0

Describe image and PDF side-panel support

• Documents supported image workflows, API restrictions, limits, build instructions, and the native side-panel experience in English.

README.md

README_ZH.mdDocument image support in Chinese +25/-0

Document image support in Chinese

• Adds the localized feature overview, compatibility constraints, limits, build steps, and side-panel screenshot.

README_ZH.md

sidebar-image-support.pngAdd native side-panel image screenshot +0/-0

Add native side-panel image screenshot

• Adds visual documentation showing image attachment controls in the browser side panel.

screenshots/sidebar-image-support.png

Other (5) +53 / -4

.eslintrc.jsonExclude browser test artifacts from linting +1/-1

Exclude browser test artifacts from linting

• Adds test-results to ESLint ignore patterns so generated profiles and screenshots are not analyzed.

.eslintrc.json

.gitignoreIgnore generated test results +1/-0

Ignore generated test results

• Prevents native browser profiles, screenshots, and other smoke-test outputs from entering version control.

.gitignore

.prettierignoreExclude test artifacts from formatting +2/-1

Exclude test artifacts from formatting

• Ignores test-results and normalizes the manifest ignore file ending.

.prettierignore

package-image-source.mjsPackage the image-support source distribution +46/-0

Package the image-support source distribution

• Creates a compressed source archive while excluding dependencies, Git metadata, build outputs, coverage, caches, and browser test profiles.

scripts/package-image-source.mjs

manifest.jsonRoute Chromium to the revised side panel +3/-2

Route Chromium to the revised side panel

• Adds a fork-specific display version and points the native side panel at the revision-tagged shared panel URL.

src/manifest.json

Description

The new Browser.contextMenus.removeAll mock accepts a callback and returns undefined, while
refreshMenu() now calls .then() on its return value. refreshMenu() therefore throws before
ready is assigned, so this added unit test cannot exercise the intended waking-click behavior.

Code

tests/unit/background/menus.test.mjs[R22-24]

+    removeAll: (callback) => {
+      removal.then(() => callback())
+    },

Evidence

The mock's removeAll function has no return statement, while the changed production implementation
immediately evaluates Browser.contextMenus.removeAll().then(...). Calling the tested function
against this mock consequently dereferences .then on undefined.

tests/unit/background/menus.test.mjs[13-24]
src/background/menus.mjs[110-118]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The context-menu mock models callback completion but does not return a Promise, whereas the production function now expects `removeAll()` to return a thenable. The test throws synchronously before it reaches its assertions.

## Fix Focus Areas
- tests/unit/background/menus.test.mjs[13-24]
- src/background/menus.mjs[116-155]

## Recommended Fix
Make the mock return the `removal` Promise and resolve it from the test, for example by implementing `removeAll: () => removal`. Do not require a callback in the mock because `refreshMenu()` uses the promise-returning API shape.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Description

InputBox.handleKeyDownOrClick clears the draft whenever onSubmit() resolves, but
ConversationCard catches a failed postMessage() and returns normally. Any request failure after
attachments were read therefore removes the user's question and selected images instead of leaving
them available to resend.

Code

src/components/InputBox/index.jsx[R223-226]

+      .then(() => {
+        if (!mountedRef.current || draftRevisionRef.current !== submittedRevision) return
+        clearDraft()
+      })

Evidence

The new InputBox success path clears its local draft after any fulfilled onSubmit promise, while
the changed ConversationCard submit callback absorbs postMessage errors without rethrowing. Thus a
transport failure fulfills the promise observed by InputBox and triggers the new image-draft
clearing behavior.

src/components/InputBox/index.jsx[218-235]
src/components/ConversationCard/index.jsx[716-745]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
A failed background request resolves `onSubmit` because its caller catches the error, so `InputBox` mistakes the failed request for a successful submission and clears the text and image attachments.

## Fix Focus Areas
- src/components/InputBox/index.jsx[218-235]
- src/components/ConversationCard/index.jsx[716-745]

## Recommended Fix
Make the submit contract distinguish a successfully accepted request from a failed one. Propagate the `postMessage` failure from `ConversationCard` after updating its error UI (or return an explicit unsuccessful result and have `InputBox` reject it), and call `clearDraft()` only after a successful result. Keep the existing error display while retaining the text and attachments for retry.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Description

addDirectory() archives every regular file below the repository root unless its directory is in a
short fixed exclusion set that omits untracked environment, credential, local configuration, IDE,
audit, and existing ZIP files. When a maintainer runs the documented packaging command from a
developer working tree containing such files, they are silently included in the distributable source
archive.

Code

scripts/package-image-source.mjs[R23-26]

+const excluded = new Set([
+  '.git',
+  'node_modules',
+  'build',

Evidence

The new script starts recursion at the repository root, excludes only seven named roots, and
archives every other regular file, while the accompanying documentation directs maintainers to
create and distribute that complete source archive. The repository’s .gitignore identifies omitted
cases such as .idea, .vscode, and ZIP files, and the existing Firefox packager separately
excludes known local and audit artifacts including .idea, .tmp-audit, and ZIP files,
demonstrating that the new packager does not account for files already recognized as non-source
artifacts.

scripts/package-image-source.mjs[21-43]
IMAGE-SUPPORT.zh-CN.md[57-59]
.gitignore[1-9]
scripts/create-firefox-sources-zip.mjs[36-50]
IMAGE-SUPPORT.zh-CN.md[47-59]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The source-packaging script recursively archives nearly every regular file in the maintainer's working tree except those covered by a small directory denylist. Ignored or untracked environment, credential, local configuration, IDE, audit, archive, and other developer-local files can therefore enter the distributable source ZIP.

## Fix Focus Areas
- scripts/package-image-source.mjs[21-43]
- IMAGE-SUPPORT.zh-CN.md[57-59]
- .gitignore[1-9]
- scripts/create-firefox-sources-zip.mjs[36-50]

## Recommended Fix
Build the archive from a tracked-file or explicit source-file allowlist containing only the project source, configuration, and documentation required for a rebuild. Alternatively, reuse the existing Firefox source-path exclusion logic and additionally reject dot-environment files, credentials, key and certificate extensions, local configuration, IDE metadata, audit artifacts, generated browser profiles, and existing archives. Add a test that creates representative ignored and sensitive files and verifies that none appear in the resulting ZIP, and verify archive contents before publishing.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Description

App renders Sidebar · images-2 directly instead of passing a localization key to t. The
visible footer takes this branch whenever surface=sidebar, across every configured language.

Code

src/pages/IndependentPanel/App.jsx[R160-162]

+              {new URLSearchParams(window.location.search).get('surface') === 'sidebar'
+                ? 'Sidebar · images-2'
+                : 'images-2'}

Evidence

Rule 2262059 prohibits new user-facing strings that bypass localization. The new footer condition
renders a literal English Sidebar label, and no matching localization key exists in the locale
resources.

Rule 2262059: Add new English localization keys before other locales
src/pages/IndependentPanel/App.jsx[156-163]
src/_locales/en/main.json[29-44]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The newly visible side-panel revision badge bypasses the localization mechanism and remains English in every locale.

## Fix Focus Areas
- src/pages/IndependentPanel/App.jsx[160-162]
- src/_locales/en/main.json[29-44]

## Recommended Fix
Add an English localization key for the side-panel badge, add it to every supported locale, and render it through `t` while keeping the revision identifier unchanged.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Description

Attach images and the other new image keys are added only to English and the two Chinese locale
files, while supported locales such as German omit them. When one of the ten omitted locales is
active, fallbackLng: 'en' resolves the controls and errors to English throughout the attachment
flow.

Code

src/_locales/en/main.json[R29-32]

+  "Images require an OpenAI-compatible vision API.": "Images require an OpenAI-compatible vision API.",
+  "Choose a model that supports image input.": "Choose a model that supports image input.",
+  "Attach images": "Attach images",
+  "Drop images here or paste a screenshot": "Drop images here or paste a screenshot",

Evidence

Rule 2262059 requires each additional supported locale to contain every new key. The resource
registry configures thirteen languages, but the new keys appear only in English, Simplified Chinese
and Traditional Chinese; German demonstrates the omission, and the i18n setup explicitly falls back
to English.

Rule 2262059: Add new English localization keys before other locales
src/_locales/en/main.json[29-44]
src/_locales/de/main.json[27-31]
src/_locales/resources.mjs[15-54]
src/_locales/i18n-react.mjs[5-8]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The new image-interface localization keys are absent from ten supported locales, causing those languages to fall back to English.

## Fix Focus Areas
- src/_locales/en/main.json[29-44]
- src/_locales/de/main.json[27-31]
- src/_locales/resources.mjs[15-54]

## Recommended Fix
Add all sixteen image keys to every configured locale file using translations or the repository's accepted placeholder convention, retaining English as the source locale.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

💡 Did you know, you can ask Qodo to dismiss a finding you disagree with, with your reason on record

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗