Description
(I originally posted this with fastmcp, but the mentioned it's really about the mcp library.)
I'm trying to use ctx.sample to convert an image into markdown.
But it seems difficult to send multiple messages with ctx.sample. I get a random McpError() which I assume is because the Anthropic-style API usually doesn't allow to consecutive user messages.
Normally with anthropic I would send two contents in a single message, but this is not allowed by the Pydantic type SamplingMessage.
See code examples below
v0 and v1 fail with McpError() (no description) mcp/shared/session.py:286 in send_request.
v2 and v3 fails because SamplingMessage doesn't allow lists on the content field.
Example Code
async def page_to_markdown_v0(
ctx: Context, raw_text: str, page_num: int, image: MCPImage
) -> mcp.types.TextContent:
return await ctx.sample( # type: ignore[call-arg]
system_prompt=textwrap.dedent(f"""
Convert the content of this PDF page (page {page_num}) into clean, well-structured Markdown format.
Here is the raw text extracted from the page:
-----------
{raw_text}
-----------
Guidelines:
- Use the raw text as reference but also analyze the image for better formatting
- Preserve the document structure (headings, lists, tables, etc.)
- Use appropriate Markdown syntax for formatting
- Maintain the logical flow and hierarchy of the content
- Include any important visual elements as descriptions
- For tables, use Markdown table syntax
- For mathematical expressions, use LaTeX syntax if needed
Return ONLY the Markdown content, no explanations or meta-text.
"""),
messages=[
SamplingMessage(
role="user",
content=image.to_image_content(),
),
],
max_tokens=8192,
)
async def page_to_markdown_v1(
ctx, raw_text: str, page_num: int, image: MCPImage
) -> mcp.types.TextContent:
return await ctx.sample( # type: ignore[call-arg]
messages=[
SamplingMessage(
role="user",
content=mcp.types.TextContent(
type="text",
text=textwrap.dedent(f"""
Convert the content of this PDF page (page {page_num}) into clean, well-structured Markdown format.
Here is the raw text extracted from the page:
-----------
{raw_text}
-----------
Guidelines:
- Use the raw text as reference but also analyze the image for better formatting
- Preserve the document structure (headings, lists, tables, etc.)
- Use appropriate Markdown syntax for formatting
- Maintain the logical flow and hierarchy of the content
- Include any important visual elements as descriptions
- For tables, use Markdown table syntax
- For mathematical expressions, use LaTeX syntax if needed
Return ONLY the Markdown content, no explanations or meta-text.
"""),
),
),
SamplingMessage(
role="user",
content=image.to_image_content(),
),
],
max_tokens=8192,
)
async def page_to_markdown_v2(
ctx, raw_text: str, page_num: int, image: MCPImage
) -> mcp.types.TextContent:
return await ctx.sample( # type: ignore[call-arg]
messages=[
SamplingMessage(
role="user",
content=[
mcp.types.TextContent(
type="text",
text=textwrap.dedent(f"""
Convert the content of this PDF page (page {page_num}) into clean, well-structured Markdown format.
Here is the raw text extracted from the page:
-----------
{raw_text}
-----------
Guidelines:
- Use the raw text as reference but also analyze the image for better formatting
- Preserve the document structure (headings, lists, tables, etc.)
- Use appropriate Markdown syntax for formatting
- Maintain the logical flow and hierarchy of the content
- Include any important visual elements as descriptions
- For tables, use Markdown table syntax
- For mathematical expressions, use LaTeX syntax if needed
Return ONLY the Markdown content, no explanations or meta-text.
"""),
),
image.to_image_content(),
],
),
],
max_tokens=8192,
)
async def page_to_markdown_v3(
ctx, raw_text: str, page_num: int, image: MCPImage
) -> mcp.types.TextContent:
return await ctx.sample( # type: ignore[call-arg]
messages=[
{
"role": "user",
"content": [
{
"type": "text",
"text": textwrap.dedent(f"""
Convert the content of this PDF page (page {page_num}) into clean, well-structured Markdown format.
Here is the raw text extracted from the page:
-----------
{raw_text}
-----------
Guidelines:
- Use the raw text as reference but also analyze the image for better formatting
- Preserve the document structure (headings, lists, tables, etc.)
- Use appropriate Markdown syntax for formatting
- Maintain the logical flow and hierarchy of the content
- Include any important visual elements as descriptions
- For tables, use Markdown table syntax
- For mathematical expressions, use LaTeX syntax if needed
Return ONLY the Markdown content, no explanations or meta-text.
"""),
},
image.to_image_content().model_dump(),
],
}
],
max_tokens=8192,
)Version Information
FastMCP version: 2.10.6
MCP version: 1.12.2
Python version: 3.12.3
Platform: macOS-15.3.1-arm64-arm-64bit
FastMCP root path: /Users/ahle/repos/verification/nectar2/.venv/lib/python3.12/site-packages
Additional Context
No response
Description
(I originally posted this with fastmcp, but the mentioned it's really about the
mcplibrary.)I'm trying to use
ctx.sampleto convert an image into markdown.But it seems difficult to send multiple messages with ctx.sample. I get a random McpError() which I assume is because the Anthropic-style API usually doesn't allow to consecutive user messages.
Normally with anthropic I would send two contents in a single message, but this is not allowed by the Pydantic type SamplingMessage.
See code examples below
v0 and v1 fail with
McpError()(no description)mcp/shared/session.py:286 in send_request.v2 and v3 fails because
SamplingMessagedoesn't allow lists on thecontentfield.Example Code
Version Information
Additional Context
No response