fix: create jaeger ExternalName services after app deploy in conductor - #663
fix: create jaeger ExternalName services after app deploy in conductor#663somyas25 wants to merge 3 commits into
Conversation
HacksonClark
left a comment
There was a problem hiding this comment.
Jaeger is not working in train-ticket:
2026-03-24 00:02:44,705 - mcp.server.lowlevel.server - INFO - Processing request of type CallToolRequest │
│ 2026-03-24 00:02:44,710 - all.mcp.utils - ERROR - Request failed: 400 Client Error: Bad Request for url: http://jaeger-out.observe.svc.c │
│ luster.local:16686/api/traces?service=&start=1774310564706000&end=1774310564706034&limit=20 │
│ 2026-03-24 00:02:44,710 - all.mcp.jaeger_server - ERROR - [ob_mcp] Error querying get_traces: 400 Client Error: Bad Request for url: htt │
│ p://jaeger-out.observe.svc.cluster.local:16686/api/traces?service=&start=1774310564706000&end=1774310564706034&limit=20 │
│
Make sure to test Jaeger with MCP inspector to make sure the tool works!
I am trying to recreate this locally and when running the problem 17 and using the MCP inspector to get services for jaeger I only get 'jaeger-all-in-one' service. When trying to get the trace for this service, I get a success response but an empty output. Could you give me more insight on how you got this error so I can analyze it? Thanks! |
|
Hmm, that means that there are not traces being collected! @Saadmrp1038 Didn't we delete the jaeger images from trainticket since they were causing some bugs? I think that may be the source of this issue. |
|
@HacksonClark yes, the current train-ticket images are not jaeger instrumented. The best way is to instrument the services ourselves as the previous jaeger-based images had some issues. |
|
Gotcha! I'll have to re-review this PR then. |
|
To update: I briefly chatted with @somyas25 in person. First @somyas25: Thanks for keeping pushing! This is not an easy first issue at all. I explained what does tracing mean to @somyas25, and now she will try to instrument the services so that Jaeger can trace them. I will thus convert this PR to a draft. |
|
@yimingsu01 sounds great! This is a great task to learn about distributed tracing, a very in-demand skill. Great work so far @somyas25! Keep it up! |
|
Hi, I’ve been looking into this issue and trying to understand the train-ticket deployment flow a little better. From what I understand right now, the conductor triggers the What I’m noticing is that Based on what @Saadmrp1038 said I am assuming that there are some bugs in the images in
Just wanted to check that I’m understanding this correctly and that I am on the right track! |
|
@somyas25 Your observations are correct. For your question, I like the I was thinking we should add Jaeger tracing instrumentation to our existing app code and build our own Jaeger-enabled images. That way we have full control over the tracing setup and can update services whenever we need to. In practice that would look like:
Feel free to ask about anything! |
Understood, thanks. I will start implementing this. |
Resolution to the hotfix (#541) for train-ticket. This PR resolves creation of Jaeger ExternalName services before deploying train-ticket.