agents/docker: Refactor Docker agent. by nrybowski · Pull Request #1119 · INGInious/INGInious

nrybowski · 2026-07-09T13:07:28Z

This PR is a major refactor of the Docker agent which focuses on two main points: (i) deprecating the kata-runtime as it is unused and adds a lot of unnecessary complexity and, (ii) implementing a robust communication scheme between the agent, the grading containers and the student containers.

Regarding to the first point, we experimented a few years ago with launching custom kernels within student container (#949). This is a solid alternative to the kata-runtime for running student code as root, e.g., for networking exercises, without compromising INGInious security. This feature has been blocked since then because of the way the Docker agent exposes its capabilities, which in turns impacts the way jobs are distributed among available agents.

Regarding to the second point, the Docker agent communicates with grading and student containers through their std{in,out} interfaces which is quite fragile and blocks a current effort of removing the Docker dependency (see #1115, #1116) in favor of generic OCI compatibility. Also, when the kata-runtime is used, the Docker agent serves as a proxy between the grading and student containers while they can communicate directly when both of them run on the same kernel. This is not required by #949.
The Docker agent also relies on the ZMQ framework for communications. This is an heavy framework whose advanced features are not used. We deprecate it in favor of a simpler and lower level approach based on simple bytestreams over TCP and Unix sockets which is already in use in the existing code. By leveraging the Transport and Protocol features of the asyncio framework, we also can get rid of the custom buffering that has been implemented. The goal here is to mainly rely on features of the standard python library and simplify the code base.
Furthermore, the messages sent between each element are not formally described and are encoded with dictionaries built-on the fly. This approach is not reliable and makes the messages hard to correctly extend.

The kata-runtime is so intermingled with the Docker agent I/O that it is nearly impossible to deprecate it without refactoring the whole Docker agent I/O.
This PR is an effort in the continuity of #1115 and #1116, and should be a cleaner basis for #949.
It implements on the following changes:

Note: This PR is WIP, the git history will be squashed and rewritten before opening it for merge request.

- Deprecate communication through std{in,out} channels in favor of simple TCP connection. - Leverage formalised messages rather than dictionnaries built on the fly.

codacy-production · 2026-07-09T13:10:00Z

Not up to standards ⛔

🔴 Issues 1 critical · 20 high · 6 medium · 1 minor

Alerts:
⚠ 28 issues (≤ 0 issues of at least minor severity)

Results:
28 new issues

Category Results

BestPractice 1 minor

ErrorProne 1 critical
1 high

Security 2 medium
19 high

Complexity 4 medium

View in Codacy

🟢 Metrics 148 complexity · 0 duplication

Metric Results

Complexity 148

Duplication 0

View in Codacy

_{NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer}
_{TIP This summary will be updated as you push new changes.}

nrybowski · 2026-07-09T13:28:41Z

+        attrs = self._docker.sync._docker.containers.get(info.container_id).attrs
        try:
-            reader_stream, write_stream = await asyncio.open_connection(sock=sock._sock)
+            ip = attrs['NetworkSettings']['Networks']['bridge']['IPAddress']
+        except KeyError:
+            self._logger.exception("Failed to fetch grading container IP from container attributes.")
+            return None
+        try:
+            reader_stream, write_stream = await asyncio.open_connection(ip, GRADING_CONTAINER_TCP_PORT)


Replace with TCP server instead, grading containers will connect to the agent when they are ready.

nrybowski · 2026-07-09T13:30:47Z

+# ==== Common types ====
+
+class MsgType(IntEnum):
+    AgentInitMsg = 1


Rename in AgentInitGrading

nrybowski · 2026-07-09T13:30:55Z

+    @staticmethod
+    def from_dict(o: dict):
+        if (msg_type := o.get('type')) is not None:
+            if msg_type == MsgType.AgentInitMsg:


nrybowski · 2026-07-09T13:31:37Z

+            elif msg_type == MsgType.GradingError:
+                return GradingError(**o)
+            else:
+                raise ValueError


Add explicit error message.

nrybowski · 2026-07-09T13:31:45Z

+            else:
+                raise ValueError
+        else:
+            raise KeyError


nrybowski · 2026-07-09T13:31:51Z

+    def deserialize(o: bytes):
+        o = msgpack.loads(o)
+        if not isinstance(o, dict):
+            raise TypeError


nrybowski · 2026-07-09T13:32:40Z

+    envtypes: dict
+    run_cmd: str
+    run_as_root: bool = False
+    shared_kernel: bool = True


To deprecate.

- Deprecate kata-runtime related code, i.e., shared_kernel and both_docker switches. - Leverage formalized Agent messages when proxied through the grading container.

- Deprecate kata-runtime related code. - Connect to Agent TCP server. - Leverage formalized Agent messages.

…iner. This commit adds a mock of the Docker agent to test the behavior of the grading / students container w.r.t. the new I/O scheme, as well as some simple test cases.

nrybowski added 6 commits July 1, 2026 16:18

agent/docker: Move base containers with agent for future lib sharing.

e96548d

squash

9df831d

agent/docker: Formalise Agent messages in dedicated frozen dataclasses

7197751

agent/docker: Refactor Agent communication scheme.

224b92b

- Deprecate communication through std{in,out} channels in favor of simple TCP connection. - Leverage formalised messages rather than dictionnaries built on the fly.

containers/base: Expose Agent messages to base container.

7756e0d

containers/base/api: Formalise internal grading container messages.

d38b1c8

nrybowski changed the title ~~Refactor Docker agent.~~ agents/docker: Refactor Docker agent. Jul 9, 2026

nrybowski requested a review from anthonygego July 9, 2026 13:16

nrybowski commented Jul 9, 2026

View reviewed changes

Comment thread inginious/agent/docker_agent/messages.py

else:

raise ValueError

else:

raise KeyError

nrybowski Jul 9, 2026

Copy link
Copy Markdown

Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bis.

nrybowski commented Jul 9, 2026

View reviewed changes

nrybowski added 3 commits July 9, 2026 15:38

containers/base/api: Refactor student container I/O loop.

40737db

- Deprecate kata-runtime related code, i.e., shared_kernel and both_docker switches. - Leverage formalized Agent messages when proxied through the grading container.

containers/base/api: Refactor grading container I/O loop.

49db411

- Deprecate kata-runtime related code. - Connect to Agent TCP server. - Leverage formalized Agent messages.

agents/docker: Add integration tests for the grading / students conta…

5c4de63

…iner. This commit adds a mock of the Docker agent to test the behavior of the grading / students container w.r.t. the new I/O scheme, as well as some simple test cases.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

agents/docker: Refactor Docker agent.#1119

agents/docker: Refactor Docker agent.#1119
nrybowski wants to merge 9 commits into
INGInious:mainfrom
nrybowski:refactor_docker_agent

nrybowski commented Jul 9, 2026 •

edited

Loading

Uh oh!

codacy-production Bot commented Jul 9, 2026 •

edited

Loading

Uh oh!

nrybowski Jul 9, 2026

Uh oh!

nrybowski Jul 9, 2026

Uh oh!

nrybowski Jul 9, 2026

Uh oh!

nrybowski Jul 9, 2026

Uh oh!

nrybowski Jul 9, 2026

Uh oh!

nrybowski Jul 9, 2026

Uh oh!

nrybowski Jul 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

nrybowski commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codacy-production Bot commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Not up to standards ⛔

Uh oh!

nrybowski Jul 9, 2026

Choose a reason for hiding this comment

Uh oh!

nrybowski Jul 9, 2026

Choose a reason for hiding this comment

Uh oh!

nrybowski Jul 9, 2026

Choose a reason for hiding this comment

Uh oh!

nrybowski Jul 9, 2026

Choose a reason for hiding this comment

Uh oh!

nrybowski Jul 9, 2026

Choose a reason for hiding this comment

Uh oh!

nrybowski Jul 9, 2026

Choose a reason for hiding this comment

Uh oh!

nrybowski Jul 9, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

nrybowski commented Jul 9, 2026 •

edited

Loading

codacy-production Bot commented Jul 9, 2026 •

edited

Loading