We have been working on a robot-arm setup in the office: two SO-101 arms placed on a white table, a set of red and green cubes, and a bucket used as part of the task environment. The setup also includes three USB cameras: one positioned above the table and one mounted near each gripper.
At first glance, this sounds straightforward. The top-down camera sees the entire table, both robot arms, the cubes, and the bucket. The gripper cameras, however, are much harder to distinguish. They are the same model, mounted in similar positions, and connected to the same machine. From the operating system's point of view, they are just video devices with names like /dev/video0, /dev/video2, and /dev/video4.
That is where a very practical robotics problem begins.
In many physical setups, identifying which camera is which is still a manual process. You unplug one camera, see which feed disappears, label it, and repeat until the mapping is clear. It is not difficult work, but it is slow, repetitive, and easy to get wrong when devices are identical.
For this setup, we asked an AI coding agent to run the data collection process. The instruction was simple: use the top camera and the left gripper camera. We did not specify which video device corresponded to each physical camera.The agent began by inspecting the available video devices. It recognized that the odd-numbered entries were metadata nodes and identified three actual camera feeds. It then captured reference frames from each feed and copied the images over SSH for closer inspection.Using the contents of the images, the agent determined how the video devices mapped to the physical cameras.
The top-down view was easy to identify. /dev/video0 showed the full table: both robot arms, the red and green cubes on the left side, and the bucket toward the top-right. That made it the overhead camera.
The two gripper cameras were the real challenge. Both showed close-up views of an orange gripper above the same white table. Their metadata did not reveal which one was mounted on the left arm and which one was mounted on the right arm.
So the agent used a visual landmark. In the top-down camera view, the red and green cubes were located on the left side of the table. One of the gripper cameras could see those same cubes in the corner of its frame. The other camera was looking toward an emptier part of the table, closer to the bucket.
From that evidence, the agent concluded that /dev/video4 was the left gripper camera, because it was the gripper view that could see the cubes. /dev/video2 was the right gripper camera and could be ignored for that data collection task.
It did not stop at the conclusion. Before continuing, the agent reported its reasoning and asked for confirmation: was the left gripper the one looking at the cubes? Once confirmed, it proceeded with the correct camera.
What stands out is not only that the agent identified the camera correctly. It is how it got there.
We did not explicitly tell it to take reference photos. We did not explain that the cubes could be used as landmarks. We did not provide a camera map or describe the spatial layout in detail. The agent built a small investigation for itself: collect evidence, inspect the visual context, compare camera views, infer the likely mapping, and verify before acting.
This is a small example, but it is a meaningful one. Real-world engineering work is full of problems that are not neatly described in an API or configuration file. Hardware behaves differently under load. Device names are ambiguous. Identical components need to be mapped back to physical reality. A useful agent needs more than the ability to run commands; it needs to adapt when the environment does not match a clean assumption.
In this case, the agent treated the cameras as sources of evidence rather than opaque device paths. It connected what the operating system exposed with what the physical world actually looked like.
That is the part worth remembering. The robot arms may be the most visible part of the demo, but the more interesting moment was quieter: an AI agent using images of a table to figure out left from right.
Source Notes
The frames were captured by the agent on June 6, 2026.
The robot arms are SO-101 arms.
