Why buy expensive motors just to read joint angles?

Cost-effective robotic teleoperation using AprilTags with a 3D printed controller to control simulated and real SO-100 robots. Replaces expensive leader-follower setups, halving the cost for imitation learning.

• SO-100 robot follower arm
• 3D printed handheld controller (just a hollow cube with a grip)
• Webcam

Hardware: 3D printed controller with AprilTags, webcam, SO-100 robot
Software: AprilTag detection, MuJoCo simulation

Prerequisites: Python 3.10+, USB webcam

1. Install dependencies:

   uv sync  # or pip install mujoco opencv-contrib-python scipy lerobot[feetech]

2. 3D print controller: Print cube from /assets/, attach 46mm AprilTags (DICT_APRILTAG_16h5, IDs 0-5)

3. Calibrate camera:

   python calibrate.py

4. Setup mjpython(macOS)( :

   bash mjpy-init.sh

Main application:

mjpython main.py

Individual components:

uv run april.py          # AprilTag detection only
uv run mujoco_loop.py     # MuJoCo simulation only

Controls:

• Hand movement: Move controller to control robot end-effector
• u/j keys: Open/close gripper (while focusing cursor in terminal)
• ESC/Q: Exit

AprilTag Detection: Multi-face detection, pose fusion, temporal smoothing, coordinate transformation

Robot Control: Inverse kinematics, Jacobian-based control, gravity compensation, joint limits

Architecture: Two parallel threads communicate via queue - AprilTag detection (april.py) and MuJoCo simulation (mujoco_loop.py) for real-time performance

• main.py - Main application
• april.py - AprilTag detection
• mujoco_loop.py - MuJoCo simulation
• calibrate.py - Camera calibration
• scene.xml - MuJoCo scene
• so_arm100.xml - Robot model
• assets/ - 3D printable STL files

AprilTags: DICT_APRILTAG_16h5 family, 46mm size, IDs 0-5

• Robotics education
• Research data collection
• Behavior prototyping
• Remote teleoperation

We are happy to share that this project was one of the winners of the HuggingFace LeRobot Hackathon 2025, where we placed #24 from over 250+ submissions worldwide (Top 10%), and won a LeKiwi as a result. We thank HuggingFace and Seeedstudio for this award.

Built with: Python, OpenCV, MuJoCo, NumPy, SciPy