soarm-ws: A Full-Stack Workspace for SO-ARM100 Manipulator Research

imu_sdk and the vendored SO-ARM100 robot description feed into m5teleop, the integration point that runs the real-time teleoperation loop (IMU → EKF → orientation control → IK → servo commands, all logged through Rerun). Recording is opt-in: pass --record and m5teleop hands frames to soarm_lerobot, which buffers episodes into a LeRobotDataset for ACT / Diffusion Policy training — with no import-level dependency back the other way. soarm_sdk and camera_calibration are standalone: the former is a parallel, self-contained Feetech servo SDK (used by its own calibration/dashboard tools, not currently wired into the teleop loop), the latter a Rerun-based camera intrinsics/ArUco toolkit with no shared dependencies at all. soarm_mjlab is the newest branch — RL training of SO-ARM100 in MuJoCo via mjlab, still in progress, planned to deploy trained policies through the same soarm_sdk.RobotInterface that drives real hardware.

soarm_sdk stable

Python SDK for the Feetech STS/SCS serial bus servo protocol — port handling, packet framing, batch calibration planning (ID reassignment, limits, torque, baud rate), and the tick↔radian conversion layer everything else builds on. Ships a 7-tab Viser browser dashboard with live 3-D FK for homing, PID tuning, and monitoring.

imu_sdk stable

Transport + firmware for M5StickC/ESP32 IMU boards (MPU6050/6500/6886/9250, 6- or 9-axis). Firmware streams JSON over serial at ~100 Hz with on-device gyro bias calibration; orientation itself is computed host-side by m5teleop's error-state Kalman filter.

m5teleop active

The integration point: a 50 Hz real-time loop wiring IMU → ESKF → cascade orientation controller → IK (pink/pinocchio) → the arm over lerobot's SO100Follower, with a parallel Viser sim and full Rerun logging. --record optionally streams frames into soarm_lerobot.

soarm_lerobot active

Dataset recording and imitation-learning training. TeleopRecorder detects episode boundaries and writes LeRobotDataset episodes; dataset.py loads them into normalized, chunked torch datasets for ACT and Diffusion Policy. Operates purely on recorded data — decoupled from how it was captured.

camera_calibration stable

Standalone camera intrinsics and ArUco tooling: chessboard calibration, live detection monitor, marker generation, webcam parameter estimation — a unified CLI, Rerun for all visualization (no cv2.imshow, works headless). Not wired into the teleop pipeline; built for future eye-in-hand and workspace-calibration work.

soarm_mjlab in progress

RL training of SO-ARM100 in MuJoCo via mjlab, targeting deployment through soarm_sdk.RobotInterface — the same interface real hardware uses, so a trained policy won't care whether it's driving sim or the physical arm. Phase 0 (repo scaffolding, uv-based install tooling, CI) is done; Phase 1 (a sample "Reach" task carried end-to-end) is underway — see the roadmap.

SO-ARM100 vendored, read-only

The hardware vendor's repo (TheRobotStudio), tracked as-is: URDF/MJCF robot descriptions consumed by soarm_sdk's dashboard and m5teleop's IK solver, plus CAD, 3D-print notes, and BOM/assembly docs for the physical arm.

Mechanical fabrication

• Printing cost: roughly €50 for a follower arm, €105 for a leader–follower pair within the EU when outsourcing prints.
• Materials: PLA+ by default; PETG or nylon for higher temperature tolerance, >35% infill on load-bearing links.
• Follow the official STL pack and BOM in the SO-ARM100 repo; check the LeRobot Discord for community upgrades (metal joint inserts, cable harnesses).

Actuation and electronics

Component	Notes
FEETECH STS3215 servos	Follower arm: six identical servos (7.4 V/20 Nm or 12 V/30 Nm). Leader arm gears: 3× 1/147 (C046), 2× 1/191 (C044), 1× 1/345 (C001).
Servo bus adapter	Waveshare Serial Bus Servo Driver Board with USB-UART bridge.
Debug tools	Official FEETECH Windows software, FT_SCServo_Debug_Qt, and soarm_sdk's own Viser dashboard.
Power	7.4–12 V DC, sized for peak current (~6 A per arm). Match barrel jack polarity (5.5/2.1 mm or 5.5/2.5 mm).

Sensing and peripherals

• Cameras: UVC webcams, Intel RealSense D405/D435, or any RGB-D sensor supported via LeRobot camera adapters; calibrate with this workspace's own camera_calibration package.
• IMU: M5StickC/ESP32 boards via imu_sdk, mounted on the wrist for inertial feedback in the teleop loop.

Compute, networking, and safety

• Control workstation: Ubuntu 22.04 LTS or macOS ≥12, 8+ CPU cores, 16 GB RAM; an NVIDIA GPU if training diffusion/RL policies locally (required for soarm_mjlab's CUDA path).
• Safety: hardware e-stop, safety relay, over-voltage protection; power servos down before swapping end effectors.

Calibration and troubleshooting

• Homing vs. kinematics: LeRobot's calibration routine sets a consistent zero; for accuracy-critical geometric calibration from motion capture or camera data, use the FIGAROH toolbox.
• Serial bus baud rate mismatches — confirm both controller and servos agree (some adapters top out below 1 Mbps).
• Mixed firmware revisions — update all servos to the same version before chaining them.
• Power polarity mistakes — double-check barrel jack wiring before powering up.

Two learning tracks are actively built in this workspace, both consuming the vendored SO-ARM100/Simulation assets (URDF for FK, MJCF for physics):

• Imitation learning (soarm_lerobot): record teleop demonstrations through m5teleop --record, then train ACT or Diffusion Policy models on the resulting LeRobotDataset.
• Reinforcement learning (soarm_mjlab, in progress): train in MuJoCo via mjlab, following the manager-based config pattern from unitree_rl_mjlab — no C++ deployment stack or terrain/locomotion machinery, since a 6-DOF arm's control loop already runs comfortably in Python at ~50 Hz. Deploys through soarm_sdk.RobotInterface so trained policies drive sim and real hardware identically.

Beyond this workspace's own tracks, the broader SO-ARM ecosystem has useful options worth knowing about if you want to go further in a given direction: MoveIt/OMPL for classical motion planning, ros2_so_arm100 for a ROS2 description/control bring-up, IsaacSim/IsaacLab for GPU-accelerated, photorealistic RL at larger scale.

The most active front right now is soarm_mjlab. See the full roadmap document for details; current status:

• done Phase 0 — repo scaffolding: submodule setup, uv-based install tooling (mjlab gates torch behind CPU/CUDA extras), CI (lint + test on the empty-but-structured package).
• in progress Phase 1 — a sample "Reach" task carried end-to-end: train → validate → sim-deploy → real hardware, with scaffolding built to hold a second and third task without rework.

This section will be updated as phases land — treat it as a live status snapshot, not a fixed plan.

Clone the workspace with all submodules:

git clone --recurse-submodules https://github.com/thanhndv212/soarm-ws.git
cd soarm-ws

Each package installs independently — pip install -e . is the default:

cd soarm_sdk && pip install -e .
cd ../imu_sdk && pip install -e .
cd ../m5teleop && pip install -e .
cd ../soarm_lerobot && pip install -e .
cd ../camera_calibration && pip install -e .

soarm_mjlab is the one exception — it uses uv because mjlab gates torch behind mutually-exclusive CPU/CUDA extras routed to different package indices, which plain pip can't express:

cd soarm_mjlab
make sync-cpu   # or: uv sync --extra cpu --group dev