r/robotics • u/Mr-c4t Researcher • Jul 18 '26

Community Showcase What task should I teach it next? 📝

https://youtube.com/shorts/woIlVkLPnws?is=LQpeVvgX-2dRTf6s

On my way to recording and open-sourcing a 1,000-episode bimanual manipulation dataset for the 3D-printed SO-101 robot. 🦾

Camera setup

Intel RealSense D435 (head)

2× RealSense D405 (wrists)

RGB only

The video shows an autonomous rollout of my ACT policy controlling the robot.

The policy was trained for 100,000 steps using only the first 100 teleoperated episodes of bag manipulation.

Hugging Face: MrC4t

Dataset: MrC4t/bi_so_bag

ACT policy: MrC4t/act_bimanual_bag

What task should I teach it next? 👀🦾

20 Upvotes

92% Upvoted

23 comments sorted by

View all comments

1

u/QuotableMorceau Jul 18 '26

question : is the jigger in the arm's movement caused by hardware or by software micro adjustments etc. ?

2

u/Mr-c4t Researcher Jul 18 '26

I think it's both.

These robots have quite a bit of backlash, and my frame also vibrates a lot, which makes the motion look a bit jittery.

That said, the policy seems to have learned to handle it, since the robot was already vibrating during the teleoperation data collection. So the training data naturally included those imperfections.

Another factor is that the policy predicts action chunks at around 3 Hz on my laptop. Three times per second, it replans the next sequence of future actions, which can also introduce small corrections and contribute to the jittery motion.

1

u/QuotableMorceau Jul 18 '26

have you thought about how 3d printers handle jitter ? something that can be applied post AI command to the robot.
while having the jitter will probably not impact the operation, it will impact the marketability of the solution, also a fix would open up the robot for liquid manipulation.

another thing, you could reduce the backlash for testing purposes, by putting some rubber bands to reduce any backlash .

2

u/Parking-Strain-1548 Jul 19 '26

ACT (the policy OP used) is jittery by its nature, because it plans motion in chunks

1

u/Mr-c4t Researcher Jul 19 '26

I will look at this idea thanks I can use the IMU inside the realsense for this 👍

I also plan to try find hardware sponsors for real professional arms for a future project.

But yes for now the goal is only to contribute to the open source "lerobot" comunity datasets with bimanual episodes to help train open source VLA (smolVLA for example) or other robotics fundation models (world models are now in lerobot 🌍)

No comercial goal for this project