← back to blog

Capstone - Week Three

413 words 2 min

Week Update

After attempting to do Part 4: Accessing Data from Nvidias Getting started with Isaac Sim tutorial, I found that I was having a lot of issues accessing the output data due to Docker. I have opted to install Windows onto my machine in order to run Isaac Sim directly on the computer. ROS2 is the primary extension used in part 4 of the tutorial for the visualization of data. This extension gave me a great deal of difficulty, and I opted to move on instead of wasting more time on it. I found that this extension is deprecated for the current versions of both Ubuntu and Windows. I will try to avoid using it in the future unless it is absolutely necessary, since this extension is deprecated. I do not believe that I will need to use it. If there is a reason why my project requires the usage of this application, I will come back to this in order to find a different solution to this issue.

As I have opted to move away from the first tutorial I was working on, I have started a new tutorial also offered through Nvidia: Train an SO-101 Robot from Sim-to-Real with Nvidia Isaac. I believe that this tutorial will be extremely relevant to the work that I am looking to do on this project. This tutorial is longer than the previous, consisting of 4 modules:

  • Build your Robot Lab
  • Collect Train and Evaluate
  • Augment Datasets
  • Measure the Action Gap

I will be completing module 1 and module 2 up to the real evaluation (as that is not applicable to me at this point in the project). This tutorial includes training of an autonomous robotic arm through teleoperation using GROOT, which is a possible avenue of training for this project. The usage of this arm is trained through a video of both primary vision (a small camera above the claw) and tertiary vision (an external webcam). The task that the sim is trained on is a pick-and-place. The SO-101 arm is an open-source arm with 6 DOF that can be made or purchased. There is an arm as well as a teleoperational arm for training. It is also pointed out in the tutorial overview that it is possible to only have the teleoperation arm (and not the physical arm) in order to train the simulation to do multiple tasks before committing to building or buying the arm itself.

Approx. Working hours

  • 5 hours