Human Following and Payload Delivery Robot for Unstructured and Rough Environments

Implementation of a 4 Wheel Differential Drive Human Following robot capable of carrying a load with the ability to autonomously return to starting point with terrain-aware navigation in both GPS enabled and GPS Denied environments

Key Features:

• ROS2 based software stack.
• Human Following using Apriltags (tag36h11 with id=0 for following, use id=1 for other purposes)
• Mapless and Map based Autonomous Navigation (2D LiDAR Based).
• GPS waypoint navigation
• LIO SAM based environment mapping
• Autonomous Return using GPS

Features Under Development

• Behaviour Tree based model for managing missions
• Integrating .pcd generated by LIO-SAM with planners for autonomous navigation in GPS Denied zones

Prerequisites:

• ROS2 Humble
• Ubuntu 22.04
• Gazebo Fortress v6.16

Setup

1. Visualize URDF in RViz (without launch file)

   ros2 launch urdf_tutorial display.launch.py model:={urdf file path}
   

2. Launch Gazebo Simulation in Custom World

   ros2 launch bot_description gazebo.launch.py world_name:={world name}
   

3. Launch robot controller

   ros2 launch bot_controller controller.launch.py
   

4. Launch Gazebo Simulation and Load controller

   ros2 launch bot_bringup bot_bringup.launch.py
   

5. Launch Keyboard Teleop with ros2_control

   ros2 run teleop_twist_keyboard teleop_twist_keyboard --ros-args -r /cmd_vel:=/bot_controller/cmd_vel_unstamped

6. Run Apriltag detection and Human Following

   ros2 launch apriltag_navigation apriltag_navigation.launch.py

7. SLAM with slam_toolbox

   ros2 launch slam_toolbox online_async_launch.py slam_params_file:={path to yaml file} use_sim_time:=true

8. Switch from Mapping to Localization - make change in the params file

   ros2 launch slam_toolbox online_async_launch.py slam_params_file:={path to yaml file} use_sim_time:=true

9. Control the Robot with Twist Mux

   ros2 run twist_mux twist_mux --ros-args --params-file {path to yaml file} -r cmd_vel_out:=bot_controller/cmd_vel_unstamped
   
   ros2 run teleop_twist_keyboard teleop_twist_keyboard --ros-args -r /cmd_vel:=/cmd_vel_joy
   

10. Navigation Using Nav2

    ros2 launch bot_description navigation_launch.py use_sim_time:=true

11. For Mapless Navigation - Launch Twist Mux node first and then proceed with following scripts

    ros2 launch slam_toolbox online_async_launch.py use_sim_time:=true
    
    ros2 launch bot_description navigation_launch.py use_sim_time:=true
    

12. start container and mount the custom params file

docker run --rm -it \
  --name liosam-humble-jammy-container \
  --net=host \
  --ipc=host \
  --shm-size=2g \
  --privileged \
  -e DISPLAY=$DISPLAY \
  -e QT_X11_NO_MITSHM=1 \
  -e ROS_LOCALHOST_ONLY=0 \
  -v /tmp/.X11-unix:/tmp/.X11-unix:rw \
  -v /home/shaurya/followbot_ws/src/followbot_ros/bot_description/config/lio_sam_baylands_params.yaml:/root/lio_sam_baylands_params.yaml:ro \
  liosam-humble-jammy:latest \
  bash

12. Launch LIO SAM

ros2 launch lio_sam run.launch.py params_file:=/root/lio_sam_baylands_params.yaml

14. run human detector

ros2 launch human_detector human_detector.launch.py   camera_frame_id:=rgbd_camera   horizontal_fov:=1.10   use_sim:=True   pose_landmarker_model_path:=/home/shaurya/models/pose_landmarker_full.task

14. run detector

source ~/venvs/followbot/bin/activate
export PYTHONNOUSERSITE=1

python3 -m pip install --upgrade pip
python3 -m pip install "numpy<2" mediapipe opencv-python

mkdir -p ~/models
wget -O ~/models/pose_landmarker_full.task \
  https://storage.googleapis.com/mediapipe-models/pose_landmarker/pose_landmarker_full/float16/latest/pose_landmarker_full.task

ros2 launch human_detector human_detector.launch.py     use_sim:=False     camera_frame_id:=rgbd_camera     horizontal_fov:=1.10     pose_landmarker_model_path:=/home/shaurya/models/pose_landmarker_full.task     rgb_image_topic:=/camera/image     depth_image_topic:=/camera/depth_image     camera_info_topic:=/camera/camera_info

16. naive follower

ros2 run bot_terrain_follower naive_follower --ros-args --params-file /home/shaurya/followbot_ws/src/followbot_ros/bot_terrain_follower/config/follower.yaml

Demo

Contact

Shaurya Jain - Reach me at jainshaurya.sj@gmail.com

Citation