NOTE: this software needs to interface with a running instance of the BenchBot software stack. Unless you are running against a remote stack / robot, please install this software with the BenchBot software stack as described here.

BenchBot API

The BenchBot API provides a simple interface for controlling a robot or simulator through actions, and receiving data through observations. As shown above, the entire code required for running an agent in a realistic 3D simulator is only a handful of simple Python commands.

Open AI Gym users will find the breakdown into actions, observations, and steps extremely familiar. BenchBot API allows researchers to develop and test novel algorithms with real robot systems and realistic 3D simulators, without the typical hassles arising when interfacing with complicated multi-component robot systems.

Running a robot through an entire environment, with your own custom agent, is as simple as one line of code with the BenchBot API:

from benchbot_api import BenchBot
from my_agent import MyAgent

BenchBot(agent=MyAgent()).run()

The above assumes you have created your own agent by overloading the abstract Agent class provided with the API. Overloading the abstract class requires implementing 3 basic methods. Below is a basic example to spin on the spot:

from benchbot_api import Agent
import json

class MyAgent(Agent):

    def is_done(self, action_result):
        # Go forever
        return False

    def pick_action(self, observations, action_list):
        # Rotates on the spot indefinitely, 5 degrees at a time
        # (assumes we are running in passive mode)
        return 'move_angle', {'angle': 5}

    def save_result(self, filename, empty_results, results_format_fns):
        # Save some blank results
        with open(filename, 'w') as f:
            json.dump(empty_results, f)

If you prefer to do things manually, a more exhaustive suite of functions are also available as part of the BenchBot API. Instead of using the BenchBot.run() method, a large number of methods are available through the API. Below highlights a handful of the capabilities of BenchBot API:

from benchbot_api import BenchBot, RESULT_LOCATION
import json
import matplotlib.pyplot as plt

# Create a BenchBot instance & reset the simulator / robot to starting state
b = BenchBot()
observations, action_result = b.reset()

# Print details of selected task & environment
print(b.task_details)
print(b.environment_details)

# Visualise the current RGB image from the robot
plt.imshow(observations['image_rgb'])

# Move to the next pose if we have a 'move_next' action available
if 'move_next' in b.actions:
    observations, action_result = b.step('move_next')

# Save some empty results
with open(RESULT_LOCATION, 'w') as f:
    json.dump(b.empty_results(), f)

For sample solutions that use the BenchBot API, see the examples add-ons available (e.g. benchbot-addons/examples_base and benchbot-addons/examples_ssu).

Installing BenchBot API

BenchBot API is a Python package, installable with pip. Run the following in the root directory of where this repository was cloned:

u@pc:~$ pip install .

Using the API to communicate with a robot

Communication with the robot comes through a series of "channels" which are defined by the robot's definition file (e.g. carter). A task definition file (e.g. semantic_slam:passive:ground_truth) then declares which of these connections are provided to the API as either sensor observations or actions to be executed by a robot actuator.

The API talks to the BenchBot Supervisor, which handles loading and managing the different kinds of back-end configuration files. This abstracts all of the underlying communication complexities away from the user, allowing the BenchBot API to remain a simple interface that focuses on getting observations and sending actions.

An action is sent to the robot by calling the BenchBot.step() method with a valid action (found by checking the BenchBot.actions property):

from benchbot_api import BenchBot

b = BenchBot()
available_actions = b.actions
b.step(b.actions[0], {'action_arg:', arg_value})  # Perform the first available action

The second parameter is a dictionary of named arguments for the selected action. For example, moving 5m forward with the 'move_distance' action is represented by the dictionary {'distance': 5}.

Observations lists are received as return values from a BenchBot.step() call (BenchBot.reset() internally calls BenchBot.step(None), which means don't perform an action):

from benchbot_api import BenchBot

b = BenchBot()
observations, action_result = b.reset()
observations, action_result = b.step('move_distance', {'distance': 5})

The returned observations variable holds a dictionary with key-value pairs corresponding to the name-data defined by each observation channel.

The action_result is an enumerated value denoting the result of the action (use from benchbot_api import ActionResult to access the Enum class). You should use this result to guide the progression of your algorithm either manually or in the is_done() method of your Agent. Possible values for the returned action_result are:

• ActionResult.SUCCESS: the action was carried out successfully
• ActionResult.FINISHED: the action was carried out successfully, and the robot is now finished its traversal through the scene (only used in passive actuation mode)
• ActionResult.COLLISION: the action crashed the robot into an obstacle, and as a result it will not respond to any further actuation commands (at this point you should quit)

Standard Communication Channels

Tasks and robot definition files declare actions and observations, and these files are include through BenchBot add-ons. The add-on creator is free to add and declare channels as they please, but it is a better experience for all if channel definitions are as consistent as possible across the BenchBot ecosystem.

So if you're adding a robot that move between a set of poses, declare a channel called 'move_next with no arguments. Likewise, a robot that receives image observations should use a channel named 'image_rgb' with the same format as described below. Feel free to implement the channels however you please for your robot, but consistent interfaces should always be preferred.

If you encounter a task using non-standard channel configurations, the API has all the functionality you need as a user to handle them (actions, config, & observations properties). On the other hand, maybe the non-standard channel should be a new standard. New standard communication channels are always welcome; please open a pull request with the details!

Standard action channels:

{'distance': float}

{'angle': float}

Standard observation channels:

numpy.ndarray(shape=(H,W),
 dtype='float32')

{
 'frame_id': string
 'height': int
 'width': int
 'matrix_instrinsics':
 numpy.ndarray(shape=(3,3),
 dtype='float64')
'matrix_projection':
 numpy.ndarray(shape=(3,4)
 dtype='float64')
}

[fx 0 cx]
[0 fy cy]
[0 0 1]

[fx 0 cx Tx]
[0 fy cy Ty]
[0 0 1 0]

numpy.ndarray(shape=(H,W,3),
 dtype='uint8')

{
 'frame_id': string
 'height': int
 'width': int
 'matrix_instrinsics':
 numpy.ndarray(shape=(3,3),
 dtype='float64')
'matrix_projection':
 numpy.ndarray(shape=(3,4)
 dtype='float64')
}

[fx 0 cx]
[0 fy cy]
[0 0 1]

[fx 0 cx Tx]
[0 fy cy Ty]
[0 0 1 0]

{
 'range_max': float64,
 'range_min': float64,
 'scans':
 numpy.ndarray(shape=(N,2),
 dtype='float64')
}

{
 ...
 'frame_name': {
 'parent_frame': string
 'rotation_rpy':
 numpy.ndarray(shape=(3,),
 dtype='float64')
 'rotation_xyzw':
 numpy.ndarray(shape=(4,),
 dtype='float64')
 'translation_xyz':
 numpy.ndarray(shape=(3,),
 dtype='float64')
 }
 ...
}

Using the API to communicate with the BenchBot system

A running BenchBot system manages many other elements besides simply getting data to and from a real / simulated robot. BenchBot encapsulates not just the robot, but also the environment it is operating in (whether that be simulator or real) and task that is currently being attempted.

The API handles communication for all parts of the BenchBot system, including controlling the currently running environment and obtaining configuration information. Below are details for some of the more useful features of the API (all features are also documented in the benchbot.py source code).

Gathering configuration information

Interacting with the environment

Interacting with an agent

Creating results