Counting Constraints in POMDPs based on PID Controllers

Abstract

A Bayesian architecture is proposed for integrating counting constraints in the process of decision making with Partially Observable Markov Decision Models for robotics. In addressed problems, the counted events are detected with noisy sensors, making their detection uncertain. We handle their count as a random variable to be updated on observations. In our test scenario, an iRobot Create3 moves along a hallway and needs to count how many doors it passes while following the wall. After detecting that the given number of doors have been passed, the robot should turn around and return to the starting region. For this kind of robot, events that could be counted similarly are corridor corners, intersections, gaps, and obstacles of given shapes.

To handle uncertainty, the system applies a Partially Observable Markov Decision Process (POMDP) framework together with a Proportional–Integral–Derivative (PID) controller for wall following. The PID controller keeps the robot at a roughly constant distance from the wall using infrared (IR) range measurements, while the POMDP uses probabilistic models of the sensors and environment to infer the robot’s location along the hallway by detecting door passages, and to decide when to return. The main novelty is the successful seamless integration of counting constraints in the POMDP model for action selection.