JP2006139525A - Autonomous mobile robot - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a remote monitor system for automatically operating an autonomous mobile robot and an incorporated camera by selecting an object to be monitored from the photographic image of the camera without considering the attitude of the robot or the camera. <P>SOLUTION: This autonomous mobile robot incorporated with a camera is configured to automatically control the movement of the robot and the camera, and to project an object to be monitored at the center of the picture of the camera by storing the direction of the autonomous mobile robot and the direction and the angle of view of the incorporated camera at the time of photographing an image, and designating a position where the object to be monitored is photographed from among the photographic images. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a system for performing remote monitoring by an external operation terminal using video from a camera built in an autonomous mobile robot.

A system that includes a moving mechanism and a camera and performs remote monitoring using an external operation terminal using a robot that moves autonomously has been proposed.
Conventionally proposed technology displays an image captured by a camera built in the robot on an external operation terminal, and the user recognizes the posture of the robot and the camera by looking at the image, the robot moves forward, By instructing operation units such as backward movement, camera pan, and tilt, an image of a monitoring object is acquired, and remote monitoring is realized (see, for example, Patent Document 1 or 2).

JP 2002-321180 A

JP 2003-62776 A

  The camera image visually recognized by the user is generally accompanied by a time delay, and since the image angle and the viewing angle are different from the human field of view, the posture of the robot and the camera is recognized based on the image captured by the camera. However, it is difficult for the user to manually control the postures of the robot and the camera and acquire an image of the monitored object.

  An object of the present invention is to automatically enable operation of the robot and the camera by selecting an object to be monitored from the camera image without the user being aware of the posture of the robot and the camera. To provide a remote monitoring system.

  Representative inventions disclosed in the present application are as follows. An imaging device that captures an image, a transmission unit that transmits the captured image to the input terminal, a control unit that stores the traveling direction of the autonomous mobile robot at the time of shooting, the direction and angle of view of the built-in camera, and an input A self-supporting mobile robot comprising a receiving unit for receiving object position designation information in the captured image from a terminal, wherein the control unit performs movement control of the autonomous mobile robot and camera, and the camera Control is performed so that the monitoring object is reflected in the center of the screen.

  According to the present invention, in a remote monitoring system using an autonomous mobile robot with a built-in camera, a user can efficiently acquire an image of a monitoring object without manually controlling the posture of the robot and the camera. Can do.

As shown in FIG. 1, the autonomous mobile robot 1 of the present invention can perform bidirectional communication with the external operation terminal 100 via the communication means 200.
For the communication means 200, a public line such as a public telephone line, a cellular phone line, a PHS line, a power line, an optical fiber line, or an ADSL line can be used. Furthermore, a dedicated line can be used for the communication means 200. Further, the communication form of the communication means 200 can be either wired or wireless. Moreover, the communication means 200 can enable communication to a farther place through a repeater.

The external operation terminal 100 includes a display device 101 that displays an image and a user interface 102 that selects a position of a monitoring target.
The external operation terminal 100 includes a receiving device 103 for receiving an image transmitted from the autonomous mobile robot 1.
Further, the external operation terminal 100 includes a transmission device 104 that transmits a monitoring object selection signal 302, a robot control signal, a camera control signal, a monitoring start request signal, and a monitoring end request signal to the autonomous mobile robot 1.

In addition to a mobile phone, a PDA, a general-purpose computer, a dedicated device, or the like can be used as the external operation terminal 100.
For the user interface 102, an operation button, a joystick, or a touch panel can be used.

The structure of the autonomous mobile robot of the present invention is shown in FIG. FIG. 2 is a top view and a side view showing an embodiment of the structure of the autonomous mobile robot of the present invention.
The autonomous mobile robot 1 controls the movement of the main body 2, the right and left wheels 3a and 3b, which are moving means for moving the main body 2 on the floor, and the motors 4a and 4b for driving them. And a power supply unit 6 for supplying power to the system, and a camera 7 for photographing the front periphery of the main body 2. Rotary encoders 13a and 13b for detecting the rotation speed of the wheels are attached to the left and right motors 4a and 4b. An auxiliary wheel 11 that supports the main body 2 is provided on the lower surface of the main body 2. The entire main body is covered with a side cover 12 and an upper cover 14. The motors 4a and 4b can be controlled independently, and the main body 2 can be moved forward, backward, and swiveled on the floor surface by controlling the rotational speed and direction of rotation.

The main body 2 is equipped with a gyro 8 as azimuth angle detecting means. The gyro 8 is an angular velocity sensor such as a piezoelectric vibration gyro, and detects the speed of turning on the floor surface of the main body 2. An azimuth angle is obtained by integrating the detected angular velocity inside the control device 5.
The main body 2 is provided with a front proximity sensor 9 that detects a front obstacle and a side proximity sensor 10 that detects a side obstacle. The front proximity sensor 9 and the side proximity sensor 10 are proximity distance sensors that detect the distance to an opposing object, and detect the distance by emitting an infrared beam and detecting the direction of reflected light from the object. To do.

  Further, step sensors 15 a, 15 b, 15 c, and 15 d are provided downward from the main body 2 in front of the lower surface of the main body 2. The step sensors 15a, 15b, 15c, and 15d are proximity distance sensors that detect the distance to an opposing object, and are defined from the front of the sensor by emitting an infrared beam and detecting the direction of reflected light from the object. The presence or absence of objects within a certain distance is detected. The step sensors 15a, 15b, 15c, and 15d can detect when there is a step difference in the traveling direction of the autonomous mobile robot 1.

The camera 7 is also provided with a motor 20 for moving the left and right direction of the camera 7, a motor 21 for moving the vertical direction of the camera 7, and means for changing the lens magnification of the camera 7. And can be controlled based on the camera control signal.
In addition, the main body 2 includes a data transmission device 18 that transmits data to the external operation terminal 100, and can transmit an image 300 captured by the camera 7 to the external operation terminal 100.
The main body 2 includes the data receiving device 19 and receives a monitoring object selection signal 302, a robot control signal, a camera control signal, a monitoring start request signal, and a monitoring end request signal sent from the external operation terminal 100. Can do.

  The side cover 12 is separate from the main body 2 and is coupled to the main body 2 by a piano wire 17 so as to be relatively displaced in the horizontal direction. Inside the side cover 12, contact sensors 16a, 16b, 16c, and 16d that are pressed by the horizontal displacement of the side cover 12 are provided to be fixed to the main body 2, and the side cover 12 and external obstacles are provided. Detects contact with. A part of the side cover has high light permeability, and the camera 7 can take a surrounding image through a part of the side cover.

The control device 5 is a control computer system including a CPU, a memory, and an input / output circuit, and the control algorithm is realized by a computer program built in the memory. A part of the memory can be used for storing the image 300. Furthermore, a part of the memory can be used as a map storage means for storing room map information.
The control device 5 includes sensors obtained from the rotary encoders 13a and 13b, the gyro 8, the front proximity sensor 9, the side proximity sensor 10, the contact sensors 16a, 16b, 16c and 16d, and the step sensors 15a, 15b, 15c and 15d. Based on the information, the main body 2 can be moved by controlling the motors 4a and 4b.

The robot control signal is a rotation speed, a rotation direction, and a rotation angle of a wheel, which are parameters necessary for moving the main body 2. The control device 5 can control the motors 4a and 4b based on the robot control signal to realize the movement of the main body 2. The camera control signal is a rotation speed, a rotation direction, and a rotation angle of the motor 20 and the motor 21, which are parameters necessary for moving the direction of the camera 7. Further, the camera control signal also includes a lens magnification value, which is a parameter necessary for changing the lens magnification of the camera 7. The control device 5 can control the lens magnification of the motor 20, the motor 21, and the camera 7 based on the camera control signal, thereby realizing movement of the camera direction. The control device 5 automatically generates the robot control signal and the camera control signal based on the monitoring object selection signal 302 from the external operation terminal 100, and the monitoring object is displayed on the screen of the camera 7. The main body 2 and the camera 7 can be moved so as to be reflected in the center. Here, when the camera 7 can be rotated 360 degrees in the horizontal direction and a 360-degree peripheral image can be acquired by only rotating the camera 7 360 degrees, the monitoring object can be moved to the camera 7 by only the camera control signal. Can be projected in the center of the screen.
The monitoring object selection signal 302 is a signal that designates a part of the image 300, and indicates either the identification number of the image, the pixel position of the designated object position in the image, or the pixel block position. It becomes a form including information that can be specified.

Hereinafter, processing of the remote monitoring system of the present invention will be described.
FIG. 3 is a flowchart on the side of the autonomous mobile robot 1 in the process of operating the autonomous mobile robot 1 from the external operation terminal 100 and acquiring an image of the monitored object. When the autonomous mobile robot 1 receives the monitoring start request signal from the external operation terminal 100, as shown in step 400, the control device 5 causes the camera 7 to turn the main body 2 360 degrees, and the camera 7 displays a plurality of images 300. A 360-degree surrounding image is acquired by photographing. The 360-degree surrounding image is taken with the same lens magnification, and the magnification value is stored in the control device 5. Further, the direction 307 of the autonomous mobile robot 1 at the moment when each of the plurality of images 300 constituting the 360-degree surrounding image is captured is stored in the control device 5. Therefore, even after acquiring the 360-degree surrounding image, by designating an arbitrary direction, the autonomous mobile robot 1 turns on the spot again until it turns to the specified direction, and the image captured in that direction is reproduced again. You can shoot. Each of the plurality of images 300 constituting the 360-degree surrounding image is assigned an ID for identifying the image and is stored in the control device 5. Then, as shown in step 401, the 360-degree surrounding image is transmitted to the external operation terminal 100. Here, using the direction 307 when the image 300 was taken, projective transformation is performed on the plurality of images 300 constituting the 360-degree surrounding image, and an image is synthesized to generate a processed image (panoramic image) can do. This processed image can be stored in the control device 5 and transmitted to the external operation terminal 100 as with the previous image 300. The processed image can be inversely converted into the plurality of images 300.

  The external operation terminal 100 displays the received image 300 or processed image on the display device 101, and the user finds the monitoring target object from the display image and designates the monitoring target object via the user interface 102. . The external operation terminal transmits the pixel position in the image of the designated monitoring object as the monitoring object selection signal 302 to the autonomous mobile robot 1. Here, when the external operation terminal 100 receives the image 300, the monitoring target selection signal 302 includes an ID for identifying the image 300.

  Then, as shown in step 402, when the autonomous mobile robot 1 receives the monitoring object selection signal 302 from the external operation terminal 100, the pixel position indicated by the monitoring object selection signal 302 comes to the center of the screen of the camera 7. For this purpose, a horizontal turning angle A1 and a vertical movement angle A2 are obtained (step 403). FIG. 4 shows the relationship among the monitoring object selection signal 302, the horizontal turning angle A1, and the vertical movement angle A2 in the image 300. The horizontal turning angle A1 is a horizontal angle from the center position of the image 300 to the monitoring object, and the vertical movement angle A2 is a vertical movement angle from the center position of the image 300 to the monitoring object. Is an angle. FIG. 5 shows the relationship between the magnification value of the camera 7 in the image 300 and the angle of view A1, FIG. 5 (a) shows when the magnification value is 1, and FIG. 5 (b) shows that the magnification value is α. The times are shown. The angle of view A3 in FIG. 5A is an angle that matches the viewing angle in the real world, and the autonomous mobile robot 1 stores the value of the angle of view A3 in advance.

  Here, since the angle of view A3 depends on the number of pixels of the CCD constituting the camera 7, etc., the angle of view A3 varies depending on the mounted camera. The angle of view A4 in FIG. 5B depends on the angle of view A3, and its value is generally (A3 / α). FIG. 6 shows the relationship between the pixel resolution (horizontal direction) of the image 300 and the horizontal turning angle A5. By using the number of pixels 309 in the horizontal direction and the value of the angle of view, the horizontal turning angle A5 per pixel can be calculated. In this way, the horizontal turning angle A1 can be obtained from the monitoring object selection signal 302 and the horizontal turning angle A5 per pixel.

  5 and 6, only the horizontal turning angle has been described, but the same applies to the vertical movement angle. Therefore, the vertical movement angle A2 can be obtained from the monitoring object selection signal 302.

  Then, as shown in step 404, a robot control signal and a camera control signal are generated from the horizontal turning angle A1 and the vertical movement angle A2. A robot control signal is generated from the horizontal turning angle A1, and as shown in steps 405 and 406, the main body 2 is turned on the spot by the designated angle A6. FIG. 7 shows the relationship between the designated angle A6, the current traveling direction 500 of the autonomous mobile robot 1, the horizontal turning angle A1, and the direction 307 of the image 300. In FIG. 7, the monitoring object exists in a direction left-turned from the direction 307 of the image 300 by the horizontal turning angle A1. That is, this is a direction in which the autonomous mobile robot 1 turns right from the current traveling direction 500 by the specified angle A6. In order to display the monitoring object in the center of the camera screen, it is necessary to obtain the designated angle A6, which can be calculated from the traveling direction 500, the direction 307 of the image 300, and the horizontal turning angle A1. In FIG. 7, only the generation of the designated angle A6, which is a robot control signal, has been described, but the camera control signal can be similarly calculated.

  In the above embodiment, it is assumed that there is one camera 7, but when two or more cameras are mounted, the distance between the autonomous mobile robot 1 and the monitoring object can be calculated by stereo vision. . That is, the distance can be calculated using the difference (parallax) of the position of the monitoring object in each camera screen. By automatically generating a robot control signal from the distance to the monitoring object, the autonomous mobile robot 1 can be moved from the monitoring object to an arbitrary distance.

When the control of the robot and the camera is finished, an image 300 is taken by the camera 7 as shown in step 409. In the center of the image 300, the monitoring object specified by the monitoring object selection signal 302 is reflected. Then, the process returns to step 401, and an image in which the designated target is imaged is transmitted to the external operation terminal 100.
Further, as shown in step 410, when the autonomous mobile robot 1 receives a monitoring end request signal from the external operation terminal 100, the above flow is ended.

In addition, the signal received by the autonomous mobile robot 1 from the external operation terminal 100 is not only the monitoring target selection signal 302 generated by an external device, but also a robot control signal and a camera control signal. it can. In this case, the process proceeds from step 402 to step 405 in FIG. The robot control signal transmitted from the external operation terminal 100 can cause the autonomous mobile robot to move forward, backward, and turn on the spot by the control device 5. The camera control signal transmitted from the external operation terminal 100 can move the camera 7 up, down, left, and right and change the lens magnification value via the motor 20.
According to the present invention, in a remote monitoring system using an autonomous mobile robot with a built-in camera, by selecting a monitoring object from a captured image, the robot automatically performs movement control and camera control. The user can efficiently acquire an image of the monitoring object without manually controlling the posture of the robot and the camera.

4, 5, 6, and 7, the process for the image 300 has been described, but the same process can be performed for the processed image. That is, if the processed image is inversely converted into a plurality of images 300, the same processing as the above processing can be executed.
In the description of step 400 above, it is described that a 360-degree surrounding image is captured at the current position of the autonomous mobile robot 1, but room map information is stored in the control device 5 of the autonomous mobile robot 1. If it is, it is possible to search for a position suitable for acquiring a 360-degree surrounding image and move the autonomous mobile robot 1 to that position, and then execute step 400. The position suitable for acquiring the 360-degree surrounding image is, for example, the distance between the autonomous mobile robot 1 and an obstacle existing in the vicinity is longer than the maximum value or the maximum. It is a position that makes it easy to capture an overview of the entire room.

  As described above, in order to solve the above-described problems, the autonomous mobile robot according to the present invention includes a main body, a moving means for moving the main body on the floor surface, and the moving means by controlling the moving means. A movement control means for realizing movement; a camera provided in the main body capable of photographing the periphery of the robot; a camera control means capable of controlling the orientation and lens magnification of the camera; and an image captured by the camera Transmission means for transmitting to the operation terminal, and reception means for receiving a monitoring object selection signal, robot control signal, and camera control signal from the external operation terminal are provided. The monitoring object selection signal is a signal indicating a pixel position in an image received by the external operation terminal. The robot control signal is a signal for instructing the operation of the autonomous mobile robot such as forward movement, backward movement, and turn on the spot. The camera control signal is a signal for instructing the rotational movement of the camera up, down, left, and right, and changing the lens magnification.

In addition, the autonomous mobile robot includes an all-around image obtaining unit that records 360-degree images as a plurality of images by performing 360-degree turning on the spot.
The autonomous mobile robot includes means for recognizing the current position and direction of the robot, and means for storing map information of the room, and is suitable for acquiring the all-around image. Means for searching for a position where monitoring can be performed.

In addition, the autonomous mobile robot and the camera monitor the movement control unit and the camera control unit based on the monitoring object selection signal so that the monitoring object is reflected in the center of the screen of the camera. Control means are provided.
In addition, an external operation terminal for remotely operating the autonomous mobile robot includes a receiving unit for receiving an image captured by the camera, a display unit for displaying the image, and a monitoring target from the display image. Monitoring object selection means for selecting an object, and transmission means for transmitting the monitoring object selection signal, robot control signal, and camera control signal.

The figure which shows one structure of the remote monitoring system of this application. The top view and side view which show one Example of the structure of the autonomous mobile robot of this application. The flowchart which shows the algorithm of one Example of the remote monitoring system process of this application. Explanatory drawing of the camera picked-up image and control parameter of this application. Explanatory drawing of the lens magnification and a field angle in the camera picked-up image of this application. Explanatory drawing of the resolution and pixel unit angle in the camera picked-up image of this application. Explanatory drawing of the turning angle of the autonomous mobile robot of this application.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Autonomous mobile robot 2 Main body 3a, 3b Wheel 4a, 4b Wheel motor 5 Control apparatus 6 Battery 7 Camera 8 Gyro 9 Front proximity sensor 10 Side proximity sensor 11 Auxiliary wheel 12 Main body side cover 13a, 13b Rotary encoder 14 Main body upper surface Cover 15a, 15b, 15c, 15d Step sensor 16a, 16b, 16c, 16d Contact switch sensor 17 Side cover supporting piano wire 18 Data transmitter 19 Data receiver 20 Camera drive motor 100 External operation terminal 101 Display device 102 User interface 103 Data receiver 104 Data transmitter 200 Communication means 300 Captured image 301 Processed image 302 Monitoring object selection signal 303 Robot control signal 304 Camera control signal 305 Monitoring start request signal 306 Monitoring end request signal 307 Direction of the robot at the time of acquiring the captured image 308 Lens magnification value 309 Number of pixels in the horizontal direction in the captured image 500 Current direction of the robot A1 Horizontal turning angle A2 Vertical movement angles A3, A4 Horizontal angle of view A5 Horizontal direction per pixel Angle of view A6 Turn specified angle.

Claims (4)

An autonomous mobile robot connected to an input terminal having a user interface,
An imaging device for capturing an image;
A transmission unit for sending the captured image to the input terminal;
A control unit that stores the traveling direction of the autonomous mobile robot at the time of shooting, the direction of the built-in camera, and the angle of view;
A receiving unit for receiving object position designation information in the captured image from the input terminal;
The control unit controls the movement of the autonomous mobile robot and camera so that the monitoring object is reflected in the center of the screen of the camera.   2. The autonomous system according to claim 1, wherein the object position designation information includes an identification number of an image in which the object exists and information for specifying a position of the designated object in the image. Mobile robot. The image transmitted from the imaging device via the transmission unit is a moving image obtained by imaging 360 degrees around the autonomous mobile robot obtained via the imaging unit. The autonomous mobile robot described. Furthermore, it has a storage unit that stores the map information of the travel location,
The autonomous mobile robot according to any one of claims 1 to 3, wherein the control unit includes means for searching for a position where the distance to the obstacle is equal to or longer than a threshold or the longest.

Images