JP4849178B2 - Mobile robot - Google Patents

Description

  The present invention relates to an autonomous mobile robot. In particular, the present invention relates to a robot that moves by recognizing the surrounding environment using a camera.

  A mobile robot has been developed that includes a trunk and a connection part that is connected to the trunk through a joint having a drive mechanism. Some robots of this type create an environment map from an environment image captured by a trunk-side camera provided on the trunk, and plan a movement route to a target position based on the obtained environment map. is there. In Patent Document 1, a parallax image or a distance image is calculated from a captured image by a camera provided on a robot head, a plane parameter is calculated from the calculated parallax image or distance image, and the calculated plane parameter is calculated. The environment map is created by extracting a plurality of planes including the floor from the environment map, the range in which the robot can move is recognized from the created environment map, and the movement path of the robot is planned based on the recognized range.

JP 2005-92820 A

In the case of a robot having a connection part connected to the trunk through a joint having a drive mechanism, the field of view of the camera provided on the trunk of the robot depends on the position of the connection part such as an arm or a leg. May be blocked by the connection site, and some surrounding objects may not be imaged. That is, an unknown environment may exist behind the connection site.
If there is an unknown environment behind the connection site, it will be impossible to plan a trunk movement route. Or it becomes impossible to plan the movement path | route of connection parts, such as an arm and a leg. In particular, when planning a route for moving a connection site to a target position, the target position is in an unknown environment, and a situation in which the target position cannot be recognized easily occurs. For example, if it is possible to obtain two or more trunk side image information by using two or more of the trunk side camera that is arranged at a position that different depth information of the object being imaged in both images was unknown However, the relative positional relationship of the object with respect to the mobile robot can be calculated. However, since that is a shadow of the connecting portion, the surrounding objects not imaged in one of the trunk side image information, it is impossible to identify the position.

In the current technology, every time an unknown environment is generated, the trunk or the connected part is moved to image the unknown environment, and after the unknown environment is not an unknown environment, a process for determining the movement path is required.
However, the above-described processing not only requires an operation that is not essentially required, but is also difficult to cope with in a robot that continuously moves the trunk or in an environment where surrounding objects move.

  An object of the present invention is to realize a mobile robot that does not generate an unknown environment.

In the present invention, a camera is also prepared at the connection site in order to capture an area that is blocked by the connection site and cannot be imaged. If a camera is also prepared at the connection site, it is possible to prevent a situation where an image cannot be captured by the connection site .

For example, when depth information is known, useful information can be obtained by obtaining a single composite image itself. If the depth information is unknown, it is useful to obtain a plurality of pieces of image information picked up from different viewpoints and calculate the positions of surrounding objects from them.
The robot according to the present invention includes a trunk, a connection portion connected to the trunk via a joint having a drive mechanism, two or more trunk side cameras provided on the trunk, and a connection portion. First calculating means for calculating a relative positional relationship of an object existing in an imaging range with respect to a mobile robot from a connected site side camera and two or more trunk side images captured by two or more trunk side cameras And calculating a relative positional relationship of an object existing in the imaging range with respect to the mobile robot from the trunk side image captured by at least one of the trunk side cameras and the connection site side image captured by the connection site side camera. Bei Eteiru 2 calculation means. The second calculation means selects the trunk side image with the smallest range in which the connection site is imaged and uses it in the calculation. The robot of the present invention calculates position information by processing image information. It is not always necessary to create a composite image.

In the robot according to the present invention, the trunk side image captured by the two trunk side cameras is used in combination , and in addition, the trunk side image captured by one of the trunk side cameras and the connection part side camera are used. The connection site side images imaged in the above are used in combination. These are also two or more pieces of image information picked up from different viewpoints, and the relative positional relationship of surrounding objects with respect to the mobile robot can also be calculated from these pieces of image information. Even if it cannot be captured by both trunk-side cameras due to the shadow of the connected part, if it can be captured by either one of the trunk-side cameras, it can be used to move surrounding objects by combining it with the connected part-side image information. The relative positional relationship with respect to the type robot can be specified. In this case, a plurality of composite images is obtained by imaging by changing the viewpoint, or the like when the depth information For example is unknown, it is possible such to identify the three-dimensional positions of feature points present in the imaging range Become. Thereby, the range of unknown environment can be narrowed.

  Note that the means for identifying the range where the connection site is imaged in the trunk side image is largely Two methods are possible separately. One method is the positional relationship of the connected part to the trunk side camera ( This is known to the robot) and is calculated by calculating the range where the connected part should be imaged. This is a method. The other method is to perform image processing on the trunk side image and shape the connected part (this is The area corresponding to (known to the bot) is extracted, and then the connection site is imaged This is a method for identifying. The range in which the connection site selected by the second calculation means is imaged is The least number of trunk side images may be selected by any method.
  The robot described above can calculate the subsequent movement path of the trunk or connection site. In order not to leave an unknown environment, no unnecessary operation is required. Also surroundings Even in the environment where the body moves, the movement route can be determined from moment to moment.

In the present invention, by providing a camera at the connection site, a function as a second visual recognition unit is added to the connection site. With this configuration, it is possible to specify the relative positional relationship of the surrounding object with respect to the mobile robot even in the surrounding object that is in the shadow of the connection site and cannot be captured by the trunk-side camera . According to the present invention, the range of the unknown environment can be reduced or eliminated, and the robot can be operated safely and quickly.

It is a figure which shows the external appearance of the robot of Example 1. FIG. It is a figure which shows the outline of the robot mechanism of Example 1. FIG. FIG. 3 is a view of the field of view of the robot according to the first embodiment. It is a figure which shows an example of the trunk side image. It is a figure which shows an example of a connection site | part side image. It is a flowchart which shows the process of an image composition processing apparatus. It is the figure which showed the trunk side image of FIG. 4, and the imaging range R of the connection site | part. It is the figure which showed the connection site | part side image and corresponding range S of FIG. It is a figure which shows an example of the output result of step S613. It is a figure which shows an example of the some trunk side image and the imaging range R of a connection site | part. It is a figure which shows an example of a connection site | part side image and the corresponding range. It is a figure which shows the external appearance of the robot of Example 2. FIG. It is a figure which shows the outline of the robot mechanism of Example 2. FIG. It is a visual field figure of the robot of Example 2. 10 is a flowchart illustrating a process up to an operation process of the robot according to the second embodiment. It is a flowchart which shows the process of the acquisition process of a stereo image. It is a figure which shows the outline of the robot mechanism of Example 3. FIG.

First, the main features of the embodiments described below are listed.
(Characteristic 1) The robot includes a traveling mechanism (a wheel in this embodiment, but a leg link may be used) in the trunk.
(Feature 2) The robot calculates the trunk trajectory from the environment map.
(Feature 3) The robot calculates the movement locus of the connected part from the environment map.
(Feature 4) The robot calculates the trunk trajectory and the connected part movement trajectory from the environment map.
(Feature 5) The robot controls the posture of the robot and the imaging direction of each camera by a joint angle control device, a wheel rotation speed control device, and a position information detection device. The imaging direction of the trunk side camera is calculated by the trunk side camera imaging direction calculation device, and the imaging direction of the connection site side camera is calculated by the connection site side camera imaging direction calculation device.
(Feature 6) The robot creates a movement path based on information obtained from the composite image. Based on the path, the joint angle control device, the wheel rotation speed control device, and the position information detection device control the actuator to move the trunk and / or the connection site along the created movement route.
(Characteristic 7) The robot includes a route correction device that corrects a trajectory created in the past in response to a change in the surrounding environment and changes the movement route.

FIG. 1 shows the appearance of the robot of this embodiment. The robot 100 according to the present embodiment includes a trunk 103 and one connection portion (arm) 104 connected to the trunk 103 via a shoulder joint 203. Two or more connection parts 104 may exist. The trunk 103 includes a trunk 200, a neck joint 202, and a head 201, and the trunk-side camera 105 is fixed to the head 201. The trunk-side camera 105 can change the imaging direction with respect to the trunk 200 by the neck joint 202. A pair of wheels are arranged below the trunk 200, and the trunk 103 is moved using the wheels 206.
The connection site 104 includes a shoulder joint 203, an elbow joint 204, and a wrist joint 205, and the side camera 106 is fixed to the palm. The shoulder joint 203, the elbow joint 204, and the wrist joint 205 have active mechanisms, and the connection site side camera 106 changes the imaging direction with respect to the body 200 by the shoulder joint 203, the elbow joint 204, and the wrist joint 205. Can be changed. Further, the camera position of the connection site side camera 106 with respect to the body 200 can be changed.
This robot includes means for measuring the position of the head 201 in the work environment, and recognizes the camera position and imaging direction of the trunk-side camera 105. In addition, the camera position and imaging direction of the connection site side camera 106 with respect to the trunk 103 are recognized.
The robot 100 of this embodiment approaches the switch panel 10 using the wheels 206, and operates the switch groups 12a, 12b,... Disposed on the wall surface of the switch panel 10 using the connection portion 104. The position of the wall surface of the switch panel 10 is known.

FIG. 2 is a diagram showing an outline of a mechanism 101 that recognizes the surroundings of the robot 100. The robot 100 of this embodiment has a control device 114. The control device 114 includes a composite image creation device 110, and the composite image creation device 110 inputs trunk side image information from the trunk side camera 105 and inputs connection part side image information from the connection part side camera 106. Then, the position and imaging direction of the trunk side camera 105 are input from the trunk side camera position imaging direction calculation device 111, and the position and imaging direction of the connection site side camera 106 are input from the connection site side camera position imaging direction calculation device 109. Then, composite image information is created from the trunk side image information and the connection part side image information, a subsequent movement route is created from the created composite image information (movement route creation device 113), and according to the created movement route information, The actuator 206 that rotates the wheel 206 and the joint group is controlled, and the wheel 206, the head 201, and the connection part 104 are moved along the created movement path. . The robot 100 detects a rotational speed of a wheel 206, a rotational speed detector 119 for detecting the rotational speed of the wheel 206, a joint angle detector 117 for detecting a joint angle of each joint, and a position of the trunk 200 installed in the body 200. A position information detection device 121 is provided, and information acquired by these devices is input to the trunk side camera position imaging direction calculation device 111 and the connected part side camera position imaging direction calculation device 109.
The robot has movement path creation means 113 for creating a movement path by recognizing the surrounding environment from image information captured by a camera whose position and imaging direction are known, and after moving along the created movement path, the camera position And re-calculate the imaging direction, and then move autonomously while repeating the process of imaging again.

  FIG. 3 shows a view of the robot 100. If the connection part 104 of the robot 100 enters the field of view of the trunk-side camera 105, the environmental information behind it cannot be obtained. Specifically, in the trunk side image captured from the trunk side camera 105, the point A and the point B of the switch panel are captured, but the point C and the point D are not captured. Therefore, the information on the surrounding objects at the points C and D becomes unknown, and it is impossible to recognize the environmental situation and grasp the relative positional relationship of the surrounding objects with respect to the robot 100. By providing 106, the visible range is expanded, and an unknown environment including points C and D can be grasped.

4 and 5 show examples of the trunk side image and the connection part side image, respectively. In the trunk side image of FIG. 4, the surface of the panel which is hidden behind the shadow of the connection site 104 and is not captured is captured in the connection site side image of FIG. 5. This is because the positions of the head 201 and the connection part 104 are controlled by the joint angle detection device 117, and the imaging direction of the connection part side camera 106 depends on the relative positions of the trunk side camera 105 and the connection part side camera 106. This is made possible by adjusting. The imaging direction of the connection site side camera 106 is adjusted so that the trunk side camera captures a direction that cannot be captured by the shadow of the connection site 104.
Information regarding differences in magnification, image size, and the like of each camera is grasped by the trunk side camera position imaging direction calculation device 111 and the connected part side camera position imaging direction calculation device 109, and an image based on the information at the time of synthesis is used. Since the processing is performed, the imaging ranges of the respective images do not have to completely match. For example, since the connection site side camera 106 is different from the trunk side camera 105 in the distance to the switch panel 10 and the connection site 104 is inclined with respect to the head 201, the connection site side in FIG. In the image, the magnification of the switch panel 10 taken is larger than that of the trunk side image in FIG.

FIG. 6 shows a flowchart of the image composition process. At this time, the robot 100 and the switch panel 10 are in a positional relationship as shown in FIG. The connection part 104 of the robot 100 is at the position shown in FIG. 4, and a similar image is captured by the trunk-side camera 105. The robot 100 is about to move the connection portion 104 in order to press the switch 12c (see FIG. 1) of the switch panel 10. However, since the switch 12c is within the unknown environment in the trunk side image, the composite image creation process shown in FIG. 6 is started.
In step S <b> 601, trunk-side image information is input to the composite image creation apparatus 110. In step S603, a feature point _Xn of a part that is an unknown environment in the trunk side image is extracted. FIG. 7 shows an example of feature points of the trunk side image extracted in step S603. Since the shape of the connection part 104 of the robot 100 is known, the range (imaging range of the connection part) in which the connection part is imaged can be specified from the trunk side image (first specifying unit). Points X ₁ to X ₆ in the figure are feature points extracted so as to include the imaging range of the connected part. In the present embodiment, six feature points that can be connected by a straight line are extracted, but the feature points _Xn may be taken along the outline of the connected portion. In step S605, the imaging range R of the connected region (a range in which the surrounding surrounding object information is unknown) is specified from the feature points extracted in step S603. Range surrounded by the X ₆ from the feature point X ₁ in FIG. 7 is an imaging range R of the approximate connection sites. By increasing the number of feature points _{Xn to be} extracted, the range R surrounded by the feature points can be exactly matched to the imaging range of the connected part.

In step S <b> 607, the connected part side image information is input to the composite image creating apparatus 110.
In step S609, from the connection site side image information input in step S607, the extracting the Z _n points corresponding to the feature points of the image scan range R of the connecting portion of the trunk side image. 8 shows an example of a feature point Z _n connection site side image extracted in step S609. Points Z ₁ to Z _{6 in} FIG. 8 are points corresponding to feature points X ₁ to X ₆ extracted from the trunk side image.
In step S611, from the feature point Z _n extracted in step S609, (the trunk side image which range unknown information) corresponding range S of the connection portion-side image to identify the (second specifying means).

In step S613, the corresponding range S obtained from the connected part side image is applied to the input images of the trunk side camera and the connected part side camera so as to match the imaging range R of the connected part obtained from the trunk side image. Based on the added difference information, processing such as size adjustment and tilt angle rotation adjustment is performed.
In step S615, the imaging range R of the connected part of the trunk side image is replaced with the corresponding range S for which the preparation process for synthesis has been completed. As a result, in step S617, a composite image without an unknown range is output. FIG. 9 shows an example of a composite image. The imaging range R of the connection site indicated by the broken line is replaced with the corresponding range S, and a composite image in which the connection site is not captured can be obtained.

The present embodiment may have a plurality of trunk side cameras. In that case, the imaging range R _x of the connected part is extracted from each trunk side image, the corresponding range S _x is extracted from the connected part side image for the imaging range R _x of each connected part, The imaging range R _x is replaced with the corresponding range S _x .

FIG. 10 illustrates three trunk-side images TC ₁ , TC _2, and TC ₃ captured from three trunk-side cameras. The trunk side images TC ₁ , TC ₂ and TC ₃ are subjected to the processing from step S601 to step S605, and the imaging ranges R ₁ , R ₂ and R ₃ of the connected region are specified. Since the positions of the three trunk side cameras fixed to the head are different, there is a difference in the position of the imaging range of the connection part of each image.
FIG. 11 illustrates a connection part side image AC captured by the connection part side camera that operates in conjunction with the three trunk side cameras. In order to compensate for the imaging ranges R ₁ , R _2, and R ₃ of the connected parts of the trunk-side images TC ₁ , TC _2, and TC ₃ , the connected part-side image AC is subjected to the corresponding ranges S ₁ , S, by the processing from step S 607 to step S 611. specifying the ₂ and _{S 3.}
By the processing from step S613 to step S617, the imaging ranges R ₁ , R ₂ and R ₃ of the connection site of the trunk side images TC ₁ , TC ₂ and TC ₃ are the corresponding ranges S ₁ and S ₂ of the connection site side image AC. and replaced by S _3, no trunk side camera images unknown range is created.

  Further, the present embodiment may have a plurality of connection site side cameras. In this case, it is preferable to provide a determination device that uses the connected part side image that can extract the corresponding range S in the widest range with respect to the imaging range R of the connected part of each trunk side image.

  In this embodiment, the relative position of the surrounding object with respect to the robot 100 can be calculated using the composite image obtained by the above method. The relative position of the surrounding object relative to the robot 100 here is grasped by converting the relative position coordinates of the object from the world coordinate system to a coordinate system centered on the robot.

Since Example 1 and Example 2 have a common part, the same part or device is assigned the same number, and redundant description is omitted.
FIG. 12 is a diagram illustrating an appearance of a robot using a stereo camera. The robot 100 according to the present embodiment includes a trunk 103 and a single connection portion (arm) 104 connected to the trunk 103 via a shoulder joint 203. Two or more connection parts 104 may exist. The trunk 103
A torso 200, a neck joint 202, and a head 201 are provided, and the trunk side cameras 105 (a) and 105 (b) are fixed to the head 201. The trunk side cameras 105 (a) and 105 (b) can change the imaging direction with respect to the trunk 200 by the neck joint 202. A pair of wheels 206 is disposed below the body 200 and moves using the wheels 206.
The connection site 104 includes a shoulder joint 203, an elbow joint 204, and a wrist joint 205, and the connection site camera 106 is fixed to the palm. The shoulder joint 203, the elbow joint 204, and the wrist joint 205 have active mechanisms, and the connection site side camera 106 changes the imaging direction with respect to the body 200 by the shoulder joint 203, the elbow joint 204, and the wrist joint 205. Can be changed. Further, the camera position of the connection site side camera 106 with respect to the body 200 can be changed.
This robot includes means for measuring the position of the head 201 in the work environment, and recognizes the camera positions and imaging directions of the trunk-side cameras 105 (a) and 105 (b). In addition, the camera position and the imaging direction of the connection site side camera 106 with respect to the body 200 are recognized.
The robot 100 according to the present embodiment visually recognizes the surrounding objects 30, 40, 50, and 60, creates a connection portion trajectory that does not collide with them, and places the grasped object 20 in an empty space. The trunk-side cameras 105 (a) and 105 (b) of the robot 100 normally function as stereo cameras, and points A, B, C, and D in the figure are the viewpoints of the robot 100.

FIG. 13 is a diagram illustrating an outline of a mechanism 101 that recognizes the surroundings of the robot 100 using the stereo camera of the present embodiment. The robot 100 of this embodiment has a control device 114. The control device 114 includes an image information processing device 116, and the image information processing device 116 inputs trunk side image information from the trunk side cameras 105 (a) and 105 (b) provided on the trunk 103. Then, the connection part side image information is input from the connection part side camera 106 provided in the connection part 104, and the trunk side cameras 105 (a) and 105 (b) are input from the trunk side camera position imaging direction calculation device 111. The position and the imaging direction are input, the position and the imaging direction of the connection part side camera 106 are input from the connection part side camera position imaging direction calculation device 109, and the environment map is obtained from the information on the image information and the camera position and the imaging direction. It is created and stored in the environmental map storage device 112. Based on the environmental map information, a subsequent movement route is created (movement route creation device 113), and the actuator group 115 that rotates the joint having the wheel 206 and the active mechanism is created according to the created movement route information. The wheel 206, the head 201, and the connection site 104 are moved along the movement path. The robot 100 detects a rotational speed of the wheel 206, a wheel rotational speed detection device 119 that detects the joint angle of each joint, a joint angle detection device 123 that detects the joint angle of each joint, and a body 200 that detects the position of the body 200. A position information detection device 121 is provided, and information acquired by these devices is input to the trunk side camera position imaging direction calculation device 111 and the connected part side camera position imaging direction calculation device 109.
The robot 100 recognizes the surrounding environment from image information captured by a camera with a known position and imaging orientation, calculates a movement path, moves along the calculated movement path, recalculates the camera position and imaging direction, and then The camera moves autonomously while repeating the process of imaging again.

FIG. 14 shows a bird's-eye view of the field of view of the robot 100. If the connection part 104 of the robot 100 and the grasped object 20 enter the field of view of the trunk-side cameras 105 (a) and 105 (b), the environmental information behind them cannot be obtained.
Specifically, in the example of FIG. 14, the point A and the point B can be captured by both the trunk side cameras 105 (a) and 105 (b) as indicated by solid arrows, and thus the trunk side image Thus, a stereo image can be generated, and the information can be continuously acquired in time series as the robot moves.
On the other hand, with respect to the points C and D, stereo viewing using the trunk-side cameras 105 (a) and 105 (b) at the respective points is performed only by the trunk-side camera, depending on the movement position of the lower limbs or the movement of the connection part. Cannot be obtained, and environmental information about the surroundings cannot be acquired. A broken line arrow indicates that an image having the same feature point cannot be captured because the connection site or the grasped object blocks the line of sight.
In the case of a stereo camera, especially when information such as the depth and height of surrounding objects is required, it is possible to use a camera whose position information is already known, or when a single camera is moving and capturing images. In addition, it is essential to obtain parallax information from a plurality of images. For this reason, since the points C and D cannot be viewed with both eyes, the positional relationship between the objects in the vicinity of the points C and D cannot be recognized.

In such a case, in the present embodiment, it is difficult for any of the trunk side cameras 105 to function satisfactorily as a stereo camera due to the presence or absence of an object that obstructs the field of view, such as the viewpoints of point C and point D. In this case, a stereo image can be obtained using any one of the trunk side camera 105 and the connection site side camera 106.
Specifically, when there is no object that obstructs the field of view, such as the connection part 104, such as the periphery of the point A and the point B, stereo images are acquired by the trunk-side cameras 105 (a) and 105 (b), and the surrounding objects The relative positional relationship with respect to the robot 100 can be calculated (first calculation means). On the other hand, in the case of point C, the trunk side camera 105 (b) and the connection site side camera 106 are viewed with both eyes, and in the case of point D, the trunk side camera 105 (a) and the connection site side. By performing binocular viewing with the camera 106, information necessary for recognizing the entire surrounding environment can be obtained, and the relative positional relationship of the surrounding objects with respect to the robot 100 can be calculated (second calculation means).

  When binocular vision is performed using the connection site side camera 106, which of the plurality of trunk side cameras is used to acquire a stereo image depends on the trunk where the connection site 104 is captured in the smallest range. The trunk camera from which the side image is acquired can be selected for use in the calculation. This selection is determined by the image information processing apparatus 116.

FIG. 15 is a flowchart showing an outline of the operation of the robot of this embodiment. At this time, the robot 100 is gripping the gripping object 20, and creates a connection site trajectory that does not collide with the objects 30, 40, 50 and 60, and wants to execute an operation of installing the gripping object 20 in an empty space. The connection part 104 of the robot 100 is at a position as shown in FIG. 14, and images representing such a state are taken from the positions of the trunk-side cameras 105 (a) and 105 (b). In creating the trajectory of the connection part 104, since the object 50 and a part of the object 60 are unknown by the connection part 104 and the grasped object 20, the image information processing shown in FIG. 15 is started.
In step S500, a plurality of images captured by the trunk side cameras 105 (a) and 105 (b) and the connected part side camera 106 are input to the image information processing apparatus 116, and a stereo image is acquired. The image information processing apparatus 116 may determine whether or not to use the connection part side image based on the size of the unknown range of the trunk side image, and is blocked by the connection part 104 and has a smaller range as the unknown range. Means for obtaining a stereo image from a combination of images is provided.

In step S502, the relative positions of the surrounding objects of the robot 100 are calculated based on the stereo image selected and acquired in step S500. The image data is converted from the world coordinate system to a coordinate system centered on the robot, the positional relationship between the surrounding objects is calculated, and is output as relative position information between the robot 100 and the surrounding objects. As an example, a parallax image or a distance image may be used for acquiring the relative position information.
At this time, when obtaining stereo image information from the trunk side images, and when obtaining a stereo image using the trunk side image and the connection part side image, they are referred to as first calculation means and second calculation means, respectively. It is desirable to prepare different calculation means. The trunk side cameras are parallel to each other, but the calculation means using the trunk side image and the connected part side image are different in relative positions of the respective cameras. For example, parallelization of stereo images and hand-eye calibration are performed. Various coordinate system transformations can be applied.

  The relative position information calculated in the image processing apparatus 116 is constructed as data representing the relative positions of surrounding objects in step S504 and stored in the environment map storage apparatus 112. The environmental map refers to data that allows the robot 100 to grasp the relative positions of surrounding objects. For example, when the image information processing apparatus 116 calculates a parallax image, the environmental map can be represented by an image parallax space (SDS space Spatial Disparity Space). In the SDS space, the surrounding objects 30, 40, 50, and 60 having the points A, B, C, and D appear as curved surfaces having a depth.

  In step S506, the movement route of the robot 100 is created based on the environmental map information. The robot 100 according to the present embodiment is designed to visually recognize the surrounding objects 30, 40, 50, and 60, create a connection portion trajectory that does not collide with them, and place the grasped object 20 in an empty space. Therefore, in such a case, it is desirable to detect a plane on which no surrounding object exists from the environmental map information and create a trajectory of the connection site so that the grasped object 20 can be placed there. The actual operation is realized by the actuator 115 in step S508 along the created movement path.

  FIG. 16 is a flowchart showing image information processing of the robot 100. There are two methods for acquiring the stereo image in step S500 of FIG. 15, a method of acquiring using a plurality of trunk-side images and a method of acquiring using any of the trunk-side images and the connected part-side image. . In step S <b> 160, a plurality of trunk side images captured by the trunk side cameras 105 (a) and 105 (b) and a connected part side image captured by the connected part side camera 106 are input to the image information processing apparatus 116. Is done. In step S162, it is determined whether the connection site 104 is captured in the trunk side image. When the connection part is not imaged (in the case of NO), the process proceeds to step S164, a stereo image is acquired from the trunk side images, and the first calculation unit is executed. On the other hand, when the connected part is imaged in the trunk side image (in the case of YES), the process proceeds to step S166, a stereo image is acquired from one of the trunk side image and the connected part side image, and the second calculation is performed. Execute means. The trunk-side image selected in step S166 is the trunk-side image having a smaller range where the connection site is imaged.

  In the present embodiment, the case where the three-dimensional information of the surrounding object is obtained using a total of three cameras for the trunk and the connection part is illustrated, but more cameras may be used, or one connection You may take the technique of imaging continuously, as a part side camera moves.

  In this embodiment, the case of recognizing a static surrounding object is illustrated. However, after the connected part starts to move along the route created by the movement route creation device 113, the surrounding object that becomes a new obstacle moves. It is not possible to deal with a case where it appears on the route, or when the positions of surrounding objects and the robot change due to the movement of the lower limbs or the like, and the connection site trajectory needs to be corrected.

FIG. 17 shows a form that can deal with a dynamic surrounding object. The robot mechanism 101 can appropriately correct a route once issued by providing the movement mechanism correcting device 125 for correcting the activation of the connection site in the robot mechanism of the second embodiment.
In addition, by appropriately updating the data of the surrounding environment, it is possible to simultaneously recognize the operation of the connected portion and the change of the surrounding environment at the same time and correct the trajectory accordingly.

Specific examples of the present invention have been described in detail above, but these are merely examples and do not limit the scope of the claims. The technology described in the claims includes various modifications and changes of the specific examples illustrated above.
For example, if a plurality of connection site side cameras can be rotatably mounted not only on an arm but also on a plurality of connection sites such as legs, or on different sites of one connection site, the imageable range is further expanded. .
Further, in the second embodiment, the stereo view with the trunk side camera as the main object is illustrated, but the stereo view is made possible by using a plurality of connection part side cameras, and the stereo image processing of each of the trunk side camera and the connection part side camera is performed. Equipped with a device, it can be used for more diverse applications.
The technical elements described in this specification or the drawings exhibit technical usefulness alone or in various combinations, and are not limited to the combinations described in the claims at the time of filing. In addition, the technology illustrated in the present specification or the drawings achieves a plurality of objects at the same time, and has technical utility by achieving one of the objects.

100: Robot 101: Robot mechanism 103: Trunk 104: Connection part (arm)
105: trunk-side camera 105 (a): trunk-side camera 105 (b): trunk-side camera 106: connection site-side camera 109: connection site-side camera position imaging direction calculation device 110: composite image creation device 111: body Stem side camera position imaging direction calculation device 112: Environmental map storage device 113: Movement path creation device 114: Control device 115: Actuator group 116: Image processing device 117: Joint angle detection device 119: Wheel rotation number detection device 121: Position information Detection device 125: Movement path correction device 200: Body 201: Head 202: Neck joint 203: Shoulder joint 204: Elbow joint 205: Wrist joint 206: Wheel

Claims (1)

The trunk,
A connection site connected to the trunk through a joint having a drive mechanism;
Two or more trunk side cameras provided on the trunk;
A connection site side camera provided in the connection site;
First calculation means for calculating a relative positional relationship of an object existing in the imaging range with respect to the mobile robot from two or more trunk-side images captured by two or more trunk-side cameras;
Calculating a relative positional relationship of an object existing in the imaging range with respect to the mobile robot from a trunk side image captured by at least one of the trunk side cameras and a connection site side image captured by the connection site side camera; 2 calculation means,
The mobile robot according to claim 2, wherein the second calculation means selects and uses a trunk-side image with the smallest range in which a connection site is imaged.

Images