JP2020030721A - Moving vehicle - Google Patents

Abstract

To provide a moving vehicle with which it is possible to improve the accuracy of present position using an imaged image acquired by an imaging unit mounted in a manipulator.SOLUTION: A moving vehicle 1 comprises: a manipulator 11; a manipulator control unit 12; an imaging unit 13 mounted in the manipulator 11; a position estimation unit 14 for estimating the position of the imaging unit 13 using an imaged image and a map; a present position acquisition unit 15 for acquiring the present position of the moving vehicle 1 from the estimated position; a movement mechanism 16; a movement control unit 17 for controlling the movement mechanism 16 using the present position; an acquisition unit 18 for acquiring suitability information that indicates whether the imaged image is suitable for position estimation; and an imaging direction control unit 19 for changing the imaging direction using suitability information so that an imaged image suitable for position estimation can be imaged. Thus, control can be exercised using the suitability information so that imaging suitable for position estimation is performance, and the accuracy of present position can be improved.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a moving body that obtains a current position using a captured image captured by a capturing unit mounted on a manipulator.

  2. Description of the Related Art Conventionally, a mobile robot having a manipulator provided with an imaging device is known (see Patent Document 1). The mobile robot can travel by itself using a travel information mark captured by an imaging device of the manipulator.

Japanese Utility Model Publication No. 60-066486

  However, in a moving body that autonomously moves using a captured image, there is a problem that accurate movement cannot be realized unless information used for controlling the movement is appropriately captured. For example, when the photographing is performed in backlight, there is a problem that it may not be possible to perform appropriate movement.

  The present invention has been made to solve the above problems, and it is an object of the present invention to improve the accuracy of a current position in a moving body that obtains a current position using a captured image obtained by a capturing unit mounted on a manipulator. With the goal.

In order to achieve the above object, a moving body according to the present invention is a moving body that moves autonomously, a manipulator, a photographing unit that is mounted on the manipulator, and acquires a photographed image around the moving body, and a photographed image. A position estimating unit for estimating the position of the photographing unit using the map in the moving area of the moving object, and a current position acquisition for acquiring the current position of the moving object using the position of the photographing unit estimated by the position estimating unit Unit, a moving mechanism that moves the moving body, using the current position, a movement control unit that controls the moving mechanism, and an acquisition unit that acquires matching information indicating whether the captured image is suitable for estimating the position, A manipulator control unit that controls the manipulator, and an image capturing direction that is changed by the manipulator control unit using the matching information so that a captured image suitable for position estimation is captured. Those with a direction control unit.
With such a configuration, it is possible to obtain a captured image that is more suitable for estimating the position by controlling the imaging direction using the matching information, and as a result, to improve the accuracy of the current position. Will be able to

Further, in the moving object according to the present invention, the adaptation information includes a representative value of the luminance of the captured image, and the capturing direction control unit determines that the representative value is larger when the representative value of the luminance of the captured image is larger than a predetermined threshold. The shooting direction may be changed so as to be smaller.
With such a configuration, for example, when the exposure of the captured image is over, and it is not possible to appropriately acquire a feature point for position estimation, the imaging direction is changed so as to obtain a more appropriate exposure. By doing so, more accurate position estimation can be performed.

Further, in the moving body according to the present invention, the adaptation information includes turning information indicating whether or not the movement immediately after the movement by the movement mechanism is a turn. When it is indicated that is turning, the shooting direction may be changed so that the optical axis of the shooting unit approaches the turning axis direction.
With such a configuration, when the moving body turns, for example, the photographing unit can photograph the upper side and the lower side. In the position estimation using the captured image, tracking may not be performed at the time of turning. In this way, by changing the shooting direction so as to perform upward or downward shooting, the position estimation at the time of turning can be performed. Tracking can be performed appropriately.

Further, in the moving object according to the present invention, the matching information includes ratio information indicating a ratio of matching between the feature points extracted from the captured image and the feature points included in the map. When the ratio indicated by the information is smaller than a predetermined threshold, the shooting direction may be changed so that the ratio becomes larger.
With such a configuration, for example, in a case where it is not possible to perform appropriate position estimation due to an obstacle or the like temporarily existing around the moving body, an image is captured while avoiding such an obstacle or the like. In this case, the photographing direction can be changed, and more accurate position estimation can be realized.

Further, in the moving body according to the present invention, the imaging direction control unit may change the imaging direction so that the position of the imaging unit is higher when the ratio indicated by the ratio information is smaller than a predetermined threshold.
With such a configuration, by changing the shooting direction so as to shoot from a higher place, the possibility of realizing shooting avoiding an obstacle or the like increases.

  ADVANTAGE OF THE INVENTION According to the moving body by this invention, it becomes possible to acquire the captured image which is more suitable for position estimation, and it is possible to improve the accuracy of the current position.

FIG. 2 is a schematic diagram illustrating an appearance of a moving object according to an embodiment of the present invention. FIG. 2 is a block diagram showing a configuration of a moving object according to the embodiment. 5 is a flowchart showing the operation of the moving object according to the embodiment. FIG. 4 is a diagram for explaining turning of the moving body according to the embodiment. FIG. 7 is a diagram for explaining matching between a feature point of a map and a feature point of a captured image according to the embodiment. FIG. 9 is a diagram for explaining a specific example of the operation of the moving object according to the embodiment.

  Hereinafter, a moving object according to the present invention will be described using embodiments. Note that, in the following embodiments, components and steps denoted by the same reference numerals are the same or equivalent, and repeated description may be omitted. The moving object according to the present embodiment performs position estimation using a photographed image acquired by a photographing unit mounted on a manipulator, and uses matching information indicating whether the photographed image is suitable for position estimation. Thus, the photographing direction is controlled.

  FIG. 1 is a schematic diagram illustrating an appearance of a moving body 1 according to the present embodiment, and FIG. 2 is a block diagram illustrating a functional configuration of the moving body 1. The mobile object 1 according to the present embodiment is a mobile object that autonomously moves, and includes a manipulator 11, a manipulator control unit 12, a photographing unit 13, a position estimating unit 14, a current position acquiring unit 15, a moving mechanism, 16, a movement control unit 17, an acquisition unit 18, and a shooting direction control unit 19. It should be noted that the mobile body 1 autonomously moves may not mean that the mobile body 1 moves in response to an operation instruction received from a user or the like, but may move to a destination according to its own judgment. The destination may be, for example, manually determined or automatically determined. Further, the movement to the destination may be performed, for example, along a movement route, or may not be performed. To move to the destination by own judgment may mean, for example, moving to the destination by the mobile unit 1 making its own judgment on the traveling direction, movement, stop, and the like. Further, for example, the moving body 1 may move so as not to collide with an obstacle. The moving body 1 may be, for example, a cart or a moving robot. The robot may be, for example, an entertainment robot, may be a monitoring robot, may be a transport robot, may be a cleaning robot, or may be a robot that shoots moving images and still images. , A welding robot, a painting robot, a pesticide spraying robot, an assembling robot, or another robot.

  The manipulator 11 may have a plurality of arms connected by a joint driven by a motor. At the tip of the manipulator 11, a hand effector (end effector) may be provided according to operations performed by the manipulator 11, for example, welding, gripping, painting, and assembling of an object to be conveyed. The manipulator 11 may be, for example, a manipulator of a vertical articulated robot or a manipulator of a horizontal articulated robot. Further, the number of axes of the manipulator 11 does not matter. As shown in FIG. 1, the base end side of the manipulator 11 may be fixed to the vehicle body 1 a of the moving body 1. In addition, the manipulator 11 may have an axis on the base end side that can turn the entire manipulator 11.

  The manipulator control unit 12 controls the manipulator 11. The manipulator control unit 12 may, for example, control the position and orientation of the hand of the manipulator 11, or may control the hand effector of the manipulator 11. The manipulator control unit 12 may control the manipulator 11 via, for example, a servo controller that controls each motor of the manipulator 11.

  The photographing unit 13 acquires a photographed image around the moving body 1. The imaging unit 13 is mounted on the manipulator 11. The position of the manipulator 11 on which the imaging unit 13 is mounted does not matter, but the imaging unit 13 may be mounted on the tip of the manipulator 11, for example. In the manipulator 11, it is assumed that the position where the photographing unit 13 is mounted is known. The imaging unit 13 may be mounted on the manipulator 11 so as to image the front side of the hand effector, for example. By doing so, the photographed image acquired by the photographing unit 13 can be used even when working with the hand effector. As described above, since the imaging unit 13 is mounted on the manipulator 11, the imaging direction is changed in accordance with the change in the position or the posture of the manipulator 11. The imaging unit 13 may acquire a captured image using an image sensor such as a CCD image sensor or a CMOS image sensor. Further, the photographing unit 13 may include an optical system such as a lens for forming an image on the image sensor. Further, the imaging unit 13 may be a single eye or a binocular (stereo camera). In the present embodiment, a case where the imaging unit 13 is a single eye will be mainly described. The photographing unit 13 normally photographs a moving image, that is, acquires continuous image frames. Since a captured image is obtained in order to obtain the current position of the moving moving body 1, it is preferable that the frame rate be sufficiently large with respect to the moving speed. For example, the frame rate may be about 30 fps or the like. In addition, the imaging unit 13 may acquire, for example, a captured image in the traveling direction, unless the imaging direction control unit 19 controls the imaging direction according to the adaptation information described later. Alternatively, a captured image in a direction transverse to the traveling direction may be acquired.

  The position estimating unit 14 estimates the position of the photographing unit 13 using the photographed image acquired by the photographing unit 13 and the map in the moving area of the moving body 1. By this position estimation, the position of the imaging unit 13 in the global coordinate system is estimated. This position may include an orientation. The position estimating unit 14 may extract a feature point from a captured image and perform position estimation by matching the extracted feature point with a feature point included in the map. The map is also called an environment map, and may be stored in, for example, a recording medium (not shown). The characteristic point may be, for example, a FAST key point, a SIFT key point, a SURF key point, or another characteristic point used in position estimation using a captured image. The matching may be performed by calculating a similarity between a feature descriptor of a feature point specified in the captured image and a feature descriptor of a feature point included in the map. Since the position estimation using such a captured image is already known as visual-SLAM (Simultaneous Localization and Mapping), a detailed description thereof will be omitted. Note that the position estimating unit 14 may or may not update the map using the feature points specified in the captured image. In the present embodiment, a case where the position estimating unit 14 performs position estimation using visual-SLAM, that is, a case where the map is updated will be mainly described. When the visual-SLAM is the ORB-SLAM, the feature points included in the map may be, for example, a map point or a feature point included in a keyframe. For the FAST key point, SIFT key point, SURF key point, feature point in ORB-SLAM, position estimation (tracking) and map update (local mapping) in ORB-SLAM, refer to the following documents 1 to 5, respectively. I want to be.

Reference 1: Edward Rosten, Tom Drummond, "Machine learning for high-speed corner detection", in 9th European Conference on Computer Vision, vol. 1, 2006, pp. 430-443.
Reference 2: D. Lowe, "Distinctive image features from scale-invariant keypoints", Int. Journal of Computer Vision (2004), 60 (2), pp. 91-110.
Reference 3: H. Bay, A. Ess, T. Tuytelaars, L. Van Gool, "SURF: Speeded Up Robust Features", Computer Vision and Image Understanding (CVIU) (2008), 110 (3), pp. 346- 359.
Reference 4: Ethan Rublee, Vincent Rabaud, Kurt Konolige, Gary R. Bradski, "ORB: An efficient alternative to SIFT or SURF", ICCV 2011, pp. 2564-2571.
Reference 5: R. Mur-Artal, JM Montiel, JD Tardos, "ORB-SLAM: A Versatile and Accurate Monocular SLAM System", IEEE Transactions on Robotics 31 (2015), pp. 1147-1163.

  The current position acquisition unit 15 acquires the current position of the mobile unit 1 using the position of the imaging unit 13 estimated by the position estimation unit 14. The angle at each joint of the manipulator 11 is managed by the manipulator control unit 12, and the mounting position of the imaging unit 13 in the manipulator 11 is known. The position of the part 13 can be known. The position of the imaging unit 13 may be, for example, the position of the moving body 1 in the local coordinate system. Therefore, using the position of the imaging unit 13 in the global coordinate system acquired by the position estimating unit 14 and the position of the imaging unit 13 in the moving object 1, the current position acquisition unit 15 uses the position of the moving object 1 in the global coordinate system. You can get the current location. The current position may be, for example, a position in a global coordinate system of a predetermined position (for example, a position of a center of gravity, a position predetermined by a designer, or the like) of the moving body 1. This current position may also include the orientation.

  The moving mechanism 16 moves the moving body 1. The moving mechanism 16 may or may not be able to move the moving body 1 in all directions, for example. To be able to move in all directions means to be able to move in any direction. The moving mechanism 16 may include, for example, a traveling unit (for example, wheels) and a driving unit (for example, a motor or an engine) for driving the traveling unit. The moving mechanism 16 may include a mechanism that can acquire the speed of a wheel or the like, for example, an encoder or the like. When the moving mechanism 16 can move the moving body 1 in all directions, the traveling unit may be an omni-directional moving wheel (for example, an omni wheel, a mecanum wheel, or the like). Since a known mechanism can be used as the moving mechanism 16, its detailed description is omitted.

  The movement control unit 17 controls the movement of the moving body 1 by controlling the movement mechanism 16 using the current position acquired by the current position acquisition unit 15. The control of the movement may be control such as the direction of movement of the moving body 1 or the start / stop of the movement. For example, when a movement path is set, the movement control unit 17 may control the movement mechanism 16 so that the moving body 1 moves along the movement path. More specifically, the movement control unit 17 may control the movement mechanism 16 so that the current position acquired by the current position acquisition unit 15 is along the movement route. Further, the movement control unit 17 may control the movement using a map. Since the control of the movement mechanism 16 by the movement control unit 17 is known, detailed description thereof is omitted.

  The acquiring unit 18 acquires matching information indicating whether the captured image is suitable for estimating a position. Whether or not a captured image is suitable for position estimation may be considered to mean whether or not a more accurate position can be estimated using the captured image. The adaptation information may be acquired according to, for example, a captured image, the content of the movement control immediately after the moving object 1, a matching state between a feature point of the captured image and a feature point of the map, as described later. .

  For example, when overexposure occurs in a captured image, that is, when overexposure occurs in the captured image, it becomes difficult to identify feature points in the captured image. Would not be suitable for Therefore, the matching information may include a representative value of the luminance of the captured image. The representative value may be, for example, an average value, a median value, a minimum value, or the like. The acquiring unit 18 may acquire, for example, a luminance value of each pixel included in the captured image, and acquire a representative value of the acquired plural luminance values. If the representative value of the brightness value of the captured image is larger than a predetermined threshold, it may be determined that the captured image is not suitable for estimating the position. This is because the captured image is considered to be overexposed.

  Further, for example, if the mobile unit 1 is turning and shooting in the horizontal direction is performed, it may not be possible to perform appropriate tracking in position estimation using a shot image. For this reason, a captured image in the horizontal direction captured when the moving body 1 makes a turn is not suitable for position estimation. Therefore, the adaptation information may include turning information indicating whether or not the immediately subsequent movement by the moving mechanism 16 is turning. The turning information may be flag-like information indicating whether or not the immediately following movement is a turning, and detailed information on the movement for determining whether or not the immediately following movement is a turning, for example, the traveling direction. It may be information indicating the speed of change (angular velocity) of the change in the moving direction or the degree of change (angle, etc.) in the traveling direction.

  Specifically, the acquisition unit 18 determines that the moving direction of the moving object 1 is determined within a predetermined time interval (for example, 1 second or 3 seconds) by a predetermined threshold angle (for example, 1 second or 3 seconds). , 80 degrees, 90 degrees, 120 degrees, 150 degrees, etc.) or more, it may be determined that the future movement is a turn, and turn information indicating that the immediately subsequent movement is a turn may be obtained. .

  FIGS. 4A and 4B are diagrams showing changes in the traveling direction of the moving body 1 when viewed from above. For example, when the future traveling direction of the moving body 1 sequentially changes as indicated by arrows A1 to A5 shown in FIG. 4A, the acquisition unit 18 determines that the movement immediately after the moving mechanism 16 is turning. May be obtained. In addition, when such a turn is performed within a predetermined time interval, it may be determined that the future movement is a turn. Also, FIG. 4A shows a turn having a turning radius of 0, but as shown in FIG. 4B, even if the turning has a turning radius other than 0, the above condition is satisfied. In this case, it may be determined that the future movement is a turn. If the traveling direction changes by a predetermined angle or more within the predetermined time interval, tracking in position estimation using a horizontal image becomes difficult regardless of the turning radius. Note that FIG. 4B shows that the moving body 1 moves along the route R1 and the traveling direction changes sequentially as indicated by arrows A6 to A8. Further, the acquisition unit 18 may acquire the traveling direction and the like in the future movement from the movement control unit 17, for example, or may calculate using the movement route, the current position, and the like. In addition, since the movement control unit 17 performs the movement control according to the current position, the movement control unit 17 may have only information on the movement in the latest predetermined period. In such a case, the acquisition unit 18 determines that the future movement is a turn if the change in the angle of the future traveling direction is greater than or equal to the threshold angle in the information held by the movement control unit 17. May be. When the turning information indicates that the turning is performed immediately afterward, it may be determined that the captured image is not suitable for estimating the position. This is because a horizontally photographed image photographed at the time of turning is not suitable for tracking.

  Further, for example, when there is an obstacle such as a human between the object corresponding to the feature point on the map used in the position estimation and the imaging unit 13, the feature point extracted from the captured image and the map , The degree of matching with the feature points included in the image is reduced, and as a result, the accuracy of position estimation is reduced. Therefore, the matching information may include ratio information indicating the ratio of matching between the feature points extracted from the captured image and the feature points included in the map. FIG. 5 is a diagram illustrating a matching relationship between a feature point extracted from a captured image and a feature point included in a map.

  For example, as shown in FIG. 5A, it is assumed that feature points A to D are included in the map, and feature points a to d are specified in the captured image. In addition, the feature points a to d of the captured image are changed to the feature points A to D of the map by matching using the feature amount (feature descriptor) between the feature point specified in the captured image and the feature point included in the map. Suppose that it corresponds to D. The matching may be performed by the position estimating unit 14 for position estimation by visual-SLAM, for example. In the case of FIG. 5A, four feature points are extracted from the captured image, and all the feature points match feature points included in the map, so that the ratio information is 1 or 100%. .

On the other hand, as shown in FIG. 5B, it is assumed that feature points A to D are included in the map, and feature points a and d are specified in the captured image. In addition, the feature points a and d of the captured image correspond to the feature points A and D of the map, respectively, by matching using the feature amount of the feature point specified in the captured image and the feature point included in the map. Let's say In FIG. 5B, the map also includes feature points B and C. Therefore, it is assumed that the feature points b and c corresponding to the feature points B and C of the map should be originally extracted from the captured image, but the extraction is not performed. Then, in the captured image, the feature points b and c are feature points of extraction omission. For example, when a feature point having a feature amount similar to the feature amount of the feature point B is not extracted near the position where the feature point B included in the map is projected on the captured image, the feature point B is extracted. It may be determined that there is a leak. In this case, the ratio information may be calculated by the following equation. In the case of percentage, the value of the following equation may be multiplied by 100. In the case of FIG. 5B, two feature points matching the map feature points are extracted from the captured image, and two feature points that are not extracted exist, so that the ratio information is 0.5 or 50. %.
Ratio information = N1 / (N1 + N2)
Here, N1 is the number of feature points that match the feature points included in the map among the feature points extracted from the captured image, and N2 is the number of feature points that have not been extracted for the captured image.

In addition, as shown in FIG. 5C, when there is a feature point of extraction omission, other feature points e and f may be extracted from the captured image. For example, in the ORB-SLAM, since the number of feature points (FAST key points) extracted from a captured image is determined in advance, feature points are extracted until the number of feature points reaches the determined number. In such a case, for example, if the feature points e and f are obtained from an obstacle such as a person existing between the object corresponding to the feature points B and C on the map and the image capturing unit 13, The feature points on the map corresponding to the feature points e and f of the image do not exist. In such a case, the feature points a and d match the feature points A and D on the map, but the feature points e and f do not match the feature points on the map. In this case, the ratio information may be calculated by the following equation. In the case of percentage, the value of the following equation may be multiplied by 100. In the case of FIG. 5C, four feature points are extracted from the captured image, and two feature points match feature points on the map, so that the ratio information is 0.5 or 50%.
Ratio information = N1 / N3
Here, N1 is as described above, and N3 is the number of feature points extracted from the captured image.

  As described above, the acquisition unit 18 determines whether the number of feature points extracted from the captured image (N3) or the number of feature points extracted from the captured image that match the feature points included in the map (N1). ), The ratio information may be obtained by using the number (N2) of the feature points of the extraction omission regarding the captured image. The ratio information may be calculated by any of the above two equations, or may be calculated by another equation. When the ratio indicated by the ratio information is smaller than a predetermined threshold, it may be determined that the captured image is not suitable for estimating the position. This is because there is a possibility that an obstacle that is not included in the map exists in front of the photographing unit 13 in the photographing direction.

  Here, the case where the ratio information decreases due to the presence of a movable obstacle such as a person in the moving area of the moving body 1 has been described. However, the decrease in the ratio information is caused by other factors, for example, It can also be caused by backlit photography. Therefore, by controlling the photographing direction using the ratio information, the influence of backlight or the like can be reduced.

  The photographing direction control unit 19 changes the photographing direction of the photographing unit 13 using the matching information acquired by the acquiring unit 18 so that a photographed image suitable for position estimation is photographed. The change of the imaging direction is performed by the imaging direction control unit 19 controlling the manipulator control unit 12. When performing control to change the imaging direction, for example, the imaging direction control unit 19 determines the imaging direction after the imaging unit 13 is changed, and the position of the manipulator 11 to which the imaging unit 13 is mounted is determined by the imaging direction. The manipulator control unit 12 may be instructed to be in the direction. For example, the manipulator control unit 12 calculates the angle of each joint so that the location of the manipulator 11 on which the imaging unit 13 is mounted is in a predetermined imaging direction by inverse kinematics, and each joint calculates the angle. The manipulator 11 may be controlled so that the angle is set as described above. The change in the imaging direction may be, for example, a change in the direction of the optical axis of the imaging unit 13, or a change in the position of the imaging unit 13, such as when the optical axis of the imaging unit 13 is translated. Is also good.

  When the matching information includes the representative value of the brightness of the captured image, the shooting direction control unit 19 performs shooting such that the representative value of the brightness of the captured image becomes smaller when the representative value of the brightness of the captured image is larger than a predetermined threshold value. The direction may be changed. When the representative value of the brightness of the captured image is smaller than the predetermined threshold, the capturing direction control unit 19 does not need to control the capturing direction according to the representative value of the luminance of the captured image. When the representative value of the brightness of the captured image is the same as the predetermined threshold, the capturing direction control unit 19 may or may not control the capturing direction. The change of the photographing direction may be, for example, a predetermined change or a change by trial and error for reducing the representative value of the luminance included in the adaptation information. In the latter case, for example, the imaging direction control unit 19 changes the imaging direction randomly or according to a predetermined rule, and as a result, when the representative value of the luminance becomes smaller than the threshold value, changes the imaging direction. It may end. Further, the predetermined change may be to change the shooting direction in the horizontal plane by turning the manipulator 11. The angle of the turn may be set to, for example, 60 degrees or 90 degrees.

  When the turning information is included in the matching information, the imaging direction control unit 19 sets the optical axis of the imaging unit 13 to the turning axis when the turning information indicates that the immediately subsequent movement by the moving mechanism 16 is turning. The shooting direction may be changed so as to approach the direction. Note that when the turning information does not indicate that the immediately subsequent movement is a turning, the shooting direction control unit 19 does not need to control the shooting direction according to the turning information. Here, when the turning information is flag-like information indicating whether or not the immediately following movement is a turning, the imaging direction control unit 19 determines whether or not the immediately following movement is a turning based on the turning information. be able to. On the other hand, when the turning information is information indicating the speed of the change in the traveling direction or the degree of the change in the traveling direction, the photographing direction control unit 19 uses the turning information to determine whether the immediately subsequent movement is a turning. It may be determined whether or not, and in the case of a turn, the shooting direction may be changed so that the optical axis of the shooting unit 13 approaches the turning axis direction. Specifically, the imaging direction control unit 19 may determine that the immediately subsequent movement is a turn when the speed of the change in the traveling direction is greater than the threshold and the angle of the change in the traveling direction is greater than the threshold. Good. Since the turning axis direction is usually a vertical direction, the photographing direction control unit 19 changes the photographing direction such that the photographing unit 13 photographs the upper side, the lower side, etc. Alternatively, the photographing direction may be changed so as to photograph obliquely upward or obliquely downward. Above and below are above and below in the vertical direction. Therefore, when the photographing direction is changed so that the photographing unit 13 photographs the upper side, the optical axis of the photographing unit 13 may be substantially vertical, and the ceiling or the like of the moving space may be photographed. Since the turning is usually performed in a horizontal plane, when the optical axis of the imaging unit 13 is substantially vertical at the time of the turning, the optical axis of the imaging unit 13 substantially coincides with the turning axis direction. . Therefore, only the angle of the captured image changes during turning, and as a result, tracking of feature points becomes easier. Further, the change of the shooting direction according to the turning information may be performed when the shooting direction at that time is the horizontal direction. This is because if the shooting direction at that time is close to the vertical direction, there is no need to change the shooting direction, even if the movement immediately after is turning.

  When the matching information includes the ratio information, the shooting direction control unit 19 may change the shooting direction so that the ratio becomes larger when the ratio indicated by the ratio information is smaller than a predetermined threshold. When the ratio indicated by the ratio information is larger than the predetermined threshold, the imaging direction control unit 19 does not need to control the imaging direction according to the ratio information. When the ratio indicated by the ratio information is the same as the predetermined threshold, the imaging direction control unit 19 may or may not control the imaging direction. The change of the shooting direction may be, for example, a predetermined change or a trial and error change to increase the ratio indicated by the ratio information included in the adaptation information. In the latter case, for example, the shooting direction control unit 19 changes the shooting direction at random or according to a predetermined rule, and ends the change of the shooting direction when the ratio information becomes larger than the threshold value. You may. Further, the predetermined change may be to change the shooting direction in the horizontal plane by turning the manipulator 11. The angle of the turn may be set to, for example, 60 degrees or 90 degrees. Further, the predetermined change may be to change the shooting direction so that the position of the shooting unit 13 is higher. When an object corresponding to a feature point included in the map cannot be photographed due to an obstacle, an object existing on the other side of the obstacle can be photographed by photographing from a position higher than the obstacle. Therefore, it is considered that raising the shooting position is preferable. When the photographing direction is changed so that the position of the photographing unit 13 is increased, control is performed so that the azimuth direction of photographing is maintained, that is, the optical axis of the photographing unit 13 is translated. You may. By raising the position of the photographing unit 13 so that the optical axis moves in parallel, for example, when the photographing unit 13 is photographing the traveling direction, the obstacle is kept in the state where the photographing in the traveling direction is continued. Can be reduced.

  When the shooting direction is changed by the shooting direction control unit 19, the shooting angle or the shooting position may be changed at a speed that can achieve appropriate tracking. By doing so, it is possible to keep track of the self-position when changing the shooting direction.

  Further, after the photographing direction is changed according to the adaptation information, the photographing direction may be returned to the original one after a predetermined period elapses. For example, when the photographing direction is changed in accordance with a large representative value of the brightness of the photographed image or a small ratio indicated by the ratio information, the photographing direction control unit 19 performs the photographing after a predetermined period has elapsed. The direction may be restored. Further, for example, when the shooting direction is changed in accordance with the fact that the immediately following movement is turning, the shooting direction control unit 19 indicates that the immediately following movement is not turning by the turning information included in the matching information. After that, the shooting direction may be returned to the original.

Next, the operation of the mobile unit 1 will be described with reference to the flowchart of FIG.
(Step S101) The current position acquisition unit 15 determines whether to acquire the current position. If the current position is to be obtained, the process proceeds to step S102; otherwise, the process proceeds to step S105. For example, the current position acquisition unit 15 may periodically determine that the current position is to be acquired.

  (Step S102) The photographing unit 13 acquires a photographed image around the moving body 1 in the photographing direction at that time. Note that the acquired captured image may be stored in a recording medium (not shown).

  (Step S103) The position estimating unit 14 extracts a feature point from the acquired captured image, and estimates the position of the imaging unit 13 using the extracted feature point and the feature point of the map. The position estimating unit 14 may update the map using the feature points extracted from the captured image. Note that, for example, the position may be estimated each time a captured image is acquired, and the map may be updated each time the captured image is acquired several times. Therefore, for example, when it is determined that the map is to be updated, the position estimating unit 14 may update the map using the extracted feature points. Note that the estimated position may be stored on a recording medium (not shown).

  (Step S104) The current position acquisition unit 15 acquires the current position of the mobile unit 1 using the angles of the axes of the manipulator 11 and the position of the imaging unit 13 acquired by the position estimation unit 14. The acquired current position may be stored on a recording medium (not shown). Then, the process returns to step S101.

  (Step S105) The acquiring unit 18 determines whether to acquire the conformance information. Then, when the matching information is obtained, the process proceeds to step S106, and otherwise, the process returns to step S101. The acquisition unit 18 may periodically determine, for example, to acquire the matching information. Usually, the time interval of the process of acquiring the matching information is longer than the time interval of the process of acquiring the current position.

  (Step S106) The acquisition unit 18 acquires the conformance information. For example, the obtaining unit 18 may obtain the representative value of the luminance value using the captured image, and obtains the turning information by referring to the contents of the future movement control managed by the movement control unit 17. Alternatively, the ratio information may be obtained by referring to a result of matching between the captured image and the map by the position estimating unit 14. The acquired matching information may be stored on a recording medium (not shown).

  (Step S107) Using the matching information acquired in step S106, the imaging direction control unit 19 determines whether control of the imaging direction is necessary. If it is necessary to control the photographing direction, the process proceeds to step S108; otherwise, the process returns to step S101. Note that, for example, when the matching information indicates that the representative value of the brightness of the captured image is large, or when the turning information indicates that the immediately subsequent movement is turning, If this indicates that the ratio of matching between the feature points of the captured image and the feature points of the map is low, it may be determined that the control of the shooting direction is necessary.

  (Step S108) The photographing direction control unit 19 changes the photographing direction of the photographing unit 13 by controlling the manipulator control unit 12. Specifically, when it is indicated that the representative value of the brightness of the captured image is large, the capturing direction control unit 19 changes the capturing direction so as to reduce the representative value by turning the manipulator 11 or the like. May be. When the movement immediately after is turning, the photographing direction control unit 19 may change the photographing direction so that the photographing unit 13 photographs above the ceiling or the like. When the ratio of matching between the feature points of the captured image and the feature points of the map is low, the capturing direction control unit 19 may change the capturing direction and the like so that the capturing position is increased. Then, the process returns to step S101.

  Although not included in the flowchart in FIG. 3, it is assumed that the movement control using the current position acquired by the current position acquisition unit 15 is performed by the movement control unit 17. In addition, when the manipulator 11 completes the movement to the position where the work is performed, the manipulator 11 is controlled by the manipulator control unit 12, so that a predetermined work, for example, welding, painting, assembling, or the like is performed. Good.

  In the flowchart of FIG. 3, various pieces of information are included in the matching information, and after the matching information is collectively acquired, the necessity determination of the control of the imaging direction according to the matching information is collectively performed. Although the case has been described, this need not be the case. The acquisition of the information included in the adaptation information and the determination of the necessity of controlling the imaging direction according to the acquired information may be repeatedly performed. In this case, for example, first, it is necessary to obtain the turning information and determine whether or not the control of the shooting direction is necessary, and when the control is not necessary, obtain the representative value of the brightness of the shot image, and It is also possible to determine the necessity of the control of the shooting direction, and when the control is unnecessary, obtain the ratio information and determine whether the control of the shooting direction is necessary according to the information. When it is determined that the control is necessary in any of the necessity determinations of the control of the photographing direction, the control determined to be necessary may be performed, and the process may return to step S101.

  If the shooting direction is to be returned to the original one after a predetermined period after the shooting direction is changed according to the adaptation information, it is determined as No in step S107, and the shooting direction is returned. Direction control unit 19 may perform control to return the shooting direction to the original direction when the condition for the image is satisfied. The order of the processes in the flowchart of FIG. 3 is an example, and the order of each step may be changed as long as a similar result is obtained. In addition, in the flowchart of FIG. 3, the processing is terminated by powering off or interruption for terminating the processing.

  Next, the operation of the moving body 1 according to the present embodiment will be described using a specific example. In this specific example, it is assumed that the moving body 1 moves in the order of the positions P1 to P6 along the route R2 shown in FIG. Further, the shooting direction when the moving body 1 is moving at each of the positions P1 to P6 is indicated by an arrow. In this specific example, it is assumed that the shooting in the traveling direction is performed except when the control of the shooting direction according to the matching information is performed.

  First, if it is time to acquire the current position while the moving body 1 is moving at the position P1, a shooting direction that matches the traveling direction indicated by the arrow, that is, an upward shooting in FIG. The position of the photographing unit 13 is estimated using the photographed image and the map, and the current position of the moving body 1 is obtained using the position estimation result (steps S101 to S104). The current position is used for movement control by the movement control unit 17. After that, the photographing, the position estimation, and the acquisition of the current position are repeated.

  When it is time to acquire the adaptation information, the acquiring unit 18 acquires the representative value of the luminance using the latest captured image at that time, and updates the contents of the future movement control managed by the movement control unit 17. The turning information is acquired by referring to the information, and the ratio information is acquired using the matching result between the latest captured image and the map (steps S105 and S106). Note that the representative value of the luminance of the captured image acquired at the position P1 is smaller than a predetermined threshold, the turning information indicates that the immediately subsequent movement is not a turning, and the ratio indicated by the ratio information is a predetermined ratio. It is assumed that the value is larger than the threshold. Then, the shooting direction control unit 19 determines that the control of the shooting direction is not necessary, and the control of the shooting direction is not performed (step S107). As a result, shooting in the traveling direction is continued.

  Thereafter, when the moving body 1 reaches the position P2, the exposure of the captured image is overexposed due to external light incident from the window existing on the left side in the traveling direction, and the representative of the luminance of the captured image acquired at that time. It is assumed that the value is larger than a predetermined threshold (steps S105 and S106). Then, the imaging direction control unit 19 determines that the imaging direction needs to be controlled, and turns the imaging direction by 90 degrees via the manipulator control unit 12 (steps S107 and S108). Due to the turning, the photographing direction becomes rightward as shown in FIG. As a result, the external light incident on the photographing unit 13 changes from the backlight to the normal light, and the photographing with the proper exposure can be performed. After a lapse of a predetermined time, as shown by a position P3 in FIG. 6, the photographing direction control unit 19 returns the photographing direction to the traveling direction which is the standard setting. Assuming that the range of the window is wider than that in FIG. 6 and the external light is also affected at the position P3, it is determined that the representative value of the brightness of the captured image is large again after the shooting direction is returned to the standard setting, Accordingly, the shooting direction is changed to the right in the figure.

  When the moving body 1 reaches the position P4, the movement control unit 17 controls the moving mechanism 16 so that the moving body 1 turns by 90 degrees. In this specific example, it is assumed that a change in the moving direction of 80 degrees or more is a setting that is determined as turning. Then, by referring to the content of the movement control, the acquiring unit 18 acquires turning information indicating that the immediately following movement is turning (step S106). In accordance with the turning information, the photographing direction control unit 19 determines that control of the photographing direction is necessary, and controls the manipulator control unit 12 so that the photographing unit 13 photographs vertically upward (steps S107 and S108). ). Therefore, the position estimating unit 14 can perform appropriate tracking even when the turning is performed, can appropriately obtain the turning angle, and can keep track of its own position. When the turning is completed, turning information indicating that the immediately following movement is not a turning is obtained. Therefore, according to the turning information, the photographing direction control unit 19 determines the traveling direction after the turning, that is, the rightward direction in the drawing. The manipulator control unit 12 is controlled so that photographing is performed.

  Thereafter, it is assumed that the human M has walked from the front while the moving body 1 is moving at the position P5. In this situation, the human M is present between the imaging unit 13 and the object corresponding to the feature point included in the map, and the matching ratio between the feature point of the captured image and the feature point of the map is reduced. And Then, the acquiring unit 18 acquires the ratio information in which the ratio of matching has decreased (steps S105 and S106). If the ratio indicated by the ratio information is smaller than a predetermined threshold, the imaging direction control unit 19 determines that control of the imaging direction is necessary, and causes the manipulator control unit 12 to increase the position of the imaging unit 13. Control. As a result, since the image is taken from a high place, the influence of the person M is reduced, and the ratio of matching between the characteristic points of the photographed image and the characteristic points of the map can be increased. Also in this case, when the predetermined time has elapsed and the moving body 1 reaches the position P6, the height of the imaging unit 13 is returned to the original height.

  As described above, according to mobile unit 1 of the present embodiment, by controlling the shooting direction using the matching information, it is possible to obtain a shot image that is more suitable for estimating the position. As a result, the accuracy of the current position is improved, and more accurate movement can be realized. In addition, since the representative value of the brightness of the captured image is included in the matching information, it is possible to control so that the image is captured with more appropriate exposure. In addition, since the turning information is included in the matching information, it is possible to perform shooting that can perform appropriate tracking even during turning, and to keep track of its own position. In addition, since the ratio information is included in the matching information, it is possible to prevent the matching ratio between the feature point of the captured image and the feature point of the map from decreasing.

  In the above-described embodiment, a case has been mainly described in which the adaptation information includes the representative value of the brightness of the captured image, the turning information, and the ratio information, but this is not essential. The matching information may include at least one or more of the three pieces of information. Further, the matching information may include other information, and the control of the shooting direction may be performed using the information.

  Further, in the above embodiment, each process or each function may be realized by being centrally processed by a single device or a single system, or may be realized by distributed processing by a plurality of devices or a plurality of systems. It may be realized by doing.

  Further, in the above-described embodiment, the information exchange performed between the components is performed by, for example, one of the components when the two components performing the information exchange are physically different from each other. It may be performed by outputting information and receiving information by the other component, or, when two components performing the transfer of the information are physically the same, one component may be used. May be performed by shifting from the processing phase corresponding to the above to the processing phase corresponding to the other component.

  Further, in the above embodiment, information related to the processing executed by each component, for example, information received, obtained, selected, generated, transmitted, or received by each component Also, information such as thresholds, mathematical expressions, addresses, and the like used by each component in the processing may be temporarily or long-term stored in a recording medium (not shown), even if not explicitly described in the above description. The storage of information on the recording medium (not shown) may be performed by each component or a storage unit (not shown). The reading of information from the recording medium (not shown) may be performed by each component or a reading unit (not shown).

  Further, in the above embodiment, when information used by each component and the like, for example, information such as a threshold and an address used by each component in processing and various setting values may be changed by a user, The user may or may not be able to change such information as appropriate, even if not explicitly stated in the description. When the information can be changed by the user, the change is realized by, for example, a receiving unit (not shown) that receives a change instruction from the user and a changing unit (not shown) that changes the information according to the change instruction. You may. The reception of the change instruction by the receiving unit (not shown) may be, for example, reception from an input device, reception of information transmitted via a communication line, or reception of information read from a predetermined recording medium. .

  Further, in the above embodiment, when two or more components included in the moving object 1 include a communication device, an input device, and the like, the two or more components may have a physically single device. Or may have separate devices.

  Further, in the above embodiment, each component may be configured by dedicated hardware, or a component that can be realized by software may be realized by executing a program. For example, each component can be realized by a program execution unit such as a CPU reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. At the time of execution, the program execution unit may execute the program while accessing the storage unit and the recording medium. Further, the program may be executed by being downloaded from a server or the like, or may be executed by reading a program recorded on a predetermined recording medium (for example, an optical disk, a magnetic disk, a semiconductor memory, or the like). Good. Further, this program may be used as a program constituting a program product. The computer that executes the program may be a single computer or multiple computers. That is, centralized processing or distributed processing may be performed.

  In addition, the present invention is not limited to the above-described embodiments, and various changes can be made, and it goes without saying that these are also included in the scope of the present invention.

  As described above, according to the moving object of the present invention, the effect of realizing a more accurate acquisition of the current position is obtained, and the moving object is useful as a moving object for performing position estimation using a captured image.

DESCRIPTION OF SYMBOLS 1 Moving body 11 Manipulator 12 Manipulator control part 13 Imaging part 14 Position estimation part 15 Current position acquisition part 16 Moving mechanism 17 Movement control part 18 Acquisition part 19 Imaging direction control part

Claims (5)

A mobile object that moves autonomously,
A manipulator,
A photographing unit attached to the manipulator and acquiring a photographed image around the moving body,
A position estimating unit that estimates the position of the imaging unit using the captured image and a map in a moving area of the moving body;
Using a position of the imaging unit estimated by the position estimating unit, a current position acquisition unit that acquires a current position of the moving body,
A moving mechanism for moving the moving body,
A movement control unit that controls the movement mechanism using the current position;
An acquisition unit that acquires matching information indicating whether the captured image is suitable for estimating a position,
A manipulator control unit for controlling the manipulator,
A moving direction control unit configured to change a photographing direction of the photographing unit by the manipulator control unit so that a photographed image suitable for estimating a position is photographed using the matching information. The matching information includes a representative value of the brightness of the captured image,
The moving body according to claim 1, wherein when the representative value of the brightness of the captured image is larger than a predetermined threshold, the capturing direction control unit changes the capturing direction so that the representative value becomes smaller. The adaptation information includes turning information indicating whether or not the movement immediately after the moving mechanism is turning,
The imaging direction control unit changes the imaging direction so that the optical axis of the imaging unit approaches the turning axis direction, when the turning information indicates that the movement immediately after the moving mechanism is turning, The moving object according to claim 1. The matching information includes characteristic information extracted from the captured image and ratio information indicating a ratio of matching between the characteristic points included in the map,
The moving body according to claim 1, wherein when the ratio indicated by the ratio information is smaller than a predetermined threshold, the shooting direction control unit changes the shooting direction such that the ratio becomes larger. The moving body according to claim 4, wherein when the ratio indicated by the ratio information is smaller than a predetermined threshold, the imaging direction control unit changes the imaging direction so that the position of the imaging unit is higher.

Images