JP2021103421A - Moving body and navigation system for the same - Google Patents

Abstract

To provide a navigation system which makes a moving body travel to its destination position accurately and is robust against an environmental change of a traveling condition for the moving body.SOLUTION: A moving body traveling autonomously comprises: an image acquisition part to take a present position image; a destination position image storage part to store a destination position image taken at a destination position; an approaching commanding part to generate a traveling command to approach the destination position based on an approaching commanding formula; an alignment commanding part to generate a traveling command to stop at the destination position or a traveling command to pass through the destination position, based on the alignment commanding formula; a travel switching part to select the approaching commanding part or the alignment commanding part, responding to the result of comparing the present position image or the destination position image; and a traveling control part to control a traveling mechanism based on the traveling command.SELECTED DRAWING: Figure 8

Description

The present invention relates to an autonomously moving mobile body and its navigation system, and in particular, a learning device learns and learns an approach movement command type and an alignment command type using data collected by a data collection moving body. The present invention relates to a system for generating a command for a moving body to approach or align with a target position by using an approach movement command formula or an alignment command formula.

In recent years, mobile bodies that carry, guide, and work while moving autonomously have begun to operate in many indoor facilities such as factories, hospitals, and public facilities. For navigation of such moving objects, a method of installing a guide system (indoor GPS, marking, etc.) on the environment side, or a map created from environmental information (shape, image, etc.) collected by moving the driving environment in advance. There is a method of moving to the target position while collating the environmental information currently being acquired with the landmark information on the map. However, advance preparations such as maintenance work on the environment side, creation work of a highly reliable map, and software development work such as collation processing with a map are a heavy load.

As a navigation technique for solving this problem, for example, claim 1 of Patent Document 1 states that "a teaching image captured in advance on a traveling path of a moving body provided with an imaging device and an actual image captured by the imaging device. It is a control method of a moving body that controls the running of the moving body so that the degree of coincidence between the real image and the teaching image becomes larger by comparing with the image, and is based on the teaching image and the real image. When the steady running step of running the moving body at a steady speed and the degree of coincidence between the deceleration start position image preset from the plurality of teaching images and the actual image reach a preset first threshold value. A low-speed traveling step in which the moving body is traveled at a deceleration speed smaller than the steady speed, and a degree of coincidence between a positioning start position image preset from the plurality of teaching images and the actual image are set in advance. When the two thresholds are reached, the positioning step of traveling the moving body at a positioning speed smaller than the deceleration speed, and the degree of coincidence between the target position image preset from the plurality of teaching images and the actual image are set in advance. A method for controlling a moving body, which comprises a stop step for stopping the driving device when the third threshold value is reached, is disclosed. According to the navigation technique disclosed in Patent Document 1, as a preliminary preparation, images at a plurality of positions up to the target stop position are acquired, and in particular, a plurality of images at positions where the vehicle decelerates in the traveling direction (front-back direction) are prepared. It is possible to decelerate in multiple stages and improve the accuracy of the stop position.

Japanese Unexamined Patent Publication No. 2010-140080

However, in the navigation technique of Patent Document 1, the degree of coincidence between the image (target position image / teaching image) captured in advance and the image (actual image) captured during traveling, and the position of the feature point of the teaching image and the actual image In that the left and right positions of the moving body are controlled by using the difference in the positions of the corresponding feature points, an environment with relatively small environmental changes (such as a factory corridor, a large number of objects, changes in arrangement, and the presence or absence of passersby) , Illumination brightness, etc.) and under the condition of linear movement near the stop target position, it can move to the target position with high accuracy, but it is applied to environments with large environmental changes such as shopping malls and public facilities and wide moving environments. In some cases, or when applied to a highly accurate movement method of the position and orientation (direction, orientation) of a moving body in a narrow environment, a moving body that can move in all directions (holonomic) or a moving body that cannot move sideways like a car ( When adapting to different types of autonomous moving objects such as non-holonomic), problems such as deterioration of movement accuracy, processing failure, and increase of preparatory load such as parameter tuning occur, and it is not possible to stop accurately at the target stop position. There is a possibility.

As a more specific example, in the technique described in Patent Document 1, since the moving body is controlled by calculating the degree of matching of images and comparing the feature points between images, the environment between the time of acquiring the teaching image and the time of acquiring the actual image Due to changes (arrangement of people and objects, lighting conditions, etc.), matching failures occur due to changes in the degree of matching and characteristic changes and disappearances of feature points (due to lighting, shadows, hiding, etc.), resulting in shopping malls and In a dynamic environment such as a public facility, there are problems such as deterioration of movement accuracy and control failure. In particular, when there is a distance between the moving object and surrounding objects, or when there is a distance to the target position during a wide range of movement, the change between the teaching image and the actual image is small with respect to the amount of movement of the moving object. This is noticeable in situations where a person or other object is likely to be imaged between a moving object and surrounding objects.

At the same time, the technique described in Patent Document 1 sequentially controls the speed of the moving body and the position in the left-right direction by calculating the degree of matching of the images and comparing the feature points between the images. In addition to the position, the accuracy of the orientation is required, such as when passing through a place (passing through the door), and the position and orientation (position and orientation) of the route to the target position are planned (such as a car garage). ) Is not possible, and there is a problem that high-precision position accuracy cannot be achieved. This problem is particularly remarkable in a nonholonomic moving body, and there is a problem that it cannot cope with various moving methods and moving bodies.

At the same time, in the technique described in Patent Document 1, it is necessary to acquire a teaching image at a deceleration position (control position) after performing a speed plan (running plan) in advance, and it is necessary to change the moving method and the moving body. There is a problem that the load of advance preparation increases every time.

The present invention has been made in view of such a background, and is a navigation device that does not require equipment construction or cartography and has a small load of advance preparation. In particular, robust movement and position in a dynamic environment and a wide mobile environment. The purpose is to realize high-precision movement in narrow places where posture route planning is required, and to provide a moving body navigation device with less preparatory load for various types of moving methods and autonomous moving bodies. And.

In order to solve the above-mentioned problems, the moving body of the present invention is based on an image acquisition unit that captures a current position image, a target position image storage unit that stores a target position image captured at a target position, and an approach movement command formula. An approaching movement command unit that generates a movement command that moves closer to the target position, and an alignment command unit that generates a movement command that stops at the target position or a movement command that passes through the target position based on the alignment command formula. The movement switching unit that selects either the approach movement command unit or the alignment command unit according to the comparison result between the current position image and the target position image, and the movement mechanism is controlled based on the movement command. A moving body including a moving control unit and a moving body.

According to the present invention, it is possible to provide a navigation device that does not require equipment construction or cartography and has a small load of advance preparation, and particularly requires robust movement in a dynamic environment and a wide mobile environment, and route planning of position and orientation. It is possible to realize high-precision movement in a place, and to provide a moving body navigation device with a small load of preparation for various types of moving methods and autonomous moving bodies.

The figure which shows the structural example of the moving body of the navigation system of one Example. The figure which shows the configuration example of the moving body for data collection of the navigation system of one Example. The figure which shows the configuration example of the learning apparatus of the navigation system of one Example. The flowchart which shows the whole processing of the navigation system of one Example. The figure which illustrates the collection range of the image used for generating the learning data for approach movement. The figure which illustrates the collection range of the alignment learning data. The flowchart which shows the learning procedure of the learning apparatus of one Example. The flowchart which shows the traveling procedure of the moving body of one Example. The figure which shows the example of the navigation device which a plurality of moving bodies 1 exist.

Examples of the present invention will be described with reference to the drawings as appropriate.

The navigation system 100 of the present invention uses a moving body 1 that autonomously moves toward a target position, a data collecting moving body 2 that collects learning data, and an approach movement command type or alignment using the collected learning data. It is a system including a learning device 3 for learning a command formula and a learning device 3. Then, the moving body 1 generates a movement command for autonomous movement by using the approaching movement command type and the alignment command type learned by the learning device 3, and makes a final target while passing through a predetermined waypoint. Autonomously move to a position. The learning device 3 can share information with the mobile body 1 and the data collecting mobile body 2 via wired or wireless communication.

In the following, it is assumed that the moving body 1 is either a non-holonomic moving body (such as a vehicle or a trolley) that cannot move to the side or a holonic moving body (such as a robot or a drone) that can move in all directions. It is assumed that the environment in which the body 1 moves autonomously is an environment such as a shopping mall or a public facility where the environmental changes such as the traffic volume and brightness of people are large. In this system, even in such a situation, regardless of the form of the moving body 1, autonomous movement to the target position (autonomous running, autonomous walking, autonomous) is performed by using the common approach movement command type and alignment command type. Since the movement command required for (flying, etc.) can be generated, the learning device 3 has an advantage that it is not necessary to learn the command formula for each form of the moving body 1. Hereinafter, examples of the moving body 1, the moving body for collecting data 2, and the learning device 3 in the navigation system 100 of the present invention will be described in detail with reference to the drawings.

<Mobile 1>
FIG. 1 is a diagram mainly for explaining a configuration example of the moving body 1, and also displays a partial configuration of the moving body 2 for data collection and the learning device 3. Although the navigation system 100 in which only one mobile body 1 exists is illustrated here, it may be a system in which a plurality of mobile bodies 1 having different forms exist, as will be described later in FIG. ..

As shown in FIG. 1, the moving body 1 has an image acquisition unit 11, a moving mechanism 12, a calculation unit 13, and a storage unit 14. The details of each will be described in sequence.

The image acquisition unit 11 acquires an image of the surroundings from the current position of the moving body 1, and is composed of, for example, one or more cameras provided with an image sensor such as a CCD image sensor or a CMOS image sensor. ing. The image obtained by the image acquisition unit 11 capturing the outside world is output to the movement switching unit 13a, the global movement command unit 13b, the approach movement command unit 13c, and the alignment command unit 13d, which will be described later.

It is desirable to use a camera having a high resolution in the image acquisition unit 11, and a camera provided with a wide-angle lens may be mounted in order to take a wide image of the moving environment. It is desirable that the viewing angle and resolution of the camera are the same as the viewing angle and resolution of the camera of the image acquisition unit 21 mounted on the moving body 2 for data collection, which will be described later. The conditions may be met. Furthermore, it is desirable that the camera is mounted at a position and direction in which the front of the moving body can be imaged and fixed to the moving body 1 so that the posture of the camera does not move. When mounted, the posture of the camera does not have to be fixed.

The moving mechanism 12 is a mechanism that realizes the movement of the moving body 1. If the moving body 1 is a vehicle, it is a wheel, if it is a robot, it is a leg or a wheel, if it is a drone, it is a propeller, and an actuator that drives the wheel or the like. is there. In the movement control unit 13e described later, a desired movement command (for example, translational movement speed, rotational movement speed, etc.) according to the form of the movement mechanism 12 is based on the movement control signal output from the alignment command unit 13d or the like described later. Is generated, and the moving body 1 is moved back and forth, turned, or stopped.

The calculation unit 13 includes a movement switching unit 13a, a global movement command unit 13b, an approach movement command unit 13c, an alignment command unit 13d, and a movement control unit 13e. A predetermined program is stored in the storage unit 14 of the moving body 1, and this program is loaded into the memory and executed by the CPU (Central Processing Unit), so that the movement switching unit 13a and the global movement command unit are executed. 13b, approach movement command unit 13c, alignment command unit 13d, and movement control unit 13e are embodied.

The movement switching unit 13a compares the current position image acquired by the image acquisition unit 11 with the target position image acquired from the target position image storage unit 14a, and sets the movement method to the target position as "global movement suitable for high-speed movement". , "Approaching movement" to move closer to the target position, "Alignment" to stop accurately at the target position, or "Alignment" to accurately pass the target position. The determination result is output to the global movement command unit 13b, the approach movement command unit 13c, and the alignment command unit 13d. Although the case where there are three types of moving methods is illustrated here, the present invention does not limit the number of moving methods. Hereinafter, an example of switching determination of each movement method by the movement switching unit 13a will be described.

When switching the movement method to "approaching movement", the movement switching unit 13a determines, for example, whether or not a predetermined object (landmark) imaged in the target position image is captured in the current position image. Then, when it is determined that the image is captured, switch to "approach movement" is selected. It should be noted that the approaching movement command unit 13c may make a determination based on the image, and the movement switching unit 13a may divide the role of determining the switching based on the determination result of the approaching movement command unit 13c. At this time, the target position image used by the approaching movement command unit 13c is switched in the vicinity of the waypoint, and while switching the target position image, the target position image is moved to the point where the final target position image is captured. When an object of the current position image is shown in a plurality of target position images, a target position image close to the final target position is selected.

Further, when the movement method is switched to "alignment", the movement switching unit 13a determines, for example, whether the similarity between the current position image and the target position image for alignment is equal to or higher than the threshold value, and when it becomes equal to or higher than the threshold value. Select to switch to "Alignment". As a method for calculating the similarity of images, for example, MSE (Mean Square Error), which aggregates the square of the difference in the brightness of pixels between the same positions in two images, is used. The similarity may be used as part of the processing of the alignment command unit 13d. If the same object appears in the target position image used by the approach command unit 13c and the target position image used by the alignment command unit 13d, the size of the object output by the approach command unit 13c is preset. If it exceeds the threshold value, the movement method may be switched to "alignment", and even if the relative position / orientation output of the alignment command unit 13d is continuously output to a value below the threshold value, the movement may be performed. The method may be switched to "alignment".

If none of the above applies, the movement switching unit 13a selects "global movement" as the movement method. When a plurality of movement methods are duplicated and it is determined that they can be handled, the order of priority is "alignment", "approach movement", and "global movement". Each determination may be performed in parallel at the same time, but the order of determination may be determined. By doing so, in this embodiment, it is possible to select an appropriate method from a plurality of movement methods and switch to another movement method at an appropriate timing.

The global movement command unit 13b is a command unit that is selected when the movement switching unit 13a determines that "global movement" is to be performed, and receives the current position image acquired from the image acquisition unit 11 as input and moves in the direction of the target position. A movement command such as a speed command for moving is output to the movement control unit 13e. The movement method by the global movement command unit 13b is not particularly limited in this embodiment, but the movement to the target position is performed by using a method called a movement visual odometry that estimates the movement amount of the moving body from the change of the current position image. You may. Although this method is not highly accurate, approximate movement information can be obtained only from the time-series input of the current position image. Other than that, for example, a method called visual SLAM may be used. This method is a method of estimating the current position of a moving body at the same time while creating a map in which the feature points of the image are three-dimensionally arranged from the time series data of the current position image acquired by the image acquisition unit 11. Since the load is very large and errors accumulate, it cannot be used all the time.

The approach movement command unit 13c is a command unit selected when the movement switching unit 13a determines to perform "approach movement", and is obtained from the current position image acquired by the image acquisition unit 11 and the target position image storage unit 14a. The acquired target position image is input to the approach movement command formula described later, and the direction of the target position is output. In addition, as this approach movement command type, the one obtained by the pre-learning by the learning device 3 is used.

The alignment command unit 13d is a command unit selected when the movement switching unit 13a determines that "alignment" is to be performed, and the current position image acquired by the image acquisition unit 11 and the target position image storage unit 14a are used. The acquired target position image is input to the alignment command formula described later, and the relative position / posture (relative position and relative angle) up to the target position / posture (including the direction of the moving body) is output. As this alignment command formula, the one obtained by pre-learning with the learning device 3 is used.

The movement control unit 13e is either a movement command calculated by the global movement command unit 13b, a direction of the target position calculated by the approach movement command unit 13c, or a relative position / posture to the target position / posture calculated by the alignment command unit 13d. Is used as an input, a movement plan (or route plan) is performed to realize the input command, etc., and appropriate movement control is performed. The movement control unit 13e performs a movement plan from the relative position / posture and controls the movement to the target position / posture, so that the position accuracy is high, for example, when passing through a narrow place such as a door (passing through opening the door). In addition, even in situations where high directional accuracy is required, the types of movement mechanisms 12 are different, such as a holonomic moving body that can move in all directions like a drone and a non-holonomic moving body that cannot move to the side like a car. Any moving body 1 can be dealt with by a common command formula.

The storage unit 14 includes a target position image storage unit 14a, an approach movement command type storage unit 14b, and an alignment command type storage unit 14c.

The target position image storage unit 14a stores one or more target position images captured in advance. In this embodiment, the moving body 1 is navigated by using the position where the target position image stored in the target position image storage unit 14a is captured as a waypoint or a final target position. The target position image stored here may be taken by moving the moving body 1 or another moving body to be navigated in advance, or may be a target position image taken by a person in advance, the Web, or the like. The target position image in the cloud or the like may be registered. However, when the image acquisition unit 11 of the moving body 1 is composed of a plurality of cameras, it is necessary to acquire the target position image having the same positional relationship. Further, in this embodiment, it is desirable to use an image taken from the same height as the image acquisition unit 11, but as will be described later, it is possible to deal with cases where the installation position in the height direction is different. Thereby, in this embodiment, it can be applied to a moving body having a different installation position of the image acquisition unit 11. Further, the target position image storage unit 14a may store the target position image used by the alignment command unit 13d and the designated information of the target position image used by the approach movement command unit 13c, or may be used by the approach movement command unit 13c. The object information captured in the target position image may be stored.

The approach movement command type storage unit 14b appropriately acquires the “approach movement command type” learned by the learning device 3 from the approach movement command type storage unit 33c of the learning device 3. Similarly, the alignment command type storage unit 14c appropriately acquires the “alignment command type” learned by the learning device 3 from the alignment command type storage unit 33d of the learning device 3. The approach movement command type and the alignment command type are acquired, for example, at the timing when the learning device 3 relearns.

Next, a configuration example of the data collecting mobile body 2 and the learning device 3 will be described with reference to FIGS. 2 and 3.

<Mobile 2 for data collection>
FIG. 2 is a diagram mainly for explaining a configuration example of the data collecting mobile body 2, and also displays a partial configuration of the mobile body 1 and the learning device 3. The data collection moving body 2 is a device that collects learning data in the vicinity of the target position, and has an image acquisition unit 21, a coordinate estimation sensor 22, a calculation unit 23, and a storage unit 24, as shown in FIG. doing. In this embodiment, since it is assumed that the system operator or the like acquires the learning data while moving the data collecting mobile body 2, the data collecting mobile body 2 in FIG. 2 uses a moving mechanism. Although omitted, if the data collection moving body 2 is desired to run on its own, a moving mechanism may be provided as in the moving body 1.

The image acquisition unit 21 acquires an image of the surroundings taken from the current position of the data collecting moving body 2, and is, for example, one or more cameras equipped with an image sensor such as a CCD image sensor or a CMOS image sensor. It is composed of. The image data obtained by imaging the outside world by the image acquisition unit 21 is output to the alignment learning data creation unit 23a and the image storage unit 24a, which will be described later. In addition, in order to widely image the moving environment and collect a lot of learning data, it is possible to mount multiple cameras with different installation positions (positions with respect to height and moving direction) and types, or mount cameras with wide-angle lenses. It is desirable to do. However, it is desirable that one of the plurality of cameras be mounted at a position and direction in which the front in the traveling direction can be imaged. It is desirable that the viewing angle and resolution of the camera be as large as possible. Further, it is desirable to have a mechanism capable of changing the height and mounting angle (pitch angle and roll angle) of the camera. Further, it is desirable to fix the camera so that the posture of the camera does not move when mounted. However, if a system that constantly grasps the posture of the camera is installed, the posture of the camera does not have to be fixed. In this way, by collecting various learning data using one or a small number of data collecting mobile bodies 2, learning results that can correspond to various conditions with different types of moving bodies can be obtained. There is a feature of the embodiment.

The coordinate estimation sensor 22 acquires data necessary for estimating the coordinates of the data collecting moving body 2. For example, by using a sensor such as an encoder that acquires the rotation speed of the wheel, the coordinates of the moving body 2 for data collection can be estimated from the accumulation of the rotation speed of the wheel. Further, the coordinates of the data collecting moving body 2 may be estimated by using the information of an external sensor such as a motion capture system or a beacon that generates radio waves.

The calculation unit 23 has a learning data creation unit 23a for alignment and a coordinate estimation unit 23b. A predetermined program is stored in the storage unit 24 of the moving body 2 for data collection, and when this program is loaded into the memory and executed by the CPU, the alignment learning data creation unit 23a and the coordinates The estimation unit 23b is embodied.

The coordinate estimation unit 23b estimates the coordinates and orientation of the position of the data collecting moving body 2 using the sensor data input from the coordinate estimation sensor 22, and outputs the coordinates. The coordinates (and orientation) here may be absolute coordinates with a reference set at a certain point, or may be relative coordinates in a series of learning data for alignment.

The alignment learning data creation unit 23a combines the image acquired by the image acquisition unit 21 with the coordinates (and orientation) input from the coordinate estimation unit 23b and outputs the alignment learning data. The alignment training data is for learning the relationship between the two images (input of the formula) and the movement amount of the point where each image is captured (relative position / orientation: output of the formula) in the alignment command formula. It is a data set of, and is composed of data in which images captured at a plurality of positions and coordinates of the positions where each image is captured are set. The alignment learning data creation unit 23a associates the input image with the coordinates of the position where the image is captured as a set, and outputs the alignment learning data storage unit 24b.

The storage unit 24 includes an image storage unit 24a and a alignment learning data storage unit 24b.

The image storage unit 24a temporarily stores an image used for creating learning data for approach movement for learning the approach movement command formula described later. The image temporarily stored in the image storage unit 24a is transferred to the image storage unit 33a of the learning device 3 at regular intervals or during learning.

Further, the alignment learning data storage unit 24b temporarily stores the alignment learning data input from the alignment learning data creation unit 23a. The alignment learning data temporarily stored in the alignment learning data storage unit 22b is transferred to the alignment learning data storage unit 33b of the learning device 3 at regular intervals or when learning is performed.

<Learning device 3>
FIG. 3 is a diagram mainly for explaining a configuration example of the learning device 3, and also displays a partial configuration of the moving body 1 and the data collecting moving body 2. The learning device 3 is a device that learns the approach movement command type and the alignment command type based on the learning data transferred from the data collecting moving body 2, and as shown in FIG. 3, the object area input unit 31, Calculation unit 32 (learning data creation unit 32a for approach movement, approach movement learning preprocessing unit 32b, approach movement command type learning unit 32c, alignment learning preprocessing unit 32d, alignment command type learning unit 32e), storage unit 33 ( It has an image storage unit 33a, a learning data storage unit 33b for alignment, an approach movement command type storage unit 33c, and a alignment command type storage unit 33d).

<Alignment command type learning process>
First, the alignment command type learning process by the alignment learning data storage unit 33b, the alignment learning pre-processing unit 32d, the alignment command type learning unit 32e, and the alignment command type storage unit 33d will be described.

The alignment learning data storage unit 33b appropriately accumulates the alignment learning data transferred from the alignment learning data storage unit 24b of the data collection moving body 2. Further, although not shown, if the learning data already acquired by another device or another opportunity is on the Web or the cloud, the alignment learning data is also accumulated from there.

Next, the alignment learning preprocessing unit 32d increases the variation of the alignment learning data in order to improve the estimation accuracy of the alignment command formula. For example, for each image of the alignment training data, image processing related to lighting conditions such as contrast adjustment and gamma conversion, processing related to robustness such as addition of noise and deletion of a part of the image are performed. Then, increase the variation of training data. Even if the image is converted, the corresponding coordinates are not changed. By doing so, when moving in a wide range of environments, robust learning results can be obtained even for environmental changes (arrangement of people and objects, lighting conditions, etc.) between the time of image acquisition of training data and the time of actual image acquisition. However, there is a feature of this embodiment.

Here, the alignment command formula will be described in detail. The alignment command formula is a formula that takes the images P and Q captured at the positions p and q as inputs and outputs the coordinates of the position q when the position p is the origin, that is, the movement amount. First, there is (Equation 1) as an example of the alignment command type. Here, for the sake of simplification of the explanation, a simple (Equation 1) will be used, but (Equation 1) represents one layer of the deep neural network, and the output u _{1 is actually used.} (Hereinafter, the subscript is the layer number) is further input to (Equation 1) as _{input x 2 , and the output u 2} is further input _{to (Equation 1) as input x 3} , which is repeated for the number of layers. , More complicated formulas are used.

In (Equation 1), x is a vector (hereinafter referred to as two-point image data) representing images P and Q. f is a non-linear function, u is a vector representing the amount of movement from the position p to q, and consists of three dimensions (X, Y, θ). u is a Cartesian coordinate system, and when the position p is the origin, X is the front direction of the image, Y is the left hand direction of the image, θ is 0 in the front of the image, and counterclockwise is positive. And W is a weighting coefficient matrix of x. And b is a coefficient representing the shift amount.

In the learning process of the alignment command formula in this embodiment, the coefficient matrix W and the coefficient matrix W are used so that the movement amount u of the two points output when the two-point image data x is input outputs a more accurate determination result. This is a process for adjusting the shift amount b. As a result of the learning, the moving body 1 can estimate the moving amount of the two points from the two-point image data x by the alignment command formula using the learned coefficient matrix W and the shift amount b in (Equation 1).

From here, the learning process of the alignment command type performed by the alignment command type learning unit 32e will be described. First, some learning data are randomly extracted from the alignment learning data group input from the alignment learning preprocessing unit 32d. Then, for (Equation 1) in which the initial value of the coefficient matrix W and the initial value of the shift amount b are substituted, the two-point image data x of each of the extracted data is input, and the calculation is performed by (Equation 1). The error e between the u and the actual movement amount u'at the two points is calculated, and the total value E of the errors is calculated. Then, the coefficient matrix W and the shift amount b are slightly changed so that the total value E of the errors becomes smaller. As a method of slightly changing the coefficient matrix W and the shift amount b here, for example, a gradient descent method can be used. This learning process is performed on all of the alignment training data groups.

By repeating the above procedure, the total error value E gradually becomes smaller, and the alignment command formula is continuously updated until the learning end condition is satisfied. The learning end condition includes, for example, a condition in which the learning process for the alignment learning data group is performed until a certain number of times is reached, and a condition in which the total error value E is the minimum. The movement amount of the two points output by the alignment command formula thus obtained is regarded as the optimum movement amount with respect to the two-point image x. Therefore, the distance and direction to the target position can be calculated by inputting the current position image and the target position image of the moving body 1 as x. Furthermore, by performing learning processing with a large amount of diverse and large amount of alignment training data, it is possible to calculate the movement distance to the target position even in the first place not included in the alignment training data or when there is an environmental change. Become.

The alignment command formula (formula 1) including the coefficient matrix W and the shift amount b after the learning process is completed, which is calculated in this way, is stored in the alignment command formula storage unit 33d. Then, the alignment command type is transferred to the alignment command type storage unit 14c of the moving body 1 at a timing such as before the operation of the moving body 1.

<Learning process of approach movement command type>
Next, the approach movement is performed by the object area input unit 31, the image storage unit 33a, the approach movement learning data creation unit 32a, the approach movement learning preprocessing unit 32b, the approach movement command type learning unit 32c, and the approach movement command type storage unit 33c. The command-type learning process will be described.

The image storage unit 33a appropriately stores the images transferred from the image storage unit 24a of the data collection moving body 2.

Next, the approach movement learning data generation unit 32a creates the approach movement learning data from the images stored in the image storage unit 33a. Specifically, the approach movement learning data creation unit 32a has an object (for example, a door) that can be a landmark for movement in any range of the image with respect to the image input from the image storage unit 33a. Judge whether it is reflected and specify the object area part where the landmark is reflected. The object area portion may be designated by a human through the object area input unit 31. In that case, as a method of designating the object region portion, for example, a so-called bounding box (rectangle surrounding the object) may be used. The object area input unit 31 may use, for example, a mouse, a keyboard, a touch screen, or the like. It should be noted that the designation of the object region portion may be automatically extracted from the color, shape, or the like without being performed by a human. By using the object area, it is possible to obtain information on the size of the object in addition to the position of the object. This set of object area information and images becomes learning data for approach movement. At this time, although not shown, if the learning data already acquired by another device or another opportunity is on the Web or the cloud, the learning data obtained from the learning data may be used.

Next, the approach movement learning preprocessing unit 32b increases the variation of the approach movement learning data in order to improve the estimation accuracy of the approach movement command formula. For example, for each image of the learning data for approach movement, image processing related to lighting conditions such as contrast adjustment and gamma conversion, processing related to robustness such as addition of noise and deletion of a part of the image are performed. So, increase the variation of data. Even if the image is converted, the corresponding object area is not changed. By doing so, even if the image is not included in the training data, it is robust against environmental changes (arrangement of people and objects, lighting conditions, etc.) between the time of image acquisition and the time of actual image acquisition of the training data. The feature of this embodiment is that various learning results can be obtained.

Here, the approach movement command formula will be described in detail. The approach movement command formula is composed of a two-step formula. First, a landmark object is detected from the current position image and the target position image input in the approach movement command formula. Then, the direction of the target position is calculated using the detected position of the object.

The detection of the object, which is the first stage of the approach movement command formula, can be expressed by (Equation 2) in the same manner as the alignment command formula (Equation 1), for example. Although the simple (Equation 2) is used here for the sake of simplification of the explanation, (Equation 2) represents one layer of the deep neural network, and is actually used. repeating that the output u _{'1 (hereinafter,} subscript numbers of layers) is input x' is input to the addition (equation 2) _2, the output u _'2 is further x' is input to the (formula 2) as a ₃ A more complicated calculation formula is used in which the processing is performed for the number of layers.

Here, x'is a vector of one image, and u'is a vector representing the type of the detected object, its certainty, and its position and size in the image. The certainty of the detected object is expressed as a numerical value from 0 to 1 for each detected object, and the larger the numerical value, the higher the possibility that the object is. The position and size of the detected object are represented by, for example, the width, height, and pixel position of the center point of the bounding box.

Then, by learning (Equation 2) using the learning data for approach movement as in the learning process of the alignment command type, it becomes possible to detect an object that becomes a landmark from the image. At this time, it is also possible to detect a plurality of objects at the same time by cutting out the input images into a plurality of images. Fixed objects such as plants, signage, doors, lights, poles, and windows are used as landmark objects, but those installed above to avoid the influence of moving objects such as people are used. Is good.

The second stage of the approach movement command type can be represented by, for example, the following (Equation 3).

In (Equation 3), u ₁ and u ₂ are pixel positions of an object (same object type) commonly detected from the current position image and the target position image, respectively. When detecting multiple objects, it is necessary to prepare an equation corresponding to the number of objects. Here, when detection in the height direction is not required, only the pixel position in the width direction of the image needs to be used. In this case, images captured at different heights can also be used, and the method of capturing the target position image and the type of moving body can be freely selected. y is the direction of the target position, k is the coefficient matrix, and g is the function that calculates the geometric transformation.

The approach movement command equations (Equation 2) and (Equation 3) including the coefficient matrix W'and the shift amount b'after the completion of the learning process calculated in this way are stored in the approach movement command expression storage unit 33c. Then, the approach movement command type is transferred to the approach movement command type storage unit 14b of the moving body 1 at a timing such as before the operation of the moving body 1.

<Flowchart of navigation system 100>
FIG. 4 is a flowchart showing an example of a series of processes by the navigation system 100 of the present embodiment, which includes the mobile body 1, the data collection moving body 2, and the learning device 3 described above. Note that FIG. 4 shows an example for the sake of explanation, and does not limit the procedure of the present invention.

<Step S1>
First, in step S1, the image and coordinates in the vicinity of the target position are collected by using the data collecting moving body 2. The collected images are used by the approach movement learning data creation unit 32a to create approach movement learning data, and the image and coordinate data sets are input to the alignment learning preprocessing unit 32d for alignment learning. Used as data.

Here, with reference to FIGS. 5 and 6, a method of collecting each data by the data collecting moving body 2 under the situation where the target position G is set in front of the door D will be described.

5, in the case of setting the target position G in front of the door D, a diagram illustrating a data acquisition range A ₁ that can capture an image needed to create the approaching moving learning data. As is apparent from this figure, the data collection for mobile 2 when you are in fan shape (simplified in FIG. 5, the triangular shape) data acquisition range A ₁ of a landmark when approaching the target position G Door D can be imaged. Therefore, collect learning data acquisition for mobile 2, while variously move data acquisition range A inside _1, required to create approaching moving learning data, a variety of image door D (landmark) is captured To do. Even if there are many similar images, they are not very useful as learning data, so it is desirable to collect the images evenly at various positions and conditions such as the distance and angle to the door D and the brightness of the illumination. Also, the data collection for mobile 2 by the data acquisition range A inside ₁ leave record movies when moving randomly, later, it is also possible to simplify the task of collecting image by cutting out the image from the video ..

On the other hand, FIG. 6, in the case of setting the target position G in front of the door D, is for positioning the training data, is a diagram illustrating a data acquisition range A ₂ to collect a data set of the image and coordinates. Here, the data collection moving body 2 is moved around the target position G (for example, a square area of about 1 m surrounding the target position G), and the image and coordinate data sets are collected at high density. For example, as shown in the balloon on the right side of the figure, at each point of the high-density grid set around the target position G, the image acquisition unit 21 of the data collection moving body 2 is directed in various directions to set the data set. Good to collect. At this time, the distance d of each grid and the step size Δθ in the data collection direction in each grid may be set in consideration of the target accuracy of the moving body 1 using this embodiment. The learning data at the positions between the crids may be interpolated by learning.

<Step S2>
In step S2, the learning device 3 is used to learn the approach movement command type and the alignment command type. The details of this learning process will be described with reference to the flowchart shown in FIG.

First, in step S21, the approach movement learning data creation unit 32a acquires an image in which the designated landmark (for example, the door D) is captured via the object area input unit 31 from the image storage unit 33a. Create learning data for approach movement. Next, the approach movement learning preprocessing unit 32b reads the approach movement learning data from the approach movement learning data creation unit 32a, and the alignment learning preprocessing unit 32d is for alignment from the alignment learning data storage unit 33b. Read the training data.

Next, in step S22, the approaching movement learning preprocessing unit 32b and the alignment learning preprocessing unit 32d perform preprocessing on each learning data to increase the variation of each learning data.

In step S23, the approach movement command type learning unit 32c performs the approach movement command type learning process, and the alignment command type learning unit 32e performs the alignment command type learning process.

Finally, in step S24, the learned approach movement command formula (for example, equations 2 and 3) is stored in the approach movement command type storage unit 33c, and the alignment command formula (for example, formula 1) is stored in the alignment command formula. It is stored in the storage unit 33d.

<Step S3>
In step S3, the target position (final target) of the actual moving environment of the moving body 1 is used by using the image acquisition unit of the moving body 1 or the moving body 2 for data collection, or a digital camera carried by the system operator. At the desired point (position and waypoint), a target position image including an object that can be a landmark is imaged and registered in the target position image storage unit 14a of the moving body 1. At the time of autonomous movement of the moving body 1, the target position image registered in the target position image storage unit 14a in this step becomes data indicating the target position.

Although the imaging position in this step is different from the imaging position in step S1, the surrounding environment of the imaging position in step S1 and this step is similar to some extent that each command formula learned in step S2 is useful. It is assumed that In other words, it is necessary to complete the learning process using the data collection moving body 2 and the learning device 3 for an environment similar to the target position of the actual moving environment, and the learning process for such an environment is completed. If not, care must be taken to carry out the processes of steps S1 and S2 for such an environment.

<Step S4>
In step S4, the moving body 1 is made to perform autonomous movement to the target position. If there is no change in the environment, the process of this step is executed repeatedly.

FIG. 8 is a flowchart showing an example of this step in detail. Each process shown here may be executed in parallel, but for the sake of simplicity, the procedure will be described sequentially.

Step S41: First, the movement switching unit 13a determines whether to switch the moving method of the moving body 1 to "alignment". For example, when the similarity between the current position image and the target position image exceeds a certain value, or when the landmark object becomes larger than the threshold value, the relative position and orientation become continuous with the value below the threshold value. If so, determine the switch to "alignment". As a result of the determination, when switching to "alignment", the process proceeds to (Yes) step S42. On the other hand, when the "global movement" or the "approaching movement" is continuously executed, the process proceeds to (No) step S44.

Step S42: The moving body 1 moves toward the target position by a command from the alignment command unit 11d. The movement in this step moves to the target position based on the alignment command formula. After moving for a predetermined time, the process proceeds to step S43.

Step S43: The movement switching unit 13a determines whether or not the “alignment” has been completed, that is, whether or not the moving body 1 has reached the target position. For example, if the relative position / posture value to the target position calculated from the alignment command formula is less than the permissible value, it is determined that the target position has been reached and the running is terminated. If not (No), the target position is reached. In order to continue the "alignment" of the above, the process proceeds to step S41.

Step S44: The movement switching unit 13a determines whether to switch the moving method of the moving body 1 to "approaching movement". For example, it is determined whether or not an object (landmark) imaged in the target position image acquired in advance is captured in the current position image, and if a common object is captured (Yes), "approach movement". If not (No), the process proceeds to step S46.

Step S45: The moving body 1 moves toward the target position by the approaching movement command unit 13c. The movement in this step moves toward the target position based on the approach movement command formula. After moving for a predetermined time, the process proceeds to step S41.

Step S46: The moving body moves from the movement start position to the target position by the global movement command unit 13b. After moving for a predetermined time, the process proceeds to step S41.

One feature of this embodiment is that the movement to the target position is divided into a plurality of steps of "global movement", "approaching movement", and "alignment". In particular, in "approach movement", the part of the current position image that moves by estimating only the movement direction by focusing only on that part with the object shown in the target position image as a landmark, and in "alignment". The part that estimates the relative position and orientation to the target position and moves by using the entire image of the current position image and the target position image is a feature different from the prior art.

Due to this feature, when the moving body 1 is far from the target position and there is little information about the target position in the current position image, the moving body 1 moves roughly focusing only on the object (landmark) at the target position and approaches the target position. Only after the information about the target position increases in the current position image, the relative position and orientation to the target position can be accurately estimated by using the entire image of the current position image and the target position image. However, the range in which the position and posture of the moving body 1 can be accurately estimated is only a narrow range as illustrated in FIG. As a result, even if the driving environment changes such that objects other than landmarks increase or decrease in the current position image (for example, increase or decrease in traffic volume depending on the day of the week in the shopping mall), the vehicle moves more accurately to the target position. It is possible to stop or pass.

As the current position image used in step S42, images corresponding to the number of shots taken several times in the past are taken, and the difference is calculated to select an image in which moving obstacles such as passersby are not shown. You can move more accurately.

<Step S5>
Finally, in step S5, the operator determines whether re-learning in the new environment is necessary, that is, whether it is necessary to actually drive in an environment dissimilar to the learned environment, and when it is determined that it is necessary. Steps S1 and S2 are executed again. On the other hand, if there is no such need, the process returns to step S3.

Hereinafter, an implementation example of the navigation system 100 of this embodiment will be described.

FIG. 9 is a diagram showing an example of a navigation system 100 in which a plurality of mobile bodies 1 exist. When the navigation system has a plurality of moving bodies 1A to 1C as in this example, in the data collecting moving body 2, the image acquisition unit 21 corresponds to the height and mounting angle of the image acquisition part 11 of the moving bodies 1A to 1C. A mechanism that can acquire various data is provided so that various data can be acquired, and learning data for approach movement and learning data for alignment are collected under installation conditions suitable for various cameras of the moving bodies 1A to 1C. Then, based on the accumulated learning data, learning of the approach movement command type and the alignment command type is performed.

With the above configuration and procedure, it is possible to provide a navigation device and a navigation system for a moving body that can move the moving body to a target position robustly and accurately in response to changes in the traveling environment. ..

100 Navigation system 1,1A to 1C Moving object 11 Image acquisition unit 12 Movement mechanism 13 Calculation unit 13a Movement switching unit 13b Global movement command unit 13c Approach movement command unit 13d Alignment command unit 13e Movement control unit 14 Storage unit 14a Target position image Storage unit 14b Approach movement command type storage unit 14c Alignment command type storage unit 2 Data collection moving body 21 Image acquisition unit 22 Coordinate estimation sensor 23 Calculation unit 23a Alignment learning data creation unit 23b Coordinate estimation unit 24 Storage unit 24a Image storage unit 24b Alignment learning data storage unit 3 Learning device 31 Object area input unit 32 Calculation unit 32a Approach movement learning data generation unit 32b Approach movement learning preprocessing unit 32c Approach movement command type learning unit 32d Alignment learning preprocessing Unit 32e Alignment command type learning unit 33 Storage unit 33a Image storage unit 33b Alignment learning data storage unit 33c Approach movement command type storage unit 33d Alignment command type storage unit

Claims (11)

It is a mobile body that moves autonomously
An image acquisition unit that captures the current position image and
A target position image storage unit that stores a target position image captured at the target position,
An approach movement command unit that generates a movement command to move closer to the target position based on the approach movement command formula, and
An alignment command unit that generates a movement command that stops at the target position or a movement command that passes through the target position based on the alignment command formula.
A movement switching unit that selects either the approach movement command unit or the alignment command unit according to the comparison result between the current position image and the target position image.
A movement control unit that controls the movement mechanism based on the movement command,
A moving body characterized by comprising. The target position image stored in the target position image storage unit is a target position image captured at the final target position and a target position image captured at a waypoint leading to the final target position.
The moving body according to claim 1, wherein at the time of autonomous movement, the target position image used by the movement switching unit is switched in the order of the waypoints and autonomously moved to the final target position. The alignment command unit calculates the relative position and the relative posture from the current position to the target position by comparing the current position image with the target position image, and outputs the movement command according to the calculation result. The moving body according to claim 1, wherein the moving body is characterized in that. The approaching movement command unit is characterized in that by comparing the current position image with the target position image, the movement direction from the current position to the target position is calculated, and the movement command according to the calculation result is output. The moving body according to claim 1. The movement switching unit is
When the similarity between the current position image and the target position image exceeds a predetermined threshold value,
When an object of the same type appears in the current position image and the target position image, and the size of the object in the current position image exceeds a predetermined threshold value, or
When the relative position / orientation output by the alignment command formula falls below a predetermined threshold value,
The moving body according to claim 1, wherein the alignment command unit is selected in any of the cases. The movement switching unit is
Although the condition for selecting the alignment command unit is not satisfied,
The moving body according to claim 5, wherein the approaching movement command unit is selected when an object of the same type is shown in the target position image and the current position image. If the movement switching unit does not satisfy the condition for selecting the alignment command unit and also does not satisfy the condition for selecting the approach movement command unit, the global movement command unit moves from the current position to the target position. The moving body according to claim 6, wherein the mobile body is selected. A navigation system for navigating a moving object according to any one of claims 1 to 7.
The alignment command type is a navigation system characterized in that the learning data collected by a moving body for data collection is obtained by learning processing with a learning device. The learning data is characterized by being a first image and a second image captured by the data collecting moving body, and a relative position and a relative posture from the imaging position of the first image to the imaging position of the second image. The navigation system according to claim 8. A navigation system for navigating a moving object according to any one of claims 1 to 7.
The approach movement command type is a navigation system characterized in that the learning data collected by a moving body for data collection is obtained by learning processing with a learning device. The navigation system according to claim 10, wherein the learning data is a third image captured by the data collecting moving body and a region where a predetermined object is captured in the third image.

Images