JP2022050115A - Movable body position estimation system, operator terminal device, and position estimation method - Google Patents

Abstract

To provide a position estimation system capable of estimating the position of a movable body with a little load imposed on an operator.SOLUTION: A position estimation system includes: a movable body 2; an operator terminal device 3 installed apart from the movable body; and movable body position estimation means 4. The movable body includes environment information acquisition means 22 that acquires a shape of an ambient environment, and can provide the operator terminal device with shape data acquired by the environment information acquisition means. The operator terminal device includes: image display means 31 capable of superposing the shape data on map information containing at least part of the moving range of the movable body and displaying the resultant data and information; and input means 32 capable of being manipulated to relatively move and/or turn at least one of the shape data and map information, which are displayed on the image display means, with respect to the other one. The movable body position estimation means can estimate a position and orientation of the movable body on the basis of a movement quantity and/or turn quantity manipulated by the input means.SELECTED DRAWING: Figure 2

Description

The present invention relates to a position estimation system, an operator terminal device, and a position estimation method for estimating the position of an actual moving body on a map, and in particular, an autonomous moving body and a moving body. The present invention relates to a position estimation system, an operator terminal device, and a position estimation method, which are systems including an operator terminal device that can be connected to and can be used when a moving body loses its own position (when it is lost).

Conventionally, an automatic guided vehicle that travels along a predetermined travel path has been known (Patent Document 1). The automatic guided vehicle identifies the contact position with respect to the layout screen, the storage means for storing the layout data related to the travel path and each transfer position, the display means for displaying the layout screen based on the layout data, and corresponds to the contact position. Derived by a derivation means that derives the actual position in the actual layout, a control means that reads the necessary layout data from the storage means, and controls the display means to display the layout screen based on the read layout data, and the derivation means. The control device of the automatic guided vehicle equipped with the setting means for capturing the address data of the actual position and setting it on the automatic guided vehicle guides the vehicle to the target position among the multiple transfer positions for loading or unloading. be able to. For example, when setting the reference position for the automatic guided vehicle to start up or recover from an abnormality, or the target position for loading / unloading to the automatic guided vehicle, the operator is displayed on the display means. By touching the layout screen with a finger, the contact position is identified, and the address data of the actual position corresponding to the contact position is transmitted to the communication means on the automatic guided vehicle side via the communication means, and the CPU on the automatic guided vehicle side. Imported the received address data, executed the address setting, and specified the reference position and the target position.

In addition, a robot that stores an environmental map and autonomously moves to a target point is also known. For example, in Patent Document 2, a plurality of distances for measuring the distance between an autonomous traveling unit provided with a camera for capturing a traveling route and an obstacle or the like by holding map data which is information on an area for autonomous traveling in a memory unit. An autonomous traveling robot that estimates and corrects its own position with a traveling control unit equipped with a sensor and controls the movement mechanism of the traveling control unit to travel is disclosed. The autonomous traveling robot recognizes its own position by a camera and performs autonomous traveling according to the map data stored in the memory unit. However, the autonomous traveling robot cannot accurately estimate its own position due to problems such as a deviation between the estimated position and the actual position, or the inability to measure the feature points of the traveling area, and there is a problem that the robot stops traveling. In the autonomous traveling robot of Patent Document 2, when the self-position estimation fails, the distance to the surrounding wall is measured by the distance sensor, the wall model is estimated from the measured distance data, and the wall model near the current position selected from the map data is used. It is disclosed that the self-position is restored by comparison with.

Japanese Unexamined Patent Publication No. 2004-110111 Japanese Unexamined Patent Publication No. 2008-5218

Patent Document 1 discloses that an operator touches a layout screen displayed on a display means with his / her finger to set a reference position at the time of starting up or recovering from an automatic guided vehicle. In this case, the operator had to check the position of the automatic guided vehicle every time when starting up or recovering from an abnormality, and touched the corresponding position on the layout screen with his finger, which was troublesome. Further, in Patent Document 2, it is stated that self-recovery can be performed automatically, but the self-position estimation has originally failed for some reason, and if the self-recovery cannot be performed due to such a cause, autonomous driving is eventually performed. Will be canceled.

An object of the present invention is to solve a part of the above-mentioned problems, and a position estimation system, an operator terminal device, and an operator terminal device capable of estimating the position of a moving object with less burden on the operator. One of the purposes is to provide a position estimation method.

In order to solve the above problems, the position estimation system for a moving body of the present invention comprises a moving body including the moving body, an operator terminal device provided separately from the moving body, and a moving body position estimating means. It is a position estimation system of a moving body including, and the moving body includes an environment information acquisition means for acquiring the shape of the surrounding environment of the moving body, and obtains shape data regarding the shape of the surrounding environment acquired by the environmental information acquisition means. The operator terminal device can be provided to the operator terminal device, and the operator terminal device includes an image display means capable of superimposing and displaying map information including at least a part of the shape data and the movement range of the moving body, and the image display. The moving body position estimating means includes an input means capable of moving and / or rotating at least one of the shape data and the map information displayed on the means relative to the other, and the moving body position estimating means is provided by the input means. The position and orientation of the moving body can be estimated based on the manipulated movement amount and / or rotation amount.

Further, in the moving body position estimation system, the moving body position estimating means is a moving body at the time of acquiring the shape data on the map information after being relatively moved and / or rotated by the input means. The position and orientation of the moving body may be estimated from the arrangement.

Further, in the position estimation system of the moving body, the shape data includes the arrangement information of the moving body at the time of acquiring the shape data, and the image display means has the movement at the time of acquiring the shape data together with the shape data. The body arrangement information is displayed integrally with the shape data, and the input means integrally with respect to the map information without changing the positional relationship between the shape data and the arrangement information of the moving body. It may be movable and / or rotateable.

Further, in the position estimation system of the moving body, the input means automatically moves or rotates so as to have a high degree of coincidence with the shape data or the map information operated by the input means. It may have an automatic alignment function to execute.

Further, in the position estimation system of the moving body, the moving body controls the self-position estimation means for estimating the self-position by collating the shape data with the map information, and the self-position by controlling the moving means. The moving body includes a control means for moving to a destination using the self-position estimated by the estimation means, and the moving body is attached to the operator terminal device when a problem occurs in the self-position estimation by the self-position estimation means. On the other hand, a support signal requesting the start of the position estimation process of the moving body may be transmitted, and the control means controls the movement by using the position and the direction of the moving body acquired from the moving body position estimating means. You may.

Further, in the position estimation system of the moving body, the moving body further includes an image information acquisition means for acquiring an image of the surrounding environment of the moving body, and the image data of the surrounding environment acquired by the image information acquisition means is used. It can be provided to an operator terminal device, the operator terminal device may be capable of displaying image data of the surrounding environment on the image display means, and the operator terminal device may be a moving body. The moving means may be controlled to include an operating means capable of operating the movement of the moving body, and the input means may move to the shape data, image data, or map information of the surrounding environment displayed on the image display means. It may be possible to set the destination of the body.

Further, the terminal device for an operator of the present invention is provided separately from the moving body including the moving means, and obtains shape data regarding the shape of the surrounding environment acquired by the environmental information acquisition means for acquiring the shape of the surrounding environment of the moving body. The shape data acquisition unit to be acquired, a map information acquisition unit that acquires map information including at least a part of the movement range of the moving body, an image display means capable of displaying the shape data and the map information in an overlapping manner, and the above. An input means capable of moving and / or rotating at least one of the shape data and the map information displayed on the image display means relative to the other, and the movement amount operated by the input means and It is possible to provide the moving amount and / or the rotating amount operated by the input means to the moving body position estimating means capable of estimating the position and orientation of the moving body from the rotating amount.

Further, the method for estimating the position of a moving body of the present invention includes a moving body including a moving means and an environmental information acquisition means, an operator terminal device provided separately from the moving body and including an image display means and an input means. A method for estimating the position of a moving body including a moving body position estimating means, wherein the step of acquiring shape data regarding the shape of the surrounding environment of the moving body by the moving body environment information acquisition means, and the shape data are obtained. The step provided to the operator terminal device, the step of displaying the shape data and the map information including at least a part of the moving range of the moving body on the image display means of the operator terminal device, and the above. By the input means of the terminal device for an operator, at least one of the shape data and the map information displayed on the image display means is moved and / or rotated relative to the other, and the input means. A step of providing the operated moving amount and / or rotation amount to the moving body position estimating means, and a step of estimating the position and orientation of the moving body from the moving amount and / or the rotating amount by the moving body position estimating means. And, including.

According to the position estimation system of the moving body of the present invention, the shape data of the surrounding environment of the moving body and the map information including at least a part of the moving range of the moving body are superimposed and displayed on the image display means, and the operator can display the map information. At least one of the shape data and the map information displayed by using the input means can be moved and / or rotated relative to the other, and the movement amount operated by the input means by the moving body position estimation means. And / or the position and orientation of the moving body is estimated from the amount of rotation. In the present invention, when a problem occurs in self-position estimation in an autonomous mobile body, an operator at a remote location can solve the problem of the mobile body position via an operator terminal device (hereinafter, also referred to as "terminal device"). You can ask for the judgment of. The operator intuitively superimposes the shape data displayed on the image display means and / or the map information on the map by moving and / or rotating the shape data relative to the other. It is possible to grasp whether it corresponds to, and it becomes easy to determine the position of the moving body. In particular, the operator's work is only to superimpose at least one of the shape data and the map information by relatively moving and / or rotating the other with respect to the other, and the burden on the operator can be reduced. Other effects will be described in the embodiments.

An example of a flowchart of a moving body position estimation method of the present invention Block diagram of components required for the position estimation system of the moving body of the present invention An example of hardware that realizes the position estimation system of the present invention. (A) is map information showing the floor plan of the moving range of the moving body, and (B) is a figure showing the work of acquiring the shape data of the surrounding environment. (A) and (B) are diagrams showing the work of acquiring the shape data of the surrounding environment. (A) and (B) are diagrams for explaining the position estimation process of a moving body. Diagram for explaining the position estimation process of a moving object (A) and (B) are diagrams showing the work of acquiring the shape data of the surrounding environment.

[Position estimation process]
FIG. 1 is an example of a flowchart of a moving body position estimation method in the operator terminal device (terminal device) of the present invention. In the moving body position estimation system and method of the present invention, when the moving body has a problem in self-position estimation by the self-position estimation means, for example, when the moving body loses its self-position, the moving body moves off the path. When the operator's judgment is required, such as in the case, the operator's judgment can be sought for the problem of the position of the moving body via the operator terminal device (terminal device). In S1 of FIG. 1, the terminal device acquires shape data around the moving body, and in S2, displays the received shape data and map information including at least a part of the moving range of the moving body on the image display means. Further, the terminal device makes the input means operable and requires the operator to change the obstacle information as needed. In S3, the operator puts at least one of the shape data and the map information on the other. Relative movement and / or rotation operation (hereinafter referred to as "movement rotation operation") is performed. In S4, the terminal device acquires the movement amount and / or the rotation amount (hereinafter referred to as “movement rotation amount”) operated by the operator and provides the moving body position estimation means. The terminal device may start the processing when the support signal requesting the start of the position estimation processing of the moving body from the moving body is received. Further, the mobile body acquires shape data related to the shape of the surrounding environment of the mobile body by the environment information acquisition means, and provides the acquired shape data to the terminal device for the operator. The moving body position estimating means estimates the position and orientation of the moving body from the acquired movement rotation amount. For example, the position and orientation of the moving body are estimated from the arrangement of the moving body at the time of acquiring the shape data on the map information after the relative movement rotation operation by the input means. Further, the input means of the terminal device has an automatic alignment function of automatically moving or rotating the shape data or map information operated by the operator by the input means so as to have a high degree of matching. Is preferable.

In this way, the image display means of the terminal device displays the shape data of the surrounding environment of the moving body and the map information together, and the operator compares the shape data of the surrounding environment of the moving body with the map information. Since the position of the moving body can be estimated by determining the corresponding position and moving or rotating one of them to superimpose the two, the work can be performed intuitively and easily. Further, by displaying the image data around the moving body captured by the image information acquisition means on the image display means of the terminal device, for example, obstacles that do not exist in the map information can be grasped from the surrounding images and the shape data and the map. In some cases, the difference from the information can be confirmed, and the position of the moving object can be estimated more accurately. In addition, the terminal device moves with respect to the operation means capable of controlling the movement means of the moving body and the movement of the moving body, and the shape data, image data, or map information of the surrounding environment displayed on the image display means. Since the destination of the body can be set, the moving body can be moved even if the position of the moving body is unknown, and the moving body can be moved to a position where the position of the moving body can be determined.

[Position estimation system configuration]
FIG. 2 is a block diagram of components required for the position estimation system 1 of a moving body for realizing the present invention. In FIG. 2, the position estimation system 1 includes a mobile body 2 including a mobile unit 21 and an environmental information acquisition unit 22, an operator terminal device 3 including an image display unit 31 and an input unit 32, and a mobile unit position estimation unit 4. , Includes. Further, the moving body 2 may include an image information acquisition means 23, a self-position estimation means 24, and a movement control means 25, and may include a storage means and a communication means (not shown). The operator terminal device 3 is provided separately from the mobile body 2, and may further include an operating means 33, or may include a storage means and a communication means (not shown). The mobile body position estimation means 4 may be provided in the mobile body 2, may be provided in the terminal device 3, or may be provided in hardware other than those. In FIG. 2, the lines connecting the elements mean that they are connected so that information can be provided to each other or to one side, and even if they are mounted inside the same device and connected by an internal bus, they are separated. This includes cases where the device is mounted on the device and connected by a wired or wireless network.

The moving body 2 has at least a moving means 21 for moving the moving body and an environmental information acquisition means 22 for acquiring environmental information around the moving body 2. The moving body 2 further has an image information acquisition means 23 for acquiring an image of the surrounding environment of the moving body 2, a self-position estimation means 24 for estimating a self-position by collating shape data with map information, and the like. Movement control means 25 that moves to the destination using the self-position estimated by the self-position estimation means, and storage means (shown) in which various information (shape data, image data, map information, program, destination coordinates, etc.) is recorded. It may have a wireless communication means (not shown) capable of communicating with an external network. The mobile body 2 may normally patrol a preset fixed route. In this case, for example, by making the moving body able to grasp its own position on the map, a preset route may be patrolled, or the surrounding wall, ceiling, floor, etc. may be circulated without having map information. By guiding the course by means for guiding the moving body 2, for example, a magnetic tape, a transmitter, or the like, a preset route may be patrolled. Further, the mobile body 2 may move autonomously by calculating a route to the set destination by setting a destination from an operator, a server, or the like in a normal state. Further, the moving body 2 may always provide the shape data to the terminal device so that the position estimation system of the moving body can be used in the terminal device 3, but the self-position estimation is performed due to an obstacle or the like in the surroundings. The support signal and the shape data may be transmitted to the terminal device 3 only when a problem occurs, and the position estimation process may be started in the terminal device 3. The mobile body 2 includes, for example, an automatic guided vehicle, a manned vehicle, a mobile industrial robot, a mobile robot, a mobile service robot, and an unmanned aerial vehicle (including a drone).

The moving means 21 moves the moving body 2, and includes, for example, wheels, tracks, a multi-legged walking device, a propeller, and the like, and further includes driving means (motor, engine, etc.) for driving them. By controlling the output of each moving means 21 by the moving control means 25, the moving body 2 can be moved in a predetermined direction.

The environmental information acquisition means 22 is a sensor that acquires the shape of the environmental information around the moving body 2, and can at least detect the shape of the surrounding obstacle. The environmental information acquisition means 22 may be any means that can detect the existence and position (relative position or absolute position) of the obstacle, and for example, measure the distance and direction to the obstacle existing in the surroundings. By doing so, the shape data of the surroundings is acquired. Examples of the environmental information acquisition means 22 include LiDAR (Light Detection and Ranging or Laser Imaging Detection and Ranging) using laser light, a range sensor, a stereo camera using a plurality of cameras, a pattern projection camera, and a radio wave (millimeter wave). ) Includes radars, magnetic sensors, etc. The environmental information acquisition means 22 is provided on the moving body 2 and can acquire at least a part of the environmental information around the moving body 2. For example, the environmental information within a predetermined angle range centered on the moving body 2. Can be obtained. It is preferable to be able to acquire the environmental information of the entire circumference (360 °), but at least the environmental information in front of the moving body in the moving direction is acquired. At least a part of the acquired shape data can be provided to the terminal device 3. Further, it is preferable that the environmental information acquisition means 22 can acquire not only two-dimensional (2D) environmental information but also three-dimensional (3D) environmental information. The environmental information acquired by the environmental information acquisition means 22 varies depending on the type of the sensor, but includes the distance to the measurement point of the sensor. For example, in the case of LiDAR, micropulse laser light sequentially scans within a predetermined angle range, receives the light reflected by an obstacle and returns, measures the time until it returns, and reaches the obstacle. To get the distance and direction of. Here, the measurement point is the position of an obstacle hit by the irradiated laser beam, and the shape data is a point cloud of the measurement point. In the case of a stereo camera, the difference in position (parallax) in each image and the distance between multiple cameras for a part of a specific subject (area of one pixel or several pixels) included in the images taken by multiple cameras. It is possible to recognize the distance and shape of a specific subject from the distance between the two. Here, a part of the subject for which the parallax is obtained in each pixel is a measurement point. In a stereo camera, a range in which images from a plurality of cameras overlap is a predetermined angle range, and 3D environmental information can be acquired. A plurality of cameras may be provided so as to be movable (rotatable). In addition, even with a normal camera, by attaching a marker (figure, character, shape, etc.) of a known size to an obstacle, the presence of the obstacle can be seen from the number of dots (size) and angle of the marker in the image information. And the position can be detected. Further, a camera such as an infrared camera that can detect the distance may be used. When a camera (including a stereo camera) is adopted as the environmental information acquisition means 22, it can also be used as the image information acquisition means 23.

The image information acquisition means 23 is a camera that acquires at least an image of the surroundings of the moving body 2, and for example, a CCD camera, a CMOS sensor, or the like can be used. It is preferable that the image information acquisition means 23 can acquire at least an image (still image or moving image) in front of the moving body 2 in the moving direction, and can acquire an image of the entire measurement range of the environmental information acquisition means 22. It is preferable to include a movable portion that can change the imaging direction of the 23. The image information acquisition means 23 can acquire image data of an obstacle, determine the obstacle by the image recognition means, or display the obstacle on the image display means 31 of the terminal device 3 so that the operator can confirm the obstacle. However, if a high-definition 3D LiDAR or the like is used as the environmental information acquisition means 22, it is possible to determine or confirm an obstacle based on the pattern of measurement points, and the environmental information acquisition means 22 determines or confirms an obstacle. If possible, the image information acquisition means 23 may not be provided. Further, if the camera is used as the environmental information acquisition means 22, it can also be used as the image information acquisition means 23. The image information acquisition means 23 is preferably provided on the moving body 2, but may be provided on the outside of the moving body 2, for example, on the surrounding wall, ceiling, floor, or the like. When the moving body 2 is provided with the image information acquiring means 23, the image information acquiring means 23 may be configured such that the camera is installed in a movable portion that can rotate in the circumferential direction, for example. It is preferable that the image pickup direction of the image information acquisition means 23 is to acquire an image of the measurement point in cooperation with the scanning of the measurement point of the environment information acquisition means 23. The imaging timing may be always imaging, or imaging may be performed at a predetermined timing. When always taking an image, the amount of image data becomes enormous, which puts pressure on the data area of the relatively small-capacity storage means mounted on the moving body 2. Further, the acquired image data can be provided to the terminal device 3 via the network, but if the acquired image data is always provided, a huge amount of image data is always transmitted and received, which causes a problem that the communication line is congested. Therefore, it is preferable to provide at least image information to the terminal device 3 only at a predetermined timing. That is, when it is necessary for the operator to make a judgment such as losing the self-position due to an obstacle in the surroundings by storing the image data in the storage means of the moving body 2 without transmitting the image data during normal operation. Alternatively, when a signal requesting an image is received from the terminal device 3, it is preferable to transmit the image data. If the storage capacity of the moving object is small, the oldest image may be automatically deleted. Further, in the case of taking an image at a predetermined timing without always taking an image, for example, image data may be acquired at the timing when an obstacle or the like blocking a moving path is detected by the environmental information acquisition means 22. When the image information acquisition means 23 is provided outside the moving body 2, for example, a surveillance camera may be used as the image information acquisition means 23.

The self-position estimation means 24 estimates the position of the self provided in the moving body 2, is realized by executing a position estimation program, and may be realized as a part of the function of the control means. The self-position estimation means 24 is not particularly limited as long as the self-position can be specified, but the shape data acquired by the environmental information acquisition means 22 is collated with the map information prepared in advance, and the collation result is obtained. The self-position of the moving body may be calculated based on. Other self-position estimation means include GPS identification, communication antenna and base station identification, and identification by displaying a symbol indicating the position on the surrounding wall, ceiling, and floor and recognizing it as an image. You may do it.

The movement control means 25 is realized by executing a movement control program, and controls the output of each movement means 21 along a route to a destination to move the moving body 2 to a predetermined position (destination). When a plurality of motors are provided in the moving body 2 as the moving means 21, the movement control means 25 may control the rotation angle and the rotation speed of each motor, and may control the current value supplied to each motor. You may control it. The movement control means 25 is preferably provided on the moving body 2, but may be provided on the outside of the moving body 2, for example, on the terminal device 3.

The operator terminal device 3 is provided separately from the moving body 2, and has an image display means 31 capable of superimposing and displaying map information including shape data and at least a part of the moving range of the moving body, and an image display means. Includes an input means 32 capable of moving and rotating at least one of the shape data and the map information displayed in the above relative to the other. The terminal device 3 includes an operating means 33 capable of manipulating the movement of the moving body, and a moving body position estimating means 4 for estimating the position and orientation of the moving body from the amount of movement rotation operated by the input means, if necessary. Although not shown, storage means in which various information (shape data, image data, map information, programs, destination coordinates, etc.) is recorded, wireless communication means capable of communicating with an external network, and environmental information acquisition means may be used. Includes a map creating means for creating map information in the vicinity of the moving body from the shape data provided from 22, a route calculating means for calculating the route of the moving body to the destination based on the map information, and a control means for controlling each means. You may go out. At least a part of the shape data is provided to the terminal device 3 from the environmental information acquisition means 22. Further, at least a part of the image data may be provided from the image information acquisition means 23. The terminal device 3 is a device operated by a human (operator), displays shape data and map information on the image display means 31, and the shape data acquired by the environmental information acquisition means 22 from the current location of the moving body 2 is a map. The operator can be made to determine which position of the information corresponds to, and at least one of the shape data and the map information displayed on the image display means 31 by the input means 32 with respect to the other so as to overlap the corresponding positions. The movement rotation operation can be performed relatively, and the position of the moving body can be estimated by providing the moving body position estimation means 4 with the amount of movement rotation to the overlapping position. Further, the terminal device 3 may allow the operator to operate the operation means 33 to freely move the moving body 2, and in this case, the image data from the image acquisition means 23 is displayed on the image display means 31. The operator may be able to confirm the status of the destination. Further, the terminal device 3 can set the destination of the moving body in the shape data, image data, or map information of the surrounding environment displayed on the image display means 31 via the input means 32, and sets the destination to the operator. Then, the moving body may be moved to the destination based on the shape data of the surrounding environment of the environmental information acquisition means. The terminal device 3 may include, for example, a computer, a laptop computer, a personal digital assistant (including a smartphone, a tablet terminal), etc., and the terminal device 3 may be realized by one hardware, or a plurality of hardware (for example, a plurality of hardware). , The portable information terminal is used as an image display means, an input means, a server is used as a moving body position estimation means, a storage means, a map creation means, a route calculation means, a control means, etc.) to realize the terminal device 3. May be good. Further, a plurality of mobile bodies 2 may be connected to one terminal device 3. In the present invention, it is sufficient for a human to support only when the moving body 2 recognizes an obstacle that affects the route calculation, and it is possible for one person to support a plurality of moving bodies.

"Map information" indicates the placement of obstacles, etc. within the moving range of a moving object, and incorporates existing maps such as road maps, topographic maps, building floor plans, and architectural drawings in advance. It may be created, or the moving range of the moving object may be acquired in advance by performing self-position estimation and environment map creation at the same time by SLAM (Simultaneous Localization and Mapping) using an environmental information acquisition means, or others. Basic map information created by the moving object or other computer may be used. The map information may be stored, for example, in the storage means of the moving body, may be stored in the storage means of the terminal device, or may be stored in the storage server. The map information preferably includes at least information to the destination in order to calculate the route by the route calculation means, and the map information may be sequentially accumulated and updated by the environmental information acquisition means while moving. As map information, a two-dimensional or three-dimensional map is divided into a plurality of cells (typically, a grid having a uniform width, a grid of voxels, etc., but is not limited to the grid), and for each cell, a moving object is used. Costs related to mobility may be set. The map information displayed on the image display means 31 includes at least a part of the moving range of the moving body. For example, the map information in a certain range may be displayed centering on the last self-position recognized before the moving body 2 loses its self-position.

The image display means 31 is a means for displaying various information, and includes, for example, a liquid crystal display (LCD), an organic EL display, a projector, and the like. If a touch panel type display is used as the image display means 31, it can also be used as the input means 32 and the operation means 33. Further, a plurality of image display means 31 may be provided corresponding to the plurality of moving bodies 2, or information of the plurality of moving bodies 2 may be simultaneously or sequentially displayed on one image display means 31. .. The image display means 31 displays at least shape data and map information of the surrounding environment of the moving body acquired from the moving body 2 (for example, stored in a storage means (not shown)), and further acquired from the moving body 2. Image data of the surrounding environment of the moving object may be displayed. In addition, other information may be displayed, for example, a message may be displayed to inform the operator that the support signal from the mobile body 2 has been received. The shape data and the map information may be displayed separately at the beginning or may be displayed overlapping from the beginning, but when one of them is moved and rotated by the input means, the shape data and the map information may be displayed on the other. It can be displayed in layers. In this way, since the shape data and the map information can be moved and / or rotated in a superposed state, the operator can easily grasp the map information corresponding to the shape data. In particular, it is preferable that the shape data and the map information have different colors so that they can be identified even if they are overlapped.

The input means 32 is a means capable of moving and rotating the shape data and map information displayed on the image display means 31 on the screen, and is, for example, a keyboard, a pointing device (mouse, touch pad, etc.), and a touch panel (touch screen). Includes), including controllers, etc. For the operation of the shape data and the map information by the input means 32, for example, the mouse may select the shape data or the map information to be operated and move the mouse as it is, or the shape data and / or the map information may be close to the operation. You may select and rotate the rotation handle displayed by using the mouse. Also, for example, on the touch panel, touch the shape data or map information that you want to operate with one finger and move it as it is, or touch the screen with the other finger to move it in the rotation mode. And may be rotated by moving one or both fingers. You may also use the keyboard cursor to move or rotate. The input means 32 may be capable of enlarging or reducing the shape data and / or the map information. Further, the input means may have an automatic alignment function. The automatic alignment function may align the entire shape data with the closest part of the map information having a high degree of matching, or may be a part of the components such as lines and corners of the shape data. On the other hand, it may be aligned with the components that are close to each other among the components having a high degree of matching of the map information. The reason for aligning with the close parts and components is that the operator operates to bring the shape data and the corresponding part of the map information close to each other. For example, if there are multiple parts similar to the shape data in the map information, or if the shape data is rotationally symmetric and it is unclear in which direction the moving body is oriented, the computer cannot determine the position of the moving body. Sometimes. In the present invention, the operator determines the portion of the map information corresponding to the shape data, and moves and rotates the portion so that the portion and the shape data are relatively close to each other. Therefore, the degree of agreement with the entire shape data is high. If the automatic alignment function for aligning to the closest part of the map information is executed, the final alignment can be performed automatically. Further, for example, when the shape data includes a shape such as an obstacle that is not included in the map information, the map information portion having a high degree of matching may not be found when the entire shape data is collated. In this case, the automatic alignment method that aligns to the part of the map information that has a high degree of matching with the entire shape data cannot be used, but the map information is applied to some components such as lines and corners of the shape data. Automatic alignment is possible if the method is to align to the components that are close to each other among the components with a high degree of matching. The automatic alignment function can assist the operator's operation. If the operator automatically moves and / or rotates to the corresponding position by the automatic alignment function without the operator's operation, the operator may provide the moving rotation amount to the moving body position estimation means after the final confirmation. Often, if the automatic alignment function cannot align, or if the alignment is in the wrong position, the operator may intervene in the movement rotation operation. Further, the input means may be able to set the destination of the moving object in the shape data, image data or map information of the surrounding environment displayed on the image display means, for example, the shape data, the image data or the map information. The destination may be set by selecting one point or one area with a pointing device or a finger.

The operation means 33 can control the movement means of the moving body to operate the movement of the moving body, and is, for example, a keyboard, a pointing device (mouse, touch pad, etc.), a touch panel (including a touch screen), a controller, or the like. , And the input means 32 may also be used. As the operating means 33, a key, a button, a stick, or the like corresponding to the moving direction of the moving body may be operated to move the moving body in the operated direction, or the shape data displayed on the image display means may be used. The moving body may be operated by setting the destination in the image data or the map information.

The moving body position estimation means 4 estimates the position of the moving body based on the amount of movement rotation by the input means 32, and is realized by executing a position estimation program. For example, a CPU (Central Processing Unit) and a work. It is realized by a memory for use or by an integrated circuit. The mobile body position estimation means 4 may be provided in the mobile body 2, may be provided in the terminal device 3, or may be provided in hardware other than those. The moving body position estimating means 4 estimates the position and orientation of the moving body from the arrangement of the moving body at the time of acquiring the shape data on the map information after the movement rotation operation is relatively performed by the input means. This is estimated from the coordinates of the map information part when the shape data is arranged so as to overlap the corresponding map information part by the movement rotation operation of the input means 32 and the direction of the moving body with respect to the shape data at that time. including. Further, the amount of movement rotation by the input means 32 includes the amount of movement and / or rotation by the automatic alignment function in addition to the amount operated by the operator.

FIG. 3 is an example of the hardware that realizes the position estimation system 1 of the moving body of the present invention. The position estimation system 1 in the present embodiment includes an autonomous mobile body 2 capable of calculating a route to a destination, an operator terminal device 3 including a mobile body position estimation means, a mobile body 2, and a terminal device. 3 may include a network 5 to which the 3 is connected, and may include a storage server 6 and a management server 7 that can be connected via the network 5, if necessary. The position estimation system 1 of the moving body of the present embodiment loses its own position in the autonomous moving body 2 capable of calculating the route to the destination, and cannot be restored by the moving body 2 and the management server 7, etc. In addition, it is possible to connect to the operator terminal device 3 via the network 5, estimate the position of the moving body 2 with the support of the operator, and return to autonomous movement. Further, the information acquired by the mobile body 2 is sent to the terminal device 3, the status of the mobile body 2 is confirmed by the terminal device 3, and the information created by the terminal device 3 is sent to the mobile body 2, and such information is used. The moving body 2 can be controlled.

[Shape data acquisition process]
FIG. 4A is map information 40 showing the floor plan of the moving range of the moving body, and FIGS. 4B, 5A and 5B are environmental information acquisition means and shape data of the surrounding environment. It is a figure which shows the work of acquiring 50. The map information 40 may be created by incorporating an existing map such as a floor plan or an architectural drawing of a building in advance, or SLAM (Simultaneous Localization and Mapping) using an environmental information acquisition means to determine the moving range of a moving object in advance. ) May be acquired by performing self-position estimation and environmental map creation at the same time, or basic map information created by another moving object or another computer may be used. As shown in FIG. 4A, in the map information 40, one or a plurality of continuous square obstacles 42 are arranged in a room composed of a horizontally long substantially quadrangular wall 41, and the map information 40 is located on the lower side. A part of the wall 41 is slightly projected, and the door 43 is arranged there. The mobile body 2 is located slightly above the center of the room. The mobile body 2 has four wheels as the moving means 21 and LiDAR as the environmental information acquisition means 22. As shown in FIG. 4B, the moving body 2 sequentially irradiates the micropulse laser beam 22a from the environmental information acquisition means 22 (LiDAR) at predetermined step angles, and scans the surrounding environment with the laser beam. Acquire shape data of environmental information. In FIG. 4B, scanning is performed in all directions, and each laser beam 22a is irradiated to the surrounding environment (wall, obstacle, door), and reflected light or scattered light (shown) at each measurement point 22b. The distance and direction to each measurement point 22b are measured by generating (1), detecting the returned reflected light or scattered light with the LiDAR detector, and measuring the time from irradiation to returning. .. FIG. 5A shows the corresponding positional relationship between each measurement point 22b excluding the laser beam 22a and the map information 40, and FIG. 5B shows only the measurement point 22b, which is the measurement point 22b. The point cloud is the shape data 50. As shown in FIG. 5A, a point cloud of measurement points indicating the position of a part of the wall 41 of the room 40 facing the moving body 2, the position of a part of the obstacle 42, and the position of the door 43. Is obtained, and polar coordinates (distance, angle) centered on the environmental information acquisition means 22 can be obtained for each measurement point 22b. The polar coordinates of each measurement point may be converted into Cartesian coordinates. As shown in FIG. 5B, from the shape data 50, it is possible to grasp the arrangement of the moving body relative to the position and orientation of the environmental information acquisition means 22, and the distance and direction to each measurement point 22b.

[Position estimation process]
6 and 7 are diagrams for explaining the position estimation process of the moving body, and show the screen of the image display means 31 of the terminal device 3. The mobile body 2 acquired the shape data 50 shown in FIG. 5 (B), but for some reason misrecognized its own position and orientation (arrangement), and failed to estimate its own position. Therefore, a support signal was sent to the terminal device 3. And the shape data 50 was provided to request the execution of the moving body position estimation process. FIG. 6A is a screen in which the shape data 50 acquired from the moving body 2 and the map information 40 are superimposed and displayed. However, since the arrangement 61 of the moving body 2 is erroneously recognized, the shape data 50 and the map are displayed. Information 40 does not match. FIG. 6B shows a state in which the operator selects the shape data 50, moves it diagonally upward to the right as shown by the arrow 62, and further rotates it slightly clockwise as shown by the arrow 63 to change the direction. It is a screen showing. In FIG. 6B, the shape data 50 before the movement is shown by a dotted line, and the shape data 60 after the movement rotation operation is shown by a solid line. The dotted line measurement point 22b moves to the solid line measurement point 22c by the movement rotation operation, and coincides with the wall 41 of the map information 40. FIG. 7 is a screen in which the shape data 60 after the movement rotation operation and the map information 40 are superimposed and displayed. The shape data 60 after the movement rotation operation coincides with the wall 41, the obstacle 42, and the door 43 of the map information 40, and the correct arrangement 64 of the moving body 2 can be grasped by the movement rotation operation. Then, the moving body position estimation means 4 specifies the coordinates of the shape data 60 after the moving rotation operation based on the moving amount 62 and the rotation amount 63, and arranges the moving body 64 relative to the shape data 60. It is calculated and provided to the moving body 2, and the moving body 2 can return to the autonomous operation with the correct arrangement 64 as its own position.

FIG. 8A is a diagram showing measurement points 22d, 22e, and 22f when the laser beam is scanned a plurality of times from the same position by the environmental information acquisition means 22 (LiDAR), and FIG. 8B is a diagram thereof. The shape data 80. In FIG. 8A, the black circle is the measurement point 22d by the first scan, the square is the measurement point 22e by the second scan, and the triangle is the measurement point 22f by the third scan. The accuracy of the shape data can be improved by scanning a plurality of times with the laser beam. Further, the shape data 80 of FIG. 8B includes a double circle 81 indicating a position and an arrow 82 indicating an direction as arrangement information of the moving body 2.

1 Position estimation system 2 Mobile 3 Operator terminal device 4 Mobile position estimation means 21 Mobile means 22 Environmental information acquisition means 23 Image information acquisition means 24 Self-position estimation means 25 Movement control means 33 Operation means 31 Image display means 32 Input means

Claims (11)

A mobile body position estimation system including a mobile body including a mobile body, an operator terminal device provided separately from the mobile body, and a mobile body position estimation means.
The moving body includes an environment information acquisition means for acquiring the shape of the surrounding environment of the moving body, and can provide shape data regarding the shape of the surrounding environment acquired by the environment information acquisition means to the operator terminal device.
The operator terminal device has an image display means capable of superimposing and displaying the shape data and map information including at least a part of the movement range of the moving body, and the shape data and the map displayed on the image display means. Includes input means capable of moving and / or rotating at least one of the information relative to the other.
The moving body position estimation means is a moving body position estimation system capable of estimating the position and orientation of the moving body based on the movement amount and / or the rotation amount operated by the input means. The moving body position estimating means estimates the position and orientation of the moving body from the arrangement of the moving body at the time of acquiring the shape data on the map information after being relatively moved and / or rotated by the input means. The position estimation system for a moving body according to claim 1. The shape data includes the arrangement information of the moving body at the time of acquiring the shape data.
The image display means displays the arrangement information of the moving body at the time of acquiring the shape data together with the shape data, together with the shape data.
The input means can move and / or rotate relative to the map information as a unit without changing the positional relationship between the shape data and the arrangement information of the moving body, claim 1 or 2. The moving body position estimation system according to 2. The input means has an automatic alignment function that automatically executes a movement or rotation operation so as to have a positional relationship with a high degree of matching with respect to the shape data or the map information operated by the input means. Item 6. The moving body position estimation system according to any one of Items 1 to 3. The moving body uses a self-position estimating means for estimating a self-position by collating the shape data with the map information, and a self-position estimated by the self-position estimating means by controlling the moving means. Control means to move to the ground, including more
The mobile body transmits a support signal requesting the start of the position estimation process of the mobile body to the operator terminal device when a problem occurs in the self-position estimation by the self-position estimation means. 4. The position estimation system for a moving body according to any one of 4. The position estimation system for a moving body according to claim 5, wherein the control means controls movement by using the position and orientation of the moving body acquired from the moving body position estimating means. The moving body further includes an image information acquisition means for acquiring an image of the surrounding environment of the moving body, and can provide image data of the surrounding environment acquired by the image information acquisition means to the operator terminal device.
The mobile control system according to any one of claims 1 to 6, wherein the operator terminal device can display image data of the surrounding environment on the image display means. The position estimation system for a moving body according to claim 7, wherein the operator terminal device includes an operating means capable of controlling the moving means of the moving body to operate the movement of the moving body. The position estimation of the moving body according to claim 7 or 8, wherein the input means can set the destination of the moving body in the shape data, image data, or map information of the surrounding environment displayed on the image displaying means. system. It is an operator terminal device provided separately from the moving body including the moving means, and is a terminal device for an operator.
A shape data acquisition unit that acquires shape data related to the shape of the surrounding environment acquired by the environmental information acquisition means for acquiring the shape of the surrounding environment of the moving body, and a shape data acquisition unit.
A map information acquisition unit that acquires map information including at least a part of the movement range of the moving object, and a map information acquisition unit.
An image display means capable of superimposing the shape data and the map information, and
An input means capable of moving and / or rotating at least one of the shape data and the map information displayed on the image display means relative to the other.
The moving amount and / or the rotating amount operated by the input means is provided to the moving body position estimating means capable of estimating the position and orientation of the moving body from the moving amount and / or the rotating amount operated by the input means. Possible terminal device for operators. The position of a moving body including a moving body including a moving means and an environmental information acquisition means, an operator terminal device provided separately from the moving body and including an image display means and an input means, and a moving body position estimation means. It ’s an estimation method.
A step of acquiring shape data regarding the shape of the surrounding environment of the moving body by the moving body environment information acquisition means, and
A step of providing the shape data to the operator terminal device,
A step of superimposing and displaying map information including at least a part of the shape data and the moving range of the moving body on the image display means of the terminal device for an operator.
A step of moving and / or rotating at least one of the shape data and the map information displayed on the image display means relative to the other by the input means of the operator terminal device.
A step of providing the moving amount and / or the rotation amount operated by the input means to the moving body position estimation means, and
A step of estimating the position and orientation of the moving body from the moving amount and / or the rotation amount by the moving body position estimating means.
A method for estimating the position of a moving object, including.

Images