JP2020060496A - Information processing device - Google Patents

Abstract

To provide an information processing device with which it is possible to generate point group data while effectively utilizing multiple kinds of map information.SOLUTION: An information acquisition unit 2 acquires imaging information from an imaging unit 20 mounted in a mobile entity 10, acquires position information that indicates the imaging position of imaging information, and acquires map information from a map information storage unit 30 that stores one or multiple kinds of map information in accordance with a region. As the information acquisition unit 2 acquires the kind of map information that corresponds to the position information that indicates the imaging position, it is possible for a generation unit 3 to generate point group data while effectively utilizing the multiple kinds of map information.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an information processing device.

  Conventionally, a method (SLAM; Simultaneous Localization and Mapping) of simultaneously performing self-position estimation and environment map creation in a moving body is known. As a kind of such SLAM, a method using a video camera instead of a laser device (VSLAM; Visual SLAM) has been proposed (for example, refer to Patent Document 1). In the VSLAM described in Patent Document 1, the cost is reduced by using a video camera.

JP, 2014-222550, A

  In the VSLAM described in Patent Document 1, feature points are extracted from each image (frame) captured by a video camera, and self-position estimation and environment map creation are performed based on the change of the feature points between the images. To do. At this time, although various feature points are extracted from the image, not all the extracted feature points are suitable for self-position estimation and environment map creation. Further, although the point cloud data included in the generated environment map is used for self-position estimation and the like, the constituent points forming the point cloud data may include difficult-to-use ones.

  Therefore, a method of removing unnecessary feature points or constituent points by referring to existing map information when generating the point cloud data can be considered. However, there are various types of existing map information stored in the storage device, and not all types of map information are created for all areas. Therefore, if the map information is simply used when the point cloud data is generated, the map information created is selected over the entire area where the mobile body can travel, and it is difficult to use other map information. there were.

  Therefore, an object of the present invention is to provide an information processing apparatus that can generate point cloud data while effectively using a plurality of types of map information.

  In order to solve the above-mentioned problems and achieve the object, an information processing apparatus according to the present invention according to claim 1 is an imaging information acquisition unit that acquires imaging information from an imaging unit mounted on a mobile body, and the imaging information. A position information acquisition unit that acquires position information indicating the imaging position of the map information, and a map information acquisition unit that acquires the map information from a map information storage unit that stores one or more types of map information according to the area; And a generation unit that generates point cloud data based on the information and the map information, and the map information acquisition unit acquires the map information of a type according to the position information.

  An information processing apparatus according to the present invention as set forth in claim 3, wherein an imaging information acquisition unit that acquires imaging information from an imaging unit mounted on a moving body, and position information acquisition that acquires position information indicating an imaging position of the imaging information. A map information acquisition unit that acquires the map information from a map information storage unit that stores one or a plurality of types of map information according to the area and the region, and generates point cloud data based on the imaging information and the map information. And a generating unit for generating the point cloud data based on the map information of a type corresponding to the position information and the imaging information.

  An information processing method according to the present invention as set forth in claim 7, wherein an imaging information acquisition step of acquiring imaging information from an imaging unit mounted on a moving body, and position information acquisition of acquiring position information indicating an imaging position of the imaging information. Steps, and a map information acquisition step of acquiring the map information from a map information storage unit that stores one or more types of map information according to the region, and point cloud data is generated based on the imaging information and the map information. And a step of generating the map information. In the map information acquisition step, the map information of a type corresponding to the position information is acquired.

  The information processing method of the present invention according to claim 8 is an imaging information acquisition step of acquiring imaging information from an imaging unit mounted on a moving body, and position information acquisition of acquiring position information indicating an imaging position of the imaging information. Steps, and a map information acquisition step of acquiring the map information from a map information storage unit that stores one or more types of map information according to the region, and point cloud data is generated based on the imaging information and the map information. In the generating step, the point cloud data is generated based on the map information of a type corresponding to the position information and the imaging information.

It is a block diagram which shows the outline of the information processing apparatus which concerns on the Example of this invention. FIG. 3 is a plan view showing an example of a road in an area where a mobile body equipped with the information processing device can travel.

  Embodiments of the present invention will be described below. An information processing apparatus according to an embodiment of the present invention includes an imaging information acquisition unit that acquires imaging information from an imaging unit mounted on a moving body, a position information acquisition unit that acquires position information indicating an imaging position of the imaging information, A map information acquisition unit that acquires map information from a map information storage unit that stores one or a plurality of types of map information depending on the area, and a generation unit that generates point cloud data based on the imaging information and the map information. Prepare The map information acquisition unit acquires the map information of a type corresponding to the position information.

  According to the information processing apparatus of the present embodiment as described above, the map information acquisition unit acquires the map information of the type corresponding to the position information indicating the imaging position, so that not only the map information created in the entire region It is also possible to acquire locally created map information. For example, when the first map information is created and stored in the entire area, the second map information is created and stored in the predetermined first area, and the imaging position is included in the first area. In the above, the first map information and the second map information are acquired, and when the image pickup position is not included in the first area, only the first map information may be acquired.

  In this way, by acquiring the map information according to the imaging position, it is possible to generate the point cloud data while effectively using the plurality of types of map information. The imaging information acquisition unit, the position information acquisition unit, the map information acquisition unit, and the generation unit may not be mounted on the same target and may be mounted on different devices or apparatuses. That is, each component of the information processing device may span a plurality of physically separated devices.

  When the map information acquisition unit acquires a plurality of types of map information, the generation unit preferably generates a plurality of point cloud data based on each of these pieces of map information. The type and accuracy of information included in the map information, such as feature information, road shape information, road connection information, etc., varies depending on the type. By generating a plurality of point cloud data based on each of a plurality of map information, the point cloud data can be selectively used according to the purpose of use (that is, what information is required to be accurate).

  An information processing apparatus according to another embodiment of the present invention includes an imaging information acquisition unit that acquires imaging information from an imaging unit mounted on a moving body, and a position information acquisition unit that acquires position information indicating an imaging position of the imaging information. And a map information acquisition unit that acquires map information from a map information storage unit that stores one or a plurality of types of map information according to a region, and a generation unit that generates point cloud data based on imaging information and the map information. And The generation unit generates the point cloud data based on the map information of the type corresponding to the position information and the imaging information.

  According to the information processing apparatus of the present embodiment, the generation unit generates the point cloud data based on the map information of the type corresponding to the position information indicating the image pickup position and the image pickup information. Similar to the information processing apparatus according to the embodiment, it is possible to generate point cloud data while effectively using a plurality of types of map information.

  In any of the information processing devices of the above-described embodiments, the generation unit removes the predetermined feature points extracted in each frame of the imaging information based on the map information and removes the predetermined constituent points forming the point cloud data. It is preferable to execute at least one of the above. As a result, of the feature points of each frame of the imaged information, those that are not suitable for generating point cloud data are removed, and among the constituent points of the point cloud data, those that are difficult to use for self-position estimation are removed. The accuracy of the point cloud data can be improved.

  In any of the information processing apparatuses of the above-described embodiments, the map information acquisition unit may acquire map information from a map information storage unit that stores at least one of a city map, a line map, and an aerial map as map information. preferable. It is sufficient for the line map to include not only the shape of the road but also information about the connection between the roads and the direction of travel.The city map not only describes the shape of the road but also the position, size, shape, and attributes of the features. It is only necessary to include information, and the aerial chart may include information that is difficult to be expressed by letters or numerical values (for example, the detailed shape of the building).

    It is preferable that the information processing apparatus of any of the above embodiments further includes an output unit that outputs the point cloud data. Thereby, the point cloud data can be utilized at the output destination.

  An information processing method according to an embodiment of the present invention includes an imaging information acquisition step of acquiring imaging information from an imaging unit mounted on a moving body, a position information acquisition step of acquiring position information indicating an imaging position of the imaging information, A map information acquisition step of acquiring map information from a map information storage unit that stores one or a plurality of types of map information depending on the area, and a generation step of generating point cloud data based on the imaging information and the map information. Including. In the map information acquisition step, the type of map information corresponding to the position information is acquired. According to such an information processing method, the point cloud data can be generated while effectively using a plurality of types of map information by acquiring the map information according to the imaging position in the acquisition step.

  An information processing method according to another embodiment of the present invention includes an imaging information acquisition step of acquiring imaging information from an imaging unit mounted on a moving body, and a position information acquisition step of acquiring position information indicating an imaging position of the imaging information. And a map information acquisition step of acquiring map information from a map information storage unit that stores one or a plurality of types of map information according to the area, and a generation step of generating point cloud data based on the imaging information and the map information. ,including. In the generation step, the point cloud data is generated based on the map information of the type corresponding to the position information and the image pickup information. According to such an information processing method, in the generating step, the point cloud data is generated based on the map information of the type corresponding to the position information indicating the image capturing position and the image capturing information. It is possible to generate point cloud data while effectively utilizing the.

  Further, it may be an information processing program that causes a computer to execute the above-described information processing method. By doing so, it is possible to generate point cloud data while effectively utilizing a plurality of types of map information using a computer.

  Further, the above-mentioned information processing program may be stored in a computer-readable recording medium. By doing so, the program can be distributed as a single unit in addition to being installed in the device, and the version can be easily upgraded.

  Hereinafter, examples of the present invention will be specifically described. As shown in FIG. 1, the information processing device 1 of the present embodiment is a VSLAM device mounted on a moving body 10 and includes an information acquisition unit 2, a generation unit 3, and an output unit 4. The moving body 10 is, for example, a vehicle, and is equipped with an imaging unit 20, a map information storage unit 30, a self-position estimation unit 40, and a travel information sensor 50.

  The imaging unit 20 is, for example, a video camera, and is arranged so as to image the front side of the moving body 10 in the traveling direction. The moving image captured by the image capturing unit 20 is composed of a plurality of image capturing information (frames) and is transmitted to the information acquisition unit 2. The image capturing section may be capable of continuously capturing still images at predetermined time intervals.

  The map information storage unit 30 is composed of, for example, a hard disk or a non-volatile memory, and stores a line map and a city map as existing map information. The line map is created and stored over the entire target area (for example, land in Japan), and the city map is created only in the city area where features such as buildings are gathered at a certain density or more. It is remembered as well.

  As described above, the map information storage unit 30 stores one or two types of map information depending on the area. In other words, the number of types of map information stored in the map information storage unit 30 depends on the position.

  The self-position estimating unit 40 estimates the current position (absolute position) of the moving body 10, and may be, for example, a GPS receiving unit that receives radio waves transmitted from a plurality of GPS (Global Positioning System) satellites. . The current position estimated by the self-position estimation unit 40 is transmitted to the information acquisition unit 2 as position information indicating the image pickup position of the image pickup information by the image pickup unit 20.

  The travel information sensor 50 is a sensor for measuring the displacement amount of the moving body 10, and for example, for measuring the azimuth displacement amount of the moving body 10 and the vehicle speed pulse acquisition unit that acquires the vehicle speed pulse of the moving body 10. The gyro sensor and the acceleration sensor for acquiring the acceleration of the moving body 10. The traveling direction of the moving body 10 is estimated based on the displacement amount acquired by the travel information sensor 50. Therefore, the displacement amount acquired by the travel information sensor 50 is transmitted to the information acquisition unit 2 as traveling direction information indicating the traveling direction of the moving body 10. The travel information sensor may be at least capable of estimating the traveling direction.

  The information acquisition unit 2 functions as an imaging information acquisition unit by acquiring the imaging information from the imaging unit 20 as described above, and the position information acquisition unit by acquiring the position information from the self-position estimation unit 40 as described above. And functions as a map information acquisition unit by acquiring map information from the map information storage unit 30 and acquires traveling direction information from the travel information sensor 50 as described above.

  The generation unit 3 extracts a feature point from the imaging information acquired by the information acquisition unit 2 to generate a two-dimensional feature point group, and further generates point group data based on the plurality of feature point groups. The point cloud data generated by the generation unit 3 may be, for example, three-dimensional point cloud data. It should be noted that the extraction of the feature points may use, for example, an ORB (Oriented fast and Rotated Brief) algorithm or another algorithm.

  The output unit 4 includes a circuit for communicating with a network such as the Internet and a public line, an antenna, and the like, and communicates with the communication unit 101 of the external server 100 to generate information (point cloud data) generated by the generation unit 3. Is transmitted (output). As a result, information is stored in the storage unit main body 102 of the external server 100. The external server 100 is configured to communicate with the plurality of information processing devices 1.

  Here, a specific example of the map information stored in the map information storage unit 30 and a specific example of the method by which the information acquisition unit 2 acquires the map information will be described with reference to FIG. Regarding the area A0 including the city area A1 and the other area A2 around the city area A1, the map information storage unit 30 includes the map information corresponding to the city area A1 and the line map corresponding to the area A0. Is remembered as

  The city map and the line map are managed in different layers. Further, the city map is created only for the city area A1, and has no information for the other areas A2. That is, in the city map layer, the other area A2 is blank. In this way, the map information storage unit 30 stores two types of map information in the city area A1 and one type of map information in the other area A2. Even if a map information layer exists at a certain position, if the content of the layer is blank (or if there is some information, it is insufficient as map information), It is not counted as the number of map information.

  When acquiring the imaging information from the imaging unit 20, the information acquisition unit 2 acquires the position information about the current position from the self-position estimation unit 40. The current position estimated by the self-position estimation unit 40 is the image pickup position of the image pickup unit 20. The information acquisition unit 2 acquires map information corresponding to the position information acquired from the self-position estimation unit 40 from the map information storage unit 30.

  When the position information acquired by the information acquisition unit 2 from the self-position estimation unit 40 is included in the city area A1 (that is, when the imaging position is in the city area A1), the information acquisition unit 2 uses the map information as map information. Acquire city maps and diagrams. When the position information acquired by the information acquisition unit 2 from the self-position estimation unit 40 is included in another area A2 (that is, when the imaging position is in another area A2), the information acquisition unit 2 determines that the map Acquire a diagram as information.

  The generation unit 3 generates point cloud data based on the imaging information, the map information, and the traveling direction information acquired by the information acquisition unit 2. At this time, the generation unit 3 selects, from the plurality of feature points extracted in each frame of the imaging information, the feature points corresponding to, for example, a movable object (another vehicle, a pedestrian, etc.) as map information and traveling direction information. And remove based on. Based on the width, shape, and undulations of the road included in the map information, it is possible to estimate in which range of each frame the road is located, and to move the feature points located on the road area Can be determined to correspond to.

  Further, the generation unit 3 generates point cloud data based on a plurality of feature point groups, and then, among the constituent points forming the point cloud data, feature points corresponding to, for example, a feature (vegetation or the like) that can change with time. Are removed based on the map information and the traveling direction information. When the map information includes vegetation information such as roadside trees, or when the map information is an aerial map and the vegetation can be identified based on the color, the constituent points corresponding to the vegetation can be removed.

  When the imaging position is within the city area A1 and the information acquisition unit 2 acquires the city map and the line drawing, the generation unit 3 generates two types of point cloud data based on the city map and the line drawing, respectively. That is, the generation unit 3 generates the point cloud data based on the city map and independently generates the point cloud data based on the line map. The city map includes not only the shape of the road but also information about the position, size, shape, and attribute of the feature, and feature points and constituent points are removed based on this information. On the other hand, the line map includes not only the shape of the roads but also the connection between the roads (for example, whether or not two roads that intersect each other when viewed from the sky are actually connected) and information about the traveling direction. Based on this information, feature points and constituent points are removed, and information about the route is added to the generated point cloud data.

  Note that the generation unit 3 may execute only one of the above-described feature point removal and component point removal based on the map information. That is, the generation unit 3 may use the map information before generating the point cloud data, or may use the map information after generating the point cloud data. Further, the generation unit 3 may add information (flag) that the reliability is low to these feature points or constituent points without removing the feature points or constituent points.

  Further, in the present embodiment, the information acquisition unit 2 acquires the map information of the type according to the position information, but the acquisition method is not limited to such an acquisition method. That is, the information acquisition unit 2 acquires all types of map information regardless of position information, and the generation unit 3 generates point cloud data based on the type of map information according to the position information and the imaging information. May be.

  The external server 100 acquires and collects the point cloud data generated as described above from the plurality of information processing devices 1. The external server 100 integrates a plurality of collected point cloud data to generate total point cloud data. In addition, when a flag is added to the constituent point of the point cloud data as described above, the reliability of the constituent point may be lowered and the points may be integrated.

  The comprehensive point cloud data generated in this way is used, for example, for estimating the self-position of the moving body. That is, when a feature is detected by an optical sensor (so-called LIDAR; Laser Imaging Detection and Ranging) mounted on the moving body, the self-position of the moving body may be estimated by collating with the comprehensive point cloud data.

  With the above configuration, the information acquisition unit 2 acquires the map information of the type corresponding to the position information indicating the imaging position, so that the generation unit 3 effectively uses the plural types of map information and collects the point cloud data. Can be generated.

  Further, when the information acquisition unit 2 acquires a plurality of types of map information, the generation unit 3 generates a plurality of point cloud data based on each of the map information, so that the point cloud data is used for a purpose (that is, which information Can be used properly depending on whether the accuracy of is required).

  Further, the generation unit 3 removes the predetermined feature points extracted in each frame of the imaging information based on the map information, so that the feature points not suitable for generating the point cloud data can be removed. Further, by removing the predetermined constituent points that form the point cloud data, it is possible to remove constituent points that are difficult to use for self-position estimation and the like. Therefore, the accuracy of the point cloud data can be improved.

  The present invention is not limited to the above-described embodiments, but includes other configurations and the like that can achieve the object of the present invention, and the following modifications and the like are also included in the present invention.

  For example, in the above embodiment, when the information acquisition unit 2 acquires a plurality of types of map information, the generation unit 3 generates a plurality of point cloud data based on each piece of map information. May generate one point cloud data based on a plurality of types of map information. Thereby, the accuracy of the generated point cloud data can be improved.

  Further, in the above-described embodiment, the map information storage unit 30 stores the city map and the line map, but the map information storage unit 30 may store the aerial map. If the aerial chart also includes information that is difficult to represent with characters or numerical values (for example, the detailed shape of the building), point cloud data can be generated based on such information.

  Further, the imaging information acquisition unit, the position information acquisition unit, the map information acquisition unit, and the generation unit that configure the information processing device may not be mounted on the moving body, and may be mounted on different devices or apparatuses. Good. That is, each component of the information processing device may span a plurality of physically separated devices.

  Besides, the best configuration, method, and the like for carrying out the present invention have been disclosed in the above description, but the present invention is not limited thereto. That is, the present invention is mainly illustrated and described mainly with respect to a specific embodiment, but without departing from the technical idea and the scope of the object of the present invention, the shape with respect to the embodiment described above Those skilled in the art can make various modifications in terms of material, quantity, and other detailed configurations. Therefore, the description limiting the shapes and materials disclosed above is described as an example for facilitating the understanding of the present invention, and does not limit the present invention. The description with the names of members excluding some or all of the above limitations is included in the present invention.

1 Information processing device 2 Information acquisition unit (imaging information acquisition unit, position information acquisition unit, map information acquisition unit)
3 generation unit 4 output unit 10 moving body 20 imaging unit 30 map information storage unit

Claims (10)

An imaging information acquisition unit that acquires imaging information from the imaging unit mounted on the moving body;
A position information acquisition unit that acquires position information indicating an imaging position of the imaging information,
A map information acquisition unit that acquires the map information from a map information storage unit that stores one or more types of map information depending on the area;
A generation unit that generates point cloud data based on the imaging information and the map information,
The information processing apparatus, wherein the map information acquisition unit acquires the map information of a type according to the position information.   The said production | generation part produces | generates several said point cloud data based on each of the said map information, when the said map information acquisition part acquired the said map information of multiple types, The claim 1 characterized by the above-mentioned. Information processing equipment. An imaging information acquisition unit that acquires imaging information from the imaging unit mounted on the moving body;
A position information acquisition unit that acquires position information indicating an imaging position of the imaging information,
A map information acquisition unit that acquires the map information from a map information storage unit that stores one or more types of map information depending on the area;
A generation unit that generates point cloud data based on the imaging information and the map information,
The information processing apparatus, wherein the generation unit generates the point cloud data based on the map information of a type according to the position information and the imaging information.   Based on the map information, the generation unit performs at least one of removal of a predetermined feature point extracted in each frame of the imaging information and removal of a predetermined constituent point forming the point cloud data. The information processing apparatus according to any one of claims 1 to 3, characterized in that:   5. The map information acquisition unit acquires the map information from the map information storage unit that stores at least one of a city map, a line map, and an aerial map as the map information. The information processing apparatus according to any one of 1.   The information processing apparatus according to claim 1, further comprising an output unit that outputs the point cloud data. An imaging information acquisition step of acquiring imaging information from an imaging unit mounted on a moving body,
A position information acquisition step of acquiring position information indicating an imaging position of the imaging information,
A map information acquisition step of acquiring the map information from a map information storage unit that stores one or a plurality of types of map information according to an area;
A generation step of generating point cloud data based on the imaging information and the map information,
The information processing method, wherein in the map information acquisition step, the type of map information corresponding to the position information is acquired. An imaging information acquisition step of acquiring imaging information from an imaging unit mounted on a moving body,
A position information acquisition step of acquiring position information indicating an imaging position of the imaging information,
A map information acquisition step of acquiring the map information from a map information storage unit that stores one or a plurality of types of map information according to an area;
A generation step of generating point cloud data based on the imaging information and the map information,
The information processing method, wherein in the generating step, the point cloud data is generated based on the map information of a type corresponding to the position information and the image pickup information.   An information processing program, characterized by causing a computer to execute the information processing method according to claim 7.   A computer-readable recording medium storing the information processing program according to claim 9.

Images