JP2020060497A - Information processing device - Google Patents

Abstract

To provide an information processing device with which it is possible to generate point group data while reducing effects due to a shadow.SOLUTION: Imaging information, sun altitude information, sun azimuth information and imaging direction information are acquired by an information acquisition unit 2. A generation unit 3 specifies a shadow generation direction on the basis of the sun altitude information, sun azimuth information and imaging direction information and extracts constituent points, out of the constituent points constituting the point group data, that are in a row in the shadow generation direction. Thus, it is made possible to estimate the constituent points corresponding to the shadow and generate point group data while reducing effects due to the shadow.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an information processing device.

  Conventionally, a method (SLAM; Simultaneous Localization and Mapping) of simultaneously performing self-position estimation and environment map creation in a moving body is known. As a kind of such SLAM, a method using a video camera instead of a laser device (VSLAM; Visual SLAM) has been proposed (for example, refer to Patent Document 1). In the VSLAM described in Patent Document 1, the cost is reduced by using a video camera.

JP, 2014-222550, A

  In the VSLAM described in Patent Document 1, feature points are extracted from each image (frame) captured by a video camera, and self-position estimation and environment map creation are performed based on the change of the feature points between the images. To do. At this time, for example, the feature points are extracted along the boundary line of the object (feature), but the feature points may be extracted along the boundary line of the shadow formed on the ground by the feature.

  Since the shape and position of the shadow changes depending on the date and time and the weather, if the point cloud data included in the environment map has data corresponding to the shadow, it may be difficult to use it for self-position estimation, etc. There is a nature. Therefore, it is desired to reduce the influence of the shadow, but it is difficult to automatically determine whether or not the boundary line appearing in each frame is due to the shadow.

  Therefore, an object of the present invention is to provide an information processing device that can generate point cloud data while reducing the influence of shadows.

  In order to solve the above-mentioned problems and achieve the object, an information processing apparatus according to the present invention according to claim 1 is an imaging information acquisition unit that acquires imaging information from an imaging unit mounted on a mobile body, and the imaging information. A generation unit that generates point cloud data based on, a sun information acquisition unit that acquires sun altitude information and sun azimuth information at the time of imaging, and imaging direction information that acquires imaging direction information regarding the direction of the imaging unit at the time of imaging An acquisition unit, wherein the generation unit specifies a shadow generation direction based on the sun altitude information, the sun azimuth information, and the imaging azimuth information, and among the constituent points forming the point cloud data, The feature is that the constituent points arranged in the shadow generation direction are extracted.

  An information processing method according to the present invention according to claim 6, wherein an imaging information acquisition step of acquiring imaging information from an imaging section mounted on a moving body, and a generation step of generating point cloud data based on the imaging information. The solar information acquisition step of acquiring the sun altitude information and the sun azimuth information at the time of imaging, and the imaging azimuth information acquisition step of acquiring the imaging azimuth information regarding the orientation of the imaging unit at the time of imaging, including the generation step, Based on the sun altitude information, the sun azimuth information, and the imaging azimuth information, the shadow generation direction is specified, and the constituent points arranged in the shadow generation direction among the constituent points forming the point cloud data are extracted. It has a feature.

It is a block diagram which shows the outline of the information processing apparatus which concerns on the Example of this invention. It is a top view showing an example of a frame of imaging information which the information processor acquires. It is a top view which shows an example of the point cloud data which the said information processing apparatus produces | generates.

  Embodiments of the present invention will be described below. An information processing apparatus according to an embodiment of the present invention includes an imaging information acquisition unit that acquires imaging information from an imaging unit mounted on a moving body, a generation unit that generates point cloud data based on the imaging information, and A sun information acquisition unit that acquires sun altitude information and sun azimuth information, and an imaging azimuth information acquisition unit that acquires imaging azimuth information regarding the orientation of the imaging unit at the time of imaging. The generation unit specifies the shadow generation direction based on the sun altitude information, the sun azimuth information, and the imaging azimuth information, and extracts the constituent points arranged in the shadow generation direction from the constituent points that form the point cloud data.

  According to the information processing apparatus of the present embodiment as described above, based on the sun altitude information, the sun azimuth information, and the imaging azimuth information, the direction in which the edge of the feature extending along the vertical direction forms a shadow (shadow generation Direction) can be estimated. When there are constituent points arranged in the shadow generation direction, it can be estimated that these constituent points correspond to the shadows of the edges of the feature that extend along the vertical direction. By estimating the constituent points based on the shadow in this way, it is possible to generate the point cloud data while reducing the influence of the shadow. The imaging information acquisition unit, the generation unit, the sun information acquisition unit, and the imaging azimuth information acquisition unit may not be mounted on the same target, but may be mounted on different devices or apparatuses. That is, each component of the information processing device may span a plurality of physically separated devices.

  The information processing apparatus further includes a position information acquisition unit that acquires position information indicating the imaging position of the imaging information, and a map information acquisition unit that acquires map information, and the generation unit is based on the position information and the map information, It is preferable to estimate the shape of the shadow of the feature included in the imaging range. The shape of the shadow that is formed is determined by the shape of the edge of the feature that extends along the vertical direction. The accuracy of the point cloud data can be improved by not only extracting the constituent points arranged in the shadow generation direction but also estimating the shadow shape of the feature. For example, if the feature is a natural product or a building with a complicated shape, it is difficult to form a linear shadow, so even if there are constituent points that are lined up in the shadow generation direction, the configuration that corresponds to the shadow It can be determined that it is not a point.

  It is preferable that the generation unit removes a plurality of constituent points arranged in the shadow generation direction from the point cloud data. As a result, it is possible to remove constituent points that are difficult to use for self-position estimation, etc., and make it easier to use the point cloud data.

  The generation unit may add a predetermined flag to a plurality of constituent points arranged in the shadow generation direction in the point cloud data. For example, when combining a plurality of point cloud data, if the contribution of the flagged constituent point is reduced, this constituent point may be removed or the influence degree may be reduced in the combined point cloud data. You can This makes it easier to use the integrated point cloud data.

  The sun information acquisition unit preferably specifies the sun altitude information and the sun azimuth information by acquiring the date information and the time information. Further, a device other than the information processing device may specify the sun altitude information and the sun direction information, and the sun information acquisition unit may acquire the information from this device.

  An information processing method according to an embodiment of the present invention includes an imaging information acquisition step of acquiring imaging information from an imaging unit mounted on a moving body, a generation step of generating point cloud data based on the imaging information, and an imaging step at the time of imaging. It includes a sun information acquisition step of acquiring the sun altitude information and the sun direction information, and an imaging direction information acquisition step of acquiring the imaging direction information regarding the direction of the imaging unit at the time of imaging. In the generation step, the shadow generation direction is specified based on the sun altitude information, the sun azimuth information, and the imaging azimuth information, and the constituent points arranged in the shadow generation direction are extracted from the constituent points forming the point cloud data. According to such an information processing method, it is possible to generate the point cloud data while reducing the influence of the shadow by estimating the constituent points corresponding to the shadow.

  Further, it may be an information processing program that causes a computer to execute the above-described information processing method. By doing so, it is possible to generate the point cloud data using a computer while reducing the influence of the shadow.

  Further, the above-mentioned information processing program may be stored in a computer-readable recording medium. By doing so, the program can be distributed as a single unit in addition to being installed in the device, and the version can be easily upgraded.

  Hereinafter, examples of the present invention will be specifically described. As shown in FIG. 1, the information processing device 1 of the present embodiment is a VSLAM device mounted on a moving body 10 and includes an information acquisition unit 2, a generation unit 3, and an output unit 4. The moving body 10 is, for example, a vehicle, and is equipped with an imaging unit 20, a map information storage unit 30, a self-position estimation unit 40, a travel information sensor 50, and a date / time information management unit 60.

  The imaging unit 20 is, for example, a video camera, and is arranged so as to image the front side of the moving body 10 in the traveling direction. The moving image captured by the image capturing unit 20 is composed of a plurality of image capturing information (frames) and is transmitted to the information acquisition unit 2. The image capturing section may be capable of continuously capturing still images at predetermined time intervals.

  The map information storage unit 30 is composed of, for example, a hard disk or a non-volatile memory, and stores existing map information. The map information stored in the map information storage unit 30 may include at least information about the feature. Note that information about a feature is not only information about the position and size of the feature, but also information about the attribute and type of the feature (such as the distinction between residential and commercial facilities and the type of commercial facility). Shall be included.

  The self-position estimating unit 40 estimates the current position (absolute position) of the moving body 10, and may be, for example, a GPS receiving unit that receives radio waves transmitted from a plurality of GPS (Global Positioning System) satellites. . The current position estimated by the self-position estimation unit 40 is transmitted to the information acquisition unit 2 as position information indicating the image pickup position of the image pickup information by the image pickup unit 20.

  The travel information sensor 50 is a sensor for measuring the displacement amount of the moving body 10, and for example, for measuring the azimuth displacement amount of the moving body 10 and the vehicle speed pulse acquisition unit that acquires the vehicle speed pulse of the moving body 10. The gyro sensor and the acceleration sensor for acquiring the acceleration of the moving body 10. The traveling direction of the moving body 10 is estimated based on the displacement amount acquired by the travel information sensor 50. The moving direction of the moving body 10 and the mounting direction of the image capturing unit 20 on the moving body 10 determine the image capturing direction information regarding the orientation of the image capturing unit 20 during image capturing. When the imaging unit 20 is directed to the front of the moving body, the traveling direction and the imaging azimuth substantially match. Therefore, the traveling direction sensor 50 transmits the imaging direction information to the information acquisition unit 2. The travel information sensor 50 may be at least capable of estimating the traveling direction of the moving body 10.

  The date / time information management unit 60 has a calendar function and a clock function. The timepiece function is preferably a radio timepiece function that receives standard radio waves and automatically corrects errors. The date and time information is transmitted from the date and time information management unit 60 to the information acquisition unit 2. The sun altitude information and the sun direction information can be specified based on the date information and the time information. Therefore, the information acquisition unit 2 that acquires the date information and the time information can be regarded as acquiring the sun altitude information and the sun azimuth information.

  The information acquisition unit 2 functions as an imaging information acquisition unit by acquiring the imaging information from the imaging unit 20 as described above, and the position information acquisition unit by acquiring the position information from the self-position estimation unit 40 as described above. Functions as a map information acquisition unit by acquiring map information from the map information storage unit 30 and functions as an imaging direction information acquisition unit by acquiring imaging direction information from the travel information sensor 50 as described above. However, by acquiring the sun altitude information and the sun direction information from the date / time information management unit 60 as described above, the sun information processing unit 60 functions as a sun information acquisition unit.

  The generation unit 3 extracts a feature point from the imaging information acquired by the information acquisition unit 2 to generate a two-dimensional feature point group, and further generates point group data based on the plurality of feature point groups. The point cloud data generated by the generation unit 3 may be, for example, three-dimensional point cloud data. It should be noted that the extraction of the feature points may use, for example, an ORB (Oriented fast and Rotated Brief) algorithm or another algorithm.

  The output unit 4 includes a circuit for communicating with a network such as the Internet and a public line, an antenna, and the like, and communicates with the communication unit 101 of the external server 100 to generate information (point cloud data) generated by the generation unit 3. To send. As a result, information is stored in the storage unit main body 102 of the external server 100. The external server 100 is configured to communicate with the plurality of information processing devices 1.

  When generating the point cloud data, the generation unit 3 identifies and extracts the constituent points corresponding to the shadows from the constituent points forming the point cloud data. The detailed procedure at this time will be described below. First, the generation unit 3 specifies the shadow generation direction based on the sun altitude information, the sun azimuth information, and the imaging azimuth information. That is, if the position of the sun and the orientation of the imaging unit 20 are determined, the direction in which an edge of a feature such as a building that extends along the vertical direction forms a shadow at the time of imaging (shadow generation direction) should be estimated. You can That is, when there are constituent points arranged in the shadow generation direction, it can be estimated that these constituent points correspond to the shadows of the edges of the feature that extend along the vertical direction.

  Whether or not the constituent points arranged in the shadow generation direction correspond to the shadow is determined based on, for example, the number of constituent points arranged in a line or the length of a straight line formed by a plurality of constituent points. do it. That is, when the number of constituent points arranged in the shadow generation direction is small, or when the length of a straight line formed by a plurality of constituent points is short, these constituent points are accidentally arranged in the shadow generation direction, There is a high possibility that it does not correspond to the shadow.

  When the generating unit 3 extracts the constituent points arranged in the shadow generation direction and determines that these constituent points correspond to the shadow, these constituent points are removed from the point cloud data.

  Here, a specific example of a method of extracting constituent points arranged in the shadow generation direction will be described with reference to FIGS. 2 and 3, the roads and features other than the target are omitted as appropriate.

  As shown in FIG. 2, when a building B exists along the road R, the building B forms a shadow S on the ground according to the position of the sun and the orientation of the imaging unit. The building B has an edge B1 extending along the vertical direction, and the shadow S has a linear boundary line S1 corresponding to the edge B1. The feature points on the boundary line S1 are extracted in the frame of FIG. 2, and the feature points on the boundary line S1 are also extracted in the frames before and after this. Therefore, when the generation unit 3 generates the point cloud data as shown in FIG. 3, the point cloud data includes the constituent points PB corresponding to the building B and the constituent points P1 to P4 corresponding to the boundary line S1. included.

  The generation unit 3 specifies the shadow generation direction L1 based on the sun altitude information, the sun direction information, and the imaging direction information. In the illustrated example, the sun is located in the southwest, and the imaging unit 20 faces north-northwest. By specifying the shadow generation direction L1, the generation unit 3 can extract the constituent points P1 to P4 arranged in this direction.

  When the generating unit 3 extracts the constituent points arranged in the shadow generation direction and determines that these constituent points correspond to the shadow, the generating unit 3 does not remove these constituent points in the point cloud data and creates a shadow. Information indicating a possibility of correspondence may be added as a flag.

  Further, when extracting the component points arranged in the shadow generation direction as described above, the generation unit 3 is based on the position information acquired from the self-position estimation unit 40 and the map information acquired from the map information storage unit 30. The shape of the shadow of the feature included in the imaging range may be estimated. The shape of the shadow formed is determined by the shape of the edge of the feature that extends along the vertical direction. For example, if the feature along the road is a natural product or a building with a complicated shape, it is difficult to form a linear shadow due to the edge extending along the vertical direction. Even if there is, it can be determined that it is not a constituent point corresponding to the shadow of this feature.

  Further, when the map information includes the height information of the feature, the generation unit 3 may use this height information. That is, not only the shadow generation direction but also the length of the boundary line of the shadow can be estimated based on the height information, the sun altitude information, the sun direction information, and the imaging direction information. Therefore, even if the constituent points are arranged along the shadow generation direction, it is possible to determine that the region exceeding the length of the boundary does not correspond to the shadow.

  The external server 100 acquires and collects the point cloud data generated as described above from the plurality of information processing devices 1. The external server 100 integrates a plurality of collected point cloud data to generate total point cloud data. In addition, when a flag is added to the constituent point of the point cloud data as described above, the reliability of the constituent point may be lowered and the points may be integrated.

  The comprehensive point cloud data generated in this way is used, for example, for estimating the self-position of the moving body. That is, when a feature is detected by an optical sensor (so-called LIDAR; Laser Imaging Detection and Ranging) mounted on the moving body, the self-position of the moving body may be estimated by collating with the comprehensive point cloud data.

  With the above configuration, the generation unit 3 specifies the shadow generation direction based on the sun altitude information, the sun azimuth information, and the imaging azimuth information, and is arranged in the shadow generation direction among the constituent points forming the point cloud data. By extracting the points, the constituent points corresponding to the shadow can be estimated. This makes it possible to generate the point cloud data while reducing the influence of the shadow.

  In addition, the generation unit 3 removes constituent points arranged in the shadow generation direction from the point cloud data, thereby eliminating constituent points that are difficult to use for self-position estimation and the like, and making it easier to use the point cloud data. it can.

  The present invention is not limited to the above-described embodiments, but includes other configurations and the like that can achieve the object of the present invention, and the following modifications and the like are also included in the present invention.

  For example, in the above-described embodiment, the information acquisition unit 2 that acquires the date information and the time information by specifying the sun altitude information and the sun direction information based on the date information and the time information functions as the sun information acquisition unit. However, the sun information acquisition unit may directly acquire the sun altitude information and the sun direction information. That is, the external device may specify the sun altitude information and the sun direction information based on the date information and the time information, and the sun information acquisition unit of the information processing device may acquire the sun altitude information and the sun direction information from the external device. .

  Further, in the above-described embodiment, the generation unit 3 mounted on the moving body 10 specifies the shadow generation direction and extracts the constituent points arranged in the shadow generation direction. For example, the control unit 103 of the external server 100. May specify the shadow generation direction or extract constituent points arranged in the shadow generation direction. That is, the generation unit 3 and the control unit 103 may function as the generation unit.

  Further, the imaging information acquisition unit, the generation unit, the sun information acquisition unit, and the imaging direction information acquisition unit that form the information processing device may not be mounted on the moving body, and may be mounted on different devices or apparatuses. Good. That is, each component of the information processing device may span a plurality of physically separated devices.

  Besides, the best configuration, method, and the like for carrying out the present invention have been disclosed in the above description, but the present invention is not limited thereto. That is, the present invention is mainly illustrated and described mainly with respect to a specific embodiment, but without departing from the technical idea and the scope of the object of the present invention, the shape with respect to the embodiment described above Those skilled in the art can make various modifications in terms of material, quantity, and other detailed configurations. Therefore, the description limiting the shapes and materials disclosed above is described as an example for facilitating the understanding of the present invention, and does not limit the present invention. The description with the names of members excluding some or all of the above limitations is included in the present invention.

1 Information processing device 2 Information acquisition unit (imaging information acquisition unit, position information acquisition unit, map information acquisition unit, sun information acquisition unit, imaging direction information acquisition unit)
3 generation unit 10 moving body 20 imaging unit

Claims (8)

An imaging information acquisition unit that acquires imaging information from the imaging unit mounted on the moving body;
A generation unit that generates point cloud data based on the imaging information;
A sun information acquisition unit that acquires sun altitude information and sun direction information at the time of imaging,
And an imaging azimuth information acquisition unit that acquires imaging azimuth information regarding the orientation of the imaging unit at the time of imaging,
The generation unit specifies a shadow generation direction based on the sun altitude information, the sun azimuth information, and the imaging azimuth information, and is arranged in the shadow generation direction among the constituent points forming the point cloud data. An information processing device characterized by extracting points. A position information acquisition unit that acquires position information indicating an imaging position of the imaging information,
Further comprising a map information acquisition unit for acquiring map information,
The information processing apparatus according to claim 1, wherein the generation unit estimates a shape of a shadow of a feature included in an imaging range based on the position information and the map information.   The information processing apparatus according to claim 1, wherein the generation unit removes a plurality of the constituent points arranged in the shadow generation direction from the point cloud data.   The information processing apparatus according to claim 1, wherein the generation unit adds a predetermined flag to the plurality of constituent points arranged in the shadow generation direction in the point cloud data.   The information processing apparatus according to claim 1, wherein the sun information acquisition unit specifies the sun altitude information and the sun azimuth information by acquiring date information and time information. . An imaging information acquisition step of acquiring imaging information from an imaging unit mounted on a moving body,
A generation step of generating point cloud data based on the imaging information,
A sun information acquisition step of acquiring sun altitude information and sun direction information at the time of imaging,
An image pickup direction information acquisition step of acquiring image pickup direction information regarding the direction of the image pickup unit at the time of image pickup,
In the generating step, a shadow generation direction is specified based on the sun altitude information, the sun azimuth information, and the imaging azimuth information, and a configuration that is arranged in the shadow generation direction among the constituent points that configure the point cloud data. An information processing method characterized by extracting points.   An information processing program for causing a computer to execute the information processing method according to claim 6.   A computer-readable recording medium storing the information processing program according to claim 7.

Images