JP2021004936A - Map data management device and map data management method - Google Patents

Abstract

To improve the accuracy of map data.SOLUTION: In a map updating data generation device 2 that manages the updating data used for the updating of map data, an update range specification unit 210 specifies an update range which is a unit for updating the map data. An image acquisition unit 211 acquires, from a moving vehicle equipped with an imaging device, an imaged image in which the update range is imaged. A position specification unit 212 specifies the position of each of a plurality of imaged objects present in the imaged image that corresponds to each of a plurality of existing objects included in the update range. A correction target specification unit 213 specifies a correction target object that is the existing object the position of which is to be corrected, on the basis of a positional deviation that is a difference between the specified position of each of the plurality of imaged objects and the position of each of the plurality of existing objects corresponding to the imaged objects and managed as map data. A data generation unit 214 generates updating data for updating the position of the correction target object to the position of the imaged object.SELECTED DRAWING: Figure 2

Description

The present invention relates to a map update data generation device and a map update data generation method, and more particularly to a technique for generating update data used for updating map data.

Conventionally, a technique for updating map data based on a captured image is known. In Patent Document 1, map information (map data) having the same position as an image on a map is compared with traffic information in the image recognized by analyzing the image, and the map information is updated based on the comparison result. The technology is disclosed.

Japanese Unexamined Patent Publication No. 2019-040175

In general, map information is enormous, and it is not realistic to manage everything manually. Therefore, the image recognition technology as described above is used for managing map information, but the current situation is that it is not always completely reliable in terms of recognition rate and recognition accuracy. On the other hand, since map information may be referred to, for example, in automatic driving technology of vehicles, it is required to improve the accuracy of map data as much as possible.

The present invention has been made in view of these points, and one of the objects of the present invention is to provide a technique capable of improving the accuracy of map data.

The first aspect of the present invention is a map update data generation device that manages update data used for updating map data. This device is from a range including a plurality of existing objects managed as the map data, from an update range specifying unit that specifies an update range that is a unit for updating the map data, and a moving body including an imaging device. , An image acquisition unit that acquires an captured image in which the update range is captured, and a position that specifies the position of each of the plurality of imaged objects existing in the captured image corresponding to each of the plurality of existing objects included in the update range. The specific unit, each of the positions of the plurality of imaged objects specified by the position specifying unit, and the positions of a plurality of existing objects that are objects corresponding to the imaged object and are managed as the map data. For updating to update the position of the correction target identification unit that specifies the correction target object that is the existing object of the position correction target and the position of the correction target object to the position of the imaged object based on the displacement amount that is the difference. It has a data generation unit that generates data.

The position specifying unit may specify the position of each of the plurality of captured objects existing in the captured image by applying an image recognition process to the captured image.

The sum of the differences between the positions of the plurality of imaged objects specified by the position specifying unit and the positions of the plurality of existing objects that are objects corresponding to the imaged object and are managed as map data is reduced. A position correction unit that corrects by performing an affine conversion may be further provided, and the correction target identification unit corresponds to each of the positions of the plurality of imaged objects corrected by the position correction unit and each of the plurality of imaged objects. The correction target object may be specified based on the difference from each of the positions of the plurality of existing objects.

The correction target identification unit is an image pickup object having a maximum displacement amount indicating a difference between the image pickup object specified by the position identification unit and the position of an existing object corresponding to the image pickup object among the plurality of image pickup objects. Therefore, an existing object corresponding to the imaged object whose displacement amount is equal to or greater than a predetermined threshold value may be specified as the correction target object.

The correction target identification unit is a score for an imaged object corresponding to the existing object that is the correction target object, and the score indicating the certainty of object recognition calculated by the position identification unit is equal to or higher than a predetermined reliability threshold value. Further, on the condition that the existing object may be determined as the object to be corrected.

When there is a new imaged object that cannot be associated with any of the existing objects among the plurality of existing objects among the imaged objects extracted from the captured image by the position specifying unit, information on the new imaged object is mapped. A map registration unit for newly registering data may be further provided.

A second aspect of the present invention is a map update data generation method. In this method, the processor of the map update data generator that manages the update data used for updating the map data includes a plurality of existing objects managed as the map data, and is a unit for updating the map data. The step of specifying the update range, the step of acquiring the captured image in which the update range is captured from the moving body provided with the image pickup device, and the captured image corresponding to each of the plurality of existing objects included in the update range. A step of specifying the position of each of a plurality of existing imaged objects, each of the identified positions of the plurality of imaged objects, and a plurality of existing objects corresponding to the imaged object and managed as the map data. A step of acquiring a position shift amount which is a difference from each position of an object, a step of specifying a correction target object which is an existing object whose position is to be corrected based on the position shift amount, and a position of the correction target object. To generate update data for updating to the position of the imaged object.

It should be noted that any combination of the above components and the conversion of the expression of the present invention between methods, devices, systems, computer programs, data structures, recording media and the like are also effective as aspects of the present invention.

According to the present invention, the accuracy of map data can be improved as an example.

It is a figure for demonstrating the outline of the map data management system which concerns on embodiment. It is a figure which shows typically the internal structure of the map update data generation apparatus which concerns on embodiment. It is a schematic diagram for demonstrating the update range and position identification which concerns on embodiment. It is a schematic diagram for demonstrating the correction process executed by the position correction part which concerns on embodiment. It is a figure which shows typically the recognition result of the image pickup object F by the position specifying part which concerns on embodiment. It is a flowchart for demonstrating the flow of the map update data generation processing executed by the map update data generation apparatus which concerns on embodiment.

[Outline of map data management system S according to the embodiment]
FIG. 1 is a diagram for explaining an outline of the map data management system S according to the embodiment. The map data management system S is a system that generates update data for updating the map data by using the captured image captured by the image pickup device C included in the moving body V. The map data management system S includes a mobile terminal 1 and a map update data generation device 2.

The mobile terminal 1 is a terminal included in the mobile V, and is, for example, a mobile terminal (smartphone, tablet, etc.), car navigation, or the like. The mobile terminal 1 includes a GPS receiver and measures its own position, that is, the position of the mobile V at predetermined intervals. For example, each time the mobile terminal 1 measures the position of the mobile V, the mobile terminal 1 transmits position information indicating the position of the mobile V to the map update data generation device 2.

The mobile body V is a vehicle, for example, a vehicle (eg, a taxi, a delivery vehicle, etc.) used to provide a transportation service for transporting passengers or cargo. The moving body V is not limited to a vehicle, and may be, for example, a train, an airplane, a ship, a drone, a human being, an animal, or the like. As described above, the moving body V includes an image pickup device C. The image pickup device C is, for example, a camera, a mobile terminal with a camera function, a drive recorder, or the like. The image pickup device C may be a device integrated with the moving body V, or may be a retrofit device.

The mobile terminal 1 is electrically connected to the image pickup device C, and acquires the captured image captured by the image pickup device C. The captured image is, for example, a moving image. For example, when the time zone in which the mobile body V provides the transportation service is during the daytime, the mobile body terminal 1 causes the image pickup device C to capture an image captured during the daytime, and the captured image is captured as map update data at night. It is transmitted to the generator 2. The mobile terminal 1 and the image pickup device C may be the same device.

The map update data generation device 2 is a device that manages map data, for example, a server. The map data is, for example, data for presenting a map around the moving body V in car navigation and guiding a route to a destination, and includes an object type, an object shape, an object position, and the like. It is data. The object managed by the map update data generation device 2 as map data is a static object, such as a traffic light, a road marking, a road sign, and a building. The map update data generation device 2 manages static objects such as traffic lights, road markings, road signs, and buildings as existing objects to be managed. Further, the map update data generation device 2 manages a plurality of position information acquired from the mobile terminal 1 as a passage route of the mobile V.

The map update data generation device 2 divides the map data to be managed into a plurality of units, and updates the map data in units of the map data included in each division. In the example shown in FIG. 1, first, the map update data generation device 2 specifies an update range for updating the map data from the map data to be managed ((1) in FIG. 1). The map update data generation device 2 causes the mobile terminal 1 provided in the mobile V traveling in the update range to transmit an captured image in which the update range is captured ((2) in FIG. 1).

The map update data generation device 2 specifies the position of each of the plurality of captured objects existing in the captured image corresponding to each of the plurality of existing objects whose positions exist in the update range ((FIG. 1). 3)). The map update data generation device 2 identifies a correction target object, which is an existing object whose position is to be corrected, among a plurality of imaged objects ((4) in FIG. 1).

Here, the map update data generation device 2 sets the object to be corrected based on the amount of misalignment, which is the difference between the positions of each of the plurality of imaged objects and the positions of the plurality of existing objects managed as map data. Identify. The map update data generation device 2 is not a position of a single imaged object and a position of a single existing object corresponding to the imaged object, but a position that comprehensively considers the positions of a plurality of imaged objects and each existing object. Make a match.

As a result, the map update data generation device 2 can totally optimize the alignment (matching) with the positions of the plurality of imaged objects and the existing objects. Then, the map update data generation device 2 specifies an existing object having a large deviation between the position of the imaged object and the position of the existing object as the existing object to be corrected even if the entire optimization is performed. As a result, the map update data generation device 2 can improve the accuracy of estimating the position of the captured object existing in the captured image. As a result, the map update data generation device 2 can improve the accuracy of the map data.

The map update data generation device 2 generates update data for updating the position of the object to be corrected as the position of the imaged object ((5) in FIG. 1). The update data is sent to, for example, a map operator server managed by a operator responsible for generating and updating map data. The map operator server updates the map data based on the update data. In this way, the map update data generation device 2 can generate update data based on the captured image.

[Configuration of map update data generation device 2]
FIG. 2 is a diagram schematically showing an internal configuration of the map update data generation device 2 according to the embodiment. The map update data generation device 2 includes a storage unit 20 and a control unit 21. In FIG. 2, the arrows indicate the main data flows, and there may be data flows not shown in FIG. In FIG. 2, each functional block shows a configuration of a functional unit, not a configuration of a hardware (device) unit. Therefore, the functional block shown in FIG. 2 may be mounted in a single device, or may be mounted separately in a plurality of devices. Data transfer between functional blocks may be performed via any means such as a data bus, a network, and a portable storage medium.

The storage unit 20 includes a ROM (Read Only Memory) for storing the BIOS (Basic Input Output System) of the computer that realizes the map update data generation device 2, and a RAM (Random Access Memory) that serves as a work area for the map update data generation device 2. ), OS (Operating System), application program, and large-capacity storage device such as HDD (Hard Disk Drive) and SSD (Solid State Drive) that store various information referred to when the application program is executed.

The control unit 21 is a processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) of the map update data generation device 2, and by executing a program stored in the storage unit 20, the update range specifying unit 210 , Image acquisition unit 211, position identification unit 212, correction target identification unit 213, data generation unit 214, position correction unit 215, and map registration unit 216.

Note that FIG. 2 shows an example in which the map update data generation device 2 is composed of a single device. However, the map update data generation device 2 may be realized by computing resources such as a plurality of processors and memories, such as a cloud computing system. In this case, each unit constituting the control unit 21 is realized by executing a program by at least one of a plurality of different processors.

The update range specifying unit 210 specifies an update range, which is a range including a plurality of existing objects managed as map data and is a unit for updating the map data. As described above, the map update data generation device 2 divides the map data to be managed into a plurality of units, and updates the map data in units of the map data included in each division. For example, the update range specifying unit 210 specifies the update range in units of a section between a certain traffic light and another traffic light on the road on which the traffic light exists, that is, a section on the road sandwiched between two different traffic lights. As another example, the update range specifying unit 210 may specify the update range in units of sections on the road sandwiched between two different intersections (including a T-junction). In addition, the update range specifying unit 210 may specify the update range in units of sections on the road sandwiched between two different existing objects among the existing objects managed as map data.

The image acquisition unit 211 acquires an captured image in which the update range is captured from the moving body V including the image pickup device C. The position specifying unit 212 specifies the positions of the plurality of captured objects existing in the captured images corresponding to the plurality of existing objects included in the update range.

FIG. 3 is a schematic diagram for explaining an update range and position identification according to an embodiment. Specifically, FIG. 3 is a diagram schematically showing a part of map data managed by the map update data generation device 2. In FIG. 3, the first traffic light E1 and the second traffic light E2 exist on the road R. The section on the road R sandwiched between the first traffic light E1 and the second traffic light E2 is an update range in which the update range specifying unit 210 is the unit of update.

In FIG. 3, the stop line E3 and the road sign E4 are also existing objects managed by the map update data generation device 2. Hereinafter, in the present specification, the existing objects managed by the map update data generation device 2 are simply "existing objects" except for the case of distinguishing the types (for example, traffic lights, stop lines, buildings, road signs, etc.). Describe using the symbol E and numbers such as "E1". When a plurality of existing objects are not distinguished, it is simply described as "existing object E". FIG. 3 shows that eight existing objects from the existing object E1 to the existing object E8 exist in the update range sandwiched between the existing object E1 and the existing object E2.

The map data handled by the map update data generation device 2 is stored in which the position coordinates of each existing object E are associated with each existing object. Although not limited, as an example, in the map data handled by the map update data generation device 2, absolute position coordinates expressed in latitude and longitude are stored for each existing object E. As another example, the map data may store the position coordinates of each existing object E using the relative position coordinates with the existing object E as a reference as the origin for each update range. In the latter case, in the example shown in FIG. 3, the relative position coordinates of each existing object E from the existing object E2 to the existing object E8 are linked to each existing object E based on the two-dimensional coordinate system with the existing object E1 as the origin. It may be attached and managed.

The position specifying unit 212 recognizes each of a plurality of captured objects existing in the captured image by applying a known image recognition process such as DNN (Deep Neural Network) to the captured image, and identifies the position thereof. .. In FIG. 3, among the imaged objects specified by the position specifying unit 212, the imaged object corresponding to the existing object E1 is the imaged object F1. Further, among the imaged objects specified by the position specifying unit 212, the imaged object corresponding to the existing object E2 is the imaged object F2. The same applies hereinafter, and among the imaged objects specified by the position specifying unit 212, the imaged object corresponding to the existing object En is the imaged object Fn (n is an integer from 1 to 8). The position specifying unit 212 specifies the absolute position coordinates of each imaged object F from the imaged object F1 to the imaged object F8.

Returning to the description of FIG. The correction target specifying unit 213 is an object corresponding to each of the positions of the plurality of imaged objects F specified by the position specifying unit 212 and the imaged object F, and is an object corresponding to the imaged object F, and is a plurality of existing objects E managed as map data. Based on the amount of misalignment, which is the difference from each position, the object to be corrected, which is the existing object to be corrected for the position, is specified. The position correction unit 215 in FIG. 2 will be described later.

As described above, the objects managed by the map update data generation device 2 as map data are, for example, static objects such as traffic lights, road markings, road signs, and buildings. Therefore, it is generally difficult to assume that these objects will move at multiple locations at the same time. On the other hand, for some reason, for example, the position of the stop line or the position of the road sign may be changed independently.

That is, it is a small probability that all the positions of the plurality of existing objects E existing in the update range move, and even if it moves, even if it is assumed that only a small number (typically one) of the existing objects E move. It is considered to be fully realistic.

Details will be described later, but on the premise of this assumption, after the alignment is performed as a whole using the positions of the plurality of existing objects E and the positions of the plurality of imaging objects F corresponding to each existing object E, An object in which the position of the existing object E and the position of the corresponding imaged object F are still deviated by a predetermined threshold value or more may be considered as the correction target object.

FIG. 3 shows the positions of eight existing objects E from the existing object E1 to the existing object E8 and the positions of the eight imaged objects F from the imaged object F1 to the imaged object F8 corresponding to the existing object E1 to the existing object E8. An example is shown in the case where the positioning is performed by the positioning unit 212 so that the total sum of the respective errors is minimized. Further, FIG. 3 shows a case where the displacement amount e between the position of the existing object E7 and the position of the image pickup object F7 is deviated by a predetermined threshold value or more. In this example, the correction target identification unit 213 specifies the existing object E7 as the correction target object. Although not shown, the existing object E7 is a pedestrian crossing. In the update range shown in FIG. 3, it is considered that the position of the existing object E7, which is a pedestrian crossing, has been moved for some reason.

Therefore, the data generation unit 214 generates update data for updating the position of the existing object E7, which is the correction target object specified by the correction target identification unit 213, to the position of the image pickup object F7 corresponding to the existing object E7. .. In this way, the map update data generation device 2 comprehensively considers not the position of a single imaged object and the position of a single existing object corresponding to the imaged object, but the positions of the plurality of imaged objects and the existing objects. Align with consideration. As a result, the map update data generation device 2 can totally optimize the alignment with the positions of the plurality of imaged objects and the existing objects. Then, the map update data generation device 2 specifies an existing object having a large deviation between the position of the imaged object and the position of the existing object as the existing object to be corrected even if the entire optimization is performed. As a result, the map update data generation device 2 can improve the accuracy of estimating the position of the captured object existing in the captured image.

[Alignment using multiple existing objects and imaging objects]
Subsequently, the details of the alignment using a plurality of existing objects and imaging objects will be described.

The position specifying unit 212 according to the embodiment recognizes the imaged object using a known recognition technique, and then, for example, time-integrates the vehicle speed of the moving body V based on the position coordinates of the starting point of the update range, or captures the imaged point. The position coordinates of the recognized imaged object are calculated by using triangulation using a plurality of captured images having different values. Therefore, it is considered that the position coordinates of the imaged object calculated by the position specifying unit 212 include some error.

Therefore, the position correction unit 215 performs an affine transformation so that the sum of the differences between the positions of the plurality of imaged objects F specified by the position specifying unit 212 and the positions of the plurality of existing objects E corresponding to the imaged object becomes small. Correct by doing. That is, the position coordinates of each imaged object F specified by the data generation unit 214 are subjected to at least one conversion of rotation, enlargement / reduction, translation, and shearing with respect to the position coordinates of the corresponding existing object E. Suppose there is. Based on this assumption, the position correction unit 215 executes a process of affine transforming the position coordinates of the image pickup object F to bring them closer to the position coordinates of the existing object E.

Specifically, the X coordinate of the imaged object Fn is X ^f _n , and the Y coordinate is Y ^f _n . Further, the X coordinate of the imaged object Fn after the affine transformation is U ^f _n , and the Y coordinate is V ^f _n . Using the affine transformation matrix, rotation, expansion or contraction, translation, and shear can be expressed using the following equations (1) to (4), respectively.

ここで、θは回転角を表す。

Here, θ represents the angle of rotation.

ここで、Ｓ_ｘ、Ｓ_ｙは、それぞれＸ軸方向及びＹ軸方向の拡大率（縮小率）である。

Here, S _x and S _y are enlargement ratios (reduction ratios) in the X-axis direction and the Y-axis direction, respectively.

ここで、Ｔ_ｘ、Ｔ_ｙは、それぞれＸ軸方向及びＹ軸方向の移動量である。

Here, T _x and T _y are movement amounts in the X-axis direction and the Y-axis direction, respectively.

ここで、ｓｈ_ｘ、ｓｈ_ｙは、それぞれＸ軸及びＹ軸に沿ったせん断係数である。

Here, sh _x and sh _y are shear coefficients along the X-axis and the Y-axis, respectively.

By combining equations (1) to (4), the position correction unit 215 expresses arbitrary rotation, enlargement / reduction, translation, and shear with respect to the position coordinates of each image pickup object F specified by the data generation unit 214. can do. In equations (1) to (4), the vector on the left side, that is, the vector indicating the position coordinates of the imaged object Fn after the affine transformation is defined as the vector _Gn, and the vector on the right side, that is, the data generation unit 214 specifies a vector indicating the position coordinates of the imaged object Fn and F _n. Further, the affine transformation matrix expressing rotation, expansion / contraction, translation, and shear is defined as matrix A. At this time, the equations (1) to (4) can be uniformly expressed by using the following equation (5).

The X-coordinate ^X _e n existing object En corresponding to the imaging object Fn, the Y-coordinate and ^Y _{e n,} a vector indicated by the vector following equation with these elements (6) and the vector _{E n.}

ベクトルＥ_ｎは撮像物体Ｆｎの位置座標を示すベクトルをＦ_ｎに対応する既存物体Ｅｎの位置座標のベクトル表現である。

Vector E _n is a vector representation of the position coordinates of the existing object En corresponding vector indicating the position coordinates of the imaged object Fn to F _n.

2 norm sum Er of errors between all the vectors E _n and the vector G _n that exists in the update range is expressed by the following equation (7).

The position correction unit 215 is an optimal affine matrix in the sense that the sum of squares of the error between the position coordinates of the existing object E and the position coordinates of the imaged object F after the affine transformation is minimized, which is shown in the equation (7). The A _opt is derived, and the position coordinates of the imaged object F are corrected using the derived matrix A _opt . The matrix A _opt is represented by the following equation (8). Furthermore, the vector ^{G opt} _n indicating the position coordinates of the imaged object Fn corrected using the matrix Aopt is expressed by the following equation (9).

4 (a)-(b) are schematic views for explaining the correction process executed by the position correction unit 215 according to the embodiment. Specifically, FIG. 4A is a diagram schematically showing the relationship between the position of the existing object E and the position of the imaged object F before the affine transformation. Further, FIG. 4B is a diagram schematically showing the relationship between the position of the existing object E and the position of the imaged object F after the affine transformation.

In FIGS. 4A-(b), the white circles indicate the positions of the existing object E, and the white squares indicate the positions of the imaged object F. As shown in FIG. 4A, before the position correction unit 215 executes the correction process, the position of the imaged object F moves east as a whole as compared with the position of the existing object E, and the direction is north-northeast. It is tilted to and reduced.

As shown in FIG. 4B, after the position correction unit 215 executes the correction process, the position of the existing object E and the position of the imaged object F are substantially the same. However, the amount of misalignment e, which is the difference between the position of the existing object E7 and the position of the imaged object F7, is significantly larger than the measurement error. This suggests that it is highly probable that the position of the existing object E7 has been changed.

As described above, the correction target identification unit 213 includes the positions of the plurality of image pickup objects F corrected by the position correction unit 215 using the equation (9) and the plurality of existing objects E corresponding to the plurality of image pickup objects F. The object to be corrected is specified based on the amount of misalignment e, which is the difference from each of the positions of. Specifically, the correction target identification unit 213 is a position indicating the difference between the position of the image pickup object F specified by the position identification unit 212 and the position of the existing object E corresponding to the image pickup object F among the plurality of image pickup objects F. The existing object E corresponding to the imaged object F having the maximum deviation amount e and the position deviation amount e of which is equal to or greater than a predetermined threshold Th is specified as the correction target object.

Here, the "predetermined threshold value" is a "movement determination threshold value" referred to by the correction target identification unit 213 to determine whether or not the position of the existing object E has been moved. The specific value of the predetermined threshold value Th may be determined by an experiment in consideration of the frame rate and resolution of the image pickup device C, the environment in which the moving body V is expected to travel, the accuracy required for the map data, and the like. For example, 2 meters. When the displacement amount e between the position of the imaged object F after correction by the position correction unit 215 and the corresponding existing object E is equal to or greater than a predetermined threshold value Th, the correction target identification unit 213 corrects the existing object E. Identify as. As a result, the correction target identification unit 213 can improve the identification accuracy of the existing object E whose position has changed.

[Probability of matching]
As described above, the position specifying unit 212 recognizes each of a plurality of captured objects existing in the captured image by applying a known image recognition process such as DNN to the captured image, and specifies the position thereof. ..

5 (a)-(b) are diagrams schematically showing the recognition result of the imaged object F by the position specifying unit 212 according to the embodiment. 5 (a)-(b) are captured images taken at the same place. In the captured image shown in FIG. 5A, a traffic light T1, a pedestrian crossing T2, and a stop line T3 are shown as objects for which map data is to be managed. In FIG. 5A, the broken line indicates the processing result of the image recognition process used by the position specifying unit 212. In the example shown in FIG. 5A, the image recognition process used by the position specifying unit 212 encloses the range in which the recognized imaged object exists on the captured image as shown by the broken line in the figure, and then appears in that range. The type of imaged object being imaged and the score for the result are output. Here, the "score" is a numerical value indicating the reliability (Confidence) of the recognition accuracy. In the image recognition process used by the position specifying unit 212 according to the embodiment, the higher the reliability of the recognition accuracy, the larger the numerical score is output.

In the captured image shown in FIG. 5B, the truck M is shown in the foreground, and a part of the pedestrian crossing T2 and the stop line T3 is shielded. In such a case, in the image recognition process used by the position specifying unit 212, the recognition accuracy of the pedestrian crossing T2 and the stop line T3 is lower than that in the case where there is no shield as shown in FIG. 5A. That is, as compared with the case where there is no shield as shown in FIG. 5 (a), when there is a shield as shown in FIG. 5 (b), the position specifying portion 212 with respect to the pedestrian crossing T2 and the stop line T3. The score output by the image recognition process used by is low.

In particular, as shown in FIG. 5B, since the left half of the pedestrian crossing T2 in the figure is completely hidden by the truck M, the broken line rectangle indicating the existence range of the pedestrian crossing T2 is closer to the right side in the figure. There is. That is, the position of the pedestrian crossing T2 is recognized as being displaced.

As described above, the score output by the image recognition process used by the position specifying unit 212 can be an index for determining whether or not the result of image recognition is reliable. When the score output by the image recognition process is low, it is highly probable that the recognition process is inaccurate in the first place, and therefore the misalignment amount e calculated based on the recognition result is also likely to be inaccurate. On the contrary, when the score output by the image recognition process is high, it can be considered that the misalignment amount e is calculated based on the reliable image recognition result.

Therefore, the correction target identification unit 213 may determine the existing object E as the correction target object on condition that the score for the imaging object F corresponding to the existing object E, which is the correction target object, is equal to or higher than a predetermined reliability threshold value. Good. As a result, the correction target identification unit 213 can further improve the identification accuracy of the existing object E whose position has changed.

[Registration of new object]
The above description has been made on the premise that the existing object E corresponding to the imaged object F specified by the position specifying unit 212 exists. However, for example, when a road sign is newly installed, there is no existing object E corresponding to the imaged object F (newly installed road sign) specified by the position specifying unit 212 on the map data.

Therefore, in the map registration unit 216, when there is a new imaged object F that cannot be associated with any of the existing objects E among the plurality of existing objects E among the imaged objects F extracted from the captured image by the position specifying unit 212. , Information about the new imaged object F is newly registered in the map data. Specifically, the map registration unit 216 newly registers the information associated with the type and position of the imaged object F specified by the position identification unit 212 in the map data. As a result, the map registration unit 216 can register information about a new object that did not exist on the map data in the map data.

[Process flow of map update data generation process executed by map update data generation device 2]
FIG. 6 is a flowchart for explaining the flow of the map update data generation process executed by the map update data generation device 2 according to the embodiment. The process in this flowchart starts, for example, when the map update data generation device 2 is activated.

The update range specifying unit 210 specifies an update range, which is a range including a plurality of existing objects E managed as map data and is a unit for updating the map data (S2). The image acquisition unit 211 acquires an image captured by the update range specified by the update range specifying unit 210 from the moving body V including the image pickup device C (S4).

The position specifying unit 212 specifies the positions of the plurality of imaged objects F corresponding to the plurality of existing objects E included in the update range (S6). The position correction unit 215 corrects the position of the image pickup object F so that the sum of the positions of the image pickup object F and the existing object E becomes small (S8).

The correction target specifying unit 213 is corrected based on the amount of misalignment e, which is the difference between the positions of the plurality of imaging objects F specified by the position specifying unit 212 and the positions of the corresponding existing objects E. Identify the object (S10). The data generation unit 214 generates update data for updating the position of the object to be corrected to the position of the image pickup object F (S12). When the data generation unit 214 generates update data, the process in this flowchart ends.

[Effects in this embodiment]
As described above, according to the map update data generation device 2 according to the embodiment, the accuracy of the map data can be improved.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes can be made within the scope of the gist. is there. For example, all or a part of the device can be functionally or physically distributed / integrated in any unit. Also included in the embodiments of the present invention are new embodiments resulting from any combination of the plurality of embodiments. The effect of the new embodiment produced by the combination has the effect of the original embodiment.

[First modification]
In the above, the correction target specifying unit 213 has described the case where the correction target object is determined based on the score output by the image recognition process used by the position specifying unit 212. In place of or in addition to this, the correction target identification unit 213 includes the positions of the plurality of imaging objects F corrected by the position correction unit 215 and the plurality of existing objects E corresponding to the plurality of imaging objects F. The object to be corrected may be determined based on the sum of the differences from each position.

Specifically, the correction target identification unit 213, on condition that the 2-norm of the sum Er error indicated by equation (7) with respect to the matrix A _opt is less than a predetermined determination threshold value, even if to confirm the correction target object Good. When the total Er of the two norms of error is large, it can be said that there is a high probability that the position correction by the position correction unit 215 did not operate correctly as compared with the case where it is small. On the contrary, the smaller the total Er of the two norms of the error, the higher the probability that the position correction by the position correction unit 215 has operated correctly. By using Er, which is the sum of 2 norms of the error represented by the equation (7), to determine the object to be corrected, the correction target identification unit 213 can further improve the identification accuracy of the existing object E whose position has changed.

[Second variant]
The position correction unit 215 may correct the position of the imaged object F based on the speed of the moving body V when the image pickup device C captures the captured image and the shutter speed of the image pickup device C. Specifically, the second speed of the moving body V at the time of capturing the captured image is P [meters / second], and the shutter speed of the imaging device C is Q [seconds]. In this case, the subject existing in the captured image is imaged with a blur of P × Q [meters]. For example, when the moving body V has a speed of 72 kilometers per hour (20 meters per second) and a shutter speed of 1/100 second, the subject existing in the captured image will be blurred by 20 centimeters. The position correction unit 215 corrects the position of the image pickup object F based on the speed of the moving body V and the shutter speed of the image pickup device C, so that the correction target identification unit 213 improves the accuracy of the position of the image pickup object F. Can be made to.

[Third variant]
In the above, in the formula (7), an error between all the vectors E _n and the vector G _n that exists in the update range has been described the case of expressing the sum of those two norms. However, error between all the vectors E _n and the vector G _n that exists in the update range may be used the sum of other norm than the sum of the 2-norm. For example, the error between all the vectors E _n and the vector G _n that exists in the update range may be used the sum total of the 1-norm. In this case, since it is assumed that the error between the vector _En and the vector G _n follows the Laplace distribution, it is advantageous in that the outlier becomes robust.

1 ... Mobile terminal 2 ... Map update data generation device 20 ... Storage unit 21 ... Control unit 210 ... Update range identification unit 211 ... Image acquisition unit 212 ... Position identification unit 213 ... Correction target specific unit 214 ... Data generation unit 215 ... Position correction unit 216 ... Map registration unit C ... Imaging device S ... Map data management system V ... Moving object

Claims (7)

In the map update data generator that manages the update data used for updating the map data
An update range specifying unit that specifies an update range that is a unit for updating the map data and is a range including a plurality of existing objects managed as the map data.
An image acquisition unit that acquires an image captured in which the update range is captured from a moving body including an image pickup device.
A position specifying unit that specifies the position of each of the plurality of captured objects existing in the captured image corresponding to each of the plurality of existing objects included in the update range.
A position that is the difference between each of the positions of the plurality of imaged objects specified by the position specifying unit and each of the positions of a plurality of existing objects that are objects corresponding to the imaged object and are managed as the map data. A correction target identification unit that identifies a correction target object that is an existing object whose position is to be corrected based on the amount of deviation,
A data generation unit that generates update data for updating the position of the object to be corrected to the position of the imaged object, and
Map update data generator with. The position specifying unit identifies the position of each of the plurality of captured objects existing in the captured image by applying an image recognition process to the captured image.
The map update data generation device according to claim 1. The sum of the differences between the positions of the plurality of imaged objects specified by the position specifying unit and the positions of the plurality of existing objects that are objects corresponding to the imaged object and are managed as map data is reduced. It also has a position correction unit that corrects by performing affine transformation.
The correction target identification unit is based on the difference between the positions of the plurality of imaged objects corrected by the position correction unit and the positions of the plurality of existing objects corresponding to the plurality of image pickup objects, respectively. To identify,
The map update data generation device according to claim 1 or 2. The correction target identification unit is an image pickup object having a maximum displacement amount indicating a difference between the image pickup object specified by the position identification unit and the position of an existing object corresponding to the image pickup object among the plurality of image pickup objects. The existing object corresponding to the imaged object whose displacement amount is equal to or greater than a predetermined threshold is specified as the correction target object.
The map update data generation device according to claim 3. The correction target identification unit is a score for an imaged object corresponding to the existing object that is the correction target object, and the score indicating the certainty of object recognition calculated by the position identification unit is equal to or higher than a predetermined reliability threshold value. Further, on the condition that the existing object is determined as the correction target object,
The map update data generation device according to claim 4. When there is a new imaged object that cannot be associated with any of the existing objects among the plurality of existing objects among the imaged objects extracted from the captured image by the position specifying unit, information on the new imaged object is mapped. It also has a map registration unit for newly registering data.
The map update data generation device according to any one of claims 2 to 5. The processor of the map update data generator that manages the update data used to update the map data
A step of specifying an update range, which is a unit for updating the map data, including a plurality of existing objects managed as the map data, and
A step of acquiring an captured image in which the update range is captured from a moving body including an imaging device, and
A step of specifying the position of each of the plurality of captured objects existing in the captured image corresponding to each of the plurality of existing objects included in the update range, and
Acquires the amount of misalignment, which is the difference between each of the specified positions of the plurality of imaged objects and the positions of the plurality of existing objects that are objects corresponding to the imaged object and are managed as the map data. Steps and
A step of identifying an existing object to be corrected based on the amount of misalignment, and a step of identifying the object to be corrected.
A step of generating update data for updating the position of the object to be corrected to the position of the imaged object, and
Map update data generation method to execute.

Images