JP7215460B2 - Map system, map generation program, storage medium, vehicle device and server - Google Patents

Description

The present invention relates to a map system, a map generation program, a storage medium, a vehicle device, and a server for generating maps.

In Patent Document 1, an image captured by a camera mounted on a vehicle is used to record location information such as landmarks, and the information is uploaded to a server or the like to generate a sparse map. A technology related to a system that determines the position of the own vehicle by downloading a generated sparse map is disclosed.

Japanese Patent Publication No. 2018-510373

In the system described above, the probe data, which is the data to be uploaded to the server, is the data for the same location, but the specifications and installation of the imaging device such as the camera mounted on the vehicle that generated it. They may be different from each other depending on the position, posture, and the like. Conventionally, a server generates a map by aggregating all such probe data. Therefore, as the number of data in the probe data increases, the map generation accuracy in the server improves.

However, when the number of probe data is small and the probe data is biased, there is a possibility that the server cannot improve the accuracy of map generation. In other words, in the conventional technology, when the number of probe data is small, there is a possibility that the accuracy of map generation by the server will be lowered.

SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and its object is to provide a map system, a map generation program, a storage medium, a vehicle device, and a server that can improve the accuracy of map generation in a server. be.

The map system according to claim 1 is extracted from a vehicle device equipped with an image capturing device mounted on a vehicle and capturing an image of the surroundings of the vehicle, and an image captured by the image capturing device transmitted from the vehicle device. and a server that generates a map using data corresponding to the feature points. The map system includes an integration unit that weights data based on the bias of data transmitted from a plurality of vehicle devices, integrates at least a portion of the plurality of data based on the weighting, and generates a map. The integration unit uses the image corresponding to the data and uses the structure-from-motion technique to estimate the behavior of the own vehicle, which is the behavior of the vehicle. The data are weighted so that the priority of the data is higher than the priority of the data determined to have relatively low estimation accuracy.

According to the above configuration, even if data transmitted from a plurality of vehicle devices to the server, that is, probe data, is biased, the data is weighted based on the bias, and a map is generated in consideration of the weighting. be done. According to such a configuration, the server can generate a more accurate map than the conventional one, even if the number of data transmitted from the vehicle device to the server is at least. Therefore, according to the above configuration, it is possible to obtain an excellent effect of increasing the accuracy of map generation in the server.

A diagram schematically showing the configuration of a map system according to the first embodiment. FIG. 4 is a diagram schematically showing the configuration of an integration unit according to the first embodiment; FIG. 4 is a diagram schematically showing the flow of processing executed by an integration unit in the first application example according to the first embodiment; FIG. 8 is a diagram schematically showing the flow of processing executed by an integration unit in a second application example according to the first embodiment; FIG. 12 is a diagram schematically showing the flow of processing executed by the integration unit in the third application example according to the first embodiment; FIG. A diagram schematically showing the configuration of a map system according to a second embodiment. A diagram for explaining a specific example of reliability information generation by the first method according to the second embodiment. A diagram for explaining a specific example of reliability information generation by the second method according to the second embodiment. A diagram for explaining a specific example of reliability information generation by the third method according to the second embodiment.

A plurality of embodiments will be described below with reference to the drawings. In addition, the same code|symbol is attached|subjected to the substantially same structure in each embodiment, and description is abbreviate|omitted.
(First embodiment)
A first embodiment will be described below with reference to FIGS. 1 to 5. FIG.
The map system 1 shown in FIG. 1 is a map system for autonomous navigation. The map system 1 functions additionally to the conventional function of specifying the position of the own vehicle such as GPS, and is effective in specifying the position with higher accuracy. The map system 1 has two main functions: map utilization and map update.

In map utilization, the map information stored in the server 2 is downloaded to the vehicle, and the vehicle stores the downloaded map information and the positions of landmarks such as signs included in the image captured by the image sensor 3 such as a camera. The position of the own vehicle is specified based on and. In this specification, the map information stored in the server 2 may be referred to as an integrated map. In this case, based on the specified current position of the own vehicle, the vehicle control unit 4 outputs a corresponding command to an actuator for operating hardware mounted on the vehicle, thereby realizing driving assistance. Actuators are devices for hardware control of a vehicle, such as brakes, throttles, steering, and lamps.

On the other hand, in updating the map, information obtained by various sensors such as the image sensor 3, the vehicle speed sensor 5, and a millimeter wave sensor (not shown) mounted on the vehicle is uploaded to the server 2 as probe data, and the integrated map in the server 2 is updated. is updated sequentially. As a result, the position of the vehicle is always determined with high accuracy based on the latest map information, and, for example, driving assistance and automatic steering are realized.

In the map system 1, the human-machine interface 6 is a user interface for notifying the user of various types of information and transmitting predetermined operations from the user to the vehicle. In this specification, the human-machine interface may be abbreviated as HMI. The HMI 6 includes, for example, a display attached to a car navigation system, a display built into an instrument panel, a head-up display projected onto a windshield, a microphone, speakers, and the like. Furthermore, a mobile terminal such as a smart phone communicatively connected to the vehicle can also serve as the HMI 6 within the map system 1 .

In addition to visually obtaining the information displayed on the HMI 6, the user can also obtain the information by voice, warning sound, and vibration. In addition, the user can request the vehicle to perform a desired action by touch operation on the display or voice. For example, when a user wishes to receive advanced driving assistance services such as automatic steering using map information, the user activates the function via the HMI 6 . For example, tapping the "map cooperation" button shown on the display activates the map utilization function and starts downloading the map information.

In another example, a map-based feature is activated by giving a voice command. It should be noted that uploading of map information related to map updating may be performed constantly while communication between the vehicle and the server 2 is established, or tapping the "map cooperation" button enables the map utilization function. It may be executed while it is being activated, or it may be activated by another UI that reflects the user's intention.

The map system 1 of this embodiment includes a server 2 and vehicle components. Each configuration on the vehicle side includes an image sensor 3, a vehicle control unit 4, a vehicle speed sensor 5, an HMI 6, a GPS receiving unit 7, a control device 8, and the like. Of these components on the vehicle side, the image sensor 3 and the control device 8 function as a vehicle device 9 that transmits data corresponding to an image captured by the image sensor 3 to the server 2 .

The server 2 is provided at a location isolated from the vehicle on which the vehicle device 9 and the like are mounted. The server 2 generates a map using the data corresponding to the image captured by the image sensor 3 and transmitted from the vehicle device 9 , and includes a control device 10 . The control device 10 is mainly composed of a microcomputer having a CPU, a ROM, a RAM, an I/O, and the like, and includes an integrating section 11 and an updating section 12 .

Each of these functional blocks is implemented by the CPU of the control device 10 executing a computer program stored in a non-transitional physical storage medium to execute processing corresponding to the computer program. Realized. Therefore, the computer program executed by the microcomputer of the control device 10 includes a program for executing at least part of each process related to map generation and map update, that is, at least part of the map generation program. . The integrating section 11 and the updating section 12 are for executing various processes related to the above-described map updating, and the details thereof will be described later.

The GPS receiver 7 outputs data Da representing GPS information represented by a signal received via a GPS antenna (not shown) to the control device 8 or the like. The vehicle speed sensor 5 detects the vehicle speed, which is the speed of the vehicle, and is configured as a wheel speed sensor that detects the speed of the wheels of the vehicle. The vehicle speed sensor 5 outputs a signal Sa representing a detected speed, which is its detected value, to the control device 8 or the like.

The image sensor 3 is mounted on a vehicle and is an imaging device that captures an image of the environment around the vehicle, specifically, the environment in a predetermined range in front of the vehicle in the direction of travel. Note that the image sensor 3 is not limited to imaging the front in the direction of travel of the vehicle, and may, for example, image the rear or side. Information about the environment around the vehicle captured by the image sensor 3 is stored in a memory (not shown) in the form of still images or moving images (hereinafter collectively referred to as images). The control device 8 is configured to be able to read the data Db stored in the memory, and executes various processes based on the data Db.

The control device 8 is mainly composed of a microcomputer having a CPU, ROM, RAM and I/O. The control device 8 includes functional blocks such as a scale factor correction unit 13, an egomotion calculation unit 14, a landmark detection unit 15, a map generation unit 16, a localization unit 17, and the like. Each of these functional blocks is implemented by executing a computer program stored in a non-transitional physical storage medium by the CPU of the control device 8, thereby executing processing corresponding to the computer program. Realized.

The control device 8 constitutes a part of an on-vehicle device such as an electronic control device mounted on the vehicle, that is, an ECU. The computer program executed by the microcomputer of the control device 8 includes a program for executing at least part of each process related to map generation and map update, that is, at least part of the map generation program. Scale factor correction unit 13 learns the scale factor of vehicle speed sensor 5 based on signal Sa provided from vehicle speed sensor 5 and data Db representing an image captured by image sensor 3 .

In this specification, the scale factor corrector may be abbreviated as the SF corrector. The scale factor of the vehicle speed sensor 5 is the ratio of the detected value of the vehicle speed sensor 5 to the vehicle speed to be measured by the vehicle speed sensor 5, that is, the ratio of the output change to the input change of the vehicle speed sensor 5. This is a coefficient for obtaining the true value of the vehicle speed from the value. The SF correction unit 13 detects the vehicle speed of the host vehicle based on the signal Sa provided from the vehicle speed sensor 5 and the scale factor corrected by learning, and outputs data Dc representing the detected value to the egomotion calculation unit 14. do.

The egomotion calculation unit 14 estimates the own vehicle behavior, which is the behavior of the vehicle, based on the image captured by the image sensor 3 . In this case, the egomotion calculator 14 is configured to estimate the vehicle behavior using the Structure From Motion method. In this specification, Structure From Motion may be abbreviated as SFM. The egomotion calculation unit 14 includes an SFM recognition unit 18 and a running locus generation unit 19 which are configured by SFM modules.

Based on the data Db, the SFM recognizing unit 18 estimates egomotion, which is a parameter representing the behavior of the vehicle itself, that is, the attitude of the vehicle itself. Note that the egomotion includes information indicating yaw, roll, pitch and translational movements. In the above configuration, the image sensor 3 captures images of the surroundings of the vehicle while moving as the vehicle travels. The SFM recognizing unit 18 recognizes corners, edges, and the like in two-viewpoint images captured while the image sensor 3 is moving, that is, two frames of images captured at different imaging positions by one image sensor 3 at different timings. Extract feature points that can easily be matched.

The SFM recognizing unit 18 associates the feature points extracted in the two frames worth of images, and calculates the optical flow of the feature points based on the positional relationship between them. The SFM recognition unit 18 estimates the three-dimensional position of each feature point and the orientation of the image sensor 3, that is, the egomotion, based on the calculated optical flows. Although the SFM recognizing unit 18 can know the amount of movement of the own vehicle by such a method, there is a problem in the accuracy of the scale. Therefore, the SFM recognizing unit 18 acquires the moving speed of the own vehicle based on the data Da representing the GPS information and the data Dc representing the detected value of the vehicle speed, and improves the accuracy of the scale based on the moving speed. It's becoming

The SFM recognition unit 18 outputs data Dd representing the estimated egomotion to the running locus generation unit 19 and the landmark detection unit 15 . The running locus generating unit 19 integrates the egomotion estimated by the SFM recognizing unit 18 every hour, and generates a running locus representing how the host vehicle has moved. The running locus generating unit 19 outputs data De representing the generated running locus to the map generating unit 16 .

The landmark detection unit 15 has a recognition unit 20 and a target generation unit 21 . The recognition unit 20 detects the position of the landmark on the image captured by the image sensor 3 based on the data Db. It should be noted that various methods can be adopted as the method of detecting the position of the landmark. The landmarks include, for example, signs, billboards, poles such as utility poles and streetlights, white lines, and traffic lights.

Based on the data Db, the recognition unit 20 recognizes the lane of the vehicle and acquires road parameters and lane marking information representing lane markings. The road parameters include information representing the lane width, which is the width of the lane, and the shape of the lane, such as the curvature of the lane, that is, the road. In addition, the road parameters include the offset representing the distance from the center position of the lane in the width direction to the position of the vehicle, and the shape of the lane such as the yaw angle representing the angle between the tangential direction of the lane and the direction of travel of the vehicle. It also includes information representing the running state of the host vehicle relative to the vehicle.

In this case, the landmarks also include track information such as the lane marking information described above. The recognition unit 20 outputs data Df representing such landmark detection results to the target generation unit 21 . The target generation unit 21 compares the detected landmarks with the SFM points therein based on the data Df given from the recognition unit 20 and the data Dd given from the SFM recognition unit 18, thereby , landmark distances, and physical position information including lateral positions. The landmark detection unit 15 outputs data Dg representing the road parameters acquired by the recognition unit 20 to the vehicle control unit 4 . In addition, the landmark detection unit 15 outputs data Dh representing information about the positions of landmarks including track information such as lane marking information generated by the target object generation unit 21 to the map generation unit 16 .

The map generation unit 16 generates map information based on data Da representing GPS information, data De given from the egomotion calculation unit 14 and data Dh given from the landmark detection unit 15 . Specifically, the map generation unit 16 associates the GPS information, the generated landmarks, and the travel locus, thereby generating map information, which is fragmentary map data. In this specification, the map information generated by the map generation unit 16 may be referred to as a probe map.

The data Di representing the probe map generated by the map generation unit 16 is uploaded to the server 2 as probe data and output to the localization unit 17 . In this case, the data Di generated by the map generation unit 16 includes specifications such as the mounting position of the image sensor 3 on the vehicle, the mounting attitude of the image sensor 3 on the vehicle, the resolution of the image sensor 3, and the angle of view. It also contains information that represents

It is difficult to say that the accuracy of the probe map is sufficiently high, because the accuracy of SFM is limited. Therefore, the integration unit 11 of the server 2 overlays and integrates a plurality of probe maps based on the data Di transmitted from the vehicle-mounted device of each vehicle to improve the accuracy of the map, although the details will be described later. The update unit 12 of the server 2 updates the integrated map when the integration by the integration unit 11 is successful. The server 2 distributes the data Dj representing the integrated map to the vehicle-mounted equipment of each vehicle. In this case, the server 2 identifies the approximate position of the delivery destination vehicle based on GPS information or the like, and delivers the integrated map around the approximate position (for example, within a radius of several kilometers around the approximate position). If a map exists on the vehicle-mounted device, it is also possible to distribute the difference from that map.

The localization unit 17 performs localization for estimating the current position of the own vehicle. The localization unit 17 downloads the data Dj representing the integrated map from the server 2, and based on the downloaded data Dj, the data Di representing the probe map, and the data Db representing the image captured by the image sensor 3, the integrated map localize for. Note that the localization unit 17 can also perform localization without using the data Di representing the probe map.

The localization unit 17 calculates road parameters based on the map information when the localization is successful. The localization unit 17 outputs data Dk representing road parameters based on map information to the vehicle control unit 4 . The vehicle control unit 4 executes various processes for controlling travel of the own vehicle based on the data Dg supplied from the landmark detection unit 15 and the data Dk supplied from the localization unit 17 . That is, the vehicle control unit 4 executes various processes for controlling travel of the own vehicle based on road parameters.

As described in the related art, the data Di transmitted from the vehicle devices 9 mounted on a plurality of vehicles may be biased. Therefore, the integration unit 11 of the server 2 determines the bias of the data Di transmitted from the vehicle devices 9 mounted on a plurality of vehicles. The integration unit 11 weights the data Di based on the bias of the data Di, and generates a map by integrating at least part of the plurality of data Di based on the weighting. Each process executed by the integration unit 11 corresponds to an integration procedure.

As shown in FIG. 2, the integration unit 11 includes functional blocks such as a bias determination unit 22, a thinning processing unit 23, a weight integration unit 24, and the like. The bias determination unit 22 executes processing for determining bias of the data Di transmitted from a plurality of vehicles, and outputs data Dl representing the result to the thinning processing unit 23 . The bias determination unit 22 selects at least one of specifications such as the mounting position of the image sensor 3 on the vehicle, the mounting orientation of the image sensor 3 on the vehicle that affects the elevation angle, depression angle, etc. of the image sensor 3, the resolution of the image sensor 3, the angle of view, and the like. It is possible to determine the bias of the data Di based on. Also, the bias determination unit 22 can determine the bias of the data Di based on the running speed of the vehicle. Furthermore, the bias determination unit 22 can determine the bias of the data Di based on the surrounding environment of the vehicle.

The thinning processing unit 23 performs thinning processing for thinning unnecessary data out of the plurality of data Di based on the data Dl, and outputs data Dm representing the result to the weighting integration unit 24 . The weighted integration unit 24 performs an integration process of superimposing and integrating data excluding unnecessary data among the plurality of data Di based on the data Dm. In this case, the weighting integration unit 24 increases the estimation accuracy of the behavior of the vehicle by the egomotion calculation unit 14, that is, the estimation accuracy when estimating the behavior of the vehicle by using the image corresponding to the data Db and using the SFM method. Data Di can be weighted based on

Specifically, the integration unit 11 compares the priority of the data Di whose estimation accuracy is relatively high with the priority of the data Di whose estimation accuracy is relatively low. Data Di can be weighted so that As a result of such weighting, the integration unit 11 can generate a map by preferentially using the data Di having a higher priority.

Next, a specific application example of each process executed by the integration unit 11 will be described. In addition, below, the process performed by each functional block with which the integration part 11 is provided is demonstrated as what the integration part 11 performs.

[1] First Application Example The first application example is related to the mounting position, orientation, specifications, and the like of the image sensor 3 . The mounting position, orientation, specifications, etc. of the image sensor 3 are considered to be advantageous from the viewpoint of improving map generation accuracy under certain circumstances, but are considered disadvantageous under other circumstances.

For example, when the mounting position of the image sensor 3 is a high position, the distance to a target existing at a relatively high position such as a signboard on an expressway is shortened and the visibility is improved. The distance to a target existing at a relatively low position such as a road marking becomes long, and its visibility deteriorates. When the mounting position of the image sensor 3 is relatively low, the distance to a target existing at a relatively low position is shortened and the visibility is improved. The distance to the target increases and its visibility deteriorates.

For this reason, if the data Di is biased toward data where the mounting position of the image sensor 3 is high, the accuracy of the information on the target existing at a relatively low position will be low, If the data is biased, the accuracy of the information on the target existing at a relatively high position will be low. Therefore, the integration unit 11 performs a thinning process, an integration process, and the like so as to evenly integrate the data Di having various mounting positions, specifically, mounting heights of the image sensor 3 . By doing so, it is possible to prevent the precision of each piece of information described above from being lowered.

The integration unit 11 can also perform the following processing in order to improve the accuracy of information regarding a specific target. That is, in order to improve the accuracy of the information about the target existing at a relatively high position, the integration unit 11 weights the data Di with a relatively high mounting position of the image sensor 3 so as to give a higher priority. It can be performed. In addition, in order to improve the accuracy of the information about the target existing at a relatively low position, the integration unit 11 weights the data Di having a relatively low mounting position of the image sensor 3 so as to give higher priority. It can be performed.

Also, a target having the same height as the mounting position of the image sensor 3 is difficult to produce a flow amount. Therefore, the integration unit 11 can perform weighting so as to give higher priority to the data Di including targets having a large deviation from the mounting position of the image sensor 3 . In this way, it is possible to improve the accuracy of information regarding a specific target.

When the angle of view of the image sensor 3 is relatively wide, it is advantageous in acquiring information on objects relatively close to the vehicle, but on the contrary, it acquires information on objects relatively far from the vehicle. It will be disadvantageous. When the angle of view of the image sensor 3 is relatively narrow, it is advantageous in acquiring information on objects relatively far from the vehicle, but on the contrary, it acquires information on objects relatively close to the vehicle. It will be disadvantageous.

For this reason, if the data Di is biased toward data with a wide angle of view of the image sensor 3, the accuracy of information on targets existing at positions relatively far from the vehicle will be low, and the data Di will be If the data is biased toward narrow data, the accuracy of information on targets existing relatively close to the vehicle will be low. Therefore, the integration unit 11 performs thinning processing, integration processing, and the like so that the data Di having various angles of view of the image sensor 3 can be evenly integrated. By doing so, it is possible to prevent the precision of each piece of information described above from being lowered.

The integration unit 11 can also perform the following processing in order to improve the accuracy of information regarding a specific target. That is, in order to improve the accuracy of the information about the target existing relatively close to the vehicle, the integration unit 11 weights the data Di with a relatively wide angle of view of the image sensor 3 so as to give higher priority. It can be carried out. In addition, in order to improve the accuracy of information on targets that are relatively far from the vehicle, the integration unit 11 weights the data Di with a relatively narrow angle of view of the image sensor 3 so as to give higher priority. It can be carried out.

Regarding the resolution of the image sensor 3, it is basically considered that the higher the resolution, the more advantageous from the viewpoint of improving the map generation accuracy. Therefore, the integration unit 11 can perform weighting so as to give higher priority to the data Di whose resolution of the image sensor 3 is relatively high. However, in order to improve the accuracy of information on targets existing relatively close to the vehicle, or to improve the accuracy of information on targets existing relatively far from the vehicle, the data Di It is necessary not only to increase the priority of , but also to give priority after considering the angle of view.

In the case of the data Di including image data captured by the image sensor 3 so as to capture the target in front, the target moves less in the lateral direction, so there is a risk that the accuracy of distance estimation in SFM will be low. Therefore, the integration unit 11 can perform weighting so as to give higher priority to data Di including image data obtained by imaging the same target from adjacent lanes. In this way, the integration unit 11 can generate a map by preferentially using the data Di including image data obtained by picking up each target from an oblique angle, thereby improving the accuracy.

The flow of processing executed by the integration unit 11 in such a first application example is summarized as shown in FIG. As shown in FIG. 3, in step S101, the bias of the data Di based on at least one of the mounting position, mounting orientation and specifications of the image sensor 3 is determined. After execution of step S101, the process advances to step S102 to determine whether or not a specific target is targeted, specifically, whether or not it is necessary to improve the accuracy of information regarding the specific target. Here, if it is necessary to improve the accuracy of the information regarding the specific target, the result is "YES" in step S102, and the process proceeds to step S103.

In step S103, thinning processing and integration processing are performed so that the priority of predetermined data for improving the accuracy of information regarding a specific target is increased. On the other hand, if it is not necessary to improve the accuracy of the information regarding the specific target, the result is "NO" in step S102, and the process proceeds to step S104. In step S104, thinning processing and integration processing are performed so as to reduce the data imbalance determined in step S101. After execution of step S103 or S104, the process ends.

[2] Second Application Example The second application example is related to the running speed of the vehicle. It is considered that the higher the running speed of the vehicle is, the longer the baseline is, and hence the higher the accuracy of the SFM. However, if the vehicle travels too fast, the number of frames that can be used for determination is reduced, which may reduce the accuracy of the SFM. Therefore, the integration unit 11 performs a thinning process, an integration process, and the like so as to evenly integrate the data Di of various running speeds of the vehicle. By doing so, a decrease in accuracy of SFM is suppressed.

However, in the case where data Di of various vehicle running speeds can be uniformly integrated in this way, there is a possibility that the number of determinations for a specific target located around the vehicle may not be sufficient. There is a risk that the accuracy of the SFM with respect to the target will be less than the predetermined decision accuracy. Note that the value of the determination accuracy may be appropriately set according to the specifications of the map system 1 . When the integration unit 11 determines that the accuracy of a specific target does not satisfy the determination accuracy according to the traveling speed of the vehicle, the data Di can be weighted as follows. In other words, the integration unit 11 can weight the data Di so that the priority of the data Di in which the vehicle travel speed is relatively low is higher than the priority of the data Di in which the vehicle travel speed is relatively high. . By doing so, it is possible to increase the number of determinations for a specific target, and as a result, it is possible to improve the accuracy of the SFM.

For example, when a vehicle is traveling in a section of an expressway where soundproof walls of the same shape are continuously provided, depending on the running speed of the vehicle and the pattern repetition interval formed by the soundproof walls, There is a possibility that the SFM recognizing unit 18 will make an error in associating feature points, that is, an erroneous matching will occur, and the accuracy of the SFM will decrease. In order to improve the accuracy of targets such as the soundproof wall, the integration unit 11 integrates data Di with various vehicle running speeds, and lowers the priority or excludes data with large outliers. be able to.

The flow of processing executed by the integration unit 11 in such a second application example is summarized as shown in FIG. As shown in FIG. 4, in step S201, the bias of the data Di based on the running speed of the vehicle is determined. After execution of step S201, the process advances to step S202 to determine whether or not a specific target is targeted, specifically, whether or not it is necessary to improve the accuracy of information regarding the specific target. Here, if it is necessary to improve the accuracy of the information on the specific target, the result is "YES" in step S202, and the process proceeds to step S203.

In step S203, thinning processing and integration processing are performed so that the priority of predetermined data for improving the accuracy of information regarding a specific target is increased or decreased. On the other hand, if it is not necessary to improve the accuracy of the information regarding the specific target, the result is "NO" in step S202, and the process proceeds to step S204. In step S204, thinning processing and integration processing are performed so as to reduce the data imbalance determined in step S201. After execution of step S203 or S204, this process ends.

[3] Third Application Example The third application example is related to the environment such as the brightness around the vehicle. When the surroundings of the vehicle are bright, the amount of noise that affects the accuracy of the SFM is reduced, which is considered to be advantageous from the viewpoint of improving the accuracy of map generation. However, if the surroundings of the vehicle are bright, some electric displays may become overexposed and may not be visible, and electric signs may flicker. there's a possibility that.

Therefore, the integration unit 11 is configured to evenly integrate the data Di including image data with various brightnesses around the vehicle. Thinning processing, integration processing, etc. are performed so that the data Di including the image data obtained and the data Di including the image data picked up at night when the brightness around the vehicle is dark can be evenly integrated. By doing so, a decrease in accuracy of SFM is suppressed.

The integration unit 11 can also perform the following processing in order to improve the accuracy of information regarding a specific target. That is, in order to improve the accuracy of information related to targets other than some of the above-described electronic displays, electronic signs, etc., the integration unit 11 selects an image captured during a time zone when the surroundings of the vehicle are bright, for example, during the daytime. Weighting can be performed to give higher priority to data Di containing data. In addition, in order to improve the accuracy of information on targets such as some of the electric displays and electric signs described above, the integration unit 11 collects image data captured during a time period when the surroundings of the vehicle are dark, for example, at night. Weighting can be performed to give higher priority to data Di containing

The flow of processing executed by the integration unit 11 in such a third application example is summarized as shown in FIG. As shown in FIG. 5, in step S301, the bias of the data Di based on the brightness around the vehicle is determined. After execution of step S301, the process proceeds to step S302 to determine whether or not a specific target is targeted, specifically, whether or not it is necessary to improve the accuracy of information regarding the specific target. Here, if it is necessary to improve the accuracy of the information regarding the specific target, the result is "YES" in step S302, and the process proceeds to step S303.

In step S303, thinning processing and integration processing are performed so that the priority of predetermined data for improving the accuracy of information on a specific target is increased. On the other hand, if there is no need to improve the accuracy of the information regarding the specific target, the result is "NO" in step S302, and the process proceeds to step S304. In step S304, thinning processing and integration processing are performed so as to reduce the data imbalance determined in step S301. After execution of step S303 or S304, this process ends.

[4] Fourth Application Example A fourth application example relates to the state of the vehicle. Immediately after the ignition switch of the vehicle is turned on, the accuracy of the scale factor correction by the SF correction unit 13 is lowered, and as a result, the accuracy of SFM may be lowered. Therefore, the integration unit 11 performs weighting so as to lower the priority of the data Di whose elapsed time from the time when the ignition switch of the vehicle is turned on is relatively short. By doing so, a decrease in accuracy of SFM is suppressed.

[5] Fifth Application Example The fifth application example is related to the imaging time period. Some roads have different road classifications depending on the time of day, and there is a possibility that specific information about such roads can be obtained with high accuracy only at specific times. Therefore, the integration unit 11 performs a thinning process, an integration process, and the like so that the data Di can be evenly integrated in various image data capturing time zones. In this way, it is possible to accurately acquire information about roads whose road classifications change according to the time of day.

As described above, the map system 1 of the present embodiment includes the vehicle device 9 equipped with the image sensor 3 mounted on the vehicle and capturing an image of the surroundings of the vehicle, and the image sensor 3 transmitted from the vehicle device 9. and a server 2 for generating a map using data corresponding to images captured by. The control device 10 of the server 2 weights the data based on the bias of the data transmitted from the plurality of vehicle devices 9, integrates at least part of the plurality of data based on the weighting, and generates a map. An integration unit 11 is provided.

According to the above configuration, even if there is a bias in the data transmitted from the plurality of vehicle devices 9 to the server 2, that is, the probe data, the data is weighted based on the bias, and the weighting is taken into account. is generated. According to such a configuration, the number of data transmitted from the vehicle device 9 to the server 2 is at least as compared to the prior art, and the server 2 can generate a more accurate map. Therefore, according to the present embodiment, it is possible to obtain an excellent effect that the map generation accuracy in the server 2 can be improved.

According to the first application example, the second application example, the third application example, and the fourth application example, which are specific application examples of the processing of the present embodiment, the integration unit 11 performs the vehicle behavior using the SFM technique. The data Di is weighted so that the priority of the data Di judged to have relatively high estimation accuracy is higher than the priority of the data Di judged to have relatively low estimation accuracy. be able to. As a result of such weighting, the integration unit 11 can generate a map by preferentially using the data Di having a higher priority. One index for improving the precision of the integrated map in the server 2 is to improve the precision of the SFM described above. Therefore, according to the specific processing as described above, it is possible to further improve the map generation accuracy in the server 2 .

(Second embodiment)
The second embodiment will be described below with reference to FIGS. 6 to 9. FIG.
As shown in FIG. 6, the map system 31 of the present embodiment differs from the map system 1 of the first embodiment in that a vehicle device 32 is provided in place of the vehicle device 9 . The control device 33 of the vehicle device 32 is different from the control device 8 of the vehicle device 9 in that two functional blocks, a road gradient estimation unit 34 and a visibility estimation unit 35, are added, instead of the map generation unit 16. The difference is that a map generator 36 is provided at the bottom.

The road gradient estimator 34 performs predetermined machine learning based on the data Db representing the image captured by the image sensor 3 , thereby estimating the gradient of the road on the image captured by the image sensor 3 . The road gradient estimator 34 outputs data Dn representing the estimated road gradient to the map generator 36 . The visibility estimation unit 35 performs predetermined machine learning based on the data Db representing the image captured by the image sensor 3 , and estimates the visibility of the image sensor 3 . The visibility estimation unit 35 outputs data Do representing the estimated visibility to the map generation unit 36 .

The map generating section 36 includes a map information generating section 37 and a reliability adding section 38 . In addition to the data De, the data Dd is also given to the map generator 36 from the egomotion calculator 14 . The map information generator 37 generates map information in the same manner as the map generator 16 of the first embodiment. The reliability imparting unit 38 imparts reliability information, which is information regarding the reliability of data corresponding to the image captured by the image sensor 3, to the data Di. The reliability adding unit 38 evaluates the reliability of data corresponding to an image based on the reliability base data input from each functional block, and generates reliability information based on the evaluation result.

The reliability information is information used in the integration unit 11 of the server 2 . That is, as described above, the integration unit 11 weights the data Di based on the bias of the data corresponding to the images transmitted from the plurality of vehicle devices, and based on the weighting, at least one of the plurality of data Di. Generate a map by merging parts. In this case, the integration unit 11 integrates the data Di in consideration of the reliability information given to the data Di to generate the map. For example, the integration unit 11 can preferentially use and integrate data Di to which reliability information indicating relatively high reliability is added. It should be noted that the "data corresponding to the image" referred to here is data relating to the state of imaging of the image by the image sensor 3, that is, not just landmarks and lane information, but also the mounting position of the image sensor 3, the angle of view, the resolution, the vehicle speed, the It is data such as vehicle environment and reliability.

The SFM low-accuracy flag is an example of the reliability-based data whose information source is the data Dd input from the egomotion calculator 14 . The SFM low accuracy flag is turned on when there is a possibility that the egomotion estimation accuracy is degraded. The reliability base data whose information source is the data Dh input from the landmark detection unit 15 includes installation position, size, type, color, number of successful SFM position estimations, number of continuous extrapolations, position when SFM position estimation is successful. , the number of SFM points, the degree of dispersion of the SFM point group, the attributes of the SFM point group, and the like.

The installation position is the installation position of the landmark viewed from the vehicle position. Note that the installation position is a fitting error in the case of the partition line. As a specific example of reliability evaluation based on the installation position, for example, even though it is a sign, if the installation position is on the road surface, it can be estimated that the possibility of false detection is high. is mentioned. The size is the size of the landmark. Note that the size is the line width in the case of the marking line. As a specific example of reliability evaluation based on size, there is a possibility of false detection if the square is less than 0.2m even though it is a sign, or if the aspect ratio is an abnormal value. can be estimated to be high.

As a specific example of reliability evaluation based on type, for example, in the case of a sign, it is determined what kind of sign it is or whether it is not a sign, and in the case of a lane marking, the line type is identified. , the possibility of erroneous detection can be suspected depending on the results of these determinations and the results of identification. As a specific example of reliability evaluation based on color, for example, in the case of lane markings, the color of the line is identified. Here are some examples of what is possible.

The SFM position estimation success count is the number of successful three-dimensional position estimations using SFM points, specifically, an integrated value. A larger number of successful SFM position estimations is better, and in that case the possibility of being a landmark increases. Therefore, as a specific example of reliability evaluation based on the number of successful SFM position estimations, when the number of successful SFM position estimations is extremely small, the possibility that the image has no feature points, that is, is not a landmark can be suspected. , are examples.

The number of times of continuous extrapolation is the number of times the position was predicted from the egomotion when the position could not be estimated by the SFM. A smaller number of consecutive extrapolations is better, which increases the likelihood of being a landmark. Therefore, as a specific example of reliability evaluation based on the number of consecutive extrapolations, if the number of consecutive extrapolations is extremely large, the image may have no feature points, that is, it may not be a landmark, and the distance accuracy may be low. For example, it is possible to suspect that

The position at the time of successful SFM position estimation is the installation position of the landmark when the position estimation by SFM is successful. As a specific example of position-based reliability evaluation when SFM position estimation is successful, there is an example in which, in principle, the farther the object, the lower the distance accuracy. As a specific example of reliability evaluation based on SFM scores, position estimation is performed by taking the average of feature points that hit recognized landmarks, so the more feature points there are, the higher the possibility of distance accuracy being. Examples include becoming.

Specific examples of reliability evaluation based on the degree of dispersion of the SFM point cloud include the following examples. That is, the greater the degree of variation in the SFM point cloud, the lower the reliability. Especially when the distance in the depth direction varies, the possibility of being a landmark decreases. This is because landmarks are basically flat, so the distance in the depth direction for each feature point on the landmark should be constant. Therefore, if there is variation in the distance in the depth direction, there is a high possibility that the object is not a landmark.

Specific examples of reliability evaluation based on the attributes of the SFM point cloud include the following examples. That is, since SFM points have attribute information such as segmentation, that is, road signs and lane markings, the higher the ratio of feature points having the same attribute, the higher the accuracy. That is, the higher the percentage of feature points with corresponding landmark segmentation, the higher the distance accuracy.

The reliability base data whose information source is the data Dn input from the road gradient estimating unit 34 include road gradient estimation state, fitting error, and the like. The road gradient estimation state indicates whether or not the road gradient can be estimated. As a specific example of the reliability evaluation based on the road gradient estimation state, there is an example in which the accuracy of uphill and downhill gradient data is reduced when the road gradient cannot be estimated. The fitting error is the accuracy of the gradient estimation, specifically the average amount of deviation between the fitting curve and the SFM points. A specific example of the reliability evaluation based on the fitting error is that the accuracy is low when the variation of the fitting error is large.

The reliability base data whose information source is the data Do input from the visibility estimation unit 35 includes tunnel flag, glass fog level, lens shielding level, bad weather level, backlight level, raindrop adhesion level, road surface snow level, and desert level. , muddy level, road surface wetness level, and the like. A tunnel flag is a flag that is turned on while passing through a tunnel. If the same background continues in a tunnel, it becomes a factor that lowers the accuracy of SFM egomotion. Therefore, it is possible to evaluate reliability such that accuracy is low when the tunnel flag is on.

The fogging level of the windshield indicates the degree of fogging of the windshield of the vehicle, and the reliability can be evaluated according to the level. The lens shielding level indicates how much the background is hidden, and reliability can be evaluated according to the level. The bad weather level represents the condition of bad weather such as heavy rain, dense fog, heavy snow, and dust, and reliability can be evaluated according to the level.

The backlight level indicates the degree of backlighting caused by sunlight in the daytime, and the degree of backlighting caused by light such as lights at night, and reliability is evaluated according to the level. be able to. The raindrop adhesion level represents the degree of adhesion of raindrops to the windshield of the vehicle, and reliability can be evaluated according to the level. The road surface snow level indicates whether or not there is snow on the road surface, and reliability can be evaluated according to the level.

The desert level indicates whether the road surface is desert or not, and reliability evaluation can be performed according to the level. The muddy level indicates whether the road surface is muddy or not, and the reliability can be evaluated according to the level. The road surface wetness level indicates whether the road surface is wet due to rain or the like, and reliability can be evaluated according to the level. Reliability evaluation based on each of these levels can be performed in multiple stages, for example, in three stages.

GNSS azimuth, GNSS velocity, DOP, and the like are examples of reliability base data whose information source is data Da representing GPS information. GNSS is an abbreviation for Global Navigation Satellite System, and DOP is an abbreviation for Dilution Of Precision. The GNSS azimuth angle represents the azimuth angle of the vehicle obtained by GNSS positioning, that is, the yaw angle. As a specific example of the reliability evaluation based on the GNSS azimuth angle, it is possible to determine that the GNSS accuracy is low when the vehicle has a large difference from the yaw angle calculated from a yaw rate sensor or the like.

The GNSS speed is the running speed of the vehicle obtained by GNSS positioning. As a specific example of the reliability evaluation based on the GNSS speed, it can be determined that the GNSS accuracy is low when the difference between the vehicle speed calculated from the wheel speed sensor and the like is large. DOP is a precision reduction rate, and generally, the smaller the value, the higher the precision of the GNSS positioning results. Therefore, as a specific example of the reliability evaluation based on the DOP, it is possible to judge that the smaller the numerical value, the higher the accuracy.

In this way, the reliability imparting unit 38 evaluates the reliability based on the estimation accuracy when estimating the own vehicle behavior, which is the behavior of the vehicle, using the image corresponding to the data Di and using the SFM technique. , and generates reliability information based on the evaluation results. Further, the reliability imparting unit 38 evaluates the reliability based on the estimation accuracy of the road gradient, and generates reliability information based on the evaluation result. Further, the reliability adding unit 38 evaluates the reliability based on the visibility estimation accuracy of the image sensor 3, and generates reliability information based on the evaluation result. Further, the reliability adding unit 38 evaluates reliability based on information regarding landmarks on the image detected based on the image corresponding to the data Di, and generates reliability information based on the evaluation result.

Next, a specific method for generating reliability information by the reliability adding unit 38 will be described.
[1] First Method In the first method, the reliability granting unit 38 determines a "base point" and a "coefficient" based on the reliability base data, and multiplies the base point by the coefficient to obtain the reliability. For example, the reliability can be set to 100 levels indicated by numerical values from "1" to "100", and the smaller the numerical value, the lower the reliability. In this case, the base points are also set to 100 levels indicated by numerical values from "1" to "100", like the reliability.

Specifically, the reliability granting unit 38 calculates at least one base point based on the reliability base data. For example, in the case of a sign, the base score is 100 when the size is within a specified value and the type of sign such as a speed limit sign can be specified. Then, the reliability adding unit 38 calculates at least one coefficient based on the reliability base data. Note that the coefficient is set to a value of 1.0 or less. For example, it is conceivable to set a lower coefficient as the number of times of continuous extrapolation increases, or to set a lower coefficient as the position at the time of position estimation is farther from the own vehicle position by a certain value or more.

A specific example of reliability information generation by the first method is shown in FIG. 7, for example. Note that landmarks are abbreviated as LMK in FIG. 7 and the like. In this case, the basic points have contents that represent the likeness of the landmark. Also, in this case, three coefficients, ie, a first coefficient, a second coefficient, and a third coefficient are set. The first coefficient has content related to distance accuracy. The second coefficient is a content related to a recognition accuracy reduction factor, that is, a visibility reduction factor. The third coefficient has contents related to other accuracy. In addition, as other accuracies, estimation accuracy of road slope, accuracy of SFM, accuracy of GNSS, and the like can be mentioned. In a specific example of the first method shown in FIG. 7, the reliability is calculated by multiplying the base point by the first, second and third coefficients.

[2] Second Method In the second method, the reliability granting unit 38 calculates two or more basic points from the reliability base data, and adds the two or more basic points to obtain the reliability. In this case, the base points may be distributed so that the sum of all the base points is "100", or that each base point is equally weighted.

A specific example of reliability information generation by the second method is shown in FIG. 8, for example. In this case, four base points are set: a first base point, a second base point, a third base point, and a fourth base point. The first base point has a content that expresses the likeness of a landmark. The second basic point has contents related to distance accuracy. The third basic point is the content related to recognition accuracy degradation factors. The fourth basic point has other contents related to accuracy.

In this case, each base point is distributed so as to be individually weighted. Specifically, the first base point is set to 60 levels indicated by numerical values from "1" to "60", the second base point is set to 20 levels indicated by numerical values from "1" to "20", The 3rd base point and the 4th base point are set to 10 levels indicated by numerical values from "1" to "10". In a specific example of the second method shown in FIG. 8, the reliability is calculated by adding the first, second, third, and fourth basic points.

[3] Third method In the third method, the reliability granting unit 38 calculates two or more basic points from the reliability base data, and obtains the reliability by evaluating each of the two or more basic points. . In this case, each base point has 100 levels indicated by numerical values from "1" to "100". A specific example of reliability information generation by the third method is shown in FIG. 9, for example. In this case, as in the second technique, four basic points are set: the first basic point, the second basic point, the third basic point, and the fourth basic point. In a specific example of the third method shown in FIG. 9, the reliability is calculated by evaluating each of the first, second, third, and fourth basic points individually. .

According to the present embodiment described above, in the vehicle device 32, the map generation unit 36 includes the map information generation unit 37 that generates map information in the same manner as the map generation unit 16 of the first embodiment, and the image sensor 3. and a reliability adding unit 38 for adding reliability information, which is information about the reliability of data corresponding to the captured image, to data Di, which is probe data to be uploaded to the server 2 . In the present embodiment, in the server 2, the integration unit 11 integrates the data Di to generate a map, taking into consideration the reliability information given to the data Di. By doing so, it is possible to obtain an excellent effect that the map generation accuracy in the server 2 can be further improved.

(Other embodiments)
The present invention is not limited to the embodiments described above and illustrated in the drawings, and can be arbitrarily modified, combined, or expanded without departing from the scope of the invention.
The numerical values and the like shown in each of the above embodiments are examples, and are not limited to them.

In the map systems 1 and 31, each functional block may be distributed. For example, a part of each functional block included in the control device 10 on the server 2 side is provided in the control devices 8 and 33 on the vehicle side, that is, the vehicle devices 9 and 32, and each control device transmits various data via communication. may be configured to execute each processing described in each of the above-described embodiments by transmitting and receiving . As a specific example of such a configuration, the following configuration can be cited.

That is, together with the data Dj representing the integrated map, the server 2 stores insufficient probe data such as less data including image data captured at night, less data at a position where the image sensor 3 is mounted at a lower position, etc. give attributes. The vehicle device 9, 32 judges whether or not the own vehicle matches the above attribute, and uploads the data Di only when it is judged to match. In this way, it is possible to efficiently collect only the probe data required by the server 2 while suppressing the amount of communication.

Further, in the second embodiment, the reliability imparting section 38 provided in the control device 33 of the vehicle device 32 can be provided in the control device 10 of the server 2 . In this case, the vehicle device 32 may be configured to transmit the reliability base data to the server 2 . In this case, the reliability imparting unit 38 provided in the server 2 generates reliability information based on the reliability base data transmitted from the vehicle device 32, and transmits the reliability information from the vehicle device 32. It will be given to the data Di to be uploaded.

In principle, the vehicle device 32 of the second embodiment does not retain past information about data corresponding to images captured by the image sensor 3 . Then, the vehicle device 32 uploads to the server 2 the data immediately before the predetermined landmark is cut off from the image captured by the image sensor 3 . Therefore, at the time of uploading the data Di to the server 2, only the reliability-based data in the frame at that time can be evaluated, and the accuracy of reliability evaluation may be lowered.

For example, information about landmark attributes may differ from frame to frame, such as being recognized as a road sign in one frame, but being recognized as an electric light in another frame. Therefore, in the vehicle device 32, it is preferable to store each piece of reliability base data in chronological order, and use the one with the highest frequency among them to convert it into a coefficient, or convert the degree of variation into a coefficient. By doing so, it is possible to maintain the accuracy of the evaluation of the reliability and, in turn, to maintain the accuracy of the map generated by the server 2 at a high level.

In the vehicle device 32 of the second embodiment, the data Di are selected and uploaded based on the reliability information, but it cannot be said that the reliability of the data represented by the reliability information is necessarily accurate. Therefore, it is desirable for the vehicle device 32 to upload as much data Di as possible without excluding the data Di as much as possible. In this case, the integration unit 11 of the server 2 may select and integrate the data Di considered to be really necessary.

Although the present disclosure has been described with reference to examples, it is understood that the present disclosure is not limited to such examples or structures. The present disclosure also includes various modifications and modifications within the equivalent range. In addition, various combinations and configurations, as well as other combinations and configurations, including single elements, more, or less, are within the scope and spirit of this disclosure.

The controller and techniques described in this disclosure may be implemented by a dedicated computer provided by configuring a processor and memory programmed to perform one or more functions embodied by the computer program. may be Alternatively, the controls and techniques described in this disclosure may be implemented by a dedicated computer provided by configuring the processor with one or more dedicated hardware logic circuits. Alternatively, the control units and techniques described in this disclosure can be implemented by a combination of a processor and memory programmed to perform one or more functions and a processor configured by one or more hardware logic circuits. It may also be implemented by one or more dedicated computers configured. The computer program may also be stored as computer-executable instructions on a computer-readable non-transitional tangible recording medium.

1, 31... map system, 2... server, 3... image sensor, 9, 32... vehicle device, 11... integrating section, 38... reliability giving section.

Claims (13)

A vehicle device (9, 32) equipped with an imaging device (3) mounted on a vehicle and configured to capture an image of the surroundings of the vehicle; A map system (1, 31) comprising a server (2) that generates a map using data corresponding to feature points in the
an integration unit (11) for weighting the data based on the bias of the data transmitted from the plurality of vehicle devices, and integrating at least a part of the plurality of data based on the weighting to generate the map; ) ,
The integration unit uses the image corresponding to the data and uses the Structure From Motion technique to estimate the behavior of the own vehicle, which is the behavior of the vehicle. A map system that weights the data so that the priority of the data is higher than the priority of the data determined to have relatively low estimation accuracy . 2. The map system according to claim 1 , wherein the integration unit determines the bias of the data based on at least one of a mounting position, a mounting orientation, and specifications of the imaging device. 3. The map system according to claim 1 or 2 , wherein the integration unit determines the bias of the data based on the travel speed of the vehicle. When the integration unit determines that the accuracy of the target located around the vehicle is less than a predetermined determination accuracy according to the traveling speed of the vehicle, the integration unit prioritizes the data of which the traveling speed of the vehicle is relatively slow. 4. The map system according to claim 3 , wherein the data is weighted so that the priority of the data is higher than that of the data in which the vehicle is traveling at a relatively high speed. 5. The map system according to any one of claims 1 to 4 , wherein the integration unit determines the bias of the data based on the surrounding environment of the vehicle. A vehicle device (9, 32) equipped with an imaging device (3) mounted on a vehicle and configured to capture an image of the surroundings of the vehicle; a server (2) that generates a map using data corresponding to feature points in both , or only said server,
weighting the data based on the bias of the data transmitted from the plurality of vehicle devices, and integrating at least a portion of the plurality of data based on the weighting to generate the map;
let it run ,
In the integration procedure, the data that is determined to have relatively high estimation accuracy when estimating the own vehicle behavior, which is the behavior of the vehicle, by using the image corresponding to the data and using the Structure From Motion method. A map generation program for weighting the data so that the priority of the data is higher than the priority of the data judged to have relatively low estimation accuracy . A computer-readable non-transitory storage medium storing the map generation program according to claim 6 . A vehicle device (9, 32) comprising an imaging device (3) mounted on a vehicle and capturing an image of the surroundings of the vehicle,
The data corresponding to the feature points extracted from the image captured by the imaging device is weighted based on the bias of the data transmitted from the plurality of vehicle devices, and based on the weighting, a plurality of to a server (2) having an integration unit (11) for generating a map by integrating at least part of the data of
Furthermore, a reliability adding unit (38) for adding reliability information, which is information about the reliability of data corresponding to the image, to the data,
The reliability imparting unit evaluates the reliability based on the estimation accuracy when estimating the own vehicle behavior, which is the behavior of the vehicle, using the image corresponding to the data and using the Structure From Motion technique. , a vehicle device that generates the reliability information based on the evaluation result . 9. The vehicle device according to claim 8 , wherein the data corresponding to the image is data relating to the imaging condition of the image by the imaging device. The vehicle device according to claim 8 or 9 , wherein the reliability imparting unit evaluates the reliability based on the estimation accuracy of the road gradient, and generates the reliability information based on the evaluation result. 11. The reliability giving unit according to any one of claims 8 to 10 , wherein the reliability is evaluated based on an estimation accuracy of visibility in the imaging device, and the reliability information is generated based on the evaluation result. A device for a vehicle as described. The reliability adding unit evaluates the reliability based on information regarding landmarks on the image detected based on the image corresponding to the data, and generates the reliability information based on the evaluation result. A vehicle device according to any one of claims 8 to 11 . Corresponding to a feature point extracted from an image captured by an imaging device transmitted from a vehicle device (9, 32) equipped with an imaging device (3) mounted on a vehicle and configured to capture an image around the vehicle A server (2) for generating a map using the data,
an integration unit (11) for weighting the data based on the bias of the data transmitted from the plurality of vehicle devices, and integrating at least part of the plurality of data based on the weighting to generate the map; ) ,
The integration unit uses the image corresponding to the data and uses the structure-from-motion technique to estimate the behavior of the own vehicle, which is the behavior of the vehicle. A server that weights the data so that the priority of the data is higher than the priority of the data determined to have relatively low estimation accuracy .

Images