WO2020115944A1

WO2020115944A1 - Map data generation device

Info

Publication number: WO2020115944A1
Application number: PCT/JP2019/030337
Authority: WO
Inventors: 和斗大森; 林　宏樹
Original assignee: 株式会社Ｎｔｔドコモ
Priority date: 2018-12-03
Filing date: 2019-08-01
Publication date: 2020-06-11
Also published as: JP7134252B2; JPWO2020115944A1

Abstract

This map server is provided with: a storage unit that stores map data in which characteristic amounts of characteristic points included in image data and positional information related to the characteristic points are associated with each other; an acquisition unit that acquires map expansion data based on the image data captured in a plurality of communication terminals; and an updating unit that, on the basis of the map data stored in the storage unit, and the map expansion data acquired by the acquisition unit, updates the map data stored in the storage unit.

Description

Map data generator

One aspect of the present invention relates to a map data generation device.

Reference 1 is based on map data in which a reference image including an image of a feature obtained by capturing in advance and image capturing position information of the reference image are associated with each other, and a current image captured by a camera of a moving body. , A technique for estimating the current position of a moving body is disclosed.

JP, 2017-151148, A

Here, in the technology described in Patent Document 1, it is important to expand the map data in order to improve the accuracy of position estimation.

One aspect of the present invention has been made in view of the above circumstances, and an object thereof is to provide a map data generation device suitable for expanding map data.

A map data generation device according to an aspect of the present invention includes a storage unit that stores map data in which feature amounts of feature points included in image data and position information related to the feature points are associated with each other. Based on the acquisition unit that acquires map expansion data based on the captured image data, the map data stored in the storage unit, and the map expansion data acquired by the acquisition unit, and stored in the storage unit. And an updating unit for updating the existing map data.

In a map data generation device according to an aspect of the present invention, map data that is stored in advance and in which feature amounts of feature points are associated with position information, and map expansion based on image data captured by a plurality of terminals. The map data is updated based on the use data. As described above, by using the image data captured by the plurality of terminals to update the map data, the map data can be easily and surely expanded. As described above, according to one aspect of the present invention, it is possible to provide a map data generation device suitable for expanding map data.

The updating unit may match the feature points of the map data with the feature points of the map extension data, and update the map data using the map extension data based on the matching result. In this way, by matching the feature points between the map data and the map extension data, the correspondence relationship between the map data and the map extension data is appropriately specified, and based on the appropriate feature points of the map extension data. Map data is updated. That is, the map data can be updated appropriately.

The map extension data includes position information related to the map extension data, and the update unit determines that the map information is used when the position information related to the map extension data is existing position information existing in the map data. Information of feature points that do not exist in the map data among the feature points of the extension data may be added as new information of the map data. In this way, when the position information related to the map extension data exists in the map data, that is, when the correspondence between the map extension data and the map data is specified, the map data among the feature points of the map extension data By adding feature points that do not exist in the map data to the map data, the accuracy of the original map data can be improved.

The map extension data includes position information related to the map extension data, and the update unit determines that the map information is used when the position information related to the map extension data is new position information that does not exist in the map data. Even if the map data relating to the existing position information existing in the data and the feature points of the map extension data are matched and the map data is updated using the map extension data based on the matching result. Good. Even if the position information related to the map extension data is new position information that does not exist in the map data, it is considered that some feature points in the map extension data match the feature points of the map data related to the existing position information. Be done. Therefore, by matching the feature points of the map data relating to the existing position information with the feature points of the map extension data and updating the map data, the map information for the map extension and the map data are updated from the position information. Even when the correspondence is not uniquely specified, the map data can be appropriately expanded.

The acquisition unit acquires time-series continuous map expansion data, and when the position information in the time-series continuous map expansion data is changed from the existing position information to the new position information, it becomes the new position information. The feature points of the map data relating to the existing position information may be matched with the feature points of the map extension data, and the map data may be updated using the map extension data based on the matching result. In this way, when the map extension data that is continuous in time series is acquired, the characteristic point of the map data related to the existing position information before (immediately before) changing to the new position information is that the position information is the new position information. It is considered that it is easy to match the feature points of certain map extension data. Therefore, the map data can be more effectively expanded by matching the feature points of the map data relating to the existing position information before becoming the new position information with the feature points of the map extension data.

The update unit may update the map data by converting the format of the map extension data so that it can be combined with the map data, and combining it with the map data. As a result, even if the angle of view and the map scale are different between the map expansion data based on the image data captured by the terminal and the map data, the appropriate conversion processing is performed and the map data is updated appropriately. You can

The storage unit has, as map data, basic data and period-limited data for each of a plurality of periods, and the updating unit updates either the basic data or the plurality of period-limited data based on the map extension data. May be determined. Thus, by determining which of the map data is updated for each of a plurality of periods based on the content of the map expansion data, for example, in the case where the information of the object existing for each period changes, Appropriate map data can be updated according to the period.

The updating unit estimates a region related to a feature point of a dynamic object included in the map extension data, and extracts a static region of the map extension data excluding a region related to a feature point of the dynamic object. The map data may be updated based on the static area. Dynamic objects (moving people, cars, trees whose shape changes depending on the season, etc.) do not exist continuously, and thus become noise when map data is used for positioning, for example. .. That is, the accuracy of the map data may be reduced by updating the map data in consideration of the dynamic object. In this respect, by updating the map data based on the static area excluding the dynamic area, it is possible to eliminate the influence of an object that does not exist continuously and improve the accuracy of the map data. it can.

The updating unit may estimate the area related to the feature point of the dynamic object based on the difference between the feature point in the map extension data and the feature point in the map data. As a result, for example, it is possible to estimate that the feature points that do not exist in the map data are the feature points of the dynamic object, and the map data is updated based on the region related to the feature points of the dynamic object. Can be appropriately avoided.

The updating unit estimates a region related to a feature point of a dynamic object based on a difference of feature points among a plurality of map extension data based on a plurality of image data captured within a predetermined period at the same place. Good. In this way, by considering the difference of the feature points between the plurality of map extension data based on the plurality of image data imaged within the predetermined period at the same place, the dynamic object that temporarily exists It is possible to appropriately estimate the region related to the feature point and appropriately avoid updating the map data based on the region related to the feature point of the dynamic object.

According to an aspect of the present invention, it is possible to provide a map data generation device suitable for expanding map data.

It is a block diagram which shows the function structure of the map server which concerns on this embodiment. It is a figure explaining the characteristic point of image data. It is a sequence diagram which shows the process which a positioning system performs. It is a sequence diagram which shows an example of the detail of the process of step S1 and step S2 shown in FIG. It is a sequence diagram which shows an example of the detail of the process of step S1 and step S2 shown in FIG. It is a sequence diagram which shows an example of the detail of the process of step S3 shown in FIG. It is a figure which shows the hardware constitutions of a map server.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same reference numerals are used for the same or equivalent elements, and duplicate description will be omitted.

FIG. 1 is a block diagram showing the functional configuration of the positioning system 1 according to this embodiment. The positioning system 1 illustrated in FIG. 1 is a positioning system that performs positioning based on image data captured by the communication terminal 50 (terminal). The positioning system 1 is a system that measures the position of the communication terminal 50 in a service that provides AR (Augmented Reality) content according to the position of the communication terminal 50, for example. Hereinafter, the positioning system 1 will be described as a system related to a service that provides AR content, but the positioning system 1 may be a system related to another application. The positioning system 1 includes a map server 10 and a plurality of communication terminals 50.

The communication terminal 50 is, for example, a terminal capable of wireless communication, and is, for example, a smartphone, a tablet terminal, a PC, or the like. The communication terminal 50 receives the provision of the AR content from the content server (not shown) when the user uses the installed application, for example. In the communication terminal 50, when an application is executed, for example, image pickup (continuous image pickup) by the mounted camera is started. Then, the communication terminal 50, based on the positioning result estimated by the map server 10 according to the imaged image data, the imaged image data, and the information (angle of view etc.) of the camera of the communication terminal 50, A content including AR content is provided from a content server (not shown). Each communication terminal 50 included in the positioning system 1 transmits map expansion data based on the captured image data to the map server 10. The map expansion data may be captured image data (moving image or still image data) itself, or may be combined with the map data 100 by adjusting the angle of view and the map scale. Data may be included. Hereinafter, the map expansion data will be described as the captured image data itself.

Further, the communication terminal 50 acquires the position information (global position information) related to the map extension data and transmits it to the map server 10. The position information related to the map expansion data may be position information acquired when the map expansion data is imaged, or position information continuously acquired irrespective of the image acquisition of the map expansion data. May be Even when the position information is acquired regardless of the image capturing of the map expansion data, the position information is continuously acquired in a cycle that is not too long, such as a cycle of several seconds, so that each map expansion data is acquired. The position information of the communication terminal 50 at a time close to the image capturing timing can be acquired. The communication terminal 50 acquires global position information of the communication terminal 50 by performing GPS (Global Positioning System) positioning or base station positioning, for example.

The map server 10 estimates the position information of the communication terminal 50 at the time of image capturing based on the image data captured by the communication terminal 50. The map server 10 stores the map data 100 used for estimating the position information, and updates the map data 100 by acquiring map extension data (that is, imaged image data) from the plurality of communication terminals 50. To do. Updating the map data means adding a new area where the position information does not exist in the map data (expansion of the map data) and improving the accuracy of the map data information regarding the area where the position information exists in the map data. Both (increasing the accuracy of map data) are included. The map server 10 includes a storage unit 11, an acquisition unit 12, an update unit 13, and a positioning unit 14.

The storage unit 11 associates the feature amount (for example, the luminance direction vector) of the feature point included in the image data acquired in advance with the position information (global position information that is absolute position information) related to the feature point. The map data 100 (3D point cloud) that is the obtained point cloud data is stored. The original data of such map data 100 is generated, for example, based on a large amount of image data previously imaged by a stereo camera (not shown) or the like capable of simultaneously imaging an object from a plurality of different directions. Then, the map data 100 is updated by the update unit 13 based on the map extension data (image data captured by the plurality of communication terminals 50). A characteristic point is a point that is prominently detected in an image, and is, for example, a point whose brightness (intensity) is larger (or smaller) than other areas. The position information related to the feature point is position information set in association with the feature point, and is global position information in the real world about the region indicated by the feature point in the image. The position information can be associated with each feature point by a conventionally known method.

The storage unit 11 stores three-dimensional position information as position information related to the characteristic points of the map data 100. The storage unit 11 stores, for example, latitude/longitude/height of a feature point as three-dimensional position information related to the feature point. The storage unit 11 may store a plurality of divided map data obtained by dividing the map data 100 into certain areas according to position information. Areas (global position information) near the boundaries of the divided map data may overlap with each other, or may not overlap with each other.

The storage unit 11 may include, as the map data 100, basic data and period-limited data for each of a plurality of periods. The basic data is the fixed map data 100 regardless of the time. The limited-time data is the map data 100 used only at a specific time. The limited-time data is provided, for example, for each season or each time zone. The areas covered by the basic data and each limited-time data (positional information to be covered) may be the same as each other or may be partially different from each other.

The acquisition unit 12 acquires map extension data based on image data captured by the plurality of communication terminals 50. The map expansion data includes position information (global position information acquired by the communication terminal 50) related to the map expansion data. The acquisition unit 12 acquires map expansion data that is continuous in time series. The acquisition unit 12 outputs the acquired map extension data to the update unit 13 and the positioning unit 14.

The update unit 13 updates the map data 100 stored in the storage unit 11 based on the map data 100 stored in the storage unit 11 and the map extension data acquired by the acquisition unit 12. Here, in the processing by the update unit 13, first, the data conversion processing is performed, and after the data conversion processing, the map data update processing is performed.

In the data conversion process, the update unit 13 performs format conversion of the map extension data so that it can be combined with the map data 100. Specifically, the updating unit 13 acquires internal parameters of the camera of the communication terminal 50, and uses the internal parameters to convert the map extension data into the map data 100 format. The internal parameters of the camera are, for example, the optical center and focal length of the camera. The updating unit 13 converts the map extension data into the map data 100 format by, for example, correcting the angle of view using an internal parameter. The updating unit 13 may acquire the internal parameter of the camera of the communication terminal 50 from the communication terminal 50, or may derive the internal parameter based on the information indicating the communication terminal 50 transmitted from the communication terminal 50, for example. When deriving the internal parameter from the information indicating the communication terminal 50, for example, the storage unit 11 of the map server 10 stores information in which the type of the communication terminal 50 and the internal parameter are associated with each other, and the update unit 13 By referring to the information, the internal parameter corresponding to the communication terminal 50 is derived. Further, the update unit 13 performs a process (coordinate conversion process) of adjusting the three-dimensional (x, y, z) direction and the map scale to the map data 100 for the map extension data.

In the map data update process after the data conversion process, the update unit 13 matches the feature points of the map data 100 with the feature points of the map extension data, and uses the map extension data based on the matching result. To update the map data 100. As shown in FIG. 2, the image data ID that is the map expansion data includes a plurality of feature points FP related to an object such as a building. By performing matching processing between the map extension data and the map data 100 for such a plurality of feature points, the latest information not reflected in the map data 100 (for example, feature points included only in the map extension data). Information) can be reflected (combined) in the map data 100 to update the map data 100.

The updating unit 13 narrows down the matching target area in the map data 100 based on the position information (position information acquired by the communication terminal 50) related to the map extension data. That is, the updating unit 13 identifies an area of the map data 100 that matches or approximates the position information of the map extension data as an area to be matched, and the feature points of the area to be matched and the feature points of the map extension data. Match with. When the position information related to the map extension data is the existing position information existing in the map data 100, the update unit 13 is the feature point information of the map extension data that does not exist in the map data 100. Is added as new information of the map data 100. By adding the information of the map data 100 to the area where the position information exists in the map data 100, the accuracy of the map data 100 can be improved.

In addition, when the position information related to the map extension data is new position information that does not exist in the map data 100, the update unit 13 determines the characteristic points of the map data 100 related to the existing position information that exists in the map data 100. , The map data 100 is updated using the map expansion data based on the result of the matching. When the position information is changed from the existing position information to the new position information in the map extension data that is continuous in time series, the updating unit 13 is a feature point of the map data 100 related to the existing position information before becoming the new position information. And the feature points of the map extension data are matched, and the map data 100 is updated using the map extension data based on the matching result. That is, for example, the update processing is performed from an area (position of existing position information) where the position information exists in the map data 100 in a state where the application is activated and the communication terminal 50 is continuously capturing an image. This is executed when the communication terminal 50 moves to a region where 100 does not have position information (a region of new position information). When moving from the existing position information to the new position information, the matching target area in the map data 100 cannot be narrowed down based on the position information related to the map extension data, but some features in the map extension data It is considered that the points match the feature points (feature points of the peripheral region) of the map data 100 related to the existing position information before (nearest) the new position information. Therefore, when the position information related to the map expansion data is new position information that does not exist in the map data 100, the feature points of the map data 100 related to the existing position information before becoming the new position information and the map expansion Even if the communication terminal 50 moves to the new position information, the feature points can be appropriately matched by adding the feature points to the map data 100 by adding the feature points to the map data 100. it can. In this way, the map data 100 can be expanded by newly adding the feature points in the area where the position information does not exist in the map data 100.

The updating unit 13 may determine which of the basic data and the plurality of time limited data in the map data 100 is updated based on the map extension data. That is, for example, when the limited-time data is provided for each season or each time zone, the update unit 13 captures the date and time when the map expansion data was captured (or the date and time when the acquisition unit 12 acquired the map expansion data). ), it is possible to determine which of the time-limited data is updated. Further, the updating unit 13 may determine which period-limited data to update based on the brightness information and the contrast information of the image that changes according to information such as weather. In addition, the update unit 13 updates the basic data when it recognizes an object that is assumed to be installed regardless of the season or the date in the map extension data, for example, by the image recognition technique, and updates the season or the date and time. When the object that is supposed to be limited is recognized, the corresponding limited-time data may be updated.

The updating unit 13 estimates a region related to the feature point of the dynamic object included in the map extension data, and extracts a static region excluding the region related to the feature point of the dynamic object from the map extension data. However, the map data 100 may be updated based on the static area. The dynamic object is a moving object such as a person or a car, and is usually an object not included in the map data 100. That is, the static area is an area excluding the area of a moving object. Note that the dynamic object here does not include an object that is a real estate (building, standing tree, etc.) but slightly moves due to a natural phenomenon such as wind. The updating unit 13 may estimate the region related to the feature point of the dynamic object based on the difference between the feature point in the map extension data and the feature point in the map data 100. That is, the updating unit 13 may estimate the feature points relating to the object that does not exist in the map data 100 as the feature points of the dynamic object. Further, the updating unit 13 may estimate the region related to the feature point of the dynamic object based on the difference of the feature point between the plurality of pieces of map expansion data imaged within the predetermined period for the same place. That is, the updating unit 13 estimates the feature points relating to the object existing only in a part of the map extension data among the plurality of map extension data within the predetermined period in the same place as the feature points of the dynamic object. You may. It should be noted that the plurality of map extension data captured at the same place do not have to be map extension data captured at completely the same place, and for example, a plurality of map extension data having at least one characteristic point being the same as each other. Any data can be used for map expansion. The updating unit 13 may identify a dynamic object such as a person or a car in the map extension data by performing image recognition. The update unit 13 may specify a dynamic object by performing, for example, a semantic segmentation process and specifying image data in pixel units.

The map server 10 may send the updated map data 100 to a mesh server (not shown). The mesh server (not shown) is a device that generates mesh data based on the map data 100. The mesh data is 3D polygon data associated with the map data 100 that is a 3D point cloud. That is, the mesh data is 3D polygon data, and has three-dimensional position information (latitude/longitude/height) associated in the map data 100. The mesh server (not shown) may generate mesh data of only the static area used for updating the map data 100 in the updating unit 13 described above. As a result, it is possible to generate a mesh server that eliminates the effects of dynamic objects (moving people, vehicles, trees whose shape changes depending on the season) and the like.

The positioning unit 14 uses the image data captured by the communication terminal 50 and the map data 100 stored in the storage unit 11 to determine global position information (three-dimensional image) of the communication terminal 50 when the communication terminal 50 captures an image. Position information) is estimated. Specifically, the positioning unit 14 performs matching between the characteristic points of the map data 100 and the characteristic points of the image data captured by the communication terminal 50 to determine the area of the map data 100 corresponding to the captured image data. Identify. Then, the positioning unit 14 determines the imaging position of the image data (that is, the global position information of the communication terminal 50 at the time of imaging) based on the global position information associated with the feature points of the map data 100 related to the specified area. presume. The positioning unit 14 may perform positioning of the communication terminal 50 using any one of the limited-time data in the map data 100 based on the information about the season and the date and time. In addition to the global position information, the positioning result may include information about the direction (direction in the three-dimensional coordinates of roll, pitch, and yaw) estimated from the image data.

The positioning unit 14 may estimate the global position information of the communication terminal 50 using one or a plurality of division map data selected from a plurality of division map data. That is, the positioning unit 14 acquires the terminal acquisition position information which is the global position information of the communication terminal acquired by the communication terminal 50 together with the image data captured by the communication terminal 50, and a plurality of terminal acquisition position information are acquired according to the terminal acquisition position information. In the communication terminal 50, one or a plurality of pieces of division map data are selected from the division map data, and the feature points of the selected division map data are matched with the feature points of the image data captured by the communication terminal 50. Global position information of the communication terminal 50 at the time of image capturing may be estimated. The positioning unit 14 selects the division map data including the position indicated by the terminal acquisition position information, for example.

Next, the processing performed by the positioning system 1 will be described with reference to FIGS. 3 to 6. FIG. 3 is a sequence diagram showing processing performed by the positioning system 1.

As shown in FIG. 3, each communication terminal 50 transmits map expansion data based on the captured image data to the map server 10 (step S1). Subsequently, the map server 10 performs the data conversion process on the map extension data (step S2) and then performs the map data update process (step S3). The details of each process will be described below with reference to FIGS. 4 to 6.

4 and 5 are sequence diagrams showing different aspects of the details of the processing of step S1 and step S2 shown in FIG.

In the example shown in FIG. 4, the communication terminal 50 acquires the optical center and focal length of the camera, which are internal parameters of the camera (step S101). Then, the communication terminal 50 transmits the map expansion data (including the global position information related to the map expansion data) and the internal parameter to the map server 10 (step S102). In this case, the map server 10 uses the internal parameters to convert the map expansion data into the format of the map data 100 (specifically, the angle of view is corrected) (step S103). Further, the map server 10 performs a process (coordinate conversion process) of adjusting the three-dimensional (x, y, z) direction and the map scale to the map data 100 with respect to the map extension data (step S104).

In the example shown in FIG. 5, unlike the example shown in FIG. 4, the internal parameters of the camera are derived in the map server 10. In the example shown in FIG. 5, the communication terminal 50 acquires terminal information indicating the communication terminal 50 (information indicating the type of the communication terminal 50) (step S201). Then, the communication terminal 50 transmits the map expansion data (including the global position information related to the map expansion data) and the terminal information to the map server 10 (step S202). In this case, the map server 10 determines whether or not the terminal information of the communication terminal 50 exists in the information (list of compatible terminals) stored in the map server 10 in which the types of communication terminals are associated with the internal parameters. Is determined (step S203). If the terminal information does not exist in step S203, the map server 10 returns the error information to the communication terminal 50 (step S204), and the process ends. On the other hand, when the terminal information exists in step S203, the map server 10 acquires the internal parameter corresponding to the terminal information of the communication terminal 50 based on the list of compatible terminals (step S205), and uses the internal parameter. The map extension data is converted into the map data 100 format (specifically, the angle of view is corrected) (step S206). Further, the map server 10 performs a process (coordinate conversion process) of adjusting the three-dimensional (x, y, z) direction and the map scale to the map data 100 with respect to the map extension data (step S207).

FIG. 6 is a sequence diagram showing an example of details of the process (map data update process) of step S3 shown in FIG. As shown in FIG. 6, in the map data update processing, the map server 10 first determines whether or not the position information (position information acquired by the communication terminal 50) regarding the map extension data exists in the map data 100. Is determined (step S301). When it is determined in step S301 that the position information related to the map expansion data exists in the map data 100, the map server 10 narrows down the matching target area in the map data 100 based on the position information related to the map expansion data. Information of feature points that do not exist in the map data 100 among the feature points of the map extension data is added as new information of the map data 100 to improve the accuracy of the map data 100 (step S302). On the other hand, when it is determined in step S301 that the position information related to the map extension data is new position information that does not exist in the map data 100, the map server 10 relates to the existing position information before becoming the new position information. Matching the characteristic points of the map data 100 (that is, the characteristic points of the peripheral region) with the characteristic points of the map extension data, and updating the map data 100 using the map extension data based on the result of the matching, The map data is expanded (step S303).

Next, the operation and effect of the map server 10 (map data generation device) according to this embodiment will be described.

The map server 10 stores the map data 100 in which the feature amounts of the feature points included in the image data and the position information related to the feature points are stored, and the images captured by the plurality of communication terminals 50. Based on the acquisition unit 12 that acquires map expansion data based on the data, the map data 100 that is stored in the storage unit 11, and the map expansion data that is acquired by the acquisition unit 12, it is stored in the storage unit 11. The update unit 13 that updates the existing map data 100.

In the map server 10, based on the map data 100 stored in advance in which the feature amounts of the feature points and the position information are associated with each other, and the map extension data based on the image data captured by the plurality of communication terminals 50. , The map data 100 is updated. As described above, by using the image data captured by the plurality of communication terminals 50 to update the map data 100, the map data 100 can be easily and surely expanded. That is, by performing cloud sourcing by collecting image data (3D point cloud) captured by the user of the communication terminal 50 during the AR experience and expanding the map data 100 that is a point cloud for self-position recognition, Large scale map data 100 can be constructed. As described above, according to the present embodiment, it is possible to provide the map server 10 suitable for expanding the map data 100. By easily expanding the map data 100, the technical effect of reducing the processing load on the processing unit such as the CPU related to the expansion of the map data 100 is also achieved.

The updating unit 13 matches the feature points of the map data 100 with the feature points of the map extension data, and updates the map data 100 using the map extension data based on the matching result. In this way, by matching the feature points between the map data 100 and the map extension data, the correspondence relationship between the map data 100 and the map extension data is appropriately identified, and the appropriate feature points of the map extension data are identified. The map data 100 is updated based on That is, the map data 100 can be updated appropriately.

The map expansion data includes position information related to the map expansion data, and the updating unit 13 determines that the position information related to the map expansion data is the existing position information existing in the map data 100. Of the characteristic points of the map extension data, information of characteristic points that do not exist in the map data 100 is added as new information of the map data 100. In this way, when the position information related to the map extension data exists in the map data 100, that is, when the correspondence between the map extension data and the map data 100 is specified, among the feature points of the map extension data, By adding feature points that do not exist in the map data 100 to the map data 100, the original map data 100 can be made more accurate.

The map expansion data includes position information related to the map expansion data, and the updating unit 13 determines that the position information related to the map expansion data is new position information that does not exist in the map data 100. , The map data 100 relating to the existing position information existing in the map data 100 is matched with the feature points of the map extension data, and based on the result of the matching, the map extension data is used to generate the map data 100. Update. Even when the position information related to the map extension data is new position information that does not exist in the map data 100, some feature points in the map extension data match the feature points of the map data 100 related to the existing position information. It is thought that. Therefore, by matching the feature points of the map data 100 related to the existing position information with the feature points of the map extension data and updating the map data 100, the map extension data and the map data are updated from the position information. Even when the correspondence with 100 is not uniquely specified, the map data 100 can be appropriately expanded.

The acquisition unit 12 acquires time-series continuous map expansion data, and in the time-series continuous map expansion data, when the position information is changed from the existing position information to the new position information, it becomes new position information. Even if the feature points of the map data 100 relating to the previous existing position information are matched with the feature points of the map extension data, and the map data 100 is updated using the map extension data based on the matching result. Good. In this way, when the map extension data that is continuous in time series is acquired, the feature point of the map data 100 related to the existing position information before (immediately before) the change to the new position information is that the position information is the new position information. It is conceivable that the feature points of the map extension data are easily matched. Therefore, the map data 100 can be more effectively expanded by matching the feature points of the map data 100 related to the existing position information before becoming the new position information with the feature points of the map extension data. it can.

The update unit 13 updates the map data 100 by performing format conversion on the map extension data so that it can be combined with the map data 100 and combining the map extension data with the map data 100. As a result, even if the map expansion data based on the image data captured by the communication terminal 50 and the map data 100 have different angles of view and map scales, appropriate conversion processing is performed and the map data 100 is appropriately converted. Can be updated. By enabling the format conversion described above, if the format of the optimum image data in the communication terminal 50 whose processing load is to be reduced and the format of the map data 100 in the map server 10 desired to perform processing with an emphasis on accuracy are different. It becomes possible.

The storage unit 11 has basic data and period-limited data for each of a plurality of periods as the map data 100, and the update unit 13 stores either the basic data or the plurality of period-limited data based on the map extension data. Decide whether to update. Thus, by determining which of the map data is updated for each of a plurality of periods based on the content of the map extension data, for example, when the information of the object changes for each period, Appropriate map data can be updated accordingly.

The updating unit 13 estimates a region related to the feature point of the dynamic object included in the map extension data, and extracts a static region excluding the region related to the feature point of the dynamic object from the map extension data. Then, the map data 100 is updated based on the static area. Dynamic objects (moving people, vehicles, trees whose shape changes depending on the season, etc.) do not exist continuously, and thus become noise when the map data 100 is used for positioning, for example. I will end up. That is, the accuracy of the map data 100 may be reduced by updating the map data 100 in consideration of a dynamic object. In this respect, by updating the map data 100 based on the static area excluding the dynamic area, it is possible to eliminate the influence of an object that does not exist continuously, and improve the accuracy of the map data 100. be able to.

The updating unit 13 estimates the area related to the feature point of the dynamic object based on the difference between the feature point in the map extension data and the feature point in the map data 100. As a result, for example, it becomes possible to estimate that the feature points that do not exist in the map data 100 are the feature points of the dynamic object, and the map data 100 is updated based on the region related to the feature points of the dynamic object. Can be avoided appropriately.

The update unit 13 estimates a region related to a feature point of a dynamic object based on a difference in feature point between a plurality of pieces of map extension data based on a plurality of image data captured in a predetermined period for the same place. .. In this way, by considering the difference of the feature points between the plurality of map extension data based on the plurality of image data imaged within the predetermined period at the same place, the dynamic object that temporarily exists It is possible to appropriately estimate the region related to the feature point and appropriately avoid updating the map data 100 based on the region related to the feature point of the dynamic object.

Finally, the hardware configuration of the map server 10 will be described with reference to FIG. The map server 10 described above may be physically configured as a computer device including a processor 1001, a memory 1002, a storage 1003, a communication device 1004, an input device 1005, an output device 1006, a bus 1007, and the like.

Note that in the following description, the word "device" can be read as a circuit, device, unit, or the like. The hardware configuration of the map server 10 may be configured to include one or a plurality of each device illustrated in the figure, or may be configured not to include some devices.

Each function in the map server 10 causes a predetermined software (program) to be loaded on hardware such as the processor 1001 and the memory 1002, so that the processor 1001 performs an arithmetic operation, communication by the communication device 1004, memory 1002 and storage 1003. It is realized by controlling the reading and/or writing of data in.

The processor 1001 operates an operating system to control the entire computer, for example. The processor 1001 may be composed of a central processing unit (CPU) including an interface with peripheral devices, a control device, a calculation device, a register, and the like. For example, the control function of the positioning unit 14 or the like of the map server 10 may be realized by the processor 1001.

Further, the processor 1001 reads a program (program code), software module, and data from the storage 1003 and/or the communication device 1004 into the memory 1002, and executes various processes according to these. As the program, a program that causes a computer to execute at least part of the operations described in the above-described embodiments is used. For example, the control function of the positioning unit 14 or the like of the map server 10 may be realized by a control program stored in the memory 1002 and operated by the processor 1001, or may be realized similarly for other functional blocks. Although it has been described that the various processes described above are executed by one processor 1001, they may be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented by one or more chips. The program may be transmitted from the network via an electric communication line.

The memory 1002 is a computer-readable recording medium, and is composed of, for example, at least one of ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (ElectricallyErasable Programmable ROM), RAM (Random Access Memory), and the like. May be done. The memory 1002 may be called a register, a cache, a main memory (main storage device), or the like. The memory 1002 can store a program (program code) executable to implement the wireless communication method according to the embodiment of the present invention, a software module, and the like.

The storage 1003 is a computer-readable recording medium, for example, an optical disc such as a CD-ROM (Compact Disc ROM), a hard disc drive, a flexible disc, a magneto-optical disc (for example, a compact disc, a digital versatile disc, a Blu-ray disc). (Registered trademark) disk), smart card, flash memory (for example, card, stick, key drive), floppy (registered trademark) disk, magnetic strip, and the like. The storage 1003 may be called an auxiliary storage device. The storage medium described above may be, for example, a database including the memory 1002 and/or the storage 1003, a server, or another appropriate medium.

The communication device 1004 is hardware (transmission/reception device) for performing communication between computers via a wired and/or wireless network, and is also referred to as a network device, network controller, network card, communication module, or the like.

The input device 1005 is an input device (for example, a keyboard, a mouse, a microphone, a switch, a button, a sensor, etc.) that receives an input from the outside. The output device 1006 is an output device (for example, a display, a speaker, an LED lamp, etc.) that performs output to the outside. The input device 1005 and the output device 1006 may be integrated (for example, a touch panel).

Each device such as the processor 1001 and the memory 1002 is connected by a bus 1007 for communicating information. The bus 1007 may be composed of a single bus, or may be composed of different buses among devices.

Further, the map server 10 includes hardware such as a microprocessor, a digital signal processor (DSP: Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array). It may be configured, and the hardware may implement some or all of the functional blocks. For example, processor 1001 may be implemented with at least one of these hardware.

Although the present embodiment has been described in detail above, it is obvious to those skilled in the art that the present embodiment is not limited to the embodiment described in this specification. The present embodiment can be implemented as modified and changed modes without departing from the spirit and scope of the present invention defined by the description of the claims. Therefore, the description of the present specification is for the purpose of exemplifying explanation, and does not have any restrictive meaning to the present embodiment.

Each aspect/embodiment described in this specification is LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G, 5G, FRA (Future Radio Access), W-CDMA. (Registered trademark), GSM (registered trademark), CDMA2000, UMB (Ultra Mobile Broad-band), IEEE 802.11 (Wi-Fi), IEEE 802.16 (WiMAX), IEEE 802.20, UWB (Ultra-Wide) Band), Bluetooth (registered trademark), or any other system using an appropriate system and/or a next-generation system extended based on the system.

The order of the processing procedures, sequences, flowcharts, etc. of each aspect/embodiment described in this specification may be changed as long as there is no contradiction. For example, the methods described herein present elements of the various steps in a sample order, and are not limited to the specific order presented.

Information that has been input and output may be stored in a specific location (for example, memory), or may be managed in a management table. Information that is input/output can be overwritten, updated, or added. The output information and the like may be deleted. The input information and the like may be transmitted to another device.

The determination may be performed by a value represented by 1 bit (whether 0 or 1), may be performed by a Boolean value (Boolean: true or false), and may be performed by comparing numerical values (for example, a predetermined value). (Comparison with the value).

The respective aspects/embodiments described in this specification may be used alone, in combination, or may be switched according to execution. Further, the notification of the predetermined information (for example, the notification of “being X”) is not limited to the explicit notification, but is performed implicitly (for example, the notification of the predetermined information is not performed). Good.

Software, whether called software, firmware, middleware, microcode, hardware description language, or any other name, instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules. , Application, software application, software package, routine, subroutine, object, executable, thread of execution, procedure, function, etc. should be construed broadly.

Also, software, instructions, etc. may be sent and received via a transmission medium. For example, the software may use wired technologies such as coaxial cable, fiber optic cable, twisted pair and digital subscriber line (DSL) and/or wireless technologies such as infrared, wireless and microwave to websites, servers, or other When transmitted from a remote source, these wireline and/or wireless technologies are included within the definition of transmission medium.

The information, signals, etc. described herein may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description include voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any of these. May be represented by a combination of

Note that the terms described in the present specification and/or the terms necessary for understanding the present specification may be replaced with terms having the same or similar meanings.

The information, parameters, and the like described in this specification may be represented by absolute values, relative values from predetermined values, or may be represented by other corresponding information. ..

User terminals are defined by those skilled in the art as mobile communication terminals, subscriber stations, mobile units, subscriber units, wireless units, remote units, mobile devices, wireless devices, wireless communication devices, remote devices, mobile subscriber stations, access terminals, It may also be referred to as a mobile terminal, wireless terminal, remote terminal, handset, user agent, mobile client, client, or some other suitable term.

As used herein, the terms "determining" and "determining" may encompass a wide variety of actions. "Judgment" and "decision" mean, for example, calculating, computing, processing, deriving, investigating, looking up (e.g., table, database or another). (Search in data structure), ascertaining that it is regarded as "judgment" and "decision" can be included. In addition, "decision" and "decision" include receiving (eg, receiving information), transmitting (eg, transmitting information), input (input), output (output), access (accessing) (for example, accessing data in a memory) may be regarded as “judging” and “deciding”. In addition, "judgment" and "decision" are considered to be "judgment" and "decision" when things such as resolving, selecting, choosing, establishing, establishing, and comparing are done. May be included. That is, the “judgment” and “decision” may include considering some action as “judgment” and “decision”.

As used herein, the phrase “based on” does not mean “based only on,” unless expressly specified otherwise. In other words, the phrase "based on" means both "based only on" and "based at least on."

As long as the terms "include", "including", and variations thereof are used in the present specification or claims, these terms are the same as the term "comprising." It is intended to be comprehensive. Furthermore, the term “or” as used in the specification or claims is not intended to be an exclusive OR.

In the present specification, a device including a plurality of devices is also included unless it is a device in which only one clearly exists in terms of context or technology.

Throughout this disclosure, the plural is included unless the context clearly indicates the singular.

10... Map server (map data generation device), 11... Storage unit, 12... Acquisition unit, 13... Update unit, 50... Communication terminal (terminal), 100... Map data.

Claims

A storage unit that stores map data in which the feature amounts of the feature points included in the image data and the position information related to the feature points are associated with each other;
An acquisition unit that acquires map expansion data based on image data captured by a plurality of terminals,
A map including: the map data stored in the storage unit; and an update unit that updates the map data stored in the storage unit based on the map extension data acquired by the acquisition unit. Data generator.
The update unit performs matching between a feature point of the map data and a feature point of the map extension data, and updates the map data using the map extension data based on a result of the matching. Item 1. The map data generation device according to item 1.
The map expansion data includes position information related to the map expansion data,
The update unit is
In the case where the position information related to the map expansion data is existing position information existing in the map data, the information of the characteristic points that do not exist in the map data among the characteristic points of the map expansion data is The map data generation device according to claim 2, which is added as new information of map data.
The map expansion data includes position information related to the map expansion data,
The update unit is
When the position information related to the map expansion data is new position information that does not exist in the map data, the characteristic points of the map data related to the existing position information existing in the map data and the map expansion data The map data generation device according to claim 2 or 3, wherein the map data is updated by using the map extension data based on the result of the matching.
The acquisition unit acquires the map expansion data continuous in time series,
In the map expansion data continuous in time series, when the position information is changed from existing position information to new position information, the characteristic points of the map data related to the existing position information before becoming the new position information, The map data generation device according to claim 4, wherein matching is performed with a feature point of the map expansion data, and the map data is updated using the map expansion data based on a result of the matching.
6. The updating unit updates the map data by performing a format conversion on the map extension data so that the map extension data can be combined with the map data, and combining the map expansion data with the map data. The map data generation device according to claim 1.
The storage unit has, as the map data, basic data and period-limited data for each of a plurality of periods,
The map data generation device according to any one of claims 1 to 6, wherein the update unit determines which of the basic data and a plurality of the time limited data is updated based on the map extension data.
The update unit estimates a region related to a feature point of a dynamic object included in the map extension data, and statically excludes a region related to the feature point of the dynamic object from the map extension data. The map data generation device according to any one of claims 1 to 7, which extracts a region and updates the map data based on the static region.
The map data generation device according to claim 8, wherein the updating unit estimates a region related to the feature point of the dynamic object based on a difference between the feature point in the map extension data and the feature point in the map data. ..
The update unit estimates a region related to the feature point of the dynamic object based on a difference in feature point between a plurality of pieces of map extension data based on a plurality of image data captured within a predetermined period for the same place. The map data generation device according to claim 8.