WO2020115945A1

WO2020115945A1 - Positioning system

Info

Publication number: WO2020115945A1
Application number: PCT/JP2019/030360
Authority: WO
Inventors: 和斗大森; 林　宏樹
Original assignee: 株式会社Ｎｔｔドコモ
Priority date: 2018-12-03
Filing date: 2019-08-01
Publication date: 2020-06-11
Also published as: JP2022028981A

Abstract

This positioning system is provided with: a local position acquisition unit that continuously acquires local positional information being relative positional information of a communication terminal; a storage unit that stores map data; a positioning unit that, on the basis of image data captured in the communication terminal and the map data stored in the storage unit, estimates global positional information of the communication terminal at the time of imaging in the communication terminal: and a position derivation unit that, on the basis of a change in the local positional information acquired by the local position acquisition unit, estimates a movement amount from the time of imaging in the communication terminal, and on the basis of the movement amount, corrects the global positional information estimated by the positioning unit, thereby deriving positional information of the communication terminal with consideration given to the movement amount.

Description

Positioning system

One aspect of the present invention relates to a positioning system.

Reference 1 describes map data in which a reference image including an image of a feature obtained by capturing an image in advance and image capturing position information of the reference image are associated with each other, and a current image captured by a camera of a mobile body (terminal). A technique for estimating the current position of the terminal based on the above is disclosed.

JP, 2017-151148, A

Here, for example, when a user who uses the terminal moves and the position of the terminal changes after the image is captured by the terminal, the position of the terminal is highly accurately estimated based on the captured image and map data. Even if it is done, the actual position of the terminal and the positioning estimation result may deviate from each other. In this case, the positioning accuracy cannot be sufficiently ensured.

One aspect of the present invention has been made in view of the above circumstances, and aims to improve positioning accuracy.

A positioning system according to an aspect of the present invention is a positioning system that performs positioning based on image data captured by a terminal, and continuously stores local position information that is relative position information of the terminal with respect to a certain point. Map data in which the local position acquisition unit to be acquired in step S1 is associated with the feature amount of the feature points included in the image data acquired in advance and the global position information that is absolute position information associated with the feature points. A storage unit to store, a global position estimation unit that estimates global position information of the terminal at the time of image capturing in the terminal based on image data captured in the terminal, and map data stored in the storage unit, and a local position By estimating the amount of movement from the time of imaging at the terminal based on the change in the local position information acquired by the obtaining unit, and correcting the global position information estimated by the global position estimating unit based on the moving amount. And a position derivation unit that derives position information of the terminal in consideration of the movement amount.

In the positioning system according to one aspect of the present invention, the image data captured by the terminal, the map data (the feature amount of the feature point included in the image data acquired in advance, and the global position information of the feature point are associated with each other. Data) and the global position information of the terminal at the time of imaging at the terminal is estimated. Here, for example, when the user who uses the terminal moves and the position of the terminal changes after the image data is captured by the terminal, the global position information is high based on the captured image data and the map data. Even if estimated accurately, it is possible that the actual position of the terminal deviates from the positioning result. In this regard, in the positioning system according to one aspect of the present invention, the amount of movement of the terminal from the time of image capturing in the terminal is estimated from the continuously acquired local position information of the terminal, and the global position information is calculated based on the movement amount. Is corrected, and the position information of the terminal considering the amount of movement is derived. In this way, the movement amount is estimated based on the continuously acquired local position information of the terminal, the global position information is corrected according to the movement amount, and the final position information of the terminal is derived, Even if the position of the terminal changes after the image data is captured, the actual position of the terminal can be specified with high accuracy. As described above, the positioning system according to one aspect of the present invention can improve positioning accuracy.

The global position estimation unit performs matching between the feature points of the map data and the feature points of the image data captured by the terminal, and based on the global position information associated with the feature points of the map data, the Global location information of the terminal may be estimated. In this way, by matching the feature points between the map data and the imaged image data, the correspondence relationship between the map data and the image data is appropriately specified, and the global position information associated with the feature points of the map data is obtained. Based on, it is possible to properly estimate the global position information of the terminal at the time of image capturing.

The storage unit stores a plurality of divided map data obtained by dividing the map data into fixed regions according to the global position information, and the global position estimation unit stores the image data captured by the terminal and the terminal acquired by the terminal. Terminal acquisition position information that is global position information is acquired, one or a plurality of division map data is selected from a plurality of division map data according to the terminal acquisition position information, and the feature points of the selected division map data and the terminal The global position information of the terminal at the time of imaging at the terminal may be estimated based on the result of matching with the feature point of the image data captured at. In this way, the map data is divided into a plurality of pieces according to the global position information, and the division map data to be matched is selected from the plurality of pieces of division map data according to the terminal acquisition position information, so that the matching range (search The range) can be narrowed down, and the efficiency of matching processing can be improved.

The positioning system 1 determines whether or not the image data can be used for estimation of global position information by the global position estimation unit based on the content of the image data captured by the terminal, and if the image data cannot be used, Further, an image capturing instructing unit for instructing to capture new image data may be further provided. In this way, in the previous stage of the position estimation by the global position estimation unit, when the captured image data is not suitable for the estimation, the image pickup of new image data is prompted, so that the global position estimation unit performs unnecessary processing. It is possible to avoid being hit and to reduce the total positioning time.

The image capturing instruction unit determines that the image data cannot be used for estimating the global position information by the global position estimating unit when the image data captured by the terminal includes a dynamic object at a predetermined value or more. May be. It is considered that dynamic objects do not exist continuously and are not recorded in the map data. Therefore, the dynamic object becomes noise in the estimation of the global position information, which may cause a decrease in positioning accuracy. In this regard, by determining that the image data cannot be used for estimating the global position information when the image data includes a dynamic object or more by a predetermined value or more, the dynamic object described above is caused. It is possible to suppress a decrease in positioning accuracy.

Even if the image capturing instruction unit determines that the image data cannot be used for the estimation of the global position information by the global position estimating unit when the image data captured by the terminal includes motion blur of a predetermined value or more. Good. If the image data contains a large amount of motion blur, matching of feature points cannot be performed properly, which may cause a decrease in positioning accuracy. In this regard, when the image data includes motion blur of a predetermined value or more, it is determined that the image data cannot be used for estimating global position information, and thus the positioning accuracy is deteriorated due to the motion blur described above. Can be suppressed.

The storage unit stores three-dimensional position information as global position information of the map data, the global position estimation unit estimates three-dimensional position information as global position information of the terminal, and the local position acquisition unit 3D position information may be acquired as the local position information of. This allows the position deriving unit to derive three-dimensional position information.

According to one aspect of the present invention, positioning accuracy can be improved.

It is a block diagram which shows the functional structure of the positioning system which concerns on this embodiment. It is a sequence diagram which shows the process which a positioning system performs. It is a figure which shows the hardware constitutions of the positioning server and communication terminal contained in a positioning system.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same reference numerals are used for the same or equivalent elements, and duplicate description will be omitted.

FIG. 1 is a block diagram showing the functional configuration of the positioning system 1 according to this embodiment. The positioning system 1 illustrated in FIG. 1 is a positioning system that performs positioning based on image data captured by the communication terminal 50 (terminal). The positioning system 1 is a system that measures the position of the communication terminal 50 in a service that provides AR (Augmented Reality) content according to the position of the communication terminal 50, for example. Hereinafter, the positioning system 1 will be described as a system related to a service that provides AR content, but the positioning system 1 may be a system related to another application. The positioning system 1 includes a position positioning server 10 and a communication terminal 50. Although only one communication terminal 50 is shown in FIG. 1, a plurality of communication terminals 50 may actually be included.

The position positioning server 10 estimates global position information, which is absolute position information of the communication terminal 50 at the time of image capturing, based on image data captured by the communication terminal 50. The position positioning server 10 has a storage unit 11 and a positioning unit 12 (global position estimation unit).

The storage unit 11 associates a feature amount (for example, a luminance direction vector) of a feature point included in image data acquired in advance with global position information, which is absolute position information associated with the feature point, in a map. The data 100 is stored. Such map data 100 is generated, for example, based on a large amount of image data previously imaged by a stereo camera (not shown) or the like that can simultaneously image an object from a plurality of different directions. A characteristic point is a point that is prominently detected in an image, and is, for example, a point whose brightness (intensity) is larger (or smaller) than other areas. The global position information of the feature point is global position information set in association with the feature point, and is global position information in the real world about the region indicated by the feature point in the image. The global position information can be associated with each feature point by a conventionally known method.

The storage unit 11 stores three-dimensional position information as global position information of feature points of the map data 100. The storage unit 11 stores, for example, latitude/longitude/height of a feature point as three-dimensional global position information of the feature point. The storage unit 11 stores a plurality of pieces of divided map data obtained by dividing the map data 100 into certain areas according to global position information. Areas (global position information) near the boundaries of the divided map data may overlap with each other, or may not overlap with each other. By overlapping the areas near the boundaries of each divided map data with each other, the terminal acquisition position information (details described later), which is the global position information of the communication terminal acquired by the communication terminal 50, is near the boundary of each divided map data. Even in such a case, it is possible to avoid the situation where the division map data corresponding to the terminal acquisition position information does not exist (the map data cannot be referred to), and the corresponding division map data can be surely selected (details will be described later). .. In the following description, it is assumed that the storage unit 11 stores a plurality of divided map data, but the storage unit 11 may store one undivided map data.

Based on the image data captured by the communication terminal 50 and the map data 100 stored in the storage unit 11, the positioning unit 12 determines global position information (in the three-dimensional space) of the communication terminal 50 when the communication terminal 50 captures an image. Position information) is estimated. Specifically, the positioning unit 12 performs matching between the characteristic points of the map data 100 and the characteristic points of the image data captured by the communication terminal 50, and determines the area of the map data 100 corresponding to the captured image data. Identify. Then, the positioning unit 12 estimates the imaging position of the image data (that is, the global position information of the communication terminal 50 at the time of imaging) based on the global position information associated with the feature point of the map data 100 related to the specified area. To do.

More specifically, the positioning unit 12 estimates global position information of the communication terminal 50 using one or a plurality of division map data selected from a plurality of division map data. That is, the positioning unit 12 acquires the terminal acquisition position information which is the global position information of the communication terminal acquired by the communication terminal 50 together with the image data captured by the communication terminal 50 (details will be described later), and the terminal acquisition position information. According to the above, one or a plurality of divided map data are selected from the plurality of divided map data, and based on the result of matching between the characteristic points of the selected divided map data and the characteristic points of the image data captured by the communication terminal 50. Estimate global position information of the communication terminal 50 at the time of image capturing in the communication terminal 50. The positioning unit 12 selects the division map data including the position indicated by the terminal acquisition position information, for example. The position positioning server 10 transmits the positioning result by the positioning unit 12 to the communication terminal 50. In addition to the global position information, the positioning result may include information about the direction (direction in the three-dimensional coordinates of roll, pitch, and yaw) estimated from the image data.

The communication terminal 50 is, for example, a terminal capable of wireless communication, and is, for example, a smartphone, a tablet terminal, a PC, or the like. The communication terminal 50 receives the provision of the AR content from the content server (not shown) when the user uses the installed application, for example. In the communication terminal 50, when an application is executed, for example, image pickup (continuous image pickup) by the mounted camera is started. Then, the communication terminal 50 is based on the positioning result estimated by the position positioning server 10 according to the imaged image data, the imaged image data, the information (angle of view, etc.) of the camera of the communication terminal 50, and the like. , A content including AR content is provided from a content server (not shown). The communication terminal 50 includes an imaging unit 51, a positioning unit 52, a local position acquisition unit 53, a position derivation unit 54, and an imaging instruction unit 55.

The image pickup unit 51 has a function of controlling image pickup of image data by the camera. The imaging unit 51 starts imaging the image data by the camera, for example, at the timing when the execution of the application receiving the AR content is started. When the image pickup unit 51 starts image pickup, the image pickup unit 51 continuously picks up an image by the camera until the execution of the application ends. The imaged image data is used by the position positioning server 10 to estimate global position information.

Based on the content of the image data captured by the camera under the control of the image capturing unit 51, the image capturing instruction unit 55 determines whether the image data can be used for estimating the global position information by the position positioning server 10. If it is determined that the image data cannot be used, the image capturing unit 51 is instructed to capture new image data without transmitting the image data to the position positioning server 10. When the image capturing instruction unit 55 determines that the image data can be used for estimating the global position information, the communication terminal 50 sends the image data and the positioning result (terminal acquisition position information) by the positioning unit 52 described later. Is transmitted to the positioning server 10.

When the image data captured by the communication terminal 50 includes a dynamic object in a predetermined value or more, the image capturing instruction unit 55 cannot use the image data for estimating global position information by the position positioning server 10. To determine. The dynamic object is a moving object such as a person or a car, and is usually an object not included in the map data 100. The imaging instruction unit 55 may specify a dynamic object by performing, for example, a process of semantic segmentation and specifying image data in pixel units. The image capturing instruction unit 55 may specify a dynamic object by using an image recognition technique. For example, the image capturing instruction unit 55 may identify an object that exists only in one image data based on a difference between continuously captured image data. An object whose position changes significantly may be identified as a dynamic object.

When the image data captured by the communication terminal 50 includes motion blur of a predetermined value or more, the image capturing instruction unit 55 determines that the image data cannot be used for estimating global position information by the position location server 10. To do. Motion blur is blurring that occurs when a moving object is captured by a camera. The image capturing instruction unit 55 may actually analyze the image data when determining how much motion blur is included in the image data. For example, the image capturing instruction unit 55 may display the detection result of the acceleration sensor mounted on the communication terminal 50. You may use it.

The positioning unit 52 acquires terminal acquisition position information that is global position information of the communication terminal 50 by performing GPS (Global Positioning System) positioning, for example. The positioning unit 52 may periodically perform positioning while the image data is being imaged under the control of the imaging unit 51. The positioning unit 52 may acquire global position information by a positioning method other than GPS positioning (for example, base station positioning).

The local position acquisition unit 53 continuously acquires local position information, which is relative position information of the communication terminal 50 with respect to a certain point. The certain point is, for example, a point at which execution of an application (application receiving AR content) is started in the communication terminal 50. The local position acquisition unit 53 continuously acquires three-dimensional local position information from, for example, a sensor mounted on the communication terminal 50. The sensor mounted on the communication terminal 50 is not limited, but, for example, a visual SLAM (Simultaneous Localization and Mapping) using an inertial measurement unit (IMU: Inertial Measurement Unit), a camera sensor, a general acceleration sensor, a gyro sensor, or the like is used. be able to. For example, the three-dimensional latitude/longitude/height of the point where the execution of the application is started is used as a reference (coordinates (x, y, z) are (0, 0, 0)), and the three-dimensional latitude from the point is set. By continuously acquiring the amount of change in longitude and height, local position information, which is relative position information of the communication terminal 50 with respect to the point, is tracked. Note that the positioning result may include, in addition to the local position information, information about the direction (direction in the three-dimensional coordinates of roll, pitch, and yaw) estimated from the detection result of the acceleration sensor, the gyro sensor, or the like. Good.

The position derivation unit 54 estimates the amount of movement of the communication terminal 50 from the time of image capturing based on the change in the local position information acquired by the local position acquisition unit 53, and based on the amount of movement, the position positioning server 10 causes By correcting the estimated global position information, the position information of the communication terminal 50 considering the above-described movement amount is derived. For example, it is assumed that the local position information (three-dimensional coordinates) at the time when the communication terminal 50 takes an image of certain image data is (3, 5, 3). It should be noted that here, the numbers of the three-dimensional coordinates are values that are easy to understand for convenience of explanation. Then, it is assumed that the local position information (three-dimensional coordinates) is (1, 8, 2) when the position deriving unit 54 derives the position after acquiring the positioning result of the position positioning server 10 regarding the image data. In this case, the position deriving unit 54 determines the coordinates (3, 5, 3) at the time of image capturing and the coordinates (1, at the current time) as the amount of movement of the communication terminal 50 from the time of image capturing by the communication terminal 50 to the current time (time of position deriving). From (8, 2), (3-1=2, 5-8=-3, 3-1=1), that is, (2, -3, 1) is derived as the movement amount. Then, the position deriving unit 54 corrects the global position information by the movement amount. That is, for example, by correcting the global position information (global position information at the time of image capturing) by adding the movement amount (2, -3, 1), the position information of the communication terminal 50 moved after the image capturing. (Global position information) can be appropriately derived. Note that the position deriving unit 54 only needs to correct the global position information based on the movement amount (in consideration of the movement amount), and does not necessarily correct the global position information by the movement amount. Good. That is, the position deriving unit 54 takes into account the error in the calculation timing, the communication time between the communication terminal 50 and the position positioning server 10, and the like, and the global position by the value obtained by multiplying the movement amount by a predetermined adjustment value. The information may be corrected. The communication terminal 50 transmits the corrected position information of the communication terminal 50 to the position positioning server 10.

Next, the processing performed by the positioning system 1 will be described with reference to FIG. FIG. 2 is a sequence diagram showing processing performed by the positioning system 1.

As shown in FIG. 2, for example, the communication terminal 50 starts capturing image data at the timing when the execution of the application receiving the AR content is started (step S1).

Subsequently, the communication terminal 50 determines whether or not the image data can be used for the estimation of global position information by the position positioning server 10 based on the content of the imaged image data (step S2). In step S2, for example, when a dynamic object is included in the captured image data by a predetermined value or more, it is determined that the image data cannot be used for estimating the global position information by the positioning server 10. .. Further, in step S2, for example, when the captured image data includes motion blur of a predetermined value or more, it is determined that the image data cannot be used for estimating the global position information by the position positioning server 10. ..

When it is determined in step S2 that the captured image data cannot be used for estimating the global position information by the positioning server 10, the communication terminal 50 does not transmit the image data to the positioning server 10, New image data is imaged again (that is, the process of step S1 is performed again). On the other hand, when it is determined in step S2 that the captured image data can be used for estimating the global position information by the position positioning server 10, the communication terminal 50 determines that the image data and the GPS information (terminal acquisition position information). ) Is transmitted to the positioning server 10 (step S3), and tracking of local position information is started (step S4).

The positioning server 10 selects one or a plurality of division map data to be used from the plurality of division map data based on the GPS information (terminal acquisition position information) obtained from the communication terminal 50 (step S5). Then, the position location server 10 globalizes the communication terminal 50 at the time of image capturing by the communication terminal 50 based on the result of matching between the feature point of the selected divided map data and the feature point of the image data imaged by the communication terminal 50. The position information is estimated (step S6). The position positioning server 10 transmits the estimated global position information to the communication terminal 50 (step S7).

The communication terminal 50 corrects the global position information transmitted from the position positioning server 10 based on the local position information (step S8). Specifically, the communication terminal 50 estimates the amount of movement of the communication terminal 50 from the time of imaging based on the change in local position information acquired by the local position acquisition unit 53, and based on the amount of movement, the position is calculated. By correcting the global position information estimated by the positioning server 10, the position information of the communication terminal 50 in consideration of the above-described movement amount is derived. The communication terminal 50 transmits the derived corrected position information to the position positioning server 10 (step S9). Then, the AR content is provided using the position information.

Next, operation effects of the positioning system 1 according to the present embodiment will be described.

The positioning system 1 is a positioning system that performs positioning based on image data captured by the communication terminal 50, and continuously acquires local position information that is relative position information of the communication terminal 50 with respect to a certain point. The local position acquisition unit 53 that stores the map data 100 in which the feature amount of the feature point included in the image data acquired in advance and the global position information that is the absolute position information of the feature point are stored. A positioning unit that estimates global position information of the communication terminal 50 at the time of image capturing in the communication terminal 50 based on the unit 11, the image data captured by the communication terminal 50, and the map data 100 stored in the storage unit 11. 12 and the change in the local position information acquired by the local position acquisition unit 53, the movement amount from the time of imaging in the communication terminal 50 is estimated, and the global position estimated by the positioning unit 12 is estimated based on the movement amount. The position deriving unit 54 derives the position information of the communication terminal 50 in consideration of the movement amount by correcting the position information.

In the positioning system 1 according to the present embodiment, the image data captured by the communication terminal 50, the map data 100 (the feature amount of the feature points included in the image data acquired in advance, and the global position information associated with the feature points). Based on (data associated with and), global position information of the communication terminal 50 at the time of image capturing by the communication terminal 50 is estimated. Here, for example, when the user who uses the communication terminal 50 moves and the position of the communication terminal 50 changes after the image data is captured by the communication terminal 50, the captured image data and the map data 100 Even if the global position information is estimated with high accuracy based on the above, it is conceivable that the actual position of the communication terminal 50 may deviate from the positioning result. In this regard, in the positioning system 1, the amount of movement of the communication terminal 50 from the time of image capturing by the communication terminal 50 is estimated from the continuously acquired local position information of the communication terminal 50, and the global position information is calculated based on the movement amount. Is corrected, and the position information of the communication terminal 50 considering the movement amount is derived. In this way, the movement amount is estimated based on the continuously acquired local position information of the communication terminal 50, the global position information is corrected according to the movement amount, and the final position information of the communication terminal 50 is derived. By doing so, even if the position of the communication terminal 50 changes after the image data is captured, the actual position of the communication terminal 50 can be specified with high accuracy. As described above, according to the positioning system 1, the positioning accuracy can be improved. When the positioning accuracy is low, the processing load may increase due to the user repeatedly requesting the positioning result. However, the positioning accuracy improves, so that the above-mentioned problem is less likely to occur, and the processing unit such as the CPU is less likely to occur. The technical effect of reducing the processing load is also obtained.

For example, when trying to perform global positioning using the map data 100 only with the communication terminal 50, the capacity of the map data 100 that can be stored is limited depending on the performance of the communication terminal 50, and the positioning accuracy should be sufficiently improved. May not be possible. In this respect, like the present embodiment, the large-scale map data 100 used for positioning is stored in the position positioning server 10, and the image data used for positioning is acquired by the communication terminal 50, thereby improving the positioning accuracy. At the same time, quick positioning is realized. The processing on the side of the communication terminal 50 depends on the performance and the OS of the communication terminal 50 by making it less likely to depend on the performance and the OS of the communication terminal 50 such as capturing image data and acquiring local position information. It is possible to realize an architecture that is difficult to do.

The positioning unit 12 performs matching between the characteristic points of the map data 100 and the characteristic points of the image data captured by the communication terminal 50, and based on the global position information associated with the characteristic points of the map data 100, the communication terminal The global position information of the communication terminal 50 at the time of imaging at 50 is estimated. In this way, by matching the feature points between the map data 100 and the imaged image data, the correspondence relationship between the map data 100 and the image data is appropriately specified and associated with the feature points of the map data 100. Based on the global position information, it is possible to appropriately estimate the global position information of the communication terminal 50 at the time of image capturing.

The storage unit 11 stores a plurality of divided map data obtained by dividing the map data 100 into certain regions according to the global position information, and the positioning unit 12 in the communication terminal 50 together with the image data imaged in the communication terminal 50. The terminal acquisition position information that is the acquired global position information of the communication terminal 50 is acquired, one or a plurality of division map data is selected from a plurality of division map data according to the terminal acquisition position information, and the selected division map is selected. Global position information of the communication terminal 50 at the time of image capturing in the communication terminal 50 is estimated based on the result of matching between the characteristic point of the data and the characteristic point of the image data captured by the communication terminal 50. In this way, the map data 100 is divided into a plurality of pieces according to the global position information, and the division map data to be matched is selected from the plurality of pieces of division map data according to the terminal acquisition position information, whereby the matching range ( The search range) can be narrowed down, and the efficiency of matching processing can be improved. That is, for example, when the stored map data is a single vast map, the search range is too wide and the search efficiency deteriorates when the image data is transmitted from the communication terminal to the positioning server to perform self-position recognition. It is possible that In this respect, the stored map data 100 is divided into a certain unit, and the divided map data to be matched is selected from a plurality of divided map data according to the terminal acquisition position information, thereby limiting the search range and performing processing. It is possible to improve efficiency.

The positioning system 1 determines whether or not the image data can be used for the estimation of the global position information by the positioning unit 12 based on the content of the image data captured by the communication terminal 50, and when it cannot be used. In addition, an image capturing instruction unit 55 for instructing image capturing of new image data is further provided. As described above, in the previous stage of the position estimation by the positioning unit 12, when the captured image data is not suitable for the estimation, the new image data is prompted to be captured, so that the positioning unit 12 performs unnecessary processing. Can be avoided and the total positioning time can be shortened.

When the image data captured by the communication terminal 50 includes a dynamic object or more in a predetermined value, the image capturing instruction unit 55 cannot use the image data for estimating global position information by the positioning unit 12. judge. It is considered that the dynamic object does not exist continuously and is not recorded in the map data 100. Therefore, the dynamic object becomes noise in the estimation of the global position information, which may cause a decrease in positioning accuracy. In this regard, by determining that the image data cannot be used for estimating the global position information when the image data includes a dynamic object or more by a predetermined value or more, the dynamic object described above is caused. It is possible to suppress a decrease in positioning accuracy.

When the image data captured by the communication terminal 50 includes motion blur of a predetermined value or more, the image capturing instruction unit 55 determines that the image data cannot be used for estimating global position information by the positioning unit 12. .. If the image data contains a large amount of motion blur, matching of feature points cannot be performed properly, which may cause a decrease in positioning accuracy. In this regard, when the image data includes motion blur more than a predetermined value, it is determined that the image data cannot be used for the estimation of the global position information, so that the positioning accuracy is lowered due to the motion blur described above. Can be suppressed.

The storage unit 11 stores three-dimensional position information as global position information of the map data 100, and the positioning unit 12 estimates the three-dimensional position information as global position information of the communication terminal 50, and the local position acquisition unit. 53 acquires three-dimensional position information as local position information of the communication terminal 50. As a result, the position deriving unit 54 can derive three-dimensional position information.

Finally, the hardware configurations of the positioning server 10 and the communication terminal 50 included in the positioning system 1 will be described with reference to FIG. The positioning server 10 and the communication terminal 50 described above may be physically configured as a computer device including the processor 1001, the memory 1002, the storage 1003, the communication device 1004, the input device 1005, the output device 1006, the bus 1007, and the like. Good.

Note that in the following description, the word "device" can be read as a circuit, device, unit, or the like. The hardware configuration of the positioning server 10 and the communication terminal 50 may be configured to include one or a plurality of each device illustrated in the figure, or may be configured not to include some devices.

Each function of the position location server 10 and the communication terminal 50 causes a predetermined software (program) to be loaded onto hardware such as the processor 1001 and the memory 1002, so that the processor 1001 performs calculation and communication by the communication device 1004, It is realized by controlling reading and/or writing of data in the memory 1002 and the storage 1003.

The processor 1001 operates an operating system to control the entire computer, for example. The processor 1001 may be composed of a central processing unit (CPU) including an interface with peripheral devices, a control device, a calculation device, a register, and the like. For example, the control function of the positioning unit 12 and the like of the position positioning server 10 may be realized by the processor 1001.

Further, the processor 1001 reads a program (program code), software module, and data from the storage 1003 and/or the communication device 1004 into the memory 1002, and executes various processes according to these. As the program, a program that causes a computer to execute at least part of the operations described in the above-described embodiments is used. For example, the control function of the positioning unit 12 and the like of the position positioning server 10 may be realized by the control program stored in the memory 1002 and operated by the processor 1001, and may be realized similarly for other functional blocks. Although it has been described that the various processes described above are executed by one processor 1001, they may be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented by one or more chips. The program may be transmitted from the network via an electric communication line.

The memory 1002 is a computer-readable recording medium, and is composed of, for example, at least one of ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (ElectricallyErasable Programmable ROM), RAM (Random Access Memory), and the like. May be done. The memory 1002 may be called a register, a cache, a main memory (main storage device), or the like. The memory 1002 can store a program (program code) executable to implement the wireless communication method according to the embodiment of the present invention, a software module, and the like.

The storage 1003 is a computer-readable recording medium, for example, an optical disc such as a CD-ROM (Compact Disc ROM), a hard disc drive, a flexible disc, a magneto-optical disc (for example, a compact disc, a digital versatile disc, a Blu-ray disc). (Registered trademark) disk), smart card, flash memory (for example, card, stick, key drive), floppy (registered trademark) disk, magnetic strip, and the like. The storage 1003 may be called an auxiliary storage device. The storage medium described above may be, for example, a database including the memory 1002 and/or the storage 1003, a server, or another appropriate medium.

The communication device 1004 is hardware (transmission/reception device) for performing communication between computers via a wired and/or wireless network, and is also referred to as a network device, network controller, network card, communication module, or the like.

The input device 1005 is an input device (for example, a keyboard, a mouse, a microphone, a switch, a button, a sensor, etc.) that receives an input from the outside. The output device 1006 is an output device (for example, a display, a speaker, an LED lamp, etc.) that performs output to the outside. The input device 1005 and the output device 1006 may be integrated (for example, a touch panel).

Each device such as the processor 1001 and the memory 1002 is connected by a bus 1007 for communicating information. The bus 1007 may be composed of a single bus, or may be composed of different buses among devices.

In addition, the positioning server 10 and the communication terminal 50 include a microprocessor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), an FPGA (Field Programmable Gate Array), and the like. It may be configured to include hardware, and the hardware may implement part or all of each functional block. For example, processor 1001 may be implemented with at least one of these hardware.

Although the present embodiment has been described in detail above, it is obvious to those skilled in the art that the present embodiment is not limited to the embodiment described in this specification. The present embodiment can be implemented as modified and changed modes without departing from the spirit and scope of the present invention defined by the description of the claims. Therefore, the description of the present specification is for the purpose of exemplifying explanation, and does not have any restrictive meaning to the present embodiment.

For example, the positioning system 1 has been described as including the communication terminal 50 and the position positioning server 10, but the present invention is not limited to this, and each function of the positioning system 1 may be realized only by the communication terminal. Further, the global position information may be corrected using other information in addition to the local position information. For example, for a spot identified by global position information, if a subsequently visited spot can be estimated (that is, if the user's flow line can be estimated according to the characteristics of the spot), based on the estimated flow line, The global position information may be corrected. In that case, for example, the accuracy of estimating the flow line may be improved by using information such as the sex and age of the user, and the accuracy of correcting the global position information may be improved. Further, in the case of the global position information of the spot that the user has visited in the past, the global position information may be corrected in consideration of the past information (the flow line traced by the user in the past). In addition, for example, when a plurality of users are using the same application and the plurality of users collaborate to execute the application, mutual local position information is shared with each other, and The global position information may be corrected based on the position information. Further, the global position information may be corrected by improving the estimation accuracy of the flow line according to the search history of the user in the communication terminal 50 and the content of the information being browsed (content of the user's interest).

Each aspect/embodiment described in this specification is LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G, 5G, FRA (Future Radio Access), W-CDMA. (Registered trademark), GSM (registered trademark), CDMA2000, UMB (Ultra Mobile Broad-band), IEEE 802.11 (Wi-Fi), IEEE 802.16 (WiMAX), IEEE 802.20, UWB (Ultra-Wide) Band), Bluetooth (registered trademark), or any other system using an appropriate system and/or a next-generation system extended based on the system.

The order of the processing procedures, sequences, flowcharts, etc. of each aspect/embodiment described in this specification may be changed as long as there is no contradiction. For example, the methods described herein present elements of the various steps in a sample order, and are not limited to the specific order presented.

Information that has been input and output may be stored in a specific location (for example, memory), or may be managed in a management table. Information that is input/output can be overwritten, updated, or added. The output information and the like may be deleted. The input information and the like may be transmitted to another device.

The determination may be performed by a value represented by 1 bit (whether 0 or 1), may be performed by a Boolean value (Boolean: true or false), and may be performed by comparing numerical values (for example, a predetermined value). (Comparison with the value).

The respective aspects/embodiments described in this specification may be used alone, in combination, or may be switched according to execution. Further, the notification of the predetermined information (for example, the notification of “being X”) is not limited to the explicit notification, but is performed implicitly (for example, the notification of the predetermined information is not performed). Good.

Software, whether called software, firmware, middleware, microcode, hardware description language, or any other name, instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules. , Application, software application, software package, routine, subroutine, object, executable, thread of execution, procedure, function, etc. should be construed broadly.

Also, software, instructions, etc. may be sent and received via a transmission medium. For example, the software may use wired technologies such as coaxial cable, fiber optic cable, twisted pair and digital subscriber line (DSL) and/or wireless technologies such as infrared, wireless and microwave to websites, servers, or other When transmitted from a remote source, these wireline and/or wireless technologies are included within the definition of transmission medium.

The information, signals, etc. described herein may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description include voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any of these. May be represented by a combination of

Note that the terms described in the present specification and/or the terms necessary for understanding the present specification may be replaced with terms having the same or similar meanings.

The information, parameters, and the like described in this specification may be represented by absolute values, relative values from predetermined values, or may be represented by other corresponding information. ..

User terminals are defined by those skilled in the art as mobile communication terminals, subscriber stations, mobile units, subscriber units, wireless units, remote units, mobile devices, wireless devices, wireless communication devices, remote devices, mobile subscriber stations, access terminals, It may also be referred to as a mobile terminal, wireless terminal, remote terminal, handset, user agent, mobile client, client, or some other suitable term.

As used herein, the terms "determining" and "determining" may encompass a wide variety of actions. "Judgment" and "decision" mean, for example, calculating, computing, processing, deriving, investigating, looking up (e.g., table, database or another). (Search in data structure), ascertaining that it is regarded as "judgment" and "decision" can be included. In addition, "decision" and "decision" include receiving (eg, receiving information), transmitting (eg, transmitting information), input (input), output (output), access (accessing) (for example, accessing data in a memory) may be regarded as “judging” and “deciding”. In addition, "judgment" and "decision" are considered to be "judgment" and "decision" when things such as resolving, selecting, choosing, establishing, establishing, and comparing are done. May be included. That is, the “judgment” and “decision” may include considering some action as “judgment” and “decision”.

As used herein, the phrase “based on” does not mean “based only on,” unless expressly specified otherwise. In other words, the phrase "based on" means both "based only on" and "based at least on."

Where the designations "first," "second," etc. are used herein, any reference to that element does not generally limit the amount or order of those elements. These nomenclatures may be used herein as a convenient way to distinguish between two or more elements. Thus, references to the first and second elements do not mean that only two elements may be employed there, or that the first element must precede the second element in any way.

As long as the terms "include", "including", and variations thereof are used in the present specification or claims, these terms are the same as the term "comprising." It is intended to be comprehensive. Furthermore, the term “or” as used in the specification or claims is not intended to be an exclusive OR.

In the present specification, a device including a plurality of devices is also included unless it is a device in which only one clearly exists in terms of context or technology.

Throughout this disclosure, the plural is included unless the context clearly indicates the singular.

DESCRIPTION OF SYMBOLS 1... Positioning system, 11... Storage part, 12... Positioning part (global position estimation part), 50... Communication terminal (terminal), 53... Local position acquisition part, 54... Position derivation part, 55... Imaging instruction part, 100... Map data.

Claims

A positioning system that performs positioning based on image data captured by a terminal,
A local position acquisition unit that continuously acquires local position information that is relative position information of the terminal with respect to a certain point;
A storage unit that stores map data in which feature amounts of feature points included in previously acquired image data and global position information that is absolute position information associated with the feature points are associated with each other,
A global position estimating unit that estimates global position information of the terminal at the time of image capturing in the terminal, based on image data captured by the terminal and map data stored in the storage unit;
Global position information estimated by the global position estimation unit based on a change in the local position information acquired by the local position acquisition unit And a position deriving unit that derives position information of the terminal in consideration of the movement amount.
The global position estimation unit performs matching between a feature point of the map data and a feature point of image data captured by the terminal, and based on global position information associated with the feature point of the map data, The positioning system according to claim 1, which estimates global position information of the terminal at the time of imaging at the terminal.
The storage unit stores a plurality of divided map data obtained by dividing the map data into fixed areas according to global position information,
The global position estimation unit acquires terminal acquisition position information which is global position information of the terminal acquired by the terminal together with image data captured by the terminal, and the plurality of the plurality of terminal acquisition position information are acquired according to the terminal acquisition position information. One or a plurality of division map data is selected from the division map data, and based on the result of matching between the feature points of the selected division map data and the feature points of the image data captured by the terminal, the The positioning system according to claim 2, which estimates global position information of the terminal.
Based on the content of the image data captured by the terminal, it is determined whether or not the image data can be used for estimating the global position information by the global position estimating unit. The positioning system according to any one of claims 1 to 3, further comprising an imaging instruction unit that instructs imaging of image data.
The image capturing instructing unit cannot use the image data for estimating global position information by the global position estimating unit when the image data captured by the terminal includes a dynamic object in a predetermined value or more. The positioning system according to claim 4, which is determined.
The image capturing instruction unit determines that the image data cannot be used for estimating global position information by the global position estimating unit when the image data captured by the terminal includes motion blur of a predetermined value or more. The positioning system according to claim 4 or 5.
The storage unit stores three-dimensional position information as global position information of map data,
The global position estimation unit estimates three-dimensional position information as global position information of the terminal,
7. The positioning system according to claim 1, wherein the local position acquisition unit acquires three-dimensional position information as local position information of the terminal.