JP2011217207A - Altitude information acquisition system, altitude information acquisition apparatus and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To acquire the altitude information of a portable terminal device at a building floor accuracy without being affected by a place or a temperature.SOLUTION: A portable terminal device 4 has a photographing function, a two-dimensional position (latitude, longitude) measuring function and an elevation angle detecting function. An altitude calculation server 2 calculates the altitude of the portable terminal device 4 using latitude information, longitude information, elevation angle information and image information transmitted from the portable terminal device 4. The altitude calculation server 2 includes: a three-dimensional map information database; a generation means which generates a collation image matched with a photographed image transmitted from the portable terminal device 4 from three-dimensional map information; and a calculation means which sets a criterion point on the collation image, and calculates the altitude of the portable terminal device 4 based on the altitude information of the criterion point in the three-dimensional map information, the latitude information, the longitude information and the elevation angle information transmitted from the portable terminal device 4.

Description

  The present invention relates to a system, apparatus, and program for acquiring altitude information of a mobile terminal device.

  There is a system that provides information regarding an object associated with a three-dimensional position to the portable terminal device according to the three-dimensional position (latitude, longitude, altitude) of the portable terminal device (see Patent Document 1). This system has a camera-equipped mobile terminal device such as a camera-equipped mobile phone, and a server that can communicate with the camera-equipped mobile terminal device via a communication network. The position information (latitude information, longitude information, altitude information) and direction information are transmitted to the server, and the server captures information about the store in the building existing at the three-dimensional position corresponding to the received three-dimensional position information and direction information. Augmented reality is realized by adding the received information to the image of the building being photographed and displaying it on the display.

  In this system, information about an object such as a store existing in a building is stored in the database of the server in association with the three-dimensional position of the floor where the object exists, and the server has a built-in portable terminal device. 3D position information generated by GPS (Global Positioning System) receiver and direction information generated by direction sensor (geomagnetic sensor or acceleration sensor) are received, and the building that exists at the position corresponding to those information At the same time, the information about the object existing in the building is retrieved from the database and transmitted to the mobile terminal device. On the display of the mobile terminal device, information on the object existing in the building being photographed is displayed so that the floor on which the object exists can be identified.

  However, GPS has a problem that accurate altitude information cannot be obtained because positioning accuracy in the vertical direction is lower than that in the horizontal direction. Accurate altitude information can be obtained by using an altimeter (barometer) or the like, but it is difficult to implement due to large restrictions on devices and costs. In addition, errors due to temperature are large, and it is extremely difficult to obtain altitude information on building floor accuracy. Although there is a technology for recognizing the floor of a building from the position of Wi-Fi base station and radio wave intensity, there are few places where the base station can be actually used because the area where the base station is provided is limited.

JP 2004-355058 A

  The present invention has been made in order to solve such a problem, and an object of the present invention is to obtain altitude information of a mobile terminal device with accuracy of a building floor without being affected by location or temperature. It is.

  The present invention provides a portable terminal device having a function of acquiring latitude information, longitude information, and elevation angle information, and a photographing function, and the portable information device using latitude information, longitude information, elevation angle information, and image information transmitted from the portable terminal device. An altitude information acquisition system including an altitude information acquisition apparatus that acquires altitude information of a terminal device, wherein the altitude information acquisition apparatus matches a three-dimensional map information database and a captured image configured by the image information. Collation image generating means for generating an image for use from the 3D map information, setting a reference point on the image for comparison, altitude information of the reference point in the 3D map information, latitude information, longitude An altitude information acquisition system comprising altitude calculation means for calculating the altitude of the mobile terminal device based on information and elevation angle information.

[Action]
According to the present invention, the altitude information acquisition device generates a matching image that matches a photographed image by the mobile terminal device from the three-dimensional map information, sets a reference point on the matching image, and 3D of the reference point. The altitude of the mobile terminal device is calculated based on the altitude information in the map information and the latitude information, longitude information, and elevation angle information of the mobile terminal device.

  ADVANTAGE OF THE INVENTION According to this invention, the height information of a portable terminal device can be obtained with a building floor precision, without receiving to the influence of a place or temperature.

It is a figure which shows the altitude information acquisition system of embodiment of this invention. It is a block diagram of the portable terminal device in FIG. It is a block diagram of the altitude calculation server in FIG. It is a flowchart which shows operation | movement of the altitude information acquisition system of embodiment of this invention. It is a figure for demonstrating collation with the camera image and three-dimensional map image in the flow of FIG. It is a figure which shows the altitude calculation method in the flow of FIG. It is a figure which shows the information provision system using the altitude information acquisition system of embodiment of this invention. It is a block diagram of the content server in FIG. It is a flowchart which shows operation | movement of the information provision system using the altitude information acquisition system of embodiment of this invention. It is a figure which shows the example of the image displayed on the portable terminal device of the information provision system using the altitude information acquisition system of embodiment of this invention. It is a figure which shows another example of the image displayed on the portable terminal device of the information provision system using the altitude information acquisition system of embodiment of this invention.

Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings.
[Advanced information acquisition system]
<System configuration>
FIG. 1 shows an altitude information acquisition system according to an embodiment of the present invention. This system includes an altitude calculation server 2 connected to a network 1 such as the Internet and a portable terminal device 4 connected to the network 1 via a base station 3. The portable terminal device 4 may be any portable device having an imaging function, a two-dimensional position (latitude, longitude) measurement function, an angle (elevation angle, azimuth angle) detection function, and a communication function. A mobile phone, a PDA (Personal Digital Assistant), a PND (Personal Navigation Device), a portable game machine, and the like. The mobile terminal device 4 and the altitude calculation server 2 can communicate with each other via the base station 3 and the network 1.

<Configuration of mobile terminal device>
FIG. 2 is a block diagram of the mobile terminal device 4. The mobile terminal device 4 includes a control unit 41, an operation unit 42, a display unit 43, a photographing unit 44, a position measurement unit 45, an angle detection unit 46, a wireless communication unit 47, and a storage unit 48.

  The control unit 41 is a controller for controlling each unit of the mobile terminal device 4, and includes a CPU (Central Processing Unit) 411, a ROM (Read Only Memory) 412, and a RAM (Random Access Memory) 413. The CPU 411 implements various processes to be described later by loading a program stored in the ROM 412 into the RAM 413 and executing it.

  The operation unit 42 includes various buttons for the user to operate the mobile terminal device 4. The display unit 43 includes a liquid crystal display device for displaying the operation state of the mobile terminal device 4, the image acquired by the imaging unit 44, information input by the user from the operation unit 42, and the like.

  The imaging unit 44 includes an imaging device such as a CCD (Charge Coupled Device) and acquires image information. The position measuring unit 45 includes a GPS receiver, receives a radio wave transmitted from a GPS satellite every predetermined time (for example, 1 second), and measures (measures) a two-dimensional position (latitude, longitude). Then, two-dimensional position information (latitude information and longitude information) and positioning time information, which are positioning information, are output to the control unit 41 as GPS information. The angle detection unit 46 includes a geomagnetic sensor and an acceleration sensor, and generates angle information (elevation angle information, azimuth angle information) indicating an angle (elevation angle, azimuth angle) in the imaging direction (frontward in the optical axis direction) of the imaging unit 44. it can.

  The wireless communication unit 47 is a circuit for performing wireless communication with the base station 3. The storage unit 48 is composed of a nonvolatile storage medium such as an SD memory card.

<Configuration of altitude calculation server>
As shown in FIG. 3, the altitude calculation server 2 includes a control unit 21, a communication unit 22, and a storage unit 23.

  The control unit 21 includes a CPU 211, a ROM 212, and a RAM 213, and controls the entire altitude calculation server. The CPU 211 implements various processes described later by loading a program stored in the ROM 212 into the RAM 213 and executing the program.

  The communication unit 22 communicates with the mobile terminal device 4 via the network 1 and the base station 3. The storage unit 23 includes an HDD (hard disk device), and is provided with a subscriber information DB (database) 231, a three-dimensional map information DB 232, and a terrain elevation information DB 233.

  The subscriber information DB 231 stores the device ID of the mobile terminal device 4 of the subscriber of the altitude calculation service provided by the altitude calculation server 2. The 3D map information DB 232 stores 3D position information of structures such as roads and buildings that constitute the 3D map. CAD (Computer Aided Design) data for designing a structure can also be stored. The topographic altitude information DB 233 stores altitude information for each predetermined area (eg, 50 m square).

<Operation of advanced information acquisition system>
The operation of the altitude information acquisition system having the above configuration will be described using the flowchart shown in FIG. The flow in this figure is performed by the CPU 411 of the mobile terminal device 4 executing the program stored in the ROM 412 and the CPU 211 of the altitude calculation server 2 executing the program stored in the ROM 212.

  When the user of the mobile terminal device 4 starts an altitude information acquisition application (hereinafter referred to as an altitude information acquisition application), the mobile terminal device 4 transmits an access request to the altitude calculation server 2 (step S1). This access request includes the device ID of the mobile terminal device 4.

  When the altitude calculation server 2 receives the access request, the altitude calculation server 2 refers to the subscriber information DB 231, checks the device ID, reads out the image data of the top screen of the altitude information acquisition application from the storage unit 23, and transmits it to the mobile terminal device 4. (Step S11). The mobile terminal device 4 receives the image data of the top screen of the altitude information acquisition application and displays it on the display unit 43 (step S2).

  Next, in the mobile terminal device 4, the photographing unit 44, the position measurement unit 45, and the angle detection unit 46 are operated under the control of the CPU 411. Thereby, the photographing unit 44 photographs a subject in front of the optical axis to generate image information (captured image information), and the position measuring unit 45 measures the two-dimensional position of the mobile terminal device 4 to obtain the two-dimensional position information and Positioning time information is generated, and the angle detection unit 46 generates angle information (step S3).

  The image information is read into the RAM 413, sent to the display unit 43, and displayed. The image information is transmitted to the altitude calculation server 2 together with the two-dimensional position information (latitude information and longitude information) and angle information (step S4).

  When the altitude calculation server 2 receives the image information, the two-dimensional position information, and the angle information (step S12), the altitude calculation server 2 calculates the altitude of the mobile terminal device 4 by image recognition (step 13).

Details of step S13 will be described with reference to FIGS.
First, a matching image that matches the photographed image is searched from the 3D map information DB 232. At this time, azimuth information is used. FIG. 5A is an example of a captured image, and FIG. 5B is an example of a collation image.

  Next, a reference point is set at a corresponding position in the captured image and the collation image. The reference point is preferably a straight part that is easy to recognize an image, such as the upper edge of a building, a signboard, or a pedestrian crossing. 5A and 5B, reference points are set at the upper edge of the building far from the left side of the road, the approximate center of the sign on the building far from the right side of the road, the pedestrian crossing, and the entrance of the first floor of the building near the left side of the road. The reference point can be set either automatically or manually.

In FIG. 5A, a viewpoint Q ₀ (x ₀ , y ₀ , z ₀ ) is a three-dimensional position of the imaging unit 44, and among the three-dimensional position information, an x-coordinate value x ₀ and a y-coordinate value y ₀ are This is a known value measured by the position measurement unit 45. On the other hand, the z coordinate value z ₀ is an unknown value that is an altitude calculation target.

In FIG. 5B, the three-dimensional position information of the reference points P _{1 to} P ₄ (X coordinate values X _{1 to} X ₄ , Y coordinate values Y _{1 to} Y ₄ , Z coordinate values Z _{1 to} Z ₄ ) is three-dimensional. This is a known value indicating a position (latitude, longitude, altitude) on the map information DB 232. The viewpoint P ₀ is a viewpoint set when a collation image is formed based on the data on the 3D map information DB 232, and the 3D position information (X coordinate value X ₀ , Y coordinate value Y ₀ , The Z coordinate value Z ₀ ) is known.

Next, the three-dimensional position information (X _{1 to} X ₄ , Y _{1 to} Y ₄ , Z _{1 to} Z ₄ ) on the three-dimensional map information DB of the reference points P _{1 to} P ₄ of the collation image, and the latitude of the mobile terminal device 4 Information and longitude information (x ₀ , y ₀ ), elevation angle from the viewpoint to the reference point in the captured image, and the distance from the viewpoint to the reference point, by triangulation, in other words, the altitude of the viewpoint of the captured image, in other words, a portable terminal The altitude of the device 4 is calculated.

This will be described in detail with reference to FIG. In this figure, the x-axis is the horizontal direction on the paper surface, the y-axis is the direction perpendicular to the paper surface, and the z-axis is the vertical direction on the paper surface. The real space reference points, which are points in the real space corresponding to the reference points of the verification image (FIG. 5B), are designated as Q _A (x _A , y _A , z _A ), Q _B (x _B , y _B , z _B ), Let Q _C (x _C , y _C , z _C ). Q ₀ (x ₀ , y ₀ , z ₀ ) and Q ₁ (x ₀ , y ₀ , z ₁ ) are two viewpoints that differ only in altitude.

These real space reference point Q _A, Q _B, three-dimensional position of Q _C (longitude, latitude, altitude) are known values on the three-dimensional map DB 232. Of the three-dimensional position information of the viewpoints Q ₀ (x ₀ , y ₀ , z ₀ ) and Q ₁ (x ₀ , y ₀ , z ₁ ), the x coordinate value x ₀ and the y coordinate value y ₀ Thus, it is a known value measured by the position measuring unit 45. Also, the z coordinate value z ₀ or z ₁ is an unknown value that is an altitude calculation target.

The altitude of the viewpoint is calculated by triangulation using the distance from the viewpoint to the real space reference point and the elevation angle. For example, for the viewpoint Q ₀ , the intersection of the reference point Q _A , the perpendicular drawn downward from the reference point Q _A and the horizontal line drawn from the viewpoint Q _{0 with} respect to the perpendicular is designated as Q _D, and Q ₀ , Q _A , the length L of the base of the right triangle formed by Q _D can be expressed by the following formula (1). Further, the length between Q _A and Q _D , that is, the difference between altitude difference (z _A −z ₀ ) between the viewpoint Q ₀ and the reference point Q _A, and L can be expressed by the following equation [2]. it can.

^{_{L = √ {(x D -x}} 0) 2 + (y D -y 0) 2 + (z D -z 0) 2} ... formula (1)
z _A −z _D = Ltan Θ (2)

_{Here, x D = x A, y} D = y A, because it is _z D = z _0, wherein (1) becomes the following equation (3). Further, from the equation [2], the altitude z _{0 at the} viewpoint Q ₀ can be expressed by the following equation [4].

L = √ {(x _A −x ₀ ) ² + (y _A −y ₀ ) ² } Formula [3]
z ₀ = z _A −L tan Θ (4)

Since x _A , x ₀ , y _A , and y ₀ are known, L can be calculated. Θ is detected by the angle detection unit 46 of the mobile terminal device 4. Therefore, the viewpoint Q ₀ altitude can be calculated by using the calculated value of L and the detected value of Θ.

  Returning to the description of FIG. The altitude calculation server 2 transmits the calculated altitude information of the viewpoint to the mobile terminal device 4 (step S14). The mobile terminal device 4 receives the altitude information and holds it in the RAM 413 (step S5). The altitude information can also be stored in the storage unit 48.

  When the user does not input an instruction to end the altitude information acquisition application from the operation unit 42 (step S6: NO), the mobile terminal device 4 repeats steps S3 to S5 and inputs an instruction to end the altitude information acquisition application ( (Step S6: YES), an access release request is transmitted to the altitude calculation server 2 (Step S7).

  The altitude calculation server 2 repeats steps S12 to S14 while not receiving the access release request (step S15: NO), and if received (step S15: YES), releases the access of the mobile terminal device 4 (step S16).

  As described above, the altitude calculation server 2 according to the present embodiment generates a collation image that matches the captured image configured by the image information transmitted from the mobile terminal device 4 from the three-dimensional map information. Since the altitude of the mobile terminal device is calculated based on the altitude information in the 3D map information of the reference point and the latitude information, the longitude information and the elevation angle information transmitted from the mobile terminal device, The altitude information of the terminal device can be obtained with building floor accuracy without being affected by the location or temperature.

  In addition, although the above description was described as the real-time process with respect to one portable terminal device, it is actually performed by the batch process with respect to several portable terminal devices. Further, the mobile terminal device 4 may be configured not to generate and transmit azimuth angle information. Further, in FIG. 4, the mobile terminal device 4 makes an access request to the altitude calculation server 2 and receives the top screen from the altitude calculation server 2, but the mobile terminal device 4 is configured to generate the top screen. You can also. A single navigation device can also be configured to have the functions of the mobile terminal device 4 and the altitude calculation server 2.

[Information provision system]
<System configuration>
FIG. 7 shows an information providing system using the altitude information acquisition system according to the embodiment of the present invention. In this figure, the same reference numerals as those in FIG. This system is obtained by adding a content server 5 to the altitude information acquisition system of FIG. The content server 5 can communicate with the mobile terminal device 4 via the network 1 and the base station 3.

<Content server configuration>
As shown in FIG. 8, the content server 5 includes a control unit 51, a communication unit 52, and a storage unit 53.

  The control unit 51 includes a CPU 511, a ROM 512, and a RAM 513, and controls the entire content server 5. Further, the CPU 511 implements various processes to be described later by loading a program stored in the ROM 512 into the RAM 513 and executing it.

  The communication unit 52 communicates with the mobile terminal device 4 via the network 1 and the base station 3. The storage unit 53 includes an HDD (hard disk device), and is provided with a subscriber information DB 531 and a content DB with a three-dimensional position.

  The subscriber information DB 531 stores the device ID of the mobile terminal device 4 of the subscriber of the information providing service provided by the content server 5 and the like. In the three-dimensional position-attached content DB 532, information related to an object such as a store existing in a building is stored in association with the three-dimensional position (latitude, longitude, altitude) of the floor where the object exists.

<Operation of information provision system>
In the information providing system having the above-described configuration, the related operation between the mobile terminal device 4 and the altitude calculation server 2 is the same as that of the above-described altitude information acquisition system, and thus the description thereof will be omitted. The related operation will be described with reference to the flowchart shown in FIG. The flow in this figure is performed by the CPU 411 of the mobile terminal device 4 executing the program stored in the ROM 413 and the CPU 511 of the content server 5 executing the program stored in the ROM 512.

  When the user of the mobile terminal device 4 starts an information providing application (hereinafter referred to as an information providing application), the mobile terminal device 4 transmits an access request to the content server 5 (step S21). This access request includes the device ID of the mobile terminal device 4.

  Upon receiving the access request, the content server 5 refers to the subscriber information DB 531 and collates the device ID. Then, the content server 5 reads out the image data of the top screen of the information providing application from the storage unit 53 and transmits it to the mobile terminal device 4 (step). S31). The mobile terminal device 4 receives the image data of the top screen of the information providing application and displays it on the display unit 43 (step S22).

  Next, in the mobile terminal device 4, the photographing unit 44, the position measurement unit 45, and the angle detection unit 46 are operated under the control of the CPU 411. Thereby, the photographing unit 44 photographs a subject in front of the optical axis to generate image information (captured image information), and the position measuring unit 45 measures the two-dimensional position of the mobile terminal device 4 to obtain the two-dimensional position information and Positioning time information is generated, and the angle detection unit 46 generates azimuth angle information indicating an azimuth angle in front of the optical axis of the imaging unit 44 (step S23). In addition, the information providing application can be operated in parallel with the altitude information acquisition application (which is preferable), and when performing the parallel operation, by using each information generated in step S3 of FIG. Step S23 can be omitted.

  The image information is read into the RAM 413, sent to the display unit 43, and displayed. The two-dimensional position information is converted into three-dimensional position information together with the altitude information, and transmitted to the altitude calculation server 2 together with the azimuth angle information (step S24). The altitude information is received in step S5 of FIG. 4 and is held by the RAM 413 or stored in the storage unit 48.

  The content server 5 receives the three-dimensional position information and the azimuth angle information (step S32). Then, the content DB 532 with the three-dimensional position is accessed, and a predetermined image from the optical axis direction indicated by the azimuth information is within a predetermined distance (for example, 500 m) from the two-dimensional position (latitude, longitude) of the mobile terminal device 4. The content related to the object existing in the corner is searched and acquired (step S33).

  Next, display control information for displaying the acquired content on the display unit 43 of the mobile terminal device 4 is generated according to the altitude information of the mobile terminal device 4 and the three-dimensional position information of the object corresponding to the content (step S34). ) And transmitted to the portable terminal device 4 together with the content (step S35).

  The mobile terminal device 4 receives the content and the display control information (step S25), synthesizes the content with the photographed image according to the three-dimensional position information of the object corresponding to the content, and displays it on the display unit 43 (step S25). S26).

  FIG. 10 shows a display image. Here, FIG. 10A is a case where the viewpoint altitude is set to the third floor, and FIG. 10B is a case where the viewpoint altitude is set to the first floor. “A” to “i” are contents, for example, store names. The display positions of the frames surrounding “a” to “i” correspond to the three-dimensional position (latitude, longitude, altitude) of the object corresponding to the content, that is, the three-dimensional position of the floor where the store exists, for example.

  For example, since the content “f” relates to a store or the like that exists on the third floor of the left building in the photographed image, the content “f” is displayed at an altitude approximately at the center of the left part of the screen in FIG. 10A and in the left part of the screen in FIG. Displayed at the top. When content related to a plurality of objects is associated with the same floor, the left and right and front and rear display positions are controlled based on latitude information and longitude information.

  Returning to the description of FIG. When the user does not input an instruction to end the information providing application from the operation unit 42 (step S27: NO), the portable terminal device 4 repeats steps S23 to S26, and inputs an instruction to end the information providing application (step S27). : YES), an access release request is transmitted to the content server 5 (step S28).

  The content server 5 repeats steps S32 to S35 while not receiving the access release request (step S36: NO), and if received (step S36: YES), releases the access of the mobile terminal device 4 (step S37).

  As described above, according to the information providing system of the present embodiment, the mobile terminal device 4 uses the content server to store the latitude information and longitude information acquired by the built-in position measurement unit 45 and the altitude information acquired from the altitude calculation server 2. By transmitting to 5, the content transmitted from the content server 5 can be combined with the photographed image and displayed according to the altitude and the three-dimensional position of the object corresponding to the content.

  In the flow of FIG. 9, the content (for example, “a” to “i” in FIG. 10) and the display control information (for example, information indicating the display position of the frame surrounding “a” to “i”) are displayed as the content. Although the server 5 transmits to the mobile terminal device 4, the image information for configuring the image in which the content is arranged according to the three-dimensional position (for example, the image obtained by removing the photographed image from the image shown in FIG. 10) is the content. You may comprise so that it may transmit to the portable terminal device 4 from the server 5. FIG.

  Further, FIG. 10 shows a display example of content related to a store in a building, but it is also possible to display content targeting nature such as mountains and constellations. FIG. 11 shows a display example of mountain names and climbing routes.

  DESCRIPTION OF SYMBOLS 2 ... Altitude calculation server, 4 ... Portable terminal device, 5 ... Content server, 21, 41, 51 ... Control part, 44 ... Imaging | photography part, 45 ... Position measurement part, 46 ... Angle detection part, 232 ... Three-dimensional map information DB 532: Content with a three-dimensional position.

Claims (3)

A mobile terminal device having an acquisition function of latitude information, longitude information and elevation angle information, and a photographing function, and an altitude of the mobile terminal device using latitude information, longitude information, elevation angle information and image information transmitted from the mobile terminal device An altitude information acquisition system comprising an altitude information acquisition device for acquiring information,
The altitude information acquisition device includes: a 3D map information database; a collation image generating unit that generates a collation image that matches a captured image configured by the image information from the 3D map information; And a height calculation means for calculating the height of the portable terminal device based on the altitude information in the 3D map information of the reference point and the latitude information, longitude information, and elevation angle information. Advanced information acquisition system characterized by this. An altitude information acquisition device that acquires altitude information of the mobile terminal device using latitude information, longitude information, elevation angle information and image information transmitted from the mobile terminal device,
A three-dimensional map information database, a collation image generating means for generating a collation image matching the captured image constituted by the image information from the three-dimensional map information, and setting a reference point on the collation image Altitude information comprising: altitude information in the three-dimensional map information of the reference point; and altitude calculation means for calculating the altitude of the portable terminal device based on the latitude information, longitude information, and elevation angle information Acquisition device.   A program for causing a computer to function as each means of the altitude information acquisition apparatus according to claim 2.

Images