JP6447251B2 - Information processing apparatus, display control method, and display control program - Google Patents

Description

  The present invention relates to an information processing apparatus, a display control method, and a display control program.

  Augmented reality (AR) technology is known in which object data is superimposed on a part of an image captured by an imaging device such as a camera.

  In the conventional AR technology, when position information or identification information (marker ID) with respect to a reference object (image data) such as an AR marker recognized from a captured image is acquired, object data and arrangement information associated with the marker ID are obtained. AR content is acquired, the positional relationship between the marker and the camera is estimated from the size and shape (distortion) of the marker region, and the arrangement (position, orientation, etc.) of the object data is determined using the estimated positional relationship. The object data is superimposed and displayed on the captured image with the determined arrangement.

  Also, instead of displaying the object data associated with the AR marker, based on position information from a predetermined positioning method such as a Global Positioning System (GPS) and inclination information of the terminal device, Object data or the like corresponding to the image captured by the terminal device is superimposed and displayed.

JP-A-8-184436 International Publication No. 2012/033095

  However, there is an error of about 10 m to 20 m as the accuracy of the current GPS. GPS can measure a position in three dimensions (latitude, longitude, altitude), but errors in the horizontal direction (latitude, longitude) are often allowed. On the other hand, the effect of an error of 20 m on the altitude is large. For example, even when it is desired to superimpose the object data of the object when looking down from the top on the photographed image, the object data of the object when looking up from below due to the error is displayed. May end up. Thus, appropriate object data is not displayed due to the altitude error associated with positioning.

  In one aspect, an object of the present invention is to display object data corresponding to an altitude error associated with positioning.

  In one aspect, in the information processing device, an image acquisition unit that acquires an image captured by the imaging device, an altitude information acquisition unit that acquires altitude information of the imaging device, and a tilt that acquires tilt information of the imaging device An information acquisition unit; and a control unit that determines object data to be displayed on the image based on the error information of altitude included in the acquired altitude information and the acquired tilt information.

  As one aspect, object data corresponding to an altitude error associated with positioning can be displayed.

It is a figure which shows an example of a function structure of a terminal device. It is a figure which shows an example of the hardware constitutions of a terminal device. It is a figure which shows an example of AR content information. It is a figure which shows an example of the offset data used at the time of correction | amendment. It is a flowchart which shows an example of the display control process in this embodiment. It is a flowchart which shows an example of a correction process. It is a flowchart which shows an example of a calculation process of a screen position. It is a figure which shows the example of a screen which superimposed object data. It is a figure for demonstrating correction | amendment of the altitude in this embodiment. It is a figure which shows the example of switching of object data.

  Embodiments will be described below with reference to the drawings.

<Functional configuration example of information processing apparatus>
A functional configuration example of a terminal device as an example of an information processing device in the present embodiment will be described with reference to the drawings. FIG. 1 is a diagram illustrating an example of a functional configuration of the terminal device. 1 includes a communication unit 11, an imaging unit (image acquisition unit) 12, a display unit 13, a storage unit 14, a detection unit (altitude information acquisition unit, inclination information acquisition unit) 15, And a control unit 16.

  The communication unit 11 is connected to a communication network such as the Internet or a local area network (LAN) in a state where data can be transmitted and received. For example, the communication unit 11 transmits information such as a shooting position and a tilt to a management server that manages AR content including, for example, object data and arrangement (position) information thereof, and corresponding AR content information and the like from the management server. Receive. The object data is, for example, model data of an object placed in a three-dimensional virtual space corresponding to the real space, and is, for example, superimposition data displayed superimposed on an image captured by the imaging unit 12. . The object data includes various forms such as text, icons, animations, marks, patterns, images, and videos.

  The communication unit 11 can perform short-range communication with a computer such as another terminal device 10 using a communication method such as infrared communication, Wi-Fi (registered trademark), or Bluetooth (registered trademark).

  The imaging unit 12 captures images at regular frame intervals and generates image data. The imaging unit 12 is an imaging device such as a digital camera, for example, but is not limited thereto. The imaging unit 12 may be built in the terminal device 10 or may be a connectable external device. When the imaging unit 12 is mounted, it is preferable that the tilt, direction, and the like are operated integrally with the terminal device 10, but the present invention is not limited to this. In addition, the imaging unit 12 may acquire image data captured externally (imaging device). In this case, it is preferable to have position information and tilt information of the imaging device, but the present invention is not limited to this.

  The display unit 13 displays a captured image acquired from the imaging unit 12 or a composite image in which AR content is superimposed on the captured image. Further, the display unit 13 displays a menu screen or setting screen set in advance for performing the display control process in the present embodiment, an operation screen for operating the terminal device 10, and the like. The display unit 13 may input information from the screen like a touch panel.

  The storage unit 14 stores various information necessary for the present embodiment. For example, the storage unit 14 can write and read information under the control of the control unit 16 and the like. The storage unit 14 stores, for example, the above-described AR content information, offset information used when correcting position information (altitude), various setting information, and the like, but the stored content is not limited thereto. Absent.

  For example, the detection unit 15 acquires position information and tilt information of the terminal device 10 (or the imaging unit 12) using one or a plurality of positioning methods. As a positioning method of position information, for example, GPS is used, or position information (latitude, longitude, altitude) is acquired from the position of a connected Wi-Fi network (for example, router) or mobile network (for example, base station). However, the present invention is not limited to this. For example, the detection unit 15 may be an altitude information acquisition unit that acquires altitude information of the imaging unit 12 or may be an inclination information acquisition unit that acquires inclination information of the imaging unit 12.

  In addition, the detection part 15 can acquire the information of how many GPS were used for positioning. In addition, when the detection unit 15 is simultaneously connected to a plurality of Wi-Fi networks or mobile networks, the detection unit 15 uses the average value of each position information or the position information of the router or base station having the maximum reception intensity. The position information of the terminal device 10 may be acquired.

  As a method for measuring tilt information, for example, using an electronic compass or a gyro sensor, orientation information (tilt (Pitch), orientation (Azimuth), rotation (Roll)), etc. can be obtained. It is not limited. The electronic compass is an example of a geomagnetic sensor, an azimuth sensor, etc., and can detect the earth's geomagnetism in two or three dimensions, and acquire azimuth information depending on which direction it is oriented with respect to the geomagnetism. Further, the gyro sensor can acquire the azimuth information by detecting that it is rotating or detecting a change in direction.

  The detection unit 15 periodically acquires the position information and the tilt information described above. Further, the detection unit 15 can acquire detection distances and recognition distances by various sensors, the imaging unit 12, and the like, a shooting range (viewing angle information), and the like from preset setting information.

  The control unit 16 controls the entire configuration of the terminal device 10. For example, the control unit 16 executes position recognition processing, display control processing, and the like. The position recognition process is a process for recognizing whether or not specific position information for object data or the like is included in a position range that can be acquired in a captured image acquired by the terminal device 10. The display control process is a process of superimposing corresponding object data on a predetermined position of the captured image when specific position information is within a range that can be acquired in the captured image in the position recognition process. Further, the control unit 16 controls execution of various applications and software installed in the terminal device 10. In addition, the control unit 16 controls the start and end of various processes, performs a preprocess set in advance, and controls an error process when an abnormality occurs.

  The control unit 16 illustrated in FIG. 1 includes a recognition unit 21, a determination unit 22, a correction unit 23, and a display control unit 24.

  The recognizing unit 21 performs the above-described position recognizing process from information such as position information, tilt information, recognition distance, and shooting range of the terminal device 10 (or the imaging unit 12). For example, the recognizing unit 21 recognizes whether or not the object data is to be superimposed and displayed within the position range that can be acquired in the captured image from the position information, tilt information, recognition distance, and the like of the terminal device 10. For example, even if a 100 km ahead mountain that is considered unnecessary for the user is photographed by the position recognition process described above, the object data regarding the mountain is separated by a predetermined distance or more (outside the recognition distance). Can be controlled so as not to be a target to be superimposed. Note that, as described above, the criterion for determining whether or not the object data is to be superimposed and displayed may be set in advance based on, for example, the above-described azimuth, position, recognition distance, and the like, and managed by the storage unit 14. Good. The recognition unit 21 can perform the above recognition process by referring to the information.

  The determination unit 22 determines whether correction of position information (for example, altitude or the like) in the present embodiment is necessary. Whether correction is necessary or not is determined to be unnecessary if the content of the object data does not change even if high-level correction is performed. In the case of a predetermined positioning method, it may be determined that there is no need to unconditionally correct. Note that the determination of the necessity of correction is not limited to this.

  When the determination unit 22 determines that the altitude correction is necessary, the correction unit 23, for example, a correction value (for example, a GPS, a Wi-Fi network, a mobile network) or the like that is set according to a positioning method or the like. The altitude correction value is calculated based on the offset value. Information regarding the offset (offset table) is stored in the storage unit 14. The correction content is not limited to this, and position information (latitude or longitude) and inclination information other than altitude may be corrected. Further, the information to be corrected may be only one type (for example, altitude) or a plurality of types.

  The display control unit 24 corresponds to, for example, an object included in the photographed image based on the position information, the tilt information, the AR content information, and the like reflecting the determination result by the determination unit 22, the correction result by the correction unit 23, and the like. Object data is superimposed on the captured image. Further, the display control unit 24 causes the display unit 13 to display the superimposed image. Thereby, the object display which considered the altitude error accompanying positioning can be realized.

  The terminal device 10 is, for example, a tablet terminal, a smartphone, a personal digital assistant (PDA), a notebook PC, or the like, but is not limited thereto, and may be a communication terminal such as a game machine or a mobile phone. .

<Example of Hardware Configuration of Terminal Device 10>
Next, a hardware configuration example of a computer that functions as the terminal device 10 will be described with reference to the drawings. FIG. 2 is a diagram illustrating an example of a hardware configuration of the terminal device. In the example of FIG. 2, the terminal device 10 includes a microphone (hereinafter referred to as “microphone”) 31, a speaker 32, a camera 33, a display unit 34, an operation unit 35, a sensor unit 36, and a power unit 37. The wireless unit 38, the short-range communication unit 39, the auxiliary storage device 40, the main storage device 41, the CPU 42, and the drive device 43 are connected to each other via a system bus B.

  The microphone 31 inputs a voice uttered by the user and other sounds. The speaker 32 outputs the other party's voice or sounds such as ringtones. The microphone 31 and the speaker 32 can be used, for example, when talking to a call partner using a call function or the like, but is not limited to this, and can be used for inputting and outputting information by voice.

  For example, the camera 33 captures an image of a real space (moving image, still image) within a preset angle of view. The camera 33 is an example of the imaging unit 12 described above. The camera 33 may be built in the terminal device 10 or may be externally attached.

  The display unit 34 displays to the user a screen set by an operating system (OS) or various applications (for example, an image in which object data is superimposed on a real space). The display unit 34 is an example of the display unit 13 described above. Further, the display unit 34 may be a touch panel display or the like. In that case, the display unit 34 has a function as an input / output unit. The display unit 34 is a display such as a Liquid Crystal Display (LCD) or an organic Electro Luminescence (EL).

  The operation unit 35 is an operation button displayed on the screen of the display unit 34 or an operation button provided outside the terminal device 10. The operation button may be, for example, a power button or a volume adjustment button, or may be an operation key for character input arranged in a predetermined order. For example, when the user performs a predetermined operation on the screen of the display unit 34 or presses the operation button described above, the touch position on the screen is detected by the display unit 34. The display unit 34 can display an application execution result, content, icon, cursor, and the like on the screen.

  The sensor unit 36 detects a certain point in time or a continuous position, inclination (azimuth), and operation of the terminal device 10. The sensor unit 36 is an example of the detection unit 15 described above. For example, the sensor unit 36 detects the tilt angle, direction, position, acceleration, and the like of the terminal device 10, but is not limited thereto. The sensor unit 36 is, for example, a GPS, a gyro sensor, a tilt sensor, an acceleration sensor, or the like, but is not limited thereto.

  The power unit 37 supplies power to each component of the terminal device 10. The power unit 37 is an internal power source such as a battery, but is not limited thereto. The power unit 37 can also detect the amount of power constantly or at predetermined time intervals and monitor the remaining amount of power.

  The radio unit 38 is a communication data transmission / reception unit that receives a radio signal (communication data) from a base station (mobile network) using an antenna or the like, or transmits a radio signal to the base station via an antenna.

  The short-range communication unit 39 can perform short-range communication with a computer such as another terminal device 10 using a communication method such as infrared communication, Wi-Fi, or Bluetooth. The wireless unit 38 and the short-range communication unit 39 described above are communication interfaces that enable data transmission / reception with other computers.

  The auxiliary storage device 40 is a storage means such as a hard disk drive (HDD) or a solid state drive (SSD). The auxiliary storage device 40 stores an execution program (display control program) in the present embodiment, a control program provided in a computer, and the like based on a control signal from the CPU 42, and performs input / output as necessary. The auxiliary storage device 40 can read and write necessary information from each stored information based on a control signal from the CPU 42 and the like.

  The main storage device 41 stores an execution program read from the auxiliary storage device 40 in accordance with an instruction from the CPU 42, and stores various information obtained during the program execution. The main storage device 41 is, for example, a Read Only Memory (ROM) or a Random Access Memory (RAM).

  The CPU 42 controls processing of the entire computer, such as various operations and data input / output with each hardware component, based on a control program such as an OS and an execution program stored in the main storage device 41. Each process in the display control in this embodiment is realized. The CPU 42 is an example of the control unit 16 described above.

  More specifically, the CPU 42 executes a program corresponding to the program on the main storage device 41 by causing the program installed in the auxiliary storage device 40 to be executed based on, for example, a program execution instruction obtained from the operation unit 35 or the like. I do. For example, by executing a display control program, the CPU 42 performs communication of various data by the communication unit 11 described above, imaging by the imaging unit 12, display by the display unit 13, display of various information by the storage unit 14, and detection by the detection unit 15. Processing such as detection of position information and tilt information is performed. Further, the CPU 42 executes the display control program, thereby determining the position recognition by the recognition unit 21 described above, determining whether it is a correction processing target by the determination unit 22, correcting the position information, etc. by the correction unit 23, and the display control unit. 24, such as display control, is performed. The processing content in the CPU 42 is not limited to the above-described content. The content executed by the CPU 42 is stored in the auxiliary storage device 40 or the like as necessary.

  The drive device 43 can detachably set the recording medium 44, for example, and can read various information recorded on the set recording medium 44 and write predetermined information on the recording medium 44. The drive device 43 is, for example, a medium loading slot or the like, but is not limited to this.

  The recording medium 44 is a computer-readable recording medium that stores an execution program and the like as described above. The recording medium 44 may be a semiconductor memory such as a flash memory, for example. The recording medium 44 may be a portable recording medium such as a USB memory, but is not limited thereto.

  In the present embodiment, by installing an execution program (for example, a display control program) in the hardware configuration of the computer main body described above, the hardware control and the software cooperate to perform the display control processing in the present embodiment. Can be realized. Further, the display control program corresponding to the above-described display control processing may be in a resident state on the terminal device 10, for example, or may be activated by an activation instruction.

<Data example>
Next, an example of data used in the display control process in the present embodiment will be described with reference to the drawings. FIG. 3 is a diagram illustrating an example of AR content information. The AR content information shown in FIG. 3 includes object data and its arrangement (position) information.

  Examples of the AR content information items shown in FIG. 3 include “position information”, “camera information”, “object ID”, “object data”, “coordinates”, and the like, but are not limited thereto. Absent. “Position information” is position information of the terminal device 10 and is information measured by a preset positioning method. The position information can be represented by three-dimensional coordinates (X, Y, Z) composed of latitude, longitude, and altitude. “Camera information” refers to camera parameters such as camera angle-of-view information, and includes tilt information of the terminal device 10 or the imaging unit 12. The “object ID” is identification information for identifying object data corresponding to the position information and camera information described above. “Object data” is information of actual object data. In the example of FIG. 3, image data is shown as object data, but the present invention is not limited to this, and may be data of text, icons, animations, marks, patterns, videos, or combinations thereof. . The object data is at least one of various objects such as buildings, chairs, desks, pipes, and people, arrows, marks, signs, character information, and the like.

  “Coordinates” are coordinates (x, y, z) indicating a position where object data is superimposed and displayed in correspondence with position information, camera information, and the like. The coordinates are associated with each object data. The AR content information shown in FIG. 3 may be acquired in advance by the terminal device 10 from the management server or the like and stored in the storage unit 14, and when the object data is displayed, the current position information, inclination information, and the like are managed. The server may be inquired to obtain corresponding AR content information.

  FIG. 4 is a diagram illustrating an example of offset data used at the time of correction. The items of offset data shown in FIG. 4 include, for example, “positioning method”, “offset content”, but are not limited thereto. In the present embodiment, the offset content is set according to the positioning method when the terminal device 10 measures the current position. “Positioning method” is information indicating the type of positioning. “Offset content” is information relating to the content of the corresponding offset. For example, the maximum value of the error determined according to the positioning method (measurement method) is set, but the present invention is not limited to this.

  In the present embodiment, when the positioning method is GPS in the example of FIG. 4, the offset value (error value) is determined by the number of artificial satellites used for positioning (hereinafter referred to as “the number of satellites”). This is because the error is smaller when the number of satellites is larger. For example, when the “number of satellites” is 4 or less, the offset is 100 m. When the number of satellites is more than 4 and less than 7, the offset is 50 m. When the number of satellites is more than 7, The offset is 20 m.

  In the case of a Wi-Fi router, since the distance from the terminal device 10 is short, an offset value of 100 m is set, and in the case of a mobile network, an offset value of 1000 m is set. The positioning method and its offset value shown in FIG. 4 are not limited to this, and for example, an offset value determined according to the type and performance of the positioning method, the positioning environment, etc. can be set.

<Example of display control processing>
Next, an example of display control processing in the present embodiment will be described using a flowchart. FIG. 5 is a flowchart illustrating an example of the display control process in the present embodiment. In the example of FIG. 5, the control unit 16 of the terminal device 10 performs, as preprocessing, activation of a display control program corresponding to AR display, activation of the imaging unit 12, acquisition of an image area, and the like (S01). Next, the imaging unit 12 starts shooting and acquires an input image (image data) (S02).

  Next, the detection part 15 acquires the positional information and inclination information of the terminal device 10 (S03). In the process of S03, the position information is measured by GPS, for example. Next, the control unit 16 determines whether or not to perform correction (S04), and when performing correction (YES in S04), the position information (latitude, longitude, and altitude) obtained by GPS includes altitude. Error correction processing is performed (S05). The process of S05 will be described later. Further, in the process of S05, error correction other than altitude may be performed.

  Further, the control unit 16 recognizes position information including the acquired altitude when correction is not performed in the process of S04 (NO in S04) or after the process of S05 (S06). Next, the display control unit 24 acquires AR content information corresponding to the position information and tilt information (camera information) of the terminal device 10 (S07). The position information at this time is position information whose altitude is corrected. In the process of S07, AR content information corresponding to position information and tilt information may be acquired from the storage unit 14, for example. In the process of S07, an inquiry may be made to the management server that manages the AR content information, and the content information corresponding to the position information and the tilt information may be acquired.

  Next, the display control unit 24 generates a conversion matrix (S08). For example, the “coordinate” information included in the AR content information is converted into coordinates in the real space. By using the transformation matrix, for example, the size and orientation of the object data can be changed corresponding to the shooting position (coordinates), size, and orientation of the input image.

  Next, the display control unit 24 performs image conversion on the acquired object data (for example, coordinate conversion using a conversion matrix in S07, model-view conversion, perspective conversion, etc.) (S09), and then superimposes and displays it on the input image (S09). S10). Model-view conversion is a process of converting object data into display data such as actual objects (arrows, marks, character information). Further, the perspective conversion is a process of converting into display data to which information such as inclination, transmittance, and shadow is added in the case of a three-dimensional object, for example. Note that the type of image conversion is not limited to this.

  Next, it is determined whether or not to end the process (S11). If the process is not ended (NO in S11), the process returns to S02. In addition, when the process is terminated by a termination instruction or the like of the user or the like (YES in S11), the AR display control process is terminated.

<Correction processing; S05>
Next, the correction process (S05) described above will be described using a flowchart. FIG. 6 is a flowchart illustrating an example of the correction process. In the example of FIG. 6, the correction unit 23 acquires position information such as latitude, longitude, and altitude acquired by the detection unit 15 such as GPS (S21), and the inclination (Pitch), direction (Azimuth) of the terminal device 10, Orientation information such as rotation is acquired (S22).

  In the present embodiment, only the altitude and inclination in the processing of S21 and S22 are used. In the process of S22, the inclination is, for example, an inclination angle of the terminal device 10 and a rotation angle around the X axis (horizontal direction), but is not limited thereto. In the processing of S22, the azimuth is the azimuth angle of the terminal device 10 and is a rotation angle centered on the y-axis (vertical direction), but is not limited to this. The rotation (Roll) is a rotation angle of the terminal device 10 and is a rotation angle centered on the z axis (depth) as shown in the image diagram, but is not limited to this.

  Next, the correction unit 23 calculates a correction value by a predetermined correction formula using the offset data shown in FIG. 4 (S23). An example of the correction formula is “altitude after correction = altitude−sin θ * offset”, but is not limited thereto. θ is, for example, the angle of inclination described above.

  Next, the correction unit 23 sets the acquired value (position information) of GPS as latitude, longitude, and altitude after correction (S24). In the above-described processing, GPS is used as an example of the positioning method. However, for example, altitude correction is similarly performed on the Wi-Fi network (router) and the mobile network (base station) using offset data. Can do. In the present embodiment, not only altitude correction but also latitude and longitude may be corrected using a similar method.

<About the display position (overlapping position) of the object data>
Here, the display position (superimposition position) of the object data in this embodiment will be described. Object data in real space is measured by location (latitude, longitude, altitude), and the information can be acquired from AR content information information set in advance. In the present embodiment, location information is included in advance for each object data. In the display control processing in the present embodiment, calculation of the screen position of the object data, calculation of the size (size) in two or three dimensions, calculation of rotation of the object data, etc. are performed, for example, generated as a transformation matrix. .

  Here, the calculation processing of the screen position of the object data in the display control unit 24 will be described using a flowchart. FIG. 7 is a flowchart illustrating an example of a screen position calculation process.

  In the example of FIG. 7, the display control unit 24 inputs the object data and the location (latitude, longitude, altitude) of the terminal device 10 (S31), and the distance between the terminal device 10 and the object data in latitude, longitude, and altitude. Is calculated (S32). In the process of S31, the location of the corresponding object data may be input from the location (latitude, longitude, altitude) of the terminal device 10 and the tilt information.

  Next, the display control unit 24 calculates the distance between the object data and the screen center on the x, y, and z axes corresponding to the display screen of the terminal device 10 (S33). At this time, orientation information is input. The orientation information is direction position information, for example, and includes azimuth, inclination, rotation, and the like. Next, the display control unit 24 inputs the viewing angle and the altitude of the screen (S34), and calculates OpenGL-related position parameters (S35). OpenGL is a programming interface for graphics processing. By inputting a position parameter or the like, a three-dimensional image or the like corresponding to the input image can be superimposed and displayed. In the present embodiment, OpenGL is used, but the present invention is not limited to this. Next, the display control unit 24 outputs the OpenGL position parameter (S36).

  Here, the above-mentioned content is demonstrated concretely. First, the distance between the terminal device 10 and object data is calculated by the following calculation formulas (1) to (3) based on latitude, longitude, and altitude. In Expressions (1) to (3), for example, the distance between the terminal device 10 and the object data is calculated using latitude, longitude, and altitude.

ここで、Ｌ_ａ１は端末装置１０の緯度を示し、Ｌ_ａ２はオブジェクトデータの緯度を示し、Ａ_１は、端末装置１０の高度を示す。また、Ａ_２は、オブジェクトデータの高度を示し、Ｌ_ｏ１は、端末装置１０の経度を示し、Ｌ_ｏ２は、オブジェクトデータの経度を示す。また、Ｄ_ｘは緯度での距離を示し、Ｄ_ｙは高度での距離を示し、Ｄ_ｚは経度での距離を示す。また、式（１）、式（３）の１１１１１１は、Ｌａｔｉｔｕｄｅ１度の変化における距離（１１１，１１１メートル）を示す。

Here, L _a1 indicates the latitude of the terminal device 10, L _a2 indicates the latitude of the object data, and A ₁ indicates the altitude of the terminal device 10. A ₂ represents the altitude of the object data, L _o1 represents the longitude of the terminal device 10, and L _o2 represents the longitude of the object data. Further, D _x represents a distance in latitude, D _y represents a distance in altitude, and D _z represents a distance in longitude. Moreover, 111111 of Formula (1) and Formula (3) shows the distance (111,111 meters) in the change of Latitude 1 degree.

  Next, the distance between the object data and the screen center is calculated using the following calculation formulas (4) to (6) on the x, y, and z axes with the screen center as the origin.

ここで、上述した式において、Ｄ_ｘは、緯度で、端末装置１０とオブジェクトデータの距離を示し、Ｄ_ｙは高度で、端末装置１０とオブジェクトデータの距離を示し、Ｄ_ｚは、経度で、端末装置１０とオブジェクトデータの距離を示す。また、ｘは端末装置１０の傾斜（ｐｉｔｃｈ）であり、ｙは、端末装置１０の方位（ａｚｉｍｕｔｈ）であり、ｚは端末装置１０の回転（ｒｏｌｌ）である。また、Ｓ_ｘはｘ軸で、オブジェクトデータと画面中央の距離を示し、Ｓ_ｙはｙ軸で、オブジェクトデータと画面中央の距離を示し、Ｓ_ｚはｚ軸で、オブジェクトデータと画面中央との距離を示す。

Here, in the above-described equation, D _x is latitude and indicates the distance between the terminal device 10 and object data, D _y is altitude and indicates the distance between the terminal device 10 and object data, and D _z is longitude, The distance between the terminal device 10 and object data is shown. Further, x is the pitch of the terminal device 10, y is the azimuth of the terminal device 10, and z is the rotation of the terminal device 10. S _x is an x-axis indicating the distance between the object data and the center of the screen, S _y is a y-axis indicating the distance between the object data and the center of the screen, and S _z is a z-axis between the object data and the center of the screen. Indicates distance.

  Next, the OpenGL position parameter is calculated using the following calculation formulas (7) to (11).

ここで、上述の式において、Ｌ_ｘは、画面の幅を示し、Ｌ_ｙは画面の高度を示す。また、Ａ_ｘは、左右から見る視野角を示し、Ａ_ｙは、上下から見る視野角を示す。また、Ｖ_ｘは、ｘ座標を示し、Ｖ_ｙは、ｙ座標を示す。また、Ｓ_ｘはｘ軸で、オブジェクトデータと画面中央の距離を示し、Ｓ_ｙはｙ軸で、オブジェクトデータと画面中央の距離を示し、Ｓ_ｚはｚ軸で、オブジェクトデータと画面中央の距離を示し、Ｏ_ｘはｘ軸で、ＯｐｅｎＧＬの位置パラメータを示し、Ｏ_ｙはｙ軸で、ＯｐｅｎＧＬの位置パラメータを示し、Ｏ_ｚはｚ軸で、ＯｐｅｎＧＬの位置パラメータを示す。

Here, in the above formula, L _x indicates the width of the screen, and L _y indicates the altitude of the screen. Also, A _x represents the viewing angle viewed from the left, A _y indicates the viewing angle viewed from above and below. V _x represents the x coordinate, and V _y represents the y coordinate. S _x is an x-axis indicating the distance between the object data and the center of the screen, S _y is a y-axis indicating the distance between the object data and the center of the screen, and S _z is a z-axis indicating the distance between the object data and the center of the screen. O _x is the x-axis and the OpenGL position parameter, O _y is the y-axis and the OpenGL position parameter, and O _z is the z-axis and the OpenGL position parameter.

  Further, 20 shown in Expression (9) indicates a minimum value (meter) that can be recognized by the distance between the terminal apparatus 10 and the object data, and 200,000 shown in Expression (9) indicates the distance between the terminal apparatus 10 and the object data. The maximum value (meter) that can be recognized by the distance is shown, and 50 shown in the equation (9) indicates the minimum value of the OpenGL position parameter.

  The object data acquired by the above-described method is displayed in a superimposed manner at a predetermined position in association with position information (such as three-dimensional coordinates) of the input image. FIG. 8 is a diagram illustrating an example of a screen on which object data is superimposed. In the example of FIG. 8, a mountain is displayed as an example of real content (target object) 51 corresponding to the content of the object data with respect to the input image 50.

  In this case, the object data 52-1 and 52-2 can be displayed as information corresponding to the content (target object) 51 from the position information and tilt information of the terminal device 10 that captured the input image 50. The object data 52 is text data such as a name or image data such as an arrow, but is not limited thereto. In the present embodiment, the position information of the terminal device 10 is corrected for altitude information corresponding to the positioning method.

  Here, FIG. 9 is a diagram for explaining altitude correction in the present embodiment. In the example of FIG. 9, a castle 60 is displayed on the screen as an example of object data. In the example of FIG. 9, for example, if the altitude measured by GPS is 0 m and the actual user's position is 20 m, the castle looked up from the bottom even though it is looking down from the top (20 m position). 60 may be displayed.

  For example, even if different object data is set for object data that can be seen from an altitude of 0 to 10 m and object data that can be seen from a position of 10 m or more, different object data is displayed due to an error in altitude. Therefore, in the present embodiment, the above-described correction process is performed.

  As an example of correction, as described above, calculation is performed using “corrected altitude = acquired altitude (including error) −tilt of terminal (no error) * offset (for example, maximum error value)”. Can do. In correspondence with the example of FIG. 9 described above, when the inclination θ of the terminal device 10 is −50 °, the corrected altitude is “altitude−sin θ * 20 m (when the offset is 20 m) = 0 m−sin ( −50 °) * 20 = 0 m − (− 15.32 m) = 15.32 m (error corrected by 15.32 m) ”, the altitude can be corrected.

  Further, the value of θ described above is reversed between positive and negative depending on whether the terminal device 10 faces upward or downward. Therefore, the altitude is corrected in the upward direction if the inclination of the terminal device 10 is downward, and the altitude is corrected in the downward direction if it is upward. In addition, by using a sin sine wave, changes in the range of −1 to +1 and around 90 ° and −90 ° become more natural. In the present embodiment, when the positioning method is a Wi-Fi network or a mobile network, correction is performed using an offset value corresponding to the positioning method.

  Here, the correction value is created on the assumption that the user is at a position of 20 m, but the calculation result has an error of 15.32 m. However, in the present embodiment, the accuracy of correction is not so necessary. For example, when displaying object data such as a large castle where object data is switched based on an altitude of 10 m by correcting to 15.32 m, for example. , It can display the object data that suits its altitude. In the present embodiment, the correction value can be calculated by simple calculation using the offset value according to the positioning method, so that the processing load can be reduced and real-time object data superimposed display can be realized.

  FIG. 10 is a diagram illustrating an example of switching of object data. As shown in FIG. As shown in FIG. 10, when there is no GPS altitude correction in the present embodiment, the looked up image (castle 60-1) is displayed when viewed from below, but this embodiment is applied. By performing altitude correction, as shown in FIG. 10, it is possible to display an image (castle 60-2) with a composition looking down when viewed from above from the terminal device 10.

  As described above, according to the present embodiment, it is possible to display object data corresponding to an altitude error associated with positioning such as GPS. For example, in the present embodiment, location information (latitude, longitude, altitude) is acquired by GPS or the like, and error range information based on GPS recognition status is obtained. Further, the angle information of the terminal is acquired by an electronic compass, and the altitude is corrected by the altitude information, the error range information, and the terminal angle information. The present embodiment can be applied to, for example, an AR (vision based AR) using an AR marker and a location based AR.

  Although the embodiments have been described in detail above, the invention is not limited to the specific embodiments, and various modifications and changes can be made within the scope described in the claims. Moreover, it is also possible to combine a part or all of each Example mentioned above.

In addition, the following additional remarks are disclosed regarding the above Example.
(Appendix 1)
An image acquisition unit that acquires an image captured by the imaging device;
An altitude information acquisition unit for acquiring altitude information of the imaging device;
An inclination information acquisition unit for acquiring inclination information of the imaging device;
A control unit that determines object data to be displayed on the image based on error information of altitude included in the acquired altitude information and the acquired tilt information;
An information processing apparatus comprising:
(Appendix 2)
The controller is
The information processing apparatus according to appendix 1, wherein the altitude information is corrected using an error value determined according to a type of positioning method that has acquired the altitude information.
(Appendix 3)
The controller is
The information according to appendix 1 or 2, wherein whether or not the altitude information needs to be corrected is determined, and when it is determined that the altitude information needs to be corrected, the altitude information is corrected. Processing equipment.
(Appendix 4)
The information processing apparatus according to attachment 2, wherein the positioning method includes at least one of GPS, Wi-Fi network, and mobile network.
(Appendix 5)
The information processing apparatus according to appendix 2, wherein the error value is a maximum value of an error corresponding to the positioning method.
(Appendix 6)
Obtain an image captured by the imaging device,
Obtaining altitude information of the imaging device;
Obtaining tilt information of the imaging device;
Determining object data to be displayed on the image based on the error information of altitude included in the acquired altitude information and the acquired tilt information;
A display control method in which processing is executed by a computer.
(Appendix 7)
Obtain an image captured by the imaging device,
Obtaining altitude information of the imaging device;
Obtaining tilt information of the imaging device;
Determining object data to be displayed on the image based on the error information of altitude included in the acquired altitude information and the acquired tilt information;
A display control program for causing a computer to execute processing.

10 Terminal device (information processing device)
11 Communication Unit 12 Imaging Unit (Image Acquisition Unit)
13, 34 Display unit 14 Storage unit 15 Detection unit (Altitude information acquisition unit, inclination information acquisition unit)
DESCRIPTION OF SYMBOLS 16 Control part 21 Recognition part 22 Judgment part 23 Correction part 24 Display control part 31 Microphone 32 Speaker 33 Camera 35 Operation part 36 Sensor part 37 Electric power part 38 Radio | wireless part 39 Short-range communication part 40 Auxiliary storage device 41 Main storage device 42 CPU
43 Drive device 44 Recording medium 50 Input image 51 Content (object)
52,60 Object data

Claims (6)

An image acquisition unit that acquires an image captured by the imaging device;
An altitude information acquisition unit for acquiring altitude information of the imaging device;
An inclination information acquisition unit for acquiring inclination information of the imaging device;
A control unit that determines object data to be displayed on the image based on error information of altitude included in the acquired altitude information and the acquired tilt information;
An information processing apparatus comprising: The controller is
The information processing apparatus according to claim 1, wherein the altitude information is corrected using an error value determined according to a type of a positioning method that acquired the altitude information. The controller is
The determination as to whether or not the altitude information needs to be corrected, and the altitude information is corrected when it is determined that the altitude information needs to be corrected. Information processing device.   The information processing apparatus according to claim 2, wherein the positioning method includes at least one of a GPS, a Wi-Fi network, and a mobile network. Obtain an image captured by the imaging device,
Obtaining altitude information of the imaging device;
Obtaining tilt information of the imaging device;
Determining object data to be displayed on the image based on the error information of altitude included in the acquired altitude information and the acquired tilt information;
A display control method in which processing is executed by a computer. Obtain an image captured by the imaging device,
Obtaining altitude information of the imaging device;
Obtaining tilt information of the imaging device;
Determining object data to be displayed on the image based on the error information of altitude included in the acquired altitude information and the acquired tilt information;
A display control program for causing a computer to execute processing.

Images