JP2018120412A - Method and program to control mobile communication terminal, and mobile communication terminal - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent situations where a display provided as an augmented reality from being difficult to view for users.SOLUTION: A mobile communication terminal 100 displays air tags 901 to 905 onto a monitor 35 by superposing images taken with a camera. Each of the air tags 901 to 905 displays each sightseeing spot selected following the direction of the mobile communication terminal 100. The mobile communication terminal 100 further displays, onto the monitor 35, a slide bar 910 for setting the sensitivity of detecting the direction of the mobile communication terminal 100. The position in the horizontal direction of a slider 911 in the slide bar 910 corresponds to the sensitivity of detection. A user can set the sensitivity of detection of the mobile communication terminal 100 by operation to adjust the position in the horizontal direction of the slider 911. The mobile communication terminal 100 detects the direction of the mobile communication terminal 100 of interest in the set sensitivity.SELECTED DRAWING: Figure 1

Description

  The present disclosure relates to providing augmented reality, and more particularly to providing augmented reality according to the position and orientation of a mobile communication terminal.

  A technique related to providing information using augmented reality is known. For example, Japanese Patent Laying-Open No. 2015-149032 (Patent Document 1) states that “a technology for realizing augmented reality with greater flexibility and versatility by accurately determining various situations such as a user and the surrounding environment”. Is disclosed. More specifically, Patent Document 1 states that “a storage device 21 that stores content information 112 in the augmented reality providing system and location information 215 that indicates the current location of the mobile terminal device 2 that is carried by the user. From the position information acquisition unit 201 that acquires the state information 216 indicating the state of the mobile terminal device 2, the content information 112 stored in the storage device 21 according to the acquired position information 215 and the acquired state information 216. An augmented reality composing unit 202 for selecting the fictitious information 217 constituting the augmented reality and a display unit 23 for expressing the augmented reality by outputting the selected fictitious information 217 are provided. ]reference).

Japanese Patent Laying-Open No. 2015-149032

  In recent years, information provided as augmented reality may be generated according to not only the position of the mobile communication terminal but also the orientation. In such a case, the higher the detection sensitivity of the orientation of the mobile communication terminal, the higher the accuracy with which the change in display provided as augmented reality follows the change in the orientation of the mobile communication terminal. Accordingly, when the orientation of the mobile communication terminal changes, the provided augmented reality display changes smoothly.

  On the other hand, if the detection sensitivity of the orientation of the mobile communication terminal becomes too high, the display provided as augmented reality changes following the minute change in the orientation of the mobile communication terminal. Therefore, even when the user's hand holding the mobile communication terminal is slightly shaken, the display provided as augmented reality provided in response to the shake can change. As a result, the display provided as augmented reality is difficult to stabilize and may be difficult for the user to see.

  The present disclosure has been made in order to solve the above-described problems, and an object thereof is to avoid a situation in which a display provided as augmented reality is difficult for a user to see.

  According to certain aspects of the present disclosure, a method for controlling a mobile communication terminal is provided. The method includes obtaining a setting of sensitivity for detecting the orientation of the mobile communication terminal, adjusting the sensitivity for detecting the orientation of the mobile communication terminal according to the setting, and adjusting the orientation of the mobile communication terminal with the adjusted sensitivity. A step of detecting, and a step of displaying information for realizing augmented reality generated according to the detected orientation of the mobile communication terminal.

  When the other situation of this indication is followed, the program for making the processor of a mobile communication terminal perform the said method is provided.

  According to still another aspect of the present disclosure, a mobile communication terminal is provided. The mobile communication terminal displays orientation detection means configured to detect the orientation of the mobile communication terminal, and information for realizing augmented reality generated according to the orientation detected by the orientation detection means. A configured information generation unit, a sensitivity acquisition unit configured to acquire a detection sensitivity setting of the direction detection unit, and a sensitivity of the direction detection unit adjusted according to the sensitivity acquired by the sensitivity acquisition unit Configured sensitivity adjustment means.

  According to the present disclosure, the detection sensitivity of the orientation of the mobile communication terminal can be adjusted. The mobile communication terminal generates information for realizing augmented reality according to the orientation of the mobile communication terminal detected according to the adjusted detection sensitivity. As a result, the display provided as augmented reality can appropriately follow the change in the orientation of the mobile communication terminal, and the situation where the display becomes difficult to see following the minute change in the orientation of the mobile communication terminal is avoided. obtain.

It is a figure which shows the portable communication terminal which displays an air tag as an example of the information which implement | achieves augmented reality in a certain embodiment. It is a figure showing an example of the hardware constitutions of a portable communication terminal. It is a figure which shows an example of the data structure of the information for displaying an air tag. It is a figure for demonstrating an example of the relationship between the structure of the air tag displayed by a portable communication terminal, and the content of an air tag database. It is a figure showing an example of a functional structure of a portable communication terminal. It is a figure for demonstrating an example of the predetermined conditions regarding selection of a sightseeing spot. It is a flowchart of the process performed for the display of an air tag in a portable communication terminal. It is a figure which shows the specific example of the display mode of the air tag in a portable communication terminal. It is a figure which shows the specific example of the display mode of the air tag in a portable communication terminal. It is a figure which shows an example of the screen which displays a slide bar separately from the display screen of an air tag. It is a figure which shows an example of the display of the relevant information of an air tag. It is a figure which shows an example of a structure of the communication system containing a portable communication terminal and a server.

  Hereinafter, embodiments of a mobile communication terminal will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

[Technology]
The mobile communication terminal displays an air tag as an example of information for realizing augmented reality. More specifically, the mobile communication terminal detects the orientation of the mobile communication terminal 100 during execution of an application for realizing augmented reality, for example, and selects a tourist spot according to the orientation from among a plurality of tourist spots. Select and display an air tag corresponding to each selected tourist spot.

  FIG. 1 is a diagram illustrating a mobile communication terminal that displays an air tag as an example of information for realizing augmented reality in an embodiment. As shown in FIG. 1, the mobile communication terminal 100 displays air tags 901 to 905 on a monitor 35 so as to be superimposed on an image photographed by a camera (camera 26 in FIG. 2 described later). Each of the air tags 901 to 905 represents each of the tourist spots selected according to the orientation of the mobile communication terminal 100. In FIG. 1, a hand H represents a user's hand holding the mobile communication terminal 100.

  The mobile communication terminal 100 further displays a slide bar 910 for setting the sensitivity for detecting the orientation of the mobile communication terminal 100 on the monitor 35. The horizontal position of the knob 911 on the slide bar 910 corresponds to the detection sensitivity. The user can set the detection sensitivity of the mobile communication terminal 100 by an operation of adjusting the position of the knob 911 in the horizontal direction. An example of the operation is an operation on the monitor 35 (touch operation type). Another example is an operation on an input switch (an input switch 25 in FIG. 2 described later). The mobile communication terminal 100 detects the orientation of the mobile communication terminal 100 with the set sensitivity.

[Hardware configuration of mobile communication terminal]
FIG. 2 is a diagram illustrating an example of a hardware configuration of mobile communication terminal 100 according to an aspect. With reference to FIG. 2, the configuration of mobile communication terminal 100 will be described.

  The mobile communication terminal 100 includes a CPU (Central Processing Unit) 20, an antenna 23, a communication device 24, an input switch 25, a camera 26, a flash memory 27, a RAM (Random Access Memory) 28, and a ROM (Read). -Only Memory) 29, memory card drive device 30, microphone 32, speaker 33, audio signal processing circuit 34, monitor 35, LED (Light Emitting Diode) 36, communication interface 37, vibrator 38, , A GPS (Global Positioning System) antenna 39, a GPS module 40, an acceleration sensor 41, and a geomagnetic sensor 42. The CPU 20 includes an augmented reality (AR) module 50. A memory card 31 can be attached to the memory card drive device 30.

  The antenna 23 receives a signal transmitted by the base station or transmits a signal for communicating with another communication device via the base station. The signal received by the antenna 23 is subjected to front-end processing by the communication device 24, and the processed signal is sent to the CPU 20.

  The CPU 20 executes processing for controlling the operation of the mobile communication terminal 100 based on a command given to the mobile communication terminal 100. When the mobile communication terminal 100 receives the signal, the CPU 20 executes a predetermined process based on the signal sent from the communication device 24 and sends the processed signal to the audio signal processing circuit 34. The audio signal processing circuit 34 performs predetermined signal processing on the signal, and sends the processed signal to the speaker 33. The speaker 33 outputs sound based on the signal.

  The input switch 25 receives an instruction input to the mobile communication terminal 100. The input switch 25 is realized by a touch sensor, a button provided on the housing of the mobile communication terminal 100, or the like. A signal corresponding to the input command is input to the CPU 20.

  The microphone 32 receives an utterance from the mobile communication terminal 100 and sends a signal corresponding to the uttered voice to the voice signal processing circuit 34. The audio signal processing circuit 34 executes a predetermined process for a call based on the signal, and sends the processed signal to the CPU 20. The CPU 20 converts the signal into data for transmission, and sends the converted data to the communication device 24. The communication device 24 generates a signal for transmission using the data, and transmits the signal to the antenna 23.

  The flash memory 27 stores data sent from the CPU 20. In addition, the CPU 20 reads data stored in the flash memory 27 and executes a predetermined process using the data. In one aspect, the flash memory 27 stores information for displaying an air tag.

  FIG. 3 is a diagram schematically illustrating an example of an information data structure (air tag database) for displaying an air tag. Referring to FIG. 3, the air tag database includes spot names, position information, spot photographs, and related information files for each tourist spot.

  The spot name represents the name of each tourist spot. FIG. 3 shows “spot name (A)” as an example of data constituting the spot name.

  The position information represents the position of each tourist spot and includes, for example, latitude information and longitude information. FIG. 3 shows “position information (A)” as an example of data constituting the position information.

  The spot photograph is an image (photograph) representing each tourist spot. FIG. 3 shows “image data (A)” as an example of data constituting a spot photograph.

  The related information file represents detailed information regarding each tourist spot. The related information file is, for example, a file written in a markup language for providing information about each tourist spot. FIG. 3 shows “related information file (A)” as an example of data constituting the related information file.

  FIG. 4 is a diagram for explaining an example of the relationship between the configuration of the air tag displayed by the mobile communication terminal 100 and the contents of the air tag database. The air tag 901 includes a name part 9011, an image part 9012, and a distance instruction part 9013.

  The name part 9011 displays the spot name in the air tag database. “Toppan Parliament” displayed in the name portion 9011 in FIG. 4 is an example of a spot name.

  The image unit 9012 displays a spot photograph in the air tag database. In the example of FIG. 4, a photograph of “Toppan Parliament” is arranged in the image portion 9012.

  The distance instruction unit 9013 displays the distance calculated using the current position of the mobile communication terminal 100 and the position information of the tourist spot in the air tag database. The calculation of the distance may be a linear distance between the mobile communication terminal 100 and the tourist spot, or when moving from the current position of the mobile communication terminal 100 to the tourist spot according to a road set in a map generated in advance. It may be the length of the route passing through.

  When the user selects an air tag, the mobile communication terminal 100 reproduces a related information file of a tourist spot corresponding to the air tag. The reproduction of the related information file will be described later with reference to FIG.

  Returning to FIG. 2, the RAM 28 temporarily holds data generated by the CPU 20 based on an operation performed on the input switch 25. The ROM 29 stores a program or data for causing the mobile communication terminal 100 to execute a predetermined operation. The CPU 20 reads the program or data from the ROM 29 and controls the operation of the mobile communication terminal 100.

  The memory card driving device 30 reads data stored in the memory card 31 and sends it to the CPU 20. The memory card drive device 30 writes the data output by the CPU 20 in the storage area of the memory card 31.

  The audio signal processing circuit 34 performs signal processing for a call as described above. In the example shown in FIG. 2, the CPU 20 and the audio signal processing circuit 34 are shown as separate configurations. However, in other aspects, the CPU 20 and the audio signal processing circuit 34 may be integrated. Good.

  The monitor 35 is a touch operation type monitor, but the type of mechanism that receives the touch operation is not particularly limited. The monitor 35 displays an image defined by the data based on the data acquired from the CPU 20. For example, still images, moving images, maps, etc. stored in the flash memory 27 are displayed.

  The LED 36 emits light based on a signal output from the CPU 20. In one aspect, the communication interface 37 is realized by WiFi (Wireless Fidelity), Bluetooth (registered trademark), NFC (Near Field Communication), or the like. In another aspect, the communication interface 37 accepts attachment of a data communication cable. The communication interface 37 transmits a signal output from the CPU 20. Alternatively, the communication interface 37 transmits data included in a signal received from the outside of the mobile communication terminal 100 to the CPU 20.

  Vibrator 38 performs an oscillating operation at a predetermined frequency based on a signal output from CPU 20.

  The GPS antenna 39 receives GPS signals transmitted from, for example, four or more satellites. Each received GPS signal is input to the GPS module 40. The GPS module 40 performs a positioning process using each GPS signal and a known technique, and acquires position information of the mobile communication terminal 100.

  The acceleration sensor 41 detects acceleration acting on the mobile communication terminal 100. In one aspect, the acceleration sensor 41 is realized as a three-axis acceleration sensor. The detected acceleration is input to the CPU 20. The CPU 20 detects the movement and posture (tilt) of the mobile communication terminal 100 based on this acceleration.

  The geomagnetic sensor 42 detects the direction in which the mobile communication terminal 100 is facing. Information acquired by this detection is input to the CPU 20.

  The AR module 50 adds, deletes, emphasizes, or attenuates the information read in the RAM 28 to the actual environment around the mobile communication terminal 100, and the reality viewed from the user of the mobile communication terminal 100 Execute the process to extend the world. In one aspect, as described with reference to FIG. 1, the AR module 50 superimposes an air tag corresponding to a tourist spot selected according to the orientation of the mobile communication terminal 100 on an image captured by the camera 26. To display. The display positions of the plurality of air tags 901 to 905 shown in FIG. 1 may be movable by a drag operation on each air tag. The AR module 50 may be configured as a dedicated processing circuit separate from the CPU 20.

[Functional configuration of mobile communication terminal]
FIG. 5 is a diagram illustrating an example of a configuration for realizing a function of displaying an air tag in the mobile communication terminal 100. An example of the function of the mobile communication terminal 100 will be described with reference to FIG.

  As illustrated in FIG. 5, the mobile communication terminal 100 includes an adjustment value acquisition unit 101, a sensitivity adjustment unit 102, low-pass filters 103 and 104, a position specification unit 105, a direction specification unit 106, a distance specification unit 107, and an AR tag. A display unit 108 is included. In an embodiment, each of the adjustment value acquisition unit 101, the sensitivity adjustment unit 102, the low-pass filters 103 and 104, the position specifying unit 105, the direction specifying unit 106, the distance specifying unit 107, and the AR tag display unit 108 is a CPU 20. Is realized by executing an appropriate program. In another embodiment, some and all of the adjustment value acquisition unit 101 to the AR tag display unit 108 may be realized by elements other than the CPU 20 such as a dedicated hardware element.

  In the mobile communication terminal 100, the position specifying unit 105 specifies the current position of the mobile communication terminal 100 according to the data output from the GPS module 40.

  The distance specifying unit 107 specifies the distance between the mobile communication terminal 100 and each tourist spot using the current position specified by the position specifying unit 105 and the position information of each tourist spot in the air tag database (FIG. 3). . In one example, the distance specifying unit 107 specifies the distance as a linear distance between the mobile communication terminal 100 and the tourist spot. In another example, the distance specifying unit 107 uses the distance as the length of a route that passes when moving from the current position of the mobile communication terminal 100 to a tourist spot according to a road set in a previously generated map. Identify.

  The adjustment value acquisition unit 101 acquires an adjustment value of the sensitivity for detecting the orientation of the mobile communication terminal 100 according to the position of the knob 911 on the slide bar 910 (FIG. 1). In one example, the sensitivity specified by the acquired adjustment value is higher as the knob 911 is positioned to the right in the slide bar 910.

  The sensitivity adjustment unit 102 determines the cutoff frequency of each of the low-pass filter 103 and the low-pass filter 104 according to the adjustment value acquired by the adjustment value acquisition unit 101. In determining the cut-off frequency, the sensitivity adjustment unit 102 is, for example, data stored in the flash memory 27 (FIG. 2), and includes an adjustment value, a cut-off frequency of the low-pass filter 103, and a cut-off frequency of the low-pass filter 104. Use data that defines the relationship. In one example, the higher the sensitivity specified by the adjustment value, the higher the cutoff frequency. Thus, as the detection sensitivity is set higher, the low-pass filter 103 outputs a detection output up to a higher frequency output from the geomagnetic sensor 42 to the direction specifying unit 106.

  The direction specifying unit 106 acquires data output from the acceleration sensor 41 via the low-pass filter 104, acquires data output from the geomagnetic sensor 42 via the low-pass filter 103, and the mobile communication terminal 100 according to both data. Specify the direction of.

  The AR tag display unit 108 stores information in the air tag database according to the position specified by the position specifying unit 105, the direction specified by the direction specifying unit 106, and the distance specified by the distance specifying unit 107. A tourist spot according to a predetermined condition is selected from the tourist spots, an air tag of the selected tourist spot is generated, and the generated air tag is displayed on the monitor 35.

  The AR tag display unit 108 may display an air tag generated by another device on the monitor 35. In this case, the AR tag display unit 108 may transmit the position specified by the position specifying unit 105 and the direction specified by the direction specifying unit 106 to another device. The other device selects a sightseeing spot using the position and orientation transmitted from the AR tag display unit 108, generates an air tag of the selected sightseeing spot, and transmits the generated air tag to the mobile communication terminal 100. .

  FIG. 6 is a diagram for explaining an example of a predetermined condition regarding selection of a sightseeing spot. In the example of FIG. 6, the predetermined condition is that the tourist spot is located in the area A1. The region A1 has a sector shape arranged on a horizontal plane defined by the XY plane. The sector center is a point P, each center is 2α, and its radius is a length Ra. In the example of FIG. 6, the point P represents the position of the mobile communication terminal 100. A line L1 represents the direction of the mobile communication terminal 100. The region A1 is a region located within a distance Ra from the mobile communication terminal 100 in a range in which the angle α extends from the direction of the mobile communication terminal 100 in different directions on the horizontal plane.

[Process flow]
FIG. 7 is a flowchart of processing executed by the CPU 20 to display the air tag. The contents of the process will be described with reference to FIG.

  In step S <b> 10, the CPU 20 specifies the position of the mobile communication terminal 100 using the detection output of the GPS module 40 as the position specifying unit 105 (FIG. 5).

  In step S20, CPU 20 determines whether or not there has been an operation on slide bar 910. If CPU 20 determines that operation has been performed on slide bar 910, control proceeds to step S30 (YES in step S20), and if not, control proceeds to step S50 (NO in step S20).

  In step S30, the CPU 20 detects, as the sensitivity adjustment unit 102, the low-pass filter 104 and the geomagnetic sensor 42 applied to the detection output of the acceleration sensor 41 according to the operation amount of the knob 911 of the slide bar 910 (FIG. 1). Each cut-off frequency of the low-pass filter 103 applied to the output is determined.

  In step S <b> 40, the CPU 20 sets the cutoff frequency determined in step S <b> 30 for each of the low-pass filter 103 and the low-pass filter 104 as the sensitivity adjustment unit 102.

  In step S <b> 50, the CPU 20 specifies the direction of the mobile communication terminal 100 using the detection outputs of the acceleration sensor 41 and the geomagnetic sensor 42 as the direction specifying unit 106.

  In step S60, CPU 20 determines whether any of the following (condition 1) and (condition 2) is satisfied.

  <Condition 1> The position specified in step S10 has changed by a predetermined amount or more with respect to the position specified in the previous step S10.

  <Condition 2> The direction specified in step S50 has changed by a predetermined amount or more with respect to the direction specified in the previous step S50.

  When CPU 20 determines that at least one of <condition 1> or <condition 2> is satisfied, the control proceeds to step S70 (YES in step S60), and both <condition 1> and <condition 2> are satisfied. If not, control returns to step S10 (NO in step S60).

  In step S70, CPU 20 selects, as AR tag display unit 108, a sightseeing spot that is a target for air tag display from sightseeing spots whose information is stored in the air tag database (FIG. 3) according to a predetermined condition. . Furthermore, the CPU 20 generates information for displaying an air tag corresponding to the selected sightseeing spot. In one example, in order to display the air tag 901, the CPU 20 arranges “spot name” in the name part 9011, arranges “spot photo” in the image part 9012, and displays the current position of the mobile communication terminal 100 in the distance instruction part 9013. And the distance calculated using the “position information” of the spot.

  In step S80, CPU 20 displays an air tag corresponding to the sightseeing spot selected in step S70 as AR tag display unit 108. Thereafter, the CPU 20 returns the control to step S10.

  As described above, in the process described with reference to FIG. 7, when at least one of the position and orientation of the mobile communication terminal 100 changes by a predetermined amount or more, the CPU 20 executes the control after step S70. The display of the air tag on the monitor 35 is updated.

[Specific example of operation corresponding to display mode of air tag]
FIG. 8 and FIG. 9 are diagrams showing specific examples of the display mode of the air tag in the mobile communication terminal 100. Hereinafter, a specific example of an operation corresponding to the display mode of the air tag in the mobile communication terminal 100 will be described with reference to FIGS. 8 and 9.

<When the air tag display is blurred>
In the example shown in FIG. 8, the mobile communication terminal 100 displays one air tag 901. In FIG. 8, trajectories 901A to 901D represent display positions of the air tag 901 in the immediately preceding short period. In the state shown in FIG. 8, the display of the air tag 901 is shaken following the slight shaking of the casing of the mobile communication terminal 100.

  In such a case, the user operates the slide bar 910 so as to move the knob 911 to the left. As a result, the CPU 20 changes the adjustment value for the direction detection sensitivity so that the detection sensitivity decreases, so that the cutoff frequencies of the low-pass filter 103 and the low-pass filter 104 are increased. Thereby, the sensitivity of detecting the orientation of the mobile communication terminal 100 is lowered.

  When the sensitivity of detecting the orientation of the mobile communication terminal 100 decreases, the amount of change in the orientation of the mobile communication terminal 100 required to change the orientation specified by the detection outputs of the acceleration sensor 41 and the geomagnetic sensor 42 increases. For this reason, the amount of change in the orientation of the mobile communication terminal 100 required for the CPU 20 to update the display of the air tag increases. Thereby, the degree to which the display of the air tag follows the change in the orientation of the mobile communication terminal 100 is reduced, and the display blurring as shown in FIG. 8 is less likely to occur.

<If the air tag display is not smooth>
In the example shown in FIG. 9, the mobile communication terminal 100 displays one air tag 901. In FIG. 9, trajectories 901X and 901Y represent display positions of the air tag 901 in the immediately preceding short period. In the state shown in FIG. 9, the update frequency of the display of the air tag 901 is relatively low, and the user feels that the display of the air tag is not smooth.

  In such a case, the user operates the slide bar 910 so as to move the knob 911 to the right. As a result, the CPU 20 changes the adjustment value for the direction detection sensitivity so that the detection sensitivity increases, thereby changing the cutoff frequencies of the low-pass filter 103 and the low-pass filter 104 to be low. Thereby, the sensitivity of detection of the direction of the mobile communication terminal 100 increases.

  When the sensitivity for detecting the orientation of the mobile communication terminal 100 increases, the amount of change in the orientation of the mobile communication terminal 100 required for the CPU 20 to update the display of the air tag decreases. Thereby, the degree to which the display of the air tag follows the change in the direction of the mobile communication terminal 100 increases, and the user can feel that the display of the air tag changes smoothly.

[Classification of air tag display]
The CPU 20 may classify the air tag displayed on the monitor 35 according to the distance from the mobile communication terminal 100. For example, the CPU 20 attaches a first color frame to an air tag of a tourist spot located at a place where the distance from the mobile communication terminal 100 is less than 1.00 km, and the distance from the mobile communication terminal 100 is 1.00 km 2 or more. A second colored frame is attached to the air tag of the tourist spot located at a location less than 0.000 km, and the third color is attached to the air tag of the tourist spot located at a distance of 2.00 km or more from the mobile communication terminal 100. A frame may be attached.

[Display slide bar]
FIG. 10 is a diagram illustrating an example of a screen that displays a slide bar separately from the air tag display screen. The display screen of the monitor 35 shown in FIG. 10 is a screen for setting an application. As illustrated in FIG. 10, the CPU 20 may accept the sensitivity setting by displaying a slide bar 910 on a screen different from the display screen of the air tag.

[Display related information of air tag]
FIG. 11 is a diagram illustrating an example of display of air tag related information on the monitor 35. In an embodiment, the CPU 20 may display related information corresponding to an air tag when an operation such as a touch operation is performed on the air tag after the air tag such as the air tag 901 is displayed on the monitor 35. The display shown in FIG. 11 is an example of display of related information corresponding to the air tag 901 of FIG.

  More specifically, the air tag 901 in FIG. 1 represents information about a tourist spot with the name “Toppan Parliament”. The display shown in FIG. 11 represents detailed information regarding “Toppan Parliament”. The display shown in FIG. 11 is a display according to the related information file (FIG. 3) of the tourist spot corresponding to the air tag 901.

  The display of FIG. 11 includes a return button 990. When the return button 930 is operated, the CPU 20 may return the display on the monitor 35 to the screen including the captured image and the air tag as shown in FIG.

[Involvement of other devices]
<Air tag database storage location>
FIG. 12 is a diagram illustrating an example of a configuration of a communication system 500 including the mobile communication terminal 100 and the server 150. In one embodiment, information defining the air tag database (FIG. 3) is stored in the storage device of the server 150. In step S70, the CPU 20 reads the air tag database stored in the storage device of the server 150, selects a tourist spot, and generates information for displaying the air tag.

<Generation of information for displaying air tags>
In the communication system 500 according to an embodiment, a part of the process (for example, step S70) illustrated in FIG. 7 may be executed by a processor of the server 150 (an apparatus other than the mobile communication terminal 100). In this example, CPU20 of the mobile communication terminal 100 performs step S10-step S60. If it is determined in step S60 that at least one of condition 1 and condition 2 is satisfied, CPU 20 transmits information specifying the position and orientation of portable communication terminal 100 to server 150. The processor of the server 150 executes control corresponding to step S70.

  More specifically, the processor selects a tourist spot using the position and orientation of the mobile communication terminal 100, generates information for displaying an air tag corresponding to the selected tourist spot, and generates the generated information. Transmit to the mobile communication terminal 100.

  The CPU 20 displays the air tag on the monitor 35 using the information received from the server 150.

[Summary of disclosure]
The disclosure herein is summarized below.

  (1) The CPU 20 of the mobile communication terminal 100 acquires the setting of the sensitivity for detecting the orientation of the mobile communication terminal 100 by determining the cutoff frequency of the low-pass filters 103 and 104 in step S30, and in step S40. The detection sensitivity is adjusted by setting the cutoff frequency of the low-pass filters 103 and 104. In step S50, CPU 20 detects the orientation of portable communication terminal 100 with the adjusted sensitivity. In step S <b> 70, the CPU 20 displays information for realizing the augmented reality generated according to the orientation of the mobile communication terminal 100 on the monitor 35.

  In the mobile communication terminal 100, the sensitivity of detecting the orientation of the mobile communication terminal 100 can be adjusted. Thereby, even when there is an individual difference in the sensitivity of the acceleration sensor 41 and / or the geomagnetic sensor 42, the individual difference is covered by adjusting the sensitivity. Thereby, the display of an air tag can change so that it may be easy to see for a user.

  Furthermore, even if the sensitivity of the installed acceleration sensor 41 and / or geomagnetic sensor 42 changes due to a change in the model of the mobile communication terminal 100, the change in sensitivity can be covered by sensitivity adjustment.

  In the mobile communication terminal 100, the method for detecting (identifying) the orientation of the mobile communication terminal 100 is not limited to the method using the acceleration sensor and the geomagnetic sensor 42. For example, the mobile communication terminal 100 may detect (specify) the orientation of the mobile communication terminal 100 in accordance with the ratio of reception strengths of beacon signals transmitted from a plurality of directions. Even in such a case, the CPU 20 can eliminate the blur in the display of the air tag and can eliminate the situation where the display of the air tag is not smooth by adjusting the sensitivity of the direction detection.

  Note that items displayed as information for realizing augmented reality in the present disclosure are not limited to air tags representing sightseeing spots. As long as the item to be displayed is selected according to the orientation of the mobile communication terminal, it may be of a type other than an air tag, or may be other than a tourist spot.

  (2) In order to acquire the sensitivity setting, the CPU 20 may display an object (slide bar 910) for setting the sensitivity on the display (monitor 35).

  (3) The CPU 20 may display the object on a display together with information (air tag 901) for realizing augmented reality.

  (4) The operation on the object may be the setting of the position of the knob (knob 911) on the slide bar.

  (5) The CPU 20 detects the detection output of the geomagnetic sensor 42 from which the high-frequency component has been removed by the first filter (low-pass filter 103) and the acceleration sensor from which the high-frequency component has been removed by the second filter (low-pass filter 104). The direction of the mobile communication terminal 100 may be specified using the detected output 41. The CPU 20 may change the frequency region removed by the first filter and the frequency region removed by the second filter by setting the cutoff frequencies of the low-pass filter 103 and the low-pass filter 104 in step S40.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  100 mobile communication terminal, 150 server, 500 communication system, 901-905 air tag, 901A-901D, 901X, 901Y locus, 910 slide bar, 911 knob, 9011 name part, 9012 image part, 9013 distance instruction part.

Claims (7)

A method for controlling a mobile communication terminal, comprising:
Obtaining a setting of sensitivity for detecting the orientation of the mobile communication terminal;
Adjusting the sensitivity of detecting the orientation of the mobile communication terminal according to the setting;
Detecting the orientation of the mobile communication terminal with the adjusted sensitivity;
Displaying information for realizing augmented reality generated according to the detected orientation of the mobile communication terminal. Obtaining the sensitivity setting comprises:
Displaying an object for setting the sensitivity on a display;
The method according to claim 1, further comprising: obtaining a content of an operation on the object as the sensitivity setting.   The method according to claim 2, wherein the object is displayed on the display together with information for realizing augmented reality. The object includes a slide bar;
The method according to claim 2, wherein the operation on the object includes setting a position of a knob on the slide bar. Detecting the orientation of the mobile communication terminal comprises:
Obtaining a detection output of a geomagnetic sensor provided in the mobile communication terminal after removing a high-frequency component by a first filter;
Obtaining a detection output of an acceleration sensor provided in the mobile communication terminal after removing a high-frequency component by a second filter,
The step of adjusting the sensitivity of detecting the orientation of the mobile communication terminal includes:
Changing the frequency domain that the first filter removes according to the acquired settings;
The method according to any one of claims 1 to 4, further comprising: changing a frequency domain that the second filter removes according to the acquired setting. A program for controlling a mobile communication terminal,
The program for making the processor of the said mobile communication terminal perform the method of any one of Claims 1-5. A mobile communication terminal,
Orientation detection means configured to detect the orientation of the mobile communication terminal;
An information generation unit configured to display information for realizing augmented reality generated according to the direction detected by the direction detection unit;
A sensitivity acquisition means configured to acquire a detection sensitivity setting of the orientation detection means;
A portable communication terminal comprising: a sensitivity adjustment unit configured to adjust the detection sensitivity of the direction detection unit according to the sensitivity acquired by the sensitivity acquisition unit.

Images