JP2010171664A - Personal digital assistant, information display control method, and information display control program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To display a message in accordance with a situation of a real world which changes with the elapse of time. <P>SOLUTION: A GPS module 13 and a position information calculation module 14 detect the current position of a self-terminal. A detailed information acquiring module 15 searches an imaging range of a camera module 11 and measures a distance from the absolute position of the self-terminal to each target object. A control section 10 acquires message data and absolute position information from a message server, determines a sequence between the display range of the each target object and a drawing range of the message and their overlapping on the basis of the distance to the each target object, the absolute position information of the message, the display range of the each target object and the drawing range of the message, superposes a message having the former sequence and only a message of the drawing range that does not overlap the display range of the target object on a picked-up video for display. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a portable information terminal, an information display control method, and an information display control method capable of displaying a message virtually written in association with an object in the real world on, for example, an image of the real world captured by a built-in camera, and The present invention relates to an information display control program.

  In recent years, for example, in addition to conventional communication methods such as direct conversation, telephone conversation via telephone, communication by e-mail, and bidirectional communication via the Internet, for example, Japanese Patent Application Laid-Open No. 2008-71298 A next-generation communication method such as (Patent Document 1) has also been proposed.

  This Patent Document 1 proposes a communication technique that allows messages to be exchanged using the real space by superimposing and displaying the bulletin board image on the real space image captured by the digital camera. . That is, in the technique described in Patent Document 1, the spatial bulletin board input device converts the movement of drawing the memo contents in the real space into the three-dimensional coordinate information, and acquires the position information specifying the place where the memo is described. And send it to the spatial communication server. The spatial bulletin board display device displays a captured image of the real space and transmits a browsing request to the spatial communication server. The spatial bulletin board display device acquires the three-dimensional coordinate information and the memo position information of the memo contents near the shooting location from the spatial communication server, and displays the acquired memo contents on the captured image. According to the technique described in Patent Literature 1, virtual communication using the real space can be realized by providing a virtual bulletin board in the real space and enabling writing and browsing on the terminal. ing.

  Further, in the published patent publication (Patent Document 2) of Japanese Patent Application Laid-Open No. 2004-317715, the direction and distance to an object are measured and associated with the object without going to the position of the object in the real space. A technique has been proposed in which a message can be registered even for an object in a place where the user cannot enter by storing the message. That is, in Patent Document 1, the mobile terminal device receives an input of text data from a user via the operation unit, and also sets a relative position setting dialog for a distance to a registration target (object) for registering the text data. Accept input to. In addition, the mobile terminal device acquires direction information indicating the orientation of the mobile terminal device using the built-in geomagnetic sensor and gyroscope while being supported by the user so that the registration target exists ahead. Furthermore, the mobile terminal device measures the position (latitude, longitude, and altitude) of the mobile terminal device using a built-in GPS receiver. Then, the mobile terminal device calculates the position information of the registration target based on the position information, the direction information, and the distance to the registration target, and associates the calculated position information with the text data in the storage unit. Remember.

JP 2008-71298 A (FIG. 1) JP 2004-317715 A (FIG. 6)

  By the way, in the real world, for example, a new building is built, a road is built, or conversely, the building is destroyed and disappears. Therefore, for example, it is fully conceivable that a new building is built around the position where the message described in Patent Document 1 described above is registered.

  However, in the case of the technique described in Patent Document 1, a message that has already been registered at a certain position is always displayed even when, for example, a building or the like is newly built around that position. become. That is, in the image obtained by imaging the real space, a message before the building is constructed is displayed even though a new building exists.

  Thus, the conventional technology cannot cope with the real world situation that changes with the passage of time.

  The present invention has been proposed in view of such circumstances, and a portable information terminal, an information display control method, and information for enabling a message to be displayed in response to a real-world situation that changes with the passage of time. An object is to provide a display control program.

  The portable information terminal of the present invention captures an image of the real world using an imaging lens and an image sensor, displays the captured image, and communicates with a message server that manages a plurality of messages in association with absolute positions in the real world. Then, a message is acquired from the message server, and is displayed superimposed on the captured image. At this time, the portable information terminal of the present invention detects the current absolute position of the terminal, obtains the imaging range in the real world, and each object present in the real space within the imaging range from the absolute position of the terminal. Is further connected to the message server through the communication unit, and the message data managed in association with the absolute position within the imaging range and the absolute position information are acquired from the message server. Then, the portable information terminal of the present invention provides the distance to each object, the absolute position information of the message, the display range of each object in the captured image, and the drawing when the message is superimposed on the captured image. Based on the range, the front-rear relationship between the display range of the object and the drawing range of the message in the captured image, and the overlap between the display range of the target object and the drawing range of the message are determined. Only the message in the drawing range that does not overlap the display range of the object is drawn while being superimposed on the captured image. Thereby, this invention solves the subject mentioned above.

  That is, according to the present invention, when the drawing range of the message is before the display range of the object in the captured video, the message is drawn, and the drawing range of the message is after the display range of the target in the captured video. In some cases, only when the drawing range of the message is out of the display range of the object, only the drawing range of the part that is out of the display range is displayed.

  In the present invention, the front / rear relationship between the display range of the object and the drawing range of the message and their overlap are determined, and only the message of the drawing range that does not overlap the display range of the target object and the message having the front / rear relationship before are captured. By superimposing and drawing on the video, it is possible to display a message corresponding to the situation in the real world that changes over time.

1 is a diagram illustrating a schematic overall configuration example of a message management system according to an embodiment of the present invention. It is a block diagram which shows the schematic internal structural example of the mobile telephone terminal of embodiment of this invention. It is a figure which shows a mode that the message is superimposed and displayed on the picked-up image displayed on a view finder screen according to the message browsing request from a user, and its image | video. It is a figure used for description of the viewfinder screen after the mobile telephone terminal is moved from the example of FIG. 3 and the scenery of the real space. It is a figure used for description of the positional relationship between an imaging range in a real space by a camera module, a target (object) such as a building in the real space existing in the imaging range, and a message associated with the object in the imaging range. . FIG. 6 is a diagram showing a display example of objects and messages displayed on the viewfinder screen of the mobile phone terminal of the present embodiment in the case of the arrangement example of each object (object) and message in FIG. 5. 6 is a flowchart when a message is displayed according to the positional relationship between an object within an imaging range and a message in the mobile phone terminal of the present embodiment. It is a sequence diagram of this embodiment system in the case of displaying a message in the viewfinder screen of a mobile phone terminal. It is a figure which shows the example of a display of the captured image (view finder screen) of a camera module in the case of matching absolute position information with the input of a message, and the input message. FIG. 10 is a diagram used for explaining the viewfinder screen and the scenery in the real space after the mobile phone terminal is moved from the example of FIG. 9. It is a flowchart at the time of the message input in the mobile telephone terminal of this embodiment. It is a sequence diagram of this embodiment system when inputting a message and registering it. It is a flowchart in the case of inputting the message which limited the terminal which can browse a message. It is a flowchart in a message server in case the message for which the browsing object was limited is browsed. It is a flowchart in the mobile telephone terminal which requests | requires browsing of the message by which browsing object was limited. It is a sequence diagram of this embodiment system in the case of inputting and registering a message whose browsing target is limited. 12 is a flowchart for controlling a message display position while avoiding an object moving on a viewfinder screen.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  In the following description, as an example of application of the portable information terminal of the present invention, a mobile phone terminal having a camera function is given, but of course it is merely an example, and the present invention is not limited to this example. For example, the present invention can also be applied to various portable information terminals other than mobile phone terminals.

[Entire system configuration]
FIG. 1 shows a schematic overall configuration example of a message management system according to an embodiment of the present invention.

  In FIG. 1, a mobile phone terminal 2 has all the functions of a general mobile phone terminal, and a camera module is mounted as one of the functions. When the mobile phone terminal 2 captures an image of a real space with a camera module, the captured image can be displayed on a display screen. The mobile phone terminal 2 also includes a GPS (Global Positioning System) module and a position information calculation module. The GPS module receives a radio wave from a GPS (Global Positioning System) satellite 1, and the position information calculation module calculates the position (latitude, longitude, altitude) of the terminal using the received information. Although details will be described later, the mobile phone terminal 2 of the present embodiment communicates with the message server 4 via the packet network 3 to send a message related to the current position of the terminal and the imaging range of the camera module. The message is obtained from the message server 4 and the message can be displayed on the display screen.

  The message server 4 performs registration and management of messages written in association with various objects in the real space, transmission of messages in response to message browsing requests from mobile phone terminals and the like. Specifically, the message server 4 registers and manages a plurality of messages in association with the absolute position information in the real space. The plurality of messages and absolute position information are stored in the message database 5.

[Schematic configuration of mobile phone terminal]
FIG. 2 shows a schematic internal configuration example of the mobile phone terminal 2 according to the embodiment of the present invention. Note that FIG. 2 illustrates only main components necessary for displaying a message, inputting a message, and the like in the cellular phone terminal 2 of the embodiment of the present invention. Such components are not shown. Of course, it goes without saying that the cellular phone terminal 2 of the present embodiment also has general various functions.

  As shown in FIG. 2, the mobile phone terminal 2 of the present embodiment includes a control unit 10, a camera module 11, a memory 12, a GPS module 13, a location information calculation module 14, and detailed information acquisition as main components according to the invention. The module 15 includes a communication module 16, a display module 17, a message display module 18, a message input module 19, and a key button module 20.

  The communication module 16 is for performing communication via the packet network 3, and in particular in the present embodiment, communicates with the message server 4.

  The display module 17 includes, for example, a display panel such as a liquid crystal and a display driving circuit for displaying an image or the like on the display panel.

  The camera module 11 includes an imaging lens system, an imaging device, a video processing circuit that converts an imaging signal obtained from the imaging device into a video signal for display display, and the like. The real-world video signal imaged by the image sensor of the camera module 11 is sent to the display module 17. As a result, a real-time captured image of the real world captured by the camera module 11 is displayed on the screen of the display module 17.

  The memory 12 stores the information display control program of the present invention as one of application programs for the mobile phone terminal. In the case of the present embodiment, the application program is executed by a CPU (not shown), so that messages can be browsed and input as described later. In the present embodiment, when the CPU executes the application program, for example, the control unit 10, the position information calculation module 14, the detailed information acquisition module 15, the message display module 18, the message input module 19, and the like in FIG. Each function can be realized. Although details will be described later, when the cellular phone terminal 2 of the present embodiment acquires message data and absolute position information corresponding to the message from the message server 4, the message data and absolute position information are also stored in the memory 12. save. Although details will be described later, when a message is input from the user via the message input module 19, the mobile phone terminal 2 according to the present embodiment has data of the input message and absolute position information corresponding to the message. Are stored in the memory 12.

  The GPS module 13 receives radio waves from the GPS satellite 1 and sends the received information to the position information calculation module 14.

  The position information calculation module 14 calculates the absolute position (latitude, longitude, altitude) of the terminal itself in the real world using information received from the GPS satellite 1 by the GPS module 11. Further, the position information calculation module 14 stores the calculated absolute position information in the memory 12.

  In the present embodiment, the GPS module 13 and the position information calculation module 14 are configured separately, but the position information calculation function may be included in the GPS module 13.

  The detailed information acquisition module 15 includes a gyro sensor, an acceleration sensor, an electronic compass, and the like. The gyro sensor detects the amount and direction of rotation when a rotational motion is applied to the mobile phone terminal, and the tilt of the mobile phone terminal due to the rotation. The acceleration sensor detects the movement acceleration when the acceleration movement is applied to the mobile phone terminal, the movement amount, and the movement direction. The electronic compass detects the current orientation of the mobile phone terminal, particularly the orientation in which the lens of the camera module 11 faces in the present embodiment.

  Further, the detailed information acquisition module 15 uses the detection signals from the gyro sensor, the acceleration sensor, and the electronic compass, and the absolute position of the own terminal calculated by the position information calculation module 4 at the present time, so that the camera module 11 of the own terminal. Finds a detailed imaging range of the real space being imaged. That is, in the case of the present embodiment, the detailed information acquisition module 15 obtains the attitude of the own terminal at the present time in the real space from both detection signals of the gyro sensor and the acceleration sensor, and further in that attitude Based on the detection signal from the electronic compass and the absolute position of the terminal calculated by the position information calculation module 14, it is calculated which direction the lens of the camera module 11 is facing. Then, the detailed information acquisition module 15 projects the extent of any orientation in the real space by the camera module 11 based on the orientation of the terminal in the real space and the orientation of the lens in the orientation. Is calculated. A specific example of the detailed imaging range in the real space obtained by the detailed information acquisition module 15 will be described later.

  Further, the detailed information acquisition module 15 accesses the message server 4 via the communication module 16 and requests transmission of a message registered within the imaging range. When the detailed information acquisition module 15 acquires the message data and the absolute position information corresponding to the message from the message server 4, the detailed information acquisition module 15 stores them in the memory 12.

  When the message data acquired from the message server 4 and the absolute position information corresponding to the message are stored in the memory 12, the message display module 18 captures the real-world image captured by the camera module 11. The above message is superimposed on the screen and displayed on the screen of the display module 17. That is, the message display module 18 in this case is associated with the absolute position information on the position indicated by the absolute position information in the imaging range of the real space calculated by the detailed information acquisition module 15. Display each message. A specific example of a message displayed on the screen of the display module 17 by the message display module 18 will be described later.

  The key button module 20 is mounted on a mobile phone terminal such as a numeric keypad, a speech key, an end key (power button), a clear key, an application key, a cross key, a determination key, and a shutter button for a camera. It consists of various keys and a circuit for generating an operation signal when these keys are operated.

  The message input module 19 displays message data such as text input by the user through the key button module 20 on the screen of the display module 17 and stores it in the memory 12. Although details will be described later, the message input by the user is associated with an object (object) or the like selected by the user within the imaging range of the camera module 11. Although details will be described later, the message input by the user is associated with absolute position information of the position designated by the user within the imaging range of the camera module 11. Accordingly, the memory 12 stores absolute position information associated with the message together with the input message. That is, when the user inputs a message after selecting a desired object (target object) or the like within the imaging range and designates a position within the imaging range, the message input module 19 The associated absolute position information is stored in the memory 12.

  The message and the absolute position information stored in the memory 12 are sent from the communication module 16 to the message server 4 via the packet network 3 and registered in the message server 4 according to a transmission instruction from the user, for example. Will be.

  The control unit 10 controls the overall control of the mobile phone terminal of this embodiment and the operation of each module.

[Display display example when viewing messages]
In FIG. 3, a message browsing request is input from the user through the key button module 20, whereby the message and the absolute position information corresponding to the message are acquired from the message server 4, and the captured image of the camera module 11 (view of the display module 18 is displayed). A display example in the case where the message is displayed in a superimposed manner on an image of the real space on the finder screen 30) is shown. In FIG. 3, for example, a part of the real space landscape 31 where the house 31a and the tree 31b exist is the imaging range of the camera module 11, and the viewfinder screen 30 displaying the imaging range is shown. For example, a display example when two messages 35 and 36 are displayed in a superimposed manner is shown.

  As in the example of FIG. 3, in the mobile phone terminal of the present embodiment, when the camera module 11 captures an image and a message browsing request is input from the user, the detailed information acquisition module 15 From the detection signals from the acceleration sensor and the electronic compass and the absolute position of the terminal calculated by the position information calculation module 14, a detailed imaging range of the real space currently captured by the camera module 11 is obtained.

  At this time, the message display module 18 displays the messages 35 and 36 at any position in the viewfinder screen 30 displaying the imaging range of the real space from the absolute position information acquired from the message server 4. The message 35, 36 is displayed at the position through the message display module 18 after calculating whether to arrange.

  In FIG. 3, the position between the house 31a and the tree 31b and near the ground within the imaging range of the real space displayed on the viewfinder screen 30 is the absolute position information of the message 35. In the imaging range, an example is shown in which the position of the message 36 is almost directly above and near the tree 31b.

  Here, it is assumed that the mobile phone terminal is moved in the direction indicated by the arrow 32 in the figure when the camera module 11 is imaging the real world as shown in FIG.

  In this case, the detailed information acquisition module 15 can know from the detection signals from the gyro sensor, acceleration sensor, and electronic compass that the mobile phone terminal has been moved in the direction of arrow 32 in the figure. Then, the detailed information acquisition module 15 at this time uses the detection signals from the gyro sensor, the acceleration sensor, and the electronic compass and the own terminal absolute position calculated by the position information calculation module 4 in the direction of the arrow 32. The detailed imaging range in the real world that the camera module 11 is imaging at the time after the movement is obtained.

  FIG. 4 shows a viewfinder screen 30 and a real space landscape 31 after the mobile phone terminal is moved in the direction of the arrow 32 in FIG. In this case, the message display module 18 of the cellular phone terminal 2 of the present embodiment uses the viewfinder screen 30 based on the absolute position information of each message 35 and 36 and the imaging range currently captured by the camera module 11. It is calculated at which position the messages 35 and 36 should be placed, and the messages 35 and 36 are displayed at the positions through the message display module 18.

  In the example of FIG. 4, since the imaging range has moved in the direction of the arrow 32 from the example of FIG. 3, it corresponds to a position between the house 31 a and the tree 31 b and near the ground in the landscape 30 in the real space. An example in which the message 35 deviates from the viewfinder screen 30, while the message 36 corresponding to a position substantially directly above and near the tree 31 b in the scenery 30 in the real space remains in the viewfinder screen 30. Is shown.

  In order to realize such a thing, in this embodiment, the imaging range calculation module 15 performs the sensing operation (detection operation) of the gyro sensor, the acceleration sensor, and the electronic compass at a sufficiently short time interval that can follow the movement of the mobile phone terminal. )It is carried out. Further, the message display module 18 updates the display position of the message in accordance with the update of the shooting range in the shooting range calculation module 15.

  As described above, in this embodiment, the calculation process of the imaging range for the real space is repeated at short time intervals, and the message display position is updated in correspondence with the imaging range obtained every short time. Even when the user arbitrarily moves the mobile phone terminal, it is possible to appropriately control the display position of the message in the real space and display the message on the viewfinder screen.

  In addition, as described above, the mobile phone terminal according to the present embodiment has the position, orientation, and orientation of the mobile phone terminal based on the position information by GPS and the detection information by various sensors such as a gyro sensor, an acceleration sensor, and an electronic compass. The corresponding photographing range can be grasped, and further, it is possible to determine in which position on the viewfinder screen 30 the message should be arranged and displayed from the absolute position information of the message. It is also possible to update the determination of the shooting range and the display of messages at any time.

  Therefore, according to the message management system of the present embodiment, a place where users communicate with each other using a real-world image projected through the camera module 11 and a message superimposed on the image, or a company gives the user It is possible to provide a place for providing advertisements to be distributed.

[Message display control according to the positional relationship between the object in the imaging range and the message]
By the way, in the real world, a situation change such as a new building being built often occurs.

  For this reason, in the embodiment of the present invention, the following processing is performed in order to cope with such a change in the situation in the real world.

  That is, the mobile phone terminal according to the present embodiment has the absolute position of each object such as a building in the real space captured by the camera module 11 and the display range of each object such as a building displayed on the viewfinder screen 30. And the display position of the message in the viewfinder screen using the absolute position of the message associated with the object in the imaging range by the camera module 11 and the drawing range occupied by the message displayed in the viewfinder screen To control.

  In FIG. 5, the imaging range 23 in the real space by the camera module 11, objects (objects) 41 and 42 such as buildings in the real space existing in the imaging range 23, and the imaging range 23 in the real space An example of the positional relationship between messages 43, 44, and 45 associated with an object (object) or the like is shown.

  6 shows an object 41a (video of the target object 41) and an object 42b (target object) displayed on the viewfinder screen 30 of the display module 17 by imaging the imaging range 23 of FIG. 42) and messages 43, 44, and 45 displayed in the viewfinder screen 30 of the imaging range 23.

  Here, the imaging range 23 by the lens 22 of the camera module 11 is a fan-shaped range in the real space as shown in FIG. Note that the actual imaging range of the lens 22 is included up to an infinite distance within the field angle range, but it is not realistic to include all the infinite distance within the imaging range. For this reason, in the present embodiment, a certain limit is set for the distance from the lens 22 in the imaging range 23. In the present embodiment, as shown in FIG. 5, the imaging range 23 is, for example, within a distance within a radius R meter from the focal point of the lens 22.

  In the example of FIG. 5, two objects 41 and 42 such as buildings exist in the fan-shaped imaging range 23, and three messages 43, 44, and 45 are registered in the imaging range 23. Has been. The message 43 is such that the absolute position of the shooting range 23 in the real space overlaps with the direction in which the object 41 exists as viewed from the lens 22 and is farther from the object 41. The range is a part of the display range of the object 41 a of the target object 41 displayed in the viewfinder screen 30. On the other hand, in the case of the message 44, the absolute position of the shooting range 23 in the real space overlaps with the direction in which the object 42 exists when viewed from the lens 22, but is closer to the object 42. Yes. In the case of the message 45, the absolute position of the shooting range 23 in the real space is a position that does not overlap with any object as viewed from the lens 22.

  In the case of the positional relationship as shown in FIG. 5, the display on the viewfinder screen 30 corresponds to the positional relationship between the objects 41a and 42a of the objects and the messages 43, 44, and 45 as shown in FIG. Shall. That is, according to the present embodiment, the message 43 is displayed in a state where a part of the drawing range is obstructed by the object 41a in order to indicate that it is behind the object 41a of the object 41. To. On the other hand, the message 44 is displayed so as to be superimposed on the object 42a in order to indicate that it is in front of the object 42a of the object 42. Note that the message 45 is displayed as it is because it does not overlap any of the objects 41a and 42a.

  Here, in order to realize the display of the viewfinder screen 30 as shown in FIG. 6, the mobile phone terminal 2 of the present embodiment performs the following processing.

  In the mobile phone terminal 2 of the present embodiment, when the real space is imaged, the real space in the imaging range is scanned using the autofocus function of the camera module 11 as an example. In other words, the camera module 11 in this case uses the autofocus function to obtain points that are in focus at substantially the same distance in the real space within the imaging range. Then, the imaging range calculation module 15 determines that there is a group of objects in a portion composed of a set of points that are approximately the same distance by the scan, and determines the position of the object in the imaging range and the The distance to the object is stored in the memory 12.

  The distance to the object in the real space is measured, for example, by moving the mobile phone terminal 2 in the horizontal direction or the vertical direction without moving in the focus direction (depth direction), and imaging before and after the movement. The distance to the object may be obtained by using two images and the principle of a so-called stereo camera. That is, in this case, when the same object is imaged at the center of the screen while moving the mobile phone terminal 2, for example, the absolute position before and after the movement and the change in orientation during the imaging before and after the movement (that is, the change in angle) And the distance to the object is calculated by the so-called triangulation method.

  In addition, the presence determination of the object in the real space and the measurement of the distance may be performed by, for example, an ultrasonic sensor. In this case, the mobile phone terminal includes an ultrasonic sensor module, emits ultrasonic waves from the ultrasonic sensor module toward the imaging range, and measures the arrival time difference between the reflected waves, thereby determining the presence of the target object. And distance calculation.

  Next, the message display module 18 includes a range in which the object of each target object extracted within the shooting range is displayed in the viewfinder screen, a distance to each target object calculated by the shooting range calculation module 15, The viewfinder screen 30 as shown in FIG. 6 is displayed using the absolute position of the message existing in the shooting range and the drawing range when each message is displayed on the viewfinder screen.

  That is, for example, when the display range of the object 41a and the drawing range of the message 43 overlap in the viewfinder screen 30 like the object 41a and the message 43, the message display module 18 displays the object 41a up to the object 41. The distance is compared with the distance obtained from the absolute position of the message 43. When the distance to the target object 41 is closer, the message 43 is displayed for the part hidden in the display range of the object 41a of the target object 41. Only the drawing range outside the display range of the object 41a is displayed without being displayed.

  On the other hand, even if the display range of the object 42a and the drawing range of the message 44 overlap in the viewfinder screen 30 like the object 42a and the message 44, the distance to the object 41 of the object 41a, As a result of the comparison with the distance obtained from the absolute position of the message 43, when the distance to the message 44 is closer, the message display module 18 causes the message 44 to be superimposed on the object 42a and displayed.

  In the example of FIG. 6, an example is given in which a part of the message 43 is hidden by the object 41a. For example, the drawing range of the message 43 hidden by the object 41a is displayed, for example, translucently or displayed in a different color. Thus, the content of the message may be known to the user to some extent.

[Flowchart for viewing messages]
FIG. 7 shows a flowchart when a message is displayed according to the positional relationship between the object in the imaging range and the message in the mobile phone terminal 2 of the present embodiment.

  In FIG. 7, when imaging by the camera module 11 is performed and a message browsing start instruction is input from the user through the key button module 20, the control unit 10 first activates the GPS module 13.

  Then, as a process of step S <b> 1, the GPS module 13 receives a radio wave from the GPS satellite 1 and sends the reception information to the position information calculation module 14. In step S1, the position information calculation module 14 uses the information from the GPS module 11 to calculate the absolute position (latitude, longitude, altitude) of the terminal in the real world at the current time, and calculates the calculated absolute The position information is stored in the memory 12.

  Next, as the process of step S2, the detailed information acquisition module 15 accesses the message server 4 via the communication module 16, and whether the message is registered within a range of radius R meters from the absolute position information of the own terminal, for example. Inquire whether or not. If the message is registered, the shooting range calculation module 15 advances the process to step S3. If the message is not registered, the imaging range calculation module 15 ends the process of the flowchart of FIG.

  When the processing proceeds to step S3, the detailed information acquisition module 15 obtains the posture of the own terminal at the present time in the real space from both detection signals of the gyro sensor and the acceleration sensor, and in that posture Based on the detection signal from the electronic compass and the absolute position of the terminal calculated by the position information calculation module 4, it is calculated which direction the lens of the camera module 11 is facing.

  Next, the detailed information acquisition module 15 uses the camera module 11 to execute the real space based on the orientation of the terminal in the real space obtained in step S3 and the orientation of the lens in that posture as the process of step S4. Which range of which direction is projected, that is, an imaging range is obtained.

  Next, in step S5, the detailed information acquisition module 15 inquires of the message server 4 accessed in the previous step S2 whether or not the message is registered within the imaging range. The imaging range calculation module 15 proceeds to step S6 when a message is registered within the imaging range, and proceeds to step S12 when a message is not registered within the imaging range.

  When the processing proceeds to step S6, the detailed information acquisition module 15 responds to the message server 4 accessed in the previous step S2 with the data of all messages registered within the imaging range and their respective messages. Get the attached absolute position information.

  Next, the detailed information acquisition module 15 performs the processing of step S7 from the distance of the object of each target existing in the imaging range of the real space and the distance obtained from the absolute position information of the message in the imaging range. Then, the positional relationship between each object (object) and the message is compared.

  Further, the detailed information acquisition module 15 determines whether or not an object exists before the message as a process in step S8 from the positional relationship comparison result in step S7. If the detailed information acquisition module 15 determines that an object exists before the message, the detailed information acquisition module 15 notifies the message display module 18 of the fact and passes the processing to step S9 of the message display module 18. On the other hand, if it is determined that the object does not exist before the message, the detailed information acquisition module 15 notifies the message display module 18 to that effect, and passes the processing to step S11 of the message display module 18.

  In step S11, the message display module 18 displays the message on the viewfinder screen. That is, in the process of step S11, the display range of the object 42a and the drawing range of the message 44 overlap in the viewfinder screen 30 as in the object 42a and the message 44 in FIG. However, when the distance of the message 44 is closer than the object 42a, the message display module 18 causes the message 44 to be superimposed on the object 42a and displayed. After the process of step S11, the control unit 10 of the present embodiment advances the process to step S12.

  On the other hand, when the processing proceeds to step S9, the message display module 18 compares the object display range with the message drawing range, and then proceeds to step S10.

  When the processing proceeds to step S10, the message display module 18 does not display the portion hidden in the object display range in the message drawing range, but displays only the drawing range outside the object display range. In other words, in the case of the processing of step S10, the message display module 18 does not display the message 43 for the portion hidden in the display range of the object 41a, such as the object 41a and the message 43 in FIG. Only the drawing range out of the display range of 41a is displayed. After the process of step S10, the control unit 10 advances the process to step S12.

  When the process proceeds to step S12, the control unit 10 determines whether or not a message browsing end instruction is input from the user through the key button module 20, and returns to step S3 when the end instruction is not input. On the other hand, when the end instruction is input, the processing of the flowchart of FIG. 7 is ended.

[Sequence when viewing messages]
FIG. 8 shows a sequence diagram of the system of the present embodiment when a message is displayed on the viewfinder screen of the mobile phone terminal 2.

  In FIG. 8, when an instruction for requesting message browsing is input from the user through the key button module 20 at an arbitrary place where GPS information can be acquired, the mobile phone terminal 2 performs message browsing mode in the process of step S <b> 21. Set to ON.

  When the message browsing mode is turned on in step S21, the cellular phone terminal 2 acquires current position information (latitude, longitude, altitude) of the terminal using radio waves from the GPS satellite 1 as processing in step S22. . Then, the cellular phone terminal 2 stores the position information in the memory 12 as the process of step S23.

  Next, the cellular phone terminal 2 transmits a message read request to the message server 4 via the communication module 16 as the process of step S24. The message read request at this time includes position information of the own terminal and information indicating the radius R meter that can be captured as an imaging range in the own terminal. Note that the radius R meter that can be captured by the mobile phone terminal as the imaging range may be managed by the message server 4 side. In this case, the message server 4 manages the value of the radius R meter that can capture the imaging range for each model of the mobile phone terminal.

  The message server 4 that has received the message read request performs a search for all records (registered messages) in the database 5 using the position information included in the message read request as a trigger as the process of step S31. Do. And message server 4 takes out all the messages matched with the position within radius R meter from the above-mentioned position information as processing of Step S32, and these messages and the absolute position information matched with the message, The message response is transmitted to the mobile phone terminal 2.

  The mobile phone terminal 2 that has received the message response stores the message and the absolute position information in the memory 12 as the process of step S25.

  Next, the cellular phone terminal 2 obtains the imaging range as described above as the process of step S26, and displays a message on the viewfinder screen of the imaging range as the process of step S27. In step S27, as described above, the message is displayed based on the comparison between the position and display range of the object displayed in the viewfinder screen, the absolute position of the message and the drawing range.

  Thereafter, when an instruction for requesting the end of browsing of the message is input from the user through the key button module 20, the cellular phone terminal 2 turns off the message browsing mode as the process of step S28.

[Example of display image at message input]
In FIG. 9, a display example of a captured image (viewfinder screen 30) of the camera module 11 when a message is input from the user through the key button module 20 and absolute position information associated with the message is set. Is shown. In FIG. 9, as in the example of FIG. 3 described above, a part of the real space landscape 31 where the house 31 a and the tree 31 b exist is an imaging range of the camera module 11, and the imaging is performed. A display example when the user inputs a desired message 46 in the viewfinder screen 30 displaying the range is shown.

  As in the example of FIG. 9, in the mobile phone terminal of the present embodiment, when a message is input from the user, the detailed information acquisition module 15 includes a gyro sensor, an acceleration sensor, Based on each detection signal from the compass and the absolute position of the terminal calculated by the position information calculation module 14, a detailed imaging range of the real space currently being imaged by the camera module 11 is obtained.

  At this time, the message input module 19 displays the input message 46 at, for example, the center position in the viewfinder screen 30 displaying the imaging range of the real space.

  Here, it is assumed that the mobile phone terminal is moved in a direction indicated by an arrow 33 in the figure when the real space is being imaged by the camera module 11 as shown in FIG.

  In this case, the detailed information acquisition module 15 knows from the detection signals from the gyro sensor, acceleration sensor, and electronic compass that the mobile phone terminal has been moved in the direction of arrow 33 in the figure. Then, the detailed information acquisition module 15 at this time uses the detection signals from the gyro sensor, the acceleration sensor, and the electronic compass, and the absolute position calculated by the position information calculation module 14 in the direction of the arrow 33. The detailed imaging range of the real space which the camera module 11 is imaging at the time after the movement is obtained.

  FIG. 10 shows the viewfinder screen 30 and the real space landscape 31 after the mobile phone terminal is moved in the direction of the arrow 33 in FIG. In this case, the message input module 19 of the mobile phone terminal of this embodiment maintains the state where the input message 46 is displayed at the center position of the viewfinder screen 30.

  In the example of FIG. 10, since the imaging range has moved in the direction of the arrow 33 from the example of FIG. 9, a part of the house 31 a is removed from the viewfinder screen 30 in the landscape 30 in the real space, while the message 46 is A state in which the image is displayed at the center position of the viewfinder screen 30 is shown.

  Also in this example, as described above, the imaging range calculation module 15 performs the sensing operation of the gyro sensor, the acceleration sensor, and the electronic compass at a sufficiently short time interval that can follow the movement of the mobile phone terminal. Further, the message input module 19 updates the display position of the message 46 in accordance with the update of the shooting range in the shooting range calculation module 15 so that the message 46 is positioned at the center of the viewfinder screen 30. To do.

  In this embodiment, the fine information acquisition module 15 sets the absolute position in the real space corresponding to the center position of the viewfinder screen 30 on which the message 46 is displayed as the registered position of the message 46.

  Here, the absolute position of the message 46 in the depth direction, that is, the distance from the current position of the terminal itself to the message registration position, for example, is the same as the distance measurement example for each object in the real space at the time of browsing the message. It can be obtained using the autofocus function of the camera module 11. That is, for example, by measuring the distance to the object in the real space corresponding to the object displayed at the center of the viewfinder screen 30, the absolute position of the message 46 in the depth direction can be obtained.

  In addition, the absolute position (distance) in the depth direction of the input message 46 can be determined from the distance to the target object other than the center position of the viewfinder screen 30. For example, it is possible to select a desired object within the imaging range, measure the distance to the object, and set the measured distance as the absolute position of the message 46 in the depth direction. In this case, for example, a desired object is selected by displaying a distance measurement mark on the viewfinder screen 30 and moving the distance measurement mark onto the object of the desired object by operating the key button module 20 or the like. It can be done by things. Then, when a desired object is instructed by the distance measurement mark, for example, when a distance measurement start instruction is input from the key button module 20, the camera module 11 measures the distance to the object by the autofocus function. To do.

  In the example of FIGS. 9 and 10 described above, an example in which the message size (drawing range) is made constant is given. However, the message size can be arbitrarily set by, for example, a size designation input through the key button module 20 or the like. It is also possible to change the size.

  Further, the size of the message may be in accordance with the absolute position (distance) in the depth direction of the message. That is, in this case, for example, the size is increased when the distance to the message is short, and conversely, when the distance is long, the size is decreased.

  After the message is input as described above and the registration position of the message is specified, for example, when a determination instruction is input from the user through the key button module 20, the control unit 10 of the mobile phone terminal 2 of the present embodiment The data of the input message and the absolute position information of the message (that is, information indicating which distance the message is away from the current position of the terminal itself) are determined.

[Flowchart for message input]
FIG. 11 shows a flowchart at the time of message input in the mobile phone terminal 2 of the present embodiment.

  In FIG. 11, when imaging is performed by the camera module 11 and a message input start instruction is input from the user through the key button module 20, the control unit 10 first activates the GPS module 13 and starts from the GPS satellite 1. Start receiving radio waves.

  At this time, the position information calculation module 14 determines whether or not the GPS module 13 can receive the radio wave from the GPS satellite 1 and can acquire the current position of the terminal itself as the process of step S61. If the current position of the terminal itself can be acquired, the control unit 10 stores the information in the memory 12 and proceeds to step S62. On the other hand, if the current position cannot be acquired, the control unit 10 in the flowchart of FIG. The process ends.

  When the processing proceeds to step S62, the message input module 19 can accept a message input from the user through the key button module 20. When a message is input from the user through the key button module 20, the message data is stored in the memory 12.

  Next, the detailed information acquisition module 15 obtains the imaging range in the same manner as described above as the process of step S63, and further obtains the absolute position (distance) in the depth direction of the message as the process of step S64.

  Then, as the process of step S65, the control unit 10 transmits the input message data and the absolute position information of the message from the communication module 16 to the message server 4 and causes the message server 4 to register the message data.

  Thereafter, the control unit 10 determines whether or not a message input end instruction is input from the user through the key button module 20 as the process of step S66, and returns to step S62 when the end instruction is not input. On the other hand, when the end instruction is input, the processing of the flowchart of FIG. 11 is ended.

[Sequence for message input]
FIG. 12 shows a sequence diagram of the system of this embodiment when inputting a message and registering the message.

  In FIG. 12, when an instruction for requesting message input is input from the user through the key button module 20, the cellular phone terminal 2 turns on the message input mode in the process of step S41.

  When the message input mode is turned on in step S41, the cellular phone terminal 2 acquires the current position information (latitude, longitude, altitude) of the terminal using the radio wave from the GPS satellite 1 as the process of step S42. . Then, the cellular phone terminal 2 stores the position information in the memory 12 as the process of step S43.

  Next, when a message is input from the user through the key button module 20 as a process of step S44, the mobile phone terminal 2 further obtains an imaging range as a process of step S44, and then processes the message as a process of step S46. Obtain absolute position information.

  Thereafter, the mobile phone terminal 2 transmits a message record request to the message server 4 via the communication module 16. The message record request at this time includes message data and absolute position information of the message.

  The message server 4 that has received the message record request registers the message included in the message record request and its absolute position information in the database 5 as the process of step S51. When the registration is completed, the message server 4 transmits a registration completion response to the mobile phone terminal 2 as a process of step S52.

  After receiving the registration completion response, when an instruction for requesting the end of viewing of the message is input from the user through the key button module 20, the cellular phone terminal 2 turns off the message input mode as a process of step S48. .

[Limited viewing of messages]
In the message management system of this embodiment, the message server 4 can read a message (that is, can read the message) by user authentication using the telephone number or mail address of the mobile phone terminal that has transmitted the message read request. It is also possible to limit terminals).

  In this way, by limiting the terminals that can read messages (hereinafter referred to as browsing targets), for example, personal messages can be prevented from being browsed by an unspecified number of people, Only the other party can view the message.

  That is, in this case, information can be transmitted to a specific browsing target, for example, a message that can be browsed only by a person registered in the address book of the mobile phone terminal is registered.

  In the case of this example, it is also possible to display the name of the person who registered the message as well as the message.

  Note that this is especially possible with mobile phone terminals that have an address book and manage personal information and other people's information, and do not manage personal information such as so-called digital cameras. This is not possible with this terminal.

  FIG. 13 shows a flowchart in the case of inputting a message in which terminals that can read the message are limited. In the flowchart of FIG. 13, the same steps as those in FIG. 11 are denoted by the same reference numerals, and the description thereof is omitted.

  In FIG. 13, the control unit 10 advances the process to step S67 after the process of step S64. In step S67, when a user inputs a telephone number, an e-mail address, or the like for limiting the message viewing target through the key button module 20, the control unit 10 advances the process to step S65.

  In the case of step S65 of FIG. 13, the control unit 10 transmits the input message data, the absolute position information of the message, and the information for limiting the browsing target from the communication module 16 to the message server 4, and the message The server 4 is registered.

  FIG. 14 shows a flowchart in the message server 4 when a message whose browsing target is limited is browsed.

  In FIG. 14, when the message server 4 receives the message read request, as a process in step S71, the message server 4 performs a search for all records in the database 5 based on the position information included in the request, and the position information All messages associated with positions within a radius of R meters are extracted.

  Next, the message server 4 discriminates a public message whose browsing target is not limited and a message whose browsing target is limited from the messages of all records by the search.

  For the public message, the message server 4 transmits the message to the requesting mobile phone terminal as the process of step S76, and then ends the process of the flowchart of FIG. On the other hand, for a message whose browsing target is limited, the process proceeds to step S73.

  When the processing proceeds to step S73, the message server 4 performs an authentication process to determine whether the request source terminal is a terminal to be browsed. Then, when the terminal is the browsing target terminal as the process of step S74, the message server 4 advances the process to step S75 and transmits the message, and then ends the process of the flowchart of FIG. On the other hand, if it is not the browsing target terminal, the message server 4 ends the processing of the flowchart of FIG. 14 without transmitting the message.

  FIG. 15 shows a flowchart in the mobile phone terminal 2 that requests browsing of a message whose browsing target is limited. In the flowchart of FIG. 15, the same steps as those in FIG. 7 are denoted by the same reference numerals, and the description thereof is omitted.

  In FIG. 15, the control unit 10 of the mobile phone terminal 2 advances the process to step S13 after the process of step S5.

  When the process proceeds to step S13, the control unit 10 displays the telephone number or mail address of the terminal itself stored in the address book or the like of the memory 12 and the telephone number or mail address for specifying the browsing target together with the message read request. The message is sent to the message server 4.

  Thereafter, in step S6, all messages sent from the message server 4 are acquired.

  FIG. 16 shows a sequence diagram of the system of the present embodiment when inputting and registering a message whose browsing target is limited. In FIG. 16, the same steps as those in FIG. 12 are denoted by the same reference numerals, and the description thereof is omitted.

  In FIG. 16, after the process of step S46, the cellular phone terminal 2 advances the process to step S50.

  In step S50, when a user inputs a telephone number, an e-mail address, or the like for limiting a message viewing target through the key button module 20, the mobile phone terminal 2 advances the process to step S47.

  In the case of step S47 in FIG. 16, the mobile phone terminal 2 sends a message record request having the data of the input message, the absolute position information of the message, and information for limiting the browsing object from the communication module 16 to the message server 4. Send to.

  Accordingly, in step S51, the message server 4 registers the message included in the message record request, its absolute position information, and information for limiting the browsing target in the database 5.

[Message display control to avoid moving objects]
When the mobile phone terminal 2 of the present embodiment displays a message on the viewfinder screen 30 of the display module 18, it is possible to solve the problem that the message becomes difficult to see due to, for example, a passing person or a car in the real space. Yes. Specifically, the mobile phone terminal 2 of the present embodiment controls the movement of the display position of the message so that it is most visible from the user who is viewing the viewfinder screen 30.

  FIG. 17 shows a flowchart for controlling the display position of a message while avoiding an object moving on the viewfinder screen 30. In the flowchart of FIG. 17, the same steps as those in FIG. 7 are denoted by the same reference numerals, and the description thereof is omitted.

  In the flowchart of FIG. 17, when the detailed information acquisition module 15 determines in step S8 that no object exists before the message, the detailed information acquisition module 15 proceeds to step S14.

  When the processing proceeds to step S14, the detailed information acquisition module 15 determines whether or not a moving object representing a person or a car has moved from another place before the message. Note that whether or not an object is a moving object can be determined by detecting a motion vector of an image as an example. That is, the detailed information acquisition module 15 finds a moving object by detecting a motion vector of an image displayed in the viewfinder screen 30. Further, the detailed information acquisition module 18 determines whether or not the object detected as the moving object moves before the message from, for example, the comparison result of the positional relationship in step S7. That is, in this example, in step S7, for each object in the viewfinder screen 30, the positional relationship of all the objects is obtained regardless of whether or not there is movement. The control unit 10 may perform the detection of the motion vector.

  If it is determined in step S14 that a moving object does not come, the detailed information acquisition module 15 advances the process to step S11. As a result, the message display module 18 displays a message as described above. On the other hand, if it is determined that a moving object is coming, the detailed information acquisition module 15 advances the process to step S15 and passes the process to the message display module 18.

  In step S15, when the moving object moves on the message, the message display module 18 displays the message at a position to avoid the moving object. For example, when a moving object such as a person or a vehicle comes, the message display module 18 changes the display position of the message to the upper side of the moving object, or an object that does not move, such as a building in the vicinity of the position where the message should be displayed. The display position of the message is changed near the wall or ground object. After the process of step S15, the message display module 18 advances the process to step S12.

  In the flowchart of FIG. 17, the process proceeds to step S16 after step S9. When the processing proceeds to step S16, the detailed information acquisition module 15 determines whether or not a moving object representing a person, a car, or the like moves from another place before the message, as in step S14. If it is determined in step S16 that a moving object does not come, the detailed information acquisition module 15 advances the process to step S10. As a result, the message display module 18 displays a message as described above. On the other hand, if it is determined in step S16 that a moving object is coming, the message display module 18 executes the process of step S17.

  In the processing of step S17, the message display module 18 does not display the portion hidden in the display range of the object (fixed object that does not move) in the drawing range of the message, as in step S10, and deviates from the display range of the fixed object. Displays only the drawn area. On the other hand, when the moving object moves before the message in which only the drawing range out of the display range of the fixed object is displayed, the message display module 18 sends the message to the moving object as in step S15. Display at a position to avoid. After the process of step S15, the message display module 18 advances the process to step S12.

[Summary]
As described above, according to the embodiment of the present invention, when the object display range and the message drawing range overlap in the viewfinder screen 30, the message is closer in distance than the object target. The message is not displayed for the portion hidden in the display range of the object, and only the drawing range outside the display range of the object is displayed. Thus, according to the embodiment of the present invention, even when a situation change occurs such as a new building being built in the real world after the message is registered, the situation change in the real world is appropriately performed. Corresponding message display is possible.

  The above description of the embodiment of the present invention is an example of the present invention. For this reason, the present invention is not limited to the above-described embodiment, and it is needless to say that various modifications can be made according to the design or the like as long as the technical idea according to the present invention is not deviated.

  1 GPS satellite, 2 mobile phone terminal, 3 packet network, 4 message server, 5 message database, 10 control unit, 11 camera module, 12 memory, 13 GPS module, 14 position information calculation module, 15 detailed information acquisition module, 16 communication Module, 17 Display module, 18 Message display module, 19 Message input module, 20 Key button module, 23 Imaging range, 30 Viewfinder screen, 31 Landscape, 31a House, 31b Tree, 35, 36, 43, 44, 45, 46 Message, 41, 42 Object, 41a, 42a Object

Claims (8)

An imaging unit that images the real world with an imaging lens and an imaging element;
A video display unit for displaying video;
A communication unit that communicates with a message server that manages a plurality of messages in association with absolute positions in the real world, and
A position detector that detects the current absolute position of the terminal;
An imaging range calculation unit that calculates an imaging range captured by the imaging unit in the real world;
A distance measuring unit that measures the distance from the absolute position of the terminal itself to each object existing in the real space within the imaging range;
A message information acquisition unit for connecting to the message server through the communication unit, and acquiring, from the message server, message data managed in association with the absolute position within the imaging range and the absolute position information;
The real-time captured image within the imaging range captured by the imaging unit is displayed on the video display unit, the distances to the objects measured by the distance measuring unit, and the message information acquisition unit Based on the absolute position information of the acquired message, the display range of each object in the captured image, and the rendering range when the message is superimposed on the captured image, the target in the captured image The display range of the object and the drawing range of the message, and the overlap of the display range of the object and the drawing range of the message are determined. A message display control unit that draws only a message in a drawing range that does not overlap with the above-described captured image,
A portable information terminal. The imaging range calculation unit
A gyro sensor that detects the amount and direction of rotation when a rotational motion is applied to the terminal, and the inclination of the terminal due to the rotation;
An acceleration sensor for detecting a movement acceleration when the acceleration motion is applied to the terminal, a movement amount thereof, and a movement direction thereof;
An electronic compass for detecting an orientation in which the imaging lens of the imaging unit faces,
Based on the detection signals of the gyro sensor and the acceleration sensor, the orientation of the terminal at present is obtained in the real space, and the orientation in which the imaging lens of the imaging unit faces is obtained from the detection signal obtained from the electronic compass. The portable information terminal according to claim 1, wherein an imaging range in the real world that is imaged by the imaging unit is calculated based on an attitude of the terminal in the camera and an orientation in which the imaging lens faces in the attitude.   The portable information terminal according to claim 2, wherein the imaging range calculation unit sets the imaging range within a certain distance from the focal point of the imaging lens of the imaging unit.   The distance measuring unit determines, as a group of objects, a portion including a set of points in which the focus of the imaging lens of the imaging unit is approximately the same distance in the real space within the imaging range, The portable information terminal according to claim 1, wherein a distance is measured for each object.   The distance measurement unit uses the absolute position before and after the movement when the terminal is moved and the change in orientation when the same object is imaged by the imaging unit before and after the movement. The portable information terminal according to claim 1, wherein the distance to is calculated.   The portable information terminal according to claim 1, wherein the distance measuring unit measures a distance to the object based on a reflected wave arrival time when an ultrasonic wave is emitted into the real space of the imaging range. Imaging the real world with an imaging unit having an imaging lens and an imaging element;
A step in which the position detection unit detects the current absolute position of the terminal;
An imaging range calculating unit calculating an imaging range captured by the imaging unit in the real world;
A distance measuring unit measuring a distance from the absolute position of the terminal itself to each object existing in the real space within the imaging range;
The message information acquisition unit connects to a message server that manages a plurality of messages in association with the absolute positions in the real world through the communication unit, and manages the messages in association with the absolute positions within the imaging range. Obtaining the message data and its absolute position information,
The message display control unit causes the video display unit to display a real-world captured image within the imaging range captured by the imaging unit, and the distance to each object measured by the distance measurement unit, Based on the absolute position information of the message acquired by the message information acquisition unit, the display range of each object in the captured video, and the rendering range when rendering the message superimposed on the captured video, the imaging A front-to-back relationship between the display range of the object and the drawing range of the message in the video, and an overlap between the display range of the target object and the drawing range of the message, Drawing only the drawing range message that does not overlap the display range of the object on the captured image;
An information display control method comprising: An imaging control process for imaging the real world to an imaging means having an imaging lens and an imaging element;
A position detection process for causing the position detection unit to detect the current absolute position of the terminal;
An imaging range calculation step of calculating an imaging range captured by the imaging unit in the real world;
A distance measurement control process that causes the distance measurement means to measure the distance from the absolute position of the terminal itself to each object existing in the real space within the imaging range;
Data of messages managed by associating and managing a plurality of messages in association with the absolute position in the imaging range by connecting to a message server managing each of the messages in correspondence with absolute positions in the real world through the communication unit. And message information acquisition processing for acquiring the absolute position information,
The real-world captured image within the imaging range captured by the imaging unit is displayed on the video display unit, and the distance to each object measured by the distance measuring unit and the message information acquisition process are acquired. The object in the captured video is based on the absolute position information of the message, the display range of each target in the captured video, and the rendering range when the message is superimposed on the captured video. The display range of the message and the drawing range of the message, and the overlap of the display range of the object and the drawing range of the message are determined. A message display control process for superimposing and drawing only a message in a drawing range that does not overlap on the captured image;
Display control program for causing a computer to execute.

Images