JP2016066146A - Display system, display method, computer program, and computer-readable storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a display system capable of providing more contents to a user.SOLUTION: A display system 1 includes a server 51 including a storage part 52 which stores additional information data A including contents and a portable terminal 21 including an imaging part 24 which generates the image data of an object and a display part 25 which displays the image data. The portable terminal 21 acquires an additional information file F including the additional information data A related to the object from the server 51. The display system 1 allows the display part 25 to display the contents of the additional information data A. The display system 1 includes transition information for the transition of the additional information data A to other contents.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a display system for displaying content in an overlapping manner in a real environment, a display method, a computer program that causes a computer to execute steps included in the display method, and a computer-readable storage medium that stores the computer program.

  In recent years, augmented reality (AR) technology has been developed in which the real environment perceived by humans is augmented by computers. This augmented reality technology is a technology that literally expands the real society seen by humans by adding and emphasizing information such as content to the real environment.

  This augmented reality technology can be embodied on a portable terminal by improving the functions of the portable terminal. For example, the augmented reality technique can be realized by installing it on a portable terminal as an AR application (hereinafter referred to as an AR application). it can.

  The AR application captures a subject with the camera of the mobile terminal, acquires position information with a GPS (Global Positioning System) of the mobile terminal, and specifies the subject from the position information. For example, Patent Document 1 proposes a technique for registering content and position information in association with each other.

  The AR application acquires contents such as character strings, images, sounds, moving pictures, and CG (computer graphics) related to the subject from the server, and displays the contents together with the subject on the display screen of the portable terminal. ing.

  As described above, the user can acquire information expanded about the subject through the content through the display screen of the mobile terminal.

JP 2010-026546 A

  However, if the current augmented reality technology is to be embodied on a mobile terminal, the augmented reality technology has a configuration in which a subject and content are superimposed on the display screen of the mobile terminal, and the amount of content that can be displayed is portable. Depending on the size of the display screen of the terminal, there was a possibility that not all contents intended by the creator could be displayed.

  SUMMARY An advantage of some aspects of the invention is that it provides a display system that can provide more content to a user.

  A display system according to the present invention includes a server including a storage unit that stores additional information including content, an imaging unit that generates image data of an object, and a display unit that displays the image data. A display system in which the terminal acquires additional information related to the object from the server and displays the content of the additional information on the display unit, and the additional information transitions to another content. It is characterized by including transition information.

  A plurality of the additional information is associated with one object and stored in the storage unit. The additional information includes transition information for transitioning at least one or more of the plurality of additional information to another content. It is characterized by being.

  The transition information includes an additional transition for newly displaying other content on the display unit, an update transition for displaying the other content while hiding the displayed content, and an external web server from the displayed content. It is characterized by including an external transition for transition to a web page.

  The server further includes a real-time data acquisition unit that acquires real-time data, and a real-time content generation unit that generates real-time content based on the real-time data acquired by the real-time data acquisition unit, The content displayed on the display unit includes the real-time content generated by the real-time content generation unit.

  The terminal includes an additional information requesting unit that requests the server for additional information related to the object based on the image data of the object, and the storage unit includes at least one object for one object. The additional information file in which the additional information is described in association with the image and the image data of the object associated with the additional information file stored in advance as image data obtained by photographing the target object in advance is stored. An additional information file acquisition unit that searches for image data of an object that matches the image data of the object from the unit, acquires an additional information file associated with the image data of the object, and transmits the additional information file to the terminal It is characterized by that.

  The storage unit of the server stores the position data of the object in association with the image data of the object, the terminal further includes a position information acquisition unit that acquires the position data of the object, and the additional information request unit includes The position data of the object is also transmitted to the server, and the additional information file acquisition unit of the server obtains the image data of the object from the storage unit based on the image data of the object and the position data of the object. It is characterized by searching.

  The additional information file is created for each region, and the additional information file acquisition unit stores the object information from the storage unit based on the image data of the object or the image data of the object and the position data of the object. Image data is retrieved and acquired, and the additional information file is acquired based on image data and position data of the acquired object.

  The additional information file is created for each time period, and the server further includes a clock unit that provides time, and the additional information file acquisition unit is stored in the storage unit based on the image data of the object. The image data is retrieved and acquired, and the additional information file is acquired based on the image data and time of the acquired object.

  The display method of the present invention includes a step of storing additional information including content and additional information including transition information to other content in a storage unit, a step of generating image data of a target, and the target The method includes a step of acquiring additional information related to the object based on image data of the object, and a step of displaying a content of the additional information on a display unit.

  The computer program of the present invention is characterized by causing a computer to execute the steps included in the display method described above.

  A computer-readable storage medium stores the computer program described above.

  According to the display system of the present invention, more content can be provided to the user.

It is a schematic block diagram of a display system. It is the schematic diagram which showed the additional information file. (A) shows an object image table, and (B) shows an additional information file table. It is a flowchart figure which shows the process of a positional information acquisition part. It is a flowchart figure which shows the process of an additional information request | requirement part. It is a flowchart figure which shows the process of an additional information file acquisition part. It is a flowchart figure which shows the process of a display process part. It is a flowchart figure which shows the process of an input process part. It is the figure which showed a mode that the guidance display board of a station was displayed on the display part of a portable terminal. It is a figure which shows a mode that the relevant content was superimposed and displayed on the guidance display board of the station displayed on the touch panel. It is a partial enlarged view of the portable terminal which shows a mode that one content was selected and all the transitions to the other content group regarding the content were performed. It is a partial enlarged view of the portable terminal which shows a mode that one content was selected and the update transition was performed to the other content group regarding the content. It is a partial enlarged view of the portable terminal which shows a mode that one content was selected and an additional transition was made to other content related to the content. It is the whole portable terminal figure which shows a mode that the one content was selected and it externally changed to the web page. It is the schematic which showed the exchange of the data of a portable terminal and a server. It is the figure which showed a mode that the book which opened the page is displayed on the touch panel of a portable terminal. It is a figure which shows a mode that the relevant content was superimposed and displayed with respect to the book displayed on the touch panel. It is a figure which shows a mode that the image content of "Tower" was tapped and the moving image content of "Tower" was reproduced | regenerated and completed. It is a figure which shows a mode that the one content was selected and the other content group regarding the content was displayed.

  Hereinafter, an example of a mode for carrying out the present invention (hereinafter referred to as an example) will be described with reference to FIG. FIG. 1 is a schematic configuration diagram of the display system 1. As shown in FIG. 1, the display system 1 includes a server 51 and a mobile terminal 21 that performs data communication with the server 51.

(Mobile device)
First, the mobile terminal 21 (terminal) will be described. The mobile terminal 21 is, for example, a touch panel tablet. Here, the portable terminal 21 is not limited to a touch panel type tablet, and may be a wearable terminal such as a smart watch, a smart glass, a smart band, a smart contact lens, or a smart wig worn by a user, and further, a camera function. It may be a terminal device such as a notebook computer, desktop computer, or digital signage.

  For a device that does not have a camera function, a terminal of the present invention can be obtained by attaching an external camera such as a web camera later. Moreover, the portable terminal 21 and the terminal are not limited to devices having a camera function from the beginning, but also include devices to which an external camera such as a web camera can be attached later.

  The portable terminal 21 operates as a storage unit 22 that operates as a storage function, a control unit 23 that operates as a control function, an imaging unit 24 that operates as a camera function, a display unit 25 that operates as a display function, and an input function. It includes at least a touch panel 27 having an input unit 26, a GPS unit 28 that operates as a GPS function, a clock unit 30 that functions as a clock function, and a wireless communication unit 31 that operates as a communication function. Hereinafter, each functional unit will be described in detail.

  The storage unit 22 is configured by combining a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an HDD (Hard Disk Drive), and the like. The storage unit 22 stores various programs including an OS (Operating System) that provides basic functions of the mobile terminal 21.

  The storage unit 22 stores position data and state data. The position data is data indicating the position of the portable terminal 21 acquired by the GPS unit 28. When the GPS unit 28 acquires the position data, the position data is stored together with the time at that time acquired by the clock unit 30.

The status data refers to two modes, a recognition mode and a tracking mode. The recognition mode refers to a mode for recognizing an object for displaying content. The tracking mode refers to a mode for tracking an object on which content is displayed.

  The imaging unit 24 is a so-called camera and generates image data of a subject from an image formed by a lens. The subject image data generated by the imaging unit 24 is stored in the storage unit 22 as necessary.

  The display system 1 of the present invention recognizes an object whose content is to be displayed by using image data generated by the imaging unit 24 when the state data in the storage unit 22 is in the recognition mode. After the state data is changed to the recognition mode, the image data of the object cannot be generated by the imaging unit 24, that is, when the object is out of frame, the control unit 23 changes the state data to the recognition mode.

  The touch panel 27 includes a display unit 25 having a liquid crystal panel and an input unit 26 having a touch pad. The mobile terminal 21 of the present embodiment embodies the functions of the display unit 25 and the input unit 26 using the touch panel 27. The display unit 25 and the input unit 26 are not limited to the touch panel 27. For example, the display unit 25 and the input unit 26 may be divided into liquid crystal and touch keys.

  On the display unit 25 of the touch panel 27, the image data generated by the imaging unit 24 and the content are displayed together with the object. Operation information is input to the input unit 26 of the touch panel 27 from the user of the portable terminal 21.

  Here, the operation information is, for example, a tap, double tap, long tap, flick (up, down, left, right flick), swipe, pinch (pinch in, pinch out), multi tap, etc. This is information about whether or not

  The wireless communication unit 31 (communication unit) enables data communication between the portable terminal 21 and the server 51 by network communication. Network communication is communication based on wireless communication standards such as 3G (3rd Generation), LTE (Long Term Evolution), 4G (4th Generation), WiFi (registered trademark), and Bluetooth (registered trademark).

  Here, the wireless communication part 31 should just have as a function the transmission speed which can transmit / receive image data, and is not limited to what was illustrated. Instead of the wireless communication unit 31, a wired communication unit that performs data communication with the server 51 by wire, and a configuration including both the wireless communication unit 31 and the wired communication unit may be employed. Further, when the function of the display system 1 is realized by the mobile terminal 21, the wireless communication unit 31 is omitted.

  The GPS unit 28 acquires the position information of the mobile terminal 21 using a global positioning system. Specifically, the GPS unit 28 includes a GPS receiver, receives a GPS signal transmitted from a GPS satellite, and calculates and acquires current position data of the mobile terminal 21 based on the GPS signal. When the GPS unit 28 acquires the current position data of the mobile terminal 21, the GPS unit 28 acquires the time from the clock unit 30 and stores it in the storage unit 22 together with the time.

  The clock unit 30 provides time. Here, the mobile terminal 21 can perform network communication with the server 51 by the wireless communication unit 31, and the time of the clock unit 30 is periodically synchronized with the time of the server 51.

  The control unit 23 includes a CPU (Central Processing Unit), and controls the storage unit 22, the imaging unit 24, the touch panel 27, the GPS unit 28, the clock unit 30, and the wireless communication unit 31 described above. In addition, the control unit 23 executes a position information acquisition unit 35, an additional information request unit 36, a display processing unit 37, and an input processing unit 38.

  These position information acquisition unit 35, additional information request unit 36, display processing unit 37, and input processing unit 38 are installed as an AR application in the storage unit 22 of the mobile terminal 21 as an application on the client side of the display system 1, for example. .

  The position information acquisition unit 35 uses the GPS unit 28 to calculate current position data, and uses the current position data as position data of the mobile terminal 21 or position data of an object on which content is displayed.

  Here, when the mobile terminal 21 is located in a building or a tunnel, the GPS unit 28 blocks a GPS signal from a GPS satellite and cannot receive the GPS signal. That is, the position information acquisition unit 35 may not acquire position data even if it tries to acquire position data.

  In this case, the position information acquisition unit 35 confirms the presence of the position data stored in the storage unit 22 within a predetermined time, and if the position data stored within the predetermined time exists, the position information is stored in the mobile terminal. 21 position data or the position data of an object on which content is displayed. Here, the predetermined time is set to 5 minutes as an initial set value, for example. Note that the administrator can set an arbitrary numerical value for the predetermined time.

  On the other hand, when the GPS signal from the GPS satellite cannot be received and there is no position data stored in the storage unit 22 within a predetermined time, the position information acquisition unit 35 performs processing without acquiring the position data. Exit.

  As described above, the position information acquisition unit 35 may not be able to acquire the position data of the mobile terminal 21, and in that case, the subsequent processing is performed without using the position data. Therefore, in the configuration of the mobile terminal 21, the position information acquisition unit 35 is not an essential configuration and may be omitted. Similarly, the GPS unit 28 and the clock unit 30 used in the position information acquisition unit 35 may be omitted from the configuration of the mobile terminal 21.

  When the status data in the storage unit 22 is in the recognition mode, the additional information request unit 36 transmits the additional information request data D to the server, and sends the additional information file F associated with the object for displaying the content to the mobile terminal 21. Request to send.

  Here, the additional information request data D refers to data including image data and position data of an object for displaying content, or image data of an object for displaying content. The additional information request data D is also used for recognizing an object for displaying content.

  When the position information acquisition unit 35 described above can acquire the position data, the additional information request unit 36 adds the additional information based on the position data and the image data of the object on which the content generated by the imaging unit 24 is displayed. Request data D is generated and transmitted to the server 51.

  On the other hand, when the position information acquisition unit 35 cannot acquire the position data, the additional information request unit 36 generates the additional information request data D based on the image data of the object on which the content generated by the imaging unit 24 is displayed. To the server 51.

  When the additional information request unit 36 transmits the additional information request data D to the server 51, the server 51 transmits the additional information file F to the mobile terminal 21. Hereinafter, the additional information file F will be described with reference to FIG. FIG. 2 is a schematic diagram showing the additional information file F.

  The additional information file F (additional information data set) is a file in which for each content to be displayed on the object, information on the storage position of the content on the server 51, its display, its operation, execution, etc. is described. is there. Here, for a certain content, data having the storage information, display information, operation information, and execution information of the content is referred to as additional information data A (additional information).

  The additional information file F is a file in which the additional information data A is described one or more times. Further, the additional information file F includes regional data and time zone data in addition to the additional information data A as necessary.

  The time zone data refers to data indicating a period from a certain time to a certain time in a day. The display system 1 of the present invention can change the content to be displayed according to the time zone by having the additional information file F have this time zone data.

  For example, morning from 4 am to 11 am, noon from 11 am to 18 pm, evening from 18 pm to 20 pm, night from 20 pm to 24 pm, and midnight at midnight Settings can be made from 4:00 to 4:00, and the contents can be changed according to these morning, noon, evening, night, and midnight.

  Regional data refers to land that is grouped together for a given reason, such as similar topography or the same climate. The display system 1 of the present invention can change the contents to be displayed depending on the area by having the additional information file F include the area data.

  For example, this regional data can divide Japan into 8 regions, Hokkaido, Tohoku, Kanto, Chubu, Kinki, China, Shikoku, Kyushu, prefectures, and municipalities, and change the contents accordingly. be able to.

  When one additional information file F is associated with one piece of image data, regional data and time zone data are not set. On the other hand, when time zone data and region data are set for an image of one object, a plurality of additional information files F are created and associated with each time zone and region.

  The content storage position, content display information, content operation information, and content execution information constituting the additional information data A will be described in detail below.

  The content storage position refers to a storage position in the server 51 of content data such as a character string, an image, a sound, a moving image, and CG. Here, the content data in the display system 1 of the present invention includes real-time content, which is also stored in the storage unit 52 of the server 51.

  Real-time content refers to character string data, image data, audio data, moving image data, CG data, etc. generated based on real-time data acquired from the server 51 or an external server. The real-time data refers to real-time data such as vacant seat information, weather data, exchange information, stock price information, operation information, news data, and the like.

  The content display information refers to the size, angle, display position, and transmission information of the content based on the object.

  The content operation information refers to information such as whether or not the content is automatically played when the content is a moving image, CG, or the like, and whether or not the content is repeatedly played when the playback ends.

  The content execution information refers to an operation that is executed when the content displayed on the display unit 25 is tapped, double-tapped, or the like. In this operation, the content includes character string, enlargement, reduction, transition in the image, and the content includes reproduction, stop, transition in audio, moving image, CG, and is set in association with the tap or double tap.

  In the additional information file F, a transition is set to execution information of at least one additional information data A among the additional information data A constituting the additional information file F. Transition among operations will be described. Transitions include add transition, update transition, all transition, external transition, and transition end. Hereinafter, it demonstrates in order.

  In the additional transition, for example, when a content is tapped, another content is newly displayed without changing the content. This additional transition has a storage position in the server 51 of the additional information data A of content to be newly displayed as transition information.

  In the update transition, when a content is tapped, the content or specific content is hidden from the display unit 25 and another content is newly displayed on the display unit 25. This update transition has a storage position in the server 51 of the additional information data A of the content to be hidden and the content to be newly displayed as the transition information.

  When all the contents are tapped, all contents displayed on the display unit 25 are hidden from the display unit 25, and another designated additional information file F is downloaded from the server 51. The content is displayed on the display unit 25 based on the newly downloaded additional information file F. This all transition has the storage position in the server 51 of the new additional information file F as transition information.

  Here, the display of new content includes using a part of an external application using an API (Application Programming Interface). Here, the API refers to, for example, Twitter API, Facebook API, and YouTube (registered trademark) API.

  In the external transition, when the content is tapped, the AR application on the mobile terminal 21 side of the display system 1 is terminated, and a web browser is activated to display a designated external URL (Uniform Resource Locator) web page. This external transition has external URL information as transition information.

  When the content is tapped, the specific content is hidden from the display unit 25 when the content is tapped, and the transition from one content to another content is terminated. This transition end has information on contents to be hidden as transition information.

  The display processing unit 37 displays the image generated by the imaging unit 24 on the display unit 25. When the display processing unit 37 receives the additional information file F from the server 51, the display processing unit 37 holds the additional information data A described in the additional information file F in the temporary storage area of the storage unit 22, and stores the content from the server 51. While downloading, the content is displayed on the display unit 25 based on the display information described in the additional information data A.

  Further, when the content of the additional information file F held in the temporary storage area of the storage unit 22 is updated by the input processing unit 38, the display processing unit 37 transmits the content related to the updated additional information data A from the server 51. While downloading, the content is displayed on the display unit 25 based on the display information described in the updated additional information data A.

  The input processing unit 38 processes operation information from the input unit 26 when the AR application is executed on the mobile terminal 21. For example, when the content is tapped, double-tapped, long-pressed, or the like in the input unit 26, the input processing unit 38, based on the contents described in the additional information file F held in the temporary storage area of the storage unit 22, The action associated with is executed.

  Hereinafter, the input processing unit 38 will be described in detail. When the input processing unit 38 is activated, the input processing unit 38 waits for acquisition of operation information from the input unit 26. When the operation information is input from the input unit 26, the input processing unit 38 reads the additional information file F held in the temporary storage area when the state is the tracking mode and the operation information is displayed. It is determined whether or not the operation associated with the operation information is a transition.

  If the operation associated with the operation information is other than transition, for example, it is assumed that the operation is expansion. In that case, the content is enlarged and the operation information from the input unit 26 is again waited.

  Hereinafter, a case where the operation associated with the operation information relates to transition will be described. If the operation associated with the operation information is an additional transition, an update transition, or a transition end among the transitions, the contents of the additional information file F in the temporary storage area of the storage unit 22 are updated as such, and the input unit 26 is again updated. Waiting for operation information from.

  If all the operations associated with the operation information are transitions, the specified additional information file F is downloaded and the data of the additional information file F held in the temporary storage area of the storage unit 22 is downloaded. to erase.

  If the operation associated with the operation information is an external transition among the transitions, the web application is activated, the page of the designated URL is displayed, and the AR application is terminated.

(server)
Next, the server 51 will be described. The server 51 includes a storage unit 52 that operates as a storage function, a control unit 53 that operates as a control function, a clock unit 60 that operates as a clock function, and a wireless communication unit 61 that operates as a communication function.

  The storage unit 52 of the server 51 stores various programs including a server OS that provides basic functions of the server 51. In addition, the storage unit 52 of the server 51 stores in advance a countless number of image data such as a guide display board of a station existing in the real world, each page of a published book, and a building as image data of an object. In addition, character string data, image data, audio data, moving image data, and CG data are also stored as content data.

  Further, the additional information file F described above is stored in the storage unit 52 of the server 51 in association with the image data of the object. When there is no position data and time zone data, this additional information file F is uniquely associated with the object image data, and when it has position data and time zone data, a plurality of this additional information file F is associated with the object image data. Yes.

  In the additional information file F, transition is set to execution information in at least one additional information data A among the additional information constituting the additional information file F. As a result, one additional information data A is linked to the other additional information data A to form a virtual network inside the storage unit 52 of the server 51.

  In addition, the storage unit 52 of the server 51 includes a database 62. The database 62 has two tables, an object image table 62A and an additional information file table 62B.

  The two tables will be described with reference to FIG. FIG. 3A shows an object image table 62A, and FIG. 3B shows an additional information file table 62B. As shown in FIG. 3A, the object image table 62A has two attributes: storage position data of object image data and object position data.

  Here, the product image data refers to pre-captured image data such as a station information display board and a restaurant menu. The image data of an object includes image data obtained by photographing a part of the object in addition to image data obtained by photographing the entire object in advance. The image data obtained by capturing a part of the image is covered by a plurality of images. The image data of these objects is stored in advance in the storage unit 52, and the storage position data is stored in the image table 62A of the objects.

  Further, the position data of an object is position data indicating the actual installation location of the object in the real world by longitude, latitude, and altitude. The object position data is stored in the object image table 62A in association with the storage position data in the object image data.

  As shown in FIG. 3B, the additional information file table 62B has two attributes: storage position data of the image data of the object and storage position data of the additional information file F of the object. The storage position data of the object image data is stored in the additional information file table 62B in association with the storage position data of the additional information file F of the object.

  The clock unit 60 of the server 51 manages time. The server 51 and the mobile terminal 21 can communicate with each other through the wireless communication unit 61, and the clock unit 30 of the mobile terminal 21 is periodically synchronized with the clock unit 60 of the server 51.

  The control unit 53 of the server 51 includes a CPU and controls the storage unit 52, the clock unit 60, and the wireless communication unit 61 described above. In addition, the control unit 53 of the server 51 executes an additional information file acquisition unit 66, a real time data acquisition unit 67, and a real time content generation unit 68.

  The additional information file acquisition unit 66 receives the additional information request data D transmitted from the additional information request unit 36 of the portable terminal 21 by the wireless communication unit 61, and displays content based on the additional information request data D An object is recognized, and the additional information file F associated therewith is searched and acquired.

  Hereinafter, the procedure for recognizing the object by the additional information file acquisition unit 66 will be described with respect to the case where the additional information request data D includes position data and the case where it does not include the position data.

  When the additional information request data D includes position data, first, the additional information file acquisition unit 66 refers to the object image table 62A and extracts image data of an object close to the position data. Here, the image data of an object close to the position data refers to image data of an object existing within a radius of 300 m from the position indicated by the position data.

  Next, the additional information file acquisition unit 66 matches the image data of the target object on which the content is to be displayed from among the extracted image data of the extracted object by performing a similar image search using local features such as SIFT and SURF. Acquire image data of the object to be matched.

  When the position information is not included in the additional information request data D, the additional information file acquisition unit 66 displays the content by the similar image search from the image data of all objects included in the object image table 62A. Image data of an object that matches the image data of the object is acquired.

  Note that if the additional information file acquisition unit 66 cannot acquire the image data of the object that matches the image data of the object on which the content is to be displayed, the additional information file acquisition unit 66 ends the process in any case, It does not transmit to the portable terminal 21.

  Next, a procedure for acquiring the additional information file F associated with the image data of the object based on the image data of the object acquired by the previous processing by the additional information file acquisition unit 66 will be described.

  The additional information file acquisition unit 66 refers to the additional information file table 62B and searches for the additional information file F that matches the storage address of the image data of the object acquired in the previous processing. As a result, when the number of hits is one, the additional information file F is transmitted to the portable terminal 21 by the wireless communication unit 61.

  A case where a plurality of additional information files F are hit and they have time zone data will be described. In that case, the clock unit 60 acquires the time, selects one additional information file F in which the time zone data corresponding to the time is set, and the additional information file F is stored in the portable terminal 21 by the wireless communication unit 61. Send to.

  A case where a plurality of additional information files F are hit and they have regional data will be described. In that case, using the position data of the additional information request data D, one additional information file F in which the area data corresponding to the position is set is selected, and the additional information file F is portable terminal by the wireless communication unit 61 To 21.

  A case where a plurality of additional information files F are hit and they have time zone data and regional data will be described. In that case, using the time obtained by the clock unit 60 and the position data of the additional information request data D, the additional information file F corresponding to the time and position is selected, and the additional information file F is selected as the wireless communication unit 61. Is transmitted to the portable terminal 21.

  The real time data acquisition unit 67 periodically acquires real time data from an external server. For example, the real-time data acquisition unit 67 acquires movie theater vacancy information every 5 seconds from an external server that manages movie theater vacancy information, or obtains real-time weather information every 10 seconds from the Japan Meteorological Agency server. Acquire it yourself or receive emergency earthquake bulletins. In this manner, exchange information, stock price information, operation information, and the like are acquired from an external server that manages other real-time data. The acquisition of real-time data includes a case where data is actively acquired such as going to the server, and a case where it is acquired passively as automatically notified from the server.

  In addition, although the real time data acquisition part 67 demonstrated with the structure which acquires real time data from an external server, it is not limited to this. That is, the real time data acquisition unit 67 includes a configuration for acquiring real time data managed in the server 51.

  The real-time content generation unit 68 generates real-time content based on the real-time data immediately or periodically when the real-time data acquisition unit 67 acquires real-time data, and stores the real-time content in the storage unit 52 of the server 51 in the same manner as other content. save. For example, when the real-time data acquisition unit 67 acquires real-time data of movie theater vacancy information, the real-time content generation unit 68 generates real-time content indicating the vacancy status of the movie theater based on the real-time data. If the acquisition unit 67 acquires real-time weather information from the Japan Meteorological Agency, based on this, real-time content indicating the weather situation is generated.

(Description of flow)
Next, the flow of the position information acquisition unit 35, additional information request unit 36, additional information file acquisition unit 66, display processing unit 37, and input processing unit 38 described above will be described.

  The flow of the position information acquisition unit 35 will be described with reference to FIG. FIG. 4 is a flowchart showing processing of the position information acquisition unit 35. In step S21, the position information acquisition unit 35 determines whether or not the GPS signal can be received. If the GPS signal can be received, the process proceeds to step S22, where the current position data is calculated and acquired from the GPS signal, and the process proceeds to step S23. In step S23, the position data is stored in the storage unit 52 together with the time acquired by the clock unit 30, and the process ends.

  In step S21, the position information acquisition unit 35 proceeds to step S24 when the GPS signal cannot be received. Then, in step S24, the position information acquisition unit 35 determines whether or not position data stored for a predetermined time is stored on the storage unit 52. If the position data is stored, the process proceeds to step S25. In step S25, the past position data stored on the storage unit 52 is acquired, and the process ends.

  In step S <b> 24, if there is no position data stored in the storage unit 52 for the stored time within the predetermined time, the position information acquisition unit 35 terminates the process as no position data.

  The flow of the additional information request unit 36 will be described with reference to FIG. FIG. 5 is a flowchart showing processing of the additional information request unit 36. First, in step S31, the additional information requesting unit 36 determines whether or not the state data in the storage unit 22 is in the recognition mode. If it is in the recognition mode, the process proceeds to step S32. If not, the process returns to step S31.

  In step S32, the additional information request unit 36 acquires the position data by the position information acquisition unit 35, acquires the image data of the object on which the content generated by the imaging unit 24 is displayed in step S33, and proceeds to step S34.

  Then, in step S34, the additional information request unit 36 generates additional information request data D from the position data and the object on which the content is displayed. In step S35, the additional information request data D is transmitted to the server by the wireless communication unit 31. 51, and returns to step S31 again.

  Thereafter, in step S31, the additional information request unit 36 repeats this series of processing when the state data is in the recognition mode. On the other hand, when the state data is not in the recognition mode, the additional information request unit 36 waits for a state change until the state data enters the recognition mode.

  The flow of the additional information file acquisition unit 66 will be described with reference to FIG. FIG. 6 is a flowchart showing the processing of the additional information file acquisition unit 66. In step S41, the additional information file acquisition unit 66 determines whether additional information request data D has been received.

  When the additional information file acquisition unit 66 receives the additional information request data D, the process proceeds to step S42. Further, the additional information file acquisition unit 66 waits for reception of additional information request data D that repeats step S41 until the received data does not arrive from the portable terminal 21.

  In step S42, the additional information file acquisition unit 66 determines whether or not the position data is included in the additional information request data D. If included, the process proceeds to step S43. In step S43, the additional information file acquisition unit 66 refers to the object image table 62A, and among the image data of the object image table 62A, acquires the image data of the object close to the position data of the object on which the content is displayed. Extract and proceed to step S44.

  On the other hand, in the case where position data is not included in step S42, the additional information file acquisition unit 66 proceeds to step S44 without performing the image data extraction work for this item.

  In step S44, the additional information file acquisition unit 66 obtains the image data of the object that matches the image data of the object on which the content is displayed from among the image data of the extracted object or the image data of all the objects by the similar image search. And the process proceeds to step S45.

  In step S45, the additional information file acquisition unit 66 determines whether there is image data of an object that matches the image data of the object on which the content is displayed. If there is, the process proceeds to step S46. In step S45, if no object image data matching the object image data for which content is to be displayed is found from the object image data, the additional information file acquisition unit 66 returns to step S41 and again adds. Step S41 is repeated until the information request data D arrives, and reception of additional information request data D is awaited.

  In step S46, the additional information file acquisition unit 66 refers to the additional information file table 62B, searches for the additional information file F associated with the image data of the matched object, and proceeds to step S47.

  In step S47, the additional information file acquisition unit 66 determines whether or not there is one additional information file F that has been found as a result of the search. If there is one search result, the process proceeds to step S49.

  If a plurality of search results are hit in step S47, the process proceeds to step S48, the time is acquired by the clock unit 60, and if the additional information request data D includes position data, the time is also used. One additional information file F is selected based on the position data, and the process proceeds to step S49.

  In step S49, the additional information file acquisition unit 66 transmits the additional information file F to the portable terminal 21 by the wireless communication unit 61, returns to step S41, and repeats step S41 until additional information request data D arrives again. Waiting for repeated reception.

  The flow of the display processing unit 37 will be described with reference to FIG. FIG. 7 is a flowchart showing the processing of the display processing unit 37. In step S51, the display processing unit 37 displays the image data generated by the imaging unit 24 on the display unit 25, and proceeds to step S52.

  In step S52, the display processing unit 37 determines whether or not the additional information file F has been received from the server 51. When the additional information file F is received, the display processing unit 37 proceeds to step S53 and stores the received additional information file F in the storage unit 22. In the temporary storage area, and the process proceeds to step S55.

  On the other hand, in step S52, when the additional information file F is not received from the server 51, the display processing unit 37 proceeds to step S54 and stores the additional information file F held in the temporary storage area of the storage unit 22. It is determined whether or not the content has been updated. If it has been updated, the process proceeds to step S55.

  If the additional information file F has not been updated, the process returns to step S51, and the process of displaying the image data captured by the imaging unit 24 is repeated until the additional information file F arrives or the additional information file F is updated.

  In step S55, the display processing unit 37 downloads the content from the server 51 based on the additional information file F, and downloads the content based on the display information of the additional information data A described in the additional information file F. The information is displayed on the display unit 25 together with the object, and the process proceeds to step S56.

  In step S56, the display processing unit 37 changes the state data in the storage unit 22 to the tracking mode, and returns to step S51 again.

  The flow of the input processing unit 38 will be described with reference to FIG. FIG. 8 is a flowchart showing processing of the input processing unit 38.

  In step S801, the input processing unit 38 determines whether or not there is operation information from the input unit 26. If there is operation information from the input unit 26, the process proceeds to step S802. If there is no operation information from the input unit 26, the process returns to S801 again and waits for operation information to repeat step S801 until the operation information from the input unit 26 arrives.

  In step S802, the input processing unit 38 determines whether or not the state data is in the tracking mode. If the state data is in the tracking mode, the process proceeds to step S803. If the status data is not in the tracking mode, the process returns to S801 again and waits for operation information to repeat step S801 until the operation information from the input unit 26 arrives.

  In step S803, the input processing unit 38 determines whether the operation information from the input unit 26 is operation information related to the displayed content. If the operation information is related to content, the process proceeds to step S804.

  In step S803, if the operation information from the input unit 26 is operation information that is not related to the displayed content, the process returns to S801 again and waits for input to repeat step S801 until the operation information from the input unit 26 arrives. Become.

  In step S804, the input processing unit 38 determines whether or not the operation associated with the operation information is a transition. If the operation relates to the transition, the process proceeds to step S805. On the other hand, if the operation associated with the operation information is not a transition, the process proceeds to step S806, where the input processing unit 38 is based on the additional information file F held in the temporary storage area of the storage unit 22. Then, the operation associated with the operation information is performed, and the process returns to step S801.

  In step S805, the input processing unit 38 determines whether or not the operation corresponds to “additional transition”, “update transition”, or “transition end” among the transitions, and if so, the process proceeds to step S807. Based on these, the additional information file F held in the temporary storage area of the storage unit 22 is updated, and the process returns to step S801.

  In step S805, if the operation does not correspond to “additional transition”, “update transition”, or “transition end” among the transitions, the input processing unit 38 proceeds to step S808.

  In step S808, the input processing unit 38 determines whether or not the operation corresponds to “all transition”. If the operation corresponds to “all transition”, the process proceeds to step S809, and the additional information data in the additional information file F is obtained. A new additional information file F is downloaded from the server 51 based on the storage position described in A, and the process proceeds to step S810.

  In step S810, the input processing unit 38 deletes the old additional information file F stored on the storage unit 22, and returns to step S801.

  In step S808, if the operation does not correspond to “all transition”, the input processing unit 38 proceeds to step S811, determines whether the operation corresponds to “external transition”, and the operation corresponds to “external transition”. When the web browser is started, the URL described in the additional information file F is opened by the web browser, and the process proceeds to step S813 to end the AR application.

(Example in which the display system 1 is used as a station information display board 81)
With reference to FIGS. 9 to 15, a state in which a group of contents related to the station guidance display board 81 is displayed using the display system 1 of the present invention will be described. Here, FIG. 15 shows a schematic diagram showing the exchange of data between the portable terminal 21 and the server 51.

  The user operates the touch panel 27 of the mobile terminal 21 to activate the AR application. When the AR application is activated, the position information acquisition unit 35, the additional information request unit 36, the display processing unit 37, and the input processing unit 38 operate in parallel with each other.

  With reference to FIG. 9, how the display system 1 of the present invention recognizes the guidance display board 81 of the target station is described. FIG. 9 is a diagram illustrating a state in which the station guidance display board 81 is displayed on the touch panel 27 of the mobile terminal 21.

  The user adjusts the orientation of the imaging unit 24 of the mobile terminal 21 so that the imaging unit 24 of the mobile terminal 21 can generate an image of the guidance display board 81 at the station. Then, the display processing unit 37 continues to display on the touch panel 27 the image data of the station guidance display board 81 generated by the imaging unit 24 based on the flow shown in FIG.

  At this time, the position information acquisition unit 35 continues to acquire position data based on the flow shown in FIG. Further, the additional information request unit 36 continues to generate additional information request data D based on the position data and the image data based on the flow shown in FIG. 5 and transmit it to the server 51 via the wireless communication unit 31.

  At this time, in the server 51, the additional information file acquisition unit 66 is waiting to receive the additional information request data D. When the additional information request data D is received from the mobile terminal 21, the server 51 stores the additional information request data D based on the flow shown in FIG. Attempt to acquire the additional information file F stored in the unit 32.

  Here, the additional information request data D transmitted from the mobile terminal 21 includes the position data of the station guidance display board 81. Therefore, the additional information file acquisition unit 66 refers to the object image table 62A and extracts object image data close to this position data from the object image table 62A.

  And the additional information file acquisition part 66 searches the image data of the thing which matches the image data of the guidance display board 81 of a station from the image data of the extracted thing by similar image search. Then, the additional information file acquisition unit 66 refers to the additional information file table 62B and searches for the additional information file F associated with the image data of the matched object.

  If the image data of the matching object is not found, the additional information file acquisition unit 66 ends the processing for this request, and the additional information file acquisition unit 66 of the object that matches the image data on the guidance display board 81 of the station to be sent next. Search for an image.

  Here, it is assumed that the additional information file F associated with the guidance display board 81 of this station is prepared differently for each time zone, and a plurality of hits are made. In this case, the additional information file acquisition unit 66 selects one additional information file F corresponding to the time zone acquired by the clock unit 60 from the additional information files F prepared for each time zone.

  The additional information file F has a data structure as shown in FIG. And the additional information file F linked | related with the guidance display board 81 of this station has time zone data set, area data is not set, and has four additional information data A.

  Specifically, the additional information file F includes four stores, “XX movie theater”, “□ △ bowling”, “XX department store”, and “XX instrument”, which are stores near the information display board 81 of this station. It has additional information data A about one store. In this additional information data A, the storage location of content, display information, operation information, and execution information are described.

  The description of the additional information data A will be described by taking “XX movie theater” as an example. In the additional information data A of “Ox movie theater”, the storage location of the image content 82 of “Ox movie theater” in the server 51, that is, the address is described in the content storage location.

  In addition, in the additional information data A, the content display information describes that the content is displayed on the upper right of the station guide display board 81, and the content execution information transitions when the content is tapped. And the storage location of the other additional information file F required for performing all transitions in the server 51 is described. In this content, nothing is set for the operation information. Further, additional information data A for “□ Δ bowling”, “◯ × department store”, and “◯ □ musical instrument” is also described.

  When the additional information file F is transmitted to the portable terminal 21, the display processing unit 37 stores the four additional information data A described in the received additional information file F on the basis of the flow shown in FIG. In the temporary storage area.

  Then, the display processing unit 37 downloads each content from the server 51 based on the contents of the additional information file F held in the temporary storage area of the storage unit 22, and uses these contents as additional information of the additional information file F. Based on the display information of the data A, the information is displayed on the display unit 25 together with the station guidance display board 81.

  The manner in which content is displayed around the station guidance display board 81 will be described with reference to FIG. FIG. 10 is a diagram showing a state in which related contents are displayed in a superimposed manner on a station guidance display board 81 displayed on the touch panel 27 of the mobile terminal 21 by the display system 1 of the present invention.

  As shown in FIG. 10, when the display processing unit 37 of the mobile terminal 21 downloads and displays each content based on the content of the additional information file F, “○ × The image content 82 of “movie theater”, the image content 83 of “□ △ bowling” in the lower right, the image content 84 of “XX department store” in the upper left, and the image content 85 of “XX instrument” in the lower left are displayed. The The above contents are all image data.

  Thus, the user can obtain store information near the station via the display unit 25 only by holding the station information display board 81 with the portable terminal 21. Here, it is assumed that the user is interested in “XX movie theater” in the store information near the station and taps the image content 82 of “XX movie theater” as shown in FIG.

  As described above, the image content 82 of “XX movie theater” describes the operation when tapped in the content execution information of the additional information data A indicating “XX movie theater” of the additional information file F. , All transitions are set when tapped.

  Therefore, when the image content 82 of “XX movie theater” is tapped, the input processing unit 38 downloads the additional information file F necessary for all transitions from the server 51 based on the flow shown in FIG. At the same time, the old additional information file F held on the storage unit 22 is deleted.

  When the additional information file F is downloaded from the server 51, the display processing unit 37 holds the new additional information file F on the temporary storage area of the storage unit 22 based on the flow shown in FIG. Based on the new additional information file F, each content is displayed on the touch panel 27. Then, it is displayed on the touch panel 27 as shown in FIG.

  Referring to FIG. 11, a state where “XX movie theater” is selected and the content group indicating the store information is completely transitioned to the content group indicating the information “XX movie theater” will be described. FIG. 11 is a partial enlarged view of the mobile terminal 21 showing a state in which one content is selected and all transitions are made to another content group related to the content.

  As shown in FIG. 11, when the display processing unit 37 of the mobile terminal 21 downloads and displays each content based on the contents of the new additional information file F, from the upper right to the lower right with the station information display board 81 as the center. Toward the image image 86 of “XX movie theater” indicating that the content related to “XX movie theater” is being displayed, and “fee” indicating a fee for watching a movie at “XX movie theater” ”And a“ map ”image content 88 indicating a map from the station where the station information display board 81 is set to“ ◯ × cinema ”are displayed.

  Also, from the upper left to the lower left, currently being screened at “○ × Movie Theater” such as “Augmented Reality Story” image content 89, “AR Great Adventure” image content 90, “Air Woman” image content 91 The movie title is displayed.

  Thus, the user simply taps the image content 82 (see FIG. 10) of “XX movie theater” showing interest, and displays the content information group related to this “XX movie theater” on the display unit 25. Can be obtained through. In other words, the display system 1 of the present invention has a hierarchical structure in which the content displayed over the object has a hierarchical structure, and can efficiently provide content information.

  It is assumed that the user is interested in the movie “augmented reality story” and taps the image content 89 of “augmented reality story”.

  Here, the content execution information of the additional information data A indicating the “augmented reality story” of the additional information file F is set to perform an update transition when tapped. Specifically, the image content 86 of “Ox movie theater” is not displayed, and the image content 92 of “augmented reality story” is displayed in the same part.

  Further, the image content 89 of “augmented reality story”, the image content 90 of “AR Great Adventure”, and the image content 91 of “A woman of AA” are not displayed, and “screening time” is displayed on the left side of the station information display board 81. Image content and “reserved” image contents 94, 95, and 96 associated with each screening time are set to be displayed between the “screening time” content 93 and the station guide display board 81.

  Therefore, when the image content 89 of “augmented reality story” is tapped, the input processing unit 38 configures the additional information file F held in the temporary storage area of the storage unit 22 based on the flow shown in FIG. Delete the additional information data A representing “○ × Movie Theater”, “Augmented Reality Story”, “AR Great Adventure”, “A Woman of Earl”, “Augmented Reality Story”, “Screening Time”, “Reservation”, The additional information data A representing “reservation” and “reservation” is added to update the contents of the additional information file F held in the temporary storage area of the storage unit 22.

  When the additional information file F is updated, the display processing unit 37 displays content on the touch panel 27 based on the updated additional information file F based on the flow shown in FIG. Then, it is displayed on the touch panel 27 as shown in FIG.

  With reference to FIG. 12, a state where “augmented reality story” is selected and the content group indicating “Ox movie theater” is updated to a content group indicating information of “augmented reality story” will be described. FIG. 12 is a partially enlarged view of the mobile terminal 21 showing a state in which one content is selected and an update transition is made to another content group related to the content.

  As shown in FIG. 12, when the display processing unit 37 of the mobile terminal 21 displays each content on the basis of the updated content of the additional information file F, the right part centering on the station information display board 81 is The image content 86 (see FIG. 11) of “XX movie theater” is updated to the image content 92 of “augmented reality story” indicating that the content related to the movie of “augmented reality story” is displayed.

  In addition, on the left side, image content 93 of “screening time” indicating the screening time of “augmented reality story” is displayed, and the content 93 of “screening time” and the guidance display board 81 of the station are associated with each screening time. The displayed “reserved” image contents 94, 95, 96 are displayed.

  As a result, the user can select “Ox movie theater” from the store information group and obtain a content group related to “Ox movie theater”. By simply tapping the image content 89 of the “augmented reality story” from the related content group, the content group related to the “augmented reality story” can be obtained via the display unit 25.

  Next, it is assumed that the user decides to watch a movie called “augmented reality story”, decides to make a reservation purchase for the time from 12:20, and taps the image content 94 of “reservation”.

  Here, the content execution information of the additional information data A indicating “reservation” of the additional information file F is set to perform an additional transition when tapped. Specifically, it is set to display the real-time content 97 of “vacant seat information” associated with the screening time.

  In the real-time content 97 of the “vacant seat information”, the real-time data acquisition unit 67 acquires the real-time data of the movie theater vacancy information from an external server, and the real-time content generation unit 68 converts the real-time data of the vacant seat information into the acquired real-time data. Content generated based on this.

  Then, when the “reserved” content 94 is tapped, the input processing unit 38 adds the additional information constituting the additional information file F held in the temporary storage area of the storage unit 22 based on the flow shown in FIG. The additional information data A representing “vacant seat information” is added to the data A to update the contents of the additional information file F.

  When the content of the additional information file F is updated, the display processing unit 37 displays the content on the touch panel 27 based on the updated content of the additional information file F based on the flow shown in FIG. . Then, it is displayed on the touch panel 27 as shown in FIG.

  Referring to FIG. 13, a state in which the “reserved” image content 94 is selected and the real-time content 97 “vacant seat status” is newly added and displayed will be described. FIG. 13 is a partial enlarged view of the mobile terminal 21 showing a state in which one content is selected and an additional transition is made to another content related to the content.

  As shown in FIG. 13, when each content is displayed based on the content of the additional information file F updated by the display processing unit 37 of the mobile terminal 21, a real-time content 97 of "vacant seat status" is newly displayed. A square in the real-time content 97 of “vacant seat status” indicates a seat. The hatched portion of the seat indicates a reserved seat, and whether or not the seat is reserved is based on the real time data acquired by the real time data acquisition unit 67. In addition, the seat surrounded by a thick line indicates the S seat.

  As a result, the user can obtain vacant seat information based on the real-time data of the desired times only by tapping the image content 94 of the desired times. Subsequently, it is assumed that the user decides to settle the desired number of times for the movie “augmented reality story” and taps a vacant seat that is not shaded among the seats.

  Here, the content execution information of the additional information data A indicating “vacant seat information” in the additional information file F is set to perform an external transition when tapped. Specifically, the web server is activated with the URL described in the execution information and the selected seat as arguments, and the AR application is terminated.

  Accordingly, when the user taps the desired seat, the user can make a payment by changing to the payment screen of the external server shown in FIG. FIG. 14 is an overall view of the mobile terminal 21 showing a state in which one content is selected and external transition is made to a web page. The user can enter the ID and password shown in FIG. 14 and proceed to the settlement screen of the external server to pay for the movie.

(Example in which the display system 1 is used for an opened page of the book 100)
A state in which a related content group is displayed on the opened page of the book 100 using the display system 1 of the present invention will be described with reference to FIGS. FIG. 16 is a diagram illustrating a state in which a book whose page is opened is displayed on the touch panel 27 of the mobile terminal 21.

  The user activates the AR application and adjusts the orientation of the imaging unit 24 of the mobile terminal 21 so that the imaging unit 24 can generate an image of the station book 100. Then, the book 100 is displayed on the touch panel 27 by the display processing unit 37 as shown in FIG.

  At this time, in the AR application, the position information acquisition unit 35 continues to acquire position data, the additional information request unit 36 generates additional information request data D based on the position data and the image data generated by the imaging unit 24, and wireless It continues to be transmitted to the server 51 via the communication unit 31.

  In the server 51, when the additional information request data D is received, the additional information file acquisition unit 66 uses the image data of the book that opened the page included in the additional information request data D for the book that opened this page. Search for image data of objects that match the image data.

  Then, the additional information file acquisition unit 66 searches the additional information file F associated with the image data of the object based on the image data of the found object. In this example, a plurality of additional information files F are associated with the image data of the book whose page is opened for each region, and when the search is performed, a plurality of additional information files F are hit.

  In this case, the additional information file acquisition unit 66 refers to the position data included in the additional information request data D, and selects the additional information file F in which the regional data included in the position data is set.

  When region data is set for an image of one object and there are a plurality of additional information files F, the additional information file F that is selected when position data cannot be acquired is always included. It shall be assumed.

  The additional information file F has a data structure as shown in FIG. The additional information file F associated with the book 100 that has opened this page has three pieces of additional information data A in which the time zone data is not set, the regional data is set.

  The three additional information data A relate to the image content 101 of “tower”, the image content 102 of “gate”, and the image content 103 of “warehouse”. In this additional information data A, the storage location of content, display information, operation information, and execution information are described.

  The additional information data A of “tower” is the same image data as the photograph of the tower whose content is published in the book 100. The display information of the contents describes that the image data of this tower is displayed in the same manner on the photograph posted in the book 100.

  When the image content 101 of the “tower” is tapped in the execution information of the additional information data A of “tower”, the image content 101 of the “tower” is hidden, and the moving image content of the “tower” is instead displayed. An update transition that displays is set.

  The video content 104 of “Tower” is set so that the display information in the additional information data A is displayed on the left page of the book 100 with the page opened, and the operation information is automatically played once. When the execution information is tapped, an end transition is set so that this content is not displayed.

  Further, the additional information data A of “Gate” is set in substantially the same manner as the additional information data A of “Tower”, and when tapped, the moving image content is displayed on “Gate”. . The “tower” and “gate” described above have been described with the configuration in which the image data of “tower” and “gate” are displayed in the same manner on the photo posted in the book 100, but are not limited thereto. Not. For example, a transparent sheet having the same size as the photograph may be displayed on the photograph.

  The additional information data A of “related information” is the image content 103 of “related information” whose content is related to the content of the book 100, for example, a warehouse built in the same era as “tower” and “gate” About.

  This additional information data A is set to be displayed on the right side of the book 100 in the display information, and when the image content 103 of this “related information” is tapped in the execution information, an additional content group is displayed. Transition is set.

  When the additional information file F is transmitted to the mobile terminal 21 by the additional information file acquisition unit 66, the display processing unit 37 causes the display unit 25 to display a content group based on the received additional information file F.

  With reference to FIG. 17, a state where related contents are displayed on a book will be described. FIG. 17 is a diagram illustrating a state in which related content is superimposed and displayed on the book 100 displayed on the touch panel.

  As shown in FIG. 17, when the display processing unit 37 of the mobile terminal 21 displays each content based on the content of the additional information file F, the image content 101 of “tower” is displayed on the tower photo on the left page of the book 100. , The image content 102 of “Gate” is superimposed on the picture of the gate of the page on the right side of the book, and the image content 103 of “Related Information” is displayed on the right side of the book 100.

  The user can obtain the image content 103 of the related information related to the content of the book 100 and the position of the user by simply holding the book 100 with the page open on the portable terminal 21. The user can also see the image content of the “tower” and the image content of the “gate” superimposed on the image of the book 100, so that the image of the tower and the gate can be seen floating from the page, and the transition or the like can be performed. I can understand.

  Then, it is assumed that the user is interested in the image content 101 of “tower” and taps this. Then, the image content 101 of “tower” is hidden, and the moving image content 104 of “tower” is displayed instead.

  A state in which the image content 101 of “tower” is updated to the moving image content of “tower” will be described with reference to FIG. FIG. 18 is a diagram illustrating a state where the image content 101 of “tower” is tapped and the moving image content 104 of “tower” is reproduced and finished.

  As shown in FIG. 18, when the “tower” image content 101 is tapped by the user, the “tower” moving image content 104 related to this tower is reproduced. As a result, if the user does not use the display system 1 of the present invention, he / she could not obtain only the photos of the towers published in the books and the sentences related to the towers described in the books. You can also get tower information. Next, it is assumed that the user taps the image content 103 of “related information” in the state shown in FIG.

  Referring to FIG. 19, how the user taps the image content 103 of “related information” and a related content group is displayed in the state shown in FIG. 18 will be described. FIG. 19 is a diagram illustrating a state in which one content is selected and another content group related to the content is displayed.

  As shown in FIG. 19, when the image content 103 of “related information” is tapped by the user, the character string content 105 of “description”, here, the description of the warehouse is displayed on the page portion on the right side of the book 100. Also, under the page portion on the left side of the book 100, the image content 106 of “nearest station” indicating the nearest station of the warehouse is displayed.

  Further, on the right side of the image content 106 of “Nearest Station”, an image content 107 of “Map” to the location where the warehouse is located from the nearest station is displayed, and further, a restaurant near the warehouse is displayed on the left side of the book 100. An image content 108 of “restaurant” indicating information and an image content 109 of “hotel” indicating hotel information in the vicinity of the warehouse are displayed below the image content 108.

  That is, the user can organically obtain information related to this by tapping one content. In addition, when the displayed content is further tapped, the transition is made to the next content. That is, the related content information has a hierarchical structure, so that the content information is efficiently provided to the user.

  Then, for example, when the image content 106 of “Nearest station” is tapped, content indicating train arrival / departure information and delay information is displayed, and when the image content 108 of “restaurant” is tapped, the restaurant content is displayed. When the content representing the menu is displayed and the image content 109 “hotel” is tapped, the content indicating the access information to the hotel, the fee, etc. is displayed. Provide efficiently.

(Effect of the display system 1 of the present invention)
According to the display system 1 of the present invention, the additional information including one content includes transition information for transitioning to another content. As a result, more content can be provided to the user in the limited display area of the display unit 25. Moreover, content can be efficiently provided to the user by associating the transition to other content with one content.

  Further, according to the display system 1 of the present invention, a plurality of additional information data A is associated with one object and stored in the storage unit 52, and at least one or more of the plurality of additional information data A is stored. Transition information for transitioning to another content is included in the execution information in the additional information data A.

  As a result, a plurality of contents can be displayed with respect to one object, and a transition can be made from the plurality of contents to other contents. Conventional augmented reality technology often displays a single content for a single marker. On the other hand, according to the display system 1 of the present invention, it is possible to display a plurality of contents on one object at the same time, and furthermore, since it has a hierarchical structure, more to the user. Content can be provided efficiently.

  Further, according to the display system 1 of the present invention, the transition information includes an additional transition for newly displaying other content on the display unit 25 and an update transition for displaying the other content while hiding the displayed content. And external transition for transitioning from the displayed content to the web page of the external web server. By having various transition information in this way, it can be designed to provide content more efficiently to the user.

  In addition, according to the display system 1 of the present invention, the server 51 has a real-time data acquisition unit 67 that acquires real-time data, and a real-time content that generates real-time content based on the real-time data acquired by the real-time data acquisition unit 67. And a content generation unit 68. The content displayed on the display unit 25 of the mobile terminal 21 includes the real-time content generated by the real-time content generation unit 68. As a result, the user can obtain not only the content created in advance but also information based on the real-time data from the real-time content.

  In addition, according to the display system 1 of the present invention, the mobile terminal 21 includes the additional information requesting unit 36 that requests the server 51 for additional information related to the object based on the image data of the object. The storage unit 52 includes an additional information file F in which one or more additional information data A is associated with one target object, and image data of an object associated with the additional information file F. It is remembered.

  Further, according to the display system 1 of the present invention, the server 51 searches the storage unit 52 for image data of an object that matches the image data of the object, and the additional information file F associated with the image data of the object. And an additional information file acquisition unit 66 that transmits the information to the portable terminal 21. This eliminates the need for manufacturing and embedding an IC chip in an object on which content is to be displayed or attaching a barcode to manage the content.

  Further, according to the display system 1 of the present invention, the storage unit 52 of the server 51 stores the position data of the object in association with the image data of the object. And the portable terminal 21 is further provided with the position information acquisition part 35 which acquires the position data of a target object. Then, the additional information request unit 36 also transmits the position data of the object to the server 51, and the additional information file acquisition unit 66 of the server 51 reads from the storage unit 52 based on the image data of the object and the position data of the object. It searches for image data of objects. As a result, it is possible to more quickly and accurately recognize the object on which the content is displayed.

  Further, according to the display system 1 of the present invention, the additional information file F is created for each region. Then, the additional information file acquisition unit 66 searches and acquires the image data of the object from the storage unit 52 based on the image data of the object or the image data of the object and the position data of the object, and acquires the additional information file F. Acquisition is based on the acquired image data and position data.

  As a result, the additional information file acquisition unit 66 can acquire the additional information file F corresponding to the region, and when the content is displayed on the display unit 25 based on this, the user can select the content corresponding to the region. Content can be obtained more efficiently.

  Further, in the display system 1 of the present invention, the additional information file F has time zone data and is created for each time zone. The server 51 further includes a clock unit 60 that provides time. Further, the additional information file acquisition unit 66 searches for and acquires the image data of the object from the storage unit 52 based on the image data of the object, and acquires the additional information file F based on the image data and time of the object acquired. get.

  As a result, the additional information file acquisition unit 66 can acquire the additional information file F corresponding to the time, and when the content is displayed on the display unit 25 based on this, the user can select the content corresponding to the time. Content can be obtained more efficiently.

  In addition, this invention is not limited only to the above-mentioned Example. For example, the transition of the operation of the additional information file F may include an automatic transition in addition to an additional transition, an update transition, a full transition, an external transition, and a transition end. In the automatic transition, for example, when a predetermined time elapses after the content is displayed, the content is not displayed and another content is newly displayed. This automatic transition has the storage position in the server of the additional information data A of the content to be newly displayed as transition information, and the display time.

  The display system 1 of the present invention also includes a step of storing additional information data A including additional information including content and transition information to other content in the storage unit 52, and generating image data of the object. A display method comprising the steps of: acquiring additional information data A related to the object based on the image data of the object; and displaying the content of the additional information data A on the display unit 25; And everything that includes this display method.

  Further, the present invention includes a computer program for causing a computer to execute the steps included in the display method described above, and all devices in which the program is incorporated. Further, the present invention includes a computer-readable storage medium storing the above-described computer program and all devices including the storage medium. Moreover, it is good also as a display system of this invention only with the portable terminal 21, without having the server 51, for example, only with a smart phone and a tablet.

  A display system for displaying content superimposed on the real environment of the present invention, a display method, a computer program for causing a computer to execute steps included in the display method, and a computer-readable storage medium storing the computer program Various modifications can be made without departing from the scope of the present invention.

DESCRIPTION OF SYMBOLS 1 Display system 21 Portable terminal 22 Memory | storage part 23 Control part 24 Image pick-up part 25 Display part 26 Input part 27 Touch panel 28 GPS part 30 Clock part 31 Wireless communication part 34 Setting language acquisition part 35 Position information acquisition part 36 Additional information request | requirement part 37 Display Processing unit 38 Input processing unit 51 Server 52 Storage unit 53 Control unit 60 Clock unit 61 Wireless communication unit 62 Database 62A Object image table 62B Additional information file table 66 Additional information file acquisition unit (additional information acquisition unit)
67 Real-time data acquisition unit 68 Real-time content generation unit 81 Station information display board 82 Image content of “○ × cinema” 83 Image content of “□ △ bowling” 84 Image content of “○ × department store” 85 “○ □ Musical instrument Image content of “86” Image content of “XX movie theater” 87 Character string content of “Charge” 88 Image content of “Map” 89 Image content of “Augmented Reality” 90 Image content of “AR Adventure” 91 “AAR” The image content of the woman of 92 The image content of the augmented reality story 93 The image content of the screening time 94 The image content of the reservation 95 The image content of the reservation 96 The image content of the reservation 97 The vacancy status Real-time content 100 Book 101 “Tower” image content 1 2 “Gate” image content 103 “Warehouse” image content 104 “Tower” video content 105 “Related information” character string content 106 “Nearest station” image content 107 “Map” image content 108 “Parking” "Image content 109""Image" content of "Hotel" 110 Image content of "Restaurant" D Additional information request data A Additional information data F Additional information file

Claims (11)

A server comprising a storage unit for storing additional information including content;
A terminal including an imaging unit that generates image data of the object and a display unit that displays the image data, the terminal acquires additional information related to the object from the server, and the additional information A display system for displaying the content of the display on the display unit,
The display system, wherein the additional information includes transition information for transitioning to another content.   A plurality of the additional information is associated with one object and stored in the storage unit. The additional information includes transition information for transitioning at least one or more of the plurality of additional information to another content. The display system according to claim 1, wherein: The transition information is
Additional transitions to display other content on the display,
An update transition that hides the displayed content and displays other content,
The display system according to claim 1, further comprising an external transition for transitioning from the displayed content to a web page of an external web server. The server
A real-time data acquisition unit for acquiring real-time data;
A real-time content generation unit that generates real-time content based on the real-time data acquired by the real-time data acquisition unit;
The display system according to claim 1, wherein the content displayed on the display unit of the terminal includes real-time content generated by the real-time content generation unit. . The terminal
Based on the image data of the object, an additional information request unit that requests the server for additional information related to the object;
The storage unit includes an additional information file in which one or more additional information is associated with one object, and
The object is previously photographed as image data, and the image data of the object associated with the additional information file is stored,
The server searches the storage unit for image data of an object that matches the image data of the object, acquires an additional information file associated with the image data of the object, and transmits the additional information file to the terminal The display system according to claim 1, further comprising an acquisition unit. In the storage unit of the server, position data of an object is stored in association with image data of the object,
The terminal
A position information acquisition unit for acquiring position data of the object;
The additional information requesting unit also transmits position data of the object to the server,
6. The server according to claim 1, wherein the additional information file acquisition unit of the server searches the image data of the object from the storage unit based on the image data of the object and the position data of the object. The display system according to claim 1. The additional information file is created for each region,
The additional information file acquisition unit searches and acquires the image data of the object from the storage unit based on the image data of the object or the image data of the object and the position data of the object, and acquires the additional information file. The display system according to claim 6, wherein the display system is acquired based on the acquired image data and position data. The additional information file is created for each time zone,
The server further includes a clock unit that provides time,
The additional information file acquisition unit retrieves and acquires the image data of the object from the storage unit based on the image data of the object, and acquires the additional information file based on the image data of the acquired object and the time. The display system according to claim 1, wherein the display system is a display system. Storing additional information including content and additional information including transition information to other content in a storage unit;
Generating image data of the object;
Obtaining additional information related to the object based on image data of the object;
And displaying the content of the additional information on the display unit.   A computer program for causing a computer to execute the steps included in the display method according to claim 9.   A computer-readable storage medium storing the computer program according to claim 10.

Images