JP4624765B2 - Information superposition terminal and additional image superposition system - Google Patents

Description

  The present invention relates to an information superimposing terminal and an additional image superimposing system including the information superimposing terminal.

  Conventionally, various image composition techniques have been proposed. For example, Patent Document 1 proposes an optical image composition display device. Here, the customer can see the customer trying on the costume without actually trying it on. Further, for example, Patent Document 2 proposes a clothing try-on system. In this configuration, people with various body shapes, physiques, and appearances can try on clothes by synthesizing images on the screen.

JP-A-7-175012 JP 2004-86662 A

  However, in the configuration of Patent Document 1, a customer's image and a costume image are optically synthesized using a half mirror. In this configuration, a large half mirror is required. For this reason, an apparatus will enlarge. When the customer moves, the position of the costume image remains fixed. Therefore, there arises a problem that the customer's image and the costume image are shifted.

  Moreover, in the structure of patent document 2, it superimposes and displays on the display, matching the size of clothes with a human body shape, physique, and the like. In this configuration, since the images are superimposed on the display, there is a problem of lack of reality as compared to observation in a real field of view.

  The present invention has been made in view of the above problems, and realistically superimposes an additional image based on additional image data according to the state of the observation target such as the position, size, and orientation of the observation target on the real field of view. An object is to provide an additional image superimposing system that can be observed, and a small information superimposing terminal used in the system.

  In order to solve the above-described problems and achieve the object, according to the first aspect of the present invention, there is provided an additional image superimposing system for superimposing and displaying an additional image on an observation target in an actual field of view. An imaging unit for imaging an object, an electronic information display unit, an information superimposing terminal including at least a transflective optical element that superimposes and displays an additional image displayed on the electronic information display unit on an observation target in an actual visual field, and an imaging unit An observation target recognition unit that identifies an observation target imaged by the image recognition, and additional image data processing that acquires additional image data associated with the observation target and processes the additional image data according to the state of the observation target And an additional image superimposing system in which an additional image based on the additional image data processed by the additional image data processing unit is displayed on the electronic information display unit.

  Moreover, according to a preferable aspect of the present invention, it is desirable to have an additional image content setting unit that sets the content of the additional image superimposed on the observation target. According to a preferred aspect of the present invention, it is desirable to have an observation target setting unit that sets an observation target on which an additional image is superimposed.

  Further, according to a preferred aspect of the present invention, the observation target data storage unit for storing the observation target data for the observation target recognition unit to recognize the observation target and the additional image data associated with the observation target are stored. An image data storage unit, and the observation object data storage unit and the additional image data storage unit are provided in a server computer that can communicate with the information superimposing terminal. It is desirable to include an additional image content setting unit and an additional image data storage unit.

  Moreover, according to a preferable aspect of the present invention, it is desirable to have a user setting unit that sets a user who can display and observe an additional image in a superimposed manner.

  Moreover, according to a preferable aspect of the present invention, it is desirable to have a time setting unit that sets a time zone in which the additional image can be displayed in a superimposed manner and observed.

  Moreover, according to a preferable aspect of the present invention, it is desirable to have an area setting unit that sets an area where the additional image can be displayed in a superimposed manner and observed.

  Moreover, according to a preferable aspect of the present invention, it is desirable that the additional image is a costume image that is superimposed and displayed on the observation target.

  According to the first aspect of the present invention, the additional image data processing unit processes the additional image data according to the state of the observation target. The electronic information display unit displays an additional image based on the processed additional image data. The transflective optical element displays the additional image displayed on the electronic information display unit in a superimposed manner in the actual field of view to be observed. As a result, an additional image corresponding to the position, size, orientation, and the like of the observation target can be superimposed and displayed on the actual visual field.

  According to the second aspect of the present invention, the electronic information display unit displays an additional image based on the additional image data corresponding to the state of the observation target. The transflective optical element displays the additional image displayed on the electronic information display unit in a superimposed manner in the actual field of view to be observed. The user can observe the observation object through the transflective optical element. For this reason, the semi-transmissive optical element can be miniaturized regardless of the size of the observation target. Thereby, the small information superimposition terminal used for the above-mentioned additional image superimposition system can be provided.

  Embodiments of an information superimposing terminal and an additional image superimposing system according to the present invention will be described below in detail with reference to the drawings. In addition, this invention is not limited by this Example.

  FIG. 1 is a diagram illustrating a schematic configuration of an additional image superimposing system according to the first embodiment of the present invention. The user OBS observes the observation target OBJ using the information superimposing terminal 100. The information superimposing terminal 100 and the server computer 150 are configured to be communicable. The observation object OBJ is a dinosaur skeleton model displayed in a museum. The skeletal model is composed only of dinosaur frames.

  The information superimposing terminal 100 includes a main body 110, an imaging unit 104, an electronic information display unit 101, and a transflective optical element 102. A small camera can be used as the imaging unit 104. As the electronic information display unit 101, a liquid crystal display panel or an organic EL panel can be used. The transflective optical element 102 can be a hologram element or a half mirror.

  2A and 2B show a perspective configuration of the information superimposing terminal 100. FIG. The transflective optical element 102 is configured to be opened and closed. FIG. 2A shows a state where the transflective optical element 102 is opened. FIG. 2-2 shows a state where the transflective optical element 102 is stored. The main body 110 has a substantially egg shape.

  FIG. 3 shows a schematic configuration of the information superimposing terminal 100. The transflective optical element 102 transmits light from the observation object OBJ and reflects light from the electronic information display unit 101 toward the user OBS. Thereby, the semi-transmissive optical element 102 superimposes an additional image (see FIGS. 4A and 4B) based on the additional image data 120a displayed on the electronic information display unit 101 on the actual field of the observation target OBJ. To display.

  FIG. 4A shows an image in which the observation object OBJ in the real field of view and the additional image based on the additional image data 120a displayed in a superimposed manner are combined. The additional image data 120a is a muscle part of a dinosaur. As a result, the user OBS can recognize the additional image based on the additional image data 120a while superimposing the observation target OBJ while observing the observation target OBJ. As a result, the user OBS can simultaneously observe the observation object OBJ in the real field and the additional image based on the additional image data 120a displayed in a superimposed manner. Further, when the user OBS changes the position and distance at which the observation object OBJ is observed, the appearance of the observation object OBJ also changes. For example, FIG. 4B illustrates a composite image when the user OBJ views the dinosaur skeleton model from the back (the direction of the tail). When the relative positional relationship between the user OBS and the observation object OBJ changes due to the configuration described later, the additional image data 120a is processed according to the observation object OBJ and displayed in a superimposed manner.

  FIG. 5 is a functional block diagram of the additional image superimposing system according to the present embodiment. FIG. 6 is a flowchart showing a procedure for displaying an additional image based on the additional image data 120a so as to be superimposed on the real field of view. First, a rough procedure will be described with reference to FIG. 6, and then further detailed with reference to the functional blocks of FIG.

  In step S601 in FIG. 6, the observation target recognition unit 106 (see FIG. 5) recognizes the observation target OBJ imaged by the imaging unit 104 by image processing. In step S602, it is determined whether or not the additional image data 120a corresponding to the recognized observation object OBJ is stored in the additional image data database 154 (see FIG. 5). When the determination result of step S602 is true (Yes), the process proceeds to step S603. In step S603, the additional image data 102a to be displayed in a superimposed manner is acquired. In step S605, the acquired additional image data 120a is processed according to the state of the observation target OBJ. The electronic information display unit 101 displays an additional image based on the processed additional image data 120a. The additional image based on the displayed additional image data 120a is displayed superimposed on the actual field of view by the transflective optical element 102. When the determination result in step S602 is false (No), the process is terminated without displaying the superimposed image.

  Next, the description will be continued with reference to FIG. The information superimposing terminal 100 and the server computer 150 are configured to be capable of wireless communication. The server computer 150 is operated and managed by a museum exhibiting a dinosaur framework model. The imaging unit 104 images the observation target OBJ. The observation target data acquisition unit 105 selects and acquires a part necessary for specifying the observation target OBJ from the captured image. The observation target recognition unit 106 identifies the observation target OBJ by image recognition processing, for example, pattern matching processing. In the present embodiment, the observation target recognition unit 106 identifies and recognizes the type of dinosaur that is the observation target OBJ. The communication unit 107 on the information superimposing terminal 100 side transmits / receives data to / from the communication unit 151 on the server computer 150 side. Data transmission / reception is controlled by the control units 108 and 152.

  The server computer 150 includes an observation target database 153 and an additional image data database 154. The observation target database 153 stores information on various exhibits displayed in the museum. The additional image data database 154 stores image data to be added later to the observation target OBJ. For example, the additional image data database 154 stores image data of various dinosaur muscle parts, preferably three-dimensional image data. The observation target database 153 corresponds to the observation target data storage unit. The additional image data database 154 corresponds to the additional image data storage unit.

  The additional image data 120a selected from the additional image data database 154 is sent to the information superimposing terminal 100 by wireless communication. The additional image data acquisition unit 109 acquires the information about the observation target OBJ identified by the observation target recognition unit 106, here, the type of the dinosaur, and the muscle portion of the dinosaur skeleton model corresponding to the information about the type of the dinosaur Image data (additional image data 120a) is acquired. In addition to the function of specifying the observation target OBJ, the observation target recognition unit 106 has a function of recognizing the state of the specified observation target OBJ. The data of the observation target OBJ specified by the observation target recognition unit 106 and the information on the state of the recognized observation target OBJ are sent to the additional image data processing unit 111. The additional image data processing unit 111 processes the acquired additional image data 120a according to the state of the observation target OBJ. Here, “the state of the observation target OBJ” refers to the position, orientation, size, and the like of the observation target OBJ with respect to the user OBS. The additional image data acquisition unit 109 and the additional image data processing unit 111 constitute an additional image data processing unit 112 indicated by a dotted line. Thereby, the position, orientation, size, and the like of the additional image data 120a are processed according to the position, orientation, size, etc. of the observation object OBJ. The electronic information display unit 101 displays an additional image based on the processed additional image data 120a. When the user OBS observes the dinosaur from the side, the image is synthesized as shown in FIG. Similarly, when the user OBS observes the dinosaur from behind, the image is synthesized as shown in FIG. As a result, an additional image based on the additional image data 120a corresponding to the position, size, orientation, and the like of the observation target OBJ can be superimposed and displayed in the real field of view.

  In the present embodiment, the user OBS can observe the observation object OBJ through the transflective optical element 102. For this reason, the semi-transmissive optical element 102 can be downsized regardless of the size of the observation object OBJ. Thereby, the small information superimposition terminal 100 used for the additional image superimposition system mentioned above can be provided.

  Further, the user OBJ of the information superimposing terminal 100 can register additional image data to be superimposed in the server computer 150 in advance. The registration method includes a method using a numeric keypad (not shown) of the information superimposing terminal 100 and a method using an operation button (not shown).

  Further, it may be configured to select the additional image data 120a related to the observation target OBJ that the user OBS is gazing at. For example, the observation target OBJ that the user OBS is gazing at is detected using a line-of-sight detection technique. The observation object OBJ that the user OBS intentionally sees can be determined from the above-described image recognition result and the point of gaze position. As a result, an additional image based on the additional image data 120a can be superimposed and displayed on the observation target OBS that is being watched by the user OBS.

(Modification)
FIG. 7 shows a schematic configuration of an additional image superimposing system according to a modification of the present embodiment. In this modification, the additional image data acquisition unit 109 and the additional image data processing unit 111 are provided in the server computer 150. As described above, the additional image data 120a is processed and displayed in real time according to the state of the observation object OBJ in the real field of view. For example, the user OBS may move around the dinosaur skeleton model on display while observing using the information superimposing terminal 100. At this time, the additional image data 120a is processed and displayed in real time according to the state of the observation object OBJ that changes every moment. For this purpose, high-speed arithmetic processing capability is required. The server computer 150 that manages a large amount of electronic information has a high processing speed. Therefore, by providing the image processing unit in the server computer 150, it is possible to perform faster image processing.

  Further, the observation target data acquisition unit 105 and the observation target recognition unit 106 may be provided in the server computer 150. Thereby, the structure of the information superimposition terminal 100 can further be simplified.

  8A and 8B are diagrams for explaining images observed by the additional image superimposing system according to the second embodiment of the present invention. In FIG. 8A, the observation object OBJ is a girl (hereinafter referred to as “A child”). By using the information superimposing terminal 200, a user OBS (not shown) can superimpose an additional image based on the costume image data 120b on the real child A and observe it. The costume image data 120b is, for example, an image of a heel ear. The costume image data 120b corresponds to the additional image data. In FIG. 8-2, the observation object OBJ is a pet cat. By using the information superimposing terminal 200, a user OBS (not shown) can observe an additional image based on the additional image data 120c superimposed on a real-field cat. The additional image data 120c is, for example, an image of a lion's mane. Thus, the present embodiment is configured so that a so-called costume image can be observed. The same parts as those in the first embodiment are denoted by the same reference numerals, and redundant description is omitted.

In this embodiment, the child A, who is the observation object OBJ, can be set to the following (1) and (2).
(1) Costume content,
(2) User OBS that can observe the superimposed image

  FIG. 9 is a functional block diagram of the additional image superimposing system according to the present embodiment. FIG. 10 is a flowchart showing a procedure for displaying an additional image based on the additional image data 120b so as to be superimposed on the real field of view. First, based on FIG. 10, a basic procedure for superimposing and displaying a costume image will be described. Then, it demonstrates using the functional block diagram of FIG.

  In step S1001 in FIG. 10, a costume target is recognized. The costume target corresponds to the observation target OBJ. In step S1002, transformation data regarding what kind of costume is to be obtained is acquired. In step 1003, it is determined whether or not the costume image data is stored in a storage unit to be described later. The costume image data corresponds to the additional image data 120b. When the determination result of step S1003 is true (Yes), the process proceeds to step S1004. In step S1004, a costume image that is an additional image based on the additional image data 102b displayed in a superimposed manner is acquired. Furthermore, in step S1005, image data of the costume target that is the observation target OBJ is acquired. In step S1006, the costume image data is processed according to the state of the costume target. Then, the electronic information display unit 101 displays a costume image based on the processed additional image data 120b. The costume image based on the displayed costume image data 120b is displayed superimposed on the actual field of view by the transflective optical element 102.

  Next, the description will be continued with reference to FIG. The additional image superimposing system of the present embodiment includes an information superimposing terminal 200, an information superimposing terminal 300, and a server computer 400. The information superimposing terminal 200, the information superimposing terminal 300, and the server computer 400 are configured to be able to communicate with each other. Here, the information superimposing terminal 200 is a device used by the user OBS who observes the observation target OBJ. In contrast, the information superimposing terminal 300 is an apparatus used by Mr. A, shown in FIG.

  First, the description starts with the information superimposing terminal 200 used by the user OBS. The imaging unit 104 images A child who is the observation target OBJ. The costume target data acquisition unit 105 selects and acquires a part necessary for specifying the observation target OBJ from the captured image. The costume target recognition unit 206 identifies the observation target OBJ by image processing, for example, pattern matching processing. The costume target recognition unit 206 corresponds to the observation target recognition unit. In the case of the present embodiment, the costume target recognition unit 206 identifies and recognizes the person of child A who is the observation target OBJ. The communication unit 207 on the information superimposing terminal 200 side transmits / receives data between the communication unit 301 of the information superimposing terminal 300 and the communication unit 401 of the server computer 400. Data transmission / reception is controlled by the control units 208, 302, and 402.

  The information superimposing terminal 300 used by Mr. A, who is the observation target OBJ, includes a costume basic database 303. The costume basic database 303 includes a costume content setting unit 304, a user setting / determination unit 305, an area setting unit 306, and a time setting unit 307. The costume content setting unit 304 corresponds to the additional image content setting unit. The user setting / determination unit 305 corresponds to a user setting unit.

  In the costume content setting unit 304, the content of the costume image data 120b that is superimposed and displayed by the child A who is the observation target OBJ is set. In the present embodiment, the above-described image of the heel ear is set as the costume image data 120b.

  The user setting / determination unit 305 is set with information on the user OBS that can observe the composite image on which the costume image data 120b is superimposed and displayed by Mr. A, who is the observation target OBJ. For example, it is set so that only the friend and acquaintance of child A who is the observation object OBJ can observe the composite image in which the costume image data 120b is superimposed and displayed. Information on the user OBS using the information superimposing terminal 300 is registered in the server computer 400 in advance. When the user OBS is a friend of A child who is the observer OBJ, transmission / reception of the costume image data 120b is permitted. As a result, another person who is not related to the child A who is the observation object OBJ cannot observe the superimposed composite image.

  In the area setting unit 306, information on a region where a composite image on which the costume image data 120b is superimposed and displayed can be observed by Mr. A, who is the observation target OBJ. For example, it is set so that a composite image in which the costume image data 120b is superimposed and displayed can be observed only in the park of the theme park. As a result, the superimposed composite image cannot be observed outside the theme park.

  In the time setting unit 307, information of a time zone in which the child image A, who is the observation target OBJ, can observe the composite image on which the costume image data 120b is superimposed and displayed is set. For example, only in the morning time zone, it is set so that the synthesized image on which the costume image data 120b is superimposed and displayed can be observed. As a result, the superimposed composite image cannot be observed in the afternoon time zone.

  The server computer 400 includes a costume object database 403, a costume image database 404, and a transformation data database 405. The costume target database 403 stores information for specifying the person of child A who is the observation target OBJ. The costume image database 404 stores various additional image data to be displayed in a superimposed manner. The costume image database 404 corresponds to an additional image data storage unit. For example, image data of the ear of the eyelid that is the costume image data 120b is stored. The transformation data database 405 stores data to be transformed in the costume image. For example, “兎” in FIG. 8-2 and “Lion” in FIG. 8-2. Mr. A, who is the observation target OBJ, selects the target name that he / she wants to transform.

  The transformation data and the additional image data are not limited to costumes such as animals and people. For example, an additional image based on the aura-shaped additional image data can be superimposed and displayed on the contour portion of the child A in the real field of view. In addition, by limiting the user OBS that can observe the superimposed image, advertisements and messages can be superimposed and displayed near the A child in the real field of view. For example, it can be configured such that only a teenage user OBS can recognize a specific message when looking at Mr. A using the 3D information device 200. Further, the additional image data may be, for example, animated character image data or background image data.

  Returning to the information superimposing terminal 200, the description will be continued. The costume image data 120b selected from the costume image database 404 is sent to the information superimposing terminal 200 by wireless communication. The costume image data acquisition unit 209 acquires the costume image data 120b. The costume target recognition unit 206 has a function of recognizing the state of the specified observation target OBJ in addition to the function of specifying the observation target OBJ. The data about the identified observation object OBJ and the information on the state of the recognized observation object OBJ are sent to the costume image data processing unit 211 by the costume object recognition unit 206. The costume image data processing unit 211 processes the acquired costume image data 120b according to the state of the child A who is the observation object OBJ. The costume image data processing unit 211 corresponds to an additional image data processing unit. The costume image data acquisition unit 209 and the costume image data processing unit 211 constitute an additional image data processing unit 212 indicated by a dotted line. As a result, the position, orientation, size, and the like of the costume image data 120b are processed according to the position, orientation, size, etc. of the child A, who is the observation object OBJ. Then, the electronic information display unit 101 displays an additional image based on the processed costume image data 120b. As a result, when the user OBS permitted by Mr. A is observing Mr. A, the superimposed image can be observed as shown in FIG.

  In this embodiment, the costume image data 120b is superimposed and displayed in real time according to the state of the child A who is the observation object OBJ. For example, when the child A turns sideways with respect to the user OBS, the position, size, and orientation of the costume image data 120b change according to the state of the child A. As a result, it is possible to superimpose and display the costume image data corresponding to the position, size, orientation, etc. of the child A who is the observation object OBJ.

  FIG. 11 shows functional blocks of the additional image superimposing system according to the third embodiment of the present invention. The same parts as those in the first embodiment and the second embodiment are denoted by the same reference numerals, and redundant description is omitted.

In the present embodiment, the user OBS who observes the observation target OBJ can be set to the following (1) and (2).
(1) OBJ to be observed
(2) Contents of costume

  The information superimposing terminal 250 is a device used by the user OBS who observes the observation target OBJ. The information superimposing terminal 250 includes a costume basic database 303. In the costume content setting unit 304, the content of the costume image data 120b to be superimposed and displayed on the observation target OBS is set by the user OBS. In the present embodiment, the above-described image of the heel ear is set as the costume image data 120b.

  In the costume target setting unit 308, information for specifying the observation target OBJ to be costumed is set. For example, the costume image data 120b is set to be superimposed and displayed only for the child A who is the observation target OBJ. The costume target setting unit 308 corresponds to the observation target setting unit. Information for identifying A's person is stored in advance in the costume target database 403 of the server computer 400. When the user OBS observes the child A with the information superimposing terminal 250, transmission / reception of the costume image data 120b is permitted. As a result, the superimposed composite image cannot be observed even if another person who is not related to the child A who is the observation object OBJ is observed.

  FIG. 12 shows functional blocks of the additional image superimposing system according to the fourth embodiment of the present invention. The same parts as those in the first embodiment to the third embodiment are denoted by the same reference numerals, and redundant description is omitted. The other computer 500 is, for example, a computer operated by a theme park. Note that the server computer 400 described above may also have the functions of the other computer 500.

In this embodiment, the other computer 500 is configured to be able to set the following (1), (2), and (3).
(1) OBJ to be observed
(2) Contents of costumes,
(3) User OBS that can observe the superimposed image

  The other computer 500 includes a costume basic database 303. The costume basic database 303 includes the costume content setting unit 304, the costume target setting unit 308, the user setting / determination unit 305, the area setting unit 306, and the time setting unit 307 as described above.

  As a result, the other computer 500, for example, when the child A entered the theme park is the child A ’s birthday, the additional image data 120b for the child A in the field of view in the theme park. The additional image based on is superimposed and displayed. At this time, the other computer 500 can also set a user OBS that can observe the superimposed image.

  As another example, the other computer 500 counts and stores the number of times Mr. A has visited the theme park. Next, an additional image based on specific additional image data is displayed in a superimposed manner at the time of 10th visit or 20th visit. Then, the user OBS set by the other computer 500 can observe the superimposed composite image by the information superimposing terminal 200. In this example, the other computer 500 performs a costume target and a user setting. For this reason, for example, both the child A and the user OBS who are costume targets do not know the setting contents. For this reason, the user OBS can observe an unexpected superimposed image.

  The present invention is not limited to the configurations described in the above embodiments. For example, the various data stored in the server computer may be stored in the information superimposing terminal of the person who is the observation target OBJ or the information superimposing terminal of the user OBS. Further, the costume target recognition unit may be provided in the server computer.

  The area setting is not limited to the theme park. For example, a travel destination area or a specific prefecture can be set. Furthermore, the information displayed in a superimposed manner in the real field of view may display icons, symbols, etc. in a superimposed manner. In addition, the communication mode between the information superimposing terminal and the server computer may be any type capable of transferring data such as a PHS line, a mobile phone line, a wired LAN, a wireless LAN, and infrared communication.

  Further, in each of the above-described embodiments, an example in which the user OBS holds and uses the information superimposing terminal by hand is described. However, the present invention is not limited to an information superimposing terminal held by a hand, and for example, a head-mounted display, a spectacle-type face-mounted display, or a wristwatch-type device may have the above-described information superimposing terminal function. Thus, the present invention can be modified as appropriate without departing from the spirit of the present invention.

The information superimposing system of the present invention will be described below with reference to the drawings.
An information display system according to an embodiment of the present invention includes:
An imaging unit for imaging an observation object having related electronic information;
An image recognition unit that identifies an imaged observation object by image recognition;
An observation object information storage unit for storing electronic information and observation object information added in association with the electronic information;
An electronic information display unit for displaying predetermined image information;
A transflective optical element that transmits light from the user's line-of-sight direction real field of view and superimposes the predetermined image information displayed on the electronic information display unit on the real field of view;
A selection specifying means for selecting and specifying a specific observation object from a plurality of observation objects in the real field of view grasped through the semi-transmissive optical element;
A determination unit that determines whether observation object information is added to the observation object recognized by the image recognition unit;
When it is determined by the determination unit that the observation target information is added to the observation target, predetermined identification information is displayed on the electronic information display unit, and is superimposed on the actual field of the observation target. Information to be displayed and the observation object information added by the image display control unit in association with the specific observation object by specifying the specific observation object on which the identification information is displayed in a superimposed manner by the selection specifying means. Is displayed on the electronic information display unit, and is superimposed on the actual field of the observation object.

  Hereinafter, the information display system according to the present embodiment will be described in detail. Note that the present invention is not limited to the embodiments.

  FIG. 13 is a diagram showing a schematic configuration of the information display system according to the embodiment of the present invention. The user OBS uses the portable information terminal 100 to observe the observation object, here, the printed material 1110. There is no restriction | limiting in particular as said observation object, According to the objective, it can select suitably, For example, the building in real field of view, a book, the guidance map of a theme park, etc. are mentioned. In the following embodiment, an example of a theme park will be described. Further, the printed material 1110 is printed on a paper medium based on a digital block.

  Since the portable information terminal 100 can be the same as the information superimposing terminal described above, the description thereof is omitted.

  FIG. 14 is a flowchart showing a procedure for displaying the printed material information superimposed on the real field of view. FIG. 15 is a functional block diagram of the information display system according to the present embodiment. Hereinafter, the procedure of the information display system according to the present embodiment will be described with reference to FIG. In step S <b> 401 of FIG. 14, the imaging unit 104 images the printed material 1110. In step S <b> 402, the pattern matching unit 1105 performs pattern matching processing based on the image data of the printed material 1110. In step S402, the image recognition unit 1106 identifies the content of the printed material 1110 based on the pattern matching result. In step S404, the determination unit 126 determines whether or not printed material information is added to the specified printed material. In step S405, the semi-transmissive optical element 102 displays the identification information for identifying the specific print object for which the determination unit 126 has determined that the print object information is added superimposed on the actual field of view of the specific print object. . In step S406, the selection specifying unit 1108 allows the user to select and specify a specific printed matter on which the identification information is displayed. In step S407, information regarding a specific printed matter is acquired. In step S <b> 408, the acquired print information is displayed on the electronic information display unit 101. The displayed printed matter information is displayed superimposed on the real field of view by the transflective optical element 102.

  Next, the description will be continued with reference to FIG. In FIG. 15, the information terminal device 100 and the server computer 120 are configured to be capable of wireless communication. As described above, the server computer 120 is managed by the theme park operating company. The imaging unit 104 images the printed material 1110. The input / output unit 103 of the portable information terminal 100 transmits / receives data to / from the server computer 120. Image data of the printed material 1110 captured by the imaging unit 104 is transmitted to the server computer 120.

  The server computer 120 includes a pattern matching unit 1105, an image recognition unit 1106, a printed material information storage unit 122, and a determination unit 126. The printed material information storage unit 122 further includes a digital block 123, a block information database 124, and a latest information database 125. The pattern matching unit 1105 performs pattern matching processing based on the image data of the printed material 1110. The image recognition unit 1106 identifies the content of the printed material 1110 based on the pattern matching result. The determination unit 126 determines whether or not printed material information is added to the specified printed material 1110.

  The theme park management company prints and publishes many other books, maps, etc. on paper media in addition to this theme park guide map. It is a database stored as electronic information. In addition, the composition information database 124 stores electronic information for managing the digital composition 123. Further, the latest information database 125 stores printed material information added in association with electronic information. Specifically, in the latest information database 125, for example, information on the current date and time of the theme park is updated and stored in real time.

  In general, books, maps, and the like issued by the digital version 123 are very large. In particular, in a large-scale theme park, the information stored in the digital version 123 and the version information database 124 may range from hundreds to tens of thousands of types. In this case, it is necessary to refer to and compare a very large amount of electronic information data when specifying the printed material 1110 captured by the imaging unit 104 by image recognition. In this embodiment, an image recognition unit 1106 is provided in the server computer 120. The server computer 120 that manages a large amount of electronic information has a high processing speed. The image recognition unit 1106 in the server computer 120 refers to a plurality of pieces of electronic information stored in the digital block 123 and the block information database 124. Then, the image recognition unit 1106 searches the digital block 123 and the block copy information database 124 for data that matches the image data captured by the image capturing unit 104. Thereby, the printed material 1110 can be specified with high efficiency.

  Next, the determination unit 108 determines whether or not printed material information is added to a specific observation object in the specified printed material. Here, the printed material information is information stored in the latest information database 125. For example, the current waiting time (congestion status) of the vehicle in the theme park, or the location of the event held on the day in the theme park And holding time. Normally, the printed matter information is associated with the information of the printed matter 1110 stored in the digital block 123, so that when the image recognition unit 1106 and the pattern matching unit 1105 search the composition information database 124, At the same time, it is determined whether or not printed material information is added.

  Next, when the printed material information is added to a specific observation object in the printed material, an identification signal for specifying the observation object to which the printed material information is added is sent to the portable information via the input / output unit 103. It is transmitted to the terminal 100 by radio.

The portable information terminal 100 to which the identification signal is transmitted causes the electronic information display unit 101 to display identification information for identifying a specific observation object in the printed material. The displayed identification information is displayed by the transflective optical element 102 so as to be superimposed on a specific observation object in the printed material being observed.
FIG. 13 shows an example in which identification information for indicating that printed matter information is added to an attraction 1111 in the printed matter 1110 is displayed. It is also possible to display a plurality of identification information in the printed material.

The portable information terminal 100 includes a selection specifying unit 1108 for selecting and specifying a specific target in the printed material 1110 on which the identification information is displayed. The selection specifying unit 1108 is not particularly limited as long as it can select and specify a specific target in the printed material 1110 on which the identification information is displayed from the portable information terminal 100 side, and may be appropriately selected according to the purpose. For example, an item selected indirectly via a display screen, an item selected directly by an optical probe, or the like can be used. As the selection specifying unit 1108 in the portable information terminal 100 of the present embodiment, as shown in FIG. 13, the selection specifying unit 1108 is displayed by being superimposed on a real field printed matter via the electronic information display unit 101 and the semi-transmissive optical element 102. An operation button 1010 for moving the cursor 1112 to a predetermined position is provided. The operation button 1010 has a function of moving the cursor 1112 to a predetermined position and a function of selecting a specific target when the specific target and the cursor 1112 coincide in the real field of view. Accordingly, as shown in FIG. 13, the user who recognizes that the identification information notifying that the printed material information is added to the attraction 1111 in the printed material 1110 is displayed, the cursor 1112 is moved to the attraction 1111 by the operation button 1010. To obtain the printed material information on the attraction 1111 can be selected and specified.
Furthermore, as a modification of the selection specifying unit 1108, there is a method of selecting and specifying an observation object on which identification information is displayed without using the cursor 1112. In this case, when the identification information is displayed on a plurality of attractions, the operation button 1010 is appropriately operated to display the color of the identification information superimposed on a specific observation target, here a specific attraction, Select a specific attraction by changing the blinking pattern. Next, for example, a specific attraction is specified by a determination button common to or different from the operation button 1010.
Further, as another modification example of the selection specifying unit 1108, there is a method of selecting and specifying an observation object using a target mark displayed in a superimposed manner in an actual visual field. In this case, the target mark is displayed at the center of the electronic information display unit 101, and the image information of the target mark is displayed in a superimposed manner in the actual visual field. Next, the user moves the portable information terminal 100 to move the target mark onto a specific attraction where the identification information is displayed. At this time, the target mark is always displayed on the electronic information display unit 101 so that the target mark can be observed at the center of the transflective optical element 102. Even if the user moves the portable information terminal 100, the identification information is always displayed. And displayed on the electronic information display unit 101 so as to be superimposed on the position of the observation object on the real field of view.

  Next, when the user OBS selects a specific printed material object in the printed material, that is, the attraction 1111, the printed material information associated with the specific printed material object is acquired from the latest information database 125. The latest print information to be acquired is, for example, the current waiting time (congestion status) of vehicles without the theme park, the location and time of events held on the day in the theme park, etc. The latest printed matter information is wirelessly sent to the portable information terminal 100 via the acquisition output unit 103.

  The electronic information display unit 101 displays printed material information acquired from the database. The displayed image of printed matter information is reflected by the transflective optical element 102 in the direction of the user OBS. The user OBS can confirm the latest information by superimposing the printed material 1110 while observing the printed material 1110 in a real field of view. For example, in this embodiment, as shown in FIG. 16, real-time character information “vehicle A: wait 5 minutes” and “vehicle B 15 minutes” is displayed in a superimposed manner while viewing the guidance map of the theme park. . For this reason, the user can identify the presence or absence of the printed matter information in the printed matter by the identification information, and can acquire the printed matter information and display it superimposed only when necessary. As described above, the user OBS can obtain the printed matter information quickly and easily only when necessary.

Hereinafter, the information display system will be described with reference to the drawings.
An information display system according to an embodiment of the present invention includes:
An information acquisition unit for acquiring image information and electronic information associated with the image information;
An image information storage unit for storing image information;
An electronic information storage unit for storing electronic information;
A template data creation unit for creating template data from image information;
A template database that stores the template data created by the template data creation unit;
A template data collation unit that collates whether the template data created by the template data creation unit matches the template data in the template database by image recognition;
If it is determined that the template data created by the template data creation unit does not match the data in the template database, the template data created by the template data creation unit is stored in the template database, and the template data creation unit Identification data for creating image information obtained by adding image identification data to image information corresponding to the template data created by the template data creation unit when it is determined that the created template data matches the data in the template database A processing unit;
An imaging unit that captures an image reproduced based on the image information stored in the image information storage unit;
An image recognition unit for collating and identifying an image captured by the imaging unit by image recognition;
An electronic information display unit that displays electronic information associated with at least the identified image;
An information display system including a transflective optical element that transmits light from an actual visual field of a user's line of sight and displays electronic information displayed on an electronic information display unit in a superimposed manner in the actual visual field.

  Hereinafter, the information display system according to the present embodiment will be described in detail. Note that the present invention is not limited to the embodiments.

FIG. 17 is a diagram showing a schematic configuration of the information display system according to the embodiment of the present invention.
A mobile terminal 30 connected to a network (not shown) has a plurality of image information (image 1 to image n) 10 such as photographs and a plurality of text data associated with the image information, for example, character information, via the network. Electronic information such as (text1 to textn) 20 is acquired. The acquired image information (image 1 to image n) and electronic information (text 1 to text n) are stored in the database 120.
The acquired image information such as a photograph is printed as printed matter using the printer 40. The printed matter that has been printed is observed using the portable information terminal 100. When the user observes the printed matter through the semi-transmissive optical element 102 provided in the portable information terminal 100, electronic information such as character information associated with the printed matter is displayed superimposed on the real field of view of the printed matter. .

  Since the portable information terminal 100 can be the same as the information superimposing terminal described above, the description thereof is omitted.

FIG. 18 is a flowchart illustrating a procedure for displaying the printed material information superimposed on the real field of view. FIG. 19 is a functional block diagram of the information display system according to the present embodiment. Hereinafter, the procedure of the information display system according to the present embodiment will be described with reference to FIG.
In step S501 in FIG. 18, the information acquisition unit 505 acquires image information such as a photograph and electronic information such as a sentence associated with the image information. In step S502, the template data creation unit 506 creates template data from the acquired image information. In step S504, the template data matching unit in the recognition unit 507 matches the template data created by the template data creation unit 506 with the template data stored in advance in the template database (template DB). It is collated by image recognition. If it is determined from the matching result of the template data matching unit that the template data created by the template data creating unit 506 cannot be identified by the template DB, in step S503, the processing in the processing unit 508 is performed. The identification data processing unit creates image information obtained by adding image identification data to image information corresponding to the template data created by the template data creation unit 506.
Next, in step S <b> 505, the imaging unit 104 provided in the portable information terminal 100 captures an image such as a photograph reproduced based on the image information acquired by the information acquisition unit 505. Next, in steps S506 and S507, the processing unit 508 having a pattern matching unit and an image recognition unit (not shown) identifies image information from the captured image.
Next, in step S508, electronic information such as text relating to the specified image information is acquired. In step S509, the acquired electronic information is displayed on the electronic information display unit 101. The displayed printed matter information is displayed superimposed on the actual field of view by the transflective optical element 102.

Next, the description will be continued with reference to FIG. In FIG. 19, the information terminal device 100 and the server computer 120 are configured to be capable of wireless communication. The server computer 120 includes an information acquisition unit 505 that acquires image information such as a photograph and electronic information such as a sentence associated with the image information from the network via the information terminal device 100 or not via the information terminal device 100. Whether the information storage unit 222, the template data generation unit 506 that generates the template data 506 from the image information, and the template data generated by the template data generation unit 506 match the template data stored in the database A recognition unit 507 having a template data collation unit that collates whether or not by image recognition, and the recognition unit 507 determines that the template data created by the template data creation unit 506 does not match the data in the template database. Information acquisition unit 505 The image data is stored in the information storage unit 222, and the template data created by the template data creation unit 506 is created by the template data creation unit 506 when it is determined that the template data matches the data in the template database. From the identification data processing unit 508 that creates image information with image identification data added so that the image information corresponding to the template data can be identified from other image information already stored in the information storage unit 222. It is configured. The recognizing unit 507 includes an image recognizing unit that collates and identifies an image captured by the image capturing unit 104 by image recognition.
The information storage unit 222 further includes an image information storage unit 223 that stores image information, an electronic information storage unit 224 that stores electronic information associated with the image information, and template data created by the template data creation unit 506. And a template database to be stored.

When the information acquisition unit 505 is provided in the information terminal device 100, the user uses the information terminal device 100 to, for example, image information such as a photograph and text data associated with the photograph, that is, a sentence or the like. Get electronic information. Of the acquired information, electronic information such as text is stored in the electronic information storage unit 224 of the server computer 120. Next, the template data creation unit 506 of the server computer 120 creates template data from image information such as a photograph. Next, the template data collation unit 507 collates the template data created by the template data creation unit 506 by image recognition to determine whether the template data matches the template data stored in the template database 225.
When it is determined that the template data created by the template data creation unit 506 does not match the data in the template database 225 as a result of matching by the template data matching unit 507, the image information corresponding to the matched template is image information. While being stored in the storage unit 223, template data created by the template data creation unit 506 from the image information is stored in the template database 225.
On the other hand, if it is determined that the template data created by the template data creation unit 506 matches the data in the template database 225 as a result of matching by the template data matching unit 507, the image information corresponding to the matched template is image data. The identification data processing unit 508 adds image identification data to the image information corresponding to the collated template so as to be distinguishable from other image information stored in the information storage unit 223. Then, the image information to which the image identification data is added is stored in the image information storage unit 223, and the template data created by the template data creation unit 506 from the image information to which the image identification data is added is the template database 225. It is stored in. As described above, all pieces of image information such as photographs acquired by the information acquisition unit 505 can be easily identified by image recognition. The configuration of the server computer 120 is not limited to the above configuration, and can be appropriately selected according to the purpose. For example, the information terminal device 100 may have all the same functions as the server computer 120.

  Here, the image identification data is not particularly limited and can be appropriately selected according to the purpose. For example, the image identification data is displayed on the outer frame portion of a photograph that displays a colored image frame around the photograph or the like. Examples are those that display specific characters, symbols, patterns, or the like. Moreover, there is no restriction | limiting in particular as electronic information linked | related with the said image information, According to the objective, it can select suitably, For example, a text, an audio | voice, a music etc. are mentioned.

  A user of the information display system according to the present embodiment prints specific image information stored in the image information storage unit 223 of the server computer 120, for example, a photograph or the like, using an inkjet printer or the like. For example, the image information may be printed by a cradle-type printer device for a portable terminal that includes a connection unit that can be connected to the information terminal 100.

  In the example shown in FIG. 19, the electronic information of the document 113T is stored in the electronic information storage unit 224 of the server computer 120 in association with the printed photograph 113G. Further, electronic information is not associated with the printed photograph 114G.

  Then, when the user of the information display system images the photograph 113G with the imaging unit 104 provided in the information terminal 100, pattern matching processing is performed based on the image data of the photograph 113G. The image recognition unit 507 specifies the content of the photograph 113G based on the result of pattern matching. The processing unit 508 determines whether or not electronic information is added to the specified photograph 113G. If electronic information is added, the electronic information, here, the information of the electronic information 113T is stored in the electronic information storage. Acquired from the unit 224.

  Next, the acquired electronic information 113T is wirelessly transmitted to the portable information terminal 100 via the communication control unit 505.

  The portable information terminal 100 to which the electronic information 113T is sent displays the information of the electronic information 113T, here, the text on the electronic information display unit 101. The displayed image such as a sentence is reflected by the transflective optical element 102 in the direction of the user OBS. The user OBS can confirm by superimposing the sentence 113T while observing the photograph 113G in a real field of view.

Hereinafter, modifications of the information terminal device of the present invention will be described with reference to the drawings.
As illustrated in FIG. 20, the information terminal device 100 according to the present modification includes a main body 100A and an optical function unit 100B connected to the main body 100A. The main unit 100A is a portable information terminal that can be connected to a network such as a cellular phone. In addition, the main body 100A includes a connector 110B in which terminals for transmitting / receiving data to / from an external device or database are arranged. The main body 100A can be folded as shown in FIGS. 20 (a) and 20 (b).

  On the other hand, the optical function unit 100B includes an electronic information display unit 104, a transflective optical element 102, and a housing 110. The electronic information display unit 104 and the transflective optical element 102 are rotatably connected to the housing 1200 via hinges. For this reason, when not in use, the electronic information display unit 104 and the semi-transmissive optical element 102 are folded compactly as shown in FIG. 20C, and when used, as shown in FIG. 20D. Can be arranged facing each other with a predetermined interval. The optical function unit 100B includes a connection unit 110A that can be electrically connected to a connector 110B disposed in the main body unit 100A. Therefore, by connecting the connecting part 110A of the optical function part 110B to the connector 110B of the main body part 110A, the image data on the main body part 110A side can be displayed by the electronic information display part 104.

  As described above, the additional image superimposing system according to the present invention is useful for superimposed display when the observation target and the user move relatively.

1 is a diagram illustrating a schematic configuration of an additional image superposition system according to Embodiment 1. FIG. It is a figure which shows the isometric view structure of the information superimposition terminal of Example 1. FIG. It is another figure which shows the isometric view structure of the information superimposition terminal of Example 1. FIG. It is a figure which shows schematic structure of the information superimposition terminal of Example 1. FIG. FIG. 6 is a diagram illustrating a composite image superimposed and displayed according to the first embodiment. It is another figure which shows the composite image by which the superimposed display of Example 1 was carried out. It is a figure which shows the functional block of the additional image superimposition system which concerns on Example 1. FIG. 3 is a flowchart illustrating a procedure of superimposed display according to the first embodiment. It is a figure which shows the functional block of the additional image superimposition system which concerns on the modification of Example 1. FIG. It is a figure which shows the synthesized image by which the superimposed display of Example 2 was carried out. It is another figure which shows the synthesized image by which the superimposed display of Example 2 was carried out. It is a figure which shows the functional block of the additional image superimposition system which concerns on Example 2. FIG. 3 is a flowchart illustrating a procedure of superimposed display according to the first embodiment. It is a figure which shows the functional block of the additional image superimposition system which concerns on Example 3. FIG. It is a figure which shows the functional block of the additional image superimposition system which concerns on Example 4. FIG. It is a figure which shows schematic structure of the information display system which concerns on another Example. It is a flowchart which shows the procedure of the superimposed display of printed matter information. It is a figure which shows the functional block of an information display system. It is another figure which shows schematic structure of an information display system. It is a figure which shows schematic structure of the information display system which concerns on another Example. It is a flowchart which shows the procedure of the superimposed display of printed matter information. It is a figure which shows the functional block of an information display system. It is a figure which shows the modification of a portable information terminal.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Information superimposition terminal 101 Electronic information display part 102 Transflective optical element 104 Imaging part 105 Observation object data acquisition part 106 Observation object recognition part 107 Communication part 108 Control part 109 Additional image data acquisition part 111 Additional image data processing part 110 Main body part 112 , 212 Additional image data processing unit 120a, 120b, 120c Additional image data 150 Server computer 151 Communication unit 152 Control unit 153 Observation target database 154 Additional image data / database 200 Information superimposing terminal 205 Kigurumi target data acquisition unit 206 Kigurumi target recognition unit 207 Communication unit 208 Control unit 209 Kigurumi image data acquisition unit 211 Kigurumi image data processing unit 250, 300 Information superimposing terminal 301 Communication unit 302 Control unit 303 Kigurumi basic database 3 04 Kigurumi content setting unit 305 User setting / determination unit 306 Area setting unit 307 Time setting unit 308 Kigurumi target setting unit 400 Server computer 401 Communication unit 402 Control unit 403 Kigurumi target database 404 Kigurumi image database 405 Transformation data database 500 Others Computer OBJ Observation object OBS User 1110 Printed material 1111 Attraction 1112 Cursor 1010 Operation button 1108 Selection specifying unit 1105 Pattern matching unit 1106 Image recognition unit 126 Judgment unit 1107 Display information determination unit 10 Image information 20 Character information 30 Portable terminal 40 Printer 113T Document 113G 114G Photo 505 Information acquisition / communication control unit 506 Template data creation unit 507 Recognition unit 508 Identification data Data processing unit 222 information storage unit 223 image information storage unit 224 the electronic information storing part 225 Template DB
100A body part 100B connector 1200 housing

Claims (8)

An additional image superimposing system for displaying an additional image superimposed on an observation target in a real field of view,
An information superimposing terminal including at least a semi-transmissive optical element that superimposes and displays an additional image displayed on the electronic information display unit, and an additional image displayed on the electronic information display unit. When,
An observation object recognition unit that identifies the observation object imaged by the imaging unit by image recognition;
An additional image data processing unit that acquires additional image data associated with the observation target and processes the additional image data according to the state of the observation target;
An additional image superposition system, wherein an additional image based on the additional image data processed by the additional image data processing unit is displayed on the electronic information display unit.   The additional image superimposing system according to claim 1, further comprising an additional image content setting unit configured to set the content of the additional image superimposed on the observation target.   The additional image superimposing system according to claim 1, further comprising an observation target setting unit configured to set the observation target on which the additional image is superimposed. An observation target data storage unit for storing observation target data for the observation target recognition unit to recognize the observation target;
An additional image data storage unit that stores the additional image data associated with the observation target;
The observation object data storage unit and the additional image data storage unit are provided in a server computer capable of communicating with the information superimposing terminal,
The other computer or the server computer that can communicate with the information superimposing terminal includes the additional image content setting unit and the additional image data storage unit. Additional image superposition system.   5. The additional image superimposing system according to claim 1, further comprising a user setting unit configured to set a user who can display and observe the additional image in a superimposed manner.   The additional image superimposing system according to claim 1, further comprising a time setting unit that sets a time zone during which the additional image can be displayed and observed in a superimposed manner.   The additional image superimposing system according to claim 1, further comprising an area setting unit configured to set a region where the additional image can be displayed and observed in a superimposed manner.   The additional image superimposing system according to claim 1, wherein the additional image is a costume image superimposed and displayed on the observation target.

Images