JP2011130386A - Image display device, image display method, and portable terminal - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image display device in which contents intended by a user can be displayed while being superimposed on a captured image of a real space, without any complicated work. <P>SOLUTION: An image display device includes: an imaging section 1 for imaging a subject to produce captured image data; a QR code-reading section 5 for reading retrieval condition data representing retrieval conditions of content data from a QR code contained in the captured image data; a content retrieving section 6 for acquiring from a content DB 12 content data meeting the retrieval conditions based on the retrieval condition data read by the QR code-reading section 5; a content combining section 8 for combining the content data acquired by the content retrieving section 6 and the captured image data to produce display image data for combining contents based on the content data with an image based on the captured image data to display the combined content; and a display control section 9 for displaying an image based on the display image data on a display section 10. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image display apparatus and method, and a portable terminal.

  A card with an identification image (two-dimensional code such as QR code (registered trademark)) is imaged with a digital camera, the identification image is detected from the image data obtained by this imaging, and the card is detected from the detected identification image. Patent Document 1 discloses a method for detecting identification information.

  Then, by developing this method, a captured image obtained by imaging with a digital camera is displayed on the monitor, and an object corresponding to the detected identification information is displayed on the card image displayed on the monitor. An example of superimposing and displaying a three-dimensional image is disclosed in Patent Document 2. According to this method, it is possible to express a visual effect that fuses real space and virtual space.

JP 2000-82108 A JP 2000-322602 A

  However, the method disclosed in Patent Document 2 requires that the card identification image and the corresponding object have a one-to-one correspondence, so that the object to be superimposed on the card image is changed to another one. To do this, it is necessary to newly associate the card identification image with the object, which is troublesome. Also, the card creator needs to create a desired object to be superimposed on the card or to select a desired object from a plurality of objects at the time of card creation. It is troublesome.

  The present invention has been made in view of the above circumstances, and provides an image display apparatus and method, and a portable terminal capable of superimposing and displaying content intended by a user on a captured image without troublesome work. The purpose is to do.

  An image display device of the present invention includes an imaging unit that captures an image of a subject and generates captured image data, and a search condition reading that reads the search condition from search condition data that represents a search condition for content data included in the captured image data. A content data acquisition unit that acquires from the database content data that matches the search condition read by the search condition reading unit, and the content data acquired by the content data acquisition unit and the captured image data Then, an image composition unit that generates display image data for combining and displaying content based on the content data on an image based on the captured image data, and a display that displays an image based on the display image data on the display unit And a control unit.

  The portable terminal of this invention is provided with each part of the said image display apparatus, the said display part, and the communication part connected to the said database via a network.

  The image display method of the present invention includes an imaging step of imaging a subject to generate captured image data, and a search condition reading for reading the search condition from search condition data representing a search condition for content data included in the captured image data. A content data acquisition step of acquiring content data that matches the search condition from a database; and combining the content data and the captured image data to add content based on the content data to an image based on the captured image data An image composition step for generating display image data for composite display, and a display control step for displaying an image based on the display image data on a display unit.

  According to the present invention, it is possible to provide an image display apparatus and method and a portable terminal capable of displaying content intended by a user in a superimposed manner on a captured image without troublesome work.

The figure for demonstrating the outline of the image display system which is one Embodiment of this invention. The figure which shows schematic structure of the image display system for describing one Embodiment of this invention The figure explaining the process of the marker detection part in the system shown in FIG. 2, a marker image extraction part, and a marker image normalization part The figure which showed an example of the table registered into content DB in the image display system shown in FIG. The figure explaining an example of the service using the image display system shown in FIG. The figure which showed the example of a screen display at the time of making a user select content by the image display system shown in FIG. The figure explaining another example of the service using the image display system shown in FIG. The figure which showed the 1st modification of the image display system shown in FIG. The figure explaining the example of the service using the image display system shown in FIG. The figure which showed the 2nd modification of the image display system shown in FIG. The figure which showed the 3rd modification of the image display system shown in FIG. The figure which showed the 4th modification of the image display system shown in FIG.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a diagram for explaining an outline of an image display system according to an embodiment of the present invention. In this image display system, a plurality of content data is registered in a database, and an object B such as a card on which a QR code A that encodes a search condition for searching for optimal content data from the database is pasted or printed. Create The object B has a marker C on its contour. The object B is a Christmas card, a direct mail for advertising, a photo album, or the like. When the object B is imaged by the imaging unit, the marker C is detected from the captured image, and a range surrounded by the marker C is extracted as an image of the object B. Further, the QR code included in the image of the object B is recognized, and the content data matching the search condition indicated by the QR code is acquired from the database. Then, the image B 'of the object B and the content D that matches the search condition are combined and displayed. Hereinafter, a specific system configuration will be described.

  FIG. 2 is a diagram showing a schematic configuration of an image display system for explaining an embodiment of the present invention. The image display system shown in FIG. 2 includes an image display device 100 and a content database 12.

  The image display device 100 includes an imaging unit 1, a marker detection unit 2, a marker image extraction unit 3, a marker image normalization unit 4, a QR code reading unit 5, a content search unit 6, a communication unit 7, The content composition unit 8, the display control unit 9, and the display unit 10 are provided. A network 11 such as the Internet is connected to the communication unit 7, and a content database (DB) 12 is connected to the network 11.

  The imaging unit 1 captures a subject and generates captured image data. The imaging unit 1 performs signal processing on an imaging system output from an optical system such as an imaging lens and a diaphragm, an imaging element, and the imaging element ( And a signal processing unit for generating data in which one pixel has red, blue, and green color components.

  As shown in FIG. 3, the marker detection unit 2 detects a marker C provided on the contour of the object B from captured image data 30 obtained by imaging the object B shown in FIG. 1 by the imaging unit 1. As a method of detecting the marker C, for example, a method of performing matching processing with a template image of a marker registered in advance, a method of registering a marker shape, and extracting a subject having the shape by contour extraction processing, etc. Adopt it.

  As shown in FIG. 3, the marker image extraction unit 3 extracts the marker C detected by the marker detection unit 2 and the data in the range inside the marker image data 31 from the captured image data. The marker image data 31 includes data (search condition data representing content search conditions) corresponding to the QR code A shown in FIG.

  The marker image normalization unit 4 normalizes the marker image data extracted by the marker image extraction unit 3. Specifically, as shown in FIG. 3, the marker image data 31 is deformed from a diamond shape to a square shape. The normalized marker image data 31 has a form in which the QR code A can be read.

  The QR code reading unit 5 extracts data corresponding to the QR code A included therein from the marker image data 31 normalized by the marker image normalizing unit 4, and decodes the QR code A from the extracted data. Read the search condition.

  The communication unit 7 is a communication interface that transmits and receives data by connecting to the network 11 by wire or wireless.

  The content search unit 6 acquires content data that matches the search condition from the content DB 12 via the communication unit 7 in accordance with the search condition read by the QR code reading unit 5.

  The content synthesizing unit 8 synthesizes the content data searched by the content search unit 6 and the captured image data obtained from the imaging unit 1, and combines the content based on the content data and the image based on the captured image data. Display image data to be combined and displayed on the display unit 10 is generated.

  The display control unit 9 outputs the display image data generated by the content composition unit 8 to the display unit 10 and causes the display unit 10 to display an image based on the display image data.

  The display unit 10 displays an image using a display element such as an organic EL element or a liquid crystal display element. The display unit 10 may be separate from the image playback device 100.

  The content DB 12 registers a plurality of content data to be combined with captured image data by the content combining unit 8. New content data can be registered in the content DB 12 at any time. The content DB 12 may be built in the image display device 100.

  Next, a specific example of a service using the image display system will be described. In the following description, it is assumed that a table as shown in FIG. 4 is registered in the content DB 12.

  FIG. 4 is a diagram showing an example of a table registered in the content DB in the image display system shown in FIG. As shown in FIG. 4, a table 40 in which 2D or 3D image data is registered and a table 41 in which 3D polygon data is registered are recorded in the content DB 12.

  Different image data G1 to G6 are registered in the table 40. In each of the image data G1 to G6, the attribute information indicating the contents in the form of text and the registration indicating the date when the data was registered are registered. Date information is registered in association with each other.

  In the example of FIG. 4, since the image data G1 is Santa Claus image data, text data “Santa Claus” indicating the content is registered as attribute information of the image data G1. Further, since the image data G3 and G6 are image data related to Christmas such as a Christmas tree and Santa Claus, text data “Christmas” indicating the contents is registered as attribute information of the image data G3 and G6. Further, since the image data G2 and G4 are car image data, text data “Car Name” indicating the contents (car name, etc.) is registered as attribute information of the image data G2 and G4. The image data G4 is, for example, data obtained by slightly editing the image data G2, and the attribute information is the same as the image data G2. However, since the image data G4 is registered after the image data G2, the registration date of the image data G4 is newer.

  Different polygon data P1 to P6 are registered in the table 41, and each piece of polygon data P1 to P6 is registered in association with attribute information indicating the contents in text format.

  In the example of FIG. 4, since the polygon data P1 is data modeled on an infant girl, text data “Infant woman” indicating the content is registered as attribute information of the polygon data P1. Further, since the polygon data P2 is data modeled on an infant boy, text data “infant man” indicating the content is registered as attribute information of the polygon data P2. Further, since the polygon data P3 is data modeled on a middle-aged man, text data “middle-aged man” indicating the content is registered as attribute information of the polygon data P3. Further, since the polygon data P4 is data modeled on a middle-aged woman, text data “middle-aged woman” indicating the content is registered as attribute information of the polygon data P4. Further, since the polygon data P5 is data modeled on an elementary school boy, text data “elementary man” indicating the content is registered as attribute information of the polygon data P5. Further, since the polygon data P6 is data modeled on an elementary school girl, text data “elementary school girl” indicating the content is registered as attribute information of the polygon data P6.

  FIG. 5 is a diagram for explaining an example of a service using the image display system shown in FIG. In this example, a Christmas card 50 on which a marker 51 and a QR code 52 are printed is prepared. The QR code 52 includes the word “Santa Claus” or “Christmas” related to the Christmas card in the attribute information among the content data registered in the content DB 12 and searches for content data that is image data. The search condition is coded.

  A person who receives the Christmas card 50 images the Christmas card 50 with the imaging unit 1 of the image display device 100. When the imaging is completed, the image display apparatus 100 extracts a portion corresponding to the marker 51 from the captured image data obtained by the imaging, and extracts the portion and data inside it as marker image data. Subsequently, the image display apparatus 100 extracts a portion corresponding to the QR code 52 from the marker image data, and reads the search condition by decoding the QR code 52 therefrom. Next, the image display apparatus 100 acquires appropriate content data from the content DB 12 according to the read search condition.

  If there is only one content data that matches the search condition, the image display apparatus 100 acquires the content data. If there is a plurality of content data that matches the search condition, one of the content data is acquired.

  In the example of FIG. 5, the search condition is “retrieve content data including the word“ Santa Claus ”or“ Christmas ”in the attribute information among the content data registered in the content DB 12”. For this reason, there are a plurality of pieces of content data matching the search condition, such as image data G1, G3, and G6. The content search unit 6 selects one from the plurality of image data. As a selection method, there are a method of selecting one randomly from a plurality of content data, a method of selecting the latest registration date, and the like. Or you may make it select what was selected by the user of the apparatus from several content data which satisfy | fills search conditions.

  For example, the content search unit 6 acquires all the image data G1, G3, and G6. Then, as shown in FIG. 6, the content search unit 6 transmits an image g1 based on the image data G1, an image g3 based on the image data G3, and an image g6 based on the image data G6 via the display control unit 9. Displayed on the display unit 10 simultaneously. Then, when any of the displayed images is selected by the user using an operation unit (not shown) provided in the image display device 100, the content search unit 6 selects image data corresponding to the selected image. The result is output to the content composition unit 8 as a search result.

  Thus, when one content data is searched by the content search unit 6, the content data and the captured image data are combined to generate display image data. Finally, an image based on the display image data is displayed on the display unit 10. When the content data searched by the content search unit 6 is the image data G1, as shown in FIG. 5, the captured image 53 including the marker image 50 ′ corresponding to the Christmas card 50 is based on the image data G1. A Santa Claus image g1 is superimposed and displayed.

  FIG. 7 is a diagram for explaining another example of a service using the image display system shown in FIG. In this example, in order to promote a new product, a catalog 70 in which a marker 71 and a QR code 72 are printed is placed in a direct mail sent by a car dealer to a large number of customers. The QR code 72 includes, in the content data registered in the content DB 12, the word “Car Name”, which is the name of the vehicle to be advertised, in the attribute information, the registration date is the latest, and A search condition for searching for content data as image data is coded.

  The customer who has received the direct mail images the catalog 70 included in the direct mail with the imaging unit 1 of the image display device 100. When the imaging is completed, the image display apparatus 100 extracts a portion corresponding to the marker 71 from the captured image data obtained by the imaging, and extracts the portion and data inside it as marker image data. Subsequently, the image display apparatus 100 extracts a portion corresponding to the QR code 72 from the marker image data, decodes the QR code 72 therefrom, and reads the search condition. Next, the image display apparatus 100 acquires appropriate content data from the content DB 12 according to the read search condition.

  Two pieces of content data including “Car Name” as attribute information are registered in the content DB 12. However, the content search unit 6 selects the one with the latest registration date (image data G4) from the two content DBs 12. Get from.

  Next, the acquired content data and captured image data are combined to generate display image data. Finally, an image based on the display image data is displayed on the display unit 10. As a result, as shown in FIG. 7, the car image g4 based on the image data G4 is superimposed and displayed on the captured image 74 including the marker image 70 'corresponding to the catalog 70.

  As described above, according to this image display system, a creator of an article such as a Christmas card or a catalog simply embeds a QR code that encodes a search condition in the article, and an image obtained by imaging the article The content intended by the creator can be superimposed and displayed. As a result, the example shown in FIG. 5 can give a surprise to the person who received the Christmas card, and even a Christmas card that cannot be realized in the real space can be realized virtually, You can create value-added Christmas cards. Further, in the example shown in FIG. 7, the vehicle to be advertised can be presented to the customer in a form closer to the reality than the catalog, and an effect of increasing the advertising effect of direct mail can be expected. Further, as in the example shown in FIG. 7, by setting a condition that the registration date is the latest as the search condition, it is possible to always display the latest content in a superimposed manner. For this reason, it is not necessary to change the QR code every time the product content is changed, and a simple system can be realized.

  Conventionally, in order to display a desired content superimposed on an image of an article, the desired content is first created or the desired content is selected from a large number of contents, and then the desired content is selected. It was necessary to create a QR code corresponding to the content on a one-to-one basis and embed this QR code in the article. On the other hand, in this image display system, even if the content to be displayed in a superimposed manner is not clearly determined, it is only necessary to embed a search condition for roughly specifying the content to be displayed in a superimposed manner as a QR code. It is possible to superimpose and display content in line with the intention. In other words, since the creator of the article can freely create a QR code regardless of the data registered in the content DB 12, it is possible to create an article with added value without troublesome work. Also, the content data registered in the content DB 12 can be freely added regardless of the QR code, so that the content data can be easily managed.

  For example, in the example of FIG. 7, it is assumed that the content data desired to be superimposed and displayed is upgraded after sending a direct mail to the customer. Specifically, it is assumed that the image data G2 shown in FIG. 4 is registered first and then the image data G4 obtained by slightly correcting the image data G2 is registered. If the image data G2 is not used in the future, the image data G2 and the image data G4 may be overwritten and saved on the content DB 12, but there are cases where it is desired to keep the old image data G2. For example, it is a case where new and old products are compared. In such a case, according to the conventional method, when the image data G4 is registered, it is necessary to create a QR code corresponding to the registered image data. However, according to the present system, such work can be made unnecessary. .

  Further, according to the present system, even when there are a plurality of content data matching the search condition, it is possible to superimpose and display on the captured image what is randomly selected from the plurality of content data. For this reason, in the example shown in FIG. 5, every time a Christmas card is captured, the content superimposed on the captured image changes randomly, and a timeless Christmas card can be realized.

  In addition, according to the present system, even when there are a plurality of content data that matches the search condition, it is possible to superimpose and display on the captured image a plurality of content data selected by the user of the image display device 100. is there. Therefore, it is possible to display content that suits the user's preference. For example, in the example shown in FIG. 7, if the image data indicating the exterior of the car and the image data indicating the interior of the car are registered on the same day as the content data matching the search condition, the user can The exterior and interior of the car can be checked sequentially.

  Note that, when there are a plurality of pieces of content data that match the search condition, the image display device 100 may switch and display the content based on the plurality of pieces of content data at predetermined time intervals. In this case, the content search unit 6 acquires all content data that matches the search conditions. Then, the content synthesizing unit 8 synthesizes each of the acquired plurality of content data and the captured image data to generate a plurality of display image data. The display control unit 9 may output the plurality of display image data sequentially to the display unit 10 at predetermined time intervals so that the display unit 10 sequentially displays a plurality of images based on the plurality of display image data. . For example, in the example shown in FIG. 7, the exterior and interior of the car can be automatically displayed in sequence, and direct mail with a high advertising effect can be realized. With the example shown in FIG. 5, a more interesting Christmas card can be realized.

  Further, according to the present system, anyone with software that can generate a QR code can easily create an original Christmas card as shown in FIG. Therefore, the service provider only has to sell the Christmas card with the marker and register the content data in the content DB 12, and the service user purchases the Christmas card with the marker and adds it to himself / herself. Just paste the created QR code. Thus, according to the present system, it is possible to realize a service with less burden on both the service provider and the service user.

  In the above description, the marker is provided on the outline of the Christmas card or catalog item. However, this marker is for specifying the range in the captured image data of the QR code. It is not necessary to provide the outline of the article. For example, a marker may be provided around the QR code so as to surround the QR code. In the image display apparatus 100, the QR code can be read by detecting this marker, extracting the image in the marker, and normalizing it.

  In this way, when providing a marker around the QR code, a software that can create a QR code has a function that can create not only a QR code but also a marker surrounding the QR code. Etc., the user can create a Christmas card as shown in FIG. By doing so, the service provider side does not have to sell the card, and the cost for service construction can be suppressed.

  Note that when the imaging unit 1 captures an article, if the QR code is used as a main subject, the QR code can be recognized without a marker. For this reason, this service can be realized without providing a marker on the article side.

  Further, so far, the image display apparatus 100 can read the search condition by pasting or printing the QR code on the article. However, the present invention is not limited to this. For example, a color code may be used instead of the QR code. Further, instead of providing a QR code, a search condition may be coded and embedded in an article by digital watermark technology. When using a digital watermark, a block that can read search condition data encoded by the digital watermark detection software may be provided instead of the QR code reading unit 5.

  Next, a modification of the image display system shown in FIG. 2 will be described.

  FIG. 8 is a diagram showing a first modification of the image display system shown in FIG. The system shown in FIG. 8 is the same as the system configuration shown in FIG. 2 except that the face extraction unit 13 is added to the image display apparatus 100 and some functions are added to the content composition unit 8.

  The face extraction unit 13 extracts face image data indicating a face from the marker image data normalized by the marker image normalization unit 4 by a known face detection process.

  When the content data input from the content search unit 6 is polygon data, the content composition unit 8 performs a texture mapping process for pasting the face image data extracted by the face extraction unit 13 to the polygon data, Display image data for stereoscopically displaying the face included in the subject with polygons is generated.

  Next, a specific example of a service using the image display system shown in FIG. 2 will be described. In the following description, it is assumed that a table as shown in FIG. 4 is registered in the content DB 12.

  FIG. 9 is a diagram illustrating an example of a service using the image display system illustrated in FIG. In this example, a page 90 on which a marker 91 is printed is placed in a photo album. Then, the album creator pastes a favorite photograph (here, a baby girl's photograph) and a QR code 92 on this page 90 to complete the album. The QR code 92 includes a search condition that, among the content data registered in the content DB 12, the attribute information includes the word “infant girl” that is the subject of the photograph and the content data that is polygon data is searched. Should be coded.

  The person who has received the photo album images the page 90 of the photo album with the imaging unit 1 of the image display device 100. When the imaging is completed, the image display apparatus 100 extracts a portion corresponding to the marker 91 from the captured image data obtained by the imaging, and extracts the portion and data inside it as marker image data. Subsequently, the image display device 100 extracts a portion corresponding to the QR code 92 from the marker image data, decodes the QR code 92 therefrom, and reads the search condition. Further, the image display device 100 extracts face image data indicating a face from the marker image data. Next, the image display apparatus 100 acquires appropriate content data (here, polygon data P1) from the content DB 12 according to the read search condition.

  Next, the image display apparatus 100 pastes the extracted face image data to the acquired polygon data P1 by texture mapping processing, and generates display image data for stereoscopic display of the face based on the face image data. . Finally, the image display device 100 displays an image based on the display image data on the display unit 10. As a result, as shown in FIG. 9, the infant's face 93 shown in the album photo is displayed in a three-dimensionally superimposed manner on the captured image 94 including the marker image 90 ′ corresponding to the page 90.

  As described above, according to the system shown in FIG. 8, the face of the person shown in the album can be displayed three-dimensionally on the display unit 10, and an album with high added value can be provided. In this system, it is preferable that the polygon is stereoscopically displayed from the portion of the captured image where the photograph exists, and therefore, the marker 91 is preferably provided around the photograph pasted on the album.

  FIG. 10 is a diagram showing a second modification of the image display system shown in FIG. The system shown in FIG. 10 is the same as the system configuration shown in FIG. 2 except that the spectral calculation unit 14 and the content correction unit 15 are added to the image display device 100.

  The spectral calculation unit 14 captures the marker image data from the spectral reflectance characteristic data of the marker stored in advance in the image display device and the pixel data values of the normalized marker image data. Calculate the spectral characteristics of the light source at the time.

  The content correction unit 15 corrects the color of the content data acquired by the content search unit 6 based on the spectral characteristics calculated by the spectral calculation unit 14.

  The content synthesizing unit 8 synthesizes the content data corrected by the content correcting unit 15 and the captured image data obtained by imaging by the imaging unit 1.

  With the configuration as described above, an image displayed on the display unit 10 can be improved.

  FIG. 11 is a diagram showing a third modification of the image display system shown in FIG. The system shown in FIG. 11 is the same as the system configuration shown in FIG. 2 except that the marker image analysis unit 16 is added to the image display device 100 and the function of the content search unit 6 is partially changed.

  The marker image analysis unit 16 analyzes the marker image data normalized by the marker image normalization unit 4 and transmits the analysis result to the content search unit 6. Specific analysis items relate to the image quality such as the overall color, brightness, and contrast of the image.

  The content search unit 6 searches the content DB 12 for content data that matches the search condition and that is closest to the image quality of the marker image data analyzed by the marker image analysis unit 16.

  In the example of the service shown in FIG. 5, the image data satisfying the search condition is three image data G1, G3, and G6. For example, the content data closest to the marker image data) is selected.

  In the example of FIG. 5, since the green Christmas tree occupies most of the marker image data, “green” is transmitted to the content search unit 6 as the analysis result as the color of the marker image data. Then, among the image data G1, G3, and G6, the image data G6 having the largest proportion of green is acquired by the content search unit 6 as the optimum content.

  Here, image data close to the color of the marker image data is selected as the optimum content. However, image data having most of the opposite color in the color space to the color of the marker image data is the optimum content. You may choose as For example, you may select the image data G1 which has most red.

  Close to image quality means that the difference between the numerical values indicating the image quality (color, brightness, or contrast) between the marker image data and the content data is less than or equal to the threshold value. Say small.

  If there is only one content data that matches the search condition, the content search unit 6 may acquire the content data regardless of the analysis result.

  Further, the image quality information of the content data may be registered in advance in association with each content data of the content DB 12, or the content search unit 6 may obtain the content data satisfying the search condition.

  The content DB 12 may register a plurality of patterns of content data with different image quality for one piece of content data. In this case, even if there is one piece of content data that matches the search condition, it is only necessary to select the one with the closest image quality among the plurality of patterns of the content data. In this case, if there are a plurality of content data that matches the search condition, the content data selected at random, the content data selected by the user, or the content data with the latest registration date is selected, and the selected content data is selected. What is necessary is just to select the pattern with the closest image quality among a plurality of patterns of data.

  With the configuration as described above, when the captured image data and the content data are displayed in a superimposed manner, the image quality of the marker image and the content included in the captured image data can be made uniform, and a composite display without any sense of incongruity becomes possible. .

  Note that the configuration shown in FIG. 11 can also be applied to the modified examples shown in FIGS.

  FIG. 12 is a diagram showing a fourth modification of the image display system shown in FIG. The system shown in FIG. 12 is the same as the system configuration shown in FIG. 2 except that the image analysis DB 17 is added to the image display device 100 and the function of the content search unit 6 is partially changed.

  The content search unit 6 in the system shown in FIG. 12 has a function of acquiring the marker image data normalized by the marker image normalization unit 4 and transmitting it to the image analysis DB 17 via the network 11. Further, the content search unit 6 searches the content DB 12 for optimal content data by adding the analysis result returned from the image analysis DB 17 to the search conditions read by the QR code reading unit 5.

  In the image analysis DB 17, for example, image data registered in the content DB 12 is registered together with content information that specifically represents the content. For example, in the case of the image data G1 shown in FIG. 4, content information indicating that it is Santa Claus is registered. In the case of the image data G3 shown in FIG. 4, content information indicating a Christmas tree is registered. In the case of the image data G6 shown in FIG. 4, content information indicating that the character “Merry Christmas” is the main image data is registered. This content information is registered as tag information of the image data.

  It is assumed that the same content information registered in the image analysis DB 17 is also registered in each content data registered in the content DB 12.

  The image analysis DB 17 performs matching between the marker image data received from the content search unit 6 and the image data in the database, and determines image data whose content is closest to the marker image data. The image data whose content is closest to the marker image data refers to the image data having the highest evaluation value among the image data having the matching evaluation value equal to or higher than a certain value.

  In the service example shown in FIG. 5, since the Christmas tree is printed on the Christmas card 50, the image data of the Christmas tree in the image analysis DB 17 is determined as the image data whose content is closest to the marker image data. Is done. The image analysis DB 17 returns the content information of the image data determined to be closest to the marker image data to the content search unit 6 as an analysis result.

  The content search unit 6 searches for content data that matches the search condition obtained by the QR code reading unit 5 and that is closest to the content of the marker image data. In the case of the service example shown in FIG. 5, the image data satisfying the search condition is three image data G1, G3, and G6, but the content information received from the image analysis DB 17 is information indicating a Christmas tree. The image data G6 associated with this content information is acquired by the content search unit 6 as the optimum content.

  If there is only one content data that matches the search condition, the content search unit 6 may acquire the content data regardless of the content information.

  With such a configuration, content close to the content of an image printed on an article to which a QR code is attached can be superimposed and displayed with less discomfort.

  The configuration shown in FIG. 12 can also be applied to the modification examples shown in FIGS.

  The image display device 100 described above can be applied to portable terminals such as a head mounted display with a camera using a head mounted display as the display unit 10, a digital camera, and a camera-equipped mobile phone.

  As described above, the following items are disclosed in this specification.

  The disclosed image display device extracts an image capturing unit that captures an image of a subject and generates captured image data, and retrieves search condition data representing a search condition for content data from the captured image data, and based on the search condition data A search condition reading unit that reads a search condition, a content data acquisition unit that acquires content data that matches the search condition read by the search condition reading unit, and the content data acquired by the content data acquisition unit And the captured image data, and an image composition unit that generates display image data for combining and displaying the content based on the content data on the image based on the captured image data, and an image based on the display image data Is displayed on the display unit.

  In the disclosed image display device, when there are a plurality of content data matching the search condition in the database, the content data acquisition unit selects and acquires one of the plurality of contents at random To do.

  In the disclosed image display device, when there are a plurality of content data that match the search condition, the content data acquisition unit acquires the plurality of content data and displays content based on each of the plurality of content data. The image is displayed on the display unit, and the image synthesis unit synthesizes the selected content data from the plurality of content data displayed on the display unit with the captured image data.

  In the disclosed image display device, the content data acquisition unit acquires content data that matches the search condition and is closest to the image quality of the captured image data.

  In the disclosed image display device, the content data acquisition unit acquires content data that matches the search condition and is closest to the content of the captured image data.

  The disclosed image display device includes a face image data extraction unit that extracts face image data indicating a face from the captured image data, and the content data acquired by the content data acquisition unit displays a person's face in three dimensions. When the polygon data is the polygon data, the image composition unit displays the face based on the face image data in a three-dimensional manner by texture mapping processing for pasting the face image data extracted by the face image data extraction unit to the polygon data The display image data to be generated is generated.

  In the disclosed image display device, in the database, the content data and attribute information indicating the contents of the content data are registered in association with each other, and the search condition is specified in the text. This is a condition that specifies that the content data associated with the attribute information including the keyword is searched.

  In the disclosed image display device, in the database, the content data, attribute information indicating the contents of the content data in text, and registration date information of the content data are associated and registered, The search condition is a condition that specifies that the attribute data including a predetermined keyword is associated with the text and the content data with the latest registration date is searched.

  In the disclosed image display apparatus, the display unit is a head mounted display.

  The disclosed portable terminal includes each unit of the image display device, the display unit, and a communication unit connected to the database via a network.

  According to the disclosed image display method, an imaging step of imaging a subject to generate captured image data, search condition data representing a search condition for content data is extracted from the captured image data, and based on the search condition data A search condition reading step for reading a search condition, a content data acquisition step for acquiring content data that matches the search condition from a database, and a combination of the content data and the captured image data into an image based on the captured image data An image composition step for generating display image data for combining and displaying content based on the content data; and a display control step for displaying an image based on the display image data on a display unit.

  According to the disclosed image display method, in the content data acquisition step, when there are a plurality of content data matching the search condition on the database, one of the plurality of contents is randomly selected and acquired. To do.

  In the disclosed image display method, in the content data acquisition step, when there are a plurality of content data that matches the search condition, the plurality of content data are acquired and content based on each of the plurality of content data is obtained. The image is displayed on the display unit, and in the image composition step, one selected from the plurality of content data displayed on the display unit is synthesized with the captured image data.

  In the disclosed image display method, in the content data acquisition step, content data that matches the search condition and that is closest to the image quality of the captured image data is acquired.

  In the disclosed image display method, in the content data acquisition step, content data that matches the search condition and is closest to the content of the captured image data is acquired.

  The disclosed image display method includes a face image data extraction step for extracting face image data indicating a face from the captured image data, and the content data acquired in the content data acquisition step is used for stereoscopically displaying a human face. In the case of polygon data, in the image composition step, the face based on the face image data is stereoscopically displayed by texture mapping processing in which the face image data extracted by the face image data extraction unit is pasted to the polygon data. The display image data for generating is generated.

  In the disclosed image display method, the content data and attribute information indicating contents of the content data in text are registered in the database in association with each other, and the search condition is specified in the text. This is a condition that specifies that the content data associated with the attribute information including the keyword is searched.

  In the disclosed image display method, in the database, the content data, attribute information indicating the contents of the content data in text, and registration date information of the content data are associated and registered, The search condition is a condition that specifies that the attribute data including a predetermined keyword is associated with the text and the content data with the latest registration date is searched.

DESCRIPTION OF SYMBOLS 1 Imaging part 5 QR code reading part 6 Content search part 8 Content synthetic | combination part 9 Display control part 10 Display part 12 Content DB

Claims (18)

An imaging unit that images a subject and generates captured image data;
A search condition reading unit that extracts search condition data representing a search condition for content data from the captured image data, and reads the search condition based on the search condition data;
A content data acquisition unit that acquires from the database content data that matches the search condition read by the search condition reading unit;
An image that combines the content data acquired by the content data acquisition unit and the captured image data, and generates display image data for combining and displaying content based on the content data on an image based on the captured image data A synthesis unit;
An image display apparatus comprising: a display control unit that causes a display unit to display an image based on the display image data. The image display device according to claim 1,
An image display device in which the content data acquisition unit selects and acquires one of the plurality of contents at random when there are a plurality of content data matching the search condition on the database. The image display device according to claim 1,
When there are a plurality of content data matching the search condition, the content data acquisition unit acquires the plurality of content data and causes the display unit to display content based on each of the plurality of content data,
The image synthesizing unit is an image display device that synthesizes a selected one of the plurality of content data displayed on the display unit with the captured image data. The image display device according to claim 1,
An image display device in which the content data acquisition unit acquires content data that matches the search condition and is closest to the image quality of the captured image data. The image display device according to claim 1,
An image display device in which the content data acquisition unit acquires content data that matches the search condition and is closest to the content of the captured image data. The image display device according to any one of claims 1 to 5,
A face image data extraction unit that extracts face image data indicating a face from the captured image data;
When the content data acquired by the content data acquisition unit is polygon data for stereoscopically displaying a person's face, the image composition unit extracts the face image data extracted by the face image data extraction unit. An image display device that generates the display image data for stereoscopically displaying a face based on the face image data by texture mapping processing to be pasted to polygon data. The image display device according to any one of claims 1 to 6,
In the database, the content data and attribute information indicating the content data in text are associated and registered,
The search condition is an image display device that is a condition that specifies searching for the content data in which the attribute information including a predetermined keyword is associated with the text. The image display device according to any one of claims 1 to 7,
In the database, the attribute data indicating the content data and the content data content in text and the registration date information of the content data are registered in association with each other,
The search condition is an image display apparatus that is designated to search for the content data having the latest registration date and the attribute information including a predetermined keyword associated with the text. The image display device according to any one of claims 1 to 8,
An image display device in which the display unit is a head-mounted display. Each part of the image display device according to any one of claims 1 to 9,
The display unit;
A mobile terminal comprising a communication unit connected to the database via a network. An imaging step of imaging a subject and generating captured image data;
A search condition reading step of extracting search condition data representing a search condition of content data from the captured image data and reading the search condition based on the search condition data;
A content data acquisition step of acquiring content data matching the search condition from a database;
An image synthesis step of synthesizing the content data and the captured image data, and generating display image data for combining and displaying the content based on the content data on the image based on the captured image data;
A display control step of displaying an image based on the display image data on a display unit. The image display method according to claim 11, comprising:
In the content data acquisition step, when there are a plurality of content data matching the search condition on the database, an image display method for selecting and acquiring one randomly from the plurality of contents. The image display method according to claim 11, comprising:
In the content data acquisition step, when there are a plurality of content data that match the search condition, the plurality of content data is acquired and the content based on each of the plurality of content data is displayed on the display unit,
In the image synthesis step, an image display method for synthesizing a selected one of the plurality of content data displayed on the display unit with the captured image data. The image display method according to claim 11, comprising:
In the content data acquisition step, an image display method for acquiring content data that satisfies the search condition and is closest to the image quality of the captured image data. The image display method according to claim 11, comprising:
In the content data acquisition step, an image display method for acquiring content data that matches the search condition and is closest to the content of the captured image data. The image display method according to any one of claims 11 to 15,
A face image data extraction step for extracting face image data indicating a face from the captured image data,
When the content data acquired in the content data acquisition step is polygon data for stereoscopically displaying a person's face, in the image synthesis step, the face image data extracted by the face image data extraction unit is used as the polygon data. An image display method for generating the display image data for stereoscopically displaying a face based on the face image data by a texture mapping process to be pasted on the data. The image display method according to any one of claims 11 to 16, comprising:
In the database, the content data and attribute information indicating the content data in text are associated and registered,
The search condition is an image display method that is a condition that specifies searching for the content data in which the attribute information including a predetermined keyword is associated with the text. The image display method according to any one of claims 11 to 17,
In the database, the attribute data indicating the content data and the content data content in text and the registration date information of the content data are registered in association with each other,
The search condition is an image display method in which the attribute information including a predetermined keyword is associated with the text and a search condition is designated to search for the content data with the latest registration date.