JP2011043545A - Image display device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image display device which can make a user perceive that projected related image is related to a real substance. <P>SOLUTION: An imaging means 30 picks up a region 999 to be projected and acquires a related image 501 relating to a real image 801a on the basis of discrimination identification information 801b. A projection means 20 projects the related image 501 to a position superimposed on the real image 801a. The projection means 20 projects the related image 501 so that it may move gradually to the outside direction of the real image 801a, and standstill-projects the related image 501 at the region 999 to be projected of the outside of the real image 801a. The related image 501 is displayed as if it is generated from the real image 801a, the related image 501 is animation-displayed moving continuously from a first position to a second position, and it can be perceived by the user that the related image 501 is related to the real image 801a. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image display device that projects a related image related to a real image in the vicinity of the real image.

  Conventionally, as disclosed in Patent Document 1, an information display device that can simultaneously view information related to an entity such as a printed matter within the same field of view has been proposed. Such an information display device identifies the entity by reading an identification information transmission source provided on the entity, searches for a related image related to the entity, and locates the related image next to the entity. It is a device that projects to. For example, if the entity is a painting and the user wants to know information about this painting, the explanatory text of the painting is projected next to the painting as a related image, so that the user can appreciate the painting while viewing the painting. Can read.

  However, when there are a large number of entities, there is a problem that it is difficult for the user to recognize to which entity the projected related image is related only by projecting the related image next to the entity. It was.

JP 2009-8905 A

  The present invention solves the above problems and provides an image display device that allows a user to perceive that a projected related image is related to an entity.

The invention according to claim 1, which has been made to solve the above problems,
A projecting means that is arranged to face a projection surface on which an entity on which a real image is described or an entity on which a real image is displayed is placed, and projects an image on the projection surface;
An imaging means for imaging the projection surface;
Entity position recognition means for recognizing a position on the projection surface of the real image from the picked-up image picked up by the image pickup means, and generating real image position information;
Identification information acquisition means for acquiring identification information of the entity;
Based on the identification information of the entity acquired by the identification information acquisition unit, the related image acquisition unit acquires the related image corresponding to the entity from the storage unit storing the related image related to the real image. When,
Based on the real image position information, first projection image data in which the related image is projected onto a first position that is a position to be superimposed on the real image, and an entity and a projection surface that are outside the real image Projection image data generation means for generating second projection image data in which the related image is projected to a second position which is at least one of the positions above;
Image signal generating means for outputting one of the first projection image data and the second projection image data as an image signal to the projection means and then outputting the other as an image signal to the projection means;
It is characterized by having.

According to a second aspect of the present invention, in the first aspect of the present invention, the projection image data generating means includes a first position where the first projection image data is projected and a second position where the second projection image data is projected. Generating third projection image data in which a related image is projected at a third position that is a position that continuously complements
The image signal generation means further outputs one of the first projection image data and the second projection image data as an image signal to the projection means, and outputs the other as an image signal to the projection means. Before performing, the third projection image data is output as an image signal to the projection means.

  According to a third aspect of the present invention, in the invention according to the first or second aspect, the projection image data generating means is configured to change the brightness of the related image to be projected between the object and the exposed projection surface. It is characterized by making it.

  According to a fourth aspect of the present invention, in the first or second aspect of the present invention, the projection image data generating means is on at least one of the entity and the exposed projection surface and has a constant brightness. The related image is projected onto an area having the above-described contrast and a certain level or less.

  According to a fifth aspect of the present invention, in the first or second aspect of the present invention, the projection image data generating means projects a related image on the exposed projection surface.

  The invention according to claim 6 is the invention according to claim 4 or claim 5, wherein the projection image data generation means cannot project the related image to the region where the related image is to be projected. The related image is reduced and projected onto the area.

The invention according to claim 7 is the invention according to claims 1 to 6,
Based on the identification information of the entity acquired by the identification information acquisition unit, further comprising presence determination means for determining whether a related image corresponding to the entity exists in the storage unit;
The related image acquisition unit acquires the related image from the storage unit when the presence determination unit determines that the related image exists in the storage unit.

The invention according to claim 1 is disposed opposite to a projection surface on which an entity on which a real image is described on a surface or an entity on which a real image is displayed is placed, and is disposed on the projection surface. Recognizing the position of the real image on the projection surface from the projection unit for projecting the image, the imaging unit for imaging the projection surface, and the captured image captured by the imaging unit, and generating real image position information An entity position recognition unit, an identification information acquisition unit that acquires the identification information of the entity, and a related image associated with the actual image is stored based on the identification information of the entity acquired by the identification information acquisition unit. The related image is acquired at a first position that is a position to be superimposed on the actual image based on the actual image position information, and the related image acquisition unit that acquires the related image corresponding to the entity from the stored storage unit. First projected image data to be projected; Projection image data generating means for generating second projection image data in which the related image is projected to a second position which is outside the image and at least one position on the entity and the projection surface; Image signal generating means for outputting either one of the projection image data and the second projection image data as an image signal to the projection means and then outputting the other as an image signal to the projection means is provided.
Thereby, before the related image is projected onto the second position, the related image is projected at a position where the related image is superimposed on the real image, so that the user feels as if the related image is generated from the real image. . In other words, the user can perceive that the projected related image is related to the entity.
Alternatively, after the related image is projected to the second position, the related image is projected to the first position where the actual image is, so that the user can perceive that the related image is related to the actual image. .

According to a second aspect of the present invention, in the first aspect of the present invention, the projection image data generating means includes a first position where the first projection image data is projected and a second position where the second projection image data is projected. Generating third projection image data on which the related image is projected at a third position, which is a position that continuously complements between the first projection image data and the second projection. After outputting one of the image data as an image signal to the projection unit and before outputting the other as an image signal to the projection unit, the third projection image data is output to the projection unit as an image signal. It is characterized by outputting.
Thereby, since the related image is displayed while continuously moving from the first position to the second position, the user is strongly given an impression that the related image is related to the actual image, and the projected related image is further projected. It is possible to make the user perceive that the image is related to the entity.
Alternatively, since the related image is displayed while continuously moving from the second position to the first position, the user is given an impression as if the related image is stored in the actual image, and is projected onto the projection area. The user can perceive that the related image is related to the actual image.

According to a third aspect of the present invention, in the invention according to the first or second aspect, the projection image data generating means is configured to change the brightness of the related image to be projected between the object and the exposed projection surface. It is characterized by making it.
Thereby, it becomes easy for a user to visually recognize an entity by projecting a related image at a reduced brightness on the entity.
Alternatively, when the related image is projected with the brightness increased on the entity, the related image is displayed in an emphasized manner on the entity, so that the user can easily view the related image.

According to a fourth aspect of the present invention, in the first or second aspect of the present invention, the projection image data generating means is on at least one of the entity and the exposed projection surface and has a constant brightness. The related image is projected onto an area having the above-described contrast and a certain level or less.
As a result, even when a large number of entities and the like are placed on the projection area, it is possible to project a related image corresponding to the entity.

According to a fifth aspect of the present invention, in the first or second aspect of the present invention, the projection image data generating means projects a related image on the exposed projection surface.
As a result, the related image is not projected onto the entity, so that the user can clearly see the related image.

The invention according to claim 6 is the invention according to claim 4 or claim 5, wherein the projection image data generation means cannot project the related image to the region where the related image is to be projected. The related image is reduced and projected onto the area.
As a result, even if a large number of entities and the like are placed on the projection area and the area where the related image can be projected is narrow, the related image corresponding to the entity can be projected.

The invention according to claim 7 is the invention according to claims 1 to 6,
Based on the identification information of the entity acquired by the identification information acquisition unit, further comprising presence determination means for determining whether a related image corresponding to the entity exists in the storage unit;
The related image acquisition unit acquires the related image from the storage unit when the presence determination unit determines that the related image exists in the storage unit.
Thus, an information table representing the relationship between the identification information and the related image is stored in the storage unit, and the existence determination unit determines whether the entity identification information acquired by the identification information acquisition unit exists in the information table. If it is determined whether or not the related image corresponding to the entity exists in the storage unit, it is possible to quickly and surely search for the related image corresponding to the entity and quickly It is possible to acquire related images.

1 is a schematic diagram of an image display device according to an embodiment of the present invention. It is explanatory drawing of the projection state of a related image. It is a block diagram of an image display apparatus. It is explanatory drawing of an information table. It is a flowchart in a main process. (First embodiment) It is explanatory drawing of the projection state of a related image. (Second Embodiment) It is explanatory drawing of the projection state of a related image. (Third embodiment) It is explanatory drawing of the projection state of a related image. (Fourth embodiment)

(Outline of the image display device of the present invention)
Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. As shown in FIG. 1, the image display device 100 of the present invention includes a control unit 10, a projection unit 20, and an imaging unit 30. The projection unit 20 and the imaging unit 30 are disposed to face a projection area 999 such as a plate surface of a table. The projection unit 20 is a device that projects the related image 501 onto the projection area 999 based on the “image signal” output from the control unit 10. The imaging unit 30 is a device that images the projection area 999. The related image 501 is an image related to the real image 801a placed on the projection area 999 and described on the surface of the entity 801 such as a printed material. For example, when the real image 801a is a painting, The related image 501 is a commentary on the picture. In the entity 801, identification information 801b such as a two-dimensional code and a barcode is described.

In the present invention, when the entity 801 is placed on the projection area 999 (the state of FIG. 2A), the imaging unit 20 images the projection area 999, so that the identification information 801b is controlled by the control device. 10, based on the identification information 801 b, the control device 10 acquires a related image 501 related to the real image 801 a from the storage unit 200 (shown in FIG. 3) that is a database.
Then, the projecting means 20 projects the related image 501 so as to float and be displayed at the position superimposed on the real image 801a (first position) (state (B) in FIG. 2). Specifically, the related image 501 blinks on the real image 801a to give the user the impression that the related image 501 is generated from the real image 801a.
Next, the projecting means 20 projects the related image 501 so that the related image 501 gradually moves in the outward direction of the real image 801a (state (C) in FIG. 2), and further, the related image 501 is displayed as the real image. Still projection is performed on the second projection area 999 outside the area 801a (state (D) in FIG. 2).
In this way, the related image 501 is displayed as if it is generated from the actual image 801a, and the related image 501 is animated while continuously moving from the first position to the second position. The user is strongly given an impression related to the real image 801a.

(Block diagram of image display device)
Hereinafter, a block diagram of the image display device 100 will be described with reference to FIG. The control unit 10 includes a CPU 11, a RAM 12, a ROM 13, an imaging controller 15, a projection image controller 17, a VRAM 18, and an input unit 19, which are connected to each other via a bus 9. The projection image controller 17 and the VRAM 18 are connected to each other.

The CPU 11 performs various calculations and processes in cooperation with the RAM 12 and the ROM 13.
The RAM 12 temporarily stores a program processed by the CPU 11 and data processed by the CPU 11 in its address space. The RAM 12 includes a captured image data storage area 12a, an entity position information storage area 12b, an identification information storage area 12c, an information table search result storage area 12d, and a related image data storage area 12e.
In the captured image data storage area 12 a, “captured image data” captured by the imaging unit 30 and generated by the imaging controller 15 is stored.
In the entity position information storage area 12b, “entity position information” and “actual image position information” described later are stored.
The “identification information” described later is stored in the identification information storage area 12c.
The “information table” acquired from the information table storage area 200a of the storage means 200 is stored in the information table search result storage area 12d.
In the related image storage area 12e, “related images” acquired by the input unit 19 as described later are stored.

  Various programs for controlling the image display device 100 are stored in the ROM 13. The ROM 13 stores programs such as an imaging control program 13a, an entity position recognition program 13b, an identification information acquisition program 13c, an information table search program 13d, a related image data acquisition program 13e, and a projection image data generation program 13f. The various programs are processed by the CPU 11, thereby realizing various functions of the image display device 100. It should be noted that the program may be realized as an ASIC (Application Specific Integrated Circuit).

  The imaging control program 13a outputs an “imaging control signal” to the imaging controller 15, controls the imaging controller 15 and the imaging means 30, and generates “captured image data”. The generated “captured image data” is stored in the captured image data storage area 12a.

  The entity position recognition program 13b analyzes the “captured image data” stored in the captured image data storage area 12a, detects the outer edge position of the entity 801 on the projection area 999, Information ". Further, the entity position recognition program 13b analyzes the “captured image data” to detect the outer edge position of the actual image 801a on the projection area 999 and generates “actual image position information”. The generated “entity location information” and “real image location information” are stored in the entity location information storage area 12b.

  The identification information acquisition program 13c extracts “identification information” of the entity 801 from the “captured image data” by analyzing the “captured image data” stored in the captured image data storage area 12a. The extracted “identification information” is stored in the identification information storage area 12c.

  The information table search program 13d searches the “information table” corresponding to the “identification information” stored in the identification information storage area 12b by referring to the information table storage area 200a of the storage means 200, and the “information table” Is acquired and stored in the information table search result storage area 12d. Note that the “information table” is a table representing the relationship between the identification information 801b, the actual image 801a, and the related image 501, as shown in FIG. In the embodiment shown in FIG. 4, the “identification information” is alphanumeric, the “real image” is the name of the real image 801 a, and the “related image” is the file name or URL (Uniform Resource Locator) of the related image 501. Has been. When the “identification information” of a certain real image 801a is known, referring to the “information table”, the related image 501 related to the real image 801a is found.

  The related image data acquisition program 13e refers to the “information table” stored in the information table search result storage area 12d, and stores the “related image data” corresponding to the “identification information” in the related image data storage area of the storage unit 200. This is a program for acquiring “related image data” from 200b and storing it in the related image data storage area 12e.

Based on the “real image position information” stored in the entity position information storage area 12b, the projection image data generation program 13f projects “related image data” on the first position that is a position to be superimposed on the real image 801a. The “drawing command” for generating the “first projection image data” is output to the projection image controller 17.
Further, the projection image data generation program 13f projects the “related image” at a position near the outside of the entity 801 based on the “entity location information” stored in the entity location information storage area 12b. A “drawing command” for generating “2 projection image data” is output to the projection image controller 17.

The imaging controller 15 has an A / D converter and a DSP (Digital Signal Processor). The A / D converter converts a “signal voltage” output from an image sensor of the imaging unit 30 described later into a “digital signal”.
The DSP generates “captured image data” that is two-dimensional pixel data from the “digital signal” generated by the A / D converter. The generated “captured image data” is stored in the captured image data storage area 12d.

  The projection image controller 17 has an image processing circuit such as a GPU (Graphics Processing Unit). The projection image controller 17 processes the “related image data” stored in the related image storage area 12 a based on the “drawing command” output from the projection image data generation unit 13 c to obtain “projection image data”. Generated and stored in the VRAM 18. The projection image controller 17 has an external output interface, and outputs “projection image data” stored in the VRAM 18 to the projection unit 20 as an “image signal”.

The input unit 19 acquires “related image data”. The input unit 19 includes a LAN (abbreviation for Local Area Network) interface. A storage unit 200 that functions as a database is connected to the input unit 19 via a LAN, a WAN (abbreviation for Wide Area Network), or the Internet. The storage means 200 includes a hard disk and a flash memory. The storage means 200 has an information table storage area 200a and a related image data storage area 200b.
An “information table” is stored in the information table storage area 200a.
In the related image storage area 200b, “related image data” is stored.

  The projection unit 20 includes a lamp driving circuit, a lamp, an illumination optical system, an image generation element, and an imaging optical system. The lamp driving circuit is a circuit that supplies current to the lamp. The lamp supplied with current from the lamp driving circuit emits diffused light. The illumination optical system collects the diffused light emitted from the lamp and makes it uniform illumination light. This illumination light is applied to the image generating element. The image generation element is an LCD (Liquid Crystal Display) or a DMD (Digital Micromirror Device), and generates an image by expressing the irradiated illumination light in gradation on each pixel. The imaging optical system is composed of a large number of lenses, and images the illumination light that has passed through the image generating element on the projection area 999.

  The imaging unit 30 is a device that images the projection area 999. The imaging device 30 has an image sensor and an imaging optical system. The image sensor has photodiodes arranged two-dimensionally. Each photodiode converts the light intensity into an electric charge, and outputs the electric charge as a “signal voltage”. The image sensor includes a charge coupled device image sensor (CCD) and a complementary metal oxide semiconductor (CMOS). The imaging optical system is composed of a single lens or a plurality of lenses, and forms an incident image on an image sensor.

(Description of the main flow of the first embodiment)
Below, the flow of the main process of 1st Embodiment is demonstrated. When the image display apparatus 100 is powered on, the process proceeds to S11 “initialization”.
In the process of S11 “initialization”, various programs stored in the ROM 13 are activated. When the process of S11 ends, the process proceeds to S12.

  In the process of S <b> 12 “start imaging”, the imaging control program 13 a outputs an “imaging control signal” to the imaging controller 15 to start imaging of the projection area 999 by the imaging unit 30. The imaging controller 15 generates “captured image data”. The generated “captured image data” is stored in the captured image data storage area 12a. The imaging control program 13a outputs an “imaging control signal” for imaging the projection area 999 by the imaging unit 30 to the imaging controller 15 every predetermined period (for example, several hundred milliseconds). When the process of S12 ends, the process proceeds to the determination process of S13.

  In the determination processing of S13 “acquisition of identification information?”, The identification information acquisition program 13c analyzes “captured image data” stored in the captured image data storage area 12a, and stores the entity 801 in the “captured image data”. It is determined whether there is identification information 801b. If the identification information acquisition program 13c determines that the “captured image data” includes the identification information 801b (YES in S13), the “identification information” is stored in the identification information storage area 12c, and S14 Proceed to the process.

  In the process of S14 “information table search”, the information table search program 13d refers to the “information table” stored in the information table storage area 200a of the storage means 200, and uses the “identification information” acquired in the process of S13. Search for. When the process of S14 ends, the process proceeds to the determination process of S15.

In the determination process of S15 “identification information exists in information table?”, When the information table search program 13d determines that the “identification information” acquired in the process of S13 exists in the “information table” searched in the process of S14. (The determination process in S15 is YES), the “identification information” and the file name of the “related image” corresponding thereto are associated and stored in the information table search result storage area 12d, and the process proceeds to S16.
On the other hand, when the information table search program 13d determines that the “identification information” acquired in the process of S13 does not exist in the “information table” searched in the process of S14 (the determination process of S15 is NO), S13 Return to the determination process.

  In the process of S16 “acquire related image data”, the related image data acquisition program 13e refers to the “information table” stored in the information table search result storage area 12d and “identification information” detected in the process of S13. Is acquired from the related image data storage area 200b of the storage unit 200 and stored in the related image data storage area 12e. If the “related image data” corresponding to the “identification information” is a URL, the related image data acquisition program 13e acquires the WEB page of the URL via the Internet. When the process of S16 ends, the process proceeds to S17.

In the process of S17 “detecting the position of an entity”, the entity position recognition program 13b analyzes the “captured image data” stored in the captured image data storage area 12a to thereby identify the entity on the projection area 999. The outer edge position of the object 801 is detected to generate “entity position information”. Specifically, by performing threshold processing and edge processing on the “captured image data”, a boundary (outline of the entity 801) with the outer edge projection region 999 of the entity 801 is detected, and projection of the boundary is performed. The position coordinates on the area 999 are calculated.
Similarly, the entity position recognition program 13b detects the outer edge position coordinates of the actual image 801a on the projection area 999 by analyzing the “captured image data” stored in the captured image data storage area 12a. , “Real image position information” is generated. The generated “entity location information” and “real image location information” are stored in the entity location information storage area 12b. When the process of S17 ends, the process proceeds to S18.

  In the process of S18 “first projection image data generation”, the projection image data generation program 13f refers to the “real image position information” stored in the entity position information storage area 12b, so that the target of the real image 801a is detected. The projection image controller 17 generates a “drawing command” for generating “first projection image data” for recognizing the position on the projection area 999 and projecting the related image 801 a on the first position that is superimposed on the real image 801 a. Output to. The projection image data generation program 13 f stores “related image data” stored in the related image data storage area 12 e in the VRAM 18. Based on the “drawing command”, the projection image data generation program 13 f performs image processing on the “related image data” stored in the VRAM 18 to generate “first projection image data”, and stores it in the VRAM 18. The image processing includes a process of arranging “related image data” so that the related image 801a is projected at the first position. Since the distance between the projection unit 20 and the projection area 999 is constant, the projection image data generation program 13f can recognize the coordinate position of the projection area 999. When the process of S18 is completed, the process proceeds to S19.

In the process of S19 “first projection image data projection”, the external output interface of the projection image controller 17 outputs the “first projection image data” stored in the VRAM 18 as the “image signal” to the projection unit 20. The projection unit 20 to which the “image signal” is input projects the related image 501a on the real image 801a (first position) (state shown in FIG. 2B). When the process of S19 ends, the process proceeds to the determination process of S20.
In the process of S18, in order to make the user visually recognize that the actual image 801a and the related image 501 are related, the projection image data generation program 13f causes the related image 501 to blink on the actual image 801a. It is preferable to output to the projection image controller 17 a “rendering command” that generates “first projection image data” that is displayed and appears as if the related image 501 is lifted from the actual image 801 a.

In the determination process of S20 “identification information disappears?”, The identification information acquisition program 13c determines whether or not 801b has disappeared. Specifically, the identification information acquisition program 13c analyzes “captured image data” stored in the captured image data storage area 12a, and whether the “captured image data” includes the identification information 801b of the entity 801. Determine whether.
When the identification information acquisition program 13c determines that there is no identification information 801b in the “captured image data” (YES in S20), the process proceeds to S40.
On the other hand, if the identification information acquisition program 13c determines that the “captured image data” includes the identification information 801b (NO in S20), the process proceeds to S21.
When the entity 801 is removed from the projection area 999, the identification information 801b is not captured, and therefore it is determined that the identification information 801b is not included in the “captured image data”.

  In the determination process of S21 “first projection image data projection completed?”, The projection image generation program 13f determines whether or not the projection of “first projection image data” has been completed. If the projection image generation program 13f determines that the projection of the “first projection image data” has been completed (YES in S21), the process proceeds to S22. On the other hand, when the projection image generation program 13f determines that the projection of the “first projection image data” has not been completed (NO in S21), the process returns to S17.

  In the process of S22 “Projection of third projection image data?”, The projection image generation program 13f determines whether or not the projection of “third projection image data” has been completed. When the projection image generation program 13f determines that the projection of the “third projection image data” has been completed (YES in S22), the process proceeds to S37. On the other hand, when the projection image generation program 13f determines that the projection of the “third projection image data” has not been completed (NO in S22), the process proceeds to S27.

  In the process of S27 “detecting the position of the entity”, the entity position recognition program 13b analyzes the “captured image data” stored in the captured image data storage area 12a, thereby identifying the entity on the projection area 999. The outer edge position coordinates of the object 801 and the actual image 801a are detected, and “entity position information” and “actual image position information” are generated and stored in the entity position information storage area 12b. The process of S27 is the same as the process of S17. When the process of S27 ends, the process proceeds to S28.

In the process of S28 “third projection image data generation”, the projection image data generation program 13f refers to “actual image position information” and “substance position information” stored in the entity position information storage area 12b. Thus, the positions of the real image 801a and the entity 801 on the projection area 999 are recognized, and the related image 501 gradually moves from the real image 801a (first position) to the outside (second position) of the real image 801a. A “drawing command” for generating the “third projection image data” to be moved is output to the projection image controller 17. Specifically, the projection image data generation program 13f calculates a position (third position) that equally divides the first position and the second position into a plurality. Each time the process of S28 is executed, the projection image data generation program 13f sequentially determines a third position closer to the second position, and the related image 501 is projected onto the third position. Projection image data "is generated. In the determination process of S22, the process of S27 to S29 is repeated until the projection image generation program 13f determines that the projection of the “third projection image data” is completed, and the related image 501 is moved from the first position. It gradually moves to the second position.
Alternatively, the projection image data generation program 13f sets “third projection image data” in which the related image 501 gradually moves from the actual image 801a (first position) to the outside (second position) of the actual image 801a. You can create video data.
The projection image data generation program 13 f performs image processing on the “related image data” stored in the VRAM 18 based on the “drawing command”, generates “third projection image data”, and stores it in the VRAM 18. When the process of S28 ends, the process proceeds to S29.

In the process of S29 “third projection image data projection”, the external output interface of the projection image controller 17 outputs “third projection image data” stored in the VRAM 18 to the projection unit 20 as an “image signal”. The projection unit 20 to which the “image signal” is input causes the related image 501 to be gradually moved from the real image 801a (first position) to the outside (second position) of the real image 801a. And project (state (C) in FIG. 2).
When the process of S29 ends, the process proceeds to S20.

  In the process of S37 “detecting the position of the entity”, the entity position recognition program 13b analyzes the “captured image data” stored in the captured image data storage area 12a to thereby identify the entity on the projection area 999. The outer edge position coordinates of the object 801 and the actual image 801a are detected, and “entity position information” and “actual image position information” are generated and stored in the entity position information storage area 12b. The process of S37 is the same as the process of S17. When the process of S37 ends, the process proceeds to S38.

  In the process of S38 “second projection image data generation”, the projection image data generation program 13f refers to “actual image position information” and “substance position information” stored in the entity position information storage area 12b. Thus, the position of the real image 801a and the entity 801 on the projection area 999 is recognized, and the related image 501 is statically projected to the position outside the real image 801a (second position). Is output to the projection image controller 17. When the process of S38 ends, the process proceeds to S39.

In the process of S39 “projection of second projection image data”, the external output interface of the projection image controller 17 outputs the “second projection image data” stored in the VRAM 18 to the projection unit 20 as an “image signal”. The projecting means 20 to which the “image signal” is input projects the related image 501 onto the second image of the actual image 801a (the second position) (state (D) in FIG. 2). When the process of S39 ends, the process proceeds to S20.
In the embodiment shown in FIG. 2, the second position is on the exposed projection surface 999 outside the entity 801. Thus, the related image 501 is clearly displayed by projecting the related image 501 to a position that does not overlap the entity 801. Further, in order for the user to recognize that the related image 501 is an image related to the real image 801, the related image 501 is projected in the vicinity of the corresponding entity 801.

  In the determination process of S40 “end?”, When the user selects to end the image display apparatus 100 by operating an operation unit (not shown), the main flow shown in FIG. 5 ends. On the other hand, when the user does not select to end the image display apparatus 100, the process returns to the determination process of S13.

  As described above, in the present embodiment, as shown in the processing of S27 to S29, the gap between the first position superimposed on the real image 801a and the second position outside the real image 801a is continuously complemented. The related image 501 is projected to the third position, which is the position to be moved, and the related image 501 is animated while being continuously moved from the first position to the second position. Therefore, the related image 501 is the real image 801a. Give the user a relevant impression.

When a predetermined time has elapsed since the related image 501 is projected onto the projection area 999, the projection image 501 is projected onto the projection area 999 while gradually moving from the second position to the first position. But it doesn't matter. This gives the user the impression that the related image 501 is stored in the real image 801a, and allows the user to perceive that the related image 501 projected on the projection area 999 is related to the real image 801a. It becomes possible.
Such control is performed in steps S37 to S39 in the flowchart shown in FIG. 5 (second image projection), and when a predetermined time has elapsed, steps S27 to S29 are performed (third image projection). Next, this is realized by executing the processing of S17 to S19 (first image projection).

(Second Embodiment)
The second embodiment will be described with respect to differences from the first embodiment. The second embodiment is an embodiment in which the brightness of the projected related image 501 differs between the entity 801 and the exposed projection surface 999 on which the entity 801 is not placed.
In the embodiment shown in FIG. 6, when the related image 501 moves a predetermined distance from the real image 801a (first position) to the outside of the real image 801a, the related image 501 is not projected onto the entity 801. A part of the image 501 is cut out and projected only onto the exposed projection area 999 (state (C) in FIG. 6). Alternatively, when the related image 501 moves from the first position to the outside of the real image 801a by a predetermined distance, the brightness of the related image 501 is lowered on the entity 801, and the related image 501 is displayed on the entity 801 and the projection area 999. Project to.

In the second embodiment, in the process of S28 of the main flow shown in FIG. 5, the projection image generation program 13f executes the “actual image position information” and “substance position” stored in the entity position information storage area 12b. By referring to the “information”, the positions of the real image 801a and the entity 801b on the projection area 999 are recognized, and the related image 501 is a predetermined distance from the real image 801a (first position) to the outside of the real image 801a. When moved, a part of the related image 501 is cut out so that the related image 501 is not projected on the entity 801, and “third projection image data” that is projected only on the exposed projection area 999 is generated. The “drawing command” is output to the projection image controller 17.
Alternatively, when the related image 501 moves a predetermined distance from the real image 801a (first position) to the outside of the real image 801a, the shadow image generation program 13f reduces the brightness of the related image 501 on the entity 801, A “drawing command” for generating “third projection image data” for projecting the image 501 onto the entity 801 and the projection area 999 is output to the projection image controller 17.

In this way, since the related image 501 is animated from the first position to a predetermined distance in the outward direction of the real image 801a, the user is made to recognize that the related image 501 is an image related to the real image 801a. Can do. On the other hand, when the related image 501 moves from the first position to the outside of the real image 801a by a predetermined distance, the related image 501 is not projected on the entity 801, or the related image 501 is projected with reduced brightness. . Therefore, it becomes easy for the user to visually recognize the explanatory text 801c or the like described on the entity 801.
Alternatively, the brightness of the related image 501 may be increased on the entity 801 and the related image 501 may be projected. In this case, the related image 501 is displayed highlighted on the entity 801. The image 501 is easier to visually recognize.

(Third embodiment)
The third embodiment will be described with respect to differences from the first embodiment. In the third embodiment, as the second position, the related image 501 is a blank area on at least one of the entity 801 and the exposed projection surface 999 that has a lightness of a certain level or more and a contrast of a certain level or less. Is an embodiment for projecting. First, the entity 801 is already placed in the projection area 999, and the related image 501 related thereto is projected on the projection area 999. In this state, the entity 802 is further placed on the projection area 999 (state (A) in FIG. 7). When the entity 802 is placed, the related image 502 is projected so as to be superimposed on the actual image 802a (first position) (the state shown in FIG. 7B). The related image 502 is projected onto a blank area (second position) having a lightness of a certain level or more and a contrast of a certain level or less while moving (state (C) in FIG. 7). As shown in FIG. 7C, the blank area may extend over the entity and the exposed projection area 999 as long as the above condition is satisfied.

  In the third embodiment, in the processes of S27 and S37 in the main flow shown in FIG. 5, the entity position recognition program 13b analyzes the “captured image data” stored in the captured image data storage area 12a. Thus, a blank area having a lightness of a certain level or more and a contrast of a certain level or less is detected. Then, the entity position recognition program 13b calculates the position coordinates on the projection surface 999 of the outer edge of the blank area, and stores it in the entity position information storage area 12b as “blank area position information”.

  In the process of S28, the projection image generation program 13f refers to the “real image position information” and “blank area position information” stored in the entity position information storage area 12b, thereby determining the real image 802a and the blank area. “Drawing” that recognizes the position on the projection area 999 and generates “third projection image data” in which the related image 502 gradually moves from the actual image 802a (first position) to the blank area (second position). Command ”is output to the projection image controller 17.

  In the process of S38, the projection image generation program 13f recognizes the position of the blank area on the projection area 999 by referring to the “blank area position information” stored in the entity position information storage area 12b. A “drawing command” for generating “second projection image data” for still-projecting the related image 502 in the blank area (second position) is output to the projection image controller 17.

  As described above, in the third embodiment, the related image is projected onto a blank area where the brightness is equal to or higher than a certain level and the contrast is equal to or less than a certain level. Therefore, even when a large number of entities are placed on the projection area 999, it is possible to project related images corresponding to the entities.

(Fourth embodiment)
The fourth embodiment will be described with respect to differences from the third embodiment. In the fourth embodiment, as shown in FIG. 8, when the related image cannot be projected onto the blank area, the related image 502 is reduced and projected onto the area.

  In the fourth embodiment, in the process of S28 of the main flow shown in FIG. 5, the projection image generation program 13f executes the “real image position information” and “blank area position” stored in the entity position information storage area 12b. By referring to the “information”, the positions of the real image 802a and the blank area on the projection area 999 are recognized, and if it is determined that the related image 502 cannot be projected onto the blank area, the real image 801a ( A “drawing command” for generating “third projection image data” that moves while the related image 502 is gradually reduced and displayed from the first position) to the blank area (second position) is output to the projection image controller 17.

  In the process of S38, the projection image generation program 13f recognizes the position of the blank area on the projection area 999 by referring to the “blank area position information” stored in the entity position information storage area 12b. The related image 502 is reduced to a size that can be projected onto a blank area, and a “drawing command” for generating “second projection image data” for statically projecting the reduced related image 501 onto the blank area (second position) is projected. Output to the image controller 17.

  As described above, in the fourth embodiment, when the related image 502 cannot be projected onto the blank area, the related image 502 is reduced and projected onto the area. Even if a large number of entities and the like are placed and the blank area is narrow, the related image 502 corresponding to the entity 802a can be projected.

  If the related image cannot be projected onto the region where the related image is to be projected, the technical idea that the related image is reduced and projected onto the region is applicable to the first embodiment. Needless to say.

(Summary)

  In the above description, the present invention has been described with respect to an example of an entity on which identification information and an actual image are printed. However, the identification information is included in the image display device of an entity having an image display device such as a PDA (Personal Digital Assistant). And a real image is displayed, a related image related to the real image is acquired from the related image data storage area 200b of the storage unit 200 based on the identification information, and the related image is projected onto the projection area 999. It goes without saying that the technical idea of the present invention can also be applied to the form.

  In the embodiment described above, the storage unit 200 is external to the image display device 100 and is connected to the input unit 19 via a LAN, WAN, or the Internet. However, the storage unit 200 is built in the image display device 100. It does not matter.

  It should be noted that a real image may be captured from “captured image data” stored in the captured image data storage area 12a, and the captured real image may be projected as a related image.

  Although the present invention has been described above in connection with the most practical and preferred embodiments at the present time, the present invention is not limited to the embodiments disclosed herein. However, the present invention can be appropriately changed without departing from the spirit or concept of the invention that can be read from the claims and the entire specification, and an image display device with such a change should also be understood as being included in the technical scope. I must.

9 Bus 10 Control unit 11 CPU
12 RAM
12a Captured image data storage area 12b Entity position information storage area 12c Identification information storage area 12d Information table search result storage area 12e Related image data storage area 13 ROM
13a imaging control program 13b entity position recognition program 13c identification information acquisition program 13d information table retrieval program 13e related image data acquisition program 13f projection image data generation program 15 imaging controller 17 projection image controller 18 VRAM
DESCRIPTION OF SYMBOLS 19 Input part 20 Projection means 30 Imaging means 100 Image display apparatus 200 Storage means 200a Information table storage area 200b Related image data storage area 801 Entity 801a Real image 801b Identification information 801c Description 802 Entity 802a Real image 802b Identification information 501 Image 502 Related image 999 Projected area

Claims (7)

A projecting means that is arranged to face a projection surface on which an entity on which a real image is described or an entity on which a real image is displayed is placed, and projects an image on the projection surface;
An imaging means for imaging the projection surface;
Entity position recognition means for recognizing a position on the projection surface of the real image from the picked-up image picked up by the image pickup means, and generating real image position information;
Identification information acquisition means for acquiring identification information of the entity;
Based on the identification information of the entity acquired by the identification information acquisition unit, the related image acquisition unit acquires the related image corresponding to the entity from the storage unit storing the related image related to the real image. When,
Based on the real image position information, first projection image data in which the related image is projected onto a first position that is a position to be superimposed on the real image, and an entity and a projection surface that are outside the real image Projection image data generation means for generating second projection image data in which the related image is projected to a second position which is at least one of the positions above;
Image signal generating means for outputting one of the first projection image data and the second projection image data as an image signal to the projection means and then outputting the other as an image signal to the projection means;
An image display device comprising: The projection image data generation means relates to the third position, which is a position that continuously complements between the first position where the first projection image data is projected and the second position where the second projection image data is projected. Generating third projection image data on which the image is projected;
The image signal generation means further outputs one of the first projection image data and the second projection image data as an image signal to the projection means, and outputs the other as an image signal to the projection means. The image display device according to claim 1, wherein the third projection image data is output to the projection unit as an image signal before performing.   The image display device according to claim 1, wherein the projection image data generation unit makes the brightness of the related image to be projected different between the object and the exposed projection surface.   The projection image data generation means projects a related image on a region on at least one of an entity and an exposed projection surface, the brightness of which is greater than or equal to a certain level and the contrast is not greater than a certain level. The image display device according to claim 1 or 2.   The image display device according to claim 1, wherein the projection image data generation unit projects a related image on the exposed projection surface.   5. The projection image data generation means, when the related image cannot be projected onto the area where the related image is to be projected, reduces the related image and projects it onto the area. 5. The image display device according to 5. Based on the identification information of the entity acquired by the identification information acquisition unit, further comprising presence determination means for determining whether a related image corresponding to the entity exists in the storage unit;
7. The related image acquisition unit acquires the related image from the storage unit when the presence determination unit determines that the related image exists in the storage unit. An image display device according to claim 1.

Images