JP2011180712A - Projection type image display apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To achieve touch panel operation for a projected image of a projector. <P>SOLUTION: A projection part 42 projects an image to a screen 16. A camera 12 picks up an area including at least the image projected to the screen 16. An infrared camera 14 picks up an upper space of the screen 16. A contact determination part 38 determines whether a user's finger is brought into contact with the screen 16, based on the image picked up by the infrared camera 14. When the contact determination part 38 determines contact of the user's finger, a coordinate determination part 34 outputs the coordinates of a tip of the user's finger as a pointing position to the projected image. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a projection display apparatus that projects an image on a screen.

  2. Description of the Related Art Conventionally, a personal computer is connected to a projection video display device (hereinafter also referred to as a projector as appropriate) and an image on a display is projected onto a screen. When a presentation or the like is performed using this projection image, a mouse operation on the personal computer side is required when performing operations on the projection image such as enlargement / reduction or click.

  Several techniques are known to enable operations on projected images without leaving the screen. Japanese Patent Application Laid-Open No. H10-228561 describes that a touch screen can be detected by configuring a projector screen with a touch panel. Japanese Patent Application Laid-Open No. 2004-228688 discloses a technique for detecting an irradiation position of a laser pointer with respect to an image projected on a screen by a camera and projecting an appropriate image corresponding to the irradiation position on the screen.

JP 2004-212488 A JP 2008-27080 A

  The techniques described in Patent Documents 1 and 2 require a special device such as a touch panel or a laser pointer in order to realize a touch operation on a projected image.

  The present invention has been made in view of such circumstances, and an object thereof is to provide a technique for realizing a touch operation on a projected image without a user using a special device.

  One embodiment of the present invention is a projection display apparatus. The apparatus includes a projection unit that projects an image on a screen, a first imaging unit that captures an area including at least the image projected on the screen, and a second imaging unit that captures an upper space of the screen. And a contact determination unit that determines whether or not a predetermined object is in contact with the screen based on an image captured by the second imaging unit, and a contact determination unit that determines that the predetermined object is in contact with the screen, A coordinate determination unit that outputs the coordinates of the predetermined object as a pointing position with respect to the video based on the image captured by the first imaging unit;

  Another aspect of the present invention is also a projection display apparatus. The apparatus moves above the screen from a projection unit that projects an image on a screen, an imaging unit that captures at least a region including the image projected on the screen, and an image captured by the imaging unit. A real image detecting unit for detecting a real image of a predetermined object, a shadow detecting unit for detecting a shadow of the predetermined object generated by projection light from the projection unit, from an image captured by the imaging unit, and a real image of the predetermined object; When the distance between corresponding points of the shadow is equal to or less than a predetermined threshold value, it is determined that the predetermined object is in contact with the screen, and the contact determination unit determines that the predetermined object is in contact with the screen. A coordinate determining unit that outputs the coordinates of the predetermined object as a pointing position with respect to the video.

  Yet another embodiment of the present invention is also a projection display apparatus. The apparatus includes: a projection unit that projects an image on a main screen; an imaging unit that captures at least an area including the image projected on the main screen; and the main screen in a projection area by the projection unit. A sub-screen specifying unit for specifying the position and range of the sub-screen from the image captured by the imaging unit, and a predetermined screen to be displayed on the sub-screen in accordance with the range of the sub-screen A sub video processing unit that forms the sub video, and a composition that superimposes the sub video on the sub screen position specified by the sub screen specifying unit and synthesizes the predetermined main video to be displayed on the main screen A section.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, and the like are also effective as an aspect of the present invention.

  According to the present invention, it is possible to realize a touch operation on a projected image without a user using a special device.

It is a figure which shows the positional relationship of the projection type video display apparatus which concerns on Embodiment 1, and a screen, FIG. 1 (a) is a figure which shows a mode that it saw from the side, FIG.1 (b) is seen from upper direction. 2 is a functional block diagram of the projection display apparatus according to Embodiment 1. FIG. (A) shows an example of an image taken by the camera, and (b) shows an example of an image taken by the infrared camera 14. It is a figure which shows the positional relationship of the projection type video display apparatus concerning Embodiment 2, and a screen. 6 is a functional block diagram of a projection display apparatus according to Embodiment 2. FIG. (A), (b) is a figure which shows an example of the user's finger | toe F and shadow S imaged with a camera. It is a figure which shows the positional relationship of the projection type video display apparatus concerning Embodiment 3, and a screen. It is a figure which shows an example of the image | video projected on a main screen and a subscreen by a projection type video display apparatus. 6 is a functional block diagram of a projection display apparatus according to Embodiment 3. FIG. It is a figure explaining the correlation of a subscreen and a sub video file.

Embodiment 1 FIG.
FIG. 1 is a diagram illustrating a positional relationship between the projection display apparatus 10 and the screen 16 according to the first embodiment. FIG. 1A is a side view, and FIG. 1B is a top view. Indicates.

  A projection port 18 for image light is formed on the front surface of the projection display apparatus 10. Since the projection display apparatus 10 is a so-called short focus projector, the projection port 18 is provided to face obliquely downward. As a result, an image can be projected onto a projection surface located close to the apparatus 10.

  In FIG. 1, the projection display apparatus 10 projects an image on a screen 16 disposed on a plane such as a table top. The screen 16 is preferably a dedicated screen having high diffusibility. Alternatively, an image may be projected directly on a table top or the like without installing a dedicated screen.

  The camera 12 is installed adjacent to the projection port 18 (for example, above). The camera 12 is preferably a digital camera provided with a solid-state image sensor and a signal processing circuit. The solid-state imaging device is, for example, a complementary metal oxide semiconductor (CMOS) image sensor or a charge coupled device (CCD) image sensor. The signal processing circuit performs various signal processing such as A / D conversion and conversion from the RGB format to the YUV format on the signal output from the solid-state imaging device.

  As shown in FIG. 1B, the lens position and the angle of view are set so that the camera 12 can capture at least the entire projection area projected from the projection port 18 onto the screen 16. The image captured by the camera 12 is used to detect the two-dimensional coordinates of an object that approaches or touches the screen 16, for example, the user's finger F.

  Below the projection port 18 is installed an infrared camera 14 that captures an image through a filter that transmits near infrared rays. The infrared camera 14 is arranged so that its optical axis is substantially parallel to the surface of the screen 16, and the lens position and the angle of view are set so that at least the image above the screen 16 is imaged. The image captured by the infrared camera 14 is used to detect an object that touches the screen 16, for example, the user's finger F.

  The camera 12 and the infrared camera 14 may be separate from the projection display apparatus 10 or may be built in the apparatus 10.

  A personal computer 20 is connected to the projection display apparatus 10 and provides an image signal to be projected. Alternatively, the projection display apparatus 10 may be provided with an external recording medium drive such as an SD card slot, and the apparatus 10 may be configured to project an image in the recording medium without using an external device.

  The projection display apparatus 10 detects the two-dimensional coordinates of the user's finger F that touches the screen 16, thereby realizing an operation like a touch panel on the image projected on the screen.

  FIG. 2 is a functional block diagram of the projection display apparatus 10. This configuration can be realized by an arbitrary processor, memory, other LSI, optical component, and mechanical component in terms of hardware, and is realized by a program loaded in the memory in terms of software. Describes functional blocks realized through collaboration. Therefore, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  The personal computer (PC) 20 supplies a video signal to be projected to the projection display apparatus 10. The video signal is, for example, a moving image such as a movie, an application image, a web browser image, or the like, but is not limited thereto. The video holding unit 40 temporarily holds a video signal supplied from the outside.

  The projection unit 42 includes a light source, a light modulation unit that modulates light incident from the light source according to a supplied video signal, a focus lens that adjusts a focal position of light incident from the light modulation unit, and the like. ing. The image light generated by the light modulation unit is projected onto the screen 16 via the focus lens.

  As the light source, a halogen lamp having a filament-type electrode structure, a metal halide lamp having an electrode structure for generating arc discharge, a xenon short arc lamp, a high-pressure mercury lamp, an LED lamp, or the like can be employed.

  A DMD (Digital Micromirror Device) can be employed for the light modulation unit. The DMD includes a plurality of micromirrors corresponding to the number of pixels, and desired video light is generated by controlling the direction of each micromirror according to each pixel signal.

  The lens position of the focus lens is moved on the optical axis by a lens driving unit such as a stepping motor, a voice coil motor (VCM), or a piezo element.

  The camera 12 captures an image projected on the screen 16 as a main subject. The infrared camera 14 captures an infrared image above the screen 16.

  The control unit 30 detects a touch operation on the screen 16 by the user's finger. The control unit 30 includes a target detection unit 32, a coordinate determination unit 34, a tip detection unit 36, and a contact determination unit 38.

  The target detection unit 32 acquires an image captured by the camera 12 and an image projected on the screen by the projection unit 42. Then, the user's finger as a target is recognized by taking the difference between the projected image and the captured image. Alternatively, the user's finger may be recognized by executing matching with a sample image prepared in advance.

  The coordinate determination unit 34 recognizes the tip of the finger from the image of the user's finger recognized by the target detection unit 32, and determines two-dimensional coordinates in the projected image of the finger tip. The coordinates can be determined from the pixel position at the tip of the finger and the size and position of the pixel area occupied by the projected image in the captured image.

  The tip detection unit 36 acquires an infrared image captured by the infrared camera 14 and detects the tip of the user's finger facing the screen. This recognition can be realized by matching with a sample image prepared in advance, for example.

  The contact determination unit 38 determines whether or not the finger tip of the user detected by the tip detection unit 36 is in contact with the screen 16 in the infrared image.

  Only when the contact determination unit 38 determines that the finger is in contact with the screen, the two-dimensional coordinates determined by the coordinate determination unit 34 are transmitted to the personal computer 20 as pointing position information. Further, when it is determined by the contact determination unit 38 that the user's finger is tapped, that is, contact that is intermittent in a short time is made, it is transmitted to the personal computer 20 as a tap event.

  The personal computer 20 provides pointing position information and tap event information as inputs to the running application. For example, when a tap event is received at a certain coordinate, processing is performed assuming that a click at that coordinate on the screen has been performed. By doing this, it is possible to handle the screen like a touch panel, such as processing the movement with the finger in contact with the screen as a cursor movement or processing as a tap event click.

  3A shows an example of an image captured by the camera 12, and FIG. 3B shows an example of an image captured by the infrared camera 14. In FIG. 3A, the coordinate determining unit 34 determines the two-dimensional coordinates of the tip of the finger at each of the positions A and B. In FIG. 3B, the contact determination unit 38 determines whether or not the tip of the finger has contacted the screen 16 at each of the positions A and B. When the position is A, it is determined as non-contact and when the position is B, it is determined as contact. Only when the finger is in position B where the finger is in contact with the screen, the two-dimensional coordinates of the finger tip are transmitted to the personal computer as pointing position information.

  As described above, according to the projection display apparatus 10 according to the first embodiment, the two-dimensional coordinates of the user's finger on the screen are determined and whether or not the user's finger has touched the screen is detected. can do. Since the pointing position information is determined by the action when the finger is touching the screen, erroneous input can be reduced compared to the case where the pointing position is determined only by the movement of the user's hand on the screen.

  According to this embodiment, in a usage mode in which an image is projected onto a table using a short focus projector or the like, a user around the projected image can perform a touch operation on the image with his / her hand. A very intuitive operation is realized.

  In addition, since a special screen having a contact detection function is not used, the projection target of the projection display apparatus 10 is not limited. Further, the user does not need to use a touch pen or a laser pointer having a special function.

  In addition, by permitting recognition of a plurality of targets in the captured image, multi-touch, that is, touches to a plurality of locations by a plurality of users can be processed. When a plurality of targets are recognized in the captured image, it may be configured such that which target should be recognized with priority can be changed by a finger tap operation or the like. Alternatively, when a plurality of targets are recognized, the sum of the target information may be input. For example, an intermediate point of the touched coordinates may be determined as a two-dimensional coordinate, or a vector in the movement direction of each touch may be obtained, and the cursor may be moved in the direction of the combined vector.

  In the above description, a personal computer is used as a video signal supply source. However, the projection display apparatus 10 may incorporate a computer function.

  Instead of the infrared camera 14, a digital camera similar to the camera 12 may be used to capture an image above the screen 16. In this case, the tip of the user's finger is recognized by matching with a finger sample image prepared in advance.

  A special function may be associated with a tap on the screen, that is, a continuous touch operation. For example, at the time of tapping operation, it is set to change the direction of the projected image so that the side close to the tapped position is down, to set to enlarge a predetermined area, or to a predetermined projector menu May be set to be displayed.

When an operation is performed on a projected image with a finger, the projected image is not projected in a portion that becomes a shadow of the user's hand. In order to avoid this, sub-windows for operation recognition are projected and displayed at the four corners of the projected image, and finger operations (drag and drop, tap, etc.) in this sub-window are the main projected images. You may comprise so that it may be recognized as operation with respect to. The sub window may be superimposed on the main projection image or may be displayed as a separate window.
A virtual keyboard may be projected and displayed on a part of the projected image, and keyboard input may be performed by touch recognition by the camera.

  A plurality of infrared cameras for imaging the upper part of the screen may be provided, and imaging may be performed from different directions. In this way, for example, even if there are a plurality of recognition targets on the same straight line for one camera, they can be distinguished by another camera.

Embodiment 2. FIG.
FIG. 4 is a diagram showing a positional relationship between the projection display apparatus 50 according to the second embodiment and the screen 16. Elements denoted by the same reference numerals as those in FIG. 1 have the same structure and function as described above. Similar to the first embodiment, the projection display apparatus 50 projects an image on a screen 16 arranged on a plane such as a table top.

  The camera 52 is a digital camera similar to the camera 12 of FIG. 1, and is installed at a position different from the projection port 18 in the horizontal direction and the height direction. The camera 52 is set with a lens position and an angle of view so that at least the entire projection area projected from the projection port 18 onto the screen 16 can be imaged.

  In the second embodiment, a touch operation on the screen 16 by the user's finger F is detected using the single camera 52 by using the shadow S of the user's finger F generated by the projection light.

  FIG. 5 is a functional block diagram of the projection display apparatus 50 according to the second embodiment. Also in this figure, those skilled in the art will understand that each functional block can be realized in various forms by hardware only, software only, or a combination thereof.

  The configurations of the personal computer 20, the video holding unit 40, and the projection unit 42 are the same as those described with reference to FIG. The control unit 60 recognizes a touch operation on the screen 16 by the user's finger. The control unit 60 includes a finger detection unit 62, a shadow detection unit 64, a contact determination unit 66, and a coordinate determination unit 68.

  The finger detection unit 62 acquires an image captured by the camera 52 and an image projected on the screen by the projection unit 42. Then, by taking the difference between the projected image and the captured image, the real image of the user's finger F as a target is recognized. Alternatively, the user's finger may be recognized by executing matching with a sample image prepared in advance.

  The shadow detection unit 64 acquires an image captured by the camera 52 and detects a shadow S of the user's hand. For example, an area where the luminance of the pixel is smaller than a predetermined threshold value may be detected as the shadow S.

  The contact determination unit 66 specifies the tip of the real image of the finger F and the tip of the shadow S, and determines whether the finger is in contact with the screen based on the distance between them. For example, when the distance between the two is less than or equal to a predetermined threshold, it may be determined that the screen is in contact.

  The coordinate determination unit 68 determines two-dimensional coordinates in the projected image of the finger tip when the contact determination unit 66 determines that the finger is in contact with the screen. The coordinates can be determined from the pixel position at the tip of the finger and the position and size of the pixel area occupied by the projected image in the captured image.

  The two-dimensional coordinates determined by the coordinate determination unit 68 are transmitted to the personal computer 20 as pointing position information. Further, when the user's finger is tapped by the contact determination unit 66, that is, when the contact is intermittent in a short time, it is transmitted to the personal computer 20 as a tap event. Processing in the personal computer 20 is the same as that in the first embodiment.

  6A and 6B show an example of the user's finger F and shadow S imaged by the camera. FIG. 6A is a captured image when the user's finger is away from the screen. In this case, since the tip of the real image of the finger F and the tip of the shadow S are separated from each other, it can be determined that the screen is not touched. FIG. 6B is a captured image when the user's finger is in contact with the screen. In this case, since the tip of the real image of the finger F and the tip of the shadow S substantially coincide with each other, it can be determined that the finger F is in contact with the screen.

  As described above, according to the projection display apparatus according to the second embodiment, it is possible to detect contact of the user's finger with the screen based on an image captured by a single camera. As in the first embodiment, since the pointing position information is determined by the action when the finger is touching the screen, it is erroneously input as compared with the case where the pointing position is determined only by the movement of the user's hand on the screen. Can be reduced. Further, it is not necessary to use a special screen having a contact detection function, a touch pen having a special function, a laser pointer, or the like.

By allowing a plurality of detections of the real image of the user's finger F and the shadow S in the captured image, it is possible to process multi-touch, that is, touches to a plurality of locations by a plurality of users.
Two or more cameras may be set apart in the horizontal direction. In this way, it is possible to reduce the number of shaded areas that are blocked by the user's hand, making it easier to support multi-touch.

Embodiment 3 FIG.
FIG. 7 is a diagram showing a positional relationship between the projection display apparatus 100 and the screen 16 according to the third embodiment. In the figure, elements denoted by the same reference numerals as those in FIG. 1 have the same structure and function as described above. Similar to the first and second embodiments, the projection display apparatus 100 projects an image on a screen 16 arranged on a plane such as a table top.

  The camera 54 is installed at a position different from the projection port 18 in the horizontal direction and the height direction. The camera 54 has a lens position and an angle of view so that at least the entire projection area projected onto the screen 16 from the projection port 18 can be imaged. It is preferable that the camera 54 can capture an image above the screen wider than the camera 52 in the second embodiment.

  In the third embodiment, apart from the screen 16 (hereinafter referred to as “main screen”), a sub-screen 80 that has a smaller area than the main screen and can be carried is used. The sub screen 80 is preferably a white plate, and any material can be used as long as it is light and strong. At least one surface of the sub-screen 80 is preferably surface-treated so as to have the same diffusibility as the main screen 16.

  FIG. 8 is a diagram illustrating an example of an image projected by the projection display apparatus 100. A main image 72 is projected and displayed on the main screen 16. When the sub screen 80 is outside the projection area, the main image 72 is not changed. When the sub screen 80 is brought into the projection area, a predetermined sub image 82 is projected and displayed on the sub screen 80. At this time, image light corresponding to an image obtained by synthesizing the main image 72 and the sub image 82 is projected from the projection port 18.

  A predetermined marker is printed on the surface of the sub-screen 80 so that the sub-screen 80 in the projection area can always be recognized. The markers are preferably provided at the four corners or sides of the sub screen 80 so that the sub screen 80 can be distinguished from top, bottom, left, right, and rotation. Any marker can be used as long as it can be recognized in the image.

  The marker may be printed with ink that can be imaged only at a specific wavelength, such as near infrared rays, and is not visible to the naked eye. In this case, a filter that transmits a specific wavelength is attached to the lens of the camera 54. By doing so, there is no possibility that the visibility of the sub video is lowered by the marker. Furthermore, the marker can be imaged without being affected by projection light or external light.

  Various sub-images can be displayed on the sub-screen. For example, an operation menu screen for the main video may be displayed as a sub video. In this way, the menu screen can be freely moved by hand, or the menu screen can be physically pulled to hand, so that the operability is improved. Alternatively, a material video (for example, a manual, a commentary, an annotation, etc.) for the main video may be displayed as a sub video on the sub screen.

  FIG. 9 is a functional block diagram of the projection display apparatus 100 according to the third embodiment. Also in this figure, those skilled in the art will understand that each functional block can be realized in various forms by hardware only, software only, or a combination thereof.

  The image holding unit 40 and the projection unit 42 are the same as those in the first and second embodiments.

  The control unit 110 acquires an image captured by the camera 54 and executes processing such as scaling and fitting of the main video and the sub video. The control unit 110 includes a marker recognition unit 112, a sub screen identification unit 114, a related file acquisition unit 116, a file storage unit 118, a sub video processing unit 120, a main video processing unit 122, and a synthesis unit 124.

  The marker recognition unit 112 recognizes a marker from the captured image. This recognition can be realized by matching with a sample image prepared in advance. As will be described later, when the marker is a two-dimensional code, the content embedded in the code can be read using a corresponding program.

  The sub-screen specifying unit 114 recognizes four corners or four sides of the sub-screen 80 based on the recognized marker, and specifies the position, orientation, size, and shape of the sub-screen.

  When a specific sub video file is associated with the marker recognized by the marker recognition unit 112, the related file acquisition unit 116 acquires the file from the file storage unit 118 and passes it to the sub video processing unit 120. This file association will be described later in detail.

  The sub video processing unit 120 performs scaling and fitting of the sub video according to the size and shape of the sub screen specified by the sub screen specifying unit 114. Since such processing is well known, detailed description thereof is omitted.

  The main video processing unit 122 performs scaling and fitting of the main video in accordance with the size and shape of the main screen specified by the main screen specifying unit (not shown).

  The synthesizing unit 124 acquires the main video and the sub video in units of frames. Then, the sub video frame is superimposed and synthesized on the main video frame in accordance with the position and orientation of the sub screen specified by the sub screen specifying unit 114. The synthesized frame is sent to the video holding unit 40 as a video signal.

  As described above, the main video displayed on the main screen and the sub video displayed on the sub screen are scaled and fitted separately. Thereafter, the synthesized image is formed as a synthesized image by the synthesizing unit 124, and the video signal is supplied to the video holding unit 40. As a result, even when the sub screen moves or rotates within the projection area, it is possible to fit the sub video on the sub screen in real time.

  With the configuration as described above, it is possible to project and display different images on the main screen and the sub screen using the same projector optical system.

  A sub-image enlargement / reduction function may be associated with the vertical movement of the sub-screen 80. For example, the sub image may be set to be enlarged when the sub screen 80 is moved upward (in the direction of arrow C in FIG. 7), and the sub image may be reduced when the sub screen 80 is moved downward.

  If the sub screen 80 has a certain thickness, a marker is also printed on the side surface, and this is recognized in the captured image of the camera, whereby the vertical movement distance of the sub screen can be obtained. Alternatively, the vertical movement of the sub-screen may be estimated based on the change in the sub-screen area in the captured image. That is, the vertical movement distance can be estimated from the area change amount within a predetermined time or at a predetermined frame interval.

  Subsequently, the sub video processing unit 120 sets an enlargement rate or a reduction rate corresponding to the vertical movement distance of the sub screen, enlarges or reduces the sub video at the ratio, and supplies the sub video to the combining unit 124. In this way, only the sub video on the sub screen can be enlarged and reduced without changing the main video.

  A plurality of sub-screens 80 may be prepared so that different sub-images can be automatically projected for each sub-screen. FIG. 10 is a diagram for explaining the association between the sub-screen and the sub-video file.

  As shown in FIG. 10, different markers M are printed in advance on the sub-screens A and B, respectively. These markers are associated with the video file to be projected in advance and stored in the file storage unit 118. When any of the sub-screens is placed in the projection area, the marker recognition unit 112 reads the marker, the related file acquisition unit 116 acquires the video file associated with the read marker, and passes it to the sub video processing unit 120. In this way, it is possible to easily switch the sub video projected on the sub screen simply by replacing the sub screen within the projection area.

  Alternatively, the marker on the sub screen 80 may be a two-dimensional code such as a QR code. In this case, the association between the sub screen 80 and the sub video file may not be performed in advance. For example, the directory and file name in the storage device are described in the two-dimensional code, and when the marker recognition unit 112 reads the two-dimensional code, the related file acquisition unit 116 searches for the read file. Also good. Alternatively, an IP (Internet Protocol) address or a URL (Universal Resource Locator) may be described in the two-dimensional code. In this case, when the marker recognizing unit 112 reads the two-dimensional code, the browser is activated, the corresponding web page is acquired via the network, and the projected image display is performed so that the web page is projected and displayed on the sub-screen. An apparatus may be configured.

  As described above, according to the third embodiment, when a main video is projected on the main screen and the actual sub-screen is moved into the projection area, a sub-video different from the main video is displayed on the sub-screen. Can be projected and displayed. As a result, a more intuitive operation such as physically moving the sub-screen can be performed. Further, if the function of enlargement / reduction is associated with the vertical movement of the sub-screen, the enlargement / reduction process can be performed with a more intuitive operation without the need to display a menu screen or the like.

  In the above description, it is assumed that the function of a personal computer is embedded in the projection display apparatus 100. However, by connecting the projection display apparatus and the personal computer and installing software having a function corresponding to the above-described control unit 110 in the computer, it is possible to exert the same function.

  The present invention has been described based on several embodiments. This embodiment is an exemplification, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are within the scope of the present invention. is there.

  Embodiments 1 and 3 and Embodiments 2 and 3 can be implemented in combination.

  In addition to the image light from the projection port, illumination for illuminating the user's finger may be provided. In this case, illumination with a specific wavelength such as near infrared rays may be used, and a user's finger may be imaged with a camera in which a filter that selectively transmits this wavelength is attached to the lens. By doing so, it is possible to reduce the influence of projection light and external light when recognizing the user's finger.

  In the embodiment, the touch operation with the user's finger has been described. However, when the user performs a touch operation using a physical pointing device, for example, a pointing stick, each embodiment is configured with the same configuration as above. Can be applied.

In the embodiment, the case where an image is projected onto a horizontal surface such as a table top using the projection display apparatus has been described. However, the present invention can also be applied to the case where an image is projected onto a vertical surface such as a wall surface. . In the latter case, it becomes possible for the presenter to operate the screen while touching the projected image without using a tool in a presentation or the like.
Even when an image is projected on a curved surface instead of a flat surface, the present invention can be applied by appropriately setting the number and orientation of cameras to be installed.

  10, 50, 100 Projection-type image display device, 12, 52 camera, 14 infrared camera, 16 screen, 18 projection port, 20 personal computer, 30, 60, 110 control unit, 32 target detection unit, 34 coordinate determination unit, 36 Tip detection unit, 38 Contact determination unit, 40 Video holding unit, 42 Projection unit, 62 Finger detection unit, 64 Shadow detection unit, 66 Contact determination unit, 68 Coordinate determination unit, 72 Main video, 80 Sub screen, 82 Sub video, 112 marker recognition unit, 114 sub-screen specifying unit, 116 related file acquisition unit, 118 file storage unit, 120 sub video processing unit, 122 main video processing unit, 124 synthesis unit.

Claims (6)

A projection unit that projects an image on a screen;
A first imaging unit for imaging at least a region including an image projected on the screen;
A second imaging unit for imaging the space above the screen;
A contact determination unit that determines whether a predetermined object is in contact with the screen based on an image captured by the second imaging unit;
A coordinate determination unit that outputs coordinates of the predetermined object as a pointing position with respect to the video based on an image captured by the first imaging unit when it is determined by the contact determination unit;
A projection-type image display device comprising: A projection unit that projects an image on a screen;
An imaging unit for imaging at least an area including an image projected on the screen;
A real image detection unit for detecting a real image of a predetermined object moving above the screen from an image captured by the imaging unit;
A shadow detection unit that detects a shadow of the predetermined object generated by the projection light from the projection unit, from an image captured by the imaging unit;
A contact determination unit that determines that the predetermined object is in contact with the screen when the distance between the corresponding point of the real image of the predetermined object and the shadow is equal to or less than a predetermined threshold;
A coordinate determination unit that outputs the coordinates of the predetermined object as a pointing position with respect to the video when it is determined by the contact determination unit to be in contact;
A projection-type image display device comprising:   3. The projection display apparatus according to claim 1, wherein the contact determination unit outputs a tap operation when it is determined that the contact is intermittently performed in a short time by the predetermined object. A projection unit that projects an image on the main screen;
An imaging unit for imaging at least an area including an image projected on the main screen;
A sub-screen specifying unit that specifies the position and range of the sub-screen from an image captured by the imaging unit when a sub-screen separate from the main screen exists in the projection area of the projection unit;
A sub video processing unit for shaping a predetermined sub video to be displayed on the sub screen in accordance with the range of the sub screen;
A synthesizing unit that superimposes and synthesizes the sub video at a sub screen position specified by the sub screen specifying unit with respect to a predetermined main video to be displayed on the main screen;
A projection-type image display device comprising: A marker recognition unit for recognizing a predetermined marker formed on the sub-screen;
A sub video acquisition unit for acquiring a sub video pre-associated with the recognized marker;
The projection display apparatus according to claim 4, further comprising:   6. The projection video display apparatus according to claim 4, wherein the sub video processing unit performs an enlargement or reduction process on the sub video in accordance with a vertical movement of the sub screen with respect to the main screen. .

Images