JP2005346453A - Image display device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a low-cost image display device capable of acquiring characteristics of a written image and specifying a user writing information written on a writing face without requiring any dedicated camera, a sensor or the like taking an image of the user. <P>SOLUTION: This image display device is provided with a characteristic acquisition part 11 acquiring characteristics of the written image formed on the writing face 5 based on an image projected to the writing face 5. The characteristic acquisition means 11 acquires characteristics of the written image based on the direction of the written image written on the writing face 5, a time of writing operation, or an area on the writing face 5. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image display device, for example, an image display device configured to synthesize a projected image projected on a display surface and an image written on a writing surface.

  In recent years, presentation systems using personal computers have been widely used. In this presentation system, a created document is projected and displayed on a large screen using a projector, and coordinates are directly input to the display screen on the screen, or a computer screen is projected onto the screen by a projector and a cursor is operated. By this, it points to the arbitrary position of the material.

  A touch sensor is widely used as a means for direct input to the display screen. When touching the screen with a finger or other pointing device, the touch sensor detects this, and the touched position is detected as a pointing position. There is something that moves the cursor.

  However, in the touch sensor, a plurality of people cannot operate or write on the touch sensor surface at the same time, and the hand cannot be put on the touch sensor surface.

  There exists a thing as shown in patent document 1 as what eliminates such a malfunction. This is a projection display device that includes a screen that forms a predetermined projection image, a projector that projects an image on the screen, and a camera that captures an image of the screen, and the screen uses a whiteboard marker. A writing surface for writing is provided.

  Then, the user writes directly on the screen while viewing the computer image projected on the screen, and when the writing is completed, the user captures the screen surface with the built-in camera and extracts the written image. In what is described in Patent Document 1, a written image can be extracted and stored as electronic information so that it can be used effectively.

  However, since the characteristics described in Patent Document 1 are not associated with the information written on the screen, such as the direction of the written information, it is possible to specify the user who performed the writing. There is a problem that the file becomes difficult to view when handling the stored electronic information.

  Specifically, when multiple users write on the screen, the position of the user can be specified, but the information actually written on the screen cannot be associated with the user. For example, information written by a specific user cannot be extracted from the stored file, and the stored electronic information cannot be used effectively.

  In addition, when a camera is installed on the back of the screen when the information written on the screen is captured by the camera, the characters captured by the camera are inverted 180 °, so that the written characters are correct. Although it is necessary to invert to the position (the same direction as the characters of the file), there are some meaningless images in the written image, such as arrows, figures, underlines, etc. Since the information shown in Patent Document 1 cannot associate the information actually written on the screen with the user, it cannot be specified whether the information is the information written on the screen by the same user. It is not possible to invert only a specific user's character.

  On the other hand, there are those described in Patent Document 2 or Patent Document 3 that can specify the position of a user who performs writing.

  The device described in Patent Document 2 is a screen display input device that is provided with touch-type screen input means on the front surface of the display screen, displays information on the screen, and enables handwriting input by touching the screen input means. Then, the direction and distance of deviation of the operator's eye position from a predetermined reference is detected using an image captured by the camera device provided on the upper surface of the display screen, and the detected value is stored in the correction amount storage means. By reading out the corrected amount and correcting the coordinate value input from the screen input means, it is possible to accurately grasp the position of the operator and correct the coordinate value input from the screen input means. I am doing so.

  Further, Patent Document 3 shows a touch panel with a reference setting function that can be imaged, and includes four light emitting diodes attached at specific positions in a peripheral region of the information input surface, and input / output of the light emitting diodes. And a flashing control circuit for controlling the flashing, and the flashing control circuit controls the flashing so that the flashing patterns of the light emitting diodes are different.

The device described in Patent Document 3 makes it possible to easily identify the position of a light emitting diode when a touch panel is imaged as compared with the case where the lighting pattern of the light emitting diode is controlled to be non-flashing.
JP 2003-143348 A JP 11-296304 A JP 2000-172430 A

  However, since it is necessary to image a user with the camera device provided on the upper surface of the display screen, it is necessary to provide a dedicated camera device for imaging the user, which is described in Patent Document 2 and the manufacturing cost. In addition, the characteristics of the written information, such as the direction of the information, are not associated with the information written on the screen, so it is not possible to specify which user made the write and save it. When dealing with electronic information, the file is still difficult to see.

  Moreover, since it is necessary to provide four light emitting diodes in a specific position in the peripheral region of the touch panel, the one described in Patent Document 3 increases the manufacturing cost and the direction of written information. Because the characteristics are not associated with the information written on the screen, it is not possible to determine which user the writing was made, and the file is still difficult to see when handling stored electronic information. .

  The present invention has been made to solve such a problem, and it is possible to acquire characteristics of a written image without using a dedicated camera or sensor for imaging a user, and information written on a writing surface. It is an object of the present invention to provide a low-cost image display apparatus that can specify which user has made the above.

  An image display device according to the present invention includes an image projection unit that projects an image, a display surface that displays an image projected from the image projection unit, a writing surface that directly forms a writing image so as to overlap the projection image, An imaging unit that captures a writing image formed on the writing surface, a writing image extracted from the image captured by the imaging unit, and a part of the extracted writing image and a projection image projected by the projection unit Or an image extracting / combining unit that generates a combined image by combining all, and an acquiring unit that acquires characteristics of a written image formed on the writing surface based on a shadow reflected on the writing surface Composed.

  With this configuration, the characteristics of the written image are acquired and written on the writing surface by using the shadow of the user reflected on the writing surface, which is normally removed as image noise, without using a dedicated camera or the like. Since the information can be associated with the user, it can be specified by which user the writing was made. As a result, it is possible to effectively use the stored written image while preventing the manufacturing cost of the image display device from increasing.

  In the image display device of the present invention, the acquisition unit acquires the characteristics of the written image based on the direction, time, or area of the written image written on the writing surface. Composed.

  With this configuration, since the direction, time, or area of the written image written on the writing surface is acquired as a characteristic, the written image can be easily associated with the user, and the written image can be easily organized. it can.

  Further, the image display device of the present invention is configured such that the acquisition unit acquires a direction in which the writing image is formed by determining an extending direction of a shadow existing on the writing surface.

  With this configuration, since the position of the writer is obtained by determining the direction in which the shadow extends, it is possible to determine that the writing image extending in the predetermined direction is the writing image of the same user. .

  In the image display device of the present invention, the acquisition unit refers to the image captured by the imaging unit, and includes pixels having the shadow brightness, and the minimum and maximum two-dimensional coordinates in the image. It is comprised from what determines the extending direction of a shadow based on the inclination of the straight line which connects.

  With this configuration, it is possible to reduce the time for determining a shadow extending in a predetermined direction by determining the shadow based on arbitrarily determined luminance information of the pixel, and to improve the acquisition processing capability of the written image. Can do.

  In the image display device of the present invention, the acquisition unit refers to the image captured by the imaging unit, sets the image in a small region composed of a plurality of pixels, and includes a portion where the small region exists. The color information is extracted, and the extracted color information is referenced to determine the direction in which the shadow exists based on the regularity of the arrangement of the small areas including the color information corresponding to the shadow.

  With this configuration, more accurate shadow direction is determined by determining the shadow's longitudinal direction from the color information of the shadow contained in a small area consisting of a plurality of pixels arranged at arbitrary intervals in the image and the position of the small area. Can be detected.

  In the image display device of the present invention, the acquisition unit acquires the time when the writing image is formed based on the existence time of the shadow in the same direction as the extending direction of the shadow existing on the writing surface. Consists of

  According to this configuration, by acquiring the shadow existence time, it is possible to accurately acquire the writing time and the user in association with each other.

  Further, in the image display device of the present invention, the acquisition means includes the direction of the shadow existing on the writing surface, the shadow existing time in the same direction as the extending direction of the shadow existing on the writing surface, and the tip of the shadow Based on the trajectory of the movement, the region where the written image is formed is acquired.

  With this configuration, the region where the writing image is formed based on the extending direction of the shadow existing on the writing surface, the existence time of the shadow in the same direction as the extending direction of the shadow, and the locus of movement of the tip of the shadow is determined. Therefore, the writing image and the shadow can be associated with each other, and the writing time and the user can be associated with each other for more accurate acquisition.

  The image display device of the present invention further includes a writing image control unit, and the writing image control unit writes the writing image on the writing surface extracted by the image extraction / combination unit by the acquisition unit. The image is divided into a plurality of parts based on the characteristics of the image, and the direction of the written image is unified in the same direction.

  With this configuration, it is possible to determine which user has made a plurality of written images written on the writing surface. Therefore, when storing the written information as electronic information, the direction of the written image is set to an arbitrary direction. Can be saved in a unified manner.

  The image display device of the present invention further comprises a writing image control means and a pointing position detection means for detecting a pointing position when the writing surface is pointed, and the writing image control means is provided by the image extraction / synthesis means. The extracted writing image of the writing surface is divided into a plurality based on the characteristics of the writing image acquired by the acquisition unit, and the direction of the writing image on the writing surface detected by the pointing position detection unit is the same direction It consists of things that are unified.

  With this configuration, it is possible to determine which user has made a plurality of written images written on the writing surface, and when the written information is stored as electronic information, the images written by the user are displayed. Since the selection can be made, only the direction of the necessary writing image selected can be stored in a unified manner.

  In the image display device of the present invention, the pointing position detecting means is configured to detect a pointing position based on shadow information of the pointing tool included in the writing surface.

  With this configuration, since the pointing position can be detected by the shadow of the user's finger, a written image can be selected without using a mouse or a keyboard, and the manufacturing cost of the image display device can be reduced. it can.

  As described above, according to the present invention, it is possible to acquire the characteristics of the written image without using a dedicated camera or sensor for imaging the user, and the information written on the writing surface is made by which user. It is possible to provide a low-cost image display device that can identify whether the image is a product.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  1 to 5 are diagrams showing a first embodiment of an image display apparatus according to the present invention. First, the configuration will be described. 1 and 2, an image projection display device (image display device) 1 has a main body 2 provided with a screen 3 made of a transparent resin such as acrylic, and a projection surface (display surface) 3a of the screen 3 is provided on a projection surface (display surface) 3a. An image from the projector 4 is transmitted and displayed on the user side. The screen 3 may be made of a material having high transmittance, such as glass or plastic.

  The writing surface 5 is made of a material that is sufficiently transparent to transmit the image projected on the screen 3 provided on the side opposite to the side facing the user side to the user side, for example, acrylic. An image projected from the projector 4 onto the screen 3 can be viewed through the writing surface 5. That is, the projection surface 3a and the writing surface 5 of the screen 3 are provided in an overlapping manner.

  The writing surface 5 is processed with a cloth or the like that can be easily wiped with ink applied like a general whiteboard. For example, the writing surface 5 is generally processed as a marker for whiteboard. Characters and pictures can be written using what is written.

  Further, a projector (image projection means) 4, a CCD camera (imaging means) 6 and a control unit 7 are provided in the main body 2.

  The projector 4 is configured to project an image on the projection surface 3 a of the screen 3 by being controlled by the control unit 7.

  The CCD camera 6 is installed with the angle of view adjusted to the image projected on the screen 3, and is connected to the control unit 7 by an IEEE1394 interface. The CCD camera 6 captures the writing surface 5 at an arbitrary timing by the user, and transfers the captured image to the control unit 7.

  The CCD camera 6 may be anything as long as it can capture an image as a digital image, and may be an industrial CCD camera or a commonly used digital camera. The interface between the control unit 7 and the CCD camera 6 may be a SUB, a serial cable or the like, but preferably has a higher transfer speed.

  As shown in FIG. 3, the control unit 7 includes a projection image generation unit 8, an image synthesis unit 9, a written image extraction unit 10, a characteristic acquisition unit 11, and a storage unit 12.

  The projection image generation unit 8 generates an image to be projected on the projector 4. The projection image generation unit 8 directly reads a document file or reads information stored in a memory, thereby 4 projects an image onto the screen 3.

  A written image extracting unit (image extracting / synthesizing means) 10 extracts an image written on the writing surface 5 from an image picked up by the CCD camera 6, and an image synthesizing unit (image extracting / synthesizing means) 9. In this case, the extracted image is synthesized with the image generated by the projection image generation unit 8, that is, the image projected from the projector 4. For this reason, the projector 4 can project the composite image on the screen 3.

  The characteristic acquisition unit (acquisition means) 11 acquires the characteristics of the writing image formed on the writing surface 5 based on the shadow of the user reflected on the writing surface 5, and the writing surface 5 is used as the writing image. It is acquired based on the direction, time, or area on the writing surface 5 in which a writing image such as a character or a picture is formed thereon.

  Specifically, with reference to the image captured by the CCD camera 6, the shadow extension is based on the slope of the straight line connecting the minimum and maximum two-dimensional coordinates in the image with pixels including the brightness of the shadow. The direction is determined.

  The writing image extracted by the writing image extraction unit 10 is digitized and combined with the image projected on the screen 3, and is associated with the characteristic acquired by the characteristic acquisition unit 11 and stored in the storage unit 12 as file information. It has come to be.

  The input / output unit 13 includes operation means such as a mouse and a keyboard, and reads file information stored in the storage unit 12 and outputs the file information to an external device.

  Next, a method for acquiring the characteristics of the written image will be described with reference to FIGS.

  When a document file generated by the projection image forming unit 8 is projected from the projector 4 onto the screen 3, the file is projected when a discussion is made by a plurality of people while referring to the file image projected onto the screen 3. Each conference participant writes information on the writing surface 5.

  This writing is performed on an arbitrary writing surface 5 from an arbitrary direction at an arbitrary timing. When all the writing is completed, the written information is digitized and combined into a file projected on the projection surface 3a. The writing is temporally the same, and the direction varies depending on the writer's location.

  That is, the writing is performed at an arbitrary timing, and is not necessarily performed from a direction that coincides with the vertical direction of the file projected by the position of the writer with respect to the image projection display device 1, but is performed at a necessary place. .

  In the present embodiment, since the screen 3 is imaged from the back by the CCD camera 6, the writing surface 5 is imaged, so that the image is taken when the user is looking into the display surface 3 a of the screen 3 or writing on the writing surface 5. As shown in FIG. 4, a shadow appears in the captured image. In FIG. 4, the black portion is the shadow of the user who is writing.

  In the present invention, by using the shadow of the user to be removed because it originally occurs as noise in the image, the characteristics of the time of writing, the direction and area on the writing surface 5 are acquired, The user and the written image are specified.

  As can be seen from the reflection of the shadow in FIG. 4, the user extends his hand from the position where he writes and writes, so the shadow inevitably extends from the position where the writer exists to the writing position. By determining the current direction, the direction in which writing has been performed can be determined.

  Hereinafter, a method for acquiring the writing direction will be specifically described.

  As shown in FIG. 5A, the upper left corner in the image captured by the CCD camera 6 is the origin, the x axis is in the horizontal direction, the y axis is in the vertical direction, and every 10 pixels in the x and y directions. Get the brightness value. Next, out of the luminance values of the pixels set for every 10 pixels, the coordinates of the pixels that match the preset luminance value of the shadow are acquired.

  In FIG. 5B, the pixels inside the arm shadow 21 are pixels that match the luminance value of the shadow, so among the obtained pixels, as shown in FIG. A straight line C connecting the pixels having the y-coordinate with the minimum value A and the maximum value B is obtained.

  Since the inclination of the straight line C substantially coincides with the direction representing the position of the writer with respect to the written image, the writer's direction can be obtained from the inclination of the straight line with respect to the y-axis direction or the x-axis direction. The written image is digitized and stored in the storage unit 12 in association with this characteristic, and the file information is extracted by the input / output unit 13 as necessary.

  It should be noted that it is preferable to arbitrarily set the interval of the pixels for obtaining the brightness of the shadow from the viewpoint of the processing speed and the like because the direction of the user can be more accurately determined.

  Thus, in the present embodiment, the characteristics of the written image are obtained by using the shadow of the user reflected on the writing surface 5 that is normally removed as image noise without using a dedicated camera or the like. Since the information written on the writing surface 5 can be associated with the user, it is possible to identify the user who performed the writing. As a result, it is possible to effectively use the stored written image while preventing the manufacturing cost of the image projection display device 1 from increasing.

  In the present embodiment, the direction in which the writing image is formed is obtained by determining the extending direction of the shadow existing on the writing surface, so the writing image extending in the predetermined direction is used in the same way. It can be determined that the user's written image. Further, referring to the image captured by the CCD camera 6, the shadow extension direction is determined based on the inclination of the straight line connecting the minimum and maximum two-dimensional coordinates in the image with pixels including the brightness of the shadow. Since the determination is made, the time for determining a shadow extending in a predetermined direction can be shortened, and the acquisition processing capability of the written image can be improved.

  1 to 3, 6, and 7 are views showing a second embodiment of the image display device according to the present invention. In the present embodiment, the characteristic acquisition unit 11 refers to the image captured by the CCD camera 6 and sets the captured image as a small area composed of a plurality of pixels, and the density (color) of the portion where the small area exists. Information) is extracted, and the direction in which the shadow exists is determined based on the regularity of the arrangement of the small areas including the density corresponding to the shadow with reference to the extracted density.

  Next, a method of acquiring the characteristics of the written image will be described based on the flowchart of FIG. 6 and FIG. FIG. 6 is a flowchart of processing for determining the direction of the written image.

  First, an image captured by the CCD camera 6 is filtered. As shown in FIG. 7A, this filter is a small window composed of 100 pixels 10 × 10 pixels and is arranged in the captured image at regular intervals, and has a number from 1 to N.

  When the mth small window in the small window of 10 × 10 pixels has a pixel that matches a predetermined shadow density, the small window m is a shadowed window. That is, it is determined whether or not the mth small window in the 10 × 10 pixel small window has a pixel that matches a predetermined shadow density (step S1), and the predetermined shadow density is determined. Collect those with matching pixels. When a shadowed window is acquired in this way, a small window is obtained as shown by the hatched portion in FIG. Note that the window indicated by the diagonal lines in FIG. 7B is a small window of 10 × 10 pixels.

  Next, after recording the number of the shadowed portion of the small window (step S2), it is determined whether or not the recorded number has continuity or aggregation (step S3). Next, it is determined whether or not the number of the shadowed portion of the small window has a number in the vicinity of the four sides of the reading surface 3a (step S4). Since the total number of small windows and their arrangement are known, it is possible to know in which direction the written image extends from the regularity of the numbers of the small windows in which shadows exist.

  In addition, when two or more persons are simultaneously writing on the writing surface 5 and two shadows 22 and 23 exist in the same image as shown in FIG. 7B, the shadow 22 and the shadow 23 are small windows. Because the continuity or aggregation is not maintained in the numbers, it is judged as a shadow of different users.

  As described above, in the present embodiment, the longitudinal direction of the shadow is determined from the density of the shadow included in the small area composed of a plurality of pixels arranged at arbitrary intervals in the image captured by the CCD camera 6 and the position of the small area. Since the determination is made, a more accurate shadow direction can be detected.

  FIGS. 1 to 3 and FIG. 8 are diagrams showing a third embodiment of the image display apparatus according to the present invention.

  In the present embodiment, the characteristic acquisition unit 11 acquires the characteristics of the written image based on the time when the written image is formed on the writing surface 5, and the extension direction of the shadow existing on the writing surface 5 is determined. The time when the writing image is formed is acquired based on the existence time of the shadow in the same direction.

  That is, as shown in FIG. 8, when the writing surface 5 is continuously imaged by the CCD camera 6 when writing from the left side to the right side of the writing surface 5, the shadows of the user are located at positions 24a, 24b, and 24c. As shown in FIG.

  This is because the shadow of the user appears at the time of writing and moves from the left side to the right side while writing. At this time, since the movement is performed while maintaining the direction in which the shadow extends, the time when the writing image is formed is acquired based on the existence time of the shadow in the same direction as the extending direction of the shadow existing on the writing surface 5. And the time when the writing was performed and the user can be accurately obtained in association with each other.

  1 to 3, 9, and 10 are views showing a fourth embodiment of the image display device according to the present invention. In the present embodiment, the characteristic acquisition unit 11 has a shadow direction existing on the writing surface 5, a shadow existence time in the same direction as the shadow extending direction existing on the writing surface 5, and a locus of movement of the shadow tip. The area where the written image is formed is acquired based on the above.

  In FIG. 9A, when the user writes on the writing surface 5 of the screen 3 using the marker, the tip of the marker is in contact with the writing surface 5, so that the shadow of the tip 25 of the marker is lost. Appears as the highest density point.

  Since the tip 25 draws a locus of movement of writing, if the locus of the tip 25 of this marker is recorded from each of the continuously captured images, the area where writing has been performed as shown in FIG. The trail remains.

  The point at which this locus appears is influenced by the frame rate of the CCD camera 6. If the frame rate is high, a faithful locus of writing can be obtained. In the case of the CCD camera 6 having a low frame rate, FIG. It becomes a rough trajectory like b).

  A written area can be obtained based on this locus. However, as shown in FIG. 9C, a plurality of images 25a, 25b, 25c, and 25d are written on the writing surface 5. It becomes difficult to divide the area from only.

  Since these written images always have the shadow of the writer, the image captured by the CCD camera 6 is referred to, and the pixel including the luminance of the user's shadow is the minimum of the two-dimensional coordinates in the image. The writing image was formed by associating the time from when the straight line connecting the one to the largest one appeared in the image until it disappeared with the locus of the writing image near the tip of this line. An area can be acquired.

  Hereinafter, a method of acquiring the characteristics of the written image will be described based on the flowchart of FIG. First, after the imaging of the writing surface 5 is started by the CCD camera 6, the appearance time Tm of the shadow at the tip of the marker and the coordinates (xm, ym) of the position are acquired (step S11). Thereafter, the coordinates of the tip of the marker are sequentially acquired and recorded as (xm2, ym2),... (Xmn, ymn) for each frame of the CCD camera 6.

  Next, the appearance time Ta of the arm shadow, the holding direction, and the coordinates (xa, ya) of the tip of the arm are acquired (step S12). Thereafter, it is determined whether or not there is an arm having the same direction for each frame of the CCD camera 6, that is, whether or not the shadow of the arm has disappeared (step S13), and when the shadow of the arm has disappeared Then, the time Tde is acquired (step S14).

  Next, by acquiring the coordinates (xm, ym) of the marker obtained between time Ta and Tde, a set of marker tip portions is extracted (step S14). Since the coordinates of the set of the tip of the marker and the coordinates of the tip of the shadow of the arm obtained between Ta and Tde both hold time information, the coordinates of the marker at the same time (xm , Ym) and the coordinates (xa, ya) of the arm are calculated, and a set of trajectories of the marker tip portion at coordinates whose distance dt is equal to or smaller than the threshold is extracted (step S16).

  Next, in step S17, by associating the coordinates (xa, ya) at the same time as the times ta and tda with the characteristics of the written image existing in the region of the locus of the tip of the marker of the same group, the user Thus, it is possible to obtain the existing area of the written information, and regard the written images by the markers existing in the obtained area as the same group.

  As described above, in the present embodiment, the writing image is based on the extending direction of the shadow existing on the writing surface, the existence time of the shadow in the same direction as the extending direction of the shadow, and the locus of movement of the tip of the shadow. Since the formed area is acquired, the writing image and the shadow can be associated with each other, and the writing time and the user can be associated with each other to obtain more accurately.

  1, FIG. 2, FIG. 11 to FIG. 13 are diagrams showing a fifth embodiment of the image display apparatus according to the present invention. The same reference numerals are given to the same components as those in the first embodiment. Description is omitted.

  In the present embodiment, as shown in FIG. 11, a pointing position detection unit (pointing position detection unit) 31 and a writing image control unit (writing image control unit) 32 are provided.

  The pointing position detection unit 31 detects a pointing position based on shadow information of a pointing tool such as a finger when the writing surface 5 is pointed with a finger. The writing image control unit 32 includes a writing image extraction unit. The writing image of the writing surface 5 extracted by 10 is divided into a plurality based on the characteristics of the writing image acquired by the characteristic acquisition unit 11, and the writing image on the writing surface 5 detected by the pointing position detection unit 31 is divided. The direction is unified in the same direction.

  Next, a method for unifying the direction of the written image in the same direction will be described with reference to FIGS.

  When finally combining the written image into the projected electronic file, the characteristics of the written image consisting of the time, direction, and area in which writing was performed using the user's shadow, each writing is in the positive direction of the file By synthesizing them in conformity with the information, it can be stored as easy-to-understand information later.

  This is because, as shown in FIG. 12, when the information written on the writing surface 5 is imaged by the CCD camera 6 installed on the back surface of the screen 3, the characters imaged by the CCD camera 6 are inverted by 180 °. Therefore, it is necessary to reverse the written character to the normal position.

  When the user captures an image with the camera, if the CCD camera 6 is installed on the back of the screen, the captured character is inverted by 180 °, so the written character must be inverted. The written image is not limited to characters, but also includes arrows, figures, underlines, and the like, and includes those that lose their meaning as information if the written image other than these characters is reversed. Therefore, in the present embodiment, only the written image selected by the user is inverted when the image characteristics are combined with the projected electronic file. In this inversion, only those that do not require inversion may be selected and the rest may be inverted.

  At the time of this inversion, the user points the writing surface 5 of the screen 3 with a finger to select a writing image having characteristics that require inversion or to select writing that does not require inversion. It should be noted that a writing image having a characteristic that needs to be reversed may be selected using a controller such as a mouse, or writing that does not need to be reversed may be selected.

  Hereinafter, a method of selecting a writing image to be reversed by pointing the writing surface 5 with a finger will be described.

  As shown in FIG. 12, in a configuration in which an image is projected from the back of the screen 3 by the projector 4 and the writing surface 5 on the screen 3 is imaged by the CCD camera 6, the presenter 41 performs a screen operation for presentation. When the writing surface 5 is pointed by the finger 42, the shadow of the presenter's body is reflected on the screen 3, so that the shadow of the presenter's body is displayed on the image 43 captured by the CCD camera 6, as shown in FIG. 44 appears.

  At this time, since the pointing finger shadow 45 is the darkest shadow and is left in the image, the image of the CCD camera 6 that captures the writing surface 5 has a density equal to or higher than a predetermined threshold and a predetermined size. When a black dot appears, it is determined that the shadow is a finger shadow 45, so that it can be recognized as a presenter's finger and a pointing position can be detected.

  On the other hand, it is conceivable that the presenter touches the screen 3 not for the purpose of pointing, but as shown in FIG. 13B, the same level is present in a certain area 47 around the recognized black spot 46. If there is a shadow with a density of 0, it is not regarded as a pointer.

  In this way, a shadow other than a finger or a shadow with a low density on the screen 3 such as a presenter's arm or body can be judged as noise, and only the shadow of the finger can be recognized as pointing to detect an accurate pointing position. it can.

  In particular, when a table-type rear projector as shown in FIG. 1 is used, it is conceivable that the user puts his hand on the screen or places an object. At this time, there is a possibility of causing a malfunction when pointing is detected only by the presence or absence of a shadow, but in this embodiment, an appropriate operation is performed by excluding a shadow other than one finger. be able to.

  Note that by using the pointer function described above for the written image, it is possible to select a written image that does not need to be inverted, and to invert the remaining object, in addition to selecting only that the image needs to be inverted.

  As described above, the image display apparatus according to the present invention can acquire the characteristics of the written image without using a dedicated camera or sensor for imaging the user, and can write on the writing surface with a low-cost apparatus configuration. Display device which has an effect of specifying which user has made the received information and combines the projected image projected on the display surface and the image written on the writing surface Useful as such.

It is an external view of the 1st thru | or 5th embodiment of the image display apparatus which concerns on this invention. It is sectional drawing of the image display apparatus of 1st thru | or 5th Embodiment. It is a block diagram of the image display apparatus of 1st thru | or 4th embodiment. It is a figure which shows the shadow of the arm reflected in the captured image of 1st Embodiment. (A) is a figure which shows the distribution state of the block of the luminance value of the captured image of 1st Embodiment, (b) is a figure which shows the state which compares the luminance value of the shadow of an arm, and the set luminance value (C) is a figure which shows the straight line which connects the pixels of the thing of x value and y coordinate of the minimum value A, and the maximum value B. It is a flowchart of the process which judges the direction of the writing image in 2nd Embodiment of the image display apparatus which concerns on this invention. (A) is a figure which shows the small window when filtering the image imaged with the CCD camera in 2nd Embodiment, (b) is a figure which shows the aggregate | assembly of the small window set on the captured image. is there. It is a figure which shows the image continuously imaged with the CCD camera when writing from the left side of the writing surface to the right side in 3rd Embodiment of the image display apparatus which concerns on this invention. It is a figure which shows 4th Embodiment of the image display apparatus which concerns on this invention, (a) is a figure which shows the shadow of the arm and marker which were reflected on the screen in 4th Embodiment, (b) is a marker The figure which shows the locus | trajectory of the front-end | tip part of this, (c) is a figure which shows the locus | trajectory of several writing written on the writing surface. 14 is a flowchart illustrating processing for acquiring characteristics of a written image according to the fourth embodiment. It is a block diagram of the image display apparatus of 5th Embodiment of the image display apparatus which concerns on this invention. It is a figure which shows the positional relationship of the screen of 5th Embodiment, a presenter, a CCD camera, and a product. (A) is a figure which shows the shadow of the presenter and the finger | toe of a presenter reflected in the captured image of 5th Embodiment, (b) is a figure which shows the shadow reflected in the captured image when a hand is put on a screen. .

Explanation of symbols

1. Image projection display device (image display device)
3a Projection surface (display surface)
4 Projector (image projection means)
6 CCD camera (imaging means)
9 Image composition part (Image extraction / composition means)
10 Written image extraction unit (image extraction / combination means)
11 Characteristic acquisition unit (acquisition means)
31 Pointing position detection unit (pointing position detection means)
32 Writing image control unit (writing image control means)

Claims (10)

Image projection means for projecting an image;
A display surface for displaying an image projected from the image projecting means;
A writing surface that directly forms a writing image so as to overlap the projection image;
Imaging means for capturing a written image formed on the writing surface;
An image extracting / synthesizing unit that extracts a writing image from a captured image by the imaging unit and generates a combined image by combining a part or all of the extracted writing image and a projection image projected by the projecting unit; ,
An image display apparatus comprising: an acquisition unit configured to acquire characteristics of a written image formed on the writing surface based on a shadow reflected on the writing surface. The image according to claim 1, wherein the acquisition unit acquires the characteristics of a written image based on a direction, a time, or a region on the writing surface of the writing image written on the writing surface. Display device. The image display apparatus according to claim 2, wherein the acquisition unit acquires a direction in which the writing image is formed by determining an extending direction of a shadow existing on the writing surface. The acquisition means refers to the image captured by the imaging means, and is a pixel including the brightness of the shadow, and based on the slope of a straight line connecting the smallest and largest two-dimensional coordinates in the image. The image display apparatus according to claim 3, wherein an extending direction is determined. The acquisition unit refers to the image captured by the imaging unit, sets the image in a small area composed of a plurality of pixels, extracts color information of a portion where the small area exists, and extracts the extracted color 4. The image display apparatus according to claim 3, wherein the direction in which the shadow exists is determined based on the regularity of the arrangement of the small areas including the color information corresponding to the shadow with reference to the information. 3. The acquisition unit according to claim 2, wherein the acquisition unit acquires a time when the writing image is formed based on an existence time of a shadow in the same direction as an extension direction of a shadow existing on the writing surface. Image display device. The acquisition unit is configured to write the writing based on a direction of a shadow existing on the writing surface, a shadow existing time in the same direction as a shadow extending direction on the writing surface, and a locus of movement of the tip of the shadow. The image display apparatus according to claim 2, wherein an area where an image is formed is acquired. A writing image control means, wherein the writing image control means divides the writing image of the writing surface extracted by the image extraction / combination means into a plurality based on the characteristics of the writing image acquired by the acquisition means; 8. The image display device according to claim 1, wherein directions of the written images are unified in the same direction. A writing image control means and a pointing position detection means for detecting a pointing position when the writing surface is pointed, and the writing image control means outputs the writing image of the writing surface extracted by the image extraction / synthesis means. The writing image obtained by the obtaining unit is divided into a plurality of characteristics, and the direction of the writing image on the writing surface detected by the pointing position detecting unit is unified in the same direction. The image display device according to claim 1. The image display apparatus according to claim 9, wherein the pointing position detection unit detects a pointing position based on shadow information of the pointing tool included in the writing surface.

Images