JP4500036B2 - Image projection display device, image projection display method, and image projection display program - Google Patents

Description

  The present invention relates to an image projection display device, an image projection display method, and an image projection display program.

  With the recent progress of information technology, presentation systems that display data on a computer on a display are becoming widespread. In such a system, the created material is projected and displayed on a large screen using a projector. Frequently, coordinates are directly input to the display screen, or a computer screen is projected on a screen by a projector and a cursor operation is performed. Touch sensors are widely used as means for inputting directly to a display screen.

  Thus, when the user inputs information on the screen, it is necessary to detect the position. In view of this, a technique has been disclosed in which LEDs are arranged around a touch panel display to specify the position of input information on the display (see, for example, Patent Document 1). In addition, an apparatus that photographs a user by providing a camera above the display is also disclosed (for example, see Patent Document 2).

JP 2000-172430 A JP 11-296304 A

  However, when a plurality of users operate the touch sensor surface at the same time, the touch sensor as described above cannot specify which user is performing the operation or is writing, or further writing. Since the direction of information is not uniform, there is a problem that when the write information is added to the presentation information and saved in a file and read again, the direction of the write information is different and the image becomes very difficult to view.

  Further, conventionally, since a special sensor is used to detect the position on the screen, there is a problem that the user cannot easily perform writing with a finger.

  The present invention has been made in view of the above, and a first object of the present invention is to make it possible to easily detect the presence direction of a user who has written on a display screen without using a special instrument.

  A second object of the present invention is to enable easy-to-view images to be handled by aligning the direction of information written by the user in a predetermined direction.

To solve the above problems and achieve the object, the invention according to claim 1, a projection plane image and the image projection means for projecting an image by the image projection means is projected, the pre Kito Kagemen a photographing means for photographing an image processing means for extracting the write image from the image projection plane photographed by the photographing means, from image before Kito Kagemen detects the direction of the user around the said projection plane, Direction detecting means for associating the direction with the written image, and image rotating means for rotating and storing the written image so that the directions associated with the written image are aligned .

According to the first aspect of the present invention, a sensor for detecting a special direction by detecting a user's shadow from an image of a projection plane photographed by the photographing means and detecting a user's presence position based on the shadow. It is possible to recognize in which direction the writing is performed on the projection plane by the user without using.

  According to a second aspect of the present invention, the direction detection unit divides the image area of the projection plane imaged by the imaging unit into a group of small areas consisting of several pixels, and extracts an image existing in the small area. When the image of the small area has a connected directionality, it is determined that the shadow is generated by a user operation, and the direction of the user is detected from the shadow.

According to the invention according to claim 2, in the invention according to claim 1, the image area of the projection plane photographed by the photographing means is divided into a group of small areas consisting of several pixels and exists in the small area. By extracting the image and judging the direction in which the shadow extends by the nature of the arrangement of the images in this small area, it is possible to write to the projection plane by the user in which direction without using a special direction detection sensor. It becomes possible to recognize whether it was done.

Further, the invention according to claim 3 is characterized in that the image rotation means rotates only a written image designated by a pointing operation by a user in a predetermined direction.

According to the invention of claim 3 , in the invention of claim 1 , it is necessary to write on the projection plane by rotating only the writing image designated by the pointing operation by the user in a predetermined direction. Only write information can be unified in a predetermined direction.

The invention according to claim 4 is characterized in that the direction detecting means detects a pointing position by a user from shadow information of a finger included in the image of the projection plane.

According to the invention according to claim 4 , in the invention according to claim 3 , a special pointing tool or the like is obtained by detecting the pointing position by the user from the shadow information of the finger included in the image of the projection surface. The pointing position can be detected by the shadow of the user's finger without using it and without providing a special direction detection sensor on the apparatus side.

The invention according to claim 5, further on the projection plane, at least, the extraction of the write image, characterized by comprising an input screen projection means for projecting an input button for direction indication.

According to the invention according to claim 5 , in the invention according to any one of claims 1 to 4 , an input screen for projecting at least an input button for extracting a writing image and instructing a direction onto a projection surface. By providing projection means and pointing, it is possible to perform screen operations similar to those of a normal mouse or the like.

According to a sixth aspect of the present invention, the direction detecting means detects the direction of the user from the density value of the user's shadow included in the image of the projection plane and the direction in which the shadow extends. Features.

According to the invention according to claim 6 , in the invention according to claim 1, the direction detecting means is used from the density value of the shadow of the user included in the image of the projection plane and the direction in which the shadow extends. By detecting the user's direction, the user's writing is performed without the need for a special direction detection sensor or the like by using the direction in which the user's shadow appears in the image and the density value of the shadow. It is possible to detect whether it is a shadow when

According to a seventh aspect of the invention, the direction detection unit divides the image area of the projection plane imaged by the imaging unit into a group of small areas consisting of several pixels, and the color of the image existing in the small area Information is extracted, and the extracted color information also obtains a density value, and when the density value is equal to or greater than a predetermined value, the direction in which the shadow of the user extends is determined from the arrangement of the small areas including the density value. It is characterized by that.

According to the invention according to claim 7 , in the invention according to claim 6 , the image area of the projection surface photographed by the photographing means is divided into a group of small areas consisting of several pixels and exists in the small area. Extract the color information of the image, obtain the density value of the extracted color information, and if the density value is greater than or equal to a predetermined value, the direction in which the shadow of the user extends from the arrangement of the small areas including the density value By determining, it becomes possible to recognize in which direction the user has performed writing on the projection plane without using a sensor for detecting a special direction.

Further, the invention according to claim 8 is characterized in that the image rotating means rotates only a written image designated by a pointing operation by a user in a predetermined direction.

According to the invention according to claim 8 , in the invention according to claim 7 , it is necessary that only the writing image designated by the pointing operation by the user is rotated in a predetermined direction to be written on the projection plane. Only write information can be unified in a predetermined direction.

The invention according to claim 9 is characterized in that the direction detecting means detects a pointing position by a user from shadow information of a finger included in the image of the projection plane.

According to the invention according to claim 9 , in the invention according to claim 8 , the direction detecting means is used from the density value of the shadow of the user included in the image of the projection plane and the direction in which the shadow extends. By detecting the user's direction, the user's writing is performed without the need for a special direction detection sensor or the like by using the direction in which the user's shadow appears in the image and the density value of the shadow. It is possible to detect whether it is a shadow when

The invention according to claim 10 further onto the projection plane, at least, the extraction of the write image, characterized by comprising an input screen projection means for projecting an input button for direction indication.

According to the invention of claim 10 , in the invention according to any one of claims 6 to 9 , an input screen for projecting at least an input button for extracting a writing image and instructing a direction onto a projection surface. By providing projection means and pointing, it is possible to perform screen operations similar to those of a normal mouse or the like.
The invention according to claim 11 is further characterized by comprising storage means for storing the written image.
According to the eleventh aspect of the present invention, in the invention according to any one of the first to tenth aspects, the written image can be stored electronically.
The invention according to claim 12 further includes object image generation means for generating an object image from the written image.
According to the invention according to claim 12, in the invention according to any one of claims 1 to 11, an object image is generated from the written image, so that the written image in consideration of the user's existence direction can be obtained. Can be saved.
Further, the invention according to claim 13 is characterized in that the object image generation means associates the latest one of the generated object images with the direction in which the shadow has extended at that time. .
According to the invention of claim 13, in the invention of claim 12, by associating the latest object image in time with the direction in which the shadow extends at that time, the latest image is obtained. It becomes possible to recognize a written image.

The invention according to claim 14 includes an image projecting unit that projects an image, a projection surface on which an image is projected by the image projecting unit, and an imaging unit that photographs the projection surface. In the image projection display method of projecting onto a projection surface, the direction detection unit detects a user's direction around the projection surface from the image of the projection surface captured by the imaging unit, and the image rotation unit includes the image rotation unit. An image rotation step of rotating and storing the written image so that the directions associated with the written image are aligned .

According to the fourteenth aspect of the present invention, a special direction is detected by detecting a user's shadow from the image of the projection plane photographed by the photographing means and detecting the user's presence position based on the shadow. Thus, an image projection display method for recognizing in which direction the user has performed writing on the projection surface can be realized without using a sensor.

The invention according to claim 15 includes an image projecting unit that projects an image, a projection surface on which an image is projected by the image projecting unit, and an imaging unit that photographs the projection surface. In an image projection display program for causing a computer to execute an image projection display method for projecting onto a projection plane, a direction in which a direction detection unit detects a direction of a user around the projection plane from an image of the projection plane captured by the imaging unit A detection step; and an image rotation step in which the image rotation means rotates and stores the written image so that the directions associated with the written image are aligned .

According to the fifteenth aspect of the present invention, a special direction is detected by detecting a user's shadow from the image of the projection plane imaged by the imaging means and detecting the user's location using this shadow. Without using a sensor, an image projection display method for recognizing in which direction the user has performed writing on the projection surface can be executed by a computer.

The image projection display apparatus according to the present invention (Claim 1) detects a user's shadow from the image of the projection plane photographed by the photographing means, and detects the presence position of the user based on this shadow. There is an effect that it is possible to easily recognize in which direction the user has performed writing on the projection plane based on the shadow information of the user without using a sensor or an instrument to detect.

An image projection display device according to the present invention (Claim 2) is the image projection display apparatus according to Claim 1, wherein the image area of the projection plane photographed by the photographing means is divided into groups of small areas composed of several pixels. In order to extract the image existing in the area and determine the direction in which the shadow extends by the nature of the arrangement of the images in this small area, the direction of writing on the projection surface can be determined without using a sensor that detects the special direction. There is an effect that it can be easily recognized by the user's shadow information whether it is performed by the user.

In the image projection display device according to the present invention (claim 3 ), in the invention according to claim 2 , only the written image designated by the pointing operation by the user is rotated in a predetermined direction, so that the image is written on the projection plane . Only the necessary written information, for example, only written characters can be unified in a predetermined direction.

The image projection display device according to the present invention (Claim 4 ) is the invention according to Claim 3 , in order to detect the pointing position by the user from the finger shadow information included in the image of the projection surface. There is an effect that the pointing position can be detected by the shadow of the user's finger without using a tool or the like and without providing a special direction detection sensor on the apparatus side.

An image projection display device according to the present invention (Claim 5 ) is the invention according to any one of Claims 1 to 4 , wherein at least an input button for extracting a writing image and instructing a direction is provided on the projection surface. By providing the input screen projecting means for projecting and pointing, it is possible to perform the same screen operation on the screen as the input instruction means such as a normal mouse.

In the image projection display device according to the present invention (claim 6 ), in the invention according to claim 1, the direction detecting means extends the shadow density value of the user and the shadow included in the image of the projection plane. By detecting the direction of the user from the direction, the direction of the shadow of the user reflected in the image and the density value of the shadow are used, so that there is no need for a special direction detection sensor. There is an effect that it is possible to detect whether it is a shadow when writing is performed.

The image projection display device according to the present invention (Claim 7 ) is the image projection display apparatus according to Claim 6 , wherein the image area of the projection plane photographed by the photographing means is divided into a group of small areas composed of several pixels. Extract the color information of the image existing in the area, obtain the density value of the extracted color information, and if the density value is greater than or equal to a predetermined value, the shadow extension of the user is determined from the arrangement of the small areas including the density value By determining the current direction, it is possible to easily recognize the direction in which writing on the projection plane was performed by the user without using a sensor for detecting a special direction. .

Further, in the image projection display device according to the present invention (claim 8 ), in the invention according to claim 1 , since only the written image designated by the pointing operation by the user is rotated in a predetermined direction, the image is projected on the projection surface . Only the necessary writing information, for example, only the character information can be unified in a predetermined direction.

In the image projection display device according to the present invention (claim 9 ), in the invention according to claim 8 , the direction detecting means extends the shadow density value of the user and the shadow included in the image of the projection plane. In order to detect the direction of the user from the direction, the direction of the shadow of the user reflected in the image and the density value of the shadow are used, so that no special direction detection sensor or the like is required. There is an effect that it is possible to detect whether it is a shadow at the time of writing.

The image projection display device according to the present invention (Claim 10 ) is the invention according to any one of Claims 6 to 9 , wherein at least an input button for extracting a writing image and instructing a direction is provided on the projection surface. By providing an input screen projection means for projecting and pointing, it is possible to perform the same screen operation as an input means such as a normal mouse.
In addition, the image projection display device according to the present invention (invention 11) has the effect that the written image can be stored electronically in the invention according to any one of claims 1 to 10.
An image projection display device according to the present invention (Claim 12), in the invention according to any one of Claims 1 to 11, takes into account the presence direction of the user by generating an object image from the written image. The written image can be saved.
The image projection display device according to the present invention (Claim 13) is the invention according to Claim 12, in which the latest object image in time is not related to the direction in which the shadow extends at that time. As a result, the latest written image can be recognized.

In addition, the image projection display method according to the present invention (claim 14 ) detects a user's shadow from the image of the projection plane photographed by the photographing means, and detects the presence position of the user by this shadow. There is an effect of realizing an image projection display method for recognizing which direction the user has written to the projection plane without using a sensor for detecting the direction.

In addition, the image projection display program according to the present invention (Claim 15 ) detects a user's shadow from the image of the projection plane photographed by the photographing means, and detects the user's presence position based on the shadow, thereby specially displaying the program. An image projection display method for recognizing in which direction the user has performed writing on the projection surface can be executed by a computer without using a sensor for detecting a specific direction.

  Exemplary embodiments of an image projection display device, an image projection display method, and an image projection display program according to the present invention will be explained below in detail with reference to the accompanying drawings.

(First embodiment)
FIG. 1 is a block diagram showing a configuration of an image projection display apparatus according to an embodiment of the present invention. In the figure, reference numeral 100 denotes a display surface, reference numeral 102 denotes a projector, reference numeral 103 denotes a CCD camera, reference numeral 105 denotes a control device, reference numeral 106 denotes a display device, reference numeral 107 denotes an input device, reference numeral 108 denotes a storage device, and reference numeral 110 denotes an image input unit. Reference numeral 111 denotes an image processing unit, reference numeral 112 denotes a pointing direction detection unit, reference numeral 113 denotes an image rotation unit, and reference numeral 114 denotes an I / F display unit. In addition, said control apparatus 105-I / F display part 114 is implement | achieved by the personal computer, for example.

  The display surface 100 is realized by a screen on which electronic information can be enlarged and projected. The projector 102 is a device that projects electronic information (file) such as conference material sent from a personal computer or the like. The CCD camera 103 is arranged and configured so that the entire display surface 100 falls within the angle of view, and takes a projected image of the display surface 100 and a user's shadow or written image.

  The control device 105 is constituted by a CPU, a ROM, a RAM and the like in order to comprehensively control each functional block connected on the bus. The display device 106 is a display means such as a CRT or LD used as a monitor. The input device 107 is input means such as a keyboard and a mouse. The storage device 108 stores, for example, conference materials and various files so that they can be read and written.

  The image input unit 110 performs input processing on an image captured by the CCD camera 103. The image processing unit 111 performs predetermined image processing on the input captured image and extracts a specific image from the captured image. As will be described later, the pointing direction detection unit 112 detects the presence direction (writing direction) of a user who touches or writes on the projection surface around the display surface 100. The image rotation unit 113 rotates write information in a certain direction as necessary according to the user's presence direction (write direction) detected by the pointing direction detection unit 112. The I / F display unit 114 displays, on the display surface 100, a user interface function for performing writing extraction, direction unification instructions, and the like, as will be described later.

  In the image projection display device configured as described above, an image is projected onto the display surface 100 by the projector 102. Further, the CCD camera 103 is arranged so that the entire display is within the angle of view.

  In such an image projection display device, for example, in addition to FIGS. 1 and 2, a configuration in which the display is horizontal as shown in FIG. 3 is conceivable. In this case, as shown in FIG. As described above, an image or a file displayed on the display surface 100 is projected, and the screen is discussed.

  At this time, the users P1, P2, and P3 of the apparatus display the conference material of the personal computer on the display surface 100 by the projector 102, and perform discussions based on the display. Further, if the display 100 can be written with a marker or the like, the discussion will be made by writing directly on the projected material. After the discussion, an image of the display surface 100 is taken by the CCD camera 103, and only the writing by the marker is extracted from the obtained image and pasted to the original electronic file, for example, as shown in FIG. Information can be stored electronically in the storage device 108.

  At this time, as shown in FIG. 4, when a plurality of users (P 1, P 2, P 3) have been discussed, each of them writes with a marker or the like. At this time, the written information is shown in FIG. The direction is different. Even if pasting is performed in this state when storing the writing information, it becomes very difficult to see. Therefore, in this embodiment, it is possible to identify the written direction without using a sensor or the like for detecting a special direction, and to align the direction of the written information to a predetermined direction. is there. For this, a shadow generated when the user writes on the display surface 100 is used.

  That is, while the user is writing, the image captured by the CCD camera 103 includes shadows 205 and 206 of the user's hand as shown in FIG. It always extends from the tip of the hand in the direction of one of the four sides of the image. Since there should be a user (writer) on the extension of the shadow cut by one of the four sides of the image, writing is performed on the display surface 100 from the direction in which the shadow of this hand extends. Identify if it was made from the direction.

  Next, a first operation according to the embodiment of the present invention will be described with reference to a flowchart shown in FIG. This operation is comprehensively executed by the control device 105. A filter is applied to an image taken by the CCD camera 103. As shown in FIG. 7, the filter is divided into small windows 210 consisting of several pixels, and these are managed by numbers 1 to N. First, it is determined whether or not there is a shadow on the small window 210 in the captured image (step S11). Here, if there is a shadow in the small window 210, the number of the small window 210 with shadows 207, 208 is recorded (step S12). At this time, the shaded small window in FIG. 7 includes shadow color information. Subsequently, it is determined whether or not the recorded number has continuity (step S13). Here, if it is determined that the small window has continuity, it is determined whether it has numbers near four sides on the display (step S14), and the directionality is determined (step S15).

  That is, since the total number of small windows prepared for examining the continuity of the number of recorded small windows, or the arrangement thereof, and the arrangement thereof are known, from the regularity of the number of small windows in which shadows exist, You can know in which direction it extends. Since the direction in which the shadow extends is known from the continuity obtained in step S13, the direction can be detected. When two or more persons are writing at the same time and two shadows 207 and 208 are formed in the same image as shown in FIG. 7, the continuity or aggregation is maintained in the numbers of the small windows 210 in the shadows 207 and 208. Since it does not strike, it can be recognized as a separate shadow.

  With the above method, the direction of the user's shadow can be recognized, and the pointing direction detection unit 112 can determine which direction the user is on the display surface 100. Further, it is possible to align the direction of the writing information of the file as shown in FIG. 5 as shown in FIG. 9 by judging from the writing direction of the user and rotating the written one by the image rotating unit 113. It becomes possible.

  Specifically, as shown in the flowchart of FIG. 13, the CCD camera 103 always captures the display surface 100 continuously, takes a difference between frames of the captured image (step S <b> 21), and updates the display surface 100. Monitoring is performed (step S22), and when writing is performed, it is recorded as update information (step S23). Update information that has appeared in a fixed area without a fixed time interval is recorded as the same group (step S24). When writing is done, it will have a meaning, word, and description will be done in a series of continuous flows, so the update of the image begins and increases intermittently without increasing the time interval, and constant Objects existing in the area are grouped into the same group, and are converted into objects as shown by reference numerals 215 and 216 in FIG. 10 at the time of writing extraction (step S25).

  At this time, the objected writing information is associated with the direction in which writing is performed. 11 and 12 are divided into writing information and shadow information for easy understanding of the same image. In FIG. 11, the writing information (216) converted into an object includes a small window 217 in the object area, and this small window 217 is also included in the shadow 219 of the user who has written the writing information of the object 216. Therefore, the object 216 and the shadow 219 have a common small window 217. Similarly, the object 217 and the shadow 220 have a common small window 218. As described above, the object and the direction of the shadow are associated with each other. Subsequently, based on the related information, the object is rotated to align the directions.

  As described above, when writing information is saved at an arbitrary timing of the user, the writing is unified in an arbitrary direction by rotating the object in consideration of the direction of the writing user. be able to.

(Second Embodiment)
FIG. 14 is an explanatory diagram showing a schematic configuration of an image projection display apparatus according to the second embodiment of the present invention. An image is projected from the rear surface of the display surface 100 by the projector 103, and an imaging means such as the CCCCD camera 103 is used. It is configured to image the display surface 100, and other functions are the same as those in FIG. A description of the overlapping configuration is omitted.

  When the user P who is a presenter makes a presentation and performs a screen operation, the display surface 100 is pointed with the finger Pf. At this time, as shown in FIG. 15A, a shadow 230 of the presenter appears on the image of the CCD camera 103. The pointing finger shadow 231 is the darkest shadow left in the image. When a black spot of a predetermined size appears at a density equal to or higher than a predetermined threshold in the image of the CCD camera 103 that is photographing the display surface 100, the presenter's finger is determined to be a shadow of the finger. Recognize and detect the pointing position.

  In addition, it is conceivable that the presenter has touched the display surface 100 for the purpose of pointing, but this may occur in a certain area 232 around the recognized black spot 231 as shown in FIG. If there is a shadow of the same level in density, it is not regarded as a pointer and no processing is performed. As a result, shadows other than the finger and shadows with low density on the display, such as the presenter's arm or body, are determined as noise, and only the shadow of the finger is recognized as pointing, and an accurate pointing position is detected.

  In addition, when this method is applied to a table-type rear projector as shown in FIG. 4, it is conceivable that the user puts a hand on the screen or puts an object. At this time, it is thought that a situation may occur that causes a malfunction when pointing is detected only by the presence or absence of a shadow, but by using this method and excluding shadows other than one finger, it is Operation becomes possible.

  Now, as shown in FIG. 16, when the layout diagram type is displayed, discussed, and written, for example, the writing of the characters 241 has no problem even if the characters 241 are rotated to align the directions. If the direction is changed by arrows or other figures as shown in FIG. At this time, it is possible to select only the write information that is made into an object and that needs to be rotated using the pointer function described above. Alternatively, it is possible to select only those that do not need to be rotated and rotate the remaining objects.

  FIG. 17 shows an example of a function that includes a function input unit having a predetermined function, and the processing unit that detects the pointing position from the shadow of the finger on the display includes a unit that performs a process corresponding to the input function. Is. A button dialog 245 having a predetermined function is displayed on the computer screen 243 projected onto the display.

  For example, buttons for operation as indicated by reference numerals 246 to 251 are set in the button dialog 245. Reference numerals 246 and 247 are scroll buttons for up and down the window, reference numerals 248 and 249 are left and right scrolls, and reference numerals 250 and 251 are buttons that are operated to give instructions for maximizing and minimizing the window. Thus, each window operation is set to be performed. The coordinates displayed on the screen of the button dialog 245 are fixed, and the coordinates on the button window of the dialog are inevitably fixed.

  Therefore, since the area where the button exists is known, the shadow of the finger is detected in this area by the above-mentioned processing means, and when it is determined as pointing, an event assigned to the button is generated, and processing corresponding thereto is performed. I do. The dialog 245 is not limited to a button, and can have more necessary functions as a pop-up structure. In order to prevent the button dialog from taking up more space than necessary on the display surface 100, it is possible to provide buttons along the four sides of the screen. This button function is provided with functions such as objectization, object selection, rotation, etc., and by performing this button operation appropriately, it is possible to operate the necessary operations on the display without using an interface such as a mouse. it can.

(Third embodiment)
As shown in FIG. 19A, when the user is writing on the display, the shadow 261 of the marker tip is reflected at the shadow tip of the arm (finger) 26. At this time, it is considered that the arm may be written while being placed on the display surface 100 or may be floated. However, since writing is in progress, the nib is always in contact with the display surface 100. At this time, it can be said that the shadow density of the nib is the highest.

  Therefore, as shown in FIG. 19-2, the captured image is subjected to inter-frame difference and the update information of the image is monitored (see FIG. 13). If a small shadow 261 appears in the image, and there is a shadow that maintains continuity from the small window 210 near the pen tip to the small window 210 at any one of the four sides of the image, this small shadow is displayed. Recognize that it is the tip of the pen and follow the tip of the pen. When writing is performed, the word, which has a certain meaning, will be described in a series of continuous flows, so that the update of the image starts and the movement path of the pen tip within a certain area Are made into an object at the time of writing extraction as the same group. Among the written objects, the most recent one in time and the direction in which the shadow was extended are stored in association with the information, and the writing information is saved at any timing of the user, Rotate to unify the direction of writing information.

(Fourth embodiment)
Since the size of the tip of the presenter's finger in FIG. 15A and the size of the pen tip that can be written on the display surface with a marker or the like are clearly different, the finger and the pen tip are identified. The CC camera 103 always captures the display surface 100 continuously. The captured image is obtained by taking a difference between frames and monitoring update information of the image (see FIG. 13). That is, information is updated when a user writes or when the user simply points to the display surface 100. At this time, as the updated information, a shadow of a finger appears or a pen tip appears. Depending on whether or not the process.

  If the update information is the pen tip, track the tip of the pen tip, and if the writing information has increased in the image after the pen tip has moved, whether it has increased without a certain time interval, In addition, it is determined whether or not it exists in a certain area from the first writing, and objects existing in the certain area are converted into objects within a certain time interval. When the position of the small window in which the objected writing information exists and the number of the small window in which the shadow of the user who has written is doubled, the writing and shadow information are associated with each other. When the user performs an operation to save the write information at an arbitrary timing, the information written on the display so far is converted into an object according to the above-described method and stored in the storage device 108. The storage of the electronic information in the storage device 108 is performed by compressing the image in accordance with a compression method based on Huffman coding and discrete cosine transform of a color still image defined by, for example, JPEG (Joint Photographic Experts Group), and storing it in the storage device 108. .

  At this time, all writing information is not performed from the direction of the shadow, and is stored in the storage device 108 as it is. After the storage, the user determines whether the written information needs to be rotated or not, and if it is rotated as indicated by the arrow 22 in FIG. 16, the stored information is meaningless. Select except for things. At this time, an object can be selected using a mouse or a keyboard. However, operability can be further improved by performing an operation by directly touching the display surface 100 with a finger.

  During continuous capture by the CCD camera 103, it is determined whether it is a pen tip or a finger shadow, and if it is a finger shadow, this is set as a pointing operation, and the object where the shadow is located is validated. The means for enabling the object is enabled when the small window where the shadow of the finger exists or the small window in the vicinity matches the number of the small window in the object area. If a rotation operation is performed after selecting an object to be rotated, only necessary information can be rotated and left as effective storage information.

  Among these operations, processing such as storing write information and rotating stored information is performed by preparing the above-described button dialog 245 to which a function is assigned in the display area, and storing and rotating functions for each button. By having, you will be able to discuss smoothly without using an interface such as a mouse or keyboard for a series of operations.

  The image projection display device according to the present invention allows a plurality of people to virtually write on the projected material simultaneously without using a sensor such as a touch sensor and allowing the projection surface to write with a marker or the like. There is no problem even if you go or touch your hand.

  By the way, the image projection display method (operation) described so far can be programmed, recorded on a computer-readable recording medium, and executed on the computer. Also, a part of the content contraction method can be provided on the network and realized through a communication line.

  That is, the image projection display method described in this embodiment is realized by executing a program prepared in advance on a computer (CPU 20) such as a personal computer or a workstation as shown in FIG. This program is operated by a computer such as a memory 21, a hard disk 24, a flexible disk 27, a CD-ROM (Compact-Disc Read Only Memory) 26, an MO (Magneto Optical), a DVD (Digital Versatile Disc) by operating the keyboard 25. The program is recorded on a readable recording medium, read from the recording medium by a computer (CPU 20), and displayed on the display device 23 as necessary. In addition, the content contraction data can be transmitted and received from the communication device 22 to an external device as necessary.

  Further, as shown in FIG. 21, the program can be distributed to devices 31 to 33 such as a personal computer via the recording medium via a network such as the Internet 30.

  That is, this program can be provided in a state of being installed in advance on a hard disk as a recording medium built in the computer, for example. The program can be temporarily or permanently stored in a recording medium, and can be provided as packaged software by being incorporated in a computer as a unit or being used as a removable recording medium.

  As the recording medium, for example, a flexible disk, a CD-ROM, an MO disk, a DVD, a magnetic disk, a semiconductor memory, and the like can be used.

  The program can be transferred from a download site to a computer wired or wirelessly via a network such as a LAN (Local Area Network) or the Internet, and downloaded to a storage device such as a built-in hard disk in the computer. .

  As described above, the image projection display device, the image projection display method, and the image projection display program according to the present invention, for example, project electronic information on a computer onto a display, shoot it, perform image processing, and perform special processing. This is useful when the user directly writes information with his / her finger on the display surface without using instruments or sensors. Especially, the electronic information on the computer is projected onto the display, and the written information is displayed here. It is suitable for various conferences and presentations such as shooting, image processing, and saving.

It is a block diagram which shows the structure of the image projection display apparatus concerning embodiment of this invention. It is explanatory drawing for the structure of the vertical display of the image projection display apparatus concerning embodiment of this invention. It is explanatory drawing for the structure of the horizontal display of the image projection display apparatus concerning embodiment of this invention. It is explanatory drawing which shows the example of a user's state in the display surface concerning embodiment of this invention. It is explanatory drawing which shows the projection image and written image example of the display surface concerning embodiment of this invention. It is explanatory drawing which shows the example of the user's shadow on the display surface concerning embodiment of this invention. It is explanatory drawing which shows the example which made the user's shadow and the small window on the display surface concerning embodiment of this invention. It is a flowchart which shows a user's direction determination process in the image projection display apparatus concerning embodiment of this invention. It is explanatory drawing which shows the example of rotation of the projection image in the image projection display apparatus concerning embodiment of this invention, and a writing image. It is explanatory drawing which shows the example of objectification at the time of the writing image extraction in the image projection display apparatus concerning embodiment of this invention. It is explanatory drawing which shows the example of objectification to the writing image concerning embodiment of this invention. It is explanatory drawing which shows the example of the shadow information in FIG. It is a flowchart which shows the process from the image capture in the image projection display apparatus concerning embodiment of this invention to objectification of update information. It is explanatory drawing which shows the system example which operates the display surface of the image projection display apparatus concerning embodiment of this invention from the back. It is explanatory drawing which shows the example of a user's shadow and pointing in the structure of FIG. It is explanatory drawing which shows the example of a ponting detection in FIGS. It is explanatory drawing which shows the example which projects a layout figure etc. on a display surface, and performs writing. It is explanatory drawing which shows the example of a button dialog display on a display surface. It is explanatory drawing which shows the example of a button dialog in FIG. It is explanatory drawing which shows the shadow of a user's finger | toe and marker front-end | tip part on a display surface. It is explanatory drawing which shows the example of a detection by the small window in FIGS. It is a block diagram which shows the structure of the system which performs the program concerning embodiment of this invention with the computer which can communicate. It is a block diagram which shows the structure of the system which distributes the program concerning embodiment of this invention on a network.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Display surface 102 Projector 103 CCD camera 105 Control apparatus 106 Display apparatus 107 Input apparatus 108 Storage apparatus 110 Image input part 111 Image processing part 112 Pointing direction detection part 113 Image rotation part 114 I / F display part 201 Projection image 202a, b writing Information 205, 206 Shadow 210 Small window 245 Button dialog

Claims (15)

A projection surface on which an image is projected by the image projecting means and said image projecting means for projecting an image, a photographing means for pre-shoot Kito Kagemen,
Image processing means for extracting a writing image from an image of a projection plane photographed by the photographing means;
From image before Kito Kagemen detects the direction of the user around the said projection plane, and direction detection means for associating the direction to the write image,
Image rotating means for rotating and storing the written image so that the directions associated with the written image are aligned;
An image projection display device comprising:   The direction detecting means divides the image area of the projection plane imaged by the imaging means into a group of small areas consisting of several pixels, extracts an image existing in the small area, and a direction in which the images of the small areas are connected 2. The image projection display device according to claim 1, wherein the image projection display apparatus determines that the shadow is generated by a user operation and detects the direction of the user from the shadow. The image projection display device according to claim 1 , wherein the image rotation unit rotates only a written image designated by a pointing operation by a user in a predetermined direction. The image projection display device according to claim 3 , wherein the direction detection unit detects a pointing position by a user from finger shadow information included in the image of the projection plane. Furthermore, in the projection plane, at least, the extraction of the write image, image projection according to any one of claims 1-4, characterized in that it comprises an input screen projection means for projecting an input button for direction indication Display device.   2. The image according to claim 1, wherein the direction detection unit detects the direction of the user from a density value of the shadow of the user included in the image of the projection plane and a direction in which the shadow extends. 3. Projection display device. The direction detecting means divides the image area of the projection plane imaged by the imaging means into a group of small areas consisting of several pixels, extracts color information of an image existing in the small area, and extracts the extracted color information determine a density value even if the density value is a predetermined value or more, according to claim 6, characterized in that the small region arrangement of containing the density values to determine a direction extending in the shadow of the user Image projection display device. The image projection display device according to claim 1 , wherein the image rotation unit rotates only a written image designated by a pointing operation by a user in a predetermined direction. The image projection display device according to claim 8 , wherein the direction detection unit detects a pointing position by a user from finger shadow information included in the image of the projection plane. Furthermore, in the projection plane, at least, the extraction of the write image, image projection according to any one of claims 6-9, characterized in that it comprises an input screen projection means for projecting an input button for direction indication Display device. The image projection display device according to claim 1 , further comprising storage means for storing the written image. Furthermore, the image projection display apparatus according to any one of claims 1 to 11, characterized in that it comprises an object image generating means for generating an object image from the writing image. 13. The image projection display device according to claim 12, wherein the object image generation means associates the latest object image among the generated object images with the direction in which the shadow extends at that time. . An image projection display method for projecting an image onto the projection plane, the image projection unit projecting an image, a projection plane onto which an image is projected by the image projection unit, and an imaging unit that photographs the projection plane. ,
A direction detecting step in which a direction detecting unit detects a direction of a user around the projection plane from an image of the projection plane captured by the imaging unit;
An image rotation step in which the image rotation means rotates and stores the written image so that the directions associated with the written image are aligned ;
An image projection display method comprising: An image projection display method for projecting an image on the projection plane, comprising: an image projection unit for projecting an image; a projection plane on which an image is projected by the image projection unit; and an imaging unit for photographing the projection plane. In an image projection display program to be executed by a computer,
A direction detecting step in which a direction detecting unit detects a direction of a user around the projection plane from an image of the projection plane captured by the imaging unit;
An image rotation step in which the image rotation means rotates and stores the written image so that the directions associated with the written image are aligned ;
When,
An image projection display program characterized by comprising:

Images