JP5742305B2 - Drawing image processing device - Google Patents

Description

  The present invention relates to a drawing image processing apparatus, and more particularly, to a drawing image processing apparatus that acquires an image displayed on a drawing object by a photographing apparatus while displaying the image on the drawing object by an imaging device.

  In recent years, an image displayed on a drawing object is recorded for the purpose of recording the image drawn on the drawing object while forming an image on the drawing object such as a whiteboard or blackboard by an imaging device such as a projector. What is acquired by an imaging device is known.

  For example, a projection unit that projects an image onto a predetermined projection area, an imaging unit that captures an area parallel to the projection area, an image correction unit that corrects distortion of the captured image, and a corrected captured image There is one that corrects and captures characters and figures written on a whiteboard or the like by providing writing means for writing to the storage means (see, for example, Patent Document 1).

  However, such a conventional technique has a problem that when the drawing area of the drawing object is insufficient, it is impossible to make additional writing to the contents drawn on the drawing object.

  The present invention has been made to solve such a conventional problem, and even if the drawing area of the drawing object is insufficient, the drawn image can be added to the contents drawn on the drawing object. An object is to provide a processing apparatus.

  The drawing image processing apparatus of the present invention is a drawing image processing apparatus in which an image displayed on the drawing object is acquired by a photographing device while an image is formed on the drawing object by an imaging device. An imaging image supply unit that supplies an imaging image to be imaged to the imaging device and an image captured by the imaging device are drawn in at least a part of the drawing area of the drawing object. A drawn image acquisition unit that acquires the drawn image, a reduced image generation unit that generates a reduced image obtained by reducing the display image displayed in the first specific area, and a reduced image that is required to generate the reduced image A generation request unit, and when the reduced image generation request unit requests generation of the reduced image, the reduced image generation unit includes a second specific region in the first specific region. Result in It has a configuration for generating the reduced image so as to.

  With this configuration, the drawing image processing apparatus of the present invention reduces the image displayed on the drawing object to form an image on the drawing object, so that even if the drawing area of the drawing object is insufficient, the drawing object Can be added to the contents drawn.

  The reduced image generation unit may generate the reduced image by combining the formed image and the drawn image and reducing the combined image.

  With this configuration, the drawn image processing apparatus of the present invention can reduce the image displayed on the drawing object and form an image on the drawing object.

  The drawn image processing apparatus according to the present invention includes an image storage unit that stores the formed image, and an image transfer that is stored in the image storage unit by transferring the composite image as the formed image to the image storage unit. May be provided.

  With this configuration, the drawn image processing apparatus of the present invention can save the contents displayed on the drawing object.

  The image transfer unit may be supplied to the imaging apparatus by transferring the composite image stored in the image storage unit as the imaging image to the imaging image supply unit.

  With this configuration, the drawn image processing apparatus of the present invention can reproduce the contents that have been displayed on the drawing object in the past.

  The imaging image supply unit may interrupt imaging by the imaging device when there is a request to generate the reduced image.

  The imaging image supply unit may supply a single color imaging image to the imaging device when there is a request to generate the reduced image.

  In addition, the drawing image acquisition unit, in the first specific area of the image formed by the imaging device and the image taken by the imaging device, when there is a request to generate the reduced image You may make it acquire the difference image showing a difference as the said drawing image.

  With these configurations, the drawn image processing apparatus of the present invention can acquire a drawn image drawn on a drawing object.

  The drawn image processing apparatus of the present invention may further include a template storage unit that stores a template representing a positional relationship between the first specific area and the second specific area.

  With this configuration, the drawn image processing apparatus of the present invention can reduce and image the content displayed on the drawing target on the drawing target based on the template.

  The first specific area may be composed of a plurality of areas, and the second specific area may be composed of a plurality of areas respectively corresponding to areas of the first specific area.

  With this configuration, the drawn image processing apparatus of the present invention can reduce and image the contents displayed in the plurality of regions of the drawing target on the drawing target based on the template.

  The drawing image processing apparatus of the present invention includes a receiving unit that receives an image transmitted from another drawing image processing device, and a transmission unit that transmits a drawing image acquired by the drawing image acquisition unit. The imaging image supply unit may supply the image received by the receiving unit to the imaging device.

  With this configuration, the drawing image processing apparatus of the present invention can share the content displayed on the drawing object with the drawing object at the base where the other drawing image processing apparatus is provided.

  Further, the drawing image acquisition unit is configured to transmit the drawing image by the transmission unit in the first specific region between an image formed on the imaging device and an image taken by the imaging device. A difference image representing the difference may be acquired as the drawing image.

  With this configuration, the drawn image processing apparatus of the present invention can transmit the contents drawn on the drawing object to another drawn image processing apparatus.

  Further, the drawing image processing method of the present invention is a drawing using a drawing image processing apparatus that acquires an image displayed on the drawing object by a photographing device while forming an image on the drawing object by the imaging device. In the image processing method, an imaging image supply step of supplying an imaging image to be imaged on the drawing object to the imaging device, and at least a drawing area of the drawing object from an image photographed by the imaging device A drawing image acquisition step for acquiring a drawing image drawn in a part of the first specific area; and a reduced image generation step for generating a reduced image obtained by reducing the display image displayed in the first specific area. In the reduced image generation step, when the generation of the reduced image is requested, the reduced image is generated so that the display image is formed in a second specific area of the first specific area. To generate an image.

  By this method, the content displayed on the drawing object is reduced and imaged on the drawing object, so even if the drawing area of the drawing object is insufficient, the content drawn on the drawing object can be added. Can be made.

  In the reduced image generation step, the reduced image may be generated by combining the imaged image and the drawn image and reducing the combined image.

  By this method, the content displayed on the drawing object can be reduced and imaged on the drawing object.

  The imaging apparatus includes a storage medium for storing the imaging image, and the drawn image processing method of the present invention transfers the composite image as the imaging image to the storage medium by transferring the composite image to the storage medium. You may have the image storage step to store.

  By this method, the contents displayed on the drawing object can be saved.

  The drawn image processing method of the present invention may further include a composite image supply step of supplying the composite image stored in the storage medium to the image forming apparatus as the image formed image.

  By this method, it is possible to reproduce the contents displayed on the drawing object in the past.

  In the imaging image supply step, imaging by the imaging device may be interrupted when there is a request for generation of the reduced image.

  In the imaging image supply step, a single color imaging image may be supplied to the imaging device when there is a request to generate the reduced image.

  In the drawing image acquisition step, when there is a request for generation of the reduced image, a difference in the first specific region between an image formed on the imaging device and an image photographed by the photographing device is calculated. The difference image to represent may be acquired as the drawing image.

  By these methods, a drawing image drawn on the drawing object can be acquired.

  In addition, the imaging apparatus includes a storage medium that stores a template representing a positional relationship between the first specific area and the second specific area, and the drawn image processing method of the present invention includes: A reduced image may be generated based on the template.

  According to this method, based on the template, the content displayed on the drawing object can be reduced and imaged on the drawing object.

  The first specific area may be composed of a plurality of areas, and the second specific area may be composed of a plurality of areas respectively corresponding to areas of the first specific area.

  By this method, based on the template, the contents displayed in the plurality of regions of the drawing target can be reduced and imaged on the drawing target, respectively.

  The drawing image processing method of the present invention includes an image receiving step for receiving an image transmitted from another drawing image processing apparatus, and an image transmitting step for transmitting the drawing image acquired by the drawing image acquisition unit. And in the imaging image supply step, the image received in the image reception step may be supplied to the imaging device.

  By this method, the content displayed on the drawing object can be shared with the drawing object at the base where another drawing image processing apparatus is provided.

  In the drawing image acquisition step, the first specific region between the image formed on the imaging device and the image taken by the imaging device when the drawing image is transmitted in the image transmission step. A difference image representing the difference between the two may be acquired as the drawing image.

  By this method, the content drawn on the drawing object can be transmitted to another drawing image processing apparatus.

  The drawing image processing program of the present invention is a drawing image processing program that is executed by a computer that acquires an image displayed on the drawing object by the imaging device while the image is formed on the drawing object by the imaging device. The imaging image supply step of supplying an imaging image to be imaged on the drawing object to the imaging device, and at least a part of the drawing area of the drawing object from the image photographed by the imaging device A drawing image acquisition step of acquiring a drawing image drawn in the first specific region; and a reduced image generation step of generating a reduced image obtained by reducing the display image displayed in the first specific region, In the reduced image generation step, when the generation of the reduced image is requested, the display image is formed on a second specific area in the first specific area. Generating a sea urchin said reduced image.

  Because this program reduces the image displayed on the drawing object and forms an image on the drawing object, even if the drawing area of the drawing object is insufficient, the content drawn on the drawing object is added. Can be made.

  In the reduced image generation step, the reduced image may be generated by combining the imaged image and the drawn image and reducing the combined image.

  With this program, the content displayed on the drawing object can be reduced and imaged on the drawing object.

  The computer includes a storage medium for storing the imaged image, and the drawn image processing program of the present invention stores the composite image as the imaged image in the storage medium by transferring it to the storage medium. An image storing step may be included.

  With this program, the content displayed on the drawing object can be saved.

  The drawn image processing program of the present invention may include a composite image supply step of supplying the composite image stored in the storage medium to the image forming apparatus as the image formed image.

  With this program, the contents displayed in the past on the drawing object can be reproduced.

  In the imaging image supply step, imaging by the imaging device may be interrupted when there is a request for generation of the reduced image.

  In the imaging image supply step, a single color imaging image may be supplied to the imaging device when there is a request to generate the reduced image.

  Further, in the drawing image acquisition step, when there is a request for generation of the reduced image, in the first specific region between the image formed on the imaging device and the image photographed by the photographing device A difference image representing the difference may be acquired as the drawing image.

  With these programs, it is possible to acquire a drawing image drawn on the drawing object.

  Further, the computer includes a storage medium that stores a template representing a positional relationship between the first specific area and the second specific area, and the drawing image processing program of the present invention includes the template in the reduced image generation step. A reduced image may be generated based on the above.

  With this program, the content displayed on the drawing object can be reduced and imaged on the drawing object based on the template.

  The first specific area may be composed of a plurality of areas, and the second specific area may be composed of a plurality of areas respectively corresponding to areas of the first specific area.

  With this program, based on the template, the contents displayed in the plurality of regions of the drawing target can be reduced and imaged on the drawing target, respectively.

  The drawing image processing program of the present invention includes an image receiving step for receiving an image transmitted from another drawing image processing apparatus, and an image transmitting step for transmitting the drawing image acquired by the drawing image acquisition unit. And in the imaging image supply step, the image received in the image reception step may be supplied to the imaging device.

  With this program, the content displayed on the drawing object can be shared with the drawing object at the base where another drawing image processing apparatus is provided.

  In the drawing image acquisition step, the first specific region between the image formed on the imaging device and the image taken by the imaging device when the drawing image is transmitted in the image transmission step. A difference image representing the difference between the two may be acquired as the drawing image.

  With this program, the content drawn on the drawing object can be transmitted to another drawing image processing apparatus.

  The present invention can provide a drawn image processing apparatus capable of additionally writing the content drawn on the drawing object even when the drawing area of the drawing object is insufficient.

1 is a block diagram of a drawn image processing system according to a first embodiment of the present invention. It is a hardware block diagram of the drawing image processing apparatus as the 1st Embodiment of this invention. It is a functional block diagram of the drawing image processing apparatus as the first embodiment of the present invention. It is a conceptual diagram for demonstrating the 1st specific area | region determined by the drawing image processing apparatus as the 1st Embodiment of this invention. It is a conceptual diagram for demonstrating the correspondence of the 1st specific area and the 2nd specific area which the template referred by the drawing image processing apparatus as the 1st Embodiment of this invention represents. It is a flowchart for demonstrating the image reduction operation | movement of the drawing image processing apparatus as the 1st Embodiment of this invention. FIGS. 7A to 7D are conceptual diagrams showing, in time series, examples of projected images when the image reduction operation of the drawn image processing apparatus as the first embodiment of the present invention is executed. It is a conceptual diagram for demonstrating the correspondence of the 1st specific area and the 2nd specific area which the other template referred by the drawing image processing apparatus as the 1st Embodiment of this invention represents. FIG. 9B is a conceptual diagram showing an example of a projected image when the image reduction operation is executed by applying the template shown in FIG. 8 to the drawing image shown in FIG. It is a flowchart for demonstrating the image reduction operation | movement of the drawing image processing apparatus as the 2nd Embodiment of this invention. 11 is a flowchart for explaining in detail an example of a difference image generation process in the drawn image processing apparatus shown in FIG. 10. It is a flowchart for demonstrating an example of the average difference calculation process performed in the difference image generation process shown in FIG. It is a flowchart for demonstrating an example of the difference image calculation process performed in the difference image generation process shown in FIG. It is a block diagram of the drawing image processing system which concerns on the 3rd Embodiment of this invention. It is a functional block diagram of the drawing image processing apparatus as the 3rd Embodiment of this invention.

  Embodiments of the present invention will be described below with reference to the drawings.

(First embodiment)
As shown in FIG. 1, a drawing image processing system 1 according to a first embodiment of the present invention includes a projection device 3 provided for a whiteboard 2 as a drawing object installed in a conference room or the like. , A photographing device 4 and a drawing image processing device 5 are provided.

  Here, in the present embodiment, an example in which a whiteboard is applied as a drawing object will be described. However, in the present invention, a blackboard or the like may be applied as a drawing object.

  The projection device 3 is configured by, for example, a general projector, and projects an image transmitted from the drawing image processing device 5. Here, the projection device 3 is installed so that the projection area is included in the drawing area of the whiteboard 2.

  In the present embodiment, an example in which a projection device that projects a projection image onto a drawing object will be described as an imaging device that forms an imaging image onto the drawing object. The device may be a display device such as a liquid crystal display device. When the imaging device according to the present invention is configured by a display device, a drawing-permitted board provided on the display surface of the display device is applied as the drawing object.

  Further, in FIG. 1, the drawing area of the whiteboard 2 and the projection range of the projection device 3 are shown to coincide with each other. It is good.

  The photographing device 4 is constituted by, for example, a general video camera, and is displayed on the whiteboard 2 photographed at a preset time interval such as once every 0.5 seconds or 15 times per second. Is transmitted to the drawing image processing apparatus 5. Here, the imaging device 4 is installed so that the projection area of the projection device 3 can be imaged.

  As shown in FIG. 2, the drawn image processing apparatus 5 includes a CPU (Central Processing Unit) 10, a RAM (Random Access Memory) 11, a ROM (Read Only Memory) 12, a hard disk device 13, a keyboard, and a pointing device. Are connected via a network to a device communication module 16 for communicating with a peripheral device such as a projection device 3 and a photographing device 4. It is comprised by the general computer apparatus provided with the network communication module 17 for communicating with an external apparatus. Note that the display device 15 may constitute the imaging device of the present invention.

  The ROM 12 and the hard disk device 13 store a program for causing the computer device to function as the drawing image processing device 5. That is, when the CPU 10 executes a program stored in the ROM 12 and the hard disk device 13 using the RAM 11 as a work area, the computer device functions as the drawing image processing device 5.

  As shown in FIG. 3, the drawn image processing device 5 includes an image buffer 20 that stores a projection image to be projected onto the whiteboard 2, an image storage unit 21 that stores an image, and an image buffer 20 and an image storage unit 21. At least a part of the drawing area of the whiteboard 2 from an image transfer unit 22 that transfers images between them, a formed image supply unit 23 that supplies a projection image to the projection device 3, and a captured image captured by the imaging device 4 A drawing image acquisition unit 24 that acquires a drawing image drawn in the first specific region, a reduced image generation unit 25 that generates a reduced image obtained by reducing the display image displayed in the first specific region, A reduced image generation request unit 26 that is requested to generate, and a template storage unit 27 that stores a template representing a second display area in which the reduced image is displayed in the first specific area.

  In the present embodiment, the image buffer 20 is configured by the RAM 11, and the image storage unit 21 and the template storage unit 27 are configured by the RAM 11 or the hard disk device 13. The image transfer unit 22 and the reduced image generation unit 25 are configured by the CPU 10, and the formed image supply unit 23 and the drawn image acquisition unit 24 are configured by the CPU 10 and the device communication module 16.

  The reduced image generation request unit 26 includes an operation unit such as the input device 14. Note that the operation unit may be configured by a switch or the like provided in the projection device 3 or the imaging device 4. When the whiteboard 2 is provided with an input unit such as a switch, the operation unit may be configured by the input unit of the whiteboard 2 and the device communication module 16 connected to each other.

  Further, when the drawing image processing device 5 includes a receiver such as an infrared sensor as the input device 14, the operation unit may be configured by a remote controller and a receiver that send a signal to the receiver. Good.

  The image transfer unit 22 transfers a background image selected from images stored in the image storage unit 21 (hereinafter referred to as “background image”) to the image buffer 20 in response to a request through the operation unit such as the input device 14. It is supposed to be. The image stored in the image buffer 20 may be generated by the CPU 10 executing an image display program such as a presentation program, or may represent an empty image.

  When the projected image supply unit 23 supplies the projection image stored in the image buffer 20 to the projection device 3, a marker as shown in FIG. 4 for causing the drawing image acquisition unit 24 to specify the first specific region 50. Images 51a to 51d are superimposed on the projected image.

  In FIG. 4, the marker images 51 a to 51 d are shown as rectangles respectively arranged at the four corners of the first specific area 50, but the marker image only needs to be able to specify the first specific area 50, The shape, number and position may be different from those shown in FIG.

  The drawn image acquisition unit 24 performs image correction such as trapezoid correction on the captured image based on the positions of the marker images 51a to 51d included in the captured image captured by the capturing device 4, and then the first specific region 50. The drawing image is cut out.

  Note that when the first specific region 50 is specified by the drawing image acquisition unit 24, it is not necessary to superimpose a marker image on the projection image supplied to the projection device 3 by the imaging image supply unit 23. In the embodiment, the imaging image supply unit 23 stops the superimposition of the marker image when the first specific region 50 is specified by the drawing image acquisition unit 24.

  Here, when the positional relationship among the whiteboard 2, the projection device 3, and the imaging device 4 changes, the projection state of the projection device 3 changes, the imaging state of the imaging device 4 changes, or the like, drawing is performed. When the image acquisition unit 24 needs to specify the first specific region 50 again, the imaging image supply unit 23 resumes the superimposition of the marker image in response to an instruction from the operation unit such as the input device 14. It has become.

  As shown in FIG. 5, the template storage unit 27 stores in advance a template that represents the correspondence between the first specific area 50 and the second specific area 52. Here, the template represents the coordinates of the four points of the second specific area 52 normalized in the coordinate space of the first specific area 50. Note that the template may represent the coordinates, width, and height of a specific point in the second specific region 52 normalized in the coordinate space of the first specific region 50.

  In FIG. 3, when a reduced image generation request is made via the reduced image generation request unit 26, the reduced image generation unit 25 displays a reduced image obtained by reducing the display image displayed in the first specific area 50. The template is generated based on the template stored in the template storage unit 27.

  Specifically, the reduced image generation unit 25 generates a composite image obtained by combining the projection image stored in the image buffer 20 and the drawing image acquired by the drawing image acquisition unit 24.

  More specifically, the reduced image generation unit 25 synthesizes the drawn image with the projected image on the image buffer 20 by using, as a transparent pixel, a pixel close to white as the background color of the whiteboard 2 in the drawn image. Is used to generate a composite image.

  Here, the image transfer unit 22 transfers the composite image stored in the image buffer 20 to the image storage unit 21 so that the display image displayed in the first specific area 50 can be reproduced. ing.

  Further, the reduced image generation unit 25 is configured to reduce the composite image on the image buffer 20 so as to be displayed in the second specific area 52.

  The imaging image supply unit 23 causes the drawing image acquisition unit 24 to acquire a drawing image drawn on the whiteboard 2 when there is a request for generation of a reduction image via the reduction image generation request unit 26. Temporarily, a single color image is supplied to the projector 3.

  In the present embodiment, the image formation image supply unit 23 supplies the projector 3 with an image filled with white, which is the background color of the whiteboard 2. Note that the imaging image supply unit 23 may temporarily supply an empty image to the projection device 3 when a reduced image generation request is made via the reduced image generation request unit 26. .

  The image reduction operation of the drawing image processing apparatus 5 configured as described above will be described with reference to FIG. The image reduction operation described below starts when a reduced image generation request is made via the reduced image generation request unit 26. Further, when the image reduction operation described below starts, it is assumed that a projected image is projected and a drawn image is drawn on the first specific area 50 of the whiteboard 2.

  When generation of a reduced image is requested via the reduced image generation request unit 26, first, a single color image is supplied to the projection device 3 by the imaging image supply unit 23 (step S1). As a result, only the drawn image is displayed in the first specific area 50 of the whiteboard 2.

  Next, a drawing image is acquired by the drawing image acquisition unit 24 from the captured image captured by the imaging device 4 (step S2), and the acquired drawing image is combined with the projection image on the image buffer 20 by the reduced image generation unit 25. (Step S3).

  Next, the image stored in the image buffer 20 is transferred to the image storage unit 21 by the image transfer unit 22 so that the display image displayed in the first specific area 50 can be reproduced. (Step S4).

  Next, based on the template stored in the template storage unit 27, the reduced image generation unit 25 causes the composite image synthesized on the image buffer 20 to be displayed in the second specific area 52. The projection image which has been reduced (step S5) and stored in the image buffer 20 is supplied to the projection device 3 by the imaging image supply unit 23 (step S6).

  Thus, as shown in FIG. 7A, the projected image is displayed in the first specific area 50 of the whiteboard 2, and as shown in FIG. 7B, the drawn image is displayed in the first specific area 50. When the image reduction operation is started in a state in which is drawn, a single color image is supplied to the projection device 3 by the imaging image supply unit 23 in step S1, and as shown in FIG. In 50, only a drawing image is displayed.

  The drawing image is acquired by the drawing image acquisition unit 24 in step S2, and the display image displayed on the whiteboard 2 is reduced and connected through steps S2 to S6 as shown in FIG. The image is supplied to the projection device 3 by the image image supply unit 23.

  As described above, the drawn image processing apparatus 5 according to the first embodiment of the present invention reduces the content displayed on the whiteboard 2 to form an image on the whiteboard 2. Even if there is not enough drawing area, it is possible to add to the contents drawn on the whiteboard 2.

  In the present embodiment, as illustrated in FIG. 5, it is assumed that a template representing a correspondence relationship between the first specific area 50 and the second specific area 52 is stored in the template storage unit 27 in advance. However, in the present invention, as shown in FIG. 8, the first specific area includes a plurality of areas 50 a to 50 d, and the second specific area corresponds to the areas 50 a to 50 d included in the first specific area, respectively. The template storage unit 27 includes a plurality of areas 52a to 52d, and the template storage unit 27 represents a correspondence relationship between the plurality of areas 50a to 50d of the first specific area 50 and the plurality of areas 52a to 52d of the second specific area 52. May be stored in advance.

  In this case, for example, as shown in FIG. 9A, assuming that a projection image is projected on the whiteboard 2, the image reduction operation causes the whiteboard 2 to display FIG. 9B. ) Is projected.

  In this way, the template only needs to be associated with which area the image of which area is displayed on the whiteboard 2. Alternatively, a plurality of types of templates may be stored in advance in the template storage unit 27 so that a template used in the image reduction operation can be selected in accordance with an instruction from an operation unit such as the input device 14.

(Second Embodiment)
In the drawn image processing system 1 according to the first embodiment of the present invention, when the imaging image supply unit 23 requests generation of a reduced image via the reduced image generation request unit 26, the whiteboard In order to cause the drawing image acquisition unit 24 to acquire the drawing image drawn in FIG. 2, a single color image or a sky image is temporarily supplied to the projection device 3.

  On the other hand, in the present embodiment, the drawing image acquisition unit 24 stores the image formed on the projection device 3, that is, the image buffer 20 when there is a request to generate a reduced image. The difference image representing the difference between the projected image and the display image in the first specific area obtained from the photographed image photographed by the photographing device 4 is acquired as a drawing image.

  In the present embodiment, in order to facilitate understanding of the invention, the projection image stored in the image buffer 20 is simply referred to as a “projection image” and is obtained from a photographed image photographed by the photographing device 4. An image in the first specific area is simply referred to as a “photographed image”.

  In addition, the configuration of the present embodiment is the same as that of the first embodiment of the present invention except for the configuration of the drawing image acquisition unit 24. Therefore, the configuration of the drawing image acquisition unit 24 is detailed in the present embodiment. Description of other components will be omitted.

  The drawing image acquisition unit 24 acquires a drawing image by generating a difference image by comparing the projection image and the captured image in units of pixels.

  Specifically, the drawing image acquisition unit 24 generates a differential image including pixels whose absolute value of the difference in luminance between corresponding pixels or a distance scale in the color space is larger than a predetermined threshold. ing.

  The drawn image acquisition unit 24 may generate a difference image by comparing the projected image and the captured image in units of rectangles (for example, 8 pixels × 8 pixels).

  In this case, the drawn image acquisition unit 24 generates a difference image made up of rectangles whose absolute value of the luminance difference of the corresponding rectangles or an average value with respect to the distance scale in the color space is larger than a predetermined threshold value. Configured.

  Further, the drawing image acquisition unit 24 may generate a difference image after performing a filtering process on the projected image and the captured image.

  For example, the drawn image acquisition unit 24 performs a sharpening filter process on the captured image to extract a difference between the moving average image obtained by averaging each pixel of the image with surrounding pixels and the original image. Then, after performing the smoothing filter process, the sharpening filter process may be performed, and a difference image may be generated from the image subjected to the filter process.

  Here, the drawing image acquisition unit 24 executes the smoothing filter process by applying a thickening process to the projection image using, for example, a basic calculation of the morphological calculation Erosion. Thereby, the drawing image acquisition part 24 can suppress the influence of the trapezoid distortion, the position shift, etc. that could not be corrected.

  Further, the drawing image acquisition unit 24 may perform a filtering process on the generated difference image. For example, the drawing image acquisition unit 24 may include a filter that removes at least one of yellow-green and yellow color components from the generated difference image. Thereby, the drawing image acquisition unit 24 can remove yellow green and yellow bright lines contained in the light source of the projector constituting the projection device 3 from the difference image.

  The image reduction operation of the drawn image processing apparatus 5 configured as described above will be described with reference to FIG.

  As in the first embodiment of the present invention, the image reduction operation described below starts when a reduced image generation request is made via the reduced image generation request unit 26. Similarly to the first embodiment of the present invention, when an image reduction operation described below starts, a projected image is projected onto the first specific area 50 of the whiteboard 2 and a drawn image is drawn. It shall be.

  When generation of a reduced image is requested via the reduced image generation request unit 26, first, a difference image between the projected image and the captured image is generated by the drawing image acquisition unit 24 (step S21).

  Next, the difference image generated by the drawing image acquisition unit 24 is combined with the projection image on the image buffer 20 by the reduced image generation unit 25 as a drawing image (step S22).

  Next, the image stored in the image buffer 20 is transferred to the image storage unit 21 by the image transfer unit 22 so that the display image displayed in the first specific area 50 can be reproduced. (Step S23).

  Next, based on the template stored in the template storage unit 27, the reduced image generation unit 25 causes the composite image synthesized on the image buffer 20 to be displayed in the second specific area 52. The projection image that has been reduced (step S24) and stored in the image buffer 20 is supplied to the projection device 3 by the image formation image supply unit 23 (step S25).

  Here, the difference image generation processing in step S21 will be described with reference to FIG.

  First, a thickening process is performed on the projection image by the drawing image acquisition unit 24 (step S30). Next, the projected image and the captured image subjected to the thickening process are separated into R (red), G (green), and B (blue) components by the drawn image acquisition unit 24 (step S31).

  Next, steps S32 to S34 described below are executed for each of the R, G, and B components. Here, the R, G, and B components take values from 0 to 255, and the luminance increases as this value increases. That is, when each of the R, G, and B components is 0, the color of the pixel is black, and when each of the R, G, and B components is 255, the color of the pixel is white.

  Here, an integrated image of the corresponding component of the projection image is generated by the drawing image acquisition unit 24 (step S32).

  Next, Step S33 and Step S34 described below are executed for each pixel of the captured image and each pixel of the projection image.

  First, the drawing image acquisition unit 24 executes an average difference calculation process for calculating an average difference representing a difference between the corresponding component and the average of the corresponding components of surrounding pixels with respect to the target pixel of the captured image (step S33). .

  In the average difference calculation process, as shown in FIG. 12, first, a rectangle of m × n size centering on the target pixel (m and n are predetermined constants. The size of the captured image is 1024 pixels × 768 pixels). In this case, for example, m = n = 31 pixels) is calculated by the drawing image acquisition unit 24 (step S40).

  Next, the average value of the luminance in the rectangle calculated in step S40 is calculated by the drawing image acquisition unit 24 using the integrated image calculated in step S32 (see FIG. 11) (step S41).

  Specifically, the upper left luminance of the corresponding rectangle of the integral image is LT, the upper right luminance is RT, the lower left luminance is LB, the lower right luminance is RB, and the number of pixels in the corresponding rectangle is PN. The average value AVG of the luminance in the rectangle is calculated by the following formula.

  AVG = (RB-RT-LB + LT) / PN

  The drawing image acquisition unit 24 reduces the luminance of the target pixel by the average value AVG calculated in this way, the average difference of the target pixel is calculated (step S42), and the average difference calculation process ends.

  In FIG. 11, a difference calculation process for calculating a difference value between target pixels based on the average difference of the target pixels of the captured image and the luminance value of the target pixel of the projection image is executed by the drawing image acquisition unit 24 ( Step S34).

  In the difference calculation process, as shown in FIG. 13, first, the drawing image acquisition unit 24 multiplies the constant value (for example, 1.5 times) so that the luminance value of the target pixel of the projection image becomes deep (step S51).

  Next, the luminance value of the target pixel of the projection image multiplied by a constant is subtracted from the average difference of the target pixel of the captured image by the drawing image acquisition unit 24 (step S52). Next, the drawn image acquisition unit 24 determines whether or not the difference value obtained as a result of the subtraction is larger than the threshold value TH (step S53). In this example, the threshold value TH is -5.

  Here, when it is determined that the difference value is larger than the threshold value TH, it can be determined that the target pixel of the captured image is noise such as extraneous light brighter than the background. The component is set (step S54), and the difference calculation process ends.

  On the other hand, if it is determined that the difference value is not greater than the threshold value TH, the drawing image acquisition unit 24 multiplies the constant value by a constant (for example, 1.5 times) so that the difference value becomes dark (step S55). The same component of the background color is added by the drawing image acquisition unit 24 (step S56). In the present embodiment, each component of the background color is 200.

  Here, whether or not the difference value is less than 0 is determined by the drawing image acquisition unit 24 (step S57). If it is determined that the difference value is less than 0, the difference value is set to 0 (step S58). ), The difference calculation process ends. On the other hand, if it is determined that the difference value is not less than 0, the difference calculation process ends.

  In FIG. 11, when each process described above for each R, G, B component of each pixel of the captured image and the projected image is completed, a difference image made up of each pixel having a difference value of each R, G, B component is drawn image. They are synthesized by the acquisition unit 24 (step S35).

  Finally, a filtering process for removing the yellow component from the difference image is performed on the difference image by the drawn image acquisition unit 24 (step S36). For example, when the luminances of the R, G, and B components of each pixel of the difference image are Ir, Ig, and Ib and min (Ir, Ig)> Ib is satisfied, Ib = min (Ir, Ig). As a result, the yellow component is removed from the pixel.

  In addition, in this example, although the drawing image acquisition part 24 demonstrated the example which performed the filter process which removes a yellow component with respect to a difference image in step S36, a yellowish green component is removed with respect to a difference image. Filter processing may be performed, or filter processing for removing the yellow component and the yellow-green component may be performed on the difference image.

  As described above, the drawn image processing apparatus 5 as the second embodiment of the present invention does not cause the imaging image supply unit 23 to supply a single color image or a sky image to the projection apparatus 3. The same effects as those of the drawn image processing apparatus 5 as the first embodiment of the present invention can be obtained.

  In the present embodiment, the drawing image acquisition unit 24 calculates an average difference obtained by averaging each component with each component of surrounding pixels for the target pixel of the captured image, and based on the calculated average difference. Thus, a difference image is generated by calculating a difference value of each component.

  On the other hand, in the present invention, when the drawing image acquisition unit 24 generates the difference image, the average representing the ratio of each component to the average of each component of the surrounding pixels with respect to each target pixel of the captured image. A difference image may be generated by calculating a ratio and calculating a difference value of each component based on the calculated average ratio.

  In this case, in step S33 of the difference image generation process shown in FIG. 11, an average ratio calculation process is executed instead of the average difference calculation process. Specifically, in step S42 of the average difference calculation process shown in FIG. 12, the drawn image acquisition unit 24 calculates the average difference of the target pixel by reducing the luminance of the target pixel by the average value AVG. In the average ratio calculation process, in the same step, when the average value AVG is larger than 1, the drawing image acquisition unit 24 sets the average ratio as the luminance of the background color, and when the average value AVG is 1 or less, the background By multiplying the luminance of the color by the luminance of the target pixel and dividing by the average value AVG, the average ratio of the target pixel is calculated. Also, the difference calculation process shown in FIG. 13 is executed using an average ratio instead of the average difference.

(Third embodiment)
As shown in FIG. 14, the drawing image processing system 100 according to the third embodiment of the present invention applies to the whiteboards 2 a and 2 b as drawing objects provided at bases such as a plurality of conference rooms. , Provided are projection devices 3a and 3b, photographing devices 4a and 4b, and drawn image processing devices 105a and 105b, respectively.

  Each of the whiteboards 2a and 2b is the same as the whiteboard 2 described in the first embodiment of the present invention, and is hereinafter collectively referred to as “whiteboard 2”. Similarly, since the projection apparatuses 3a and 3b are the same as the projection apparatus 3 described in the first embodiment of the present invention, they are hereinafter collectively referred to as “projection apparatus 3”. In addition, since the imaging devices 4a and 4b are the same as the imaging device 4 described in the first embodiment of the present invention, hereinafter, they are collectively referred to as “imaging device 4”.

  Each of the drawing image processing apparatuses 105a and 105b (hereinafter collectively referred to as “drawing image processing apparatus 105”) is the same as the drawing image processing apparatus 5 described with reference to FIG. 2 in the first embodiment of the present invention. It has a hardware configuration.

  The ROM 12 and the hard disk device 13 store a program for causing the computer device to function as the drawing image processing device 105. That is, when the CPU 10 executes the program stored in the ROM 12 and the hard disk device 13 using the RAM 11 as a work area, the computer device functions as the drawing image processing device 105.

  In the present embodiment, the drawing image processing apparatuses 105 are described as being connected via a network 6 such as the Internet, but may be connected via a dedicated line or the like.

  As shown in FIG. 15, the drawn image processing apparatus 105 includes an image buffer 20 that stores a projection image to be projected onto the whiteboard 2, an image storage unit 21 that stores a background image, an image buffer 20, and an image storage unit 21. An image transfer unit 122 that transfers images between the image transfer unit 122, a formed image supply unit 23 that supplies a projection image to the projection device 3, and at least one of the drawing areas of the whiteboard 2 from the captured image captured by the image capture device 4. A drawing image acquisition unit 124 that acquires a drawing image drawn in the first specific region of the image forming unit, a reduced image generation unit 25 that generates a reduced image obtained by reducing the display image displayed in the first specific region, and a reduced image A reduced image generation request unit 126 that is requested to generate a template, and a template storage unit 2 that stores a template representing a second display region in which the reduced image is displayed in the first specific region. A receiving unit 127 that receives information transmitted from another drawing image processing device, a received signal processing unit 128 that processes information received by the receiving unit 127, and transmits information to the other drawing image processing device. A transmission unit 129.

  Of the constituent elements shown in FIG. 15, the same constituent elements as those of the drawn image processing apparatus 5 according to the second embodiment of the present invention are designated by the same reference numerals, and detailed description thereof will be made. Omitted.

  In the present embodiment, the image transfer unit 122 and the reception signal processing unit 128 are configured by the CPU 10, and the drawn image acquisition unit 124 is configured by the CPU 10 and the device communication module 16.

  The reduced image generation request unit 126 is configured by an operation unit such as the input device 14. The reception unit 127 and the transmission unit 129 are configured by the CPU 10 and the network communication module 17.

  The image transfer unit 122 transfers the background image selected from the background image stored in the image storage unit 21 in response to a request via the operation unit such as the input device 14 to the image buffer 20, and the background image Is output to the transmission unit 129, so that the transmission unit 129 transmits the information to another drawing image processing apparatus.

  The drawing image acquisition unit 124 transmits information representing a difference image, that is, a drawing image, in addition to the function of the drawing image acquisition unit 24 constituting the drawing image processing apparatus 5 according to the second embodiment of the present invention. By outputting each time a captured image is obtained, the transmission unit 129 transmits the information to another drawing image processing apparatus.

  When there is a request for generating a reduced image, the reduced image generation request unit 126 outputs information indicating this request to the transmission unit 129, thereby transmitting the information to the other image processing apparatus to the transmission unit 129. It is supposed to let you.

  The received signal processing unit 128 processes information received by the receiving unit 127 from another drawing image processing apparatus. Specifically, the reception signal processing unit 128 is configured to transfer the background image to the image buffer 20 when the information received by the reception unit 127 represents a background image.

  Further, when the information received by the reception unit 127 represents a drawing image, the reception signal processing unit 128 generates a composite image by combining the background image and the projection image stored in the image buffer 20. It has become.

  More specifically, the reception signal processing unit 128 synthesizes the drawn image with the projected image on the image buffer 20 by using, as a transparent pixel, a pixel close to white as the background color of the whiteboard 2 in the drawn image. Is used to generate a composite image.

  Further, the received signal processing unit 128 causes the reduced image generating unit 25 to generate a reduced image when the information received by the receiving unit 127 indicates a request for generating a reduced image. It should be noted that the template referred to when generating the reduced image is assumed to be predetermined in advance between both the drawing image processing apparatuses 105.

  With this configuration, the display images displayed in the first specific areas of the whiteboards 2a and 2b are the same.

  As described above, the drawn image processing apparatus 105 as the third embodiment of the present invention can obtain the same effects as the drawn image processing apparatus 5 as the first embodiment of the present invention. Further, the display image displayed in each first specific area of the whiteboard 2 can be the same as that of the other drawing image processing apparatus.

DESCRIPTION OF SYMBOLS 1,100 Drawing image processing system 2, 2a, 2b White board 3, 3a, 3b Projection device 4, 4a, 4b Image pick-up device 5, 105, 105a, 105b Drawing image processing device 6 Network 10 CPU
11 RAM
12 ROM
13 Hard Disk Device 14 Input Device 15 Display Device 16 Device Communication Module 17 Network Communication Module 20 Image Buffer 21 Image Storage Unit 22, 122 Image Transfer Unit 23 Imaging Image Supply Unit 24, 124 Rendered Image Acquisition Unit 25 Reduced Image Generation Unit 26, 126 Reduction Image Generation Request Unit 27 Template Storage Unit 127 Reception Unit 128 Received Signal Processing Unit 129 Transmission Unit

JP 2004-239967 A

Claims (31)

In a drawing image processing apparatus for acquiring an image displayed on the drawing object by a photographing device while forming an image on the drawing object by an imaging device,
An imaging image supply unit that supplies an imaging image to be imaged on the drawing object to the imaging device;
A drawing image acquisition unit that acquires a drawing image drawn in at least a part of a first specific area of the drawing area of the drawing object from an image photographed by the photographing device;
A reduced image generating unit that generates a reduced image obtained by reducing the display image displayed in the first specific area;
A reduced image generation request unit that is required to generate the reduced image,
When the reduced image generation request unit is requested to generate the reduced image, the reduced image generation unit is configured to form the display image on a second specific region in the first specific region. Generate a reduced image ,
The first specific area includes a plurality of areas,
The second specific area, drawing an image processing apparatus according to claim Rukoto such a plurality of regions each corresponding to a region where the first specific area has.   The drawn image processing apparatus according to claim 1, wherein the reduced image generation unit generates the reduced image by combining the formed image and the drawn image and reducing the combined image. . An image storage unit for storing images,
Rendering the image processing apparatus according to claim 2, characterized in that and an image transfer unit to be stored in the image storage unit by transferring the composite image before Symbol image storage unit.   The said image transfer part is made to supply to the said image formation apparatus by transferring the synthesized image stored in the said image storage part to the said image formation image supply part as said image formation image. Drawing image processing apparatus.   5. The drawing according to claim 1, wherein the imaging image supply unit interrupts imaging by the imaging device when there is a request for generation of the reduced image. Image processing device.   5. The imaging image supply unit according to claim 1, wherein the imaging image supply unit supplies a single color imaging image to the imaging device when there is a request for generation of the reduced image. The drawn image processing apparatus according to 1.   The drawing image acquisition unit calculates a difference in the first specific region between an image formed on the imaging device and an image photographed by the photographing device when a request for generation of the reduced image is made. The drawn image processing apparatus according to claim 1, wherein a differential image to be expressed is acquired as the drawn image.   The drawing image processing apparatus according to claim 1, further comprising a template storage unit that stores a template representing a positional relationship between the first specific area and the second specific area. A receiving unit for receiving an image transmitted from another drawing image processing apparatus;
A transmission unit that transmits the drawing image acquired by the drawing image acquisition unit,
The drawn image processing apparatus according to claim 1, wherein the imaging image supply unit supplies the image received by the receiving unit to the imaging apparatus. The drawing image acquisition unit calculates a difference in the first specific region between an image formed on the imaging device and an image taken by the imaging device when the drawing unit transmits the drawing image. rendering the image processing apparatus according to 請 Motomeko 9 you and obtains a difference image as the drawing image representing. In a drawing image processing method using a drawing image processing apparatus that acquires an image displayed on the drawing object by a photographing device while forming an image on the drawing object by an imaging device,
An image formation image supply step of supplying an image formation image formed on the drawing object to the image formation apparatus;
A drawing image acquisition step of acquiring a drawing image drawn in at least a part of a first specific area of the drawing area of the drawing object from an image shot by the shooting device;
A reduced image generation step of generating a reduced image obtained by reducing the display image displayed in the first specific area,
In the reduced image generation step, when the generation of the reduced image is requested, the reduced image is generated so that the display image is imaged in a second specific area in the first specific area,
The first specific area includes a plurality of areas,
The drawing image processing method, wherein the second specific area includes a plurality of areas respectively corresponding to areas included in the first specific area. 12. The drawn image processing method according to claim 11, wherein, in the reduced image generation step, the reduced image is generated by combining the formed image and the drawn image and reducing the combined image. . The imaging apparatus includes a storage medium for storing an image,
13. The drawn image processing method according to claim 12, further comprising an image storing step of storing the composite image in the storage medium by transferring the composite image to the storage medium . The drawn image processing method according to claim 13, further comprising a composite image supply step of supplying the composite image stored in the storage medium as the formed image to the image forming apparatus . In the imaging image supply step, when there is a request for generation of the reduced image, drawn according to any one of claims 11 to 14, characterized in that to interrupt the image formation by said image forming apparatus Image processing method. In the imaging image supply step, when there is a request for generation of the reduced image, any one of claims 11 to 14, characterized in that for supplying the imaging image of a single color on the imaging device The drawn image processing method described in 1. In the drawing image acquisition step, when there is a request for generation of the reduced image, a difference in the first specific region between an image formed on the imaging device and an image photographed by the photographing device is calculated. drawn image processing method according to any one of claims 11 to 14 a differential image representing and acquiring as the drawn image. The imaging apparatus includes a storage medium that stores a template representing a positional relationship between the first specific area and the second specific area,
Wherein in the reduced-image generating step, the drawing image processing method according to any one of claims 11 to 17, characterized in that to generate a reduced image based on the template. An image receiving step for receiving an image transmitted from another drawing image processing apparatus;
An image transmission step of transmitting the drawing image acquired by the drawing image acquisition step,
19. The drawn image processing method according to claim 11, wherein in the imaging image supply step, the image received in the image reception step is supplied to the imaging device . In the drawing image acquisition step, when the drawing image is transmitted in the image transmission step, a difference in the first specific region between an image formed on the imaging device and an image photographed by the photographing device drawn image processing method according to 請 Motomeko 1 9 a difference image you and obtains as the drawing image representing a. In a drawing image processing program for causing a computer to acquire an image displayed on the drawing object while the image is formed on the drawing object by the imaging device, to be executed by a computer,
An image formation image supply step of supplying an image formation image formed on the drawing object to the image formation apparatus;
A drawing image acquisition step of acquiring a drawing image drawn in at least a part of a first specific area of the drawing area of the drawing object from an image shot by the shooting device;
A reduced image generation step of generating a reduced image obtained by reducing the display image displayed in the first specific area,
In the reduced image generation step, when the generation of the reduced image is requested, the reduced image is generated so that the display image is imaged in a second specific area in the first specific area,
The first specific area includes a plurality of areas,
The drawing image processing program characterized in that the second specific area includes a plurality of areas respectively corresponding to the areas of the first specific area. The drawn image processing program according to claim 21, wherein, in the reduced image generation step, the reduced image is generated by combining the formed image and the drawn image and reducing the combined image. . The computer includes a storage medium for storing an image,
23. The drawing image processing program according to claim 22, further comprising an image storing step of storing the composite image in the storage medium by transferring the composite image to the storage medium . The drawing image processing program according to claim 23, further comprising a composite image supply step of supplying a composite image stored in the storage medium as the formed image to the image forming apparatus . In the imaging image supply step, when there is a request for generation of the reduced image, drawn according to any one of claims 21 to claim 24, characterized in that to interrupt the image formation by said image forming apparatus Image processing program. 25. The image formation image supply step of supplying a single color image formation image to the image formation apparatus when a request for generation of the reduced image is made. The drawing image processing program described in 1. In the drawing image acquisition step, when there is a request for generation of the reduced image, a difference in the first specific region between an image formed on the imaging device and an image photographed by the photographing device is calculated. The drawing image processing program according to any one of claims 21 to 24 , wherein a difference image to be expressed is acquired as the drawing image. The computer includes a storage medium for storing a template representing a positional relationship between the first specific area and the second specific area,
Wherein in the reduced-image generating step, the drawing image processing program according to any one of claims 21 to claim 27, characterized in that to generate a reduced image based on the template. An image receiving step for receiving an image transmitted from another drawing image processing apparatus;
An image transmission step of transmitting the drawing image acquired by the drawing image acquisition step,
The drawing image processing program according to any one of claims 21 to 28 , wherein in the imaging image supply step, the image received in the image reception step is supplied to the imaging device . In the drawing image acquisition step, when the drawing image is transmitted in the image transmission step, a difference in the first specific region between an image formed on the imaging device and an image photographed by the photographing device drawn image processing program according to 請 Motomeko 2 9, characterized in that to obtain the differential image as the drawing image representing a. A recording medium on which the drawing image processing program according to any one of claims 21 to 30 is recorded so as to be readable by a computer.

Images