JP2015109520A - Information processing apparatus, control method of information processing apparatus, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing apparatus that can create appropriate difference information between a photographed image and a projection image, and enable the sharing of a written image without the occurrence of interruption between apparatuses.SOLUTION: An information processing apparatus 5 creates a projection image to be projected on a projection surface on the basis of image information received from another information processing apparatus 5, acquires a photographed image obtained by photographing the projection surface as a photographing surface, creates a first difference image from the projection image and the photographed image, corrects the projection image, creates a second difference image from the projection image after the correction and a photographed image obtained by photographing the projection image after the correction projected on the projection surface, composes the first and second difference images to create a composite image, and transmits the composite image to the another information processing apparatus.

Description

  The present invention relates to an information processing apparatus that shares an image between apparatuses, a control method for the information processing apparatus, and a computer program.

  When conferences are held between remote locations, images such as letters and pictures drawn on the subject such as whiteboards and blackboards provided at each location are taken with a camera or other photographic device. A sharing system has been proposed. A captured image is shared by transmission / reception between apparatuses, and a projection apparatus such as a projector provided at each base projects the image onto an imaging target. Thereby, the image drawn on the several imaging | photography object can be shared with each imaging | photography object in real time. That is, this system uses a projection device and a photographing device to synthesize images drawn on a plurality of photographing objects, and projects the same content image on each photographing target to share the images. Here, Patent Document 1 discloses a system for sharing an image having the same content without interrupting projection by the projection apparatus by transmitting a difference image between the projection image and the captured image to an apparatus at another base. .

JP 2011-151664 A

  However, such a conventional technique cannot distinguish the portion where the projection image and the writing overlap. For this reason, there is a problem that a gap occurs between a captured image obtained by photographing a written image written on a whiteboard on which the projected image is projected and a written image obtained by extracting a difference from the original projected image. When the interruption occurs, the readability of the written image at each remote site is reduced. It is an object of the present invention to provide an information processing apparatus that creates appropriate difference information between a captured image and a projected image and allows a written image that does not cause interruption to be shared between apparatuses.

  An information processing apparatus according to an embodiment of the present invention shares captured image information among a plurality of information processing apparatuses via a network. The information processing apparatus includes a creation unit that creates a projection image to be projected onto a projection plane based on image information received from another information processing apparatus, and an acquisition unit that acquires a captured image shot using the projection plane as a shooting plane. A first difference image is generated from the projected image and the captured image, the projected image is corrected, and the corrected projected image is projected onto a projection plane and the corrected projection. Generating means for generating a second difference image from an image, combining means for combining the first and second difference images to generate a combined image, and transmission for transmitting the combined image to the other information processing apparatus Means.

  ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to provide the information processing apparatus which produces the suitable difference information of a picked-up image and a projection image, and makes it possible to share the writing image which does not produce a discontinuity between apparatuses. Since there is no interruption in the image in writing at each site, the effect of improving the readability of the written image is obtained.

It is a figure which shows the example of whole structure of the image sharing system of this invention. It is a figure which shows the hardware structural example of information processing apparatus. It is a figure showing an example of functional block composition of an information processor. It is a conceptual diagram of the shared area | region which an information processing apparatus determines. It is a figure which shows the picked-up image, difference image, and writing image which are obtained by the imaging image changed in time series. It is a flowchart explaining the operation | movement of an image projection process. It is a flowchart explaining the operation | movement of a written image transmission process. It is a figure which shows the basic operation example of an image sharing system. It is a figure which shows the operation example of the image sharing system at the time of overwriting. It is a flowchart explaining the operation | movement of a difference image generation process. It is a flowchart explaining operation | movement of an average difference calculation process. It is a flowchart explaining the operation | movement of a difference calculation process. It is a flowchart explaining the operation | movement of a synthesized image generation process. FIG. 10 is a diagram illustrating a functional block configuration example of an information processing apparatus according to a second embodiment. It is a figure which shows that a proximity | contact area | region exists in a focused image and a writing image. It is a figure which shows that a proximity | contact area | region does not exist in a focused image and a writing image. It is a flowchart explaining the operation | movement of a written image transmission process. An example of the operation of the image sharing system when overwritten is shown. It is a flowchart explaining the operation | movement of a near region image generation process. FIG. 10 is a diagram illustrating a captured image, a difference image, and a written image obtained by an imaged image whose position is changed in time series in Example 3. In Example 3, it is a figure which shows the picked-up image, difference image, and writing image which are obtained by the image formation image which changes a brightness | luminance in time series.

Example 1
<System configuration>
FIG. 1 is a diagram illustrating a configuration example of an image sharing system according to an embodiment of the present invention. As shown in FIG. 1, the image sharing system 1 includes a plurality of information processing devices 5. Each information processing device 5 is installed at a plurality of bases such as conference rooms (base A and base B in this case), and can communicate via an arbitrary network 6 such as the Internet or a dedicated line. In addition, the projection device 3 and the photographing device 4 are connected to the information processing device 5 installed at each site via a wired or wireless connection. The projection device 3 may be, for example, a general projector, and projects the image transmitted from the information processing device 5 onto the projection surface. The imaging device 4 is configured by, for example, a general video camera, and captures an image captured using the projection plane as an imaging plane at a preset time interval such as once every 0.5 seconds or 15 times per second. It transmits to the information processing device 5. In other words, in the present embodiment, the projection surface on which the projection device 3 projects an image and the photographing surface on which the photographing device 4 photographs are the same. The imaging surface is a shared area for sharing images between information processing apparatuses.

  In the present embodiment, an example in which the whiteboard 2 is applied as a projection surface and a photographing surface will be described. However, the present invention is not limited to this, and a blackboard, paper, or the like may be applied. The projection device 3 uses a drawing area drawn by a marker or the like of the whiteboard 2 as a projection plane. In the present embodiment, a display device such as a liquid crystal display device may be applied instead of the projection device 3. In the case where the display device is applied, it is only necessary to apply a drawable transparent board provided on the display surface of the display device as a display target.

<Hardware configuration of information processing device>
FIG. 2 is a diagram illustrating a hardware configuration example of the information processing apparatus. The information processing apparatus 5 according to the present invention includes a CPU 100, a RAM 110, a ROM 120, an HDD 130, an input device 140, a display device 150, a device communication module 160, and a network communication module 170. CPU, RAM, ROM, and HDD are abbreviations for Central Processing Unit, Random Access Memory, Read Only Memory, and Hard Disk Drive, respectively. The CPU 100 controls each processing unit in an integrated manner. The ROM 120 and the hard disk device 130 store programs that realize various functions of the information processing apparatus of the present invention. The CPU 100 implements various functions of the information processing apparatus by executing programs stored in the ROM 120 and the hard disk device 130 using the RAM 110 as a work area.

  The input device 140 includes a keyboard, a pointing device, and the like, and accepts user input and the like. The display device 150 including a liquid crystal display is configured by a liquid crystal device or the like, and displays an image projected by the projection device 3, a screen taken by the photographing device 4, and the like. The device communication module 160 is a module for communicating with peripheral devices such as the projection device 3 and the imaging device 4. The network communication module 170 is a module for communicating with an external device such as another information processing apparatus 5 connected via a network. The information processing device 5 is configured by a general computer device including the above-described module.

<Functional blocks of information processing device>
FIG. 3 is a functional block diagram of the information processing apparatus. The information processing apparatus 5 includes an image receiving unit 200, a formed image supply unit 210, a captured image acquisition unit 220, a difference image generation unit 230, an image transmission unit 240, a write image synthesis unit 260, and a write acquisition control unit 270.

  The image receiving unit 200 receives an image transmitted from another information processing apparatus 5. The formed image supply unit 210 converts the received image received by the image receiving unit 200 into a formed image suitable for projecting on the whiteboard 2 and supplies the converted image to the projection device 3. The captured image acquisition unit 220 acquires from the imaging device 4 an image in which a shared area for sharing an image between the whiteboards 2 is captured. The difference image generation unit 230 generates a difference image representing a difference in a shared area between the formed image projected by the projection device 3 and the captured image captured by the imaging device 4.

  The written image combining unit 260 combines a plurality of difference images to generate a written image. The written image is image information such as characters and pictures written on the whiteboard 2 at the local site by the marker, and indicates an image to be transmitted to the information processing apparatus 5 at another base. The written image is not limited to the above-described image, and may be a paper medium attached to the white board 2 using a magnet or the like. The writing acquisition control unit 270 performs writing-related control such as changing the formed image projected onto the whiteboard 2 in time series, acquiring a captured image, generating a difference image, and synthesizing the writing image. The image transmission unit 240 transmits the generated writing image to the other information processing apparatus 5.

  Here, the image receiving unit 200 and the image transmitting unit 240 are configured by the CPU 100 and the network communication module 170. The formed image supply unit 210 and the captured image acquisition unit 220 are configured by the CPU 100 and the device communication module 160. The difference image generation unit 230, the write image composition unit 260, and the write acquisition control unit 270 are configured by the CPU 100.

<Image formation image supply unit>
As shown in FIG. 4, the formed image supply unit 210 superimposes the marker images 31, 32, 33, and 34 for specifying the shared region 30 on the image supplied to the projection device 3. In FIG. 4, the marker images 31, 32, 33, and 34 are shown as rectangles arranged at the four corners of the shared area 30, but the marker image only needs to be able to identify the shared area 30. The shape, number and position may be different from those shown in FIG.

<Captured image acquisition unit>
The captured image acquisition unit 220 constitutes a shared area image acquisition unit in the present invention. Based on the positions of the marker images 31, 32, 33, and 34 included in the image acquired from the image capturing device 4, the image acquired from the image capturing device 4 is subjected to image correction such as keystone correction, and then the shared area 30 is cut out. It is like that.

  The formed image supply unit 210 enlarges or reduces the received image so that the region of the received image received by the image receiving unit 200 matches the shared region 30 of the image supplied to the projection device 3. 3 is supplied.

<Difference image generation unit>
The difference image generation unit 230 generates a difference image representing a difference in the shared region 30 between the formed image supplied from the formed image supply unit 210 to the projection device 3 and the captured image captured by the imaging device 4. The difference image generation unit 230 compares the projection image held in the formed image supply unit 210 and the captured image captured by the imaging device 4 in units of pixels, and generates a difference image.

  Specifically, the difference image generation unit 230 generates a difference image composed of pixels in which the absolute value of the difference in luminance between corresponding pixels or the distance scale in the color space is larger than a predetermined threshold. The difference image generation unit 230 divides the projection image supplied from the imaging image supply unit 210 to the projection device 3 and the captured image captured by the imaging device 4 in units of rectangles (for example, 8 pixels × 8 pixels). A difference image may be generated by comparison. In this case, the difference image generation unit 230 is configured to generate a difference image made up of rectangles whose absolute value of the difference in luminance of the corresponding rectangles or an average value with respect to the distance scale in the color space is larger than a predetermined threshold value. Is done.

  Further, the difference image generation unit 230 generates a difference image after filtering each of the projection image supplied from the imaging image supply unit 210 to the projection device 3 and the captured image captured by the imaging device 4. You may make it do. For example, the difference image generation unit 230 extracts a difference between the moving average image obtained by averaging each pixel of the image with the surrounding pixels and the original image with respect to the captured image captured by the imaging device 4. Apply. Then, the projection image supplied from the imaging image supply unit 210 to the projection device 3 is subjected to smoothing filter processing and then subjected to sharpening filter processing, and a difference image is generated from each of the images subjected to the filter processing. You may do it.

  Here, the difference image generation unit 230 performs the smoothing filter process by applying a thickening process to the image supplied to the projection device 3 using, for example, a basic operation of EROSION of a morphological operation. As a result, the difference image generation unit 230 can suppress the influence of trapezoidal distortion, positional deviation, and the like that could not be corrected by the captured image acquisition unit 220.

<Writing image composition unit>
The written image synthesis unit 260 synthesizes a plurality of difference images generated by the difference image generation unit 230 to generate a written image. Specifically, pixels corresponding to writing included in the difference image are extracted and combined.

<Write acquisition control unit>
The writing acquisition control unit 270 performs control to change the formed image projected on the whiteboard 2 in time series, acquire a captured image, generate a difference image, and synthesize the writing image. An example of the formed image to be changed in time series is shown in FIGS. 5 (A) and 5 (E). FIG. 5A shows an example of the first imaged image, which is the image received by the image receiving unit 200 as it is. On the other hand, FIG. 5E shows an example of the second imaged image, which is an image obtained by extracting the contour of the image shown in FIG. The image in FIG. 5E is an image obtained by performing image processing so as not to overlap with the image in FIG. Specifically, the outline may be extracted after the black portion of the image in FIG. In this embodiment, the combined image shown in FIG. 5A is acquired in time series prior to FIG. 5B, but the image acquisition order is not limited to this.

  An example of the captured image acquired by the captured image acquisition unit 220 by changing the formed image in time series is shown in FIGS. 5 (C) and 5 (F). It is assumed that the whiteboard 2 on which the formed image is projected has been written as shown in FIG. When the combined images shown in FIGS. 5A and 5E are superimposed and projected in time series on the writing on the whiteboard 2 (FIG. 5B) and taken by the photographing device 4, FIG. Two types of captured images shown in FIG. 5F are acquired. Then, the difference image generation unit 230 generates a difference image between the captured image and the formed image. An example of the difference image is shown in FIGS. 5D and 5G. In each of the portions where the formed image and the writing overlap, a break occurs.

  The written image synthesis unit 260 synthesizes a plurality of difference images acquired by changing the formed image in time series to generate a written image shown in FIG. In the present invention, when the combined image shown in FIG. 5 (E) is projected onto the whiteboard, image processing is performed so as not to overlap the combined image shown in FIG. 5 (A). A written image (H) can be obtained.

<Image transmission unit>
The image transmission unit 240 transmits the written image generated by the written image composition unit 260 to another information processing apparatus 5. Here, when the writing image generated by the writing image composition unit 260 is empty (that is, when there is no difference between both images), or when it is the same as the writing image transmitted last time, The image transmission unit 240 may not transmit the written image. In this case, when transmitting the written image to another information processing apparatus 5, the image transmitting unit 240 stores the written image in a storage medium such as the RAM 110, and the written image and the written image to be transmitted are stored. Configured to compare.

Hereinafter, operations performed by the information processing apparatus 5 will be described with reference to FIGS. 6, 7, 7, 8, and 9.
<Writing image reception operation flowchart>
FIG. 6 is a flowchart for explaining an operation in which the image formation image supply unit 210 of the information processing device 5 projects an image input from the write acquisition control unit 270 onto the projection device 3. This process is realized by the CPU 100 configuring the image forming image supply unit 210 executing programs stored in the ROM 120 and the HDD 130 using the RAM 110 as a work area. Assume that the image is stored in the RAM 110 of the work area.

  The formed image supply unit 210 creates a projection image to be projected on the projection plane based on image information received from another information processing apparatus. Specifically, when an image is received from another site, the imaging image supply unit 210 generates an imaging image in which the received image is enlarged or reduced so as to coincide with the shared area of the image supplied to the projection device 3. (S1000). Next, the imaged image supply unit 210 superimposes the marker image on the imaged image (S1010), and supplies it to the projection device 3 (S1020). The projection device 3 projects the supplied image on the whiteboard 2 at its own location.

<Writing image transmission operation flowchart>
FIG. 7 is a flowchart for explaining a write image transmission operation to an information processing apparatus at another site. Note that the write image transmission operation described below is repeatedly executed at regular intervals. Alternatively, it may be started when a photographing request is generated via the input device 140 of the information processing apparatus 5. This process is realized by the CPU 100 configuring each processing unit shown in FIG. 3 executing programs stored in the ROM 120 and the HDD 130 using the RAM 110 as a work area. Assume that the image is stored in the RAM 110 of the work area. Each processing unit that performs this processing is controlled by the write acquisition control unit 270.

  First, the image formation image supply unit 210 sends a projection image, that is, a received image from another site received by the image reception unit 200 from another information processing device, to the projection device 3 for projection (S2010). The specific processing is as shown in the image projection flowchart described in FIG. The formed image supply unit 210 superimposes the marker image on the writing image at the other site, and projects the image supplied by the projection device 3 onto the whiteboard 2 at the own site. Next, the captured image acquisition unit 220 acquires a whiteboard image captured by the imaging device 4 as a first captured image (S2015).

  The difference image generation unit 230 performs image correction such as keystone correction on the image acquired by the captured image acquisition unit 220 based on the position of the marker image included in the image, and then cuts out the shared area (S2020). The difference image generation unit 230 generates a first difference image representing a difference in the shared area between the image supplied by the imaging image supply unit 210 to the projection device 3 and the image shot by the shooting device 4 (S2030).

  Next, the written image composition unit 260 clears the self-site written image to be held empty. (S2038). The written image composition unit 260 holds the first difference image passed from the difference image generation unit 230 (S2040). Since the written image is cleared to empty in S2038, the written image becomes the first difference image.

  Next, the imaging image supply unit 210 corrects the other-site written image received from another information processing apparatus, and sends the corrected image to the projection apparatus 3 for projection (S2050). Here, the difference image generation unit generates an image subjected to contour extraction.

  Next, the captured image acquisition unit 220 acquires a whiteboard image captured by the capturing apparatus 4 as a second captured image (S2055). The difference image generation unit 230 performs image correction such as trapezoid correction on the second captured image acquired by the captured image acquisition unit 220 based on the position of the marker image included in the image, and cuts out the shared area (S2060). .

  Next, the difference image generation unit 230 generates a second difference image that represents the difference in the shared region between the image supplied by the imaging image supply unit 210 to the projection device 3 and the image captured by the imaging device 4 ( S2070). The written image composition unit 260 synthesizes the second difference image passed from the difference image generation unit 230 with the first difference image, and obtains a written image (S2080).

  Here, the image transmitting unit 240 determines whether or not the written image of the local site generated by the written image combining unit 260 is empty (S2100). If the image transmission unit 240 determines that the writing image at the local site is empty, the image transmission unit 240 sends the other site writing image received from the other information processing apparatus to the imaging image supply unit, and performs the writing image transmission operation. The process ends (S2130).

  On the other hand, when it is determined that the written image at the local site is not empty (S2100), the image transmitting unit 240 has the same local site written image generated by the written image combining unit 260 as the previous local site written image. It is determined whether or not (S2110).

  Here, if the image transmission unit 240 determines that the self-site write image generated by the write image composition unit 260 is the same as the self-site write image that was transmitted last time, the other received from another information processing apparatus The base writing image is sent to the imaging image supply unit. Then, the image transmission unit 240 ends the writing image transmission operation (S2130).

  On the other hand, when it is determined that the written image of the local site generated by the written image composition unit 260 is not the same as the previously written image of the local site, the local site image is transferred to another information processing by the image transmitting unit 240. It is transmitted to the device 5 (S2120). Then, the written image of the other site received from the other information processing apparatus, that is, the received image is sent and projected to the image formation image supply unit, and the written image transmission operation is terminated (S2130).

<Operation example of image sharing system>
8 and 9 are conceptual diagrams for explaining an operation example of the image sharing system 1. Information processing devices 5 are installed at the sites A and B, respectively. In the following, examples of written images exchanged between the respective bases, projected images respectively projected by the projection device 3, and photographed images respectively photographed by the photographing device 4 are shown in time series.

  FIG. 8 shows an operation of sending a writing image from the base A to the base B. The case where the projection is not overwritten will be explained. First, it is assumed that each of the information processing apparatuses 5 at the base A and the base B holds a blank other-site writing image as an initial state and projects a formed image.

  FIG. 8A shows an initial state of the information processing apparatus 5 at the site A. The projection device 3 at the site A projects a blank image 500 on the whiteboard 2. Since the writing 501 of the whiteboard 2 is blank, the drawing board surface 502 on which the formed image is superimposed on the whiteboard 2 is also blank. FIG. 8B shows an initial state of the information processing apparatus 5 at the site B. The projection device 3 projects a blank image 510 on the whiteboard 2 by the projection device 3 at the site A. Since the writing 511 of the whiteboard 2 is blank, the drawing board surface 512 on which the formed image is superimposed on the whiteboard 2 is also blank.

  Next, when a session between the information processing device 5 at the site A and the information processing device 5 at the site B is established, it becomes possible to transmit and receive the written images of each other. Here, a case where a writing image is transmitted from the base A to the base B will be described. FIG. 8C shows a case where the written image is transmitted in a state where the written image at the site B is blank and the writing on the whiteboard 2 at the site A is blank.

  First, the projection device 3 at the site A superimposes and writes the image written at the site B as a formed image 520 onto the writing 521 of the whiteboard 2, and the imaging device 4 captures the image on the drawing board surface 522 on which the projection is performed. get. Then, the difference image generation unit 230 obtains a first difference image 524 by taking the difference between the formed image 520 and the captured image 523. Next, the projection device 3 at the site A extracts the outline of the writing image at the site B and superimposes it on the writing 531 of the whiteboard 2 as a formed image 530, and the imaging device 4 photographs the drawing board surface 532 on which the projection is performed. The captured image 533 is acquired. Then, the difference image generation unit 230 takes the difference between the formed image 530 and the captured image 533 to obtain a second difference image 534. Then, the writing image combining unit 260 combines the first difference image 524 and the second difference image 534 to generate a combined image 535 of the base A.

  Since there is neither a written image at another site nor a written image on the whiteboard 2, the generated first difference image 524 and the second difference image 534 are both blank, and thus the composite image 535 is also blank. Since the generated composite image 535 is blank, the information processing apparatus 5 at the site A does not transmit the written image to the site B.

  Next, a case where the writing image at the site B is blank and writing is performed on the whiteboard 2 at the site A will be described. Here, an example is shown in which the user has written “10” on the whiteboard with a marker pen or the like. FIG. 8D shows a state in which the writing image at the site B is blank and writing is performed on the whiteboard 2 at the site A. The projection device 3 superimposes and projects a blank image 540 on the whiteboard 2 at the site A where “10” is written. Since the formed image 540 is blank, the writing 541 becomes the drawing board surface 542 as it is. In other words, the state of the board remains “10”.

  FIG. 8E illustrates a case where the written image is transmitted from the base A to the base B in a state where the written image of the base B is blank and “10” is written to the whiteboard 2 of the base A. First, the projection device 3 at the site A superimposes and writes the image written at the other site on the writing 551 of the whiteboard 2 as a formed image 550, and the imaging device 4 shoots the drawing board surface 552 onto which the superimposed projection is performed. get. Then, the difference image generation unit 230 obtains a first difference image 554 by taking the difference between the formed image 550 and the captured image 553.

  Next, the projection device 3 at the site A superimposes the image obtained by extracting the contour of the written image at the other site as the image 560 on the writing 561 of the whiteboard 2, and the imaging device 4 captures the drawing board surface 562 on which the projection is performed. The captured image 563 is acquired. Then, the difference image generation unit 230 obtains a second difference image 564 by taking the difference between the formed image 560 and the captured image 563. Then, the writing image combining unit 260 combines the first difference image 554 and the second difference image 564 to generate a combined image 565 of the base A.

  The written image of the site B is blank, and “10” is written on the whiteboard 2 of the site A. Therefore, the generated first difference image 554 and second difference image 564 are both “10”, and thus the composite image. 565 is also “10”. Since the generated composite image 565 is not blank, the information processing apparatus 5 transmits the composite image 565 to the information processing apparatus at the base B. At the site B, the written image received by the projection device 3 is projected as a formed image 566.

  FIG. 9 shows the operation of each information processing apparatus when a written image is transmitted from the site B to the site A. A case where the user at the site B overwrites the written image will be described. By the processing performed by the information processing apparatus of the present invention, the extracted written image is not interrupted even if writing is superimposed on the projection. This indicates that the writing at each site is readable.

  First, the case where the written image received from the site A is “10” and the whiteboard 2 at the site B is blank will be described. FIG. 9A shows a case where the writing image received from the site A at the site B is projected onto the whiteboard 2 in a superimposed manner. The projection device 3 at the site B projects the formed image 600 generated based on the written image received from the site A on the whiteboard 2. Since the writing 601 of the whiteboard 2 is blank, the whiteboard 2 becomes a drawing board surface 602 on which the formed image is directly superimposed and projected. Specifically, “10” is projected as it is.

  FIG. 9B shows a case where a written image is transmitted to the whiteboard 2 at the site B in a state where the writing is blank. First, the projection device 3 at the site B superimposes the image written at the site A on the writing 611 of the whiteboard 2 as a formed image 610, and the imaging device 4 captures the imaging surface 612 on which the superimposed projection is performed, thereby obtaining a captured image 613. get. Then, the difference image generation unit 230 takes the difference between the formed image 610 and the captured image 613 to obtain a first difference image 614.

  Next, an image obtained by extracting the contour of the writing image at the site A by the projection device 3 at the site B is superimposed and projected on the writing 621 of the whiteboard 2 as an image 620, and the drawing board surface 622 on which the photographing device 4 is projected is superimposed. A photographed image 623 is obtained by photographing. Then, the difference image generation unit 230 obtains a second difference image 624 by taking the difference between the formed image 620 and the captured image 623. Then, the writing image combining unit 260 combines the first difference image 614 and the second difference image 624 to generate a combined image 625 of the base B.

  Since there is no writing to the whiteboard 2 at the site B, the generated first difference image 614 and the second difference image 624 are both blank, and thus the composite image 625 is also blank. Since the generated composite image 625 is blank, the information processing apparatus 5 does not transmit the written image to the base A.

  Next, a case where the writing image at the site A is not blank and writing is performed on the whiteboard 2 at the site B will be described. Here, an example is shown in which a writing image “10” of the base A is projected onto the whiteboard 2 and a strikethrough is written with a marker pen or the like.

  FIG. 9C shows a case where the writing image received from the site A is superimposed and projected on the whiteboard 2 at the site B, and the user writes on the whiteboard 2, that is, the writing is performed by overlapping the projected image. The projection device 3 at the site B projects the image 630 at the site A on the whiteboard 2. When the user writes a strike-through line on the whiteboard 2, the image forming image 630 is superimposed and projected on the write 631 on the whiteboard 2, and the drawing board surface 632 is formed with the strike-out line superimposed on “10”.

  FIG. 9D shows that “the writing image“ 10 ”received from the site A is superimposed and projected onto the whiteboard 2 at the site B, and the writing image is displayed from the site B in a state where a strikethrough is written on the projected image. The case transmitted to the base A is shown. First, the projection device 3 at the site B superimposes and writes the image written at the other site on the writing 641 of the whiteboard 2 as a formed image 640, and the imaging device 4 shoots the drawing board surface 642 on which the image is superimposed and the captured image 643 is captured. get. Then, the difference image generation unit 230 obtains a first difference image 644 by taking the difference between the formed image 640 and the captured image 643.

  Next, the projection device 3 at the site B superimposes and projects the image obtained by extracting the contour of the writing image at the other site onto the writing 651 of the whiteboard 2 as the imaging image 650, and the drawing board surface 652 onto which the imaging device 4 is superimposed and projected. The photographed image 653 is acquired by photographing. Then, the difference image generation unit 230 obtains a second difference image 654 by taking the difference between the formed image 650 and the captured image 653. Then, the writing image combining unit 260 combines the first difference image 644 and the second difference image 654 to generate a combined image 655 of the base B.

  Since the written image of the site A is “10” and a strikethrough is written on the whiteboard 2, the generated first difference image 644 and second difference image 654 are strikeouts where interruptions occur. The written image composition unit 260 synthesizes these two difference images, whereby a composite image 655 in which a strikeout line without interruption is reproduced can be acquired. In the conventional method, only the difference image 644 can be extracted, and another information processing apparatus that has received the difference image sometimes interprets the extracted image so as to be a dotted line instead of a strikethrough. According to the information processing apparatus of the present invention, since the composite image 655 is obtained, another information processing apparatus that has received the difference image can reliably project an imaged image showing a strikethrough. Therefore, it can be interpreted that a user at another base is a strikethrough. That is, an image written at another site can be accurately projected at the own site, and visibility and operability are improved. Since the generated composite image 655 is not blank, the information processing apparatus 5 transmits the composite image 655 to the base B. At the site B, the projection device 3 projects the received written image as a formed image 656.

  FIG. 9E shows a case where the written image received from the site B is superimposed and projected onto the whiteboard 2 at the site A. The projection device 3 at the site A projects the written image received from the site B, that is, the cancellation line, as the imaging image 660 on the whiteboard 2. The whiteboard 2 becomes a drawing board surface 662 obtained by superimposing a cancellation line of the base A on the writing 661, that is, “10”.

<Difference image generation processing>
FIG. 10 is a flowchart for explaining the operation of the difference image generation process. This processing is realized by the CPU 100 configuring the differential image generation unit 230 executing programs stored in the ROM 120 and the hard disk device using the RAM 110 as a work area. Assume that the image is stored in the RAM 110 of the work area.

  Details of the difference image generation processing (S2030, S2070) in the write image transmission operation of the information processing apparatus 5 shown in FIG. 7 will be described below with reference to FIGS. Here, the difference image generation unit 230 includes an image captured by the imaging device 4 (hereinafter simply referred to as “captured image”), an image supplied to the projection device 3 (hereinafter simply referred to as “projected image”), Filter processing is performed on the difference image generated based on these images.

  First, the difference image generation unit 230 performs a smoothing filter process to which the thickening process is applied to the projection image (S3000). Next, the difference image generation unit 230 separates the smoothed projected image and the captured image into R (red), G (green), and B (blue) components, respectively (S3010).

  Next, the difference image generation unit 230 performs steps 3050 to 3030 described below for each of the R, G, and B components. Here, the R, G, and B components take values from 0 to 255, and the luminance increases as this value increases. That is, when each of the R, G, and B components is 0, the color of the pixel is black, and when each of the R, G, and B components is 255, the color of the pixel is white. The difference image generation unit 230 generates integral images of corresponding components of the projection image and the captured image (S3020).

  Next, the difference image generation unit 230 executes S3030 to S3050 described below for each pixel of the captured image and each pixel of the projection image. First, the difference image generation unit 230 executes average difference calculation processing for calculating an average difference obtained by averaging the corresponding component with the corresponding component of surrounding pixels for the target pixel of the captured image (S3030).

  In the average difference calculation process, as shown in FIG. 11, the difference image generation unit 230 calculates a rectangle with a size of m × n with the target pixel as the center (S4000). Note that m and n are predetermined constants. When the size of the captured image is 1024 pixels × 768 pixels, for example, m = n = 31 pixels. Next, the difference image generation unit 230 calculates the average value of the luminance in the rectangle calculated in S4000 using the integrated image calculated in S3020 of FIG. 10 (S4010).

Specifically, the upper left luminance of the corresponding rectangle of the integral image is LT, the upper right luminance is RT, the lower left luminance is LB, the lower right luminance is RB, and the number of pixels in the corresponding rectangle is PN. The average value AVG of the luminance in the rectangle is calculated by the following formula.
AVG = (RB-RT-LB + LT) / PN

  The difference image generation unit 230 reduces the luminance of the target pixel by the average value AVG calculated in this way, calculates the average difference of the target pixel (S4020), and ends the average difference calculation process.

  Returning to FIG. 10, when the average difference calculation process for the target pixel of the captured image ends, the difference image generation unit 230 executes the average difference calculation process described with reference to FIG. 11 for the target pixel of the projection image. (S3040). Next, the difference image generation unit 230 executes a difference calculation process for calculating a difference value between the target pixels based on the average difference of the target pixels of the captured image and the average difference of the target pixels of the projection image (S3050). ).

  In the difference calculation process, as shown in FIG. 12, the difference image generation unit 230 determines whether or not the average difference of the target pixels of the captured image is larger than a predetermined threshold TH (S5000). In this example, the threshold value TH is -5.

  If it is determined that the average difference is greater than the threshold TH, the difference image generation unit 230 sets the difference value to the same component of the background color (S5080), and the difference calculation process ends. On the other hand, if it is determined that the average difference is not larger than the threshold value TH, the difference image generation unit 230 multiplies the constant difference (for example, 1.5 times) so that the average difference of the target pixels of the projection image becomes deeper (S5010). ).

  Next, the difference image generation unit 230 subtracts the average difference of the target pixels of the projection image multiplied by a constant from the average difference of the target pixels of the captured image (S5020). Next, the difference image generation unit 230 determines whether or not the difference value obtained as a result of the subtraction is larger than the threshold value TH (S5030).

  When it is determined that the difference value is greater than the threshold value TH, it can be determined that the target pixel of the photographed image is noise such as extraneous light brighter than the background. Therefore, the difference image generation unit 230 sets the difference value to the same component of the background color. (S5080), the difference calculation process is terminated.

  On the other hand, if it is determined that the difference value is not larger than the threshold value TH, the difference image generation unit 230 multiplies the constant value by a constant (for example, 1.5 times) so that the difference value becomes deep (S5040). Then, the difference image generation unit 230 adds the same component of the background color to the difference value (S5050). In the present embodiment, it is assumed that each component of the background color is 200.

  Here, the difference image generation unit 230 determines whether or not the difference value is less than 0. If it is determined that the difference value is less than 0, the difference image generation unit 230 sets the difference value to 0 (S5070), and performs difference calculation processing. Exit. On the other hand, if it is determined that the difference value is not less than 0, the difference image generation unit 230 ends the difference calculation process.

  Returning to FIG. 10, when the above-described processing for each R, G, B component of each pixel of the captured image and the projected image is completed, the difference image generation unit 230 has each pixel having a difference value of each R, G, B component. The difference image which consists of is produced | generated (S3060).

<Composite image generation processing>
FIG. 13 is a flowchart for explaining the operation of the composite image generation process. This processing functions when the CPU 100 configuring the writing image composition unit 260 executes programs stored in the ROM 120 and the hard disk device using the RAM 110 as a work area. Assume that the image is stored in the RAM 110 of the work area.

  Details of the composite image generation processing (S2040, S2080) in the write image transmission operation shown in FIG. 7 will be described with reference to FIG. This is a process of synthesizing the write image held by the write image synthesis unit 260 and the difference image input from the difference image generation unit 230.

  First, the writing image held by the writing image composition unit 260 and the difference image input from the difference image generation unit 230 are separated into R (red), G (green), and B (blue) components, respectively (S6000).

  Next, the written image composition unit 260 executes S6010 to S6020 described below for each of the R, G, and B components. Here, the R, G, and B components take values from 0 to 255, and the luminance increases as this value increases. That is, when each of the R, G, and B components is 0, the color of the pixel is black, and when each of the R, G, and B components is 255, the color of the pixel is white.

  Here, the written image composition unit 260 determines whether or not the difference value of the difference image is smaller than the background color component value (S6010). In this example, the background color component value is 200 for each color component. When it is determined that the difference value of the difference image is smaller than the background color component value, the writing image composition unit 260 replaces the pixel component of the writing image with the difference value of the difference image (S6020). On the other hand, the writing image composition unit 260 performs no processing when it is determined that the difference value of the difference image is not smaller than the background color component value.

  As described above, according to the image sharing system of the present invention, it is possible to create appropriate difference information between a captured image and a projected image, and to process a written image that does not cause a break between devices. An apparatus can be provided. That is, the information processing apparatus captures and captures a projected image in time series, and extracts a difference from the written image. Then, by synthesizing the differences, even if the projected image and the written image are superimposed, the written image transmitted to the other site is not interrupted. Since there is no interruption in the image in writing at each site, the effect of improving the readability of the written image is obtained.

(Example 2)
In the first embodiment, each time the acquired image of the local site is acquired, the formed image is changed in time series, the captured image is acquired, the difference image is generated, and the written image is synthesized. It is also conceivable to obtain a writing image of the local site by changing the formed image in time series when there is a close region in the formed image and the difference image.

<Functional blocks of information processing device>
FIG. 14 is a functional block diagram of the information processing apparatus according to the second embodiment. The difference from the first embodiment is that a proximity area image that generates a proximity area image of a difference image representing a difference in a shared area between a formed image projected on the projection apparatus 3 and a captured image captured by the imaging apparatus 4 This is the point that the generation unit 280 is added. The proximity area image generation unit 280 is configured by the CPU 100.

<Write acquisition control unit>
The writing acquisition control unit 270 according to the second embodiment first projects a formed image on the whiteboard 2 to acquire a captured image, and generates a difference image. Then, when there is a close region in the formed image and the difference image, control is performed to change the formed image in time series to synthesize the writing image of the local site. 15 and 16 show images of the second embodiment. The difference from the first embodiment is that a proximity region image shown in FIG. The proximity area image is an image obtained by projecting an image projected on the whiteboard and an area where writing on the whiteboard is close.

  First, as shown in FIG. 15, a state in which the projection device 3 projects 10 ”as an imaged image and“ 0 ”in the imaged image is close to the writing strike-through line will be described. 15A shows the first image formed, FIG. 15B shows the writing on the whiteboard 2, and FIG. 15C projects the first image formed on the writing on the whiteboard 2. Is a captured image. Here, as shown in FIG. 15C, a region where the writing and the first image are close is a close region. This proximity region extracts the difference image between the captured image and the imaged image shown in FIG. 15D and the pixel that is not the background color of the first imaged image shown in FIG. Image processing is performed to generate the proximity region image in FIG.

  Further, the imaging image to be changed in time series may be the imaging image shown in FIG. 15E obtained by performing contour extraction processing on the entire image in the same manner as in the first embodiment, or may be an image including an adjacent region. It may be a second image (M) shown in FIG. FIG. 15 (N) shows a second photographed image photographed using the imaged image of FIG. 15 (M). The second difference image obtained is the same regardless of whether the captured image is the image shown in FIG. 15F or FIG. 15N (FIG. 15G).

  The written image synthesis unit 260 synthesizes a plurality of difference images acquired by changing the formed image in time series, and generates a written image shown in FIG. In the present invention, when the imaged image shown in FIG. 15A, 15E, or 15M is projected onto the whiteboard, the portion including the adjacent region is not overlapped, so that a plurality of difference images are synthesized. As a result, an uninterrupted written image (FIG. 15H) can be obtained.

  Next, with reference to FIG. 16, a state in which “10” is projected as a formed image and there is an underline of writing that is not close to “0” in the formed image will be described. As shown in FIG. 16, since there is no near area, the near area image shown in FIG.

<Near-area image generator>
As shown in FIG. 14, the proximity region image generation unit 280 is a difference in the shared region 30 between the imaging image held by the imaging image supply unit 210 and the captured image captured by the imaging device 4 generated by the difference image generation unit 230. Are compared with each other in units of pixels to generate a proximity region image. Specifically, the proximity region image generation unit 280 includes pixels including pixels that are not the background color of the difference image in the peripheral region (for example, 3 pixels × 3 pixels rectangle) of the pixels that are not the background color of the formed image. And

<Writing image transmission operation flowchart>
FIG. 17 is a flowchart for explaining the write image transmission operation of the information processing apparatus 5 according to the second embodiment. As in the first embodiment, the write image transmission operation described below starts repeatedly at regular intervals. Alternatively, it may be started when a photographing request is generated via the input device 140 of the information processing apparatus 5. This processing is realized by the CPU 100 configuring each processing unit shown in FIG. 3 executing programs stored in the ROM 120 and the hard disk device using the RAM 110 as a work area. Assume that the image is stored in the RAM 110 of the work area. Each processing unit that performs this processing is controlled by the write acquisition control unit 270.

  The difference from the first embodiment is that, when a proximity region image is generated and the proximity region (region information) is not empty, the imaging image is changed in time series to extract the writing image of the local site. is there. Hereinafter, only steps different from the first embodiment will be described on the flowchart. The proximity region image generation unit 280 generates a proximity region image indicating the proximity region between the image supplied from the imaging image supply unit 210 to the projection device 3 and the first difference image generated in S2030 (S2043).

  The image transmission unit 240 determines whether or not the proximity area image generated by the proximity area image generation unit 280 is empty (S2045). If the image transmission unit 240 determines that the proximity region image is empty, the image transmission unit 240 performs the processing from S2100 onward in order to transmit the image generated in S2040 as it is to the other site as a self-written image. On the other hand, when the image transmission unit 240 determines that the proximity region image generated by the proximity region image generation unit 280 is not empty, the image transmission unit 240 performs processing for changing the imaged image in time series after S2050 and extracting the self-site-written image. Do.

<Operation example of image sharing system>
FIG. 18 is a conceptual diagram for explaining an operation example of the image sharing system 1 in the second embodiment. An information processing device 5 is installed at each of the bases A and B, and a writing image exchanged between the bases, a projection image projected by the projection device 3, and a photographed image photographed by the photographing device 4, respectively Examples are shown in time series.

  Only the difference from the first embodiment, that is, only the operation part using the proximity region image will be described. Here, an operation of sending a written image from the site B to the site A is shown. A case where writing is overwritten on the projection will be described. First, an example in which a writing image “10” of the site A is projected onto the whiteboard 2 and a strikethrough is written with a marker pen or the like is shown. Since there is a proximity region in the formed image and writing, the case where the proximity region image is not empty is shown.

  In FIG. 18A, the written image “10” received from the site A by the information processing apparatus is superimposed and projected onto the whiteboard 2 at the site B, and the written image is transmitted in a state where a strikethrough is written on the projected image. Show the case. First, the projection device 3 at the site B superimposes the image written at the other site on the writing 741 of the whiteboard 2 as a formed image 740, and captures the superimposed drawing board surface 742 to obtain a captured image 743. Then, the difference image generation unit 230 obtains a first difference image 744 by taking the difference between the formed image 740 and the captured image 743.

  Next, the proximity area image generation unit 280 obtains a proximity area image 745 in which the proximity areas of the imaging image 740 and the first difference image 744 are extracted. Since the proximity region image 745 is not empty, the projection device 3 at the site B superimposes and projects the image obtained by extracting the contour of the writing image at the other site onto the writing 751 of the whiteboard 2 as the imaging image 750. The imaging device 4 acquires the captured image 753 by imaging the drawing board surface 752 that is superimposed and projected. The difference image generation unit 230 obtains a second difference image 754 by taking the difference between the formed image 750 and the captured image 753. The difference image generation unit 230 combines the first difference image 744 and the second difference image 754 to generate a combined image 755 of the base B.

  The written image of the site A is “10”, and the strike-through line is written on the whiteboard 2. Therefore, the generated first difference image 744 and the second difference image 754 are strike-through lines where breaks occur. When these two difference images are combined, a combined image 755 in which a continuous strikeout line is reproduced is obtained. Since the generated composite image 755 is not blank, the information processing apparatus 5 transmits the composite image 755 to the base B. At the site B, the received written image is defined as a formed image 756.

<When the proximity image is empty>
Next, an example is shown in which an underline is written with a marker pen or the like under the image “10” of the site A projected onto the whiteboard 2. Since there is no adjacent area in the formed image and writing, the adjacent area image is empty.

  FIG. 18B shows the writing image “10” received from the site A by the information processing apparatus superimposed on the whiteboard 2 at the site B, and the written image is transmitted with the underline overlaid on the projected image. Show the case. First, the projection device 3 at the site B superimposes and projects the image written at the other site as the image 760 on the address 761 on the whiteboard 2. The imaging device 4 acquires the captured image 763 by imaging the drawing board surface 762 superimposed and projected. Then, the difference image generation unit 230 obtains a first difference image 764 by taking the difference between the formed image 760 and the captured image 763.

  Next, the proximity region image generation unit 280 obtains a proximity region image 765 obtained by extracting the proximity region between the imaging image 760 and the first difference image 764. Since the proximity region image 765 is empty, it is generated as a writing image 775 of the first difference image 764 site B. Since the written image of the site A is “10” and the underline is written in the whiteboard 2, the generated first difference image 764 is originally uninterrupted. Since the generated writing image 775 is not blank, the information processing apparatus 5 transmits the writing image 775 to the base B. At the site B, the received written image is defined as an image 776.

<Near-area image generation processing>
FIG. 19 is a flowchart for explaining the operation of the proximity region image generation process. Here, in the write image transmission operation of the information processing apparatus 5 shown in FIG. 17, processing (S2043) for generating a proximity region image will be described with reference to FIG. 19. This process is performed by the proximity region image generation unit 280. The difference image representing the difference in the shared area 30 between the formed image held by the formed image supply unit 210 and the captured image captured by the imaging device 4 generated by the difference image generation unit 230 is compared in units of pixels. . Then, a proximity area image is generated. This processing functions when the CPU 100 configuring the proximity area image generation unit 280 executes programs stored in the ROM 120 and the hard disk device using the RAM 110 as a work area. Assume that the image is stored in the RAM 110 of the work area.

  First, a difference image representing a difference in the shared region 30 between the formed image held by the formed image supply unit 210 and the captured image taken by the imaging device 4 generated by the difference image generating unit 230 is represented by R (red), Separated into G (green) and B (blue) components, respectively (S7000). Here, the R, G, and B components take values from 0 to 255, and the luminance increases as this value increases. That is, when each of the R, G, and B components is 0, the color of the pixel is black. When each of the R, G, and B components is 255, the color of the pixel is white.

  Next, the proximity region image generation unit 280 executes S7010 to S7050 described below for each pixel of the formed image and each pixel of the difference image. First, all the RGB color component values of the target pixel of the proximity image are set as the background color component values (S7010). Here, the component value of the background color is 200 for each of RGB. Next, S7020 to S7050 described below are executed for each of the R, G, and B components.

  Here, the proximity region image generation unit 280 determines whether the component value of the formed image is smaller than the background color component value (S7020). The proximity region image generation unit 280 does not perform any processing when the component value of the formed image is not smaller than the background color component value. On the other hand, if it is determined that the component value of the formed image is smaller than the background color component value, it is determined whether m × n centered on the target pixel of the difference image has a component value smaller than the background color component value (S7040). ).

  Here, the proximity region image generation unit 280 does not perform processing unless m × n centered on the target pixel of the difference image has a component value smaller than the background color component value. On the other hand, if m × n centered on the target pixel of the difference image has a component value smaller than the background color component value, all the RGB color component values of the target pixel of the adjacent image are set as the component values of the formed image (S7050). ).

  As described above, in the image sharing system of the present invention, when there is an imaging image to be projected on the whiteboard and an area where writing on the whiteboard is close, the projected image is changed in time series, and the difference is taken. Extract and synthesize. Thereby, even if the projection image and the writing are overlapped and written and extracted, the written image can be prevented from being interrupted. In addition to the effects of the first embodiment, the processing load is reduced because the time-series processing is performed only when the image formed on the whiteboard and the adjacent area exist for writing.

(Example 3)
In the first embodiment, as shown in FIG. 5, the image to be changed in time series is an image obtained by extracting the contour of the received image received from another base. The image to be changed in time series is not limited to the contour extraction, and it is conceivable to shift the position or change the luminance.

  FIG. 20 shows an example of shifting the position of the formed image in time series. FIG. 20A shows an example of the first imaged image, which is the image received by the image receiving unit 200 as it is. On the other hand, FIG. 20E shows an example of the second imaged image, which is an image obtained by moving the horizontal position of the imaged image to the right by the distance A with respect to FIG. Assume that the image in FIG. 20E is moved by image processing to a position that does not overlap with the image in FIG.

  An example of a captured image acquired by the captured image acquisition unit 220 by changing the formed image in time series is shown in FIGS. 20 (C) and 20 (F). It is assumed that the writing in FIG. 20B is written on the whiteboard 2 on which the formed image is projected. The combined images of FIGS. 20A and 20E are superimposed and projected in time series for writing on the whiteboard 2, and two types of formed images of FIGS. 20C and 20F are acquired.

  Then, the difference image generation unit 230 generates a difference image between the captured image and the formed image. An example of the difference image is shown in FIGS. 20D and 20G. In each of the portions where the formed image and the writing overlap, a break occurs.

  The written image synthesis unit 260 synthesizes a plurality of difference images acquired by changing the formed image in time series to generate a written image shown in FIG. In the present invention, since the imaged images of FIGS. 20A and 20E are not overlapped when projected onto a whiteboard, when a plurality of difference images are combined, an uninterrupted written image can be obtained. it can.

  FIG. 21 shows an example in which the brightness of the formed image is changed in time series. FIG. 21A shows an example of the first imaged image, which is the image received by the image receiving unit 200 as it is. On the other hand, FIG. 21E shows an example of the second image formed by contour extraction so as not to overlap the image of FIG. 21A and image processing for increasing the brightness inside the contour relative to the original image. Assumed to be performed. Specifically, the black portion of the image in (A) is thickened, the contour is extracted, and the brightness inside the contour is increased to a value at which writing can be detected from the captured image.

  An example of the captured image acquired by the captured image acquisition unit 220 by changing the formed image in time series is shown in FIGS. It is assumed that the whiteboard 2 on which the formed image is projected is written as shown in FIG. 21A and 21E are superimposed and projected in time series for writing on the whiteboard 2, and two types of formed images shown in FIGS. 21C and 21F are acquired. . Then, the difference image generation unit 230 generates a difference image between the captured image and the formed image. An example of the difference image is shown in FIGS. 21 (D) and 21 (G). In each of the portions where the formed image and the writing overlap, a break occurs.

  The written image synthesizing unit 260 synthesizes a plurality of difference images acquired by changing the formed image in time series to generate a written image shown in FIG. In the present invention, when the formed image shown in FIGS. 21A and 21E is projected onto the whiteboard, the outline portion of the original image and the outline image do not overlap, and the brightness inside the outline is detected by writing. it can. Therefore, when a plurality of difference images are combined, an uninterrupted written image (H) can be obtained.

  As described above, the image sharing system of the present invention changes the projected image in time series, captures, extracts a difference, and synthesizes the written image even if the projected image and the written image are overwritten and extracted. It is possible to prevent interruptions. Since there is no interruption in the image for writing at each site, the readability of the written image is enhanced.

(Other embodiments)
The present invention can also be realized by executing the following processing. That is, software (computer program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media. Then, the computer (or CPU, MPU, etc.) of the system or apparatus reads out and executes the program. In this case, the program and the storage medium storing the program constitute the present invention.

230 Difference Image Generation Unit 260 Written Image Synthesis Unit

Claims (7)

An information processing apparatus that shares captured image information among a plurality of information processing apparatuses via a network,
Creating means for creating a projection image to be projected on a projection surface based on image information received from another information processing apparatus;
Obtaining means for obtaining a photographed image photographed using the projection surface as a photographing surface;
A first difference image is generated from the projected image and the captured image, the projected image is corrected, and the corrected projected image is projected onto the projection plane and captured from the corrected projected image. Generating means for generating a second difference image;
Combining means for combining the first and second difference images to generate a combined image;
Transmitting means for transmitting the composite image to the other information processing apparatus. An information processing apparatus comprising: The said generation means correct | amends the said projection image by extracting the outline of the said projection image, The said projection image after correction | amendment does not overlap with the said projection image before correction | amendment. Information processing device. The information processing apparatus according to claim 1, wherein the generation unit corrects the projection image by changing a position of the projection image on the projection plane. The said generating means correct | amends the said projection image by changing the brightness | luminance of the said projection image, The said projection image after correction | amendment does not overlap with the said projection image before correction | amendment. Information processing device. In the imaging surface, when the projected image and the captured image are superimposed, the image capturing surface further includes an acquisition unit that acquires region information of a region to be superimposed,
The information processing apparatus according to claim 1, wherein the generation unit corrects the projection image when the area information is acquired. An information processing apparatus control method for sharing captured image information among a plurality of information processing apparatuses via a network,
A creation step of creating a projection image to be projected onto a projection surface based on image information received from another information processing apparatus;
An obtaining step of obtaining a photographed image photographed using the projection surface as a photographing surface;
A first difference image is generated from the projected image and the captured image, the projected image is corrected, and the corrected projected image is projected onto the projection plane and captured from the corrected projected image. A generating step of generating a second difference image;
A combining step of combining the first and second difference images to generate a combined image;
And a transmission step of transmitting the composite image to the other information processing apparatus.   A computer program causing a control method according to claim 6 to be executed by a computer.

Images