JP2014167701A - Projection control device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow a simple configuration to project information based on writing to a screen so as to be easily viewed.SOLUTION: A projection control device 30 comprises: a first control section 35 that controls a projector 10 projecting an image on a screen; a second control section 35 that controls a camera 20 photographing the screen; an image generation section 36 that generates a first image making first information written on the screen inconspicuous on the basis of image data photographed by the camera 20; and an image synthesis section 34 that adds second information based on the first information to the first image. The first control section 35 causes the first image with the second information added to be projected from the projector 10.

Description

  The present invention relates to a projection control apparatus.

  A technique is known in which when a presentation is performed using a projector, a computer recognizes characters written on a screen by a presenter and projects the recognized characters (see Patent Document 1).

JP 2011-103092 A

  In the prior art, a dedicated screen on which a dot pattern is formed and a dedicated pen for reading the dot pattern on the screen are required, and it is difficult to reduce the cost.

  A projection control apparatus according to the present invention includes a first control unit that controls a projector that projects an image on a screen, a second control unit that controls a camera that captures the screen, and a screen based on image data captured by the camera. An image generation unit that generates a first image that makes the first information written inconspicuous; and an image synthesis unit that adds second information based on the first information to the first image, and the first control unit Is characterized in that the first image to which the second information is added is projected from the projector.

  According to the projection control apparatus of the present invention, information based on writing on the screen can be projected easily with a simple configuration.

It is a figure explaining the use scene of the projection control apparatus by 1st embodiment. It is the figure which expanded the projection surface of the screen. It is a block diagram explaining the principal part structure of a projection control apparatus, a digital camera, and a projector. (a) is a diagram illustrating a frame image transmitted from a digital camera, (b) is a diagram illustrating a frame image in which a writing part is color-converted to a complementary color from (a), and (c) is (b). ) Is a diagram illustrating a frame image in which the background portion of the frame image is color-converted to gray, and (d) is a diagram illustrating an image projected on the screen. It is a flowchart which illustrates the flow of the process which the control part of a projection control apparatus performs. 10 is a flowchart illustrating an example of a flow of processing executed by a control unit that performs a projection operation of Modification 2. It is a block diagram explaining the principal part structure by 2nd embodiment. (a) is a diagram illustrating a frame image transmitted from a digital camera, (b) is a diagram illustrating a frame image in which a writing portion is color-converted to a complementary color from the frame image of (a), and (c) is (b) ) Is a diagram illustrating a frame image in which the background portion of the frame image is color-converted to gray, and (d) is a diagram illustrating an image projected on the screen.

Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings.
(First embodiment)
<Explanation of usage>
FIG. 1 is a diagram for explaining a usage scene of the projection control apparatus according to the first embodiment of the present invention. In FIG. 1, the projector 10 projects an image toward the screen 40. A video signal for projection by the projector 10 is supplied from the projection control device 30 to the projector 10 by wireless communication. The person A who is a presenter makes a presentation with reference to an image projected from the projector 10 onto the screen 40.

  The digital camera 20 captures the projection surface of the screen 40 at a predetermined frame rate (for example, 60 fps), and transmits the captured frame image to the projection control device 30 by sequential wireless communication. The projection control device 30 is configured by a personal computer, for example. The projection control device 30 controls the imaging operation by the digital camera 20 and the projection operation by the projector 10 by executing a predetermined program. The server 60 stores projection data and the like, and provides necessary data to the projection control device 30 in response to a request from the projection control device 30.

  FIG. 2 is an enlarged view of the projection surface of the screen 40. On the screen 40, presentation materials such as graphs are projected from the projector 10. The person A will be described while writing characters on the screen 40 with the marker pen 50. The projection control device 30 during the presentation recognizes characters written by the marker pen 50 on the screen 40 by the person A based on the frame image transmitted from the digital camera 20, and the recognized characters are easy to see for the participants. Then, it is projected on the screen 40 from the projector 10 so as to overlap the presentation material being projected.

<Explanation of projection control>
Since the present embodiment is characterized in that the characters recognized by the projection control device 30 are projected from the projector 10 onto the screen 40 in a manner that is easy for the participant to see, the following description will be focused on this point. FIG. 3 is a block diagram for explaining a main configuration of the projection control device 30, the digital camera 20, and the projector 10 of FIG.

  The digital camera 20 includes an imaging optical system 21, an imaging element 22, an image processing unit 23, and a control unit 24. The imaging optical system 21 forms a subject image (the subject in this example is a screen 40) on the imaging surface of the imaging element 22. The image sensor 22 captures a subject image and outputs image data. The image processing unit 23 performs predetermined image processing (for example, contour enhancement, white balance adjustment, etc.) on the image data. The control unit 24 performs control for each unit of the digital camera 20 and communication with the projection control device 30.

  The projection control device 30 includes a handwriting detection unit 31, a character recognition unit 32, a character generation unit 33, a synthesis unit 34, a control unit 35, a color conversion unit 36, and a storage unit 37. Based on the frame image transmitted from the digital camera 20, the handwriting detection unit 31 detects the handwriting written with the marker pen 50 on the screen 40 by the person A. FIG. 4A is a diagram illustrating a portion written by the person A in the frame image transmitted from the digital camera 20. According to FIG. 4A, “AIUE” is written on the screen 40 of the screen imaged by the digital camera 20.

  Here, the projection surface of the screen 40 is a white background, and the ink color of the marker pen 50 is a pastel color (for example, pink) having a low saturation and a high lightness. In FIG. It is shown in black for easy understanding.

  In the first embodiment, the timing at which the color conversion unit 36 starts color conversion and the timing at which the character recognition unit 32 starts character recognition processing are the timings at which the person A has finished a series of writing. Specifically, the control unit 35 compares the latest frame image with the previous frame image, and continuously changes the handwriting (pink on a white background) detected by the handwriting detection unit 31 for a predetermined time (for example, 2 seconds). The case where no occurrence occurs is determined as the write end timing.

  When the control unit 35 determines that a series of writing by the person A has been completed, the color conversion unit 36 writes the writing portion detected by the handwriting detection unit 31 in the frame image of FIG. Is converted to a complementary color (green when the ink color is pink). The lightness after color conversion is increased as the lightness of the writing color is higher, and is decreased as the lightness of the writing color is lower. FIG. 4B is a diagram illustrating a frame image in which a writing portion is color-converted to a complementary color from the frame image of FIG. In FIG. 4B, the written portion “Aiue” after color conversion is represented by an intermediate gradation.

  Further, the color conversion unit 36 performs color conversion on the background portion other than the writing portion in the frame image of FIG. 4B to an intermediate gradation (gray) with low saturation. The lightness after the color conversion is set to be substantially the same as the lightness of the color-converted portion. FIG. 4C is a diagram illustrating a frame image in which the background portion of the frame image in FIG. 4B is color-converted to gray.

  On the other hand, the character recognition unit 32 performs a known character recognition process on the writing portion detected by the handwriting detection unit 31. The character generation unit 33 generates a character image in which the character string “Aiue” recognized by the character recognition unit 32 is written using a predetermined font (for example, Gothic). The character color is, for example, black. The synthesizing unit 34 generates a frame image obtained by superimposing the character images of the character string “Aiue” generated by the character generating unit 33 on the frame image of FIG.

  The control unit 35 performs communication between the projection control device 30 and the digital camera 20, and communication between the projection control device 30 and the projector 10, in addition to control of each unit of the projection control device 30. The storage unit 37 stores each frame image and character image in the process described above. The reason for storing the information in the storage unit 37 is that the state written by the person A with the marker pen 50 and the contents of the character string recognized by the character recognition unit 32 can be searched later.

  The control unit 35 sends a video signal to the projector 10 in order to project the frame image synthesized by the synthesis unit 34 from the projector 10.

  The projector 10 includes a projection optical system 11, a projection image generation unit 12, a light source 13, and a control unit 14. The light source 13 illuminates the projected image generation unit 12. The projection video generation unit 12 is configured by a liquid crystal panel, for example, and modulates illumination light based on a video signal from the projection control device 30. The projection optical system 11 projects the light image modulated by the projection image generation unit 12 toward the screen 40.

  FIG. 4D is a diagram illustrating an image projected on the screen 40. According to FIG. 4 (d), the portion written by the person A has a lower saturation due to the complementary color (green) superimposed on the pink color, and is visually recognized as gray by the participant. Further, as described above, the background portion other than the writing portion is also visually recognized as gray. From the above, the part written by the person A is confused with the gray background part and is not easily seen by the participants.

  The portion of the clear text generated by the character generation unit 33 is displayed in black with a gothic style with the above-described gray as a background, so that it is easy for the participant to visually recognize. The control unit 14 controls each unit of the projector 10 and communicates with the projection control device 30.

<Description of flowchart>
FIG. 5 is a flowchart illustrating the flow of processing executed by the control unit 35 of the projection control device 30 in order to perform the projection operation described above. When the start of the presentation is instructed by the person A, the control unit 35 starts the processing shown in FIG. In step S101 of FIG. 5, the control unit 35 inputs the latest frame image data from the digital camera 20, and proceeds to step S102.

  In step S102, the control unit 35 determines whether the person A has finished a series of writing. When determining the end of writing, the control unit 35 makes a positive determination in step S102 and proceeds to step S103. When not determining the end of writing, the control unit 35 makes a negative determination in step S102 and returns to step S101. When returning to step S101, the above-described processing is repeated.

  In step S103, the control unit 35 causes the character recognition unit 32 to perform character recognition and proceeds to step S104. In step S104, the control unit 35 sends an instruction to the character generation unit 33 to generate a character image in which the character string “Aiue” recognized by the character recognition unit 32 is clarified, and the process proceeds to step S105.

  In step S105, the control unit 35 sends an instruction to the color conversion unit 36 to cause the above-described color conversion to proceed to step S106. Note that the color conversion process may be performed in parallel with steps S103 and S104.

  In step S106, the control unit 35 sends an instruction to the synthesizing unit 34, synthesizes the character image with the color-converted frame image, and proceeds to step S107. In step S107, the control unit 35 sends a video signal necessary for projecting the synthesized frame image to the projector 10 and proceeds to step S108.

  In step S108, the control unit 35 causes the storage unit 37 to store the frame image that has undergone character recognition (S103), the contents of the recognized character string, and the data of the frame image after color conversion (S105). The process proceeds to step S109.

  In step S109, the control unit 35 determines whether or not an end operation has been performed. When the end of the presentation is instructed by the person A, the control unit 35 makes an affirmative determination in step S109 and ends the processing in FIG. If the end of the presentation is not instructed, the control unit 35 makes a negative determination in step S109 and returns to step S101.

  In the present embodiment, it is assumed that the person A erases the writing of the marker pen 50 when the screen 40 becomes dirty due to the writing of the marker pen 50. When the writing is erased, the handwriting detection unit 31 does not detect the handwriting, so the control unit 35 cancels the above-described color conversion processing, character generation processing, and the like.

According to the first embodiment described above, the following operational effects can be obtained.
(1) The projection control device 30 makes a frame image (FIG. 4 (c) that makes the writing “aiue” written on the screen 40 inconspicuous based on the image data captured by the digital camera 20 that captures the screen 40. )) And a composition unit 34 for adding the character image based on “Aiue” to the frame image (FIG. 4C), and the frame image to which the character image is added is a projector. Projected from 10. Thereby, the information based on the writing of 40 on the screen can be appropriately projected with a simple configuration.

(2) In the projection control device 30 of (1) above, the character generation unit 33 generates a character image that has been clarified by changing all aspects of the writing “aiue”, and the combining unit 34 converts the character image into a frame image. (FIG. 4 (c)). As a result, even if the character “AIUE” written on the screen 40 is difficult to read, it can be projected in an easy-to-see manner.

(3) The projection control device 30 of the above (1) and (2) includes a control unit 35 that detects the end of writing of “Aiue”. When the control unit 35 detects the above, a character image is added. The frame image is projected from the projector 10. As a result, it is possible to project clear letters at an appropriate timing.

(4) In the projection control device 30, the color conversion unit 36 performs a frame so that the saturation and brightness of the writing “Aiue” on the screen 40 are close to the saturation and brightness of the background other than “Aiue”. Since the image is generated, the writing “Aiue” itself can be confused with the background portion to make it difficult for the participant to see.

(5) The projection control device 30 includes the control unit 35 that stores the image data captured by the digital camera 20, the frame image data, and the frame image data to which the character image is added in the storage unit 37. The state written on the screen 40 and the contents of the characters can be retrieved later.

(Modification 1)
In the first embodiment, in step S102 (FIG. 4), the color conversion unit 36 starts color conversion at the timing when the control unit 35 determines the end of a series of writing by the person A, and the character recognition unit 32 detects characters. An example of starting the recognition process has been described. Instead, at the timing when the handwriting detection unit 31 detects that the person A has performed writing indicating the target of character recognition (surrounding the target or drawing an underline or sideline on the target), the color conversion unit 36 changes the color. You may comprise so that conversion may be started and the character recognition part 32 may start a character recognition process.

  The character generating unit 33 generates a character image in which the character string “Aiue” recognized by the character recognizing unit 32 is printed, the combining unit 34 combines the character image with the frame image after color conversion, and the combining The point that the subsequent frame image is projected from the projector 10 is the same as in the case of the first embodiment.

  The handwriting detection unit 31 according to the first modification detects, for example, a circle or an ellipse or a quadrangle surrounded by a handwriting (pink on a white background). In this case, the color conversion unit 36 sets all writing including a box as a target of color conversion, and the character recognition unit 32 sets writing inside the box as a target of character recognition. Or the handwriting detection part 31 of the modification 1 detects an underline and a side line in a handwriting (pink in a white background). In this case, the color conversion unit 36 sets all writing including underline and side line as the target of color conversion, and the character recognition unit 32 sets writing corresponding to the underline and side line as the target of character recognition.

  According to the first modification, in the case where the character A does not want to be projected as a clear character, such as when it is difficult to separate the target range for character recognition due to the continuous character, or when the character A writes it in error Thus, it is possible to properly perform character recognition. In addition, since the box, underline, and side line are included in the target of color conversion, all writing by the marker pen 50 can be obscured by the participant by making it gray in the background portion.

(Modification 2)
In the first modification described above, the projector 10 is configured to perform projection that emphasizes the writing position until the person A performs writing indicating the object of character recognition (surrounding the object or drawing an underline or a side line on the object). May be.

  The color conversion unit 36 of Modification 2 performs color conversion (referred to as first color conversion) to make it easy to visually recognize the writing by the marker pen 50 once writing by the marker pen 50 is performed by the person A. When writing indicating the character recognition target is performed by the person A, color conversion (referred to as second color conversion) for making the writing with the marker pen 50 difficult to visually recognize is performed. The second color conversion corresponds to the color conversion performed in the first embodiment and the first modification described above.

  When the handwriting detection unit 31 detects a handwriting (pink on a white background), the control unit 35 of Modification 2 causes the color conversion unit 36 to start the first color conversion. In this case, the color conversion unit 36 converts the writing portion detected by the handwriting detection unit 31 in the frame image of FIG. 4A to magenta, for example. The control unit 35 sends a projection signal to the projector 10 so that the frame image after the first color conversion is projected on the screen 40. In the portion written by the person A, the magenta overlaps from above the pink color, and the saturation increases. Therefore, the person A can easily visually recognize his / her writing content on the screen 40 because the pink writing is emphasized by the magenta projection light.

  When the person A performs writing indicating the target range of character recognition (surrounds the object or draws an underline or a side line), the color conversion unit 36 detects the surrounding, underline, or side line by the handwriting detection unit 31. At the timing, the writing portion including the box, underline, and side line is color-converted to the complementary color of the writing color (green when the ink color is pink) as the second color conversion. Note that the process of converting the background part other than the writing part into the intermediate gradation (gray) is also included in the second color conversion process.

  The character recognition unit 32 of Modification 2 starts the character recognition process at the timing when the handwriting detection unit 31 detects writing indicating the character recognition target. A point that the character generation unit 33 generates a character image in which the character string “Aiue” recognized by the character recognition unit 32 is written, a point that the combining unit 34 combines the character image with the frame image after the second color conversion, The point that the combined frame image is projected from the projector 10 is the same as in the first embodiment and the first modification.

  FIG. 6 is a flowchart illustrating the flow of processing executed by the control unit 35 of the projection control device 30 in order to perform the projection operation of Modification 2 described above. When the start of the presentation is instructed by the person A, the control unit 35 starts the processing shown in FIG. In FIG. 6, the processing from step S102B to step S102D is different from that in FIG. 5 (first embodiment). For this reason, only differences from FIG. 5 will be described, and description of points common to FIG. 5 will be omitted.

  In step S <b> 102 </ b> B of FIG. 6, the control unit 35 determines whether or not the handwriting detection unit 31 has detected handwriting (pink on a white background) based on the frame image data input from the digital camera 20. When the handwriting is detected, the control unit 35 makes a positive determination in step S102B and proceeds to step S102C. When the handwriting is not detected, the control unit 35 makes a negative determination in step S102B and returns to step S101.

  In step S102C, the control unit 35 sends an instruction to the color conversion unit 36, performs the first color conversion process, and proceeds to step S102D. In step S102D, the control unit 35 determines whether or not writing indicating a character recognition target is detected. The control unit 35 makes an affirmative decision in step 102D when the handwriting detection unit 31 detects the above-described box, underline, or side line, and proceeds to step S103. A negative determination is made and the process returns to step S101.

  The second color conversion in step S105 has the same processing contents as the color conversion (S105) in FIG.

  According to the second modification, when the color of the marker pen 50 is too light, the projector 10 is controlled so as to emphasize the writing by the marker pen 50. Therefore, the range to be a character recognition target is surrounded, or an underline or a side line is drawn. Work for the person A is easier. After writing indicating the target range of character recognition, the projector 10 is controlled so that writing by the marker pen 50 is difficult to visually recognize. Therefore, the portion written by the person A is lost in gray in the background portion. It becomes difficult to see from the person.

  Generally, when the color of the marker pen 50 is dark, it is necessary to project a dark color that is a complementary color, and the projected image becomes dark. In order to avoid this, the modification 2 is particularly effective when using the light-colored marker pen 50.

(Second embodiment)
In the second embodiment, instead of generating a character image in which the character string recognized by the character recognition unit 32 is clarified, image data associated with the character string recognized by the character recognition unit 32 (for example, FIG. A). Since the points to acquire are different, this difference will be mainly described.

<Explanation of projection control>
FIG. 7 is a block diagram illustrating the main configuration of the projection control device 30B, the digital camera 20, and the projector 10 according to the second embodiment. The projection control device 30B is different from the projection control device 30 in FIG. 3 in that it has a data acquisition unit 38 instead of the character generation unit 33. Hereinafter, a description will be given with reference to FIG. FIG. 8A is a diagram illustrating a portion written by the person A in the frame image transmitted from the digital camera 20. According to FIG. 8A, “FIG. A” is written on the screen 40 of the screen imaged by the digital camera 20.

  The projection surface of the screen 40 is white as in the first embodiment, and the ink color of the marker pen 50 is pastel color (for example, pink). In FIG. Therefore, it is shown in black.

  In the second embodiment, the timing at which the color conversion unit 36 starts color conversion and the timing at which the character recognition unit 32 starts character recognition processing are the same as those in the first embodiment. Timing.

  When the control unit 35 determines that the series of writing by the person A has been completed, the color conversion unit 36 writes the writing portion detected by the handwriting detection unit 31 in the frame image of FIG. Is converted to a complementary color (green when the ink color is pink). FIG. 8B is a diagram illustrating a frame image in which the writing portion is color-converted to a complementary color from the frame image of FIG. In FIG. 8B, the written portion “FIG. A” after color conversion is represented by an intermediate gradation.

  Further, the color conversion unit 36 performs color conversion of the background portion other than the writing portion in the frame image of FIG. 8B to an intermediate gradation (gray). FIG. 8C illustrates a frame image in which the background portion of the frame image in FIG. 8B is color-converted to gray.

  On the other hand, the character recognition unit 32 performs a known character recognition process on the writing portion detected by the handwriting detection unit 31. The data acquisition unit 38 acquires image data associated with a character string (for example, FIG. A) recognized by the character recognition unit 32 in advance from the server 60. The data acquisition process by the data acquisition unit 38 is performed instead of the process of step S104 in the flowchart in FIG.

  The synthesizing unit 34 generates a frame image obtained by superimposing and synthesizing the image 80 based on the image data acquired by the data acquiring unit 38 from the frame image of FIG. The point that the combined frame image is projected from the projector 10 is the same as in the first embodiment and the first and second modifications. FIG. 8D is a diagram illustrating an image projected on the screen 40. According to FIG. 8 (d), the portion written by the person A is reduced in saturation due to overlapping of the complementary color (green) from above the pink color, and is visually recognized as gray by the participant. Further, as described above, the background portion other than the writing portion is also visually recognized as gray. From the above, the part written by the person A is confused with the gray background part and is not easily seen by the participants. On the other hand, since the image 80 is displayed with the above-described gray background, it is easy for a participant to visually recognize the image 80.

According to the second embodiment described above, the following operational effects can be obtained.
(1) The projection control device 30B makes a frame image (FIG. 8 (c) that makes the writing “FIG. A” written on the screen 40 inconspicuous based on the image data captured by the digital camera 20 that captures the screen 40. )) And a compositing unit 34 for adding the image 80 based on “FIG. A” to the frame image (FIG. 8C), and the frame image with the image 80 added thereto is a projector. Projected from 10. Thereby, the information based on the writing of 40 on the screen can be appropriately projected with a simple configuration.

(2) In the projection control device 30B, the compositing unit 34 adds the image 80 associated in advance with the writing “FIG. A” to the frame image (FIG. 8C). Thereby, not only characters but also a predetermined image can be projected.

(3) The projection control device 30B includes a control unit 35 that detects the completion of the writing of “FIG. A”. When the control unit 35 performs the detection, the frame image with the image 80 added is projected from the projector 10. I tried to make it. Thereby, the image 80 can be projected at an appropriate timing.

  In the above description, the person A erases the writing of the marker pen 50. Instead of this, a pen whose ink color disappears with time may be used.

  The above description is merely an example, and is not limited to the configuration of the above embodiment. Each embodiment and each modification may be appropriately combined.

DESCRIPTION OF SYMBOLS 10 ... Projector 20 ... Digital camera 30, 30B ... Projection control apparatus 33 ... Character production | generation part 34 ... Composition part 35 ... Control part 36 ... Color conversion part 40 ... Screen 50 ... Marker pen 60 ... Server 80 ... Image A ... Person

Claims (8)

A first control unit that controls a projector that projects an image on a screen;
A second control unit that controls a camera that images the screen;
An image generating unit that generates a first image that makes the first information written on the screen inconspicuous based on image data captured by the camera;
An image composition unit for adding second information based on the first information to the first image,
The projection control apparatus, wherein the first control unit projects the first image to which the second information is added from the projector. The projection control device according to claim 1,
The image composition unit generates the second information by changing the aspect of information within a predetermined range of the first information or the first information, and adds the second information to the first image. Projection control device. The projection control device according to claim 1,
The image synthesizing unit adds the second information previously associated with all of the first information or information within a predetermined range of the first information to the first image. Control device. In the projection control device according to claim 2 or 3,
A detection unit for detecting completion of writing of the first information or detecting writing indicating the predetermined range in the first information;
The first control unit, when detected by the detection unit, projects the first image to which the second information is added from the projector. The projection control device according to claim 4,
The image generation unit generates a second image that makes the first information stand out based on image data captured by the camera,
The first control unit may include the second image before the detection unit detects the end of writing of the first information, or before the detection unit detects writing indicating the predetermined range in the first information. Projection control apparatus for projecting from the projector. The projection control device according to claim 5,
The projection control apparatus, wherein the image generation unit generates the second image so as to increase a saturation of the first information on the screen. In the projection control apparatus as described in any one of Claims 1-6,
The image generation unit generates the first image so that the saturation and brightness of the first information on the screen are close to the saturation and brightness of a background other than the first information. Projection control device. In the projection control apparatus as described in any one of Claims 1-7,
A projection control apparatus, further comprising: a storage control unit that stores image data captured by the camera, data of the first image, and data of the second image in a storage member.

Images