JP2014163979A - Projector device, control method thereof, and control program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform image synthesis with improved image quality, regardless of positional relationship between a projector device and a projection plane.SOLUTION: A camera unit 103 captures a projection plane on which a projection image based on an input image is projected to form a first captured image, and captures a projection plane with no projection image projected thereon to form a second captured image. An image processing unit 110 recognizes the projection planes from the first and second captured images, respectively, and clips the projection planes as first and second projection plane images. A trapezoidal correction unit 109 corrects the shape of the input image to the shape of the projection image in the first projection plane image, to form a corrected input image. The image processing unit synthesizes the second projection plane image with the corrected input image to form a composite image.

Description

  The present invention relates to a projector device for projecting an image, a control method thereof, and a control program, and more particularly, an image (projected image) projected on a projection surface such as a whiteboard and characters and figures written on the projection surface. The present invention relates to a projector device that captures and stores such information.

  In general, when a meeting or the like is performed, an image such as a document (hereinafter also referred to as a projection image) is projected onto a projection surface such as a whiteboard using a projector device. At this time, a description of the projection image or the like may be written on the whiteboard on which the projection image is projected with a pen. In order to simultaneously store the projected image and the written board, for example, it is necessary to photograph the entire view of the whiteboard using an imaging device such as a digital camera.

  However, when taking a panoramic view of the whiteboard, there are cases where equipment in conference rooms other than the whiteboard is taken. Furthermore, the projection light from the projector device may be reflected on the whiteboard and the board can not be photographed beautifully. In addition, when shooting in a bright room, the contrast of the projected image may be lowered, and the projected image and the board may not be stored with good image quality.

  In order to preserve the projected image and the board writing with good image quality, for example, there is a projector apparatus and a screen (projection surface) for board writing integrated (see Patent Document 1).

  In Patent Document 1, a portion corresponding to a whiteboard for writing on a board is a screen, and an image is projected onto the screen by a projector device from the back side of the screen. Further, a camera is mounted on the back side of the screen, and at the time of shooting, image projection is temporarily stopped and characters and the like written on the board are shot from the back side of the screen to obtain a shot image. Then, the projected image and the photographed image are combined, and the characters written on the screen and the projected image projected by the projector device are stored.

JP 2003-143348 A

  However, in the method described in Patent Document 1, in order to perform image composition with good image quality, the positional relationship between the screen and the projector device needs to be fixedly arranged in a predetermined relationship. Cannot be used separately.

  That is, in the method described in Patent Document 1, in order to combine images with good image quality, it is necessary to fix the positional relationship between the screen and the projector device, and the screen and the projector device can be freely arranged. Can not.

  Accordingly, an object of the present invention is to provide a projector device that can perform image composition with good image quality and can freely set a positional relationship with a projection surface such as a whiteboard or a screen, a control method thereof, and a control program Is to provide.

  In order to achieve the above object, a projector device according to the present invention is a projector device that projects a projection image corresponding to an input image onto a predetermined projection surface, and the projection image is projected on the projection surface. A first photographing unit that obtains a first photographed image by photographing the projection surface; and a second photographing that obtains a second photographed image by photographing the projection surface in a state where the projection image is not projected onto the projection surface. Means, a cutout means for recognizing the projection plane from the first photographed image and the second photographed image and cutting out the projection plane as a first projection plane image and a second projection plane image, and the input A shape correction unit that corrects the shape of the image to the shape of the projection image existing in the first projection plane image to obtain a corrected input image, and the second projection plane image and the corrected input image Composite image by combining And having an image combining unit that, the.

  A control method according to the present invention is a control method for a projector apparatus that includes a projection unit that projects a projection image corresponding to an input image onto a predetermined projection plane, and a photographing unit that photographs the projection plane. A first photographing step of photographing the projection surface in a state in which the projection image is projected on the projection surface to obtain a first photographed image; and a state in which the projection image is not projected onto the projection surface by the photographing unit. A second photographing step of photographing the projection plane to obtain a second photographed image; and recognizing the projection plane from the first photographed image and the second photographed image, respectively. A cut-out step of cutting out as a projection plane image and a second projection plane image, and a shape that corrects the shape of the input image to the shape of the projection image existing in the first projection plane image to obtain a corrected input image A correction step, characterized by having a, an image combining step of the composite image to the image combining and the corrected input image and the second projection surface image.

  The control program according to the present invention is a control program used in a projector apparatus having a projection unit that projects a projection image corresponding to an input image onto a predetermined projection plane, and a photographing unit that captures the projection plane. A first imaging step of capturing a first captured image by capturing the projection plane in a state in which the projection image is projected onto the projection plane by the imaging unit; and the projection by the imaging unit. A second photographing step of obtaining a second photographed image by photographing the projection surface in a state in which the projection image is not projected on the surface, and the projection surface from the first photographed image and the second photographed image, respectively. A step of recognizing and cutting out the projection plane as a first projection plane image and a second projection plane image, and the shape of the input image as the first projection A shape correcting step for correcting the shape of the projected image present in the image to obtain a corrected input image, and an image combining step for combining the second projection plane image and the corrected input image to form a combined image Are executed.

  According to the present invention, the first photographed image obtained by photographing the projection surface with the projection image projected on the projection surface and the projection surface obtained by photographing the projection surface without projecting the projection image on the projection surface are obtained. After the projection plane is cut out from the second captured image as a first projection plane image and a second projection plane image, and the shape of the input image is corrected to the shape of the projection image existing in the first projection plane image Since the second projection plane image and the corrected input image are combined to form a composite image, a composite image with good image quality can be obtained regardless of the positional relationship between the projection plane and the projector device. it can.

It is a perspective view which shows the external appearance of the projector apparatus by embodiment of this invention. It is a block diagram which shows an example of the hardware constitutions of the projector apparatus shown in FIG. 3 is a flowchart for explaining image composition performed in the projector device shown in FIG. 2. It is a figure which shows the procedure of the image composition performed with the projector apparatus shown in FIG. It is a figure for demonstrating the trapezoid correction | amendment performed by the trapezoid correction | amendment part shown in FIG.

  Hereinafter, an example of a projector device according to an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a perspective view showing an external appearance of a projector apparatus according to an embodiment of the present invention.

  In FIG. 1, the projector device 100 has a projector housing 200, and operation buttons 201 operated by the user are arranged on the surface (upper surface in the drawing) of the projector housing 200. The operation buttons 201 include a power button for turning on / off the power and an adjustment button for focus adjustment.

  A camera opening 202 is formed on the front surface of the projector housing 200, and a projection opening 203 for projecting an image is formed. As will be described later, the lens of the camera unit is disposed in the projector housing 200 in association with the camera opening 202, and the lens of the video projection unit is disposed in association with the projection opening 203. The illustrated projector device 100 is connected to, for example, a personal computer (PC: not shown) by a cable 204.

  FIG. 2 is a block diagram illustrating an example of a hardware configuration of the projector device 100 illustrated in FIG. Hereinafter, a projector to which the present invention is applied will be described with reference to the drawings.

  The projector device 100 includes an operation unit 101, and the operation unit 101 receives a user operation using the operation buttons 201 described above. The video projection unit 102 projects a projection image corresponding to the video (input image) on a projection surface such as a whiteboard arranged outside the projector device 100. The camera unit 103 captures a projected image and a board letter drawn by the user on the whiteboard.

  The projection image management unit 104 generates an image (projection image) for projection according to a video signal (that is, an input image) input from an external device (for example, a PC). The external communication unit 105 communicates with an external device such as a PC. A calculation unit (CPU) 106 controls the entire projector device 100 based on a program. A primary storage memory (RAM) 107 is temporarily used as a work area in processing such as image composition processing. A storage memory (ROM) 108 stores a program for controlling the projector apparatus.

  As will be described later, the trapezoid correction unit 109 corrects, for example, a trapezoidal whiteboard image (projection plane image) to a predetermined shape (for example, a rectangle) and also holds an input image ( When an image received from a PC or the like is synthesized, the input image is corrected to a trapezoidal shape, for example. The image processing unit 110 recognizes a whiteboard area (that is, a projection plane) from an image (captured image) obtained as a result of shooting by the camera unit 103, and performs an input held by the projection image management unit 104 as described later. The image and the whiteboard image (that is, the whiteboard area) are synthesized.

  FIG. 3 is a flowchart for explaining image composition performed by projector apparatus 100 shown in FIG. In the following description, it is assumed that a meeting is performed while projecting a projected image onto a whiteboard using the projector device 100, and a part of the projected image overlaps the projected image or is written on the board slightly apart. To do.

  When the user combines the projected image on the whiteboard and the board and saves it as a combined image, the user operates the operation buttons 201 to notify the operation unit 101 of the start of the combining start process. As a result, the CPU 106 accepts the start of the synthesis process (step S300). When accepting the start of the synthesizing process, the CPU 106 controls the camera unit 103 to perform the first photographing and photograph the whiteboard (step S301).

  Thus, in the first shooting, shooting is performed in a state where a projection image corresponding to the input image is projected on the whiteboard.

  FIG. 4 is a diagram showing an image composition procedure performed by the projector device 100 shown in FIG.

  In step S301, when the camera unit 103 captures a whiteboard, a captured image 403 (first captured image) is obtained. The photographed image 403 includes a whiteboard (that is, a projection plane image) 400, a board drawing (board writing image) 401, and a projection image 402 as images. In the captured image 403, portions other than the whiteboard area, such as whiteboard legs, are also captured. Here, the whiteboard area refers to an area on the whiteboard 400 where a projected image is projected and a board writing 402 is performed.

  As shown in FIG. 4, here, the projector device 100, that is, the camera unit 103 is not located in front of the whiteboard area, and the whiteboard area is photographed from an oblique direction.

  Next, the trapezoid correction unit 109 and the image processing unit 110 correct the captured image 403 under the control of the CPU 106. Here, first, the image processing unit 110 recognizes a whiteboard area in the captured image 403 and cuts out the whiteboard area (projection plane) from the captured image 403. As described above, since the camera unit 103 captures the whiteboard 400 from an oblique direction, the whiteboard area has a trapezoidal shape in the captured image 403. Therefore, the trapezoidal correction unit 109 corrects the whiteboard area to a keystone and obtains a rectangular (square) projection plane image 410 (first projection plane image) (step S302). The trapezoid correction will be described later.

  When recognizing the whiteboard area, for example, the image processing unit 110 extracts a white square area using a difference in contrast ratio between the white color of the whiteboard and the other part in the captured image 403 and extracts the white square area. The square area is recognized as a whiteboard area (projection plane).

  Alternatively, markers that can be recognized by the image processing unit 110 are provided in the four corners of the whiteboard area in advance, and the image processing unit 110 detects the marker from the photographed image 403, and the area surrounded by the markers is whiteboard area (projected). Surface).

  In the projection plane image 410 (first projection plane image) after the trapezoid correction, the plate (board writing image) 411 is deformed by the trapezoid correction compared to the plate writing (board writing image) 401. The projected image 412 is also deformed by trapezoidal correction compared to the projected image 402. Here, the projection plane image 410 after the keystone correction is used as a reference image, and is used in the subsequent processing.

  Subsequently, the CPU 106 controls the video projection unit 102 to stop projecting the projection image. Then, the CPU 106 controls the camera unit 103 to perform the second shooting (step S303). A captured image 422 (second captured image) is obtained by the second capturing. The photographed image 422 includes a whiteboard 420 and a blackboard 421 as images.

  In the second shooting, since the shooting is performed in a state where the projection of the projection image is stopped, there is no projection image in the shot image 422. In addition, since the positional relationship between the projector device 100 and the whiteboard may change between the first shooting and the second shooting, in this case, in the captured image 400 and the captured image 420, the position of the whiteboard The relationship may be slightly different.

  Next, in the same manner as the processing in step S302, the image processing unit 110 cuts out a whiteboard area (projection plane) from the captured image 422, and the keystone correction unit 109 performs keystone correction on the whiteboard area (projection plane). Thus, a projection plane image 430 (second projection plane image) after trapezoidal correction is obtained (step S304).

  In addition, in the projection plane image 430 after the keystone correction, the board drawing (board writing image) 431 is deformed by the keystone correction as compared with the board writing (board writing image) 421.

  Subsequently, under the control of the CPU 106, the keystone correction unit 109 corrects the keystone so that the input image managed by the projection image management unit 104 has the same shape as the projection image 412 in the reference image (step S305). Here, assuming that the input image 440 shown in FIG. 4 is managed by the projection image management unit 104, the trapezoid correction unit 109 makes the input image 440 the same shape as the projection image 412 (its trapezoidal shape). The input image 440 is trapezoidally corrected to the corrected input image 450 shown in FIG.

  Next, under the control of the CPU 106, the image processing unit 110 synthesizes the projection plane image 430 (second projection plane image) and the corrected input image 450 to form a synthesized image 460 (step S306). The composite image 460 is sent to the PC by the external communication unit 105, for example. Then, the image processing unit 110 ends the image composition process.

  In the composite image 460, the board 461 is the same as the board 431, and the projection image 462 is the same as the input image 450 after correction. When performing image composition, the image processing unit 110 performs image processing in which a white portion of the projection plane image 430 (second projection plane image) is transmitted. Further, the image processing unit 110 performs layer composition so that the projection plane image 430 overlaps the corrected input image 450 to obtain a composite image 460.

  In the composite image 460 obtained as described above, the projection image 462 corresponding to the projection image 402 is generated from the input image 440 managed by the projection image management unit 104, so that the contrast is higher than that of the captured image 403. clarify. Further, in the captured image 403, the projection light from the projector device 100 may be reflected and reflected in the whiteboard area, but such a reflection does not occur in the composite image 460.

  Further, in the composite image 460, there are no unnecessary peripheral portions other than the whiteboard area included in the captured image 403, and a rectangular whiteboard area (that is, a whiteboard area viewed from the front) is obtained. It will be.

  Here, the keystone correction performed by the keystone correction unit 109 shown in FIG. 2 will be described.

  FIG. 5 is a diagram for explaining the trapezoid correction performed by the trapezoid correction unit 109 shown in FIG.

  When performing keystone correction, projective transformation (also called mapping transformation) is used. Assume that the figure (rectangle) 501 is corrected to the figure (rectangle) 502. When the coordinates of the vertex A of the figure 501 are (X, Y) and the coordinates of the vertex A ′ of the figure 502 are (x, y), the vertex A is expressed by the following expressions (1) and (2). Is projectively transformed.

x = (A1 * X + B1 * Y + C1) / (A3 * X + B3 * Y + 1) (1)
y = (A2 * X + B2 * Y + C2) / (A3 * X + B3 * Y + 1) (2)
Since there are eight parameters A1, A2, B1, B2, C1, C2, A3, and B3 in Expression (1) and Expression (2), the coordinates of the vertexes A to D and the vertices A ′ to D ′ are as follows. By generating simultaneous equations and solving these simultaneous equations, the figure 501 can be corrected to the figure 502. Using this projective transformation, any quadrangle including a trapezoid can be corrected to a rectangle.

  As described above, in the projector device 100 shown in FIG. 2, the projection image corresponding to the input image sent from the external device is projected onto the whiteboard region to perform the first shooting, and the whiteboard region (projection plane) Perform keystone correction. Subsequently, the projection of the projection image is stopped, the second shooting is performed, and the keystone correction of the whiteboard area is performed. Then, after correcting the input image to the same shape as the projected image in the captured image obtained by the first shooting, the corrected projected image and the whiteboard region (projection plane) by the second shooting are obtained. Compositing into a composite image. As a result, regardless of the positional relationship between the projector device 100 and the whiteboard, it is possible to perform image composition with good image quality.

  By the way, the board 431 present in the photographed image 430 is obtained as a result of photographing by the camera unit 103. For this reason, when a room such as a conference room is dim, the contrast between the whiteboard area and the board letter may not be clear. Therefore, when the composite image 460 is generated, the image processing unit 110 performs image processing such as increasing the contrast of the board 431 to obtain the board 461.

  By performing image processing such as increasing the contrast of the board writing in this way, the color of the board letter 461 can be made clear in the synthesized image 460, and the image quality of the synthesized image 460 is improved.

  Furthermore, when the projection image 462 and the board writing 461 overlap in the composite image 460, the projection image 462 and the board writing 461 may be difficult to see. That is, since the projection image 462 is generated from the input image 440 managed by the projection image management unit 104, if the input image 440 includes black or the like, it may be difficult to see if the blackboard 461 and the projection image 462 overlap.

  Therefore, here, the image processing unit 110 performs a white border on the outline of the board letter for the board 431. For example, the image processing unit 110 sets the white portion of the whiteboard area to a transparent color and the board writing portion to an opaque color. Then, the image processing unit 110 arranges white pixels at the boundary between the transparent color and the opaque color. Thereafter, the image processing unit 110 synthesizes the projection plane image 430 that has undergone the edging process on the input image 450 after correction, and generates a synthesized image 460 in which the board letter is outlined.

  As described above, in the embodiment of the present invention, it is possible to always obtain a composite image with good image quality without being bothered by the positional relationship between the projector device and the whiteboard.

  As is clear from the above description, in the example shown in FIG. 2, the CPU 106 and the camera unit 103 function as a first photographing unit and a second photographing unit, and the CPU 106 and the image processing unit 110 are a clipping unit and image composition. Functions as a means. The CPU 106 and the trapezoid correction unit 109 function as a shape correction unit.

  Further, the CPU 106 and the image processing unit 110 function as a contrast adjusting unit and an edging unit. The CPU 106 and the video projection unit 102 function as projection means.

  As mentioned above, although this invention was demonstrated based on embodiment, this invention is not limited to these embodiment, Various forms of the range which does not deviate from the summary of this invention are also contained in this invention. .

  For example, the function of the above embodiment may be used as a control method, and this control method may be executed by the projector apparatus. Further, a program having the functions of the above-described embodiments may be used as a control program, and the control program may be executed by a computer included in the projector apparatus. The control program is recorded on a computer-readable recording medium, for example.

  Each of the above control method and control program has at least a first photographing step, a second photographing step, a cutout step, a shape correction step, and an image composition step.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various recording media, and the computer (or CPU, MPU, etc.) of the system or apparatus reads the program. To be executed.

DESCRIPTION OF SYMBOLS 101 Operation part 102 Image | video projection part 103 Camera part 104 Projection image management part 105 External communication part 106 Operation part (CPU)
107 Primary storage memory (RAM)
108 Storage memory (ROM)
109 Keystone correction unit 110 Image processing unit

Claims (11)

A projector device that projects a projection image corresponding to an input image onto a predetermined projection plane,
First imaging means for capturing the projection plane in a state where the projection image is projected on the projection plane to obtain a first captured image;
Second imaging means for capturing the projection plane in a state where the projection image is not projected on the projection plane to obtain a second captured image;
Clipping means for recognizing the projection plane from the first photographed image and the second photographed image and cutting out the projection plane as a first projection plane image and a second projection plane image;
A shape correction unit that corrects the shape of the input image to the shape of the projection image present in the first projection plane image to obtain a corrected input image;
Image synthesizing means for synthesizing the second projection plane image and the corrected input image into a synthesized image;
A projector apparatus comprising:   The shape correcting unit corrects the shapes of the first projection plane image and the second projection plane image to predetermined shapes, respectively, and the projection image existing in the first projection plane image whose shape is corrected The projector apparatus according to claim 1, wherein the shape of the input image is corrected to the shape of the input image.   3. The projector device according to claim 2, wherein the image synthesizing unit synthesizes the second projection plane image corrected to the predetermined shape and the corrected input image.   The projector device according to claim 2, wherein the predetermined shape is a rectangle.   The projector apparatus according to claim 1, wherein the cutout unit recognizes the projection plane according to a marker provided in advance on the projection plane.   The cutout unit recognizes the projection plane according to the color of the projection plane in each of the first photographed image and the second photographed image. The projector apparatus described in 1.   The projector device according to claim 1, wherein the image composition unit performs the image composition using a predetermined color portion as a transmission color in the second projection plane image.   2. The apparatus according to claim 1, further comprising a contrast adjusting unit configured to change a contrast between the board writing and a region excluding the board writing in the second projection plane image when the board writing is performed on the projection plane. 8. The projector device according to any one of items 7.   8. The image forming apparatus according to claim 1, further comprising an edging unit configured to perform edging with a predetermined color on an outline of the board writing in the second projection surface image when the writing is performed on the projection surface. The projector device according to any one of the above. A control method for a projector apparatus, comprising: a projection unit that projects a projection image corresponding to an input image onto a predetermined projection plane; and a photographing unit that captures the projection plane,
A first photographing step of obtaining a first photographed image by photographing the projection surface in a state where the projection image is projected onto the projection surface by the photographing means;
A second photographing step of obtaining a second photographed image by photographing the projection surface in a state where the projection image is not projected onto the projection surface by the photographing means;
A step of cutting out the projection plane as a first projection plane image and a second projection plane image by recognizing the projection plane from the first captured image and the second captured image, respectively.
A shape correction step of correcting the shape of the input image to the shape of the projection image present in the first projection plane image to obtain a corrected input image;
An image synthesis step of synthesizing the second projection plane image and the corrected input image into a synthesized image;
A control method characterized by comprising: A control program used in a projector apparatus having a projection unit that projects a projection image corresponding to an input image onto a predetermined projection plane, and a photographing unit that captures the projection plane,
In the computer provided in the projector device,
A first photographing step of obtaining a first photographed image by photographing the projection surface in a state where the projection image is projected onto the projection surface by the photographing means;
A second photographing step of obtaining a second photographed image by photographing the projection surface in a state where the projection image is not projected onto the projection surface by the photographing means;
A step of cutting out the projection plane as a first projection plane image and a second projection plane image by recognizing the projection plane from the first captured image and the second captured image, respectively.
A shape correction step of correcting the shape of the input image to the shape of the projection image present in the first projection plane image to obtain a corrected input image;
An image synthesis step of synthesizing the second projection plane image and the corrected input image into a synthesized image;
A control program characterized by causing

Images