JP2007243707A - Image separation unit - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image separation unit capable of obtaining image data of a background image by eliminating a projected image from a background screen affected by the projected image, and reproducing only the projected image data. <P>SOLUTION: A projection means 12 projects each primary color component successively and periodically at high speed after resolving image information into a plurality of primary color components. An imaging means 16 images an image including the background screen affected by the projection in synchronization with the projection using each sensor provided correspondingly to each primary color. Based on an algorithm using a correlation function among a plurality of the sensors to each primary color which is predetermined in the projection means 12 and the imaging means 16, a calculation processing means 17 performs the calculation processing of outputs of the primary color component and one or more primary colors other than the affected primary color from the image output corresponding to each primary color so as to obtain either image data of only the background screen unaffected by the projection, or information data of the image excluding the background screen. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image separation apparatus capable of separating image data of a background screen equivalent to a state without a projection image on a background screen affected by the projection image, or separating projection image data from a synthesized image. . Furthermore, the present invention relates to an image separation apparatus that enables image separation even in a normal environment where it is not possible to use a primary color projection without an overlapping portion or a sensor that does not overlap and can accurately output only the output corresponding to the primary color. .

There is a chroma key system that separates a background from a non-background (object), which uses a specific color for the background or changes the color of the background to separate the non-background (object).
The invention described in Patent Document 1 (chroma key system) discloses that a color that changes by changing the color of the background is identified as the background, and the background and the non-background are separated.
The invention described in Patent Document 2 (object image clipping method, apparatus, system, and medium on which the program is recorded) takes the logical sum of object regions (chroma key object region clipping) that are clipped from object images in two different background colors. Thus, it is disclosed that an object image can be cut out.
The invention (image separation device) described in Patent Document 3 discloses that a projected image projected by a projector or the like can be removed to obtain a real image when there is no projected image. It cannot be removed.
Japanese Patent No. 3241327 JP 2000-152278 A JP 2005-182413 A

Even if an image that has been separated into RGB colors by a projector or the like and sequentially projected at a high speed is captured by a high-speed camera synchronized with each color of the projector, the spectral characteristics of the color filters of the projector and the image sensor of the camera are wide, and RGB Since there are portions where the spectral characteristics of the primary color components overlap each other, the camera sensor also captures GB components other than the R component during R projection. Similarly, RB components other than the G component are imaged during G projection, and RG components other than the B component are also imaged during B projection.
In this way, since primary color components other than the primary color that is originally projected are picked up by the camera, the projected image projected onto the space cannot be completely removed by a method such as simple color difference or composition.

The inventors of the present invention have developed a color synthesis algorithm on the premise that there are primary color components other than the primary color that is originally projected, and this method can remove a projected image projected in space. Furthermore, by removing the background image (background image without the projected image viewed from the imaging side) from the captured image, it is possible to obtain only the image projected on the background (separating the background image and the projected image). Focused on doing so.
An object of the present invention is to obtain an image data of a background image by completely removing the projection image from a background screen affected by the projection image, or to reproduce only the data of the projection image Is to provide.

In order to achieve the object, an image separating apparatus according to claim 1 according to the present invention is provided.
Projecting means for decomposing image information into a plurality of primary color components and sequentially projecting each primary color component on the background screen at high speed sequentially,
Imaging means for capturing an image including the background screen affected by the projection in synchronization with the projection by a sensor corresponding to each primary color;
Based on an algorithm using a correlation function between the plurality of sensors for each primary color determined in advance in the projection unit and the imaging unit,
By calculating the output of the primary color component and one or more primary color components other than the affected primary color component from the image output corresponding to each primary color, image data of only the background screen not affected by the projection Alternatively, it is constituted by arithmetic processing means for obtaining information data of an image that does not include a background screen.

The image separation device according to claim 2 of the present invention is the image separation device according to claim 1,
The information on the image projected by the projection unit includes a moving image, and the imaging unit is configured to capture an image from the foreground of the projected image.
According to the present invention, there is provided an image separating apparatus according to claim 3.
3. The image separation device according to claim 1, wherein the algorithm uses an image projected in space using one or two primary color components other than the primary color component for a captured image corresponding to each primary color, respectively. To be estimated.

  According to a fourth aspect of the present invention, there is provided the image separation device according to the first, second, or third aspect, wherein the algorithm has a second or higher order correlation function that defines the image data to be separated. If it is a product and the influence is extremely small, an approximate expression in which the calculation of the term is omitted is used.

The image separation device according to the present invention is a person who has no projection image even in a scene such as a projector projected from the foreground (non-background) side or a scene projected from the background side onto a screen or the like in a lecture or the like. An image can be obtained. Therefore, the speaker can be easily tracked by the image from which the projection image is removed, and the image of only the speaker from which the projection image is removed can be recorded.
In addition, the influence of the projected image when analyzing the gesture in a scene where the screen projected by the projector is operated interactively with the gesture, etc. (the influence of the projected image projected by the projector, etc., and the moving image in the image) Can not receive.
Furthermore, it is possible to obtain only the measurement image projected on the object by projecting the measurement image onto the object and removing the object image from the captured image, so that it is not affected by the surface color or background of the object. The shape of the object can be measured.

Embodiments of an image separation device according to the present invention will be described below in more detail with reference to the drawings.
(First embodiment)
FIG. 1 shows a schematic configuration of a first embodiment of an image separation apparatus according to the present invention, and FIG. 2 shows a schematic configuration of arithmetic processing means. In FIG. 1, an input image 11 that is an image to be projected onto a projector 12 that is a projection unit is input. As the projector, for example, a single plate digital mirror type DLP projector can be used. The projector 12 divides the primary color component into a plurality of primary color components from the foreground to the space, and sequentially projects each primary color component sequentially at a high speed. In the following description, it is assumed that the projector 12 is divided into RGB primary color components and projects from the foreground of the space.

  A synchronization signal is output from the projector 12 in synchronism with the projection of each primary color component color (RGB color) of the projector 12 which is the projection means. An image is picked up by the camera 16 which is an image pickup means in accordance with an image pickup control signal generated by the arithmetic processing means 17 based on the synchronization signal of the projector 12. The camera 16 is installed so as to capture images from the foreground side projected by the projector 12 toward the space. At this time, the image 15 of the space viewed from the camera 16 is combined with the mixed image 15 in which the projection image 14 projected from the projector 12 is superimposed on the space image 13 viewed from the foreground without projection from the projector 12. Become. Since the image of the space is below the image 14 projected from the projector 12, it becomes the background image 13 viewed from the foreground. An image captured by the camera 16 is input to the arithmetic processing means 17 together with a synchronization signal for projection of the primary color components from the projector 12.

In the arithmetic processing means 17, as shown in FIG. 2, the imaging control signal for the camera 16 of FIG. 1 and the input processing I / F unit 21 of FIG. 2 are based on the synchronization signal input by the synchronization control unit 26. An image input control signal and an arithmetic processing control signal for the arithmetic processing unit 24 are generated. In the input processing I / F unit 21, image data corresponding to each primary color captured by the camera 16 in accordance with the imaging control signal and sequentially input is temporarily stored in the image memory bus according to the image input control signal generated by the synchronization control unit 26. The image data is stored in the image memory 23 via 22. In the arithmetic processing unit 24 shown in the figure, the primary color component other than the primary color component affected by the primary color component from the image memory 23 via the image memory bus 22 according to the arithmetic processing control signal generated by the synchronization control unit 26 is shown. One or more primary color components are read out, and using this data, the primary color component of only the projected image projected on the space and the primary color component affected by this are calculated using the algorithm of the present invention. And stored in the image memory 23.
This result is subtracted from the captured image corresponding to each primary color, and an equivalent background image that is not affected by the projection image is obtained for each primary color by the arithmetic processing unit 24 and stored in the image memory 23. Further, this result is subtracted from the captured image corresponding to each primary color, and each primary color component of the projected image projected onto the space is obtained by the arithmetic processing unit 24 and stored in the image memory 23. The result is added for one period for each primary color component, and a projection image projected on the space (an image obtained by removing the background image from the captured image) is obtained by the arithmetic processing unit 24 and stored in the image memory 23.

  An input image picked up for each primary color is read from the image memory 23 and added for one period to obtain an image equivalent to what is seen with eyes (a normal visible image, here referred to as a normal image). It can also be stored in the image memory 23. The normal image obtained as a result of the operation, the projected image projected on the space, and the background image (foreground image) not affected by the projected image are selected by the image selection information such as the operator 25 or individually. 1 is output from the arithmetic processing unit 24 in synchronization with the arithmetic processing control signal read from the image memory 23 and generated by the synchronization control unit 26, and is input to the output means 18 in FIG.

  The output means 18 is a display device that uses an input image, and displays the input image on a display or the like. As an output means, an image recording apparatus that records an input image, a tracking apparatus that tracks a person using the input image, a gesture analysis apparatus that analyzes a person's gesture from the input image, and an input image A measuring device for measuring an object can be used.

The algorithm of the arithmetic processing unit 24 of FIG. 2 in the arithmetic processing means 17 of FIG. 1 will be described below.
First, a basic code is defined.
-Let RGB values of an image (background image) obtained by capturing an image of the original space (a scene not projected from the projection device) with a camera be [Rs, Gs, Bs].
・ When processing to remove a spatial image (scenery including a video projected from a projection device) from an image obtained by capturing a composite image (scenery including a video projected from the projection device) with a camera, projection is performed from the projection device. Only the processed image remains. Let the RGB values of this image be [Rp, Gp, Bp].
-Let RGB values of an image obtained by capturing a composite image (a scene including an image projected from a projection device) with a camera be [(Rs + Rp), (Gs + Gp), (Bs + Bp)].

[The purpose of the algorithm]
Even if [(Rs + Rp), (Gs + Gp), (Bs + Bp)] is obtained by capturing the situation in which the image is projected from the projector, [Rs, Gs, Bs ] And [Rp, Gp, Bp] cannot be estimated. Even if an image that has been separated into RGB colors by a projector or the like and is sequentially projected at a high speed is captured by a high-speed camera that synchronizes with each color of the projector, the characteristics of the color filters of the projector and the image sensor of the camera are wide. In the R projection, GB components other than the R component are also imaged. Similarly, RB components other than the G component are also imaged during G projection, and RG components other than the B component are also imaged during B projection. Therefore, in a method such as simple color difference or composition, [Rs, Gs, Bs] And [Rp, Gp, Bp] cannot be estimated.
In the present invention, from the image output corresponding to each primary color, using the primary color component and one or more primary color components other than the primary color component, an image projected on the space equivalent to an unprojected state and an image projected onto the space It is necessary to use an algorithm as a basis for estimating each of the above.

[Method definition]
Using the GB component information at the time of R projection, similarly, using the RB component information at the time of G projection, and using the RG component information at the time of B projection, [Rs, Gs, Bs] and [Rp, Gp, Bp] respectively Make it estimable.
Method 1. Method 1 is a method for estimating an image projected in space (with the background removed) using one primary color component other than the primary color component for each captured image corresponding to the primary color.
Method 2. Method 2 is a method for estimating an image projected in space (with the background removed) using two primary color components other than the primary color component for each primary color-captured image.
Method 3. A method using an approximate expression with respect to the arithmetic expression in Method 1 is referred to as Method 3.
Method 4. A method using an approximate expression with respect to the arithmetic expression in Method 2 is referred to as Method 4.

[Definition of sensor output of imaging means]
Rr: Rr is the value of the sensor that detects the red color of the camera in the red component output from the projector.
Gr: Gr is the value of the sensor that senses the green color of the camera in the red component output from the projector.
Br: Br is the value of the sensor that detects the blue color of the camera in the red component output from the projector.
Rg: Rg is the value of the sensor that detects the red color of the camera in the green component output from the projector.
Gg: Gg is the value of the sensor that detects the green color of the camera in the green component output from the projector.
Bg: Bg is the value of the sensor that detects the blue color of the camera in the green component output from the projector.
Rb: Rb is the value of the sensor that detects the red color of the camera in the blue component output from the projector.
Gb: Gb is the value of the sensor that detects the green color of the camera in the blue component output from the projector.
Bb: The value of the sensor that detects the blue color of the camera in the blue component output from the projector is Bb.

There are three types of images projected from the projection device: an image with only a red component, an image with only a green component, and an image with only a blue component. By repeatedly projecting these three types of images at high speed, a human eye can see a color image.
When you take a picture of the scenery including the image projected from the projector and remove the background image,
[Rr, Gr, Br] when projecting only the red component,
[Rg, Gg, Bg] when projecting only the green component,
[Rb, Gb, Bb] when projecting only the blue component.
A captured image (image of a composite image) including a projected image, [Rp, Gp, Bp] can be expressed by the following expression.
[Rp, Gp, Bp] = [(Rr + Rg + Rb), (Gr + Gg + Gb), (Br + Bg + Bb)] Equation 1
Furthermore, when taking a picture of the scenery including the image projected from the projection device with the camera,
When projecting only the first red component
[R ₁ , G ₁ , B ₁ ] = [(Rs + Rr), (Gs + Gr), (Bs + Br)] Equation 2
When projecting only the second green component
[R ₂ , G ₂ , B ₂ ] = [(Rs + Rg), (Gs + Gg), (Bs + Bg)]
When projecting only the third blue component,
[R ₃ , G ₃ , B ₃ ] = [(Rs + Rb), (Gs + Gb), (Bs + Bb)]
Respectively.

(Image Separation by Method 1) The primary color component and one primary color component other than the primary color component are used.
This is based on a method of extracting one color image from three RGB three-color image data and synthesizing the three color images. In this method, Rp is estimated from either of the GB components at the time of R projection, similarly, Gp is estimated from any of the BR components at the time of G projection, and Bp is estimated from either of the RG components at the time of B projection. .
Here, the following correlation is established for each color pixel onto which each color is projected. These x, y, and z are caused by the color filter characteristics of the projector and the image sensor, and are uniquely determined when the combination of the projection unit and the imaging unit is determined, and can be determined in advance. . In the present invention, these x, y, and z are called correlation functions.
Br = x × Rr ・ ・ ・ ・ ・ ・ Formula 5
Rg = y × Gg ・ ・ ・ ・ ・ ・ Formula 6
Gb = z × Bb (7)
Even if Br, Rg, and Gb in the above equation are replaced with Gr, Bg, and Rb, the following algorithm holds.

(Procedure 1) Calculate predetermined x, y, z and prepare for use.
(Procedure 2) Three images are captured in synchronization with the projection color order of the projection device.
In synchronism with the R, G, and B projections from the projection device, three scenes (synthesized images) including the projected images are captured by the camera to obtain exceptional image data.
These _{images, [R 1, G 1,} B 1], a _{[R 2, G 2, B} 2], [R 3, G 3, B 3], previously shown in Equation 2-4.
(Procedure 3) [Rs, Gs, Bs] is estimated by calculation using three captured images.
For each RGB color, the following relationship can be obtained from one of a set of two images including the projection color of the three images.
R ₁ -R ₂ = Rr-Rg = u Equation 8
G ₂ -G ₃ = Gg-Gb = v Equation 9
B ₃ -B ₁ = Bb-Br = w
From the equations 5 to 10, the following relationship is obtained from the solution of the ternary linear equation.
Rr = (u + yv + yzw) / (1-xyz) ··· Equation 11
Br = x (u + yv + yzw) / (1-xyz) ··· Equation 12
Rg = y (v + zw + xzu) / (1-xyz) ··· Equation 13
Gg = (v + zw + xzu) / (1-xyz) ··· Equation 14
Gb = z (w + xu + xyv) / (1-xyz)
Bb = (w + xu + xyv) / (1-xyz)
Rr, Br, Rg, Gg, Gb, and Bb are obtained by calculating the relational expressions of the above expressions 11 to 16, and further using expressions 2 to 4,
Rs = R ₂ -Rg = R ₁ -Rr
Gs = G ₃ -Gb = G ₂ -Gg
Bs = B ₁ -Br = B ₃ -Bb
To get [Rs, Gs, Bs] (background image only).

(Procedure 4) An image projected in space is estimated.
Using [Rs, Gs, Bs] obtained in Procedure 3 and Equations 2 to 4, the image projected on the space can be estimated by performing the following calculation with the captured image.
[R ₁ , G ₁ , B ₁ ]-[Rs, Gs, Bs] = [Rr, Gr, Br]
[R ₂ , G ₂ , B ₂ ]-[Rs, Gs, Bs] = [Rg, Gg, Bg]
[R ₃ , G ₃ , B ₃ ]-[Rs, Gs, Bs] = [Rb, Gb, Bb]
[Rr, Gg, Bb] is obtained only for the estimated image projected on the space by synthesizing one image using Rr, Gg, Bb obtained by the above equation.
Also, an image obtained by removing the background image from the captured image by obtaining Rp, Gp, Bp using Equation 1
Obtain [Rp, Gp, Bp].
The arithmetic elements in this method can be composed of an adder, a multiplier, a subtracter, and a divider.

(Image Separation by Method 2) The primary color component and two primary color components other than the primary color component are used.
According to a method of extracting two color images from three RGB three-color image data and synthesizing the three color images. This method is a method for estimating Rp from both of the GB components at the time of R projection, similarly, Gp from both of the BR components at the time of G projection, and Bp from both of the RG components at the time of B projection.
Here, the following correlation is established for each color pixel onto which each color is projected. As described above, x ₁ , x ₂ , y ₁ , y ₂ , z ₁ , z ₂ are unique to the combination resulting from the color filter characteristics of the projector and the image sensor, and are called correlation functions in the present invention. .
Gr = x ₁ × Rr ・ ・ ・ ・ ・ ・ Equation 17
Br = x ₂ × Rr ・ ・ ・ ・ ・ ・ Equation 18
Rg = y ₁ × Gg Equation 19
Bg = y ₂ × Gg Equation 20
Rb = z ₁ × Bb ・ ・ ・ ・ ・ ・ Formula 21
Gb = z ₂ × Bb ・ ・ ・ ・ ・ ・ Formula 22

(Procedure 1) Based on the characteristics of the filter, the values x ₁ , x ₂ , y ₁ , y ₂ , z ₁ , z ₂ that have been prepared in advance are used.
(Procedure 2) Three images are captured in synchronization with the projection color order of the projection device.
In synchronism with the R, G, and B projections from the projection device, an image obtained by capturing three scenes including the projection image with the camera is obtained. These images are [R ₁ , G ₁ , B ₁ ], [R ₂ , G ₂ , B ₂ ], [R ₃ , G ₃ , B ₃ ] and have already been shown in equations 2-4 .
(Procedure 3) [Rs, Gs, Bs] is estimated by calculation using three captured images.
For each of the RGB colors, the following relationship can be obtained from two sets of two images including the projection color of the three images.
R ₁ -R ₂ = Rr-Rg = u ₁ ··· Equation 23
R ₁ -R ₃ = Rr-Rb = u ₂ ··· Equation 24
G ₂ -G ₁ = Gg-Gr = v ₁ ··· Equation 25
G ₂ -G ₃ = Gg-Gb = v ₂ ··· Equation 26
B ₃ -B ₁ = Bb-Br = w ₁
B ₃ -B ₂ = Bb-Bg = w ₂
From the equations 17-22, the following relationship is obtained from the solution of the 6-ary linear equation.
Rr = (y ₁ v ₁ + u ₁ ) / (1-x ₁ y ₁ ) = (z ₁ w ₁ + u ₂ ) / (1-x ₂ z ₁ ) ... Equation 29
Gr = x ₁ (y ₁ v ₁ + u ₁ ) / (1-x ₁ y ₁ ) = x ₁ (z ₁ w ₁ + u ₂ ) / (1-x ₂ z ₁ ) ... Equation 30
Br = x ₂ (y ₁ v ₁ + u ₁ ) / (1-x ₁ y ₁ ) = x ₂ (z ₁ w ₁ + u ₂ ) / (1-x ₂ z ₁ ) ... Equation 31
Rg = y ₁ (x ₁ u ₁ + v ₁ ) / (1-x ₁ y ₁ ) = y ₁ (z ₂ w ₂ + v ₂ ) / (1-y ₂ z ₂ ) ... Equation 32
Gg = (x ₁ u ₁ + v ₁ ) / (1-x ₁ y ₁ ) = (z ₂ w ₂ + v ₂ ) / (1-y ₂ z ₂ )
Bg = y ₂ (x ₁ u ₁ + v ₁ ) / (1-x ₁ y ₁ ) = y ₂ (z ₂ w ₂ + v ₂ ) / (1-y ₂ z ₂ ) ... Formula 34
Rb = z ₁ (x ₂ u ₂ + w ₁ ) / (1-x ₂ z ₁ ) = z ₁ (y ₂ v ₂ + w ₂ ) / (1-y ₂ z ₂ ) ... Formula 35
Gb = z ₂ (x ₂ u ₂ + w ₁ ) / (1-x ₂ z ₁ ) = z ₂ (y ₂ v ₂ + w ₂ ) / (1-y ₂ z ₂ ) ... Equation 36
Bb = (x ₂ u ₂ + w ₁ ) / (1-x ₂ z ₁ ) = (y ₂ v ₂ + w ₂ ) / (1-y ₂ z ₂ ) ... Equation 37
Rr, Gr, Br, Rg, Gg, Bg, Rb, Gb, Bb are obtained by calculating the relational expressions of the above equations 29 to 37, and further using equations 2 to 4,
[R ₁ , G ₁ , B ₁ ]-[Rr, Gr, Br] = [Rs, Gs, Bs]
[R ₂ , G ₂ , B ₂ ]-[Rg, Gg, Bg] = [Rs, Gs, Bs]
[R ₃ , G ₃ , B ₃ ]-[Rb, Gb, Bb] = [Rs, Gs, Bs]
To get [Rs, Gs, Bs] (background image only).

(Procedure 4) An image projected in space is estimated.
Using [Rs, Gs, Bs] obtained in Procedure 3 and Equations 2 to 4, the image projected on the space can be estimated by performing the following calculation with the captured image.
[R ₁ , G ₁ , B ₁ ]-[Rs, Gs, Bs] = [Rr, Gr, Br]
[R ₂ , G ₂ , B ₂ ]-[Rs, Gs, Bs] = [Rg, Gg, Bg]
[R ₃ , G ₃ , B ₃ ]-[Rs, Gs, Bs] = [Rb, Gb, Bb]
[Rr, Gg, Bb] is obtained only for the estimated image projected on the space by synthesizing one image using Rr, Gg, Bb obtained by the above equation.
Also, an image obtained by removing the background image from the captured image by obtaining Rp, Gp, Bp using Equation 1
Obtain [Rp, Gp, Bp].
The arithmetic elements in this method can be composed of an adder, a multiplier, a subtracter, and a divider.

(Image Separation by Method 3) Image separation is performed using an approximate expression for the arithmetic expression of Method 1.
This is a method of simply obtaining the equations 11 to 16 in the procedure 3 of the method 1 using approximate equations.
Depending on the values of u, v, w, x, y, and z, and the product terms that combine them, you can omit negligible product terms or use methods such as Taylor expansion. An approximate expression can be obtained.
For example, in the case of x, y, z << 1, the product terms of xy, xz, yz, and xyz can be ignored, so that the equations 11 to 16 are as follows.
Rr ≒ u + yv
Br ≒ xu ・ ・ ・ ・ ・ ・ Equation 39
Rg ≒ yv ・ ・ ・ ・ ・ ・ Formula 40
Gg ≒ v + zw ・ ・ ・ ・ ・ ・ Formula 41
Gb ≒ zw ・ ・ ・ ・ ・ ・ Formula 42
Bb ≒ w + xu ・ ・ ・ ・ ・ ・ Formula 43
Therefore, in this method, the equations 11 to 16 in the procedure 3 of the method 1 are replaced with the above equations 38 to 43. By using the approximate expressions of the above expressions 38 to 43, the calculation elements in this method can be configured by only an adder, a multiplier and a subtractor. In addition, since the number of terms and the number of addition / subtraction / division are reduced in the calculation formula, the circuit scale corresponding to the calculation formula can also be reduced as compared with the method 1.

(Image Separation by Method 4) Image separation is performed using an approximate expression for the arithmetic expression of Method 2.
This is a method of simply obtaining the equations 29 to 37 in the procedure 3 of the method 2 using approximate equations.
Approximate expression Determination of the combination of these and each value of _{u 1, u 2, v 1} , v 2, w 1, w 2, x 1, x 2, y 1, y 2, z 1, z 2 Depending on the value of the product term, an approximate expression can be obtained by omitting negligible product terms or using a method such as Taylor expansion.
For example, if x ₁ , x ₂ , y ₁ , y ₂ , z ₁ , z ₂ << 1, then the second and higher order product terms that combine them can be ignored, so equations 29 to 37 are as follows: become.
Rr ≒ y ₁ v ₁ + u ₁ ≒ z ₁ w ₁ + u ₂
Gr ≒ x ₁ u ₁ ≒ x ₁ u ₂
Br ≒ x ₂ u ₁ ≒ x ₂ u ₂・ ・ ・ ・ ・ ・ Equation 46
Rg ≒ y ₁ v ₁ ≒ y ₁ v ₂
Gg ≒ x ₁ u ₁ + v ₁ ≒ z ₂ w ₂ + v ₂
Bg ≒ y ₂ v ₁ ≒ y ₂ v ₂
Rb ≒ z ₁ w ₁ ≒ z ₁ w ₂ ··· Formula 50
Gb ≒ z ₂ w ₁ ≒ z ₂ w _{2
Bb ≒ x 2 u 2 + w} 1 ≒ y 2 v 2 + w 2 ······ formula 52
Therefore, in this method, the equations 29 to 37 in the procedure 3 of the method 2 are replaced with the above equations 44 to 52. By using the approximate expressions of the above expressions 44 to 52, the calculation elements in this method can be configured by only an adder, a multiplier and a subtractor. In addition, since the number of terms and the number of addition / subtraction / multiplication / division in the calculation formula are reduced, the circuit scale corresponding to the calculation formula can also be reduced as compared with the method 2.

(Second Embodiment)
Next, a second embodiment in which an image is projected from the background side of the space and imaged from the foreground side will be described with reference to FIG.
In the figure, an input image 31 that is an image to be projected is input to a projector 32 that is a projection means. As the projector, for example, a single-plate digital mirror type DLP projector can be used as described above. The projector 32 separates the primary color components on the screen 39 from the background side, and sequentially projects each primary color component sequentially at a high speed. In the same manner as described above, the projector 32 will be described as being decomposed into RGB primary color components and projected from the background side of the screen.

  A synchronization signal is output from the projector 32 in synchronism with the projection of the colors of the primary color components (RGB colors) of the projector 32 that is the projection means. An image is picked up by the camera 36 as an image pickup means in accordance with an image pickup control signal generated by the arithmetic processing means 37 based on the synchronization signal of the projector 32. The camera 36 is installed so as to capture an image from the foreground side of the screen opposite to the background side projected by the projector 32 onto the screen 39 toward the screen 39. At this time, the image 35 of the space viewed from the camera 36 is mixed with the mixed image 35 in which the projection image 34 projected from the projector 32 is superimposed on the space image 33 viewed from the foreground without projection from the projector 32. Become. Since the image of the space is above the image 34 projected from the projector 32, it becomes the background image 33 viewed from the foreground. An image picked up by the camera 36 is input to the arithmetic processing means 37 together with a sync signal for projection of the primary color components from the projector 32.

The arithmetic processing means 37 is the same as the function / operation in FIG. 2 already shown, and the output of the arithmetic processing means 37 is input to the output means 38 as described above. Further, the calculation algorithm in the calculation processing means 37 is the same as described above.
The output means 38 is a display device that uses an input image, and displays the input image on a display or the like. Examples of the output means include the same devices as described above.

(Application example of the image separation device of the present invention)
・ In a scene that is projected to the speaker from the foreground (non-background) side by a projector or a scene that is projected from the background side to a screen, etc., a background (foreground) image obtained by removing the projected image of this device It can be used to track speakers who are not affected by the projected image, and can record the images of speakers who are not affected by the projected image.
-In a scene where the screen projected by the projector is operated interactively with gestures, etc., the background (foreground) image from which the projection image of this device has been removed is used, and the influence of the projection image (especially the moving image in the projection image, etc.) It is possible to analyze a gesture of an operation state of a hand or the like that has not been affected.
・ Measurement such as grid points and grid lines as the input image of this device in a situation where a measurement image is projected onto an object and the shape of the object is measured (only the projected measurement image is obtained and measured) Since only the measurement image projected on the object can be obtained by using the image for the object, the shape of the object can be measured by analyzing the image that is not affected by the surface color or background of the object.

  The present invention can be widely used in the field of image processing. That is, it can be used for image separation requests related to the display of images in the field of creation of movies and other video-related copyrighted works, and in the field of conferences and entertainment. The present invention can be used for manufacturing the image separation device in the field of manufacturing projectors and imaging devices.

1 is a block diagram showing a schematic configuration of a first embodiment of an image separation device according to the present invention. It is a block diagram which shows schematic structure of the arithmetic processing means of the said image separation apparatus. It is a block diagram which shows schematic structure of 2nd Embodiment of the image separation apparatus by this invention.

Explanation of symbols

11, 31 Input image 12, 32 Projector 13, 33 Background image 14, 34 Projected image 15, 35 Mixed image 16, 36 Camera 17, 37 Arithmetic processing means 18, 38 Output means 21 Input processing I / F unit 22 For image memory Bus 23 Image memory 24 Arithmetic processing unit 25 Operator 26 Synchronous control unit 39 Screen

Claims (4)

Projecting means for decomposing image information into a plurality of primary color components and sequentially projecting each primary color component on the background screen at high speed sequentially,
Imaging means for capturing an image including the background screen affected by the projection in synchronization with the projection by a sensor corresponding to each primary color;
Based on an algorithm using a correlation function between the plurality of sensors for each primary color determined in advance in the projection unit and the imaging unit,
By calculating the output of the primary color component and one or more primary color components other than the affected primary color component from the image output corresponding to each primary color, image data of only the background screen not affected by the projection Alternatively, an image separating apparatus constituted by arithmetic processing means for obtaining information data of an image not including a background screen.   The image separation apparatus according to claim 1, wherein the information on the image projected by the projection unit includes a moving image, and the imaging unit is configured to capture an image from a foreground of the projected image.   3. The algorithm according to claim 1 or 2, wherein the algorithm estimates an image projected in space by using one or two primary color components other than the primary color component for a captured image corresponding to each primary color. Image separation device.   The algorithm uses an approximate expression in which the correlation function defining the image data to be separated is a product of second order or higher, and when the influence is extremely small, the operation of the term is omitted. 2. The image separation device according to 2 or 3.

Images