KR101251049B1 - Stereoscopic 3d image system - Google Patents

Abstract

The present invention, in the stereoscopic three-dimensional image system including a camera, an image processing unit, a half mirror and the first and second monitors, the image processing unit to extract the foreground from the input image photographed by the camera to process the hologram synthesis and; The extracted foreground is displayed on the first monitor; It relates to a stereoscopic three-dimensional imaging system, characterized in that the foreground displayed on the first monitor is synthesized holographically with the image displayed on the second monitor through a half mirror.
The present invention relates to a stereoscopic three-dimensional imaging system, characterized in that for processing the image processing unit to be holographically synthesized in the foreground using a luma-key matte.

Description

Stereoscopic 3D Imaging System {STEREOSCOPIC 3D IMAGE SYSTEM}

The present invention, in the stereoscopic three-dimensional image system including a camera, an image processing unit, a half mirror and the first and second monitors, the image processing unit to extract the foreground from the input image photographed by the camera to process the hologram synthesis and; The extracted foreground is displayed on the first monitor; It relates to a stereoscopic three-dimensional imaging system, characterized in that the foreground displayed on the first monitor is synthesized holographically with the image displayed on the second monitor through a half mirror.

The present invention relates to a stereoscopic three-dimensional imaging system, characterized in that for processing the image processing unit to be holographically synthesized in the foreground using a luma-key matte.

Recently, researches to implement stereoscopic 3D images through mass media have been actively conducted.

Such a stereoscopic image display method can be divided into a glasses method using a special glasses and a lenticular (Lenticular) method, the glasses method has been pointed out that it is inconvenient and unnatural when watching the image because the glasses should be worn.

Meanwhile, there is a demand for realizing autostereoscopic stereoscopic stereoscopic images by synthesizing two or more viewpoint change images in real time.

This is a remote image system that is already implemented in the fields of education, meetings, seminars, medical treatment, interviews and counseling, and realizes stereoscopic images by synthesizing the foreground images and contents images of educators, presenters, doctors, and counselors in real time. This is to maximize the effect of the system.

In general, in order to produce 3D stereoscopic content, a subject of a foreground image is photographed in a special equipment or lighting environment in a virtual studio space, and a content is generated using an expensive 3D professional program.

In addition, real-time image synthesis requires a considerable amount of computation, which usually requires expensive specialized equipment.

Therefore, the technology that can synthesize the foreground image with the remote video interface in real time by reducing the amount of computation without using expensive equipment while interworking with the remote video system installed or provided as a solution is needed.

Meanwhile, Korean Unexamined Patent Publication No. 2000-0074007 describes a half mirror variable apparatus of a stereoscopic image display.

This focuses on selectively displaying two-dimensional and three-dimensional images by varying the half mirror up and down, and is not related to a system for synthesizing images using the half mirror.

An object of the present invention is to operate in conjunction with a remote image system that is already installed or provided as a solution, a stereoscopic three-dimensional that can synthesize the foreground image with the remote image interface in real time by reducing the amount of computation without using expensive equipment In providing an imaging system.

The present invention, in the stereoscopic three-dimensional image system including a camera, an image processing unit, a half mirror and the first and second monitors, the image processing unit to extract the foreground from the input image photographed by the camera to process the hologram synthesis and; The extracted foreground is displayed on the first monitor; By providing a stereoscopic three-dimensional imaging system, characterized in that the foreground displayed on the first monitor is synthesized with the image displayed on the second monitor through the half mirror, to solve the technical problem.

The present invention is to solve the technical problem by providing a stereoscopic three-dimensional imaging system, characterized in that for processing the image processing unit to be hologram-synthesized the foreground using a Luma-Key matte. .

The stereoscopic three-dimensional imaging system of the present invention can be combined with a remote image system that is installed or provided as a solution, and can reduce the amount of computation without using expensive equipment to synthesize the foreground image with the remote image interface in real time. Has a significant effect.

1 is a view schematically showing the main configuration of a stereoscopic 3D imaging system according to an embodiment of the present invention.
2 is a diagram illustrating an example of a result of separating the primary foreground mirror using an input image.
3 is a diagram illustrating an example of a result of separating the secondary front mirror using a mask image.
4 is a diagram illustrating an example of a result of removing a noise pixel from a secondary mirror.
5 is a diagram illustrating an example of a result of filling empty spaces in an image from which noise pixels are removed.
6A is a diagram illustrating an example of a result of applying an alpha mat once.
6B is a photograph showing an example of the result of applying the alpha matte three times.
7A to 7C are photographs illustrating an example of a process of holographically processing a foreground image using a half mirror to synthesize a remote image interface.
FIG. 8 is a diagram illustrating an example of a result obtained by holographically processing a foreground image using a half mirror and synthesizing it into a remote image interface.

The terms and words used in the present specification and claims should not be construed as limited to ordinary or dictionary terms and the inventor may properly define the concept of the term in order to best describe its invention It should be construed as meaning and concept consistent with the technical idea of the present invention.

Therefore, the embodiments described in the present specification and the configurations shown in the drawings are merely the most preferred embodiments of the present invention and are not intended to represent all of the technical ideas of the present invention. Therefore, various equivalents It should be understood that water and variations may be present.

Before describing the present invention with reference to the accompanying drawings, it should be noted that the present invention is not described or specifically described with respect to a known configuration that can be easily added by a person skilled in the art, Let the sound be revealed.

1 is a view schematically showing the main configuration of a stereoscopic 3D imaging system according to an embodiment of the present invention.

Real-time image synthesis system according to an embodiment of the present invention is configured to include a camera 100, the image pre-processing unit 200 and the image processing unit 300.

First, each module included in the image preprocessor 200 and the image processor 300 is designed to operate in conjunction with various systems, and depending on conditions, each module included in the image preprocessor 200 may be an image processor 300. It should be noted that it may be configured to be included in or vice versa.

In addition, since the present invention focuses on image synthesis for realizing stereoscopic stereoscopic images, it turns out that the camera 100 is installed so that the entire body, ie, head to toe, of the person can be photographed.

Preferably, the camera 100 according to the present invention is configured to be rotated 90 degrees fixedly, in which case the input is as if the person is lying down, but can reduce the distance between the camera and the person.

That is, since the camera's image is generally 4: 3, the camera and the person need to be separated from each other in order to shoot the whole person because this may cause a space limitation.

The image preprocessor 200 is configured to perform a function of generating a comparison image, extracting a threshold value from the comparison image, and preprocessing the background image. The comparison value processor 210, the background image processing module, and the image rotation module are provided. It is configured to include.

The comparison value processor 210 performs a function of generating and updating a comparison image and extracting a threshold value to extract the foreground of the person, and includes a comparison image update module and a threshold value extraction module.

The comparison image update module performs a function of generating and updating a comparison image for separating the primary foreground image from the input image.

In other words, in the area where the person moves, the difference between the background and the color appears, and in order to extract the person foreground, an image that is a reference to be compared with the input image is generated and updated.

The comparison image update module is configured to generate a comparison image in advance before performing real-time synthesis, and continue to update in real time when there is no person in the input image thereafter.

The threshold extraction module performs a function of extracting a threshold value from the comparison image, which serves as a comparison value for extracting the foreground from the input image.

That is, after subdividing the comparative image into units of several regions, respective RGB average values, color thresholds (HUE), and brightness thresholds are extracted.

Preferably, the area unit is set by the user since the threshold value may be sensitively applied according to the influence of the lighting.

Also preferably, since the comparison image is continuously updated in real time, the threshold value extracted therefrom may also be configured to be continuously updated.

The background image processing module performs a function of processing a background image to be used when the user synthesizes the image data. The image data is configured to be processed by changing a pixel RGB value into a structure.

In other words, when composing an image, the background image is processed so that the pixels of the part except the foreground can be filled.

Depending on the design conditions, the remote image interface screen may be configured to receive and process the background image or set a separate background image.

The image rotation module performs a function of rotating the comparative image and the background image by 270 degrees, which is due to the camera 100 being rotated by 90 degrees.

That is, although the image input through the camera 100 is rotated by 90 degrees, the monitor should be normally displayed on the monitor, so that a function of rotating the image is required.

The image processor 300 extracts the foreground from the input image and generates alpha matte and luma key mats to hologram synthesize the foreground, and includes an image rotation module, a foreground extractor 310, and a hologram processor 320. ).

The image rotation module is for correcting that the camera 100 is installed by being rotated by 90 degrees as described above, and serves to rotate the input image by 270 degrees.

The foreground extracting unit 310 performs a function of extracting the foreground from the input image, and includes a primary foreground separation module, a mask image generation module, a secondary foreground separation module, a noise pixel removing module, and a blank space filling module.

The primary foreground separating module performs a function of separating the primary foreground from the input image using an RGB average value which is a threshold value extracted by the comparison value processor 210.

That is, after comparing the RGB value of each pixel of the comparison image and the input image, the pixels exceeding the RGB average value of each region are separated into the primary mirror.

Here, the primary foreground has a meaning as a rough rough foreground, and has a meaning to reduce the amount of computation to be performed in the next step.

FIG. 2 is a photograph showing an example of a result of separating the primary foreground using an input image. In the separated primary foreground on the right side, pixels having a color similar to a background color are included in the primary foreground. Able to know.

The mask image generation module performs a function of generating a mask image using the primary foreground mirror. The mask image generation module has a value of 255 for the part separated by the foreground and a value of 0 for the part separated by the background.

The secondary foreground separation module performs a function of separating the secondary foreground by using a color (HUE), a luminance value, and a mask image, which are threshold values extracted by the comparison value processor 210.

That is, by comparing the color (HUE) and the brightness (Luminance) values with respect to the pixel having a value of 255 in the mask image is processed again whether the foreground.

FIG. 3 is a photograph showing an example of a result of separating the secondary front mirror using a mask image, and it can be seen that noise between the human legs and the background disappears.

However, you can see that the foreground continues to contain noise, such as losing a foreground that has a color similar to the background, or creating an empty ball in the area that should be recognized as the foreground.

This is because the background contains noise in response to light or light reflections.

The noise pixel removal module removes the noise pixel by using the number of pixels connected in the horizontal and vertical directions.

In other words, in order to remove pixels having a small width, the number of consecutive foreground pixels is smaller than the set threshold value.

Preferably, the directionality of the noise pixel is different depending on the location, and although the order of removal is arbitrary, the noise pixel is removed in the horizontal direction and the vertical direction, and thus, the improved result is configured.

Also, preferably, the amount of computation is reduced by removing the noise pixel around the foreground pixel having the 255 value of the mask image.

FIG. 4 is a photograph showing an example of a result of removing a noise pixel from a second foreground, in which a background noise is almost removed and only a foreground of a person remains.

However, a problem remains that the empty space removed from the foreground has to be solved because of the color similar to the background.

The empty space filling module performs a function of filling an empty space in the foreground with an input image.

That is, the pixel of the input image is filled with the foreground by comparing whether the empty space pixel is changed and / or the correlation of the pixels.

Preferably, the unchanged state in the previous step is regarded as the inside of the person and is also weighted in the vertical direction to compare and fill the pixels above and below.

This is a case where the foreground of the person is weighted in the vertical direction by paying attention to the fact that the arms and legs are facing downward, and of course, it can be variously implemented according to the design conditions.

Also, preferably, the amount of computation is reduced by filling the empty space around the foreground pixel having the value of 255 of the mask image.

FIG. 5 is a photograph showing an example of a result of filling empty spaces in an image from which noise pixels are removed, and it can be seen that the foreground is precisely extracted.

The hologram processing unit 320 performs a function of generating an alpha matte and a luma key mat to process the extracted foreground image with the remote image interface and hologram synthesis. The hologram processing unit 320 includes an alpha mat generating module and a luma key mat generating module.

The alpha matte generation module performs the function of generating an alpha matte to naturally synthesize the extracted foreground with the background image.

In other words, there are pixels mixed with the background color at the boundary of the extracted foreground, and it shows a step phenomenon. To remove it, an alpha matte must be generated and used for sophisticated composition.

Preferably, since the present invention aims at real time synthesis, the alpha matte is implemented using an average filter mask without using Knockout mating, Bayesian mating, or Poisson mating.

The alpha matte generation module generates an 8-bit grayscale alpha matte having a value between 0 and 255 using the mask image generated by the mask image generation module, and then applies an average value filter mask.

FIG. 6A illustrates an example of the result of applying the alpha matte once, and FIG. 6B illustrates a photograph of an example of the result of applying the alpha matte three times.

In the example of FIG. 6, a 3x3 average filter mask is applied, since an image size according to an example of the present invention is 960x720 and thus an effective result can be obtained compared to 320x240 size.

It goes without saying that other types of average value filter masks may be employed depending on the design conditions.

As shown in FIG. 6, it can be seen that the result of applying three times is reduced compared to the result of applying once.

However, if the number of times is applied, the boundary line becomes ambiguous and the foreground may appear awkward in the synthesized image. Therefore, it is preferable to set this properly according to the design conditions.

The luma key generation module converts the extracted foreground to grayscale to generate a luma-key matte, and the foreground is hologramized using the luma key matte.

7A to 7C are photographs illustrating an example of a process of holographically processing a foreground image using a half mirror to synthesize a remote image interface, and FIG. 8 is a remote image by hologram processing a foreground image using a half mirror. Figure 1 is a diagram showing an example of the result synthesized in the interface.

It can be seen that the foreground image is displayed on the first monitor, and this image is synchronized to the second monitor through the half mirror to perform hologram synthesis.

Depending on the design conditions, it is a matter of course that the means for separately storing the holographic synthesis processed image may be further configured.

Meanwhile, the above descriptions using only FIGS. 1 to 6 describe only the main matters of the present invention, and the present invention is limited to the configurations and functions of FIGS. 1 to 9 as long as various designs are possible within the technical scope. It is obvious that it is not limited.

100: camera
200: image preprocessing unit
210: comparison value processing unit
300: image processing unit
310: foreground extracting unit
320: hologram processing unit

Claims (6)

camera; An image preprocessor; An image processor; Half mirror; In the stereoscopic three-dimensional imaging system comprising;
The image processor extracts the foreground from the input image captured by the camera and processes the hologram to be synthesized.
The extracted foreground is displayed on the first monitor,
The foreground displayed on the monitor 1 synthesizes the hologram with the image displayed on the monitor 2 through the half mirror,
The extraction process of the foreground,
(a) the image preprocessing unit updates in real time a comparison image without a foreground, and extracts RGB average value, color (HUE) and brightness threshold values for each reference region from the image area, and
(b) the image processing unit compares the input image input from the camera with the RGB values of each pixel of the comparison image, and then separates the pixels exceeding the RGB average value of each region into the primary foreground,
(c) generating a mask image using the separated foreground;
(d) a threshold of color and brightness of each area after comparing the color (HUE) and brightness (Lumination) values of pixels corresponding to the foreground of the mask image with the values of the corresponding pixels of the comparison image. Separate pixels beyond the value into a second foreground,
(e) If the number of pixels connected in the horizontal or vertical direction in the second foreground is less than the reference value, the noise is detected and removed;
and (f) comparing the pixels that were not recognized as the noise with the upper, lower, left, and right pixels, and then filling the input image when the empty space is detected.
The method of claim 1,
And processing the image processing unit so that the foreground can be hologram-synthesized by using luma-key matte.
delete delete delete The method of claim 1,
The image processing unit, after generating an alpha matte to which the average value filter mask is applied, removes the step phenomenon from the boundary line of the foreground using the alpha matte, stereoscopic three-dimensional imaging system.

Images