KR20050018034A - Apparatus for converting 2D image signal into 3D image signal - Google Patents

Abstract

PURPOSE: A device for converting a 2D image signal into a 3D image signal is provided to convert the 2D image signal into the 3D image signal in real time by simply generating a 3D image source using motion information. CONSTITUTION: A decoder(100) decodes an NTSC(National Television System Committee) 2D complex image signal, and outputs horizontal and vertical synchronous signals, a luminance signal and a color signal. The first delay(110) delays the luminance signal and the color signal outputted from the decoder. An image signal storing unit(130) stores the luminance signal and the color signal delayed in the first delay. A storing/reading control unit(120) controls the first delay according to the horizontal and vertical synchronous signals outputted from the decoder, and controls the image storing unit to store and read the luminance signal and the color signal delayed in the first delay. A 3D image converting unit(140) converts the luminance signal and the color signal read from the image signal storing unit into a 3D image signal. An encoder(150) encodes the 3D image signal outputted from the 3D image converting unit.

Description

Apparatus for converting 2D image signal into 3D image signal}

The present invention relates to an apparatus for converting a 2D video signal into a 3D video signal. In particular, a 2D video signal for converting a 2D composite video signal of a National Television Standards Committee (NTSC) method into a 3D composite video signal is 3D. An apparatus for converting into a video signal.

The technology of 3D stereoscopic image is one of the next generation information and communication services and is one of the advanced advanced technologies with high demand along with the advancement of society and the fierce competition for technological development. In addition, the technology of 3D stereoscopic image can be used in various fields such as information communication, broadcasting, medical, education and training, military, game, animation, virtual reality, CAD and industrial technology, and its application fields are very diverse. In other words, it is the core foundation technology of 3D stereoscopic multimedia information communication that provides the next-generation presence that is commonly required in many fields.

In general, since the information that people actually see a certain object through the eyes is a three-dimensional image, a lot of efforts are being made in the video system to make people look closer and more natural to the scenes of the natural world that people normally see. Currently, many countries in the world are developing realistic 3D stereoscopic multimedia services that can be viewed and enjoyed more naturally and realistically from 3D information-oriented multimedia services.

Three-dimensional information technology is the next generation of high value-added core technology. Many countries are competing for the practical use of three-dimensional information technology in terms of technology preoccupation in the future information and communication market. In addition, since 3D information technology is in the early stages of technology development worldwide, it is still less dependent on foreign technology and is expected to be a promising technology that can possess Korea's own technology.

Therefore, one of the three-dimensional information technology to be preoccupied in the Republic of Korea is a technology that automatically converts a two-dimensional image signal to a three-dimensional image signal in real time.

People can see a given object in three dimensions by binocular disparity, which is when a human sees the same object and sees different images through his right and left eyes. These two different images are made into one stereoscopic image in the brain.

However, since video display devices such as a television receiver or a monitor display a two-dimensional image on a plane, both the left and right eyes see the same image, and thus the user cannot feel the stereoscopic effect when viewing an object on a daily basis.

Therefore, in order to allow a user to feel a stereoscopic effect when viewing a predetermined image on a television receiver or a monitor, conventionally, a stereoscopic camera is used to make a three-dimensional image or to manually generate a two-dimensional image signal by using a three-dimensional image signal. Converted to.

However, the above-described conventional technology consumes a lot of cost and time, and thus, it is impossible to convert a vast amount of existing two-dimensional image signals into three-dimensional image signals. Above all, this converted 3D video signal is ineffective because it is expensive to purchase for general users.

In addition, the conventional 3D image is separated into a left eye image and a right eye image in consideration of the convergence of both eyes and the parallax of both eyes in a planar image of 2D in advance, or after taking images according to the binocular disparity with a left eye and right eye camera, There is a need for converting a 3D image having a parallax.

Therefore, in order to watch 3D still images and moving pictures, not only a 3D image reproducing apparatus but also a 3D image source must be prepared in advance.

SUMMARY OF THE INVENTION An object of the present invention is to provide an apparatus for converting a 2D video signal into a 3D video signal, which can simply convert an NTSC 2D composite video signal into a 3D composite video signal.

Another object of the present invention is to provide an apparatus for converting a two-dimensional video signal capable of converting a two-dimensional composite video signal of the NTSC system into a three-dimensional video signal suitable for the format of an external display device.

The apparatus for converting a 2D video signal to a 3D video signal having the above object has different perspective depths regardless of the direction and speed of movement of a moving object in the image by using motion parallax in the 2D video signal. Convert stereoscopic images in real time.

Here, the present invention is for real-time processing, which is difficult to process when the amount of calculation is large. Therefore, in the present invention, an image having a resolution smaller than that of the current and previous images is sampled at equal intervals for efficient calculation and real-time processing. The images sampled at equal intervals have the same luminance distribution as the shape information of the original image. This means that the histogram mean and standard deviation of the sampled image and the original image are the same, so that there is no problem even when using the sampled image when calculating the motion parallax in real time.

The video signal is converted into a color signal and a luminance signal in order to extract shape information of the obtained sampling image. The absolute value of the pixel difference between the current and previous images is obtained from the luminance signal of the sampled image, and the threshold value is compared to classify the still and moving pixels.

In this case, the stationary pixel is generally a background pixel, and is located at a relatively long distance, while the motion pixel is at a relatively close position.

The detected pixels are pixels representing a background or an object, and the sample image is divided into eight equal parts in the vertical direction to distinguish the background from the area, and the average and standard deviation of the moving pixel values in each area are calculated.

A depth map of the original image is created by using eight averages and standard deviations calculated for each region in the sample image. That is, if the pixel value of the original image is within the range of the average value of the upper and lower regions added to or subtracted from the standard deviation of each region, the pixel of the original image is defined as the group of pixels constituting the moving object. The depth value is set small, and other pixel groups set the depth value large.

Masking process is performed to remove the impulse noise caused from the generated depth map to produce a natural stereoscopic image. Both parallax processing of the background and the moving object performs positive parallax so that the moving object is located inside the screen and the background is located further behind the moving object. And the problem of occlusion by depth is to copy the background pixel as it is.

Therefore, the apparatus for converting the 2D video signal to the 3D video signal of the present invention includes a decoder for outputting horizontal and vertical synchronizing signals, luminance signals, and color signals by decoding NTSC two-dimensional composite video signals; A first delay delaying the luminance signal and the color signal outputted from the first delay delayer according to horizontal and vertical synchronization signals outputted from the decoder, and a delayed luminance signal delayed by the first delayer; A storage / reading control unit controlling the storage and reading of the video signal of the color signal, and an image signal storage unit storing and reading the luminance signal and the color signal delayed by the first delay unit under the control of the storage / reading control unit; A three-dimensional image converter for converting the luminance signal and the color signal read out from the image signal storage into a three-dimensional image signal, and the three-dimensional image converter And the encoding consists in a three-dimensional video signal that is output encoder characteristics.

And a second delayer for delaying the horizontal and vertical synchronization signals output from the decoder for a predetermined time, and a shutter for alternately operating the left and right eyes of the shutter goggles according to the horizontal and vertical synchronization signals delayed by the second delayer. It further comprises a drive unit.

The image signal storage unit may include: a multiplexer configured to switch and output the luminance signal and the color signal output from the first delay unit under the control of the storage / reading control unit, and to store the luminance signal and the color signal switched by the multiplexer. And first and second demultiplexers for selectively outputting a luminance signal and a color signal outputted by the first to third memories, and storing and reading under the control of the read control unit. .

The 3D image converter may include an area division sampling unit sampling a luminance signal output from the image signal storage unit and dividing an area, a luminance signal sampled by the area division sampling unit, and an output luminance signal of the first delay unit; A motion detection unit for detecting a motion by comparing a signal, a depth map generator for generating a depth map according to the motion detected by the motion detection unit with respect to an output video signal of the video signal storage unit, and an output video signal of the video signal storage unit. And a parallax processing unit for converting the parallax value into a 3D video signal according to the depth map generated by the depth map generation unit. It is done.

The present invention may further include a format conversion unit converting a format of a 3D video signal output from the 3D video conversion unit into a plurality of formats suitable for a plurality of external display apparatuses and outputting the converted format to the encoder, wherein the encoder comprises: The apparatus may further include an encoder for a head mount display (HMD), an encoder for a three-dimensional television receiver, and an encoder for a two-dimensional television receiver, respectively encoding a plurality of three-dimensional image signals converted by the format converter.

Hereinafter, an apparatus for converting a 2D video signal to a 3D video signal of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram showing the configuration of an embodiment of an apparatus for converting a two-dimensional image signal to a three-dimensional image signal of the present invention. As shown in the figure, the NTSC-type two-dimensional composite video signal is decoded to separate the horizontal synchronizing signal HS and the vertical synchronizing signal VS, and the luminance signal Y and the color signal CB and CR are separated. A decoder 100 for outputting, a first delay unit 110 for delaying the luminance signal Y and the color signals CB and CR output from the decoder 100, and a horizontal output from the decoder 100 And controlling the delay of the first delay unit 110 according to the vertical synchronization signals HS and VS, and of the luminance signal Y and the color signals CB and CR delayed by the first delay unit 110, respectively. A storage / read control unit 120 that controls storage and readout, and a luminance signal Y and a color signal CB and CR which are delayed by the first delay unit 110 to be controlled by the storage / read control unit 120. 3D image of the image signal storage unit 130 that stores and reads along with the luminance signal Y and the color signals CB and CR that are read from the image signal storage unit 130. A three-dimensional image converter 140 for converting to an arc, an encoder 150 for encoding a three-dimensional image signal output from the three-dimensional image converter 140, and horizontal and vertical synchronization output from the decoder 100 The second delay unit 160 delays the signals HS and VS for a predetermined time and the horizontal and vertical synchronization signals HS and VS are delayed by the second delay unit 160. The shutter driver 170 operates the shutters of the left and right eyes alternately.

The image signal storage unit 130, the multiplexer for switching the luminance signal (Y) and the color signals (CB, CR) output from the first delay unit 110 under the control of the storage / read control unit 120. 131 and first to third memories for storing and reading the luminance signal Y and the color signals CB and CR switched by the multiplexer 131 under the control of the storage / reading control unit 120. First and second selecting and outputting the luminance signal Y and the color signals CB and CR which are read by the first and third memories 132, 133, and 134 and the first to third memories 132, 133, and 134. The demultiplexer 135,136 was comprised.

The 3D image converter 140 is an area division sampling unit 141 for sampling and dividing the luminance signal Y and the color signals CB and CR of the immediately preceding frame output from the first demultiplexer 135. And a motion detector 142 for detecting motion by comparing the luminance signal Y divided by the region division sampling unit 141 with the luminance signal Y of the current frame output from the first retarder 110. And a depth map generator 143 for generating a depth map according to the movement detected by the motion detector 142 with respect to the output luminance signal Y and the color signals CB and CR of the second demultiplexer 136. And a parallax value given to the output luminance signal Y and the color signals CB and CR of the second demultiplexer 136 according to the depth map generated by the depth map generator 143 as a 3D image signal. It consisted of the parallax process part 140 to convert.

In the apparatus for converting a 2D video signal of the present invention configured as described above into a 3D video signal, an NTSC type 2D composite video signal is decoded by the decoder 100 so that a horizontal sync signal HS and a vertical sync signal VS are obtained. In addition to the separation, the luminance signal Y and the color signals CB and CR are output to the first delay unit 110.

Then, the storage / read control unit 120 controls the delay operation of the first delay unit 110 according to the horizontal and vertical synchronization signals HS and VS output from the decoder 100, and further stores the image signal storage unit. The multiplexer 131 of FIG. 130 is controlled to switch the luminance signal Y and the color signals CB and CR output from the first retarder 110 and to synchronize and control the multiplexer 131. The first to third memories 132, 133, and 134 are selectively controlled to sequentially store and read the luminance signal Y and the color signals CB and CR switched by the multiplexer 131.

For example, as illustrated in FIG. 2A, luminance signals Y and color signals CB and CR of a plurality of frames each having a first field and a second field are sequentially output from the first delay unit 110. Assuming that the first delay unit 110 outputs the luminance signals Y and the color signals CB and CR of the first and second fields of the first frame, the storage / reading control unit 120 In addition to controlling the multiplexer 131, the first memory 132 may be stored and enabled as shown in FIG. 2B, so that the first field and the second field of the first frame output from the first delay unit 110 may be controlled. The luminance signal Y and the color signals CB and CR are input to and stored in the first memory 132 through the multiplexer 131. In addition, when the luminance signal Y and the color signals CB and CR of the second frame are output from the first delay unit 110, the storage / reading control unit 120 controls the multiplexer 131 and the second. As shown in FIG. 2C, the memory 133 is stored enabled so that the luminance signal Y and the color signals CB and CR of the first frame output from the first delay unit 110 may be used to store the multiplexer 131. The second memory 133 is input and stored through the second memory 133. When the luminance signal Y and the color signals CB and CR of the third frame are output from the first retarder 110, the storage / reading control unit 120 ) Controls the multiplexer 131 and stores the third memory 134 as shown in FIG. 2D, thereby enabling the luminance signal Y of the third frame output from the first retarder 110. The color signals CB and CR are repeatedly input and stored to the third memory 134 through the multiplexer 131.

That is, the luminance signal Y and the color signals CB and CR of each frame output from the first retarder 110 are sequentially stored in the first to third memories 132, 133, and 134.

The storage / reading control unit 120 stores the image signal when the luminance signal Y and the color signals CB and CR of the current frame are output from the first delay unit 110 and stored in the image signal storage unit 130. The luminance signal Y and the color signals CB and CR immediately stored in the unit 130 are read.

For example, after storing the luminance signal Y and the color signals CB and CR of the first frame in the first memory 132, the luminance signal Y and the second frame of the second frame are stored in the first delay unit 110. When the color signals CB and CR are output and stored in the second memory 133, the storage / reading control unit 120 controls the first memory 132 and the first and second demultiplexers 135 and 136. The luminance signal Y and the color signals CB and CR of the first frame stored are read out, and the read luminance signal Y and the color signals CB and CR are read out from the first and second demultiplexers 135 ( 136 to output the luminance signal Y and the color signals CB and CR of the first frame output from the image signal storage unit 130 by the 3D image converter 140 as shown in FIG. 2E. ) And the 3D video signal generated from the luminance signal Y and the color signals CB and CR of the second frame output from the first delay unit 110, and the generated 3D video signal is shown in FIG. 2F. As shown When the luminance signal Y and the color signals CB and CR of the third frame and the fourth frame are respectively stored in the image signal storage unit 130, the image signal storage unit 130 performs the second frame and the third frame. The luminance signals Y and the color signals CB and CR of the frames are sequentially output so that the 3D image converter 140 outputs the luminance signals of the second and third frames output from the image signal storage unit 130. (Y) and color signals CB and CR, the luminance signal Y and the color signals CB and CR of the third and fourth frames output from the first delay unit 110, and the three-dimensional image signal. Generate them sequentially.

As such, when the luminance signal Y and the color signals CB and CR immediately before are output from the image signal storage unit 130, the region division sampling unit 141 of the 3D image conversion unit 140 may have a resolution greater than that of the image. A luminance signal is sampled at equal intervals for an image having a small resolution, and the sampled image is horizontally divided into a predetermined number.

That is, since a large amount of computation is required and a large amount of time is required when the entire screen is computed at once, in the present invention, the region division sampling unit 141 of the 3D image conversion unit 140 is larger than the resolution of the image for efficient calculation and real time processing. The luminance signal is sampled at equal intervals for a small resolution image, and divided into 8 equal parts horizontally. The sampled image has the same luminance signal as that of the original image, and dividing the sampled image into eight equals the detection error when the moving pixel value is composed of different gradation values instead of the same gradation value throughout the image. It is to reduce.

When the image divided by the region division sampling unit 141 is input to the motion detector 142, the motion detector 142 may output luminance signals of the first and second fields of the current frame input from the first delay unit 110. By comparing the value of (Y) with the value of the luminance signal Y just before input from the region division sampling unit 141 for each pixel, a representative value for the movement of each part is obtained, and the obtained representative The value is output to the depth map generator 143.

Then, the depth map generator 143 may calculate the representative values of the eight movements calculated for each region of the eight equal luminance signals according to the luminance signal Y and the color signals CB and CR. If the result is larger than the motion representative value, it is determined as a motion pixel, and if it is smaller, it is determined as a still image and output to the parallax processor 144.

The parallax processor 144 divides the motion pixel and the still image pixel according to the output signal of the depth map generator 143 to lower the parallax value for the motion pixel, and gives a lot of parallax values for the still pixel. The pixels are processed into a background and converted into a 3D video signal.

The 3D video signal converted by the parallax processor 144 is encoded by the encoder 150 and converted into a composite video signal, and then output to be displayed on a display panel.

The horizontal synchronizing signal HS and the vertical synchronizing signal VS output from the decoder 100 may output a three-dimensional composite image signal from the encoder 150 as described above by the second delayer 160. The time required until the delay is delayed, and the shutter driver 170 selectively operates the shutters of the left and right eyes of the shutter goggles 180 according to the output signal of the second retarder 160 so that the user can release the shutter goggles 180. ) And watch a 3D image displayed on a display panel or the like.

3 is a block diagram showing the configuration of another embodiment of an apparatus for converting a 2D video signal to a 3D video signal of the present invention. As described above, another embodiment of the present invention provides a format of a 3D image signal output from the parallax processor 144 of the 3D image converter 140 into a plurality of formats suitable for a plurality of external display devices. The apparatus further includes a format converter 200 for converting and outputting the encoder 150 to the encoder 150. The encoder 150 includes an HMD for encoding a plurality of three-dimensional image signals converted by the format converter 200, respectively. An encoder 210 for a head mount display, an encoder 220 for a 3D television receiver, and an encoder 230 for a 2D television receiver.

According to another embodiment of the present invention configured as described above, the format conversion unit 200 receives a 3D image signal output from the parallax processing unit 144 of the 3D image conversion unit 140 and is suitable for a plurality of external display devices. The format is converted into a and is output to the encoder 150. For example, the format converter 200 converts a 3D video signal into a format suitable for a head mount display (HMD), a 3D television receiver, and a 2D television receiver, and outputs the converted format to the encoder 150.

The 3D video signal that is format converted by the format converter 200 is encoded by the encoder 150 and output to the outside, that is, the 3D video signal that is converted into the HMD format is encoded by the HMD encoder 210 for HMD. The 3D video signal is converted into a 3D video signal and then output and displayed on the HMD. The 3D video signal converted into a 3D television receiver format is encoded by the 3D television receiver encoder 320 to be 3D video signal. The 3D video signal, which is converted into and displayed on the 3D television receiver, is converted into the format of the 2D television receiver, and is encoded by the 2D television receiver encoder 330 to be a 3D video signal for the 2D television receiver. After being converted, it is output and displayed on a two-dimensional television receiver.

On the other hand, while the present invention has been shown and described with respect to specific preferred embodiments, various modifications and changes of the present invention without departing from the spirit or field of the invention provided by the claims below It can be easily understood by those skilled in the art. For example, in the above description, the format of the 3D video signal of the present invention is converted and provided as an HMD, a 3D television receiver, and a 2D television receiver. However, the present invention is not limited thereto, and the parallax processor is used. The format of the 3D video signal outputted from the projector may be variously converted and output to a video display device such as a projection TV, a projector, and a holographic 3D television receiver.

As described above, the present invention can easily create a three-dimensional image source using motion information. By using the present invention, a three-dimensional conversion apparatus for converting a two-dimensional image signal into a three-dimensional image signal in real time can be made. In addition, the 3D conversion device enables users to enjoy 3D images easily without purchasing expensive specialized equipment in a general home.

1 is a block diagram showing the configuration of an embodiment of the present invention.

FIG. 2 is an operational waveform diagram of each part of FIG. 1. FIG.

Figure 3 is a block diagram showing the configuration of another embodiment of the present invention.

Explanation of symbols on the main parts of the drawings

100: decoder 110: first delay

120: storage / read control unit 130: video signal storage unit

131: multiplexer 132 to 134: first to third memories

135 and 136: first and second demultiplexers 140: 3D image converter

141: region division sampling unit 142: motion detection unit

143: depth map generator 144: parallax processing unit

150: encoder 160: second delay

170: shutter driving unit 180: shutter goggles

200: format conversion unit 210: HMD encoder

220: 3D television receiver encoder

230: encoder for two-dimensional television receiver

Claims (6)

A decoder for decoding the NTSC two-dimensional composite video signal and outputting horizontal and vertical synchronization signals, luminance signals, and color signals; A first delayer for delaying the luminance signal and the color signal output from the decoder; An image signal storage unit for storing and reading the luminance signal and the color signal delayed by the first delay unit; A storage / reading which controls the delay of the first delayer according to the horizontal and vertical synchronization signals output from the decoder, and controls the storage and reading of the luminance signal and the color signal delayed by the first delayer. Control unit; A three-dimensional image converter for converting the luminance signal and the color signal read from the image signal storage into a three-dimensional image signal; And The apparatus for converting a two-dimensional image signal consisting of an encoder for encoding a three-dimensional image signal output from the three-dimensional image conversion unit into a three-dimensional image signal. The method of claim 1, A second delayer for delaying the horizontal and vertical synchronization signals output from the decoder by a predetermined time; And a shutter driver for alternately operating the shutters of the left and right eyes of the shutter goggles in response to the horizontal and vertical synchronization signals delayed by the second retarder. . The display apparatus of claim 1, wherein the image signal storage unit; A multiplexer for switching and outputting the luminance signal and the color signal output from the first delay unit under the control of the storage / reading control unit; First to third memories for storing and reading the luminance signal and the color signal switched by the multiplexer under the control of the storage / reading control unit; And And a first and second demultiplexers for selectively outputting luminance signals and color signals output by the first to third memories, and converting the two-dimensional image signals into three-dimensional image signals. The apparatus of claim 1, wherein the 3D image converter; An area division sampling unit sampling a luminance signal from an output signal of the image signal storage unit and dividing an area; A motion detection unit for detecting a motion of an image by comparing the value of the luminance signal sampled by the area division sampling unit with the value of the output luminance signal of the first retarder for each pixel; A depth map generator configured to generate a depth map according to the movement detected by the motion detector with respect to the output signal of the image signal storage unit; A 3D image of a 2D image signal comprising a parallax processor for converting an output image signal of the image signal storage unit into a 3D image signal by giving a parallax value according to the depth map generated by the depth map generator Device that converts signals. The apparatus of claim 4, wherein the area division sampling unit; An apparatus for converting a two-dimensional image signal into a three-dimensional image signal, characterized by horizontally dividing the area of the sampled luminance signal. The method of claim 1, The apparatus may further include a format converting unit converting a format of the 3D image signal output from the 3D image converting unit into a plurality of formats suitable for a plurality of external display devices and outputting the converted format to the encoder. The encoder is; A two-dimensional video signal comprising an encoder for a head mount display (HMD), an encoder for a three-dimensional television receiver, and an encoder for a two-dimensional television receiver, respectively encoding a plurality of three-dimensional image signals converted by the format converter. To convert a 3D video signal.