KR101217476B1 - An apparatus and a method for processing an image signal - Google Patents

Abstract

The image signal processing apparatus 3000 according to the present invention includes a Bayer processor 310 which performs at least one of lens shading correction, pixel replacement, automatic white correction, and color interpolation on a plurality of images output from a plurality of image sensors. A selection unit 320 for outputting an image according to an image format using images output from the plurality of Bayer processing units; And an RGB processor 330 performing at least one of gamma correction, color correction, and wide range correction on the image output from the selector 320.

Description

Apparatus and method for image signal processing {AN APPARATUS AND A METHOD FOR PROCESSING AN IMAGE SIGNAL}

The present invention relates to an image signal processing apparatus having a plurality of image sensors and an image signal processing method.

In the imaging device, a charge coupled device (CCD) or a CMOS is used as an image sensor. In addition, although the CMOS image sensor is advantageous in terms of cost and power consumption than the CCD image sensor, the image signal processor (ISP), which is a preprocessing step, is used to improve the quality since the image quality is not good.

An object of the present invention is to efficiently transmit a plurality of images generated from a plurality of image sensors under a bandwidth limitation for image transmission.

In order to achieve the above object, the present invention provides a video device characterized in that the plurality of images generated from the plurality of image sensors are sampled and transmitted in a column direction or a row direction, without transmitting the image in a full frame. present.

The present invention proposes an imaging apparatus characterized by adaptively using an image format for image signal processing by using a control signal according to an external image format selection command.

According to the present invention, by controlling a plurality of images from a plurality of image sensors with one image signal processing apparatus, it is possible to overcome the spatial limitations in the imaging apparatus by using the plurality of image signal processing apparatuses, The cost of using the signal processing device can be reduced.

1 is a block diagram schematically illustrating a configuration of an imaging apparatus having a plurality of image sensors as an embodiment to which the present invention is applied.
2 is a block diagram schematically showing a configuration of a Bayer processing unit as an embodiment to which the present invention is applied.
3 is a block diagram schematically illustrating a configuration of an RGB processing unit as an embodiment to which the present invention is applied.
4 is a block diagram schematically illustrating a configuration of a YUV processor as an embodiment to which the present invention is applied.
FIG. 5 is a flowchart illustrating an image signal processing process in an image signal processing apparatus having a selection unit according to an embodiment to which the present invention is applied.
FIG. 6 illustrates a plurality of image formats supported by an image signal processing apparatus according to an embodiment to which the present invention is applied.
FIG. 7 is a flowchart illustrating an image signal processing process in an image signal processing apparatus having an image sensor controlled by a controller, according to an embodiment to which the present invention is applied.

The present invention relates to an imaging apparatus having a plurality of image sensors, and more particularly, to a plurality of image sensors in one imaging apparatus, and to converting a plurality of images from the plurality of image sensors into one image signal processing apparatus. It relates to an imaging device for controlling. Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram schematically illustrating a configuration of an imaging apparatus having a plurality of image sensors as an embodiment to which the present invention is applied.

The imaging apparatus of the present invention may include a plurality of image sensors and an image signal processing apparatus 3000.

Referring to FIG. 1, an image sensor of the present invention may be composed of a first image sensor 1000 and a second image sensor 2000. The image sensor may allocate data of one of red, green, and blue (hereinafter, referred to as R, G, and B) for each pixel, and output an image having a pixel pattern of a bayer pattern. .

The image signal processing apparatus 3000 is for improving the image quality of the image output from the image sensor. Specifically, the Bayer processor 310, the selector 320, the RGB processor 330, and the first color space converter are provided. 340, an image buffer unit 350, a second color space converter 360, a YUV processor 370, and an output format unit 380.

The Bayer processor 310 corrects the image output from the image sensor to have original data, which will be described with reference to FIG. 2.

2 is a block diagram schematically showing the configuration of the Bayer processor 310 as an embodiment to which the present invention is applied.

Referring to FIG. 2, the Bayer processor 310 may include a lens shading corrector 312, a pixel substitute 314, an automatic white corrector 154, and a color interpolator 318 to correct an image. have. The Bayer processor 310 may include a plurality of Bayer processors corresponding to the plurality of image sensors. For example, referring to FIG. 1, the Bayer processor 310 may include a first Bayer processor 310A and a second Bayer processor 310B.

The lens shading corrector 312 may correct the brightness of the peripheral part by performing lens shading correction on the image when the phenomenon in which the brightness or the resolution decreases from the center of the image toward the peripheral part, that is, the lens shading phenomenon appears. The pixel replacement unit 314 may perform correction to remove the noise by using neighboring pixels when noise is generated in black and white dots due to impurities added to the image sensor. The automatic white corrector 316 may correct the distorted light source of the image as if it is seen from a visual point of view. Here, the color temperature may be used as an index representing the characteristics of the light source. The color temperature is a numerical value of light of a light source, and means a value measured by using a temperature of a black body that emits the same light as the light of the object when the object is illuminated. That is, the white temperature may be determined by determining the color temperature of the image, obtaining the gain of each of the R, G, and B channels based on the color temperature of the image. The color interpolator 318 reconstructs the Bayer pattern image into a complete image including R, G, and B at every pixel. This is because color cannot be determined using only one data of R, G, and B allocated for each pixel. In this case, a 2 * 2, 3 * 3, or 5 * 5 method may be used depending on the size of the neighboring pixel region used for interpolation. In addition, the R, G, B is a color expression method by adjusting the intensity of the three primary colors of light consisting of red, green and blue, hereinafter referred to as RGB data.

Referring to FIG. 1, the selector 320 may select an image format for image signal processing and output an image according to the selected image format. This will be described with reference to FIGS. 5 and 6.

The RGB processor 330 corrects the distortion of the color and contrast of the image, which will be described with reference to FIG. 3.

3 is a block diagram illustrating a schematic configuration of an RGB processor 330 as an embodiment to which the present invention is applied.

Referring to FIG. 3, the RGB processor 330 may include a gamma correction unit 332, a color correction unit 334, and a wide range correction unit 336. The gamma correction unit 332 corrects the distortion of the display device in advance, and the color correction unit 334 restores the RGB data of the image to have an original color, and analyzes the image by analyzing the wide range correction unit 336. You can automatically adjust the contrast in bright and dark areas such as the eye. In this case, a wide dynamic range (WDR) or high dynamic range (HDR) algorithm may be used.

The first color space converter 340 may perform color space conversion of RGB data of the image into YUV data. In the YUV data, Y represents a brightness signal, U and V represent a color signal, and the YUV data represents a representation method according to the three component ratios. For example, the first color space converter 340 may refer to an image having RGB data as an image having YUV data having a 4: 4: 4 component ratio of Y, U, and V (hereinafter, referred to as a first YUV image). .) The first YUV image may be converted into an image having YUV data having a 4: 2: 2 component ratio of Y, U, and V (hereinafter, referred to as a second YUV image). As such, when the first YUV image is converted into the second YUV image, the amount of data allocated to each pixel can be reduced, so that the image output from the first color space converter 340 is transferred to the image buffer 350. Can be stored efficiently

The image buffer 350 may store the YUV data output from the first color space converter 340 and transfer the YUV data to the second color space converter 360. Similarly, the image buffer unit 350 may include a plurality of image buffer units respectively corresponding to the plurality of image sensors. For example, referring to FIG. 1, the image buffer unit 350 may include a first image buffer unit 350A and a second image buffer unit 350B. The image output from the first image sensor 1000 may be stored in the first image buffer unit 350A, and the image output from the second image sensor 2000 may be stored in the second image buffer unit 350B. The image can be saved.

The image buffer 350 may be updated at regular intervals and may not include a separate refresh device. For example, the predetermined period may be set to a time for processing one image. That is, when the current image is transferred to the image buffer 350 through the Bayer processor 310, the selector 320, the RGB processor 330, and the first color space converter 340, the image buffer unit ( 350 may update the stored previous image with the current image. Therefore, the previous image may be stored in the image buffer unit 350 during the processing time of the current image. In addition, the image buffer unit 350 may be implemented as a memory such as an SDRAM.

The second color space converter 360 may convert the second YUV image into a first YUV image.

The YUV processor 370 will be described with reference to FIG. 4.

4 is a block diagram schematically illustrating a configuration of the YUV processor 370 as an embodiment to which the present invention is applied.

Referring to FIG. 4, the YUV processor 30 adjusts a contrast level of a camera, that is, a contrast enhancer that adjusts contrast of an image by applying a contrast level value input by a user to the image. 372, a noise removing unit 374 for removing noise of a luminance component or a color component of the image, and a contour improvement unit 376 for sharpening a boundary area of the image.

The output format unit 380 may convert the first YUV data output from the YUV processor 370 into an output format. For example, as the output format, an image having a Y, U, V component ratio of 4: 2: 2 or 4: 2: 0 can be used, or a component ratio of R, G, B is 5: 6: Images with 5 or 8: 8: 8 can be used.

FIG. 5 is a flowchart illustrating an image signal processing process in the image signal processing apparatus 3000 including the selector 320 as an embodiment to which the present invention is applied.

Referring to FIG. 5, the selector 320 may receive a plurality of images output from the plurality of Bayer processors (S500). For example, the first image may be received from the first Bayer processor 310A and the second image may be received from the second Bayer processor 310B. The received first and second images may be images in which at least one of lens shading correction, pixel replacement, automatic white correction, and color interpolation has been performed.

The selector 320 may set an image format for image signal processing (S510). In this case, the selector 320 may further include a format setting unit (not shown). Alternatively, the selector 320 may set an image format for image signal processing by a control signal according to an external image format selection command. In this case, the selector 320 may further include a controller (not shown), and may receive a control signal according to an external image format selection command from the controller (not shown). The control signal may mean a signal specifying one of a plurality of image formats supported by the image signal processing apparatus 3000. Accordingly, the selector 320 may set an image format according to a control signal transmitted from the controller (not shown). Hereinafter, the image format will be described with reference to FIG. 6.

FIG. 6 illustrates a plurality of image formats supported by the image signal processing apparatus 3000 as an embodiment to which the present invention is applied.

Referring to FIG. 6, the image format includes (a) a non-sampling image format, (b) a row sampling image format, and (c) a column sampling image format. format) can be used. The non-sampling image format may refer to an image format for performing image signal processing on any one of a plurality of images output from a plurality of Bayer processing units. For example, when the non-sampling image format is selected, image signal processing may be performed on the first image output from the first bayer processing unit 310A, or the first output from the second bayer processing unit 310B. Image signal processing may be performed on the two images. The thermal sampling image format may refer to an image format for performing image signal processing on a thermal sampling image, that is, an image formed by down-sampling a plurality of images output from a plurality of Bayer processing units in a row direction. Can be. The row sampling image format may mean an image format for performing image signal processing on a row sampling image, that is, an image formed by down-sampling a plurality of images output from a plurality of Bayer processing units in a column direction. Can be. Hereinafter, the column sampling image and the row sampling image will be described in detail.

The thermal sampling image may include a first thermal sampling image and a second thermal sampling image. The first column sampling image may mean an image obtained by down-sampling the first image output from the first bayer processing unit 310A in a row direction, and the second column sampling image may be a second bayer processing unit ( The second image output from 310B may refer to an image down-sampled in the column direction. For example, the first column sampling image may be an image obtained by down-sampling a first image, which is a full frame, in a half frame. In this case, the first column sampling image may be composed of odd lines of the first image or even lines of the first image. The second column sampled image may also be configured in the same manner as the first column sampled image. The row sampling image may be composed of a first row sampling image and a second row sampling image, and the first row sampling image and the second row sampling image may each column the first image and the second image. It may be an image down-sampled in the direction. Similarly, the first row sampled image may be composed of odd lines or even lines of the first image, and the second column sampled image may also be configured in the same manner as the first column sampled image. Can be configured. In this case, the selector 320 may further include a sampling unit (not shown). However, the image formats supported by the image signal processing device 3000 are not limited to the four image formats, and various images such as an image in which the sampled first and second images are interlaced. Of course, the format may be supported.

The selector 320 may generate a sampling image by sampling the plurality of received images according to the image format set in the format setting unit (not shown) (S520). The sampling image may include a column sampling image and a row sampling image. For example, when the set image format is a column sampling image format or a row sampling image format, a first image output from the first bayer processor 310A and a second image output from the second bayer processor 310B. May be sampled in the column direction or the row direction, respectively, to generate a column sampling image or a row sampling image. The sampling method will be described with reference to FIG. 6, and a detailed description thereof will be omitted.

The selector 320 may output the sampling image (eg, a column sampling image or a row sampling image) to the RGB processor 330 (S530).

Meanwhile, when the image format set in the format setting unit (not shown) is a non-sampling image format, the selector 320 may output one of a plurality of images received at the S500 to the RGB processor 330. It may be (S540).

FIG. 7 is a flowchart illustrating an image signal processing process in an image signal processing apparatus 3000 having an image sensor controlled by a controller (not shown) as an embodiment to which the present invention is applied.

Referring to FIG. 7, the controller (not shown) may generate a control signal according to an external image format selection command (S700). The controller (not shown) may generate activation information for the image sensor based on the generated control signal (S710). The activation information may mean information for operating an image sensor connected to the image signal processing apparatus 3000. For example, when the generated control signal specifies a non-sampling image format, activation information for any one of a plurality of image sensors may be generated. That is, information for operating the first image sensor or information for operating the second image sensor may be generated. When the generated control signal specifies a column sampling image format or a row sampling image format, activation information for the first image sensor and the second image sensor, that is, information for operating both the first image sensor and the second image sensor, may be provided. Can be generated.

A plurality of image sensors may generate an image based on the generated activation information (S720). For example, when activation information about the first image sensor and the second image sensor is generated, both the first image sensor 1000 and the second image sensor 2000 operate, so that the first image and the second image are generated. Can be generated. Alternatively, when only activation information for the first image sensor is generated among the plurality of image sensors, only the first image may be generated by operating only the first image sensor 1000. The Bayer processor 310 may perform correction on the generated image (S730). As described above, at least one of lens shading correction, pixel replacement, automatic white correction, and color interpolation may be performed by the correction. The selector 320 may generate a sampled image by sampling the corrected image based on the control signal (S740). For example, when the control signal specifies a thermal sampling image format, the selector 320 may set the thermal sampling image format as an image format for image signal processing. The thermal sampling image may be generated by sampling the corrected first and second images in a column direction. The selector 320 may output the generated sampling image to the RGB processor 330 (S750).

On the other hand, when the control signal specifies the non-sampling image format, the selector 320 may output the corrected image to the RGB processor 330 (S760).

On the other hand, the above-described image format selection and control method can be created by a computer program. The codes and code segments that make up the program can be easily deduced by a computer programmer in the field. In addition, the program may be stored in a computer readable media, and read and executed by a computer to implement a bit rate control method. The information storage medium may include a magnetic recording medium, an optical recording medium, and a carrier wave medium.

The terms and words used in the present specification and claims should not be construed as limited to ordinary or dictionary terms and the inventor may appropriately define the concept of the term in order to best describe its invention It should be construed as meaning and concept consistent with the technical idea of the present invention. Therefore, the embodiments described in this specification and the configurations shown in the drawings are merely the most preferred embodiments of the present invention and do not represent all the technical ideas of the present invention. Therefore, It is to be understood that equivalents and modifications are possible.

Claims (15)

A plurality of Bayer processing units which perform at least one of lens shading correction, pixel replacement, automatic white correction, and color interpolation on the images output from the plurality of image sensors;
A selector configured to output an image according to an image format for image signal processing using the images output from the plurality of Bayer processors; And
An RGB processor which performs at least one of gamma correction, color correction, and wide range correction on the image output from the selection unit;
And the image format comprises a non-sampling image format, a row sampling image format, and a column sampling image format.
delete The image processing apparatus of claim 1, wherein the non-sampling image format is an image format for performing image signal processing on any one of a plurality of images output from the plurality of Bayer processing units, and the thermal sampling image format is the plurality of Bayer processing units. Is an image format for performing image signal processing on an image composed of images down-sampled from a plurality of images output from a row in a row direction, and the row sampling image format is a plurality of images output from the plurality of Bayer processing units. And an image format for performing image signal processing on an image composed of images down-sampled in the column direction. The method of claim 3, wherein the selection unit,
And a format setting unit for setting an image format for image signal processing. The method of claim 4, wherein the image format is set based on a control signal according to an external image format selection command.
And the control signal is a signal specifying one of a plurality of image formats supported by the image signal processing apparatus. The method of claim 4, wherein the selection unit,
If the set image format is a column sampling image format or a row sampling image format, the image signal processing further comprises a sampling unit for sampling a plurality of images output from the plurality of Bayer processing unit in a column direction or a row direction, respectively Device. 7. The method of claim 6, wherein when the set image format is a non-sampling image format, one of a plurality of images output from the plurality of Bayer processing units, and when the set image format is a thermal sampling image format, the plurality of Bayers Images composed of down-sampled images of the plurality of images output from the processing unit in the column direction, respectively, and when the selected image format is the row sampling image format, the plurality of images output from the plurality of Bayer processing units in the column direction, respectively. And an image signal composed of down-sampled images. In the method for signal processing a plurality of images transmitted from a plurality of image sensors to one image signal processing device,
Receiving the plurality of images from each of the plurality of image sensors;
Acquiring an image according to an image format for image signal processing using the plurality of images;
Performing at least one of gamma correction, color correction, and wide range correction on the obtained image,
And the image format comprises a non-sampling image format, a row sampling image format, and a column sampling image format.
The method of claim 8, wherein the receiving step,
And performing at least one of lens shading correction, pixel replacement, automatic white correction, and color interpolation on the plurality of images. delete The method of claim 8,
The non-sampling image format is an image format for performing image signal processing on any one of the plurality of received images, and the thermal sampling image format down-loads the received plurality of images in a row direction, respectively. An image format for performing image signal processing on an image composed of a sampled image, wherein the row sampling image format is an image signal for an image composed of an image obtained by down-sampling each of the plurality of received images in a column direction. And an image format for performing processing.
The method of claim 11, wherein the obtaining step,
And setting an image format for processing the image signal. 13. The method of claim 12,
The image format is set based on a control signal according to an external image format selection command,
And the control signal is a signal for specifying one of a plurality of image formats supported by the image signal processing apparatus. The method of claim 12, wherein the obtaining step,
And when the set image format is a column sampling image format or a row sampling image format, sampling the plurality of received images in a column direction or a row direction, respectively. 15. The method of claim 14, wherein when the set image format is a non-sampling image format, one of the plurality of received images is opened, and when the set image format is a thermal sampling image format, the plurality of received images are respectively opened. Acquiring an image composed of an image down-sampled in a direction; and if the set image format is a row sampled image format, an image composed of an image obtained by down-sampling the plurality of received images in a column direction, respectively. Image signal processing method.

Images