KR100771138B1 - Imaging device and image correcting method - Google Patents

Abstract

The present invention relates to correcting image data obtained from an imaging apparatus for efficient image compression. According to an aspect of the present invention, the difference value of the brightness component for any pixel of the image data at time t and the image data at time t-1 is calculated and compared with a threshold value to determine whether or not the motion for the pixel If there is not, an image capturing apparatus capable of changing the pixel value of the image data at time t-1 is provided. Therefore, according to the present invention, it is possible to increase the temporal motion prediction efficiency of the captured image data.

Image data, brightness component, YUV

Description

Imaging device and image correcting method

1 is a block diagram of an image photographing apparatus according to an exemplary embodiment of the present invention.

2 to 3 is image data according to an embodiment of the present invention.

4 illustrates a brightness component data bit storage format in accordance with one preferred embodiment of the present invention.

5 is a flowchart illustrating a process of storing image data at time t according to an exemplary embodiment of the present invention.

6 is a flowchart illustrating a process of comparing image data at time t and image data at time t-1 according to an exemplary embodiment of the present invention.

FIG. 7 is a table illustrating an image data update state according to a comparison between image data at time t and image data at time t-1 according to an exemplary embodiment of the present invention. FIG.

<Explanation of symbols for main parts of the drawings>

110: image sensor

115: image signal processing unit

120: first storage unit

125: image comparison unit

130: signal processing unit

150: control unit

The present invention relates to an image capturing apparatus, and more particularly, to correcting image data acquired from an image capturing apparatus for efficient image compression.

In general, image data of the external information captured by the camera due to environmental factors of the shooting environment is recognized as different image data by adding noise even if the same external information is taken. That is, even if the same external information is taken in the same place according to the position of the camera in accordance with the lighting installed in the shooting environment, it is recognized as different image data due to the change of lighting, which causes a problem of inefficient efficiency in compressing and storing the moving image. .

That is, image compression may be divided into spatial compression through adjustment of quantization coefficient through DCT transform and temporal compression through motion estimation. Even if the same external information is captured due to the addition of noise due to the surrounding environment at the time of shooting, the compression efficiency decreases because different values are stored.

Accordingly, an object of the present invention for solving the above problems is to provide an image capturing apparatus and a storage method that can increase the temporal motion prediction efficiency of the captured image data.

Another object of the present invention is to provide an image capturing apparatus and a storage method capable of storing more image data in a storage space by increasing an image compression ratio.

It is still another object of the present invention to provide an image capturing apparatus and a storage method capable of minimizing a difference between a value of a motion vector and previous image data.

It is still another object of the present invention to provide an image photographing apparatus and a storage method having the same effect as comparing image data of two frames by comparing image data of one frame.

Other objects of the present invention will be easily understood through the description of the following examples.

In order to achieve the above object, according to an aspect of the present invention, the difference value of the brightness component for any pixel of the image data at time t and the image data at time t-1 is calculated and compared with the threshold value to the pixel According to the present invention, when there is no motion by determining whether or not the motion is about, the image capturing apparatus can be changed to the pixel value of the image data at time t-1.

According to a preferred embodiment of the present invention, an image capturing apparatus having a camera, comprising: a storage unit which stores image data of a time t-1; An image sensor which forms image data corresponding to an external image to generate image data at time t; And calculating the difference value for the brightness component of any pixel from the image data at time t and the image data at time t-1, wherein the difference value of the brightness component is less than a threshold value, and the image data at time t-1 When the flag bit of the pixel is '1', the image capturing apparatus may include an image comparator which determines that there is no motion and converts the pixel value of the image data at time t to the pixel value of the image data at time t-1. .

The image data is YUV format image data in a raw format, and the least significant bit of the brightness component of the image data may be the flag bit.

The flag bit may be an information bit indicating whether a pixel moves.

The apparatus may further include an image signal processor configured to convert the image data into image data in a YUV format and set a least significant bit of a data bit string that stores the brightness component of the converted image data as a flag bit.

According to another aspect of the present invention, there is provided a method for acquiring and correcting image data at time t in an image photographing apparatus having a camera.

According to a preferred embodiment of the present invention, in a method for acquiring and correcting image data at time t in an image capturing apparatus having a camera, (a) image data corresponding to an external image is imaged at time t. Generating a; (b) reading image data of the time point t-1 from the storage unit; (c) calculating a difference value between brightness components of any pixel of the image data at time t and the image data at time t-1; (d) comparing the difference between the brightness components with a threshold to determine whether there is motion; And (e) if it is determined that there is no motion, converting the pixel value of the image data at time t to the pixel value of the image data at time t-1.

Step (d) may include determining whether a difference value of the brightness components is greater than or equal to a threshold value; And extracting and analyzing flag bits of the pixel.

Converting the image data into image data in YUV format; And setting the least significant bit of the data bit string of the brightness component of the converted image data as a flag bit.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

In the following description, the same or corresponding elements will be described with the same reference numerals regardless of the reference numerals in the description with reference to the accompanying drawings.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of an image capturing apparatus according to a preferred embodiment of the present invention, Figures 2 to 3 are image data according to a preferred embodiment of the present invention, Figure 4 is a preferred embodiment of the present invention FIG. 7 is a table illustrating a brightness component data bit storage format according to an exemplary embodiment of the present invention, and FIG. 7 is a table illustrating an image data update state according to a comparison of image data at time t and image data at time t-1 according to an exemplary embodiment of the present invention. .

As shown in FIG. 1, the image capturing apparatus 100 according to the present invention includes an image sensor 110, an image signal processor 115, a first storage unit 120, an image comparator 125, and a signal processor ( 130, a second storage unit 135, an input unit 140, a display unit 145, and a controller 150.

The image sensor 110 is an imaging device in which an optical image corresponding to an external image is formed from a lens unit. The image sensor 110 converts a light signal related to the optical image into an analog signal, which is an electrical signal, and converts it into an A / D included in the image sensor 110. Through a unit (not shown) to convert to a digital signal. Here, the lens unit is provided at the front end of the image sensor 110 is a means for receiving the object.

 For example, the image sensor 110 may be referred to as a charge coupled device (CCD: hereinafter referred to as "CCD") or a complementary metal oxide semiconductor (CMOS) hereinafter referred to as "CMOS". Can be implemented.

For example, the image sensor 110 may output Bayer data (eg, RGB Bayer data) of each pixel collected by converting the light signal to the image signal processor 115. In the image sensor 110, pixels having a filter that recognizes RGB, respectively, are disposed, and the RGB filter is disposed according to a Bayer pattern such that 50% of green and 25% of red and blue are sensitive to human eyes in the total number of pixels. Can be. The data output from the image sensor 110 having such a Bayer pattern is referred to as Bayer data, and the image sensor 110 refers to such Bayer data as an image signal processor (ISP), for example, an 8-bit RGB Bayer. Can be output as data.

That is, the image sensor 110 captures an external image, generates image data, and transmits the image data to the image signal processor 115.

The image signal processor 115 performs a preprocessing process such as error elimination, distortion correction, etc., and converts the image data of the RGB format input from the image sensor 110 into a YUV format.

For example, the image signal processor 115 may include interpolation, auto white balance, gamma correction, and brightness correction on image data acquired through the image sensor 110. ), Color space correction, etc., to convert to YUV format. For example, the image signal processor 115 may convert the image data of the RGB format input from the image sensor 110 into the YUV format using Equation 1 below.

Y = 0.3R + 0.59G + 0.11B

0.493U = (BY)

0.493

0.877V = (RY)

0.877

or

Y = 0.299fR + 0.587fG + 0.114fB

U = -0.168fR-0.3313fG + 0.5fB + 128

V = 0.5fR-0.4187fG-0.0813fB + 128

For example, the image signal processor 115 converts the image data of the RGB format input from the image sensor 110 into a YUV format such as 4: 4: 4, 4: 2: 2, 4: 2: 0, and the like. It may be transmitted to the comparator 125. Here, since the conversion method to the YUV format such as 4: 4: 4, 4: 2: 2, 4: 2: 0, etc. is obvious to those skilled in the art, a detailed description thereof will be omitted.

In addition, the image signal processor 115 may remove a high frequency component by performing a low pass filter (LPF) on the image data input from the image sensor 110.

In addition, the image signal processor 115 may allocate a least significant bit (LSB) of the Y component of the image data converted into the YUV format as a flag bit.

In the present specification, the "flag bit" is defined as a bit for indicating the state information of a pixel determined to have motion by comparing the image data at time t and the image data at time t-1.

Accordingly, the image signal processing unit 115 sets the least significant bit LSB of the data bit representing the brightness component (i.e., Y component) for each pixel of the image data to a default value (e.g., "0"). The data may be generated and transferred to the image comparator 125.

For example, assume that the image signal processor 115 expresses the brightness component (Y component) data in 8 bits. As shown in FIG. 4, the image signal processor 115 may set the lowest bit (0th bit) as a flag bit and set the default value.

In addition, in the present specification, the image sensor 110 and the image signal processing unit 115 have been described on the assumption that they exist as independent components, but the image signal processing unit 115 is applied to the image sensor 110 according to an implementation method. It may be included and exist as a component.

The first storage unit 120 temporarily stores YUV data in a raw format output from the image signal processing unit 115. For example, the current time point is called t time point for convenience, the previous time point is called t-1 time point, and the time point after t time point is t + 1 time point. In this case, when it is assumed that the image data captured by the image sensor 110 and then converted into YUV format by the image signal processor 115 are image data at time t, the first storage unit 120 at time t-1 Stores YUV video data in the low-format format. Here, the first storage unit 120 is a storage medium capable of temporarily storing, for example, low-format YUV image data of less than three frames, and includes a dynamic RAM (DRAM), a synchronous dynamic RAM (SDRAM), and an EDORAM. extended data output RAM).

In the present specification, it will be assumed that the first storage unit 120 has a storage space capable of storing image data of at least three frames.

As shown in FIGS. 2 to 3, it can be seen that the 210 region and the 310 region are fixed positions or background portions without change, and only the 220 region is changed to the 320 region. However, noise is added due to factors of the surrounding environment at the moment of being imaged by the image sensor 110, so that the 210 region and the 310 region also include different image data values. As a result, image compression efficiency is reduced. Therefore, the same image area is obtained from the two image data and then corrected to increase the image compression efficiency. This will be clearly understood by the following description.

The image comparator 125 compares the image data at time t inputted from the image signal processor 115 with the image data at time t-1 stored in the first storage unit and compares the brightness components (Y components) for each pixel. If the value is equal to or greater than the value, the pixel value of the image data at time t is converted into the pixel value of the image data at time t-1.

That is, the image comparator 125 calculates the difference between the brightness components for each pixel of the image data at time t and the image data at time t-1 and is equal to or greater than a predetermined threshold value and the flag bit (that is, the least significant bit of the Y component). ) To determine whether the corresponding pixel moves, and if there is no motion, converts the corresponding pixel value of the image data at time t to the pixel value of the image data at time t-1. First, conversion of image data at time t will be described in detail with reference to FIG. 7.

For example, the image comparator 125 reads image data of the time point t-1 stored in the first storage unit 120 under the control of the controller 150, and then brightness of each pixel with the image data of the time point t. Calculate the absolute value of the difference between the components. The image comparison unit 125 may determine that there is motion if the absolute value of the difference between the brightness components is greater than or equal to the threshold value, and may not update the image data at time t. However, if the absolute value of the difference between the brightness components is less than the threshold value, it is determined that there is no motion and the pixel value of the image data at time t is changed to the image data at time t-1. Also, the image comparator 125 may update the flag bits of each pixel of the image data at time t-1 by analyzing the flag bits of each pixel of the image data at time t-1. A state table for this is illustrated in FIG. 7, which will be described in detail below with reference to FIG. 6.

In this way, the image comparison unit 125 compares the image data of the row format of the previous view with the image data of the current view and changes the pixel value corresponding to the pixel without movement into the pixel value of the image data of the previous view. By doing so, there is an advantage in that the compression efficiency at the time of image encoding in the signal processor 130 can be increased.

The signal processor 130 converts the image data input from the image comparator 125 into a predetermined image format (for example, MPEG-1, MPEG-2, MPEG-4, AVC, H.263, H.264, avi). , divx, etc.) to be transmitted to the controller 150. That is, the signal processor 130 may encode or decode the input image data in a predetermined image format. In addition, the signal processor 130 may enlarge, reduce, or rotate the image.

In addition, the signal processor 130 may convert the YUV format image data into the RGB format in order to output data to the display unit 145 under the control of the controller 150. For example, the signal processor 130 may convert image data in RGB format using Equation 2 below.

R = Y + 0.956U + 0.621V

G = Y + 0.272U + 0.647V

B = Y + 1.1061U + 1.703 V

For example, assuming that the data line for outputting data to the display unit 145 is a 16-bit data line, the signal processor 130 converts YUV image data into 5-bit, 6-bit, and 5-bit RGB data, respectively. The control unit 150 may transmit the output to the display unit 145 under the control of the controller 150.

The second storage unit 135 stores an operation program, a preset algorithm, and the like of the image capturing apparatus 100 according to the present invention. Here, the operation program may perform a function of temporarily storing temporary data generated while executing programs as well as a program for controlling the overall operation of the image capturing apparatus 100 and a program for image processing. In addition, the second storage unit 135 may store the image data encoded by the signal processing unit 130.

The input unit 140 is a means for receiving a control command, a parameter, etc. for controlling the operation of the image capturing apparatus 100 according to the present invention from the user. For example, the input unit may be implemented as one or more function keys (for example, a menu key, a confirmation key, etc.) or may be implemented in the form of a touch screen. For example, the input unit 140 may receive a mode selection command (for example, an interview mode) of the image capturing apparatus 100 from the user and transmit the received mode selection command (eg, an interview mode) to the controller 150.

The display unit 145 is a means for outputting image data, text data, numeric data, etc. under the control of the controller 150. For example, the display unit 145 may be a liquid crystal display (LCD).

The controller 150 may include internal components (eg, the image sensor 110, the image signal processor 115, the storage unit, the input unit 140, and the display unit) of the image capturing apparatus 100 according to the present invention. 145), etc.).

5 is a flowchart illustrating a process of storing image data at time t according to an exemplary embodiment of the present invention, and FIG. 6 is image data at time t and an image at time t-1 according to an exemplary embodiment of the present invention. A flowchart detailing the process of comparing data. Hereinafter, the image data at time t-1 is converted and stored in the first storage unit 120 by comparing the image data at time t-2 with brightness components and flag bits of each pixel. It is assumed that the image input through the image data at time t will be described. In addition, it is assumed that a selection mode (for example, an interview mode) is set by a user through a function button provided in the image capturing apparatus 100, and an application corresponding thereto is running.

In operation 510, the image sensor 110 captures an external image, generates image data, and transmits the image data to the image signal processor 115.

For example, it is assumed that a function is set to receive a mode selection command corresponding to, for example, an interview mode from a user through the input unit 140 provided in the image capturing apparatus 100 and operate according to the interview mode.

The image sensor 110 may generate an image data by capturing an external image at regular time intervals and transmit the generated image data to the image signal processor 115. As described above, the image sensor 110 may transfer the image data in the RGB format to the image signal processor 115. Hereinafter, the image data captured and generated by the image sensor 110 at the present time will be referred to as image data at time t.

In operation 515, the image signal processor 115 performs pre-processing such as error elimination and distortion correction on the RGB format image data input from the image sensor 110, and converts the image data into YUV format image data. To pass.

For example, assume that the image signal processing unit 115 stores the brightness component (Y component) as, for example, 8 bits. And, assume that the default value of the flag bit is '0', for example. The image signal processor 115 converts the image data of the RGB format input from the image sensor 110 into the image data of the YUV format. Here, the image signal processor 115 may set the least significant bit of the data bit of the brightness component (Y component) as a flag bit, and store the brightness component value in seven bits other than the flag bit. Here, the flag bit may be an information bit indicating whether the corresponding pixel moves. The brightness component data format is illustrated in FIG. 4.

In operation 520, the image comparator 125 reads image data at time t-1 from the first storage unit 120 under the control of the controller 150, and inputs image data at time t input from the image signal processor 115. If the comparison is greater than or equal to the threshold, the pixel value of the image data at time t is changed to the pixel value of the image data at time t-1.

First, for convenience of understanding and description, an operation process of the image comparison unit 125 will be described in detail with reference to FIG. 6.

Here, the image data at time t-1 read by the image comparator 125 from the first storage unit 120 is YUV-type raw data, and the image data at time t input from the image signal processor 115 is also YUV. The raw data of the format. It is assumed that each of the two image data (that is, the image data at time t-1 and the image data at time t) that are compared by the image comparator 125 are image data of the same format. Of course, each may have a different format, and in this case, the image comparator 125 may convert the same image format under the control of the controller 150.

In operation 610, the image comparator 125 may determine a brightness component value of any one pixel of the image data at time t and a brightness component of a pixel at the same position as the pixel of the image data at time t of the image data at time t-1. The difference value is calculated and the absolute value (hereinafter referred to as "difference value of brightness component") is taken. The image comparator 125 determines whether the calculated difference value of the brightness component is equal to or greater than a predetermined threshold value and the flag bit of the corresponding pixel of the image data at time t-1 is '0'.

If the difference between the brightness components is greater than or equal to the threshold and the flag bit is '0', the image comparator 125 determines that there is motion in step 615 and does not change the corresponding pixel value.

However, if the difference value of the brightness component is greater than or equal to the threshold value and the flag bit is not '0', the image comparator 125 determines whether the difference value of the brightness component is greater than or equal to the threshold value and the flag bit is '1' in step 620. Determine whether or not.

If the difference between the brightness components is greater than or equal to the threshold and the flag bit is '1', the image comparator 125 determines that there is motion in step 625 and does not change the corresponding pixel value of the image data at time t.

However, if the difference value of the brightness component is greater than or equal to the threshold and the flag bit is not '1', the image comparator 125 determines whether the difference value of the brightness component is less than the threshold and the flag bit is '0' in step 630. Determine whether or not.

If the difference value of the brightness component is less than the threshold value and the flag bit is '0', the image comparator 125 determines that there is no motion in step 635 and determines that the brightness component of the corresponding pixel value of the image data at time t is not present. The data value does not change, and since there is no movement, the flag bit is changed to '1' accordingly.

However, if the difference value of the brightness component is less than the threshold value and the flag bit is '0', the image comparator 125 determines whether the difference value of the brightness component is less than the threshold value and the flag bit is '1' in step 640. Determine whether or not.

If the difference value of the brightness component is less than the threshold value and the flag bit is '1', the image comparator 125 determines that the corresponding pixel of the image data at time t has no motion at step 645 and the image at time t-1 Change the value of the data's brightness component. The flag bit may also be set to '1' to recognize that there is no motion.

As a result, by using the flag bit, the image data at time t has the same effect as comparing two frame image data in one comparison. In this way, by adding flag bits, which are information bits corresponding to motion, to image data of one frame, the same effect as that of image data of two frames can be obtained by comparing with image data of one frame. In addition, since the first storage unit 120 needs only a storage space capable of storing only one frame of image data (that is, the first storage unit 120 stores the image data at time t-2 in view of the image data at time t). Since there is no need to store in the storage space can be used efficiently, there is an advantage to reduce the storage space.

Referring to FIG. 5 again, in step 525, the image comparator 125 calculates a difference value between the brightness components of the image data at time t and the image data at time t-1 and compares it with a predetermined threshold to determine whether there is motion. do.

If it is determined that there is no motion, the image comparator 125 may change the pixel value of the image data at time t to the pixel value of the image data at time t-1 in step 530.

However, if it is determined that there is motion, the process proceeds to step 535 without changing the image data at time t.

In operation 535, the image comparator 125 calculates a difference between brightness components of all pixels of the image data at time t and the image data at time t-1 and compares the result with a predetermined threshold.

If all the pixels have been compared, the image comparator 125 temporarily stores the changed image data at time t in the first storage unit 120 and transmits the image data to the signal processor 130 at step 540. The signal processor 130 receiving the image data at time t from the image comparator 125 encodes the image data using a predetermined image compression encoding algorithm. Of course, in the case of a video file, the signal processor 130 may compress the image data after receiving all the frames. The signal processor 130 may store the image data compression-coded by a predetermined algorithm (eg, H.263, MPEG-4, etc.) in the second storage unit 135 through the controller 150. When a command corresponding to reproduction of the corresponding video file is input by the user through the input unit 140, the corresponding image data is read from the second storage unit 135 under the control of the controller 150, and the signal processor 130 is read. ) Can be decoded by a predetermined algorithm (eg, H.263, MPEG-4) and transmitted to the display unit 145 under the control of the controller 150 to be output.

For example, when the data line for outputting data to the display unit 145 is a 16-bit data line, the signal processor 130 decodes the input image data read and input from the second storage unit 135 and each 5 bits. , 6-bit and 5-bit RGB signals may be converted and output to the display unit 145 under the control of the controller 150.

However, if all the pixels have not been compared, repeat steps 525 to 535.

As described above, by providing the image capturing apparatus and the storage method according to the present invention, it is possible to increase the temporal motion prediction efficiency of the captured image data.

In addition, the present invention has the effect of increasing the image compression ratio to store more image data in the storage space.

In addition, the present invention has the effect of minimizing the difference between the value of the motion vector and the previous image data.

Still another object of the present invention may have the same effect as comparing image data of two frames by comparing image data of one frame.

Although the above has been described with reference to a preferred embodiment of the present invention, those skilled in the art to which the present invention pertains without departing from the spirit and scope of the present invention as set forth in the claims below It will be appreciated that modifications and variations can be made.

Claims (7)

In the video photographing device having a camera, a storage unit which stores image data at time t-1; An image sensor which forms image data corresponding to an external image to generate image data at time t; And The difference value of the brightness component of any pixel is calculated from the image data at time t and the image data at time t-1, and the difference value of the brightness component is less than a threshold value, and the pixel of the image data at time t-1 And an image comparison unit for determining that there is no motion when the flag bit of the flag bit is '1' and converting the pixel value of the image data at time t into the pixel value of the image data at time t-1. According to claim 1, And the image data is YUV format image data in a raw format, and the least significant bit of the brightness component of the image data is the flag bit. The method of claim 2, And the flag bit is an information bit indicating whether a pixel moves. According to claim 1, And an image signal processor configured to convert the image data into image data in a YUV format, and set a least significant bit of a data bit string that stores brightness components of the converted image data as flag bits. Claims [1] A method of acquiring and correcting image data at time t in a video photographing apparatus having a camera, the method comprising: (a) forming image data corresponding to an external image by generating light corresponding to an external image; (b) reading previously stored image data of a time point t-1; (c) calculating a difference value between brightness components of any pixel of the image data at time t and the image data at time t-1; (d) comparing the difference between the brightness components with a threshold to determine whether there is motion; And and (e) if it is determined that there is no motion, converting the pixel value of the image data at time t to the pixel value of the image data at time t-1. The method of claim 5, Step (d) may include determining whether a difference value of the brightness components is greater than or equal to a threshold value; And And extracting and analyzing flag bits of the pixel. The method of claim 5, Converting the image data into image data in YUV format; And And setting a least significant bit of a data bit string of the brightness component of the converted image data as a flag bit.

Images