TWI514321B - Video image process device with function of preventing shake and the method thereof - Google Patents

Description

Anti-shake image processing device and method

The present invention relates to an anti-shake image processing apparatus, and more particularly to an image processing apparatus and method for using a different access architecture to enable a motion vector to directly correct a Bell pattern image.

With the popularity of handheld photographic devices, most people often have the opportunity to use these devices to record images in their daily lives. What is disturbing is that most people do not have professional photography training, often because of the need for long-time shooting. The shaking causes the out-of-focus and blur of the captured image, which makes the captured image quality poor.

There are many ways to improve this phenomenon in the prior art, which can be roughly divided into mechanical compensation and electronic compensation. The latter mostly compares the difference between two captured images or the change of a specific image area and the surrounding background to judge the two images. Is there any sway caused by the shaking of the hand, and the object to be compared is the image processed by the actual object through the image processing element after the image is captured by the photosensitive element, in other words, the image that is compared is actually output to The image of a display (such as an LCD), or an image that has been converted to a YUV domain, is very accurate and detailed, but the amount of data is therefore very large.

For example, please refer to FIG. 1 . FIG. 1 is a block diagram of an output circuit 100 of an image processing device of the prior art. The YUV domain image signal input/output circuit 100 is first stored in the image buffer 110. The buffer 110 then transmits the signal to the output frame buffer 120. In the Full HD specification, the number of pixels of one output frame is 1920×1080. In actual application, the number of pixels captured by the output frame buffer 120 is slightly lower. It is larger than the actual number of output frame pixels to compensate for the motion vector when the jitter condition is detected. For example, as shown in FIG. 1, the number of pixels captured by the output frame buffer 120 is (1920+64) x (1080). +64), the dotted line portion of the output frame buffer 120 is the actual output frame 121, and 32 pixels are taken around the actual output frame 121 as compensation motion vectors. The output frame buffer 120 then transmits the image data of the output frame to the output interface 130. The movement determining unit 140 also transmits the motion vector information of the frame to the output interface 130, and finally outputs the corrected interface frame 130. Screen signal output.

Please refer to the 2a and 2b diagrams. The 2a is a schematic diagram of the output frame pixel capture range of the output frame buffer 120 when no jitter occurs, and the 2b is the output frame buffer 120 in the case of a jitter. At the time of the output frame pixel capture range diagram, if there is no jitter, the camera lens alignment image is naturally the desired picture, as shown in Figure 2a, the output frame pixel capture range is in the output map In the center of the total pixel range of the frame buffer 120, if a jitter occurs at this time, for example, as shown in FIG. 2b, after the camera shake, the entire camera lens moves to the lower right, and thus the frame image received by the frame buffer 120 is output. The data is moved from the original frame image to the lower right, that is, from the frame of the original output frame buffer 120 indicated by the broken line in FIG. 2b to the frame of the output frame buffer 120', and if so, Taking the center of the total pixel range of the output frame buffer 120', the picture shifted by the jitter is captured. In fact, the correct captured picture should still be the output frame 121 before the shaking, so the movement The determining unit 140 detects the cause After the motion vector (dX, dY) is caused, the output frame is moved from the center of the total pixel range of the output frame buffer 120' to the upper left, that is, the correction (-dX, -dY), so that the output frame pixel 撷The range is still the output frame 121 before the shaking, so the image shake caused by the hand shake is cancelled.

However, the frame image data processed by the output frame buffer 120 is stored in the YUV domain image signal format. In the Full HD specification, one frame must be stored (1920+64) x (1080+64) pixels. The YUV domain image signal data, together with the motion vector compensation, needs to store several consecutive frame image data before and after, and the required memory space is very large, which causes the cost and the limitation of the size of the photographic device to be reduced. Therefore, how to provide an image processing apparatus for judging jitter and compensating for the impact of the jitter causes the amount of data to be processed to be reduced, which is the main object of the present invention, and the less the amount of data to be processed, the faster the processing speed. The less memory required, the positive impact on the cost and size of the handheld camera.

In view of the above problems, an object of the present invention is to provide an image processing apparatus and a processing method that enable a motion vector to directly correct a Bell pattern image by using an image access architecture different from the prior art.

In order to achieve the above object, the present invention discloses an anti-shake image processing apparatus, comprising: a sampling unit for sampling a primary color in a Bayer Pattern image and converting the primary color into a primary color data; The unit is coupled to the sampling unit for comparing the front and rear frames according to the primary color data and generating a motion vector; a resolution storage unit coupled to the sampling unit for storing a And the image processing unit is coupled to the frame resolution storage unit and the movement determining unit for correcting the Bell pattern image according to the motion vector.

The invention further provides an anti-shake image processing method, which comprises: sampling a primary color in a Bell pattern image and converting the primary color into a primary color data; comparing the front and rear frames according to the primary color data and generating a motion vector; The primary color data of the two frames generates a motion vector; stores the Bell pattern image of a frame; and corrects the Bell pattern image according to the motion vector.

By sampling one of the three primary colors directly in the Bell pattern image at the front end of the image processing flow as a basis for detecting whether the image in the two consecutive frames moves, the amount of data to be processed by the movement determination unit is greatly reduced, and another A memory unit stores the Bell pattern image of two consecutive frames, and directly corrects the Bell pattern image by using the motion vector and then converts it into a YUV domain image signal used by a general display device, thereby saving the memory required to perform the motion vector correction image in the YUV domain. In terms of Full HD, it can save about (1920+64) x (1080+64) x 2 pixels of memory space.

Please refer to FIG. 3 , which is a block diagram of a first embodiment of the image processing apparatus 300 for preventing jitter in the present invention. The embodiment includes a sensing unit 310 , a sampling unit 320 , a movement determining unit 330 , and a frame . The resolution storage unit 340 and the image processing unit 350, the sensing unit 310 includes components such as a CCD element and a filter, and captures an external image to form a Bayer Pattern image, which is also an external image to be captured. It is decomposed into a plurality of pixels, and each pixel contains three primary color data of red, green and blue (RGB). After the primary color data is converted by the sampling unit 320, one of the primary colors is sampled and converted into a primary color data by a Gamma conversion process. Please note that in principle, any of the three primary colors of red, green and blue can be sampled to achieve the object of the present invention, but since the naked eye has the strongest feeling of green, the contrast distribution of green brightness is also the most consistent with the brightness contrast of the actual image. The sampling green effect is the best, but the implementation range is not limited to green. Next, the corresponding data of the pixel and the converted green primary color brightness is transmitted to the movement determining unit 330, and the movement determining unit 330 stores the primary color data of the previous frame first, and waits until the primary color data of the next frame is received. When the motion determining unit 330 compares the primary color data of the corresponding pixels in the two consecutive frames to obtain a motion vector, for example, the primary color data values of the pixels in a certain block of a frame are the same as the previous frame. If the primary color data values of the pixels in the other block on the left side of the block are the same, it can be inferred that the photographic lens is swaying to the right during the image capturing time of the two frames. There are many ways to estimate the motion vector, and will not be described here. The sampling unit 320 stores the Bell pattern image received from the sensing unit 310 in the frame resolution storage unit 340, and the frame resolution storage unit 340 transmits the Bell pattern image of a frame to the image processing unit 350. The motion determining unit 330 also transmits the motion vector data corresponding to the frame to the image processing unit 350, and the image processing unit 350 directly corrects the Bell pattern by using the motion vector value. For example, the modified BB domain image of the Bell pattern image is converted into the image signal of the YUV domain, and the pixel value, color adjustment, image brightness, contrast, color saturation, etc. required for the image pixel value to be enlarged into the back end display. Image processing, since the number of pixel data required for the RGB domain signal of the Bell pattern image of the frame is much smaller than the number of pixel data required after the same frame is converted into the YUV domain image signal, the present invention proposes to convert the Bell pattern image into The YUV domain image signal is completed by the motion vector detection and compensation mechanism. The amount of data to be stored is much smaller than that required by the conventional YUV domain image signal for motion vector detection and compensation, so the speed is faster. The required memory capacity is also smaller.

Please refer to FIG. 4 , which is a block diagram of a second embodiment of the anti-shake image processing apparatus 400 of the present invention, wherein the sensing unit 310, the sampling unit 320 and the image processing unit 350 are the same as those in the first embodiment. The function of the same name is the same, and is not described again. The movement determining unit 430 includes three frame buffers, and the frame resolution storage unit 440 includes two storage units, and the actual operation situation is; the sampling unit 320 will be the first frame. The Bell pattern image data is stored in the first storage unit 442 in the frame resolution storage unit 440, and the Bell pattern image is sampled into a primary color data, and the primary color data is stored in the first frame buffer in the movement determining unit 430. The sampling unit 320 then stores a continuous second frame of the Bell pattern image data into the second storage unit 444 in the frame resolution storage unit 440, and samples the Bell pattern image into a primary color data. The primary color data is stored in the second frame buffer 434 in the movement determining unit 430; at this time, the movement determining unit 430 is configured according to the first frame original color stored in the first frame buffer 432. Comparing with the second frame primary color data stored in the second frame buffer 434 to generate a first motion vector; the first motion vector is transmitted to the first frame Bell pattern image stored by the first storage unit 442 to The image processing unit 350 corrects the jitter phenomenon of the first frame, and a continuous third frame of the Bell pattern image is sampled into a primary color data and then stored in the third frame buffer in the movement determining unit 430. The motion determining unit 430 generates a second motion vector according to the second frame primary color data stored in the second frame buffer 434 and the second frame primary color data stored in the third frame buffer 436. The second motion vector is transmitted to the image processing unit 350 along with the second frame Bell pattern image stored by the second storage unit 444 to correct the jitter phenomenon of the second frame, and the third frame of the Bell pattern image data is stored in the frame. The first storage unit 442 in the resolution storage unit 440, the fourth frame of the Bell pattern image is sampled into a primary color data, and the primary color data is stored in the first frame buffer in the movement determining unit 430. 434. In this way, the original color data of the front and rear frames are used to correct the Bell pattern image data of the front frame. Please note that although the three frame buffers and two storage units are taken as an example, the actual application may be The need for adjustments in accordance with the principles of the present invention is still within the scope of the present disclosure.

Please refer to FIG. 5 , which is a block diagram of a third embodiment of the image processing apparatus 500 for preventing shaking in the present invention, wherein the sensing unit 310, the sampling unit 320, the frame resolution storage unit 440, and the respective portions thereof The included functional components are as described above, and the movement determining unit 530 includes a movement determiner 534, a motion vector generator 536, and a motion vector buffer 538, in addition to the above-described frame buffer group 532, and a motion determiner. 534 compares the continuous frame primary color data stored in the frame buffer group to generate a comparison value and inputs the motion vector generator 536. The motion vector generator 536 includes a motion vector table, which can be obtained by looking up the table to obtain different comparison values. The moving vector size and direction, the initial motion vector can be adjusted by the image stabilization analyzer and the combined motion vector of a frame is input and input into the motion vector buffer 538. Finally, the image processing unit 550 outputs the frame image. The sum vector motion vector of the frame is read from the motion vector buffer 538 and the final output image is corrected, and the image processing unit 550 includes a shadow. The signal processor 551, a scaler 552, an output buffer 554, an output buffer controller 553, and an output circuit 555, the frame resolution storage unit 440 inputs the Bell pattern data of the storage frame to the image. After the processing unit 550, the image signal processor 551 converts the Bell pattern image into a YUV domain image signal that can be recognized by a general display device. Generally, the data amount of the YUV domain image signal is much larger than that of the Bell pattern image, and the image signal processing is performed. The 551 also performs some image effects such as color, saturation, contrast, etc. In order to meet the requirements of different resolutions, the image signal processor 551 inputs the YUV domain image signal into the scaler 552, and the scaler 552 utilizes such as interpolation. Wait for the image resolution to be adjusted to an appropriate size, and appropriately capture a certain range of pixels as an output frame according to the indication of the motion vector to compensate for the screen offset caused by the jitter. The processed image frame signal is not directly output, but is first stored in the output buffer 554. The output buffer 554 is controlled by the output buffer controller 553. When the buffer controller 553 controls the output buffer 554 to output an image frame signal. After the output circuit 555, the image signal is output to the display device or other multimedia processing device via the output circuit 555.

The above is only the preferred embodiment of the present invention, and the equivalent variations and modifications of the scope of the invention and the description of the invention are not intended to limit the scope of the invention. The scope of the invention patent may cover.

100. . . Output circuit

110. . . Image buffer

120, 120’. . . Output frame buffer

121. . . Output frame

130. . . Output interface

140. . . Mobile judgment unit

300, 400, 500. . . Image processing device

310. . . Sensing unit

320. . . Sampling unit

330, 430, 530. . . Mobile judgment unit

340, 440. . . Frame resolution storage unit

350, 550. . . Image processing unit

432, 434, 436. . . Frame buffer

442, 444. . . Storage unit

532. . . Frame buffer group

534. . . Mobile judger

536. . . Motion vector generator

538. . . Moving vector buffer

551. . . Image signal processor

552. . . Scaler

553. . . Output buffer controller

554. . . Output buffer

555. . . Output circuit

Figure 1 is a block diagram of an output circuit of a conventional image processing device.

Figure 2a is a schematic diagram of the pixel capture range of the output frame of the output frame buffer when no jitter occurs.

Figure 2b is a schematic diagram of the pixel capture range of the output frame of the output frame buffer in the event of a jitter.

FIG. 3 is a block diagram showing a first embodiment of an image processing apparatus capable of preventing shaking in the present invention.

Fig. 4 is a block diagram showing a second embodiment of an image processing apparatus capable of preventing shaking in the present invention.

FIG. 5 is a block diagram showing a third embodiment of the image processing apparatus capable of preventing shaking in the present invention.

310. . . Sensing unit

320. . . Sampling unit

440. . . Frame resolution storage unit

500. . . Anti-shake image processing device

530. . . Mobile judgment unit

532. . . Frame buffer group

534. . . Mobile judger

536. . . Motion vector generator

538. . . Moving vector buffer

550. . . Image processing unit

551. . . Image signal processor

552. . . Scaler

553. . . Output buffer controller

554. . . Output buffer

555. . . Output circuit

Claims (10)

An anti-shake image processing device includes: a sampling unit for sampling a primary color in a Bayer Pattern image and converting the primary color into a primary color data; a movement determining unit coupled to the The sampling unit stores the primary color data for comparing the front and rear frames according to the primary color data and generating a motion vector; a resolution storage unit coupled to the sampling unit for storing one a plurality of pixels of the image of the Bell pattern, and each pixel includes a plurality of primary colors; and an image processing unit coupled to the frame resolution storage unit and the movement determining unit for using the primary color data The generated motion vector corrects the primary colors of the pixels of the Bell pattern image. The anti-shake image processing device of claim 1, further comprising a sensing unit for generating a Bell pattern image. The anti-shake image processing device of claim 1, wherein the sampling unit comprises a conversion unit for converting the primary color in the Bell pattern into a primary color data by using a gamma conversion. The anti-shake image processing device of claim 1, wherein the movement judging unit comprises: a set of frame buffers for storing the primary color data of the plurality of frames; A motion vector generator is coupled to the set of frame buffers for comparing the primary color data of a first frame and a second frame in the set of frame buffers, and generating the motion vector. The anti-shake image processing device of claim 1, wherein the frame resolution storage unit stores a preset resolution pixel number and a reserved space pixel number of a first frame. And one of the second frames, the preset resolution pixel number plus the pixel number of the reserved space pixel number. The anti-shake image processing apparatus according to any one of claims 4 to 5, wherein the image processing unit corrects the first frame of the bell according to the motion vector generated by the primary color data. The primary colors of the pixels of the pattern. The anti-shake image processing apparatus according to any one of claims 4 to 5, wherein the set of frame buffers and the frame resolution storage unit are temporary registers. The anti-shake image processing device of claim 1, wherein the image processing unit comprises: an image signal processor for converting the Bell pattern image into one image signal of the YUV domain; a scaler coupled to the image signal processor for converting the image signal into a desired pixel ratio and capturing the frame according to the correction vector a suitable pixel area and outputting an image frame signal; and an output interface controller coupled to the scaler for outputting the image frame signal. An anti-shake image processing method includes: sampling a primary color in a Bayer Pattern image and converting the primary color into a primary color data; storing the primary color data, and comparing the front and rear frames according to the primary color data ( And generating a motion vector; storing the Bell pattern image of a frame; and correcting a plurality of primary colors of the plurality of pixels of the Bell pattern image according to the motion vector generated by the primary color data. The anti-shake image processing method of claim 9, wherein the step of storing the image of the bell pattern of the frame stores a first frame and a preset resolution pixel number plus one The number of reserved space pixels and the number of preset resolution pixels of one of the second frames plus the pixel data of the number of reserved space pixels.