TWI576789B - Image processing method, non-transitory computer-readable storage medium and electrical device thereof - Google Patents

Description

Image processing method, non-transitory computer readable recording medium and electronic device

This case is related to an image processing method, and in particular to an image processing method applied to a low light source environment.

At present, there are many electronic products equipped with camera devices, such as mobile phones, tablet computers, notebook computers, etc., which can be used to capture images. However, in low light source environments, the images captured by such cameras will be dark and have many noises.

This case provides an image processing method. According to an embodiment of the present invention, an image processing method includes: acquiring at least one original image; performing a pixel merging process on at least one original image, wherein the pixel merging process combines a plurality of adjacent pixel signals in the original image into a new pixel signal. For generating a first image having a new pixel signal; continuously obtaining a plurality of first images generated by the pixel combining process; and performing an over-resolution reconstruction process to cause the first image to be generated by an interpolation method Two images.

The present invention provides a non-transitory computer readable recording medium. According to an embodiment of the present invention, a non-transitory computer readable recording medium stores a computer program instruction, and the computer program instruction performs an image processing after loading an electronic device. The image processing method includes: acquiring at least one original image; performing a pixel merging process on the at least one original image, wherein the pixel merging process combines the plurality of adjacent pixel signals in the original image into a new pixel signal to generate a first image of the new pixel signal; successively obtaining a plurality of first images generated by the pixel binning process; and performing a super-resolution reconstruction process to cause the first image to generate a second image by an interpolation method.

The present invention provides an electronic device, comprising an image capturing unit, wherein the image capturing unit outputs an original image or combines a plurality of adjacent pixel signals in the original image into a new pixel signal to generate the new pixel signal. a first image; and a processor connected to the image capturing unit and obtaining a plurality of successively generated first images to cause the first image to generate a second image through an interpolation method.

In summary, by applying an embodiment of the present invention, the brightness of the photo is increased by pixel combination, and then the image is reconstructed by super-resolution to restore the photo to the original resolution, so that the image can be obtained in a low light source environment. Photos that are bright enough without sacrificing image resolution

In the following, a plurality of embodiments of the present invention will be disclosed in the drawings. For the sake of clarity, a number of practical details will be described in the following description. However, it should be understood that these practical details are not applied to limit the case. That is to say, in some implementations of this case, these practical details are not necessary. In addition, some of the conventional structures and elements are shown in the drawings in a simplified schematic manner in order to simplify the drawings.

FIG. 1 is a flow chart of an image processing method 100 according to an embodiment of the present disclosure. FIG. 2 is a block diagram of an electronic device 200 according to an embodiment of the present disclosure. Referring to FIGS. 1 and 2 together, the image processing method 100 can be applied to the electronic device 200. The electronic device 200 includes a processor 210 and an image capturing unit 230 that are electrically connected to each other. The image capturing unit 230 can be a camera module for outputting an original image.

FIG. 3 is a schematic diagram of an original image 220 according to an embodiment of the present disclosure. As shown in FIG. 3, the original image 220 includes a plurality of pixels 221. Here, in order to clearly present the drawing, only a part of the pixels 221 are drawn, and the pixels 221 are intentionally enlarged.

In this embodiment, the electronic device 200 can be an image-capable electronic device such as a mobile phone, a tablet computer, or a notebook computer. The electronic device 200 further includes a non-transitory computer readable recording medium 250 coupled to the processor 210, which may be a computer readable recording medium such as a read only memory, a flash memory or a hard disk. The non-transitory computer readable recording medium 250 stores computer program instructions, and the following image processing method 100 can be executed by the processor 210 of the electronic device 200.

Referring to FIG. 1, in step 110, the processor 210 receives at least one original image captured by the image capturing unit 230. Then, the processor 210 transmits a pixel merging instruction to the image capturing unit, so that the image capturing unit performs a pixel merging process on the original image (step 120). The pixel merging process combines the pixel signals of a plurality of adjacent pixels 221 (four adjacent pixels in this embodiment) into a new pixel signal to generate a first image with a new pixel signal. Here, the merging adds up the four pixel signals to a new pixel signal. Therefore, through this step, the original image can be increased by four times the brightness to form a first image. However, because of the pixel binning process, the resolution of the image will be four times lower than the original resolution. In this regard, the subsequent steps will be performed to restore the original image resolution.

In an embodiment, whether or not pixel merging is performed may also be determined depending on the actual shooting environment. That is, before step 120, the processor 210 determines an ambient brightness according to the original image received from the image capturing unit, and determines whether to cause the image capturing unit to perform a pixel combining process on the original image according to the ambient brightness. Alternatively, the brightness of the ambient light can be separately obtained by the brightness sensor provided on the electronic device 200. After the processor 210 obtains the ambient brightness value from the brightness sensor, it can determine whether to output the pixel combination instruction according to the ambient brightness value. That is, if the ambient brightness is lower than the preset value, it may be determined to belong to the low light source environment, and then step 120 may be performed, so that the processor 210 outputs a pixel merge command to the image capturing unit 230 to allow the image capturing unit 230. A pixel binning process is performed to output the first image. On the other hand, if the shooting environment is not a low light source environment, the image capturing unit 230 directly outputs the original image to the processor 210 without performing a pixel merging process, but the processor 210 can still perform super-resolution reconstruction on successive multiple original images. program.

In step 130, the processor 210 repeats the foregoing steps 110 to 120 to successively obtain a plurality of first images that have passed through the pixel combining step. The continuous first image means that the capture times of the images are separated by a short time (eg, 1/30 second), but the intervals are not necessarily the same. In some embodiments, the number of first images is four or more to obtain sufficient data for calculation.

Then, in step 140, the processor 210 performs an super-resolution reconstruction step to generate a second image by using the first image by interpolation, thereby obtaining a second image with the same resolution as the original image. Referring to Figure 4, there is shown a detailed flow chart of step 140 of Figure 1. First, the plurality of feature points of the first images are respectively searched for, and the feature points may be edges, corners, spots or ridges (step 141). The feature points can be selected, for example, by Harris detection, FAST detection, SURF detection, SIFT detection or MSER detection. Then, since the user holds the electronic device 200, the photographic range of each captured first image is inevitably slightly different. Therefore, by searching for the position of the same feature point on each of the first images, each first can be calculated. The displacement message between images. The alignment can be accomplished by aligning the first images by moving the same feature point to the same coordinate (step 142). After the alignment is completed, there must be other pixels from different first images around the pixels at any position. After comprehensively referring to the positions of the pixels and the color information, a second image can be formed, and the method can be interpolation or medium. A method such as a value filter (step 143). Herein, the interpolation method may be an interpolation method that is not an iterative type, in other words, an interpolation method that can be completed by one-time solution. For example: linear interpolation.

In some embodiments, the foregoing partial steps may be performed by the image sensing unit 230. For example, in step 110, the image sensing unit 230 captures an image to obtain an original image. In step 120, the pixel merging process may be performed by the image sensing unit 230 to generate a first image, and the first image is output to the processor 210, so that the processor 210 can continue to process the step 130 and subsequent steps.

In an embodiment, after the foregoing step 140, some edges of the second image may be less natural, and thus the image smoothing process may be performed on the second image by using an image smoothing filter. Here, the image smoothing filter may be a bilateral filter, a trilateral filter, or a Guided Image Filter.

In an embodiment, if the captured first image of the pixel combining step has excessive noise, the image smoothing process may be performed on the first images before step 140. Similarly, the image smoothing filter used may be a bilateral filter, a three-way filter or a guided image filter. Thereby, the image quality of the second image generated by the subsequent execution of step 140 can be made better.

In summary, by applying an embodiment of the present invention, the brightness of the photo is increased by pixel combination, and then the image is reconstructed by super-resolution to restore the photo to the original resolution, so that the image can be obtained in a low light source environment. A photo that is bright enough and does not sacrifice image resolution.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present case. Anyone skilled in the art can make various changes and refinements without departing from the spirit and scope of the case. Therefore, the scope of protection of this case is considered. The scope defined in the patent application is subject to change.

100‧‧‧Image processing method
110, 120, 130, 140‧‧‧ steps
141, 142, 143‧‧ steps
200‧‧‧Electronic devices
210‧‧‧ processor
220‧‧‧ original image
221‧‧ ‧ pixels
230‧‧‧Image capture unit
250‧‧‧ Non-transient computer readable recording medium

FIG. 1 is a flowchart of an image processing method according to an embodiment of the present disclosure. FIG. 2 is a block diagram of an electronic device according to an embodiment of the present disclosure. FIG. 3 is a schematic diagram of an original image according to an embodiment of the present disclosure. FIG. 4 is a detailed flow chart of step 140 of FIG. 1.

100‧‧‧Image processing method

110, 120, 130, 140‧‧‧ steps

Claims (11)

An image processing method includes: acquiring at least one original image; performing a pixel merging process on the at least one original image, the pixel merging process combining the plurality of adjacent pixel signals in the at least one original image into a new pixel signal a first image having the new pixel signal; a plurality of the first images generated by the pixel combining process are continuously obtained; and an super-resolution reconstruction program is executed to pass the first images through the first image The interpolation produces a second image. The image processing method of claim 1, wherein the step of performing an over-resolution reconstruction process to generate the super-resolution images by using the first image by an interpolation method further comprises: separately searching for the first images a plurality of feature points; aligning the first images according to the feature points on the first images; and reconstructing the second images by the interpolation according to the first images after the alignment. The image processing method of claim 1, wherein after the super-resolution reconstruction process, the method further comprises: performing image smoothing processing on the second image by using an image smoothing filter. The image processing method of claim 1, wherein before the super-resolution reconstruction process, the method further comprises: performing image smoothing processing on the first images by using an image smoothing filter. A non-transitory computer readable recording medium storing a computer program instruction for executing an image processing method after loading an electronic device, the image processing method comprising: obtaining at least one original image; The original image performs a pixel merging process, and the pixel merging process combines a plurality of adjacent pixel signals in the at least one original image into a new pixel signal to generate a first image having the new pixel signal; And generating, by the interpolation process, a plurality of the first images; and performing a super-resolution reconstruction process to generate a second image by using an interpolation method. The non-transitory computer readable recording medium according to claim 5, wherein the step of performing an over-resolution reconstruction process to generate the super-resolution images by using the first image by an interpolation method further comprises: separately searching The plurality of feature points of the first image; the first images are aligned according to the feature points on the first images; and the first images after the alignment are reconstructed by the interpolation The second image. The non-transitory computer readable recording medium according to claim 5, wherein after the super-resolution reconstruction program, the method further comprises: performing image smoothing processing on the second image by using an image smoothing filter. The non-transitory computer readable recording medium according to claim 5, wherein before the super-resolution reconstruction program, the method further comprises: performing image smoothing processing on the first images by using an image smoothing filter. An electronic device, comprising: an image capturing unit, configured to output an original image or combine a plurality of adjacent pixel signals in the original image into a new pixel signal to generate a first image having the new pixel signal; And a processor, connected to the image capturing unit, and obtaining a plurality of consecutively generated first images, so that the first images generate a second image through an interpolation method. The electronic device of claim 9, wherein the processor determines an ambient brightness based on the original image received from the image capturing unit, and determines whether to cause the image capturing unit to output the first image according to the ambient brightness. The electronic device of claim 9, wherein the processor further performs image smoothing processing on the first image or the second image.