TWI511549B - Image processing method and image processing system - Google Patents

Description

Image processing method and image processing system

The present invention relates to an image processing method, and more particularly to an image processing method and image processing system relating to digital zoom.

Digital cameras, digital cameras, or smart phones with camera functions are becoming more and more popular. In general, these photo-enabled electronic devices include a zoom lens. When the optical zoom does not meet the user's needs, the user will use the digital zoom function to zoom in on specific areas of the photo. However, when using digital zoom, the lens does not refocus, which means that the exposure value, aperture, and even the pixel value on the sensor will not change. When the average brightness of a specific area is different from the average brightness of the original image, the brightness of the enlarged specific area is made bright or dark.

Therefore, how to present an image that meets the expectations of the human eye at any time when the user uses the digital zoom is an issue of concern to those skilled in the art.

Embodiments of the present invention provide an image processing method and an image processing system, which can improve the quality of an image and make it more in line with the expectations of the human eye.

An embodiment of the invention provides an image processing method suitable for an image processing apparatus. The method includes: obtaining a first image processing parameter of the first image and the first image, where the first image processing parameter is corresponding to the first image a display effect; receiving a digital zoom command, and changing the resolution of the first image according to the digital zoom command to obtain a second image; acquiring second image processing parameters of the second image, wherein the second image processing parameter is corresponding to the second The display effect of the image, and the second image processing parameter is different from the first image processing parameter; and adjusting the display effect of the second image according to the second image processing parameter to generate the third image.

In an embodiment, the step of receiving the digital zoom command and changing the resolution of the first image according to the digital zoom command to obtain the second image further includes: acquiring a preset image; changing the resolution of the predicted image to obtain more a third image; calculating a noise ratio of each of the third images; determining whether a noise ratio of each of the third images is greater than a noise threshold to generate a determination result; and determining a resolution change range according to the determination result . The step of changing the resolution of the first image to obtain the second image comprises: changing the resolution of the first image within the range of resolution change to obtain the second image.

In an embodiment, the step of changing the resolution of the first image to obtain the second image includes: adjusting a first image processing parameter of the first image to obtain a fourth image; and changing a resolution of the fourth image to obtain a first image Two images.

In an embodiment, the second image processing parameter is a sharpness. The step of adjusting the second image according to the second image processing parameter to generate the third image includes: determining whether the second image processing parameter meets a sharpness threshold; if the second image processing parameter does not meet the sharpness threshold, Repeating the adjusting the first image processing parameter of the first image to obtain The step of the fourth image, the step of changing the resolution of the fourth image to obtain the second image, and the step of acquiring the second image processing parameter of the second image until the second image processing parameter meets the sharpness threshold. If the second image processing parameter meets the sharpness threshold, the second image is the third image.

In another aspect, an embodiment of the present invention provides an image processing system, including an image capturing module, a control module, and a resolution changing module. The image capturing module is configured to obtain first image processing parameters of the first image and the first image, wherein the first image processing parameter is a display effect corresponding to the first image. The control module is for receiving a digital zoom command. The resolution change module is configured to change the resolution of the first image according to the digital zoom command to obtain the second image. The image capturing module is further configured to obtain a second image processing parameter of the second image, where the second image processing parameter is a display effect corresponding to the second image, and the second image processing parameter is different from the first image processing parameter. The control module is further configured to adjust a display effect of the second image according to the second image processing parameter to generate a third image.

In an embodiment, the image capturing module is further configured to obtain a preset image. The resolution change module is further configured to change the resolution of the predicted image to obtain a plurality of third images. The control module is further configured to calculate a noise ratio of each of the third images, determine whether a noise ratio of each of the third images is greater than a noise threshold to generate a determination result, and determine a resolution change according to the determination result. range. The resolution change module is further configured to change the resolution of the first image to obtain the second image within the range of the resolution change.

In an embodiment, the control module is further configured to adjust the first image. The first image processing parameter is to obtain a fourth image. The resolution change module is further configured to change the resolution of the fourth image to obtain the second image.

In an embodiment, the second image processing parameter is sharpness. The control module is further configured to determine whether the second image processing parameter meets a sharpness threshold. If the second image processing parameter does not meet the threshold of the sharpness, the control module is further configured to repeat the step of adjusting the first image processing parameter of the first image to obtain the fourth image, and the resolution changing module is further configured to repeat The step of changing the resolution of the fourth image to obtain the second image, the image capturing module repeats the step of acquiring the second image processing parameter of the second image until the second image processing parameter meets the sharpness threshold. If the second image processing parameter meets the sharpness threshold, the second image is the third image.

In an embodiment, the display effect described above is an exposure effect, a color temperature effect, or a sharp effect.

Based on the above, the image processing method and the image processing system according to the embodiments of the present invention can further adjust various display effects of the image after the digital zoom, so that the displayed image is more in line with the expectations of the human eye.

The above described features and advantages of the present invention will be more apparent from the following description.

[First Embodiment]

1 is a schematic diagram of an image processing apparatus according to an embodiment.

The image processing device 100 includes an image capturing device 110 and a processor. 120. Memory 130 and display device 140. The processor 120 loads the image processing system 150 into the memory 130. The image processing device 100 is configured to capture one or more images and store or display the images. For example, the image processing apparatus 100 may be a digital camera, a digital camera, a smart phone, a tablet, a personal computer, or a server, but the invention is not limited thereto.

The image capturing device 110 is configured to acquire one or more images. For example, the image capturing device 110 includes a lens, an aperture, a shutter, and a photosensitive element.

The processor 120 is for controlling the image processing apparatus 100. For example, processor 120 is a central processing unit (CPU) or a microprocessor.

The display device 140 is for displaying an image. For example, the display device 140 is a liquid crystal display (LCD) or an Organic Light-Emitting Diode (OLED), but the invention is not limited thereto.

After the processor 120 receives the image acquired by the image capturing device 110, each module in the image processing system 150 is executed, and the processed image is displayed on the display device 140.

2 is a block diagram illustrating an image processing system in accordance with an embodiment.

Referring to FIG. 2 , the image processing system 150 includes an image capturing module 210 , a resolution changing module 220 , a control module 230 , and a display module 240 .

The image capturing module 210 is configured to acquire images captured by the image capturing device 110 and obtain one or more image processing parameters. For example, image processing parameters can be exposure, color temperature of the image, shutter time, aperture size, The motion vector between the images, the degree of the block effect in the image, or the noise ratio of the image, etc., are not limited thereto.

The resolution change module 220 is used to change the resolution of the image. The resolution referred to herein also refers to the width and height of the image, and the unit is the number of pixels. The algorithm employed by the resolution change module 220 can be linear or non-linear, regional or non-regional, and the present invention does not limit which algorithm is employed.

The control module 230 is configured to adjust the display effect of the image according to the image processing parameters acquired by the image capturing module 210. For example, the display effect may be various effects indicating color temperature, exposure, sharpness, blur degree, contrast, and the like of the image, and the present invention is not limited thereto.

The display module 240 is for driving the display device 140 to display the adjusted image on the display device 140. For example, display module 240 displays an instant preview image. The preview image is an image displayed when the user presses the shutter to let the user know the composition or exposure of the image.

In this embodiment, each module in the image processing system 150 is implemented as a plurality of programs. The operation of each module is described below, which is an operation performed by the processor 120 to execute the modules, and will not be described below.

FIG. 3 is a schematic diagram illustrating adjusting image brightness when changing resolution according to an embodiment.

Referring to FIG. 3, when the user wants to take a picture, the image capturing module 210 first acquires an image 310 (also referred to as a first image) and an image processing parameter (also referred to as a first image processing parameter) of the image 310, and this Image processing parameters correspond to a display effect. In the embodiment shown in Figure 3, the image The processing parameter is an exposure value, which is calculated by the image capturing module 210 according to the brightness of the image 310. The display effect corresponding to the image processing parameters is the exposure effect (indicating the brightness of the image).

For example, the brightness of region 312 in image 310 is darker than the brightness of region 314. The image capturing module 210 calculates an appropriate exposure value of the image 310 according to the brightness of each area. However, since there is a very bright light source (i.e., the bulb 316) in the image 310, the area 312 is maintained at a relatively low brightness even after the image 310 is adjusted according to the exposure value calculated by the image capturing module 210.

The user will place a digital zoom command (for example, rotate a dial or press a button) to adjust the size of the object in the image. For example, the user would "zoom in", thereby focusing on area 312. The digital zoom referred to herein is a method of image processing to enlarge the resolution of the image 310. After the control module 230 receives the digital zoom command, the resolution change module 220 changes the resolution of the image 310 according to the digital zoom command, and the original region 312 becomes the image 320. It is worth noting that since the area 312 has a relatively low brightness, the brightness of the image 320 is also low.

At this time, the image capturing device 110 will again obtain an exposure value (also referred to as a second image processing parameter) suitable for the image 320. The exposure value for image 320 will differ from the exposure value for image 310. For example, since the brightness of the image 320 is low, the exposure value suitable for the image 320 may be lower than the exposure value suitable for the image 310. The control module 230 adjusts the exposure effect of the image 320 according to the newly calculated exposure value, thereby generating an image. 330. At this time, the brightness of the image 330 is higher than the brightness of the image 320, which is more in line with the expectations of the human eye.

It is worth noting that the image processing parameters referred to herein are exposure values. However, the image processing parameters can also be other parameters that can adjust the brightness. For example, the control module 230 uses a non-linear function to adjust the brightness of the image, and the image processing parameter can be a parameter in the function. Alternatively, the control module 230 uses a local contrast enhancement algorithm to adjust the brightness of the image, and the image processing parameters are a variable in the algorithm. Similarly, the exposure effect referred to herein may also be referred to as a contrast effect or a brightness effect, and the present invention is not limited thereto.

4 is a diagram showing an example of adjusting color temperature when changing resolution in accordance with an embodiment.

Referring to FIG. 4, the image capturing module 110 first obtains an image processing parameter of the image 410 (also referred to as the first image) and the image 410. In the embodiment shown in FIG. 4, the image processing parameter is a color temperature, and the corresponding display effect is a color temperature effect. In other words, the image processing parameters obtained by the image capturing module 110 are used to indicate that the image 410 appears to be cold or warm. That is, the image processing parameters at this time are the colors corresponding to the image 410, not the brightness.

In image 410, the color temperature of bulb 416 is lower than the color temperature of the fluorescent lamp (ie, the emitted light is yellowish). For example, the color temperature of the bulb 416 is D55 (5500 degrees K), and the color temperature of the fluorescent lamp is D65 (6500 degrees K). Therefore, the color temperature of the region 412 is lower than the color temperature of the region 414. That is, the color of region 414 will be relatively white, while the color of region 412 will be relatively yellow.

When the user wants to focus on area 412, a digital zoom command is issued. After the control module 230 receives the digital zoom command, the resolution change module changes the resolution of the image 410 according to the digital zoom command, so that the region 412 becomes the image 420 (also referred to as the second image).

The image capturing module 210 obtains the color temperature of the image 420. It should be noted that the color temperature of the image 420 is not the same as the color temperature of the image 410. In the embodiment of FIG. 4, the color temperature of image 420 is lower than the color temperature of image 410. The control module 230 performs a white balance algorithm on the image 420 according to the color temperature of the image 420, thereby acquiring the image 430. In this way, the color temperature of the image 430 will be close to the color temperature of the fluorescent lamp, which is in line with the expectations of the human eye. In other words, the control module 230 adjusts the display effect (ie, the color temperature effect) of the image 420 according to the color temperature of the image 420 (also referred to as the second image processing parameter) to generate the image 430 (also referred to as the third image). Finally, display module 240 displays image 430 on display device 140.

In an embodiment, the algorithm of white balance may be based on a grey-world hypothesis or a white-patch hypothesis. However, the present invention does not limit which white balance algorithm is used by the control module 230. Further, in the embodiment shown in FIGS. 3 and 4, the pixel value of the image is raw data, that is, a pixel value that has not undergone gamma correction. However, in other embodiments, the image processing system 150 may also use pixel code corrected pixel values, and the invention is not limited thereto.

FIG. 5 is a flow chart illustrating an image processing method in accordance with an embodiment.

Referring to FIG. 5, in step S502, the image capturing module 210 takes The first image processing parameter of the first image and the first image is obtained.

In step S504, the control module 230 receives the digital zoom command, and the resolution change module 220 changes the resolution of the first image to obtain the second image.

In step S506, the image capturing module 210 acquires the second image processing parameter of the second image.

In step S508, the control module 230 adjusts the display effect of the second image according to the second image processing parameter to generate a third image.

The steps in Fig. 5 have been described in detail above, and will not be described again here.

[Second embodiment]

The second embodiment is similar to the first embodiment, and only the differences will be described herein. In general, when the image is enlarged, the noise is also amplified. In the second embodiment, the image processing system 150 adjusts the noise ratio in the image after the resolution of the image is changed. Moreover, the control module 230 generates a resolution change range for limiting the proportion of the image that can be enlarged or reduced.

In order to generate a range of resolution changes, the image capturing module 210 first obtains a preset image, and the preset image includes a preset pattern and color. For example, this preset image is an all black or all white image. Then, the resolution changing module 220 changes the resolution of the preset image to obtain a plurality of images (also referred to as a third image). The control module 230 calculates the noise ratio of each of the third images and determines whether the noise ratio is greater than a noise threshold. The control module 230 determines the range of resolution change based on the result of the determination.

For example, the maximum resolution that the image capturing device 210 can capture is 3840×2160 (the number of pixels), and the resolution of the display device 140 is 1920×1080 (the number of pixels). When the user places a digital zoom command, it is equivalent to the user to select an area in the picture and enlarge it. The resolution change module 220 is used to change the resolution of the area to be an image that can conform to the resolution of the display device 140 (as shown in FIGS. 3 and 4). In other words, when the resolution change module 220 changes the resolution of the preset image and generates a plurality of images, the regions of different resolutions are enlarged to the resolution of the display device 140. The resolution of the area magnified by the resolution changing module 220 and the noise calculated by the control module 230 are as shown in Table 1.

The first field of Table 1 indicates the magnification of magnification. For example, "2x2 = 4" indicates that the length and width of the image are doubled.

The second field of Table 1 indicates the resolution of the area. For example, when putting the pre When the length and width of the image are magnified by three times, the resolution of the region obtained by the image capturing module 210 is "1260x720" (3840/3=1260, 2160/3=720).

The third field of Table 1 indicates the noise ratio when the corresponding area is enlarged to the resolution of the display device 140. For example, the "15%" noise ratio indicates the noise ratio in the image when the resolution of the region is enlarged from "1260x720" to "1920x1080".

The fourth field of Table 1 indicates the noise ratio when the area corresponding to the table is enlarged to the maximum resolution of the image capturing device 110 (ie, 3840x2160). For example, the "25%" noise ratio indicates the noise ratio in the image when the resolution of the region is enlarged from "1260x720" to "3840x2160".

Since the preset image is used in calculating the noise ratio, the control module 230 can calculate the standard deviation of the noise. In detail, this preset image is taken by taking an all-black or all-white and flat object. Therefore, in the absence of noise, each pixel value in the preset image should be the same, in other words, the standard deviation of these pixel values should be zero. When there is noise in the preset image, the calculated standard deviation will not be 0. Therefore, the control module 230 can calculate the standard deviation of all pixel values in the preset image as the standard deviation of the noise. When the standard deviation is larger, it means that the noise ratio is larger. The control module 230 can calculate the noise ratio based on the standard deviation of the noise. However, in other embodiments, the control module 230 may use other algorithms to calculate a value indicating a noise ratio, and the present invention is not limited thereto.

After calculating the respective noise ratios in Table 1, the control module 230 compares whether the noise ratios are greater than a noise threshold. For example, control The module 230 sets the noise threshold to 25% and determines whether the noise ratio in the third field (ie, noise 1) is greater than the noise threshold, thereby generating a determination result. As shown in Table 1, when the magnification is "1x1=1", "2x2=4", "3x3=9" and "4x4=16", the noise ratio does not exceed the noise threshold. Therefore, the control module 230 determines the resolution change range from 1 to 4 according to the determination result, indicating that the magnification of the width and height of one image cannot exceed 4 times. After the resolution change range is set, when each resolution change module 220 wants to change the resolution of the image, the resolution of the image must be changed within the range of resolution change.

In other embodiments, the control module 230 can also calculate the resolution change range using the noise ratio of the fourth field in Table 1. Alternatively, the control module 230 can set different noise thresholds, and the present invention is not limited thereto.

In addition, each time the user wants to zoom in, the control module 230 uses a recursive method to increase the sharpness in the image.

6 is a flow chart illustrating the increase in sharpness, in accordance with an embodiment.

Referring to FIG. 6 , in step S602 , the image capturing module 210 first acquires an image 620 (also referred to as a first image) and image processing parameters of the image 620 (also referred to as a first image processing parameter). In this embodiment, the first image processing parameter is a sharpness parameter, and the corresponding display effect is a sharp effect. For example, when the image 620 and a filter are subjected to convolution calculations, the sharpness in the image increases. This sharpness parameter is used to determine the coefficient of this filter. In other words, adjust the sharpness parameter to adjust the sharpness of the image. The degree of sharpness referred to herein means whether the edge in the image is conspicuous. For example, if in the image If the edges are blurred, the sharpness in the image is very low.

In step S604, the control module 230 changes the sharpness parameter and increases the sharpness of the image 620 according to the sharpness parameter, thereby generating the image 621. For example, the control module 230 generates a filter based on the sharpness parameter and performs an operation of folding the image 620 with the filter to generate an image 621. However, the present invention does not limit the use of algorithms that increase sharpness.

In step S606, the resolution change module 220 changes the resolution of the image 621 to generate an image 622 (also referred to as a second image). The resolution of the image 622 will conform to the resolution of the display device 140.

In step S608, the image capturing module 210 acquires another image processing parameter (also referred to as a second image processing parameter) of the image 622. This second image processing parameter is the sharpness of the image 622. For example, the image capture module 210 performs an edge detection on the image 622 and determines the sharpness of the image 622 according to the result of the edge detection.

In step S610, the control module 230 determines whether the sharpness of the image 622 meets a sharpness threshold. This sharpness threshold can be customized by the user. If the result of step S610 is "NO", the control module 230 returns to step S604. In other words, at this time, the control module 230 repeats step S604, and the resolution changing module 220 repeats step S606. The image capturing module 210 repeats step S608 until the sharpness of the image 622 meets the sharpness threshold.

If the result of the step S610 is YES, the control module 230 performs step S612 to archive the image 622 (also referred to as the third image) or display the image. The display module 240 displays the image 622 on the display device 140.

In summary, the image processing method and the image processing system according to the embodiments of the present invention can further adjust the display effect of the image after the user releases the digital zoom command, thereby making the image more in line with the expectations of the human eye. The image processing method and the image processing system proposed by the embodiments of the present invention can also set the resolution change range so that images with too much noise are not obtained. Moreover, the sharpness of the image will be enhanced by recursive.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the invention, and any one of ordinary skill in the art can make some modifications and refinements without departing from the spirit and scope of the invention. The scope of the invention is defined by the scope of the appended claims.

100‧‧‧Image processing device

110‧‧‧Image capture device

120‧‧‧ processor

130‧‧‧ memory

140‧‧‧ display device

150‧‧‧Image Processing System

210‧‧‧Image capture module

220‧‧‧Clarity Change Module

230‧‧‧Control Module

240‧‧‧ display module

310, 320, 330, 410, 420, 430, 620, 621, 622‧‧ images

312, 314, 412, 414‧‧‧ areas

316, 416‧‧‧ bulbs

Steps of S502, S504, S506, S508‧‧‧ image processing methods

S602, S604, S606, S608, S610, S612‧‧‧ steps to increase sharpness

1 is a schematic diagram of an image processing apparatus according to an embodiment.

2 is a block diagram illustrating an image processing system in accordance with an embodiment.

FIG. 3 is a schematic diagram illustrating adjusting image brightness when changing resolution according to an embodiment.

4 is a diagram showing an example of adjusting color temperature when changing resolution in accordance with an embodiment.

FIG. 5 is a flow chart illustrating an image processing method in accordance with an embodiment.

6 is a flow chart illustrating the increase in sharpness, in accordance with an embodiment.

Steps of S502, S504, S506, S508‧‧‧ image processing methods

Claims (10)

An image processing method is applicable to an image processing device, including: acquiring a first image and a first image processing parameter of the first image, wherein the first image processing parameter is a display effect corresponding to the first image; Receiving a digital zoom command, and changing the resolution of the first image according to the digital zoom command to obtain a second image; acquiring a second image processing parameter of the second image, wherein the second image processing parameter is corresponding to The display effect of the second image is different from the first image processing parameter; and the display effect of the second image is adjusted according to the second image processing parameter to generate a third image. The image processing method of claim 1, wherein the step of receiving the digital zoom command and changing the resolution of the first image according to the digital zoom command to obtain the second image further comprises: obtaining a pre-preparation Setting an image; changing the resolution of the predicted image to obtain a plurality of third images; calculating a noise ratio of each of the third images; determining whether the noise ratio of each of the third images is greater than a noise a threshold value to generate a determination result; and determining a resolution change range according to the determination result, wherein the step of changing the resolution of the first image to obtain a second image comprises: changing the first in the resolution change range The resolution of the image is Obtain the second image. The image processing method of claim 1, wherein the step of changing the resolution of the first image to obtain a second image comprises: adjusting the first image processing parameter of the first image to obtain a fourth image. And changing the resolution of the fourth image to obtain the second image. The image processing method of claim 3, wherein the second image processing parameter is a sharpness, wherein the step of adjusting the second image according to the second image processing parameter to generate the third image comprises: Determining whether the second image processing parameter meets a sharpness threshold; and if the second image processing parameter does not meet the sharpness threshold, repeating the adjusting the first image processing parameter of the first image to obtain the a step of the fourth image, a step of changing the resolution of the fourth image to obtain the second image, and a step of acquiring the second image processing parameter of the second image until the second image processing parameter The sharpness threshold is met, and if the second image processing parameter meets the sharpness threshold, the second image is the third image. The image processing method according to claim 1, wherein the display effect is an exposure effect, a color temperature effect or a sharp effect. An image processing system includes: an image capture module for acquiring a first image and a first image processing parameter of the first image, wherein the first image processing parameter is corresponding a display effect of the first image; a control module for receiving a digital zoom command; and a resolution change module for changing the resolution of the first image according to the digital zoom command to obtain a second The image capture module is further configured to obtain a second image processing parameter of the second image, wherein the second image processing parameter is the display effect corresponding to the second image, and the second image processing The parameter is different from the first image processing parameter, wherein the control module is further configured to adjust the display effect of the second image according to the second image processing parameter to generate a third image. The image processing system of claim 6, wherein the image capturing module is further configured to obtain a preset image, wherein the resolution changing module is further configured to change the resolution of the predicted image to obtain more a third image, wherein the control module is further configured to calculate a noise ratio of each of the third images, and determine whether the noise ratio of each of the third images is greater than a noise threshold to generate a Determining the result, and determining a resolution change range according to the determination result, wherein the resolution change module is further configured to change the resolution of the first image to obtain the second image within the resolution change range. The image processing system of claim 6, wherein the control module is further configured to adjust the first image processing parameter of the first image to obtain a fourth image, wherein the resolution changing module further uses To change the solution of the fourth image The degree of resolution is obtained to obtain the second image. The image processing system of claim 8, wherein the second image processing parameter is a sharpness, wherein the control module is further configured to determine whether the second image processing parameter meets a sharpness threshold value, The second image processing parameter does not meet the threshold of the sharpness, and the control module is further configured to repeat the step of adjusting the first image processing parameter of the first image to obtain a fourth image, where the resolution is changed. The group is further configured to repeat the step of changing the resolution of the fourth image to obtain the second image, and the image capturing module repeats the step of acquiring the second image processing parameter of the second image until the The second image processing parameter meets the sharpness threshold. If the second image processing parameter meets the sharpness threshold, the second image is the third image. The image processing system of claim 6, wherein the display effect is an exposure effect, a color temperature effect, or a sharp effect.