TW202126026A - Computer system and image compensation method thereof - Google Patents

Abstract

A computer system and an image compensation method thereof are provided. In the method, the difference between first resolution information and second resolution information is determined. A first image is analyzed to generate image adjustment parameters in response to the difference existing between the first resolution information and the second resolution information. The first image would be adjusted according to the image adjustment parameters and the first resolution information to generate a second image. The first resolution information relates to the first image captured by an image capturing apparatus. The second resolution information relates to the second image presented on a display. The image adjustment parameters relate to the enhancement of the image clearness. The difference is the resolution corresponding to the first resolution information is less than the resolution corresponding to the second resolution information. The second image is displayed on the display. Accordingly, the clearness of the displaying image can be improved.

Description

Computer system and its image compensation method

The present invention relates to an image processing technology, and particularly relates to a computer system and an image compensation method thereof.

Devices such as mobile phones (or mobile phones), notebook computers, desktop computers, and Video See-Through Head-Mounted Displays (VST-HMD) are usually carried simultaneously (for example, external or Embedded) camera and display. In the application of these devices, images can be captured through the camera and displayed on the monitor quickly (or in real time). Take the video-transmissive head-mounted display (VST-HMD) as an example. The image display system is mainly composed of a panel display and an optical lens. The external environment can be captured by the camera on the device, and then the captured image can be displayed. The image allows the user to view the external world through the camera. This type of head-mounted display can be widely used in virtual reality (Virtual Reality, VR), Mixed Reality (MR), Augmented Reality (AR), and Extended Reality (XR). ) And other technologies.

However, the aforementioned devices equipped with cameras and displays may encounter a situation where the resolution of the source image (captured by the camera) is lower than the resolution of the display during the application process. In this situation, the screen displayed by the display may have the disadvantages of blurry image, image block effect, or flickering feeling in the dynamic image, which may affect the user experience.

In view of this, the embodiments of the present invention provide a computer system and an image compensation method thereof to improve the resolution and clarity of images captured by a camera, and make the images presented on the display clearer.

The image compensation method of the embodiment of the present invention includes but is not limited to the following steps: determining the difference between the first resolution information and the second resolution information; responding to the difference between the first resolution information and the second resolution information Difference, analyze the first image to generate image adjustment parameters; adjust the first image according to the image adjustment parameters and the second resolution information to form a second image. The first resolution information is related to the first image captured by the image capturing device, and the second resolution information is related to the second image presented on the display. The image adjustment parameter is related to enhancing the clarity of the image, and the difference is that the resolution corresponding to the first resolution information is lower than the resolution corresponding to the second resolution information. The second image is used for display by the display.

The computer system of the embodiment of the present invention includes, but is not limited to, an image capture device, a display, and a processor. The image capturing device is used for capturing the first image. The display is used for presenting the second image. The processor is coupled to the image capture device and the display. The processor is configured to perform the following steps: determine the difference between the first resolution information and the second resolution information; respond to the difference between the first resolution information and the second resolution information, analyze the first image To generate image adjustment parameters; adjust the first image according to the image adjustment parameters and the second resolution information to form a second image. The first resolution information is related to the first image, and the second resolution information is related to the second image. The image adjustment parameter is related to enhancing the clarity of the image, and the difference is that the resolution corresponding to the first resolution information is lower than the resolution corresponding to the second resolution information. The processor displays the second image on the display.

Based on the above, in the computer system and the image compensation method of the embodiment of the present invention, if the corresponding resolution of the image obtained by the image capturing device is lower than the resolution of the display, the image is analyzed and the resolution of the image is increased and the definition is enhanced accordingly. . In this way, more efficient resource allocation can be used, and the image presented by the display can be clearer.

In order to make the above-mentioned features and advantages of the present invention more comprehensible, the following specific embodiments are described in detail in conjunction with the accompanying drawings.

FIG. 1 is a block diagram of components of a computer system 1. 00 according to an embodiment of the present invention. Please refer to FIG. 1, the computer system 100 includes but is not limited to an image capture device 110, a display 120 and a processor 130. The computer system 100 may be a smart phone, a tablet computer, a notebook computer, or a desktop computer

, Head-mounted displays and other devices.

The image capturing device 110 may be a camera, a video camera, or other devices with image capturing function. In one embodiment, the image capturing device 110 is used for external shooting to capture images.

The display 120 is coupled to the processor 130, and the display 120 may be a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, or other types of displays. In one embodiment, the display 120 is used to display images.

The processor 130 is respectively coupled to the image capture device 110 and the display 120. The processor 130 may be a central processing unit (CPU), or other programmable general-purpose or special-purpose microprocessors and digital signals. Processor (Digital Signal Processor, DSP), programmable controller, Application-Specific Integrated Circuit (ASIC) or other similar components or a combination of the above components. In one embodiment, the processor 130 is used to obtain an image from the image capturing device 110 and further process the image.

It should be noted that in some embodiments, the processor 130 may be embedded in the image capture device 110 or the display 120. That is, the processor 130 is the processor of the image capture device 110 or the display 120.

In order to facilitate the understanding of the operation process of the embodiment of the present invention, a number of embodiments will be given below to describe in detail the process of image processing in the embodiment of the present invention. Hereinafter, the method described in the embodiment of the present invention will be described in conjunction with various components and modules in the computer system 100. Each process of the method can be adjusted accordingly according to the implementation situation, and it is not limited to this.

FIG. 2 is a flowchart of an image compensation method according to an embodiment of the invention. Referring to FIG. 2, the processor 130 determines the difference between the first analysis information and the second analysis information (step S210). Specifically, the processor 130 inputs the first image captured by the image capturing device 110, and obtains the first image or the first resolution information of the image capturing device 110. That is, the first resolution information is related to the first image captured by the image capturing device 110. The first resolution information may be the number of pixels of the height and/or width of the first image (ie, the height and/or width of the resolution). For example, the processor 130 obtains metadata of the first image, and the metadata records the resolution of the image. Alternatively, the processor 130 obtains the device identification information of the image capture device 110 from the image capture device 110 or the memory to obtain the resolution supported by the image capture device 110. On the other hand, the processor 130 obtains the second resolution information of the display 120. For example, the processor 130 may obtain the second resolution information from the device identification information of the display 120 or the current setting of the display resolution of the operating system. The second resolution information may be the number of pixels in height and/or width of the second image presented by the display. That is, the second resolution information is related to the second image presented by the display 120.

為：

…(1) 其中，

是顯示器120的解析度的寬，

是第一影像的解析度的寬。而解析度的高(Height)比值

為：

…(2) 其中，

是顯示器120的解析度的高，

是第一影像的解析度的高。在另一實施例中，處理器130可直接比較解析度的高及/或寬的大小。Then, the processor 130 may compare the resolution of the first image with the resolution of the display 120, and/or compare the resolution of the first image with the resolution of the display 120. In an embodiment, the processor 130 may determine the ratio of the two. For example, the aspect ratio value of the resolution

for:

…(1) Among them,

Is the width of the resolution of the display 120,

Is the width of the resolution of the first image. And the height ratio of the resolution

for:

…(2) Among them,

Is the high resolution of the display 120,

It is the high resolution of the first image. In another embodiment, the processor 130 may directly compare the height and/or width of the resolution.

Then, in response to the difference between the first resolution information and the second resolution information, the processor 130 analyzes the first image to generate image adjustment parameters (step S230). The difference of the embodiment of the present invention is that the resolution corresponding to the first resolution information is lower than the resolution corresponding to the second resolution information. It is worth noting that the lower resolution image displayed on the display 120 may cause blur, block effect, or image flicker. Therefore, if the judgment result of step S210 is that there is a difference between the two pieces of information, the first image needs further image definition compensation.

In an embodiment, the processor 130 may compare the ratio of the equation (1) and/or the equation (2) with the corresponding threshold value. If the ratio is greater than the corresponding threshold value, the processor 130 determines that there is a difference between the two pieces of information; otherwise, the processor 130 determines that there is no difference or the difference between the two pieces of information is very small.

大於第一門檻值時才進行影像清晰度補償。反之，若第一影像解析度的高大於其寬，則處理器130設定在高(Height)比值

大於第二門檻值時才進行影像清晰度補償。需說明的是，第一及第二門檻值的數值可依據實際情況而變化。例如，第一門檻值及/或第二門檻值可設定為1(即，當顯示器120的解析度(高或寬)大於第一影像(影像擷取裝置110擷取的影像)解析度(高或寬)時，處理器130才進行影像清晰度補償)。又例如，第一門檻值及/或第二門檻值可設定為1.5。For example, it is assumed that the aspect ratio of the first image is not changed. If the width of the first image resolution is greater than its height, the processor 130 sets the aspect ratio value

The image definition compensation is performed when it is greater than the first threshold value. Conversely, if the height of the first image resolution is greater than its width, the processor 130 sets the height ratio

When it is greater than the second threshold value, the image definition compensation is performed. It should be noted that the values of the first and second thresholds may vary according to actual conditions. For example, the first threshold and/or the second threshold can be set to 1 (that is, when the resolution (height or width) of the display 120 is greater than the resolution (high) of the first image (the image captured by the image capture device 110) (Or wide), the processor 130 performs image definition compensation). For another example, the first threshold and/or the second threshold may be set to 1.5.

In another embodiment, the processor 130 can directly compare the resolution height and/or width of the two pieces of information. If the first resolution information is lower than the resolution corresponding to the second resolution information, or the difference between the two resolutions is higher than a specific value (for example, 100, 200, or 300 pixels, etc.), the processor 130 may determine that there is a difference ( That is, image sharpness compensation is required).

If the image definition compensation is to be performed, the processor 130 will further determine the image adjustment parameters used to enhance the image definition. The processor 130 may analyze the first image to obtain corresponding image adjustment parameters. This image adjustment parameter may be related to contrast enhancement, sharpness enhancement, or other image processing that enhances image clarity.

In one embodiment, the processor 130 may determine the corresponding image adjustment parameter according to the comparison result between the first resolution information and the second resolution information. Specifically, the processor 130 can determine the relationship between the display resolution of the display 120 and the resolution of the first image according to the ratio calculated by the equation (1) or the equation (2). If the ratio is greater than 1, it means that the display resolution of the display 120 is greater than the resolution of the first image. If the ratio is larger, phenomena such as blurring and block effect may be more serious, and the processor 130 must adjust the image definition of the first image corresponding to the trend (that is, the higher the ratio, the stronger the intensity of the image adjustment parameter. high). If the ratio is less than 1, it means that the display resolution of the display 120 is lower than the resolution of the first image, that is, the chance of "decreasing the display image definition of the display 120" is low, and the processor 150 may not/disable Adjust the first image.

可由方程式(3)得出：

…(3)

是邊緣強度等級(其值以0~255為例)，

是像素的邊緣強度等級為

的個數。即，紋理等級是第一影像中所有像素的邊緣強度等級的算術平均值。In another embodiment, the processor 130 determines the texture level of the first image. Specifically, the processor 130 may count the corresponding edge intensity of each pixel in the first image, and obtain the texture level of the first image according to the statistical result. For example, the texture level of the first image

It can be derived from equation (3):

…(3)

Is the edge strength level (the value is 0~255 as an example),

Is the edge intensity level of the pixel is

The number of. That is, the texture level is the arithmetic average of the edge intensity levels of all pixels in the first image.

It should be noted that in other embodiments, the texture level may also be the median, mode, or other representative value of the edge intensity levels of all pixels in the first image. In addition, in some embodiments, the processor 130 may perform edge enhancement processing on the first image before the edge strength is counted.

After the texture level is determined, the processor 130 can determine the corresponding image adjustment parameters according to the texture level. If the texture level of the first image is too high (may be compared with a specific threshold value), it means that the first image itself may have more or more complex textures, and the effect of subsequent adjustments (for example, image sharpness enhancement) may be higher Therefore, the processor 130 must perform a relatively high-intensity image adjustment on the first image. That is, if the texture level is higher, the intensity of the image adjustment parameter is higher. On the other hand, the texture level of the first image is low (may be compared with a specific threshold value), which means that the first image may be a smoother picture, and the effect of subsequent adjustments may be lower. Therefore, the first image does not need to be excessively adjusted. The image is adjusted, not even adjusted. That is, if the texture level is lower, the intensity of the image adjustment parameter is lower. In addition, for texture levels between the highest and highest texture levels, the processor 150 can determine the corresponding image adjustment parameters according to the aforementioned trend.

In another embodiment, the processor 130 may determine the dynamic level of the first image. Specifically, the processor 130 may count the motion vector corresponding to at least one block (consisting of one or more pixels) in the first image, and obtain the dynamic level of the first image according to the statistical result. . For example, the processor 130 may determine the average value of the motion vector sizes of those blocks in the first image as the dynamic value or the dynamic level (hereinafter collectively referred to as the dynamic level) of the entire first image. For another example, the dynamic level may also be the median, mode, or other representative value of the dynamic vector size of those blocks in the first image.

After the dynamic level is determined, the processor 130 can determine the corresponding image adjustment parameters according to the dynamic level. If the dynamic level is lower, it means that the first image is closer to the static picture; conversely, the higher the dynamic level is, it means that the first image is closer to the dynamic picture. Generally speaking, the higher the scene dynamic level of the first image is, the higher the intensity adjustment is required. That is, if the dynamic level is higher, the intensity of the image adjustment parameter is higher. On the other hand, the lower the scene dynamic level of the first image, the less need to make excessive adjustments, or even no adjustments. That is, the lower the dynamic level, the lower the intensity of the image adjustment parameter.

In another embodiment, the processor 130 determines the texture level and/or dynamic level of the first image, and gives the difference between the first resolution information and the second resolution information, and the texture level and/or dynamic level Corresponding weight value. Specifically, the image adjustment parameters corresponding to the aforementioned different resolution ratios, different texture levels, and different dynamic levels can be given by prior experiments (for example, forming a comparison table or equation), or generated by training methods such as machine learning. The model is not limited here.

In addition, these levels or differences can also use weight distribution. In one embodiment, the weight value is determined based on the following mechanism: if the resolution difference, texture level or dynamic level is higher, the corresponding weight value is higher; if the resolution difference, texture level or dynamic level is lower , The lower the corresponding weight value. Then, the processor 130 may determine the intensity of the corresponding image adjustment parameter (for example, the intensity of sharpness, or the contrast) according to the difference and the corresponding weight value of the texture level and/or the dynamic level (for example, weight distribution or weight calculation, etc.). The strength, etc.).

For example, suppose that the first threshold (or the second threshold) used for the comparison resolution is set to 1, and the processor 130 presets the value of the ratio that may be obtained from equation (1) or equation (2) Divided into fifty groups (for example, the first resolution group to the fiftieth resolution group, the first resolution group is the group with the lowest ratio value, and the fiftieth resolution group is the ratio value The highest group, the rest and so on). The possible level values of equation (3) are divided into thirty groups (for example, the first texture group to the thirtieth texture group, the first texture group is the group with the lowest value, and the thirtieth texture The group is the group with the highest value, and the rest can be deduced by analogy). In addition, the possible values of the dynamic level are also divided into twenty groups (for example: the first dynamic group to the twentieth dynamic group, the first dynamic group is the group with the lowest value, and the twentieth dynamic group Is the group with the highest value, the rest can be deduced by analogy). In addition, it is assumed that the intensity of the image adjustment parameter used for sharpness adjustment has one hundred levels (for example, the first level to the hundredth level).

Then, the processor 130 can determine the intensity of the final image adjustment parameter by means of weight distribution. For example, the resolution group accounts for 50%, the texture group accounts for 30%, and the dynamic group accounts for 20%. The processor 130 distributes the results of the three reference items (for example, the resolution difference, the texture level, and the dynamic level) according to the proportions (for example, fifty, thirty, and twenty percent). That is, the priority of the three reference items is different. These three reference items and the intensity of the image adjustment parameters are all positively correlated in value. Therefore, when the weight value is assigned, the processor 130 can give the highest weight value to the one with the highest corresponding value in the group (for example, if the texture item is assigned to 30%, the 30th texture group corresponds to The weight value is 30%), and the group with the lowest corresponding value is given the lowest weight (for example, the weight value corresponding to the first texture level is 1%), and finally three references The sum of the weight values corresponding to the items (that is, the sum of the weight values assigned) is used as the intensity of the image adjustment parameter.

For example, if the results obtained from the current first image are the first resolution group, the first texture group, and the first dynamic group, the intensity of the corresponding image adjustment parameter is the third level (three hundredths) Add up to 3%). For another example, if the result obtained according to the current first image is the fiftieth resolution group, the thirtieth texture group, and the twentieth dynamic group, the intensity of the corresponding image adjustment parameter is the hundredth level (100 The solution of 50% plus 30% plus 20% is 100%).

It should be noted that the aforementioned weight ratio and the number of groups of each reference item are merely illustrative, and may be changed according to actual conditions, and the embodiment of the present invention is not limited.

Then, the processor 130 may adjust the first image according to the image adjustment parameters and the second resolution information obtained in step S230 to form a second image (step S250). Specifically, the second image is an image used for the display 120 to present. The processor 150 can adjust the image resolution of the first image to a resolution corresponding to the second resolution information. For example, if the first resolution information is 480P (720×480) and the second resolution information is 1080P (1920×1080) (that is, the display resolution of the display 120), the processor 130 may increase the 480P first image Up to 1080P resolution. It should be noted that in other embodiments, the processor 130 may also adjust the image resolution of the first image to a resolution between the corresponding resolutions of the first resolution information and the second resolution information. For example, if the first resolution information is 480P and the second resolution information is 1080P, the processor 130 can increase the 480P first image to a resolution of 720P (1280×720).

Then, the processor 130 may perform image processing related to sharpness enhancement, such as sharpness enhancement and contrast enhancement, on the first image with the increased resolution according to the image adjustment parameters obtained in step S230 to form a second image. That is, the second image is an image with increased resolution and sharpness of the first image. The processor 130 can then display the second image on the display 120.

It should be noted that the time interval between the generation of the first image and the display of the second image may be less than a specific time threshold (for example, 1 second, 500 milliseconds, or 100 milliseconds, etc.), so that the display 120 can display the image in real time. The function of capturing images captured by the device 110 (for example, Live View). In addition, if the determination result of step S210 is that there is no difference or the difference is very small, the processor 130 may directly use the first image as the second image to be displayed by the display 120. That is, the processor 130 displays the first image on the display 120.

In summary, the computer system and the image compensation method of the embodiment of the present invention analyze the first image and the display resolution for the case where the resolution of the first image captured by the image capture device is less than the display resolution of the display According to the difference between the, the texture level and the dynamic level of the first image, the image adjustment parameters corresponding to these reference items are obtained. Then, the resolution of the first image can be increased, and the corresponding image sharpness enhancement adjustment can be performed according to the image adjustment parameters. In this way, more efficient resource allocation can be used to make the display image of the display clearer.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention. Anyone with ordinary knowledge in the relevant technical field can make some changes and modifications without departing from the spirit and scope of the present invention. The protection scope of the present invention shall be subject to those defined by the attached patent application scope.

100: computer system 110: Image capture device 120: display 130: processor S210~S250: steps

FIG. 1 is a block diagram of components of a computer system according to an embodiment of the invention. FIG. 2 is a flowchart of an image compensation method according to an embodiment of the invention.

S210~S250: steps

Claims (10)

An image compensation method, including: Determine a difference between a first resolution information and a second resolution information, where the first resolution information is related to a first image captured by an image capturing device, and the second resolution information is related A second image presented on a display; In response to the difference between the first resolution information and the second resolution information, the first image is analyzed to generate an image adjustment parameter, where the image adjustment parameter is related to enhanced image clarity, and the difference is the The resolution corresponding to the first resolution information is lower than the resolution corresponding to the second resolution information; and The first image is adjusted according to the image adjustment parameter and the second resolution information to form the second image, wherein the second image is used for presentation by the display. The image compensation method according to claim 1, wherein the first resolution information includes the resolution height or width of the first image, and the second resolution information includes the resolution height or width of the display , And the step of determining the difference between the first resolution information and the second resolution information includes: Compare the height of the resolution of the first image with the height of the resolution of the display, or compare the width of the resolution of the first image with the width of the resolution of the display; and analyze the first image to The steps of generating the image adjustment parameters include: According to the comparison result, the corresponding image adjustment parameters are determined. According to the image compensation method described in claim 1, wherein the step of analyzing the first image to generate the image adjustment parameters includes: Determining the texture level of the first image, wherein the corresponding edge intensity of each pixel in the first image is counted, and the texture level of the first image is obtained according to the statistical result; and The corresponding image adjustment parameter is determined according to the level of the texture, where If the texture level is higher, the intensity of the image adjustment parameter is higher, and If the texture level is lower, the intensity of the image adjustment parameter is lower. According to the image compensation method described in claim 1, wherein the step of analyzing the first image to generate the image adjustment parameters includes: Determining the dynamic level of the first image, wherein the corresponding motion vector (motion vector) of at least one block in the first image is counted, and the dynamic level of the first image is obtained according to the statistical result; and The corresponding image adjustment parameter is determined according to the level of the dynamic level, where If the dynamic level is higher, the intensity of the image adjustment parameter is higher, and If the dynamic level is lower, the intensity of the image adjustment parameter is lower. According to the image compensation method described in claim 1, wherein the step of analyzing the first image to generate the image adjustment parameters includes: Determining at least one of a texture level and a dynamic level of the first image, wherein the texture level is related to edge strength, and the dynamic level is related to a dynamic vector; The difference between the first resolution information and the second resolution information, and the corresponding weight value of the at least one of the texture level and the dynamic level are given, where If the difference, the texture level, or the dynamic level is higher, the corresponding weight value is higher, and If the difference, the texture level, or the dynamic level is lower, the corresponding weight value is lower; and The corresponding image adjustment parameter is determined according to the difference and the corresponding weight value of the at least one of the texture level and the dynamic level. A computer system including: An image capture device to capture a first image; A display that presents a second image; and A processor, coupled to the image capture device and the display, and configured to: Determining a difference between a first resolution information and a second resolution information, wherein the first resolution information is related to the first image, and the second resolution information is related to the second image; In response to the difference between the first resolution information and the second resolution information, the first image is analyzed to generate an image adjustment parameter, where the image adjustment parameter is related to enhanced image clarity, and the difference is the The resolution corresponding to the first resolution information is lower than the resolution corresponding to the second resolution information; and The first image is adjusted according to the image adjustment parameter and the second resolution information to form the second image, wherein the second image is displayed on the display. For the computer system described in claim 6, wherein the first resolution information includes the height or width of the resolution of the first image, and the second resolution information includes the height or width of the resolution of the display, And the processor is further configured to: Comparing the height of the resolution of the first image with the height of the resolution of the display, or comparing the width of the resolution of the first image with the width of the resolution of the display; and According to the comparison result, the corresponding image adjustment parameters are determined. The computer system described in item 6 of the scope of patent application, wherein the processor is further configured to: Determining the texture level of the first image, wherein the corresponding edge intensity of each pixel in the first image is counted, and the texture level of the first image is obtained according to the statistical result; and The corresponding image adjustment parameter is determined according to the level of the texture, where If the texture level is higher, the intensity of the image adjustment parameter is higher, and If the texture level is lower, the intensity of the image adjustment parameter is lower. The computer system described in item 6 of the scope of patent application, wherein the processor is further configured to: Determining the dynamic level of the first image, wherein the corresponding dynamic vector of at least one block in the first image is counted, and the dynamic level of the first image is obtained according to the statistical result; and The corresponding image adjustment parameter is determined according to the level of the dynamic level, where If the dynamic level is higher, the intensity of the image adjustment parameter is higher, and If the dynamic level is lower, the intensity of the image adjustment parameter is lower. The computer system described in item 6 of the scope of patent application, wherein the processor is further configured to: Determining at least one of a texture level and a dynamic level of the first image, wherein the texture level is related to edge strength, and the dynamic level is related to a dynamic vector; The difference between the first resolution information and the second resolution information, and the corresponding weight value of the at least one of the texture level and the dynamic level are given, where If the difference, the texture level, or the dynamic level is higher, the corresponding weight value is higher, and If the difference, the texture level, or the dynamic level is lower, the corresponding weight value is lower; and The corresponding image adjustment parameter is determined according to the difference and the corresponding weight value of the at least one of the texture level and the dynamic level.

Images