WO2021104394A1

WO2021104394A1 - Image processing method and apparatus, electronic device, and storage medium

Info

Publication number: WO2021104394A1
Application number: PCT/CN2020/131892
Authority: WO
Inventors: 姚坤
Original assignee: RealMe重庆移动通信有限公司
Priority date: 2019-11-28
Filing date: 2020-11-26
Publication date: 2021-06-03
Also published as: CN110889809A; CN110889809B; US20220261961A1; CN110889809B9; EP4053784A1; EP4053784A4

Abstract

An image processing method and apparatus, an electronic device, and a storage medium, relating to the technical field of image processing. The method comprises: obtaining a first image and a second image, wherein the resolution of the second image is greater than that of the first image (S210); determining difference value information between a target pixel point in the second image and a reference pixel point, corresponding to the target pixel point, in the first image (S220); and according to the difference value information, performing difference value processing on a preset image that conforms to the resolution of the first image, so as to obtain a target image conforming to the resolution of the second image (S230). The method can obtain the target image according to the difference value information, and improve image quality.

Description

Image processing method and device, electronic equipment, and storage medium

This application claims the priority of the Chinese patent application filed on November 28, 2019, with the application number 201911194628.9, titled "Image processing method and device, electronic equipment, storage medium", the entire content of which is incorporated into this application by reference in.

Technical field

The present disclosure relates to the field of image processing technology, and in particular, to an image processing method, an image processing device, an electronic device, and a computer-readable storage medium.

Background technique

In the process of capturing images and videos through a terminal device with multiple cameras, in order to improve the shooting effect, high-pixel cameras are mostly used to capture high-resolution images.

In the related art, when high-resolution images are continuously acquired, the imaging effect and image quality of the images are poor. In addition, obtaining high-resolution images will lead to higher power consumption of the terminal equipment.

Summary of the invention

The purpose of the present disclosure is to provide an image processing method and device, electronic equipment, and storage medium, so as to overcome at least to a certain extent the problem of poor image quality caused by limitations and defects of related technologies.

Other characteristics and advantages of the present disclosure will become apparent through the following detailed description, or partly learned through the practice of the present disclosure.

According to one aspect of the present disclosure, there is provided an image processing method, including: acquiring a first image and a second image, the resolution of the second image is greater than the resolution of the first image; determining that the second image The difference information between the target pixel point of the first image and the reference pixel point corresponding to the target pixel point in the first image; according to the difference information, the difference information is determined in a preset image that meets the resolution of the first image Perform difference processing on all pixels of, to obtain a target image that meets the resolution of the second image.

According to one aspect of the present disclosure, there is provided an image processing device, including: an image acquisition module for acquiring a first image and a second image, the resolution of the second image is greater than the resolution of the first image; A value information determining module, used to determine the difference information between a target pixel in the second image and a reference pixel in the first image corresponding to the target pixel; a target image generating module, used to Perform difference processing on all pixels in the preset image that meets the resolution of the first image according to the difference information, and obtain a target image that meets the resolution of the second image.

According to one aspect of the present disclosure, there is provided an electronic device, including: an image sensor; a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to obtain information from the image sensor The executable instruction is executed on the image to execute the image processing method described in any one of the above.

According to one aspect of the present disclosure, there is provided a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the image processing method described in any one of the above is implemented.

It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure.

Description of the drawings

The drawings herein are incorporated into the specification and constitute a part of the specification, show embodiments consistent with the disclosure, and are used together with the specification to explain the principle of the disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.

FIG. 1 schematically shows a schematic diagram of a system architecture for implementing an image processing method in an embodiment of the present disclosure.

Fig. 2 schematically shows a schematic diagram of an image processing method in an embodiment of the present disclosure.

Fig. 3 schematically shows a flow chart of determining difference information in an embodiment of the present disclosure.

Fig. 4 schematically shows a flowchart of updating difference information in an embodiment of the present disclosure.

Fig. 5 schematically shows a flow chart of outputting a target image in an embodiment of the present disclosure.

Fig. 6 schematically shows a block diagram of an image processing device in an embodiment of the present disclosure.

Fig. 7 schematically shows a block diagram of an electronic device in an embodiment of the present disclosure.

Detailed ways

Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms, and should not be construed as being limited to the examples set forth herein; on the contrary, the provision of these embodiments makes the present disclosure more comprehensive and complete, and fully conveys the concept of the example embodiments To those skilled in the art. The described features, structures or characteristics can be combined in one or more embodiments in any suitable way. In the following description, many specific details are provided to give a sufficient understanding of the embodiments of the present disclosure. However, those skilled in the art will realize that the technical solutions of the present disclosure can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. can be used. In other cases, the well-known technical solutions are not shown or described in detail to avoid overwhelming the crowd and obscure all aspects of the present disclosure.

In addition, the drawings are only schematic illustrations of the present disclosure, and are not necessarily drawn to scale. The same reference numerals in the figures denote the same or similar parts, and thus their repeated description will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in the form of software, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and/or processor devices and/or microcontroller devices.

FIG. 1 schematically shows a system architecture diagram for implementing the image processing method. As shown in FIG. 1, the system architecture 100 may include a first end 101, a network 102, and a second end 103. Among them, the first terminal 101 may be a client, for example, various handheld devices (smart phones), desktop computers, in-vehicle devices, wearable devices, etc. that have a camera function and are equipped with multiple cameras. The network 102 is used to provide the medium of the communication link between the first end 101 and the second end 103. The network 102 may include various connection types, such as wired communication links, wireless communication links, etc., in the embodiments of the present disclosure. Among them, the network 102 between the first end 101 and the second end 103 may be a wired communication link, for example, a communication link may be provided through a serial port cable, or may be a wireless communication link, and a communication link may be provided through a wireless network. The second end 103 may be a client terminal, such as a portable computer, a desktop computer, a smart phone, and other terminal devices that have a camera function and an image processing function, and are used to synthesize images or perform other image processing. Wherein, when the first end and the second end are both clients, they may be the same client.

It should be understood that the numbers of the first end, the network, and the second end in FIG. 1 are only illustrative. There can be any number of clients, networks, and servers according to implementation needs.

It should be noted that the image processing method provided by the embodiments of the present disclosure can be completely executed by the second end or the first end, or partly executed by the first end and partly executed by the second end. The executive body does not make special restrictions. Correspondingly, the image processing device can be arranged in the second end 103 or in the first end 101.

In the embodiments of the present disclosure, cameras with different pixels in the client can respectively capture a first image and a second image with different resolutions, and then calculate the reference pixel points of the first image and the target pixel points of the second image. The difference information can further process the preset image into a high-resolution target image based on the difference information and the low-resolution preset image. Therefore, the process of outputting high-resolution images is avoided, and the image quality is improved.

In the image processing method, image processing device, electronic device, and computer-readable storage medium provided in the embodiments of the present disclosure, the target pixel in the second image and the reference pixel corresponding to the target pixel in the first image The difference information is used to perform difference processing on a preset image with the same resolution as the first image, so as to convert the preset image into a target image with a resolution of the second image. Since there is no need to obtain a high-resolution second image all the time, noise caused by high-resolution images can be avoided. The target image is synthesized through low-resolution preset images and difference information, so that only low-resolution images can be used. The effect of outputting high-resolution images reduces the problem of low signal-to-noise ratio caused by high resolution, reduces image noise, improves imaging effects, and improves image quality. On the other hand, since there is no need to always acquire high-resolution images, the difference information between the target pixel in the second image and the reference pixel in the first image can be determined, and the low-resolution preset The image is converted into a high-resolution target image, which reduces the problem of high power consumption caused by continuous output of high-resolution images in related technologies, reduces the power consumption of the terminal device, and improves the performance and reliability of the terminal device , It also increases applicability.

On the basis of the foregoing system architecture, an image processing method is provided in the embodiments of the present disclosure, which can be applied to any application scenario that uses a camera to collect images and merge images. Referring to FIG. 2, the image processing method may include step S210 to step S230, which are described in detail as follows:

In step S210, a first image and a second image are acquired, and the resolution of the second image is greater than the resolution of the first image.

In the embodiments of the present disclosure, multiple cameras may be used to photograph the scene and the objects to be photographed in the scene. The number of the multiple cameras can be two, three, four, etc., which is not limited here. Multiple cameras may be provided in one terminal device. The terminal device may be, for example, an electronic device that can take pictures such as a smart phone, as long as it has multiple cameras. The multiple cameras may include cameras with different pixels, and may specifically include a first camera and a second camera, etc., where the first camera may be a low-pixel camera, and the second camera may be a high-pixel camera. To facilitate the distinction, the image captured by the first camera may be used as the first image, and the image captured by the second camera may be used as the second image. The specific pixels of the first camera and the second camera can be determined according to actual needs, as long as the pixels of the first camera are lower than the second camera, which is not limited here. For example, the first camera may be 16 million pixels, and the second camera may be 64 million pixels or 100 million pixels, etc., which is not limited here. The second camera has a low signal-to-noise ratio in a low-light scene. The signal-to-noise ratio is the ratio of signal to noise in the image signal processing system. The larger the ratio, the smaller the noise and the better the effect.

The first image and the second image may be obtained after shooting the same scene, for example, the first image and the second image are respectively shot at the same time or sequentially. The first image and the second image may also be obtained after shooting different scenes, but the different scenes may include part of the same or corresponding specific scenes, for example, different scenes include the sky, a certain person, and so on. Since the pixels of the camera are different, and the pixels are used to describe the resolution, the resolution of the first image and the second image obtained are different. Specifically, the resolution of the first image obtained by the first camera may be smaller than the resolution of the second image.

In the embodiments of the present disclosure, both the first image and the second image may be images taken in any mode of the camera in the terminal device, for example, may be a portrait mode, a professional mode, etc., which are not limited here. The format of the first image and the second image may be RAW (RAW Image Format). RAW is an unprocessed and uncompressed format. RAW can be conceptualized as original image encoding data. The format of the first image and the second image can also be YUV format. Y in YUV represents brightness, which is gray value; while UV represents chromaticity, which is used to describe the color and saturation of the image, and is used to specify The color of the pixel. In the embodiments of the present disclosure, the formats of the first image and the second image are not limited.

It should be noted that the first image and the second image here are used to determine the difference between different cameras, so the first image and the second image can be part of the frame images taken by each camera, such as any frame of image . Specifically, the first image may be a frame of image data collected by the first camera, and the second image may be a frame of image data collected by the second camera. Before acquiring the first image and the second image, if the cameras are not turned on, the first camera and the second camera may be turned on first to acquire a frame of image as the first image and the second image.

In step S220, the difference information between the target pixel in the second image and the reference pixel in the first image corresponding to the target pixel is determined.

In the embodiments of the present disclosure, the target pixel refers to a pixel in the second image. For example, each pixel in the second image may be used as the target pixel, so as to traverse the entire second image. When determining the target pixel point, one can be selected randomly or selected in sequence, which is not limited here. The reference pixel point in the first image corresponding to the target pixel point refers to the pixel point in the neighborhood corresponding to the target pixel point in the first image, and the target pixel point corresponding neighborhood is adjacent to the target pixel point The area range may specifically be a plurality of pixels located outside the target pixel point and having an equal or unequal distance from the target pixel point in the area range. For example, reference pixels can be pixels corresponding to 4 neighborhoods, D neighborhoods, and 8 neighborhoods, and the number of reference pixels can be 4 or 8 within the range of adjacent areas, etc., and the reference pixel The more the number of points, the more accurate the difference information obtained. In the embodiment of the present disclosure, a pixel with a 4-neighborhood reference pixel is taken as an example for description.

The difference information refers to the pixel deviation value or pixel deviation between the pixel value of the target pixel in the second image and the pixel value of each reference pixel in the first image. Since each pixel can be used as a target pixel, the difference information can represent the deviation of the pixel value of the entire first image and the second image.

Fig. 3 schematically shows a flow chart for determining the difference information, referring to Fig. 3, which mainly includes step S310 to step S340, in which:

In step S310, each of the second images is used as the target pixel. For example, according to the arrangement order of the pixels in the second image, each pixel can be used as the target pixel in turn. It is also possible to randomly select a pixel in the second image as the target pixel until all the pixels are processed, which is not limited here. For example, a certain pixel f (x, y) in the second image can be used as the target pixel, where x, y represent the position coordinates of the target pixel, and f (x, y) represents the The pixel value of the target pixel.

In step S320, the coordinates corresponding to the target pixel are determined in the first image according to the coordinates of the target pixel to determine the reference pixel in the first image.

In the embodiments of the present disclosure, on the basis of determining the coordinates of the target pixel, the reference pixel in the first image may be determined according to the coordinates of the target pixel. Specifically, according to the coordinates of the target pixel point, the adjacent coordinates of the coordinate can be determined as the coordinate corresponding to it in the first image, and then the position of the reference pixel point can be determined. For example, the target pixel in the second image is f(x, y), and the adjacent 4 pixels can be pixels located at the top, bottom, left, and right of the target pixel. The calculated first pixel corresponds to the coordinates of the target pixel. The coordinates of the positions of the adjacent 4 pixels in the image can be g(x,y),g(x,y+1),g(x+1,y),g(x+1,y+1) . It should be noted that the x and y in the two coordinates do not represent the same coordinate point, but only represent the coordinate position in the two images. After the coordinates of the reference pixel point are obtained, the pixel point located at the coordinate can be used as the reference pixel point in the first image according to these coordinates.

In step S330, the difference coefficient between the pixel value of the target pixel and the pixel value of the reference pixel is obtained.

In the embodiment of the present disclosure, after the target pixel is obtained, the pixel value of the target pixel can be obtained; at the same time, the pixel value of the reference pixel can also be obtained. The difference coefficient refers to the pixel difference between each target pixel and its corresponding reference pixel. Specifically, it may be the pixel difference between the pixel value of each target pixel and the pixel value of each reference pixel.

Specifically, it can be calculated using a bilinear difference algorithm. The bilinear difference is a linear interpolation extension of the difference function of two variables, and its core idea is to perform a linear difference in two directions respectively. The bilinear difference algorithm makes full use of the four real pixel values around the virtual point in the source image to jointly determine a pixel value in the image, so the zoom effect is better.

Based on this, it is possible to calculate preset difference coefficients in multiple directions between the pixel value of the target pixel and the pixel value of the reference pixel according to multiple directions, so as to obtain the difference coefficient. The multiple directions may be the X direction and the Y direction, and each direction may correspond to a preset difference coefficient, and the preset difference coefficients in different directions may be the same or different, which is not specifically limited. For the X direction, the linear difference in the X direction is performed according to the combination of the X coordinate of the reference pixel, the X coordinate of the target pixel, and the respective corresponding pixel values. For the Y direction, the linear difference in the Y direction is performed according to the combination of the Y coordinate of the reference pixel, the Y coordinate of the target pixel, and the linear difference in the X direction. When performing the bilinear difference, the X-direction linear difference can be performed first, or the Y-direction linear difference can be performed first, which is not limited here. For example, a bilinear difference algorithm can be used to perform linear difference between the target pixel and its corresponding reference pixel in the X direction and the Y direction, respectively, to obtain the preset difference coefficient α in the X direction and the Y direction. The preset difference coefficient β on. Further, the preset difference coefficients in multiple directions may be spliced to obtain the difference coefficient between the target pixel and the reference pixel, which may be expressed as (α, β), for example.

In step S340, the difference information is generated according to the difference coefficient.

In the embodiment of the present disclosure, after the difference coefficient of a target pixel is obtained, each pixel in the second image can be used as the target pixel in the same way, and then each pixel in the second image can be compared with each pixel in the second image. A difference calculation is performed between corresponding reference pixels in the first image to obtain a difference coefficient for each pixel of the second image. Further, the difference coefficients of each pixel can be combined to generate a difference information. The number of difference coefficients contained in the difference information can be the same as the number of pixels in the second image, and the difference The information can be represented by a table of difference coefficients. On the basis of the coordinates of the target pixel, the difference information can be correspondingly recorded as w(x, y) according to the position of each pixel in the second image.

In step S230, performing difference processing on a preset image conforming to the resolution of the first image according to the difference information, and obtaining a target image conforming to the resolution of the second image.

In the embodiment of the present disclosure, the preset image can be any graphic with the same resolution as the first image, for example, it can be other frame images in a certain video except the first image; it can also be a continuously shot image except the first image. Frame images other than one image. The preset image may be collected by the first camera after the first image. The preset image may be collected by the first camera, so the resolution of the preset image may conform to the resolution of the first image, that is, low resolution. The target image refers to the image obtained by adjusting the pixel value and the resolution of the preset image. The resolution of the target image is different from the preset image, and the resolution of the target image is the same as the resolution of the second image. Therefore, it can be considered that the resolution of the target image is the same as that of the image captured by the second camera.

Specifically, the execution process of step S230 may include: performing a logical operation on the pixel values of all pixels in the preset image according to the difference information, and adjusting the pixel values of all pixels to match the resolution of the second image. To get the target image. In the embodiment of the present disclosure, after the difference information between the first image and the second image is obtained, the difference information can be fused with the preset image obtained by the first camera, and the result of the fusion between the pixel values And the conversion result converts the preset image that meets the resolution of the first image into an image that meets the resolution of the second image, thereby using the converted image as the target image. Since the resolution of the target image is the same as the resolution of the second image captured by the second camera, the target image can be used to replace the image captured by the second camera.

For example, after the difference coefficient table is obtained, it can be calculated according to the difference coefficient table between the first image and the second image, combined with the pixel values of the pixels of the preset image whose resolution matches the first image The resolution is consistent with the pixel value of the pixel in the high-definition target image of the second image, and the calculation formula may be as shown in formula (1):

f(x,y)=w(x,y)★{g(x,y),g(x,y+1),g(x+1,y),g(x+1,y+1) } Formula 1)

Among them, the symbol ★ only means that the difference is performed by the bilinear difference method.

In the process of performing difference processing on pixels, specifically, the pixel values of the pixels are converted. For a low-resolution preset image, the pixel value of multiple neighborhood pixels in each frame of the preset image can be converted into a high-resolution pixel value through the difference coefficient table . Repeat this until the pixel values of all pixels in all frames in the preset image are converted and fused, so as to obtain the corresponding pixel values according to the pixel values of all pixels in all frames in the low-resolution preset image collected by the first camera. The pixel value of the high-resolution pixel point, thereby generating the target image.

In the technical solution provided by the embodiments of the present disclosure, since the target image can basically be obtained by background collection and synthesis, the first camera is used for continuous shooting during the shooting process, so the second camera can greatly reduce the shooting time. Power consumption. Through the fusion of the preset image collected by the first camera and the difference information, the target image with the same resolution as the second image can be output, which improves the image quality. At the same time, in a low light environment, because the first camera is used for shooting, the high signal-to-noise ratio will greatly improve the image effect. On the basis of obtaining the same image effect as the second camera, the image quality is improved .

What needs to be added is that after the first image and the second image are collected, the second camera can be turned off, only the low-pixel first camera is used for shooting, and the calculated difference information between the first image and the second image is used Perform image fusion to output an image with the same resolution as that taken by the second camera. Since the second camera is only used to collect a small number of images in order to calculate the difference information and update the difference information, after collecting a frame of the second image, the second camera can be turned off to reduce the use time of the high-pixel camera. The power consumption of the terminal equipment is reduced, the risk of heat generation of the terminal equipment is also avoided, and the reliability is improved. And it can improve the quality and image quality of the captured images, obtain high-quality effects through low power consumption, and improve the user experience.

Further, in order to ensure the accuracy of the result, the second auxiliary image may be acquired through a preset period, and the difference coefficient may be updated according to the acquired second auxiliary image, so as to obtain the difference information according to the updated difference coefficient . The preset period can be, for example, any suitable time interval, such as 30 seconds or 2 minutes, and so on. The second auxiliary image here may be an image of another frame re-acquired by the second camera. The second auxiliary image may be different from the second image, but the resolution of the second auxiliary image and the resolution of the second image are the same. For example, the second auxiliary image may be one frame of image or multiple frames of image, and whether to collect one frame of image or multiple frames of image may be specifically determined according to the reference information. The reference information here may be the memory status of the terminal device and/or the CPU running status. If the reference information meets the update condition, multiple frames of second auxiliary images are collected. If it is not satisfied, only one frame of the second auxiliary image is collected. Specifically, the memory situation can be described by the remaining memory space, and a first threshold can be provided to evaluate the memory situation. The CPU operating condition can be represented by the CPU operating speed, and a second threshold can be provided to evaluate the CPU operating condition. On this basis, the reference information can be determined when the memory condition is greater than the first threshold, or only when the CPU operating condition is greater than the second threshold, or when the memory condition is greater than the first threshold and the CPU operating condition is greater than the second threshold, etc. Meet the update condition to collect multiple frames of second auxiliary images. Otherwise, only one frame of the second auxiliary image is collected. Collecting the second auxiliary image at a preset period and then performing real-time correction of the difference information can make the determined difference information more accurate.

After at least one frame of the second auxiliary image is acquired, the target pixel can be re-determined according to the at least one frame of the second auxiliary image, and the difference coefficient can be updated according to the re-determined target pixel and the reference pixel corresponding to the re-determined target pixel. Among them, for each frame of the second auxiliary image, a target pixel can be re-determined. After the second auxiliary image is determined, an image corresponding to the scene can be selected from the images collected by the first camera as the first auxiliary image. Then determine the pixel value of the target pixel according to the method in step S210 to step S220, and the preset difference between the pixel value of the reference pixel in the first auxiliary image corresponding to the target pixel in the x direction and the y direction The difference coefficient of the coefficient composition. There may still be multiple reference pixels in the first auxiliary image, for example, 4 or 8, and so on. Further, it is possible to traverse all pixels in the second auxiliary image to obtain difference information corresponding to the second auxiliary image, and use the difference information to synthesize the target image.

Fig. 4 schematically shows a flowchart for updating the difference information, which specifically includes step S410 and step S420, where:

In step S410, the target pixel of each frame of the second auxiliary image is re-determined, and the difference coefficient of each frame of the second auxiliary image is determined according to the re-determined target pixel of each second auxiliary image and the reference pixel corresponding to it. .

In the embodiment of the present disclosure, firstly, it can be judged whether the second auxiliary image is a frame, which can be specifically determined according to the reference information. If the second auxiliary image is a frame, the original difference coefficients are directly updated according to the difference coefficients composed of all pixels in the second auxiliary image in multiple directions. If the second auxiliary image is multiple frames, the target pixel of each second auxiliary image can be re-determined, and in multiple directions, the target pixel of each second auxiliary image and the corresponding reference pixel are determined to conform to each frame The difference coefficient of the second auxiliary image. Here, the number of difference coefficients may be determined according to the number of frames of the second auxiliary image and the number of pixels in each frame. The difference coefficient conforming to the second auxiliary image of each frame may be the pixel deviation between the pixel value of each target pixel in each frame of the second auxiliary image and the pixel value of the corresponding reference pixel.

In step S420, the difference coefficients of each frame of the second auxiliary image are averaged to obtain an average difference coefficient, and the difference information is updated according to the average difference coefficient.

In the embodiment of the present disclosure, after the difference coefficient of each frame of the second auxiliary image is obtained, in order to improve the accuracy, all the difference coefficients may be averaged to obtain the average difference coefficient. Further, the average difference coefficient may be used as the difference coefficient of each pixel in the second auxiliary image of multiple frames, so as to update the original second image according to the difference coefficient of each pixel in the second auxiliary image of the multiple frames. The difference coefficient corresponding to the target pixel point realizes the dynamic update of the difference coefficient. Furthermore, the entire difference information is updated according to the combination of the difference coefficient of each pixel. It should be noted that after the updated difference coefficient is obtained, the original difference information can be updated according to the updated difference coefficient to obtain the difference coefficients of all pixels corresponding to the second auxiliary image of multiple frames The difference information composed of w(x,y). That is, the difference coefficient of one pixel of the second auxiliary image of each frame is first calculated, averaged and combined, so as to update the difference information. In addition, it is also possible to perform averaging processing after calculating the difference coefficients of all pixels of the second auxiliary image of each frame, so as to update the difference information. By selecting multiple frames of second auxiliary images and selecting multiple neighborhood pixels to correct and update the difference coefficients, more accurate difference information can be obtained, and the quality of the output target image can be made higher based on the accurate difference information , Improve the image effect.

Fig. 5 schematically shows the entire flow chart of image processing, referring to Fig. 5, it mainly includes the following steps:

In step S510, a frame of a first image is acquired, and the pixel value of a certain pixel in the first image is g(x, y).

In step S520, a frame of a second image is acquired, and the pixel value of a certain pixel in the second image is f(x, y).

In step S530, the difference coefficient of each pixel is obtained to form difference information, that is, the difference coefficient table w(x, y).

In step S540, a high-resolution target image is obtained through a bilinear difference algorithm.

In step S550, the target image is continuously output.

The technical solution in Figure 5 calculates the difference coefficient of each pixel of the low-resolution and high-resolution images, and then performs the difference through the difference coefficient in the later stage to achieve high-resolution output only by shooting with a low-pixel camera. The effect of the video image. Since a low-pixel camera is used for shooting, the signal-to-noise ratio can be improved, so the image effect can be greatly improved. Since a low-resolution preset image is captured by a low-pixel camera, the preset image and difference information are then fused to obtain a high-resolution target image, and the synthesized target image is used as an image collected by the high-pixel camera. In the process of continuously outputting images and videos, the fusion operation can avoid the continuous shooting of high-pixel cameras, which can reduce the use time of high-pixel cameras, thereby reducing power consumption, and avoiding the signal-to-noise comparison caused by high-pixel cameras The low problem reduces the noise of the image and improves the image quality.

In an embodiment of the present disclosure, an image processing device is provided. With reference to FIG. 6, the image processing device 600 may include: an image acquisition module 601 for acquiring a first image and a second image. The resolution of is greater than the resolution of the first image; the difference information determination module 602 is used to determine the target pixel in the second image and the reference pixel in the first image corresponding to the target pixel The difference information between the points; the target image generation module 603 is configured to perform difference processing on a preset image that conforms to the resolution of the first image according to the difference information, and obtain a resolution that conforms to the resolution of the second image Target image.

In an exemplary embodiment of the present disclosure, the difference information determination module includes: a target pixel designation module, configured to use each pixel in the second image as the target pixel; and a reference pixel The determining module is configured to determine the corresponding coordinates in the first image according to the coordinates of the target pixel to determine the reference pixel in the first image; the difference coefficient determining module is configured to obtain The difference coefficient between the pixel value of the target pixel and the pixel value of the reference pixel; a difference coefficient combination module for generating the difference information according to the difference coefficient.

In an exemplary embodiment of the present disclosure, the difference coefficient determination module is configured to: calculate preset difference coefficients between the pixel value of the target pixel and the pixel value of the reference pixel in multiple directions , To obtain the difference coefficient.

In an exemplary embodiment of the present disclosure, the target image generation module is configured to: perform a logical operation on the pixel values of all pixels in the preset image according to the difference information, and then combine all the pixels The pixel value of is adjusted to a pixel value that meets the resolution of the second image to obtain the target image.

In an exemplary embodiment of the present disclosure, the device further includes: a difference information update module, configured to obtain a second auxiliary image through a preset period, and compare the difference coefficient according to the second auxiliary image The update is performed to determine the difference information according to the updated difference coefficient.

In an exemplary embodiment of the present disclosure, the difference information update module includes: an auxiliary image acquisition module, configured to acquire at least one frame of the second auxiliary image according to the reference information; The at least one frame of second auxiliary image re-determines the target pixel, updates the difference coefficient according to the re-determined target pixel and the corresponding reference pixel, and determines the difference information according to the updated difference coefficient.

In an exemplary embodiment of the present disclosure, if the second auxiliary image is of multiple frames, the difference coefficient update module is configured to: re-determine the target pixel of each frame of the second auxiliary image. The direction determines the difference coefficient of each frame of the second auxiliary image according to the re-determined target pixel point of each frame of the second auxiliary image and the corresponding reference pixel point; the difference coefficient of each frame of the second auxiliary image is averaged Obtain an average difference coefficient, and update the difference information according to the average difference coefficient.

It should be noted that the specific details of each module in the above-mentioned image processing device have been described in detail in the corresponding image processing method, and therefore will not be repeated here.

It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above may be embodied in one module or unit. Conversely, the features and functions of a module or unit described above can be further divided into multiple modules or units to be embodied.

In addition, although the various steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that these steps must be performed in the specific order, or that all the steps shown must be performed to achieve the desired result. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step for execution, and/or one step may be decomposed into multiple steps for execution, etc.

In the embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.

Those skilled in the art can understand that various aspects of the present disclosure can be implemented as a system, a method, or a program product. Therefore, various aspects of the present disclosure can be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "Circuit", "Module" or "System".

The electronic device 700 according to this embodiment of the present disclosure will be described below with reference to FIG. 7. The electronic device 700 shown in FIG. 7 is only an example, and should not bring any limitation to the function and scope of use of the embodiments of the present disclosure.

As shown in FIG. 7, the electronic device 700 is represented in the form of a general-purpose computing device. The components of the electronic device 700 may include, but are not limited to: the aforementioned at least one processing unit 710, the aforementioned at least one storage unit 720, a bus 730 connecting different system components (including the storage unit 720 and the processing unit 710), a display unit 740, and an image sensor 770 The image sensor 770 is used to obtain images of corresponding resolutions.

Wherein, the storage unit stores program code, and the program code can be executed by the processing unit 710, so that the processing unit 710 executes the various exemplary methods described in the "Exemplary Method" section of this specification. Steps of implementation. For example, the processing unit 710 may perform the steps shown in FIG. 2: in step S210, a first image and a second image are acquired, where the resolution of the second image is greater than the resolution of the first image; In step S220, determine the difference information between the target pixel in the second image and the reference pixel in the first image corresponding to the target pixel; in step S230, according to the difference The value information performs difference processing on a preset image that meets the resolution of the first image, and obtains a target image that meets the resolution of the second image.

The storage unit 720 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 7201 and/or a cache storage unit 7202, and may further include a read-only storage unit (ROM) 7203.

The storage unit 720 may also include a program/utility tool 7204 having a set of (at least one) program module 7205. Such program module 7205 includes but is not limited to: an operating system, one or more application programs, other program modules, and program data, Each of these examples or some combination may include the implementation of a network environment.

The bus 730 may represent one or more of several types of bus structures, including a storage unit bus or a storage unit controller, a peripheral bus, a graphics acceleration interface, a processing unit, or a local area using any bus structure among multiple bus structures. bus.

The electronic device 700 may also communicate with one or more external devices 800 (such as keyboards, pointing devices, Bluetooth devices, etc.), and may also communicate with one or more devices that enable a user to interact with the electronic device 700, and/or communicate with Any device (eg, router, modem, etc.) that enables the electronic device 700 to communicate with one or more other computing devices. This communication can be performed through an input/output (I/O) interface 750. In addition, the electronic device 700 may also communicate with one or more networks (for example, a local area network (LAN), a wide area network (WAN), and/or a public network, such as the Internet) through the network adapter 760. As shown in the figure, the network adapter 760 communicates with other modules of the electronic device 700 through the bus 730. It should be understood that although not shown in the figure, other hardware and/or software modules can be used in conjunction with the electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives And data backup storage system, etc.

In the embodiments of the present disclosure, a computer-readable storage medium is also provided, on which is stored a program product capable of implementing the above-mentioned method in this specification. In some possible implementation manners, various aspects of the present disclosure may also be implemented in the form of a program product, which includes program code. When the program product runs on a terminal device, the program code is used to enable the The terminal device executes the steps according to various exemplary embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of this specification.

The program product for implementing the above method according to the embodiment of the present disclosure may adopt a portable compact disk read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, the readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, device, or device.

The program product can use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or a combination of any of the above. More specific examples (non-exhaustive list) of readable storage media include: electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable Type programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.

The computer-readable signal medium may include a data signal propagated in baseband or as a part of a carrier wave, and readable program code is carried therein. This propagated data signal can take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The readable signal medium may also be any readable medium other than a readable storage medium, and the readable medium may send, propagate, or transmit a program for use by or in combination with the instruction execution system, apparatus, or device.

The program code contained on the readable medium can be transmitted by any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

The program code used to perform the operations of the present disclosure can be written in any combination of one or more programming languages. The programming languages include object-oriented programming languages—such as Java, C++, etc., as well as conventional procedural styles. Programming language-such as "C" language or similar programming language. The program code can be executed entirely on the user's computing device, partly on the user's device, executed as an independent software package, partly on the user's computing device and partly executed on the remote computing device, or entirely on the remote computing device or server Executed on. In the case of a remote computing device, the remote computing device can be connected to a user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (for example, using Internet service providers). Business to connect via the Internet).

In addition, the above-mentioned drawings are merely schematic illustrations of the processing included in the method according to the embodiments of the present disclosure, and are not intended for limitation. It is easy to understand that the processing shown in the above drawings does not indicate or limit the time sequence of these processings. In addition, it is easy to understand that these processes can be executed synchronously or asynchronously in multiple modules, for example.

Those skilled in the art will easily think of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptive changes of the present disclosure. These variations, uses, or adaptive changes follow the general principles of the present disclosure and include common knowledge or conventional technical means in the technical field that are not disclosed in the present disclosure. . The description and the embodiments are only regarded as exemplary, and the true scope and spirit of the present disclosure are pointed out by the claims.

Claims

An image processing method, characterized in that it comprises:

Acquiring a first image and a second image, the resolution of the second image is greater than the resolution of the first image;

Determining difference information between a target pixel in the second image and a reference pixel in the first image corresponding to the target pixel;

Performing difference processing on a preset image that conforms to the resolution of the first image according to the difference information, and obtains a target image that conforms to the resolution of the second image.
The image processing method according to claim 1, wherein the difference information between a target pixel in the second image and a reference pixel in the first image corresponding to the target pixel is determined include:

Use each pixel in the second image as the target pixel;

Determining the corresponding coordinates in the first image according to the coordinates of the target pixel to determine the reference pixel in the first image;

Obtaining a difference coefficient between the pixel value of the target pixel and the pixel value of the reference pixel;

The difference information is generated according to the difference coefficient.
The image processing method according to claim 2, wherein obtaining the difference coefficient between the pixel value of the target pixel and the pixel value of the reference pixel comprises:

Calculate preset difference coefficients in multiple directions between the pixel value of the target pixel and the pixel value of the reference pixel to obtain the difference coefficient.
The image processing method according to claim 3, wherein a preset difference coefficient in multiple directions between the pixel value of the target pixel and the pixel value of the reference pixel is calculated to obtain the difference Value factors include:

The preset difference coefficients in the multiple directions are spliced to obtain the difference coefficient between the target pixel and the reference pixel.
The image processing method according to claim 2, wherein generating the difference information according to the difference coefficient comprises:

The difference coefficient of each pixel is combined to generate the difference information.
The image processing method according to claim 1, wherein, according to the difference information, a preset image conforming to the resolution of the first image is subjected to difference processing to obtain a resolution conforming to the second image The target images include:

Perform a logical operation on the pixel values of all pixels in the preset image according to the difference information, and adjust the pixel values of all pixels to a pixel value that meets the resolution of the second image to obtain the Target image.
The image processing method according to claim 6, wherein the pixel values of all pixels in the preset image are logically operated according to the difference information, and the pixel values of all pixels are adjusted to The pixel values conforming to the resolution of the second image to obtain the target image include:

The difference information is fused with the preset image, and the preset image that meets the resolution of the first image is converted into an image that meets the resolution of the second image through the fusion result and the conversion result between pixel values. Obtain the target image.
The image processing method according to claim 2, wherein the method further comprises:

The second auxiliary image is acquired through a preset period, and the difference coefficient is updated according to the second auxiliary image, so as to determine the difference information according to the updated difference coefficient.
8. The image processing method according to claim 8, wherein the second auxiliary image is acquired through a preset period, and the difference coefficient is updated according to the second auxiliary image, so that the difference coefficient is updated according to the updated difference coefficient. Determining the difference information includes:

Acquiring at least one frame of the second auxiliary image according to the reference information;

The target pixel is re-determined according to the at least one frame of the second auxiliary image, the difference coefficient is updated according to the re-determined target pixel and the corresponding reference pixel, and the difference is determined according to the updated difference coefficient information.
The image processing method according to claim 9, wherein acquiring at least one frame of the second auxiliary image according to reference information comprises:

If the reference information satisfies the update condition, collecting multiple frames of second auxiliary images;

If the reference information does not satisfy the update condition, a frame of second auxiliary image is collected.
The image processing method according to claim 9, wherein if the second auxiliary image is a frame, the difference coefficient is updated according to the re-determined target pixel point and the reference pixel point corresponding thereto, and the difference coefficient is updated according to the update The difference coefficient after determining the difference information includes:

The original difference coefficient is updated according to the difference coefficients composed of all pixels in the second auxiliary image in multiple directions, and the difference information is determined according to the updated difference coefficient.
The image processing method according to claim 9, wherein if the second auxiliary image is a multi-frame, the difference coefficient is updated according to the re-determined target pixel and the reference pixel corresponding to it, and the difference coefficient is updated according to the update The difference coefficient after determining the difference information includes:

Re-determine the target pixel of each frame of the second auxiliary image, and determine the difference coefficient of each frame of the second auxiliary image according to the re-determined target pixel of each frame of the second auxiliary image and its corresponding reference pixel;

Perform averaging processing on the difference coefficients of each frame of the second auxiliary image to obtain an average difference coefficient, and update the difference information according to the average difference coefficient.
An image processing device, characterized in that it comprises:

An image acquisition module for acquiring a first image and a second image, the resolution of the second image is greater than the resolution of the first image;

A difference information determining module, configured to determine difference information between a target pixel in the second image and a reference pixel in the first image corresponding to the target pixel;

The target image generating module is configured to perform difference processing on a preset image conforming to the resolution of the first image according to the difference information, to obtain a target image conforming to the resolution of the second image.
The image processing device according to claim 13, wherein the difference information determining module comprises:

A target pixel designation module, configured to use each pixel in the second image as the target pixel;

A reference pixel determination module, configured to determine the corresponding coordinates in the first image according to the coordinates of the target pixel to determine the reference pixel in the first image;

A difference coefficient determination module, configured to obtain the difference coefficient between the pixel value of the target pixel and the pixel value of the reference pixel;

The difference coefficient combination module is used to generate the difference information according to the difference coefficient.
The image processing device according to claim 14, wherein the difference coefficient determining module is configured to: calculate the difference between the pixel value of the target pixel and the pixel value of the reference pixel in multiple directions The difference coefficient is preset to obtain the difference coefficient.
The image processing device according to claim 13, wherein the target image generation module is configured to: perform a logical operation on the pixel values of all pixels in the preset image according to the difference information, and The pixel values of all the pixels are adjusted to match the resolution of the second image to obtain the target image.
The image processing device according to claim 14, wherein the device further comprises:

The difference information update module is configured to obtain a second auxiliary image through a preset period, and update the difference coefficient according to the second auxiliary image, so as to determine the difference information according to the updated difference coefficient.
The image processing device according to claim 17, wherein the difference information update module comprises:

An auxiliary image acquisition module, configured to acquire at least one frame of the second auxiliary image according to reference information;

The difference coefficient update module is used to re-determine the target pixel according to the at least one frame of the second auxiliary image, update the difference coefficient according to the re-determined target pixel and the corresponding reference pixel, and according to the updated The difference coefficient determines the difference information.
An electronic device, characterized in that it comprises:

Image Sensor;

Processor; and

A memory for storing executable instructions of the processor;

Wherein, the processor is configured to execute the executable instruction on the image acquired by the image sensor to execute the image processing method according to any one of claims 1-12.
A computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the image processing method according to any one of claims 1-12 when the computer program is executed by a processor.