CN115809959A

CN115809959A - Image processing method and device

Info

Publication number: CN115809959A
Application number: CN202111088407.0A
Authority: CN
Inventors: 王涛
Original assignee: Beijing Jigan Technology Co ltd
Current assignee: Beijing Jigan Technology Co ltd
Priority date: 2021-09-16
Filing date: 2021-09-16
Publication date: 2023-03-17

Abstract

The application provides an image processing method and device, which are applied to the field of image processing, and the method comprises the following steps: acquiring a plurality of single-channel images contained in an input image; respectively carrying out scaling processing on a plurality of single-channel images according to the target resolution and the initial resolution corresponding to the input image to obtain a plurality of corresponding scaled images; wherein the target resolution is greater than the initial resolution; splicing a plurality of zoomed images to obtain a single-channel original image; and calculating the pixel values of the pixel points on other channels according to the channel to which the pixel value of each pixel point in the original image belongs and the pixel value of the pixel point related to the pixel point, so as to obtain the target image with the same number of channels as the input image. Because enough nonlinearity can be introduced in the process of splicing and restoring to multiple channels, the definition of the generated target image can be improved; meanwhile, because a large amount of operation processes such as training and the like are not needed, the calculation amount of the whole process can be reduced.

Description

Image processing method and device

Technical Field

The present application relates to the field of image processing, and in particular, to an image processing method and apparatus.

Background

The super-resolution algorithm is to improve the resolution of the original image by a hardware or software method, namely to obtain a high-resolution image by one or more low-resolution images. In the prior art, super-resolution algorithms are mainly divided into two categories: the method mainly comprises the following steps of: interpolation methods, sparse representation methods, local embedding methods, sample-based (Example-based) methods, and the like, and the deep learning methods mainly include various deep learning methods mainly based on neural network models. However, the super-resolution image obtained by the conventional method has low definition, and the process of obtaining the super-resolution image by the deep learning method has a large amount of calculation.

Disclosure of Invention

An object of the embodiments of the present application is to provide an image processing method and apparatus, so as to solve a technical problem of how to reduce a calculation amount in a process of improving a resolution on the basis of ensuring definition of a generated image.

In a first aspect, an embodiment of the present application provides an image processing method, including: acquiring a plurality of single-channel images contained in an input image; respectively carrying out scaling processing on the multiple single-channel images according to a target resolution and an initial resolution corresponding to the input image to obtain multiple corresponding scaled images; wherein the target resolution is greater than the initial resolution; splicing the multiple zoomed images to obtain a single-channel original image; wherein the resolution of the original image is the target resolution; and calculating the pixel values of the pixel points on other channels according to the channel to which the pixel value of each pixel point in the original image belongs and the pixel value of the pixel point related to the pixel point, so as to obtain the target image with the same number of channels as the input image. In the scheme, an input image with an initial resolution ratio is separated into a plurality of single-channel images, the single-channel images subjected to scaling processing are spliced, then an original image obtained by splicing is restored into a multi-channel image, and an obtained target image is an image meeting the target resolution ratio. In the process of splicing and recovering into multiple channels, enough nonlinearity can be introduced, so that the definition of the generated target image can be improved; meanwhile, because a large amount of operation processes such as training and the like are not needed, the calculation amount of the whole process can be reduced.

In an optional embodiment, the channels include a first-type channel, a second-type channel, and a third-type channel, and the plurality of single-channel images include two first images corresponding to the first-type channel, one second image corresponding to the second-type channel, and one third image corresponding to the third-type channel.

In an optional embodiment, the stitching the multiple scaled images to obtain a single-channel original image includes: splicing pixel points at the same position in the zoomed first image, the zoomed second image and the zoomed third image into pixel blocks at corresponding positions in the original image; each pixel block comprises four pixel points, two pixel points on one diagonal line in each pixel block are pixel points of the zoomed second image and the zoomed third image at the position respectively, and two pixel points on the other diagonal line are pixel points of the zoomed first image at the position. In the scheme, the scaled single-channel images are spliced into the original image through the splicing process, so that enough nonlinearity can be introduced, and the definition of the generated target image can be improved.

In an optional embodiment, a pixel point of the original image, the pixel value of which belongs to the first channel, is a first pixel point, a pixel point of the original image, the pixel value of which belongs to the second channel, is a second pixel point, and a pixel point of the original image, the pixel value of which belongs to the third channel, is a third pixel point; the method for calculating the pixel values of the pixel points on other channels according to the channel to which the pixel value of each pixel point in the original image belongs and the pixel value of the pixel point related to the pixel point to obtain the target image with the same number of channels as the input image comprises the following steps: traversing second pixel points and third pixel points in the original image, and determining an interpolation direction in a processing window range with each second pixel point or each third pixel point as a central point; calculating the pixel value of the second pixel point or the third pixel point on a first channel according to the interpolation direction and the pixel value of the pixel point related to the second pixel point or the third pixel point in the processing window range to obtain a first intermediate image; traversing a fourth pixel point corresponding to the second pixel point position and a fifth pixel point corresponding to the third pixel point position in the first intermediate image, and determining an interpolation direction within the processing window range by taking each fourth pixel point or each fifth pixel point as a central point; calculating the pixel value of the fourth pixel point on a third channel or the pixel value of the fifth pixel point on a second channel according to the interpolation direction and the pixel value of the pixel point related to the fourth pixel point or the fifth pixel point in the processing window range to obtain a second intermediate image; traversing sixth pixel points corresponding to the first pixel point positions in the second intermediate image, and determining an interpolation direction within the processing window range by taking each sixth pixel point as a central point; and calculating the pixel values of the sixth pixel point on a second channel and a third channel according to the interpolation direction and the pixel value of the pixel point related to the sixth pixel point in the processing window range to obtain the target image. In the scheme, the multi-channel image can be restored according to the original image by sequentially traversing the pixel points which belong to the single-channel pixel value in the original image and inserting the pixel values of other channels, and the definition of the generated target image can be improved because enough nonlinearity can be introduced in the inserting process; meanwhile, because a large amount of operation processes such as training and the like are not needed, the calculation amount of the whole process can be reduced.

In an optional implementation manner, the traversing second pixel points and third pixel points in the original image, and determining an interpolation direction within a processing window range with each second pixel point or each third pixel point as a center point includes: calculating the horizontal direction gradient and the vertical direction gradient in the processing window range; and determining the direction corresponding to the smaller value of the horizontal direction gradient and the vertical direction gradient as the interpolation direction.

In an optional implementation manner, the traversing fourth pixel points corresponding to the second pixel point position and fifth pixel points corresponding to the third pixel point position in the first intermediate image, and determining an interpolation direction within the processing window range with each fourth pixel point or fifth pixel point as a central point includes: calculating two diagonal direction gradients in the processing window range; and determining the direction corresponding to the smaller value of the two diagonal direction gradients as the interpolation direction.

In an optional implementation manner, the scaling the multiple single-channel images according to the target resolution and the initial resolution corresponding to the input image to obtain multiple corresponding scaled images includes: calculating a zoom rate according to the target resolution, the initial resolution and the total number of the single-channel images; and carrying out scaling processing on the single-channel image according to the scaling rate to obtain the scaled image. In the above scheme, during the process of scaling the single-channel image, a corresponding scaling rate may be calculated according to the target resolution and the initial resolution, so as to appropriately scale the single-channel image according to the scaling rate.

In an optional embodiment, when the scaling rate is greater than 1, the scaling the single-channel image according to the scaling rate to obtain the scaled image includes: for at least one image unit in the single-channel image, if the total number M of pixel points to be inserted supports uniform interpolation, uniformly inserting the M pixel points to be inserted between every two original pixel points in the image unit; if the total number M of the pixel points to be inserted does not support uniform interpolation, uniformly inserting N pixel points to be inserted between every two original pixel points in the image unit; wherein N is a positive integer smaller than M, and M-N +1 is smaller than the number of original pixel points in the image unit; after the N to-be-inserted pixel points are inserted, calculating a to-be-inserted pixel value between every two adjacent pixel points in the image unit; and taking the pixel value to be inserted corresponding to the minimum M-N distances in the calculated distances as the pixel points of the M-N pixel points to be inserted, and inserting the M-N pixel points to be inserted between the pixel points corresponding to the minimum M-N distances. In the above scheme, in the case that the single-channel image needs to be enlarged, the single-channel image may be scaled in an interpolation manner. When the pixel value corresponding to the pixel point to be inserted is calculated, the loss of definition in the interpolation process can be reduced according to the pixel values of two adjacent pixel points and the target definition; in addition, for the pixel points to be inserted which cannot have uniform difference values, the pixel points can be distributed on the broken line or around the broken line as much as possible to increase the overall variance.

In an optional embodiment, when the scaling rate is greater than 1, the scaling the single-channel image according to the scaling rate to obtain the scaled image includes: inserting pixel points among the pixel points on the single-channel image; the pixel value of the inserted pixel point is obtained by calculation according to the pixel values of two pixel points adjacent to the inserted pixel point and the target definition, and the pixel value closer to the central point of the image unit in the single-channel image occupies a larger proportion in the calculation process.

In an alternative embodiment, the image unit comprises at least one of the following objects: a first row, a first column, a last row, and a last column in the single-channel image.

In a second aspect, an embodiment of the present application provides an image processing apparatus, including: the acquisition module is used for acquiring a plurality of single-channel images contained in the input image; the scaling module is used for respectively carrying out scaling processing on the multiple single-channel images according to the target resolution and the initial resolution corresponding to the input image to obtain multiple corresponding scaled images; wherein the target resolution is greater than the initial resolution; the splicing module is used for splicing the multiple zoomed images to obtain a single-channel original image; wherein the resolution of the original image is the target resolution; and the generation module is used for calculating the pixel values of the pixel points on other channels according to the channel to which the pixel value of each pixel point in the original image belongs and the pixel value of the pixel point related to the pixel point, so as to obtain the target image with the same number of channels as the input image. In the scheme, an input image with an initial resolution ratio is separated into a plurality of single-channel images, the single-channel images subjected to scaling processing are spliced, then an original image obtained by splicing is restored into a multi-channel image, and an obtained target image is an image meeting the target resolution ratio. In the process of splicing and restoring to multiple channels, enough nonlinearity can be introduced, so that the definition of the generated target image can be improved; meanwhile, because a large amount of operation processes such as training and the like are not needed, the calculation amount of the whole process can be reduced.

In an optional embodiment, the splicing module is specifically configured to: splicing pixel points at the same position in the zoomed first image, the zoomed second image and the zoomed third image into pixel blocks at corresponding positions in the original image; each pixel block comprises four pixel points, two pixel points on one diagonal line in each pixel block are pixel points of the zoomed first image and the zoomed third image at the position respectively, and two pixel points on the other diagonal line are pixel points of the zoomed second image at the position. In the scheme, the scaled single-channel images are spliced into the original image through the splicing process, so that enough nonlinearity can be introduced, and the definition of the generated target image can be improved.

In an optional embodiment, a pixel point of the original image, the pixel value of which belongs to the first channel, is a first pixel point, a pixel point of the original image, the pixel value of which belongs to the second channel, is a second pixel point, and a pixel point of the original image, the pixel value of which belongs to the third channel, is a third pixel point; the generation module is specifically configured to: traversing second pixel points and third pixel points in the original image, and determining an interpolation direction in a processing window range with each second pixel point or each third pixel point as a central point; calculating the pixel value of the second pixel point or the third pixel point on a first channel according to the interpolation direction and the pixel value of the pixel point related to the second pixel point or the third pixel point in the processing window range to obtain a first intermediate image; traversing a fourth pixel point corresponding to the second pixel point position and a fifth pixel point corresponding to the third pixel point position in the first intermediate image, and determining an interpolation direction within the processing window range by taking each fourth pixel point or each fifth pixel point as a central point; calculating the pixel value of the fourth pixel point on a third channel or the pixel value of the fifth pixel point on a second channel according to the interpolation direction and the pixel value of the pixel point related to the fourth pixel point or the fifth pixel point in the processing window range to obtain a second intermediate image; traversing sixth pixel points corresponding to the first pixel point positions in the second intermediate image, and determining an interpolation direction within the processing window range by taking each sixth pixel point as a central point; and calculating pixel values of the sixth pixel point on a second channel and a third channel according to the interpolation direction and the pixel value of the pixel point related to the sixth pixel point in the processing window range to obtain the target image. In the scheme, the multi-channel image can be restored according to the original image by sequentially traversing the pixel points which belong to the single-channel pixel value in the original image and inserting the pixel values of other channels, and the definition of the generated target image can be improved because enough nonlinearity can be introduced in the inserting process; meanwhile, because a large amount of operation processes such as training and the like are not needed, the calculation amount of the whole process can be reduced.

In an optional embodiment, the generating module is further configured to: calculating the horizontal direction gradient and the vertical direction gradient in the processing window range; and determining the direction corresponding to the smaller value of the horizontal direction gradient and the vertical direction gradient as the interpolation direction.

In an optional embodiment, the generating module is further configured to: calculating two diagonal direction gradients in the processing window range; and determining the direction corresponding to the smaller value of the two diagonal direction gradients as the interpolation direction.

In an alternative embodiment, the scaling module is specifically configured to: calculating a scaling rate according to the target resolution, the initial resolution and the total number of the single-channel images; and carrying out scaling processing on the single-channel image according to the scaling rate to obtain the scaled image. In the above scheme, during the process of scaling the single-channel image, a corresponding scaling rate may be calculated according to the target resolution and the initial resolution, so as to appropriately scale the single-channel image according to the scaling rate.

In an optional embodiment, when the scaling rate is greater than 1, the scaling module is further configured to: for at least one image unit in the single-channel image, if the total number M of pixel points to be inserted supports uniform interpolation, uniformly inserting the M pixel points to be inserted between every two original pixel points in the image unit; if the total number M of the pixel points to be inserted does not support uniform interpolation, uniformly inserting N pixel points to be inserted between every two original pixel points in the image unit; wherein N is a positive integer smaller than M, and M-N +1 is smaller than the number of original pixel points in the image unit; after the N to-be-inserted pixel points are inserted, calculating a to-be-inserted pixel value between every two adjacent pixel points in the image unit; and taking the pixel value to be inserted corresponding to the minimum M-N distances in the calculated distances as the pixel points of the M-N pixel points to be inserted, and inserting the M-N pixel points to be inserted between the pixel points corresponding to the minimum M-N distances. In the above scheme, in the case that the single-channel image needs to be enlarged, the single-channel image may be scaled in an interpolation manner. When the pixel value corresponding to the pixel point to be inserted is calculated, the loss of definition in the interpolation process can be reduced according to the pixel values of two adjacent pixel points and the definition of a target; in addition, for the pixel points to be inserted which cannot have uniform difference values, the pixel points can be distributed on the broken line or around the broken line as much as possible to increase the overall variance.

In an alternative embodiment, the scaling module is further configured to: inserting pixel points among the pixel points on the single-channel image; the pixel value of the inserted pixel point is obtained by calculation according to the pixel values of two pixel points adjacent to the inserted pixel point and the target definition, and the pixel value closer to the central point of the image unit in the single-channel image occupies a larger proportion in the calculation process.

In a third aspect, an embodiment of the present application provides an electronic device, including: a processor, memory, and a bus; the processor and the memory are communicated with each other through the bus; the memory stores program instructions executable by the processor, the processor being capable of performing the image processing method of the first aspect when invoked by the program instructions.

In a fourth aspect, embodiments of the present application provide a non-transitory computer-readable storage medium storing computer instructions which, when executed by a computer, cause the computer to perform the image processing method according to the first aspect.

In a fifth aspect, the present application provides a computer program product comprising a computer program which, when executed by a processor, implements the image processing method according to the first aspect.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and that those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.

Fig. 1 is a flowchart of an image processing method according to an embodiment of the present application;

fig. 2 is a schematic diagram of a pixel block according to an embodiment of the present application;

FIG. 3 is a schematic diagram of another pixel block provided in the embodiment of the present application;

fig. 4 is a flowchart of a specific implementation of the step S102 provided in this embodiment of the present application;

FIG. 5 is a diagram illustrating a pixel within a processing window according to an embodiment of the present disclosure;

FIG. 6 is a diagram illustrating pixels within another processing window according to an embodiment of the present disclosure;

fig. 7 is a block diagram of an image processing apparatus according to an embodiment of the present application;

fig. 8 is a block diagram of an electronic device according to an embodiment of the present disclosure.

Detailed Description

The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.

Referring to fig. 1, fig. 1 is a flowchart of an image processing method according to an embodiment of the present disclosure, where the image processing method can be executed by an electronic device according to an embodiment of the present disclosure. The image processing method may include the following steps:

step S101: a plurality of single-channel images contained in an input image are acquired.

Step S102: and respectively carrying out scaling processing on the plurality of single-channel images according to the target resolution and the initial resolution corresponding to the input image to obtain a plurality of corresponding scaled images.

Step S103: and splicing the plurality of zoomed images to obtain a single-channel original image. Wherein the resolution of the original image is the target resolution.

Step S104: and calculating the pixel values of the pixel points on other channels according to the channel to which the pixel value of each pixel point in the original image belongs and the pixel value of the pixel point related to the pixel point, so as to obtain the target image with the same number of channels as the input image.

Specifically, the image may be divided into a single-channel image, a three-channel image, a four-channel image, and the like. Wherein, each pixel point in the single-channel image only corresponds to a pixel value, and the pixel value only belongs to a channel, for example: for a red single-channel image, the pixel value of each pixel point belongs to a red channel; assuming that the channels in the three-channel image include a first-type channel, a second-type channel, and a third-type channel, each pixel of the three-channel image corresponds to three pixel values, and the three pixel values respectively belong to the first-type channel, the second-type channel, and the third-type channel, for example: for an RGB image, three pixel values corresponding to each pixel point belong to a red channel, a green channel and a blue channel.

In this embodiment of the application, the electronic device may acquire a plurality of single-channel images included in one input image in step S101, where the electronic device may acquire the plurality of single-channel images in a plurality of manners. Taking an input image as an RGB image as an example, the electronic device may obtain an input image, and then separate the input image into three single-channel images, where each single-channel image corresponds to one of a red channel, a green channel, and a blue channel; or, the electronic device may directly receive an image of three single channels corresponding to one input image sent by the external device or read from the cloud server, and similarly, the image of each single channel corresponds to one of the red channel, the green channel, and the blue channel.

It is understood that, as an embodiment, the number of single-channel images acquired by the electronic device may also be greater than three, for example: the method comprises the steps that four single-channel images can be obtained, wherein the four single-channel images comprise two images corresponding to a first type channel, one image corresponding to a second type channel and one image corresponding to a third type channel, and the two images corresponding to the first type channel can be obtained by copying the image corresponding to one of the first type channels; or, five single-channel images including two images corresponding to the first type of channel, one image corresponding to the second type of channel, and two images corresponding to the third type of channel may be obtained, and similarly, the images corresponding to the two first type of channels or the two second type of channels may be obtained by copying one of the images corresponding to the first type of channel or the second type of channel.

Before the electronic device executes step S102, an initial resolution corresponding to the input image and a target resolution corresponding to the target image may also be obtained, that is, the image processing method provided in the embodiment of the present application aims to convert the input image with the initial resolution into the target image with the target resolution. Similar to the above embodiments, the electronic device may obtain the initial resolution and the target resolution in various ways. For example: the electronic device may read the locally stored initial resolution or target resolution; the electronic device can read the stored initial resolution or target resolution from the cloud server; alternatively, the electronic device may also read an initial resolution or a target resolution, etc., input by the user. The embodiments of the present application are not specifically limited, and those skilled in the art can appropriately select the embodiments according to actual situations.

It is understood that the target resolution provided by the embodiments of the present application may be greater than the initial resolution.

As an implementation manner, in addition to the initial resolution and the target resolution, the electronic device may also obtain other parameters required in the process of executing the image processing method provided by the embodiment of the present application, for example: the definition of the target, the size of the target image, etc. can be adjusted by those skilled in the art according to the actual situation.

Next, in step S102, the electronic device may perform scaling processing on the multiple single-channel images according to the acquired target resolution and the initial resolution, so as to obtain a scaled image corresponding to each single-channel image. There are various ways of scaling, for example: the resolution of a single-channel image can be increased in an interpolation mode; the resolution ratio of the single-channel image can be reduced by deleting part of pixel points; the resolution of a single-channel image can be increased by means of local embedding, etc.

It should be noted that, the following embodiments will describe in detail specific implementation processes of increasing the resolution of the single-channel image by means of interpolation and reducing the resolution of the single-channel image by means of deleting some pixel points, which are not described here for the moment.

Then, after obtaining a plurality of scaled images corresponding to the plurality of single-channel images, the plurality of scaled images may be stitched in step S103 to obtain an original image with a single channel, where the resolution of the original image is the same as the target resolution. The method for splicing the zoom images in the utterance includes a plurality of ways according to the number of single-channel images and the specific corresponding channel types, which is not specifically limited in the embodiment of the present application.

Taking the example that four single-channel images are acquired in step S101, and the four single-channel images include two images (first images) corresponding to the first type of channel, one image (second image) corresponding to the second type of channel, and one image (third image) corresponding to the third type of channel, step S103 may specifically include the following:

splicing pixel points at the same position in the zoomed first image, the zoomed second image and the zoomed third image into pixel blocks at corresponding positions in the original image; each pixel block comprises four pixel points, two pixel points on one diagonal line in each pixel block are pixel points of the zoomed second image and the zoomed third image at the position respectively, and two pixel points on the other diagonal line are pixel points of the zoomed first image at the position. Therefore, by adopting the splicing mode, the pixel values of different channels can be uniformly mixed together, and the interpolation is convenient.

It is understood that each pixel block in the original image obtained by stitching after the above steps are performed may be as shown in fig. 2 or as shown in fig. 3. Fig. 2 is a schematic diagram of a pixel block provided in the embodiment of the present application, and fig. 3 is a schematic diagram of another pixel block provided in the embodiment of the present application, where a square including an oblique line represents a pixel point in a scaled first image, a square including a vertical line represents a pixel point in a scaled second image, and a square including a horizontal line represents a pixel point in a scaled third image.

It should be noted that, when the first-type channel is a green channel, the second-type channel is a red channel, and the third-type channel is a blue channel, the pixel block shown in fig. 2 is obtained by stitching the scaled images according to a Bayer Pattern (Bayer Pattern); or, when the first type of channel is a red channel, the second type of channel is a green channel, and the third type of channel is a blue channel, the splicing may be performed in the manner shown in fig. 2.

In the case of four single-channel images, in order to ensure that the resolution of the original image is the target resolution, the resolution of the scaled image obtained in step S102 should be one-fourth of the target resolution. The reason is that, in the horizontal direction, the number of the pixel points of the original image is twice that of the pixel points of the zoomed image, so that the resolution of the zoomed image in the horizontal direction is half of the resolution of the original image; similarly, in the vertical direction, since the number of the pixels of the original image is twice the number of the pixels of the scaled image, the size of the resolution of the scaled image in the horizontal direction is also one half of the size of the resolution of the original image.

Finally, after obtaining the single-channel original image, the electronic device may completely complement the channel of each pixel point of the original image in step S104 to obtain the target image with the same number of channels as the input image. For example, if the input image is a three-channel image, each pixel point of the original image needs to be supplemented with pixel values corresponding to two other channels, so that each pixel point satisfies three channels.

It should be noted that, in the following embodiments, a detailed description will be given to a specific implementation process for supplementing a channel of each pixel point of an original image completely, which is not described herein for the moment.

Therefore, in the embodiment of the present application, an input image with an initial resolution is separated into multiple single-channel images, the single-channel images after being scaled are merged, then the merged original image is restored into a multi-channel image, and an obtained target image is an image that meets the target resolution. In the process of splicing and restoring to multiple channels, enough nonlinearity can be introduced, so that the definition of the generated target image can be improved; meanwhile, because a large amount of operation processes such as training and the like are not needed, the calculation amount of the whole process can be reduced.

The following describes a specific implementation process of step S102. Referring to fig. 4, fig. 4 is a flowchart illustrating a specific implementation manner of the step S102 according to an embodiment of the present application, where the step S102 may include the following steps:

step S401: the scaling rate is calculated from the target resolution, the initial resolution, and the total number of single channel images.

Step S402: and carrying out scaling processing on the single-channel image according to the scaling rate to obtain a scaled image.

In particular, the scaling rate may be calculated based on the target resolution, the initial resolution, and the total number of single-channel images. Wherein the scaling rate may comprise a value, for example: amplifying by two times; alternatively, the scaling rate may also include two values, for example: respectively a horizontal scaling rate and a vertical scaling rate.

For example, assuming that the target resolution is C × D, the initial resolution is a × E, and the total number of single-channel images is 4, the horizontal scaling rate may be calculated according to the following formula:

the vertical scaling rate can be calculated according to the following formula:

after the zoom ratio is obtained through calculation, if the zoom ratio is greater than 1, it is indicated that the single-channel image needs to be amplified; if the zoom ratio is less than 1, it means that the single-channel image needs to be reduced. The following describes in detail a specific implementation of the step S402, taking the example that the scaling factor includes a horizontal scaling factor and a vertical scaling factor.

As an embodiment, when the scaling rate is greater than 1, the resolution of the single-channel image may be increased by interpolation. In this case, the step S402 may specifically include the following steps:

aiming at least one image unit in a single-channel image, if the total number M of pixel points to be inserted supports uniform interpolation, the M pixel points to be inserted are uniformly inserted between every two original pixel points in the image unit.

And if the total number M of the pixel points to be inserted does not support uniform interpolation, uniformly inserting the N pixel points to be inserted between every two original pixel points in the image unit.

And after N pixel points to be inserted are inserted, calculating the pixel value to be inserted between every two adjacent pixel points in the image unit.

And taking the pixel values to be inserted corresponding to the minimum M-N distances in the calculated distances as the pixel points of the M-N pixel points to be inserted, and inserting the M-N pixel points to be inserted between the pixel points corresponding to the minimum M-N distances. The original pixel points refer to pixel points in a single-channel image before interpolation is carried out.

Wherein, N is a positive integer less than M, M-N +1 is less than the number of original pixel points in an image unit, and an image unit can be a row or a column in a single-channel image. The total number M of the pixels to be inserted can be determined according to the scaling rate and the number of pixels in one image unit in the single-channel image.

For example, when the number of primitive pixels in an image unit is 3, the horizontal scaling rate is

And then, aiming at one line in the single-channel image, the total number M of the pixel points to be inserted is =2. At this moment, since the size of M is equal to the size obtained by subtracting 1 from the number of pixels in the image unit, the total number M of pixels to be inserted supports uniform interpolation, and 2 pixels to be inserted can be directly and uniformly inserted between every two original pixels in the image unit, that is, every two pixels are uniformly inserted between every two original pixels in the image unitExactly one pixel is inserted between the original pixels.

For another example, when the number of original pixels in an image unit is 3, the horizontal scaling rate is

And then, aiming at one row in the single-channel image, the total number M =3 of the pixel points to be inserted. At this time, since the size of M is not equal to the size obtained by subtracting 1 from the number of pixel points in the image unit, the total number M of the pixel points to be inserted does not support uniform interpolation, 2 pixel points to be inserted may be uniformly inserted between every two original pixel points in the image unit, and then after 2 pixel points to be inserted are calculated, the pixel values to be inserted (four in total) between every two adjacent pixel points in the image unit are inserted, and the remaining one pixel point to be inserted is inserted between the four pixel points to be inserted corresponding to the pixel value to be inserted having the smallest distance between the folding lines formed by the pixel values of the original pixel points and the pixel values of the four pixel points to be inserted. For example, if the distance between the pixel value to be inserted and the broken line calculated according to the first pixel point and the second pixel point is the minimum, the remaining pixel point to be inserted is inserted between the first pixel point and the second pixel point.

According to the two examples, uniform interpolation is ensured as much as possible in the interpolation process, and non-uniform interpolation can be adopted for a small number of pixel points unless uniform interpolation is actually impossible.

It should be noted that, in the above embodiment, the pixel value of the inserted pixel may be obtained by calculating the pixel values of two adjacent pixels to the inserted pixel and the target definition. Further, the proportion of the pixel value of the pixel point closer to the central point of the image unit in the calculation process can be increased. Therefore, newly inserted pixel points are closer to the intermediate value, the transition zone can be effectively reduced by adopting the non-uniform transition mode, the condition that the definition loss of the traditional interpolation mode is overlarge is prevented, and the definition of the final picture effect is higher.

For example, when in a picture unitThe number of original pixels is 3, and the horizontal scaling rate is

When the pixel value of a single-channel image is determined, the total number of pixels to be inserted is M =2, and a first pixel to be inserted is inserted between a first original pixel and a second original pixel, and the pixel value can be determined according to the following formula:

wherein, pixel ₂ Pixel value of the first Pixel to be inserted, alpha is target definition ₁ Is the Pixel value of the first original Pixel point, pixel ₃ Is the pixel value of the second original pixel.

It is understood that all image units in a single-channel image can be enlarged by interpolation in the above embodiment; as an embodiment, the partial image unit may be enlarged by the interpolation method in the above-described embodiment, and the partial image unit may be enlarged by another interpolation method (for example, bilinear interpolation method, bicubic interpolation method, or the like). This is because, for the pixels in the first row, the first column, the last row, and the last column, there are no complete adjacent pixels in four directions, and therefore, interpolation cannot be performed in other interpolation manners.

Therefore, in the embodiment of the present application, in the case that the single-channel image needs to be enlarged, the single-channel image may be scaled by interpolation. When the pixel value corresponding to the pixel point to be inserted is calculated, the loss of definition in the interpolation process can be reduced according to the pixel values of two adjacent pixel points and the target definition; in addition, for the pixel points to be inserted which cannot have uniform difference values, the pixel points can be distributed on the broken line or around the broken line as much as possible to increase the overall variance.

As another embodiment, when the scaling rate is smaller than 1, a specific implementation process for reducing the resolution of a single-channel image by deleting a part of pixel points may be introduced.

The following describes a complete implementation process of channel supplementation for each pixel of an original image. Similarly, taking the example that four single-channel images are acquired in step S101, and the four single-channel images include two images corresponding to the first type of channel, one image corresponding to the second type of channel, and one image corresponding to the third type of channel, at this time, step S104 may specifically include the following contents:

and traversing the second pixel points and the third pixel points in the original image, and determining the interpolation direction in the processing window range with each second pixel point or each third pixel point as a central point.

And calculating the pixel value of the second pixel point or the third pixel point on the first channel according to the interpolation direction and the pixel value of the pixel point related to the second pixel point or the third pixel point in the processing window range to obtain a first intermediate image.

And traversing fourth pixel points corresponding to the second pixel point positions and fifth pixel points corresponding to the third pixel point positions in the first intermediate image, and determining the interpolation direction in a processing window range taking each fourth pixel point or each fifth pixel point as a central point.

And calculating the pixel value of the fourth pixel point on the third channel or the pixel value of the fifth pixel point on the second channel according to the interpolation direction and the pixel value of the pixel point related to the fourth pixel point or the fifth pixel point in the processing window range to obtain a second intermediate image.

And traversing sixth pixel points corresponding to the first pixel point positions in the second intermediate image, and determining the interpolation direction within the processing window range by taking each sixth pixel point as a central point.

And calculating pixel values of the sixth pixel point on the second channel and the third channel according to the interpolation direction and the pixel value of the pixel point related to the sixth pixel point in the processing window range to obtain the target image.

Specifically, for convenience of description, a pixel point of which a pixel value belongs to a first channel in the original image is named as a first pixel point, a pixel point of which a pixel value belongs to a second channel in the original image is named as a second pixel point, and a pixel point of which a pixel value belongs to a third channel in the original image is named as a third pixel point; the pixel points of which the pixel values in the first intermediate image belong to the first channel and the second channel are named as fourth pixel points, and the pixel points of which the pixel values in the original image belong to the first channel and the third channel are named as fifth pixel points; and naming the pixel point of which the pixel value in the second intermediate image belongs to the first channel as a sixth pixel point.

In the process of supplementing the channels of the pixel points of the original image, each pixel point in the original image can be traversed circularly, wherein the traversing process can be divided into three processes:

the first process, performing the first traversal, and inserting a pixel value corresponding to the first channel at a second pixel point when traversing to the second pixel point in the original image; similarly, when traversing to a third pixel point in the original image, inserting a pixel value corresponding to the first channel at the third pixel point; after all the pixel points are traversed, a first intermediate image is obtained;

in the second process, second traversal is carried out, and when a fifth pixel point in the first intermediate image is traversed, a pixel value corresponding to a third channel is inserted into the fourth pixel point; similarly, when traversing to a fifth pixel point in the first intermediate image, inserting a pixel value corresponding to the second channel at the fifth pixel point; after traversing all the pixel points, obtaining a second intermediate image;

a third process, namely performing third-pass traversal, and inserting a pixel value corresponding to the second channel and a pixel value corresponding to a third channel at a sixth pixel point when traversing to the sixth pixel point in the second intermediate image; and obtaining a target image after traversing all the pixel points.

In the three processes, when a pixel point is traversed, an interpolation direction in a processing window range with the pixel point as a central point can be determined, and then an interpolated pixel value is obtained according to the interpolation direction and the pixel value of the pixel point related to the pixel point in the processing window range.

As an embodiment, in the first process and the second process, the step of determining the interpolation direction may specifically include the following steps:

the horizontal direction gradient and the vertical direction gradient within the processing window are calculated.

And determining the direction corresponding to the smaller value of the horizontal direction gradient and the vertical direction gradient as the interpolation direction.

In the processing window range, pixels corresponding to the first-class channels exist in the horizontal direction and the vertical direction of the second pixel, so that the horizontal direction gradient and the vertical direction gradient of the second pixel can be calculated according to the pixel values of the pixels corresponding to the first-class channels.

Similarly, in the processing window range, pixels corresponding to the first type of channel exist in the horizontal direction and the vertical direction of the third pixel, so that the horizontal direction gradient and the vertical direction gradient of the third pixel can be calculated according to the pixel values of the pixels corresponding to the first type of channel.

Similarly, in the processing window range, pixels corresponding to the second-type channel and the third-type channel exist in the horizontal direction and the vertical direction of the sixth pixel, so that the horizontal direction gradient and the vertical direction gradient of the sixth pixel can be calculated according to the pixel values of the pixels corresponding to the second-type channel and the third-type channel.

As another embodiment, in the third process, the step of determining the interpolation direction may specifically include the following steps:

two diagonal direction gradients within the processing window are calculated.

And determining the direction corresponding to the smaller value of the two diagonal direction gradients as the interpolation direction.

In the processing window range, the pixel points corresponding to the third type of channel do not exist in the horizontal direction and the vertical direction of the fourth pixel point, but the pixel points corresponding to the third type of channel exist in the diagonal direction, so that the horizontal direction gradient and the vertical direction gradient of the fourth pixel point can be calculated according to the pixel values of the pixel points corresponding to the third type of channel.

Similarly, in the processing window range, the horizontal direction and the vertical direction of the fifth pixel point do not have a pixel point corresponding to the second type channel, but the diagonal direction has a pixel point corresponding to the second type channel, so that the horizontal direction gradient and the vertical direction gradient of the fifth pixel point can be calculated according to the pixel value of the pixel point corresponding to the second type channel.

It should be noted that after the pixel values of the pixel point on the other channels are calculated according to the interpolation direction and the pixel value of the pixel point related to the pixel point within the processing window range, it can be determined whether the calculated pixel value is within a certain pixel value range, and if the calculated pixel value is not within the pixel value range, the boundary of the range can be taken as the pixel value of the pixel point on the other channels, so as to avoid that the difference between the calculated pixel value and the pixel value of the whole image is large, which results in low image definition.

In addition, the embodiment of the present application does not specifically limit the processing window range and the size of the pixel value range, for example: the processing window range may be a 5 × 5 sized range, a 6 × 6 sized range, etc.; the pixel value range can be 0-255, the minimum value of the pixel points adjacent to the pixel point to the maximum value of the pixel points adjacent to the pixel point, and the like. Those skilled in the art can make appropriate adjustments according to actual conditions.

It can be understood that, for a pixel point located at an edge of an image in an original image, when the pixel point is taken as a center, the pixel point within a processing window range is incomplete, and at this time, a zero padding mode may be used to pad the pixel point (i.e., a pixel value corresponding to a non-existing pixel point is set to zero) or a symmetric mapping mode is used to pad the pixel point (i.e., a pixel value of an existing pixel point is mapped to a pixel value of a corresponding non-existing pixel point according to a symmetric relationship between the non-existing pixel point and the existing pixel point).

The following describes a specific implementation process of the step S104 by way of example.

Assuming that the first channel is a green channel, the second channel is a red channel, and the third channel is a blue channel; the processing window range may be a 5 x 5 sized range.

Referring to fig. 5 and 6, fig. 5 is a schematic diagram of a pixel point within a processing window range according to an embodiment of the present disclosure, and fig. 6 is a schematic diagram of a pixel point within another processing window range according to an embodiment of the present disclosure. Wherein, the square including the oblique line represents the pixel point in the first image after zooming, the square including the vertical line represents the pixel point in the second image after zooming, the square including the horizontal line represents the pixel point in the third image after zooming; the pixels are numbered 00 from the first pixel in the first row to 44 from the last pixel in the last row.

In the first traversal process, when traversing to the pixel 22 in fig. 5, the pixel 22 is the first pixel, and therefore the pixel value corresponding to the green channel should be inserted. The process of calculating the pixel value corresponding to the green channel is as follows:

first, the horizontal gradient of the pixel 22 is calculated according to the following formula:

Grad _H ＝abs(R ₂₂ -R ₂₀ )+abs(R ₂₂ -R ₂₄ )+abs(R ₂₀ -G ₂₁ )+abs(G ₂₁ -R ₂₂ )+abs(R ₂₂ -G ₂₃ )+abs(G ₂₃ -R ₂₄ )；

wherein, grad _H Is the horizontal gradient, R, of the pixel 22 ₂₂ Is the pixel value, R, of pixel 22 corresponding to the red channel ₂₀ Is the pixel value, R, of the pixel 20 corresponding to the red channel ₂₄ Is the pixel value, G, of pixel point 24 corresponding to the red channel ₂₁ Pixel 21 corresponds to the green channelPixel value of G ₂₃ The pixel value of the pixel 23 corresponding to the green channel.

Then, the vertical gradient of the pixel point 22 is calculated according to the following formula:

Grad _V ＝abs(R ₂₂ -R ₀₂ )+abs(R ₂₂ -R ₄₂ )+abs(R ₀₂ -G ₁₂ )+abs(G ₁₂ -R ₂₂ )+abs(R ₂₂ -G ₃₂ )+abs(G ₃₂ -R ₄₂ )；

wherein, grad _V Is the horizontal gradient, R, of pixel 22 ₀₂ Is the pixel value, R, of pixel point 02 corresponding to the red channel ₄₂ Is the pixel value, G, of pixel 42 corresponding to the red channel ₁₂ Is the pixel value, G, of pixel 12 corresponding to the green channel ₃₂ The pixel value of the pixel point 32 corresponding to the green channel.

Next, the horizontal direction gradient and the vertical direction gradient are compared.

If the horizontal gradient is smaller than the vertical gradient, the pixel value of the pixel point 22 corresponding to the green channel is calculated according to the following formula:

wherein Interp _G The pixel value of the pixel 22 corresponding to the green channel.

At this time, if the calculated pixel value is [ min (G) ] ₂₁ ，G ₂₃ )，max(G ₂₁ ，G ₂₃ )]Within the range, the interpolated pixel value is determined to be; if the calculated pixel value is less than min (G) ₂₁ ，G ₂₃ ) Then the interpolated pixel value is determined to be min (G) ₂₁ ，G ₂₃ ) (ii) a If the calculated pixel value is larger than max (G) ₂₁ ，G ₂₃ ) Then the interpolated pixel value is determined to be max (G) ₂₁ ，G ₂₃ )。

If the horizontal gradient is greater than the vertical gradient, the pixel value of the pixel 22 corresponding to the green channel is calculated according to the following formula:

at this time, if the calculated pixel value is [ min (G) ] ₁₂ ，G ₃₂ )，max(G ₁₂ ，G ₃₂ )]Within the range, the interpolated pixel value is determined to be; if the calculated pixel value is less than min (G) ₁₂ ，G ₃₂ ) Then the interpolated pixel value is determined to be min (G) ₁₂ ，G ₃₂ ) (ii) a If the calculated pixel value is greater than max (G) ₁₂ ，G ₃₂ ) Then the interpolated pixel value is determined to be max (G) ₁₂ ，G ₃₂ )。

In the second traversal, when the pixel 22 in fig. 5 is traversed, the pixel 22 is the fourth pixel, and therefore the pixel value corresponding to the blue channel should be inserted. The process of calculating the pixel value corresponding to the blue channel is as follows:

first, the gradient of the pixel 22 in the 45 ° direction is calculated according to the following formula:

Grad ₄₅ ＝abs(R ₂₂ -R ₀₀ )+abs(R ₂₂ -R ₄₄ )+abs(R ₀₀ -G ₁₁ )+abs(G ₁₁ -R ₂₂ )+abs(R ₂₂ -G ₃₃ )+abs(G ₃₃ -R ₄₄ )；

wherein, grad ₄₅ Is a gradient of 45 DEG, R, of pixel 22 ₂₂ Is the pixel value, R, of pixel 22 corresponding to the red channel ₀₀ Is the pixel value, R, of pixel 00 corresponding to the red channel ₄₄ Is the pixel value, G, of pixel 44 corresponding to the red channel ₁₁ The pixel value, G, of the pixel point 11 corresponding to the green channel ₃₃ The pixel value of the pixel 33 corresponding to the green channel.

Then, the 135 ° directional gradient of the pixel 22 is calculated according to the following formula:

Grad ₁₃₅ ＝abs(R ₂₂ -R ₀₄ )+abs(R ₂₂ -R ₄₀ )+abs(R ₀₄ -G ₁₃ )+abs(G ₁₃ -R ₂₂ )+abs(R ₂₂ -G ₃₁ )+abs(G ₃₁ -R ₄₀ )；

wherein, grad ₁₃₅ Is a 135 degree gradient, R, of pixel 22 ₀₄ Is the pixel value, R, of pixel point 04 corresponding to the red channel ₄₀ Is the pixel value, G, of the pixel point 40 corresponding to the red channel ₁₄ The pixel value G of the pixel point 14 corresponding to the green channel ₃₁ The pixel value of the pixel 31 corresponding to the green channel.

Next, the 45 ° directional gradient and the 135 ° directional gradient are compared.

If the 45 ° directional gradient is smaller than the 135 ° directional gradient, the pixel value of the pixel point 22 corresponding to the blue channel is calculated according to the following formula:

wherein Interp _B The pixel value of the pixel 22 corresponding to the blue channel.

At this time, if the calculated pixel value is [ min (B) ] ₁₁ ，B ₃₃ )，max(S ₁₁ ，B ₃₃ )]Within the range, the interpolated pixel value is determined to be; if the calculated pixel value is less than min (B) ₁₁ ，B ₃₃ ) Then the interpolated pixel value is determined to be min (B) ₁₁ ，B ₃₃ ) (ii) a If the calculated pixel value is larger than max (B) ₁₁ ，B ₃₃ ) Then the interpolated pixel value is determined to be max (B) ₁₁ ，B ₃₃ )。

If the 45 ° directional gradient is greater than the 135 ° directional gradient, the pixel value of the pixel point 22 corresponding to the blue channel is calculated according to the following formula:

at this time, if the calculated pixel value is [ min (B) ] ₁₃ ，B ₃₁ )，max(B ₁₃ ，B ₃₁ )]Within the range, the interpolated pixel value is determined to be; if the calculated pixel value is less than min (B) ₁₃ ，B ₃₁ ) Then determineThe interpolated pixel value is min (B) ₁₃ ，B ₃₁ ) (ii) a If the calculated pixel value is greater than max (B) ₁₃ ，B ₃₁ ) Then the interpolated pixel value is determined to be max (B) ₁₃ ，B ₃₁ )。

In the third traversal process, when the pixel point 22 in fig. 6 is traversed, the pixel point 22 is the sixth pixel point, and therefore the pixel value corresponding to the red channel and the pixel value corresponding to the blue channel should be inserted. The process of calculating the pixel value corresponding to the red channel is as follows:

Grad _H ＝abs(G ₂₂ -G ₂₀ )+abs(G ₂₂ -G ₂₄ )+abs(G ₂₀ -G ₂₁ )+abs(G ₂₁ -G ₂₂ )+abs(G ₂₂ -G ₂₃ )+abs(G ₂₃ -G ₂₄ )；

wherein, grad _H Is the horizontal gradient, G, of pixel 22 ₂₂ The pixel value, G, of pixel 22 corresponding to the green channel ₂₀ The pixel value, G, of the pixel point 20 corresponding to the green channel ₂₄ Is the pixel value, G, of pixel point 24 corresponding to the green channel ₂₁ Is the pixel value, G, of pixel point 21 corresponding to the green channel ₂₃ The pixel value of the pixel 23 corresponding to the green channel.

Grad _V ＝abs(G ₂₂ -G ₀₂ )+abs(G ₂₂ -G ₄₂ )+abs(G ₀₂ -G ₁₂ )+abs(G ₁₂ -G ₂₂ )+abs(G ₂₂ -G ₃₂ )+abs(G ₃₂ -G ₄₂ )；

wherein, grad _V Is the vertical gradient, G, of the pixel 22 ₀₂ Is the pixel value, G, of pixel point 02 corresponding to the green channel ₄₂ Is the pixel value, G, of pixel 42 corresponding to the green channel ₁₂ Is the pixel value, G, of pixel 12 corresponding to the green channel ₃₂ The pixel value of the pixel point 32 corresponding to the green channel.

Next, the horizontal gradient and the vertical gradient are compared.

If the horizontal gradient is smaller than the vertical gradient, the pixel value of the pixel point 22 corresponding to the blue channel is calculated according to the following formula:

wherein Interp _R The pixel value of the pixel 22 corresponding to the blue channel.

At this time, if the calculated pixel value is [ min (R) ] ₂₁ ，R ₂₃ )，max(R ₂₁ ，R ₂₃ )]Within the range, the interpolated pixel value is determined to be; if the calculated pixel value is less than min (R) ₂₁ ，R ₂₃ ) Then the interpolated pixel value is determined to be min (R) ₂₁ ，R ₂₃ ) (ii) a If the calculated pixel value is greater than max (R) ₂₁ ，R ₂₃ ) Then the interpolated pixel value is determined to be max (R) ₂₁ ，R ₂₃ )。

If the horizontal direction gradient is greater than the vertical direction gradient, the pixel value of the pixel point 22 corresponding to the blue channel is calculated according to the following formula:

at this time, if the calculated pixel value is [ min (R) ] ₁₂ ，R ₃₂ )，max(R ₁₂ ，R ₃₂ )]Within the range, the interpolated pixel value is determined to be; if the calculated pixel value is less than min (R) ₁₂ ，R ₃₂ ) Then the interpolated pixel value is determined to be min (R) ₁₂ ，R ₃₂ ) (ii) a If the calculated pixel value is greater than max (R) ₁₂ ，R ₃₂ ) Then the interpolated pixel value is determined to be max (R) ₁₂ ，R ₃₂ )。

And finally obtaining a three-channel target image through the three times of traversal.

It should be noted that, the above example only introduces the interpolation method of a part of pixel points, and it can be understood that, on the basis of the above example, a person skilled in the art can adopt a similar implementation method to calculate the interpolation method of other pixel points, and details are not described here.

Therefore, in the embodiment of the application, the pixel points including the pixel values belonging to the single channel in the original image are sequentially traversed, and the pixel values of other channels are inserted, so that a multi-channel image can be restored according to the original image, and the definition of the generated target image can be improved because enough nonlinearity can be introduced in the insertion process; meanwhile, because a large amount of operation processes such as training and the like are not needed, the calculation amount of the whole process can be reduced.

In addition, the image processing method provided by the embodiment of the present application can be applied to a super-resolution scheme, that is, the image resolution can be improved by using the image processing method provided by the embodiment of the present application.

Referring to fig. 7, fig. 7 is a block diagram of an image processing apparatus according to an embodiment of the present disclosure, where the image processing apparatus 700 includes: an obtaining module 701, configured to obtain multiple single-channel images included in an input image; a scaling module 702, configured to perform scaling processing on the multiple single-channel images according to a target resolution and an initial resolution corresponding to the input image, respectively, to obtain multiple corresponding scaled images; wherein the target resolution is greater than the initial resolution; a splicing module 703, configured to splice the multiple zoomed images to obtain a single-channel original image; wherein the resolution of the original image is the target resolution; a generating module 704, configured to calculate pixel values of the pixel point on other channels according to the channel to which the pixel value of each pixel point in the original image belongs and the pixel value of the pixel point related to the pixel point, so as to obtain a target image with the same number of channels as the input image.

In the embodiment of the application, an input image with an initial resolution ratio is separated into a plurality of single-channel images, the single-channel images subjected to scaling processing are spliced, then an original image obtained by splicing is restored into a multi-channel image, and an obtained target image is an image meeting the target resolution ratio. In the process of splicing and restoring to multiple channels, enough nonlinearity can be introduced, so that the definition of the generated target image can be improved; meanwhile, because a large amount of operation processes such as training and the like are not needed, the calculation amount of the whole process can be reduced.

Further, the channels include a first type channel, a second type channel, and a third type channel, and the plurality of single-channel images include two first images corresponding to the first type channel, one second image corresponding to the second type channel, and one third image corresponding to the third type channel.

Further, the splicing module 703 is specifically configured to: splicing pixel points at the same position in the zoomed first image, the zoomed second image and the zoomed third image into pixel blocks at corresponding positions in the original image; each pixel block comprises four pixel points, two pixel points on one diagonal line in each pixel block are pixel points of the zoomed second image and the zoomed third image at the position respectively, and two pixel points on the other diagonal line are pixel points of the zoomed first image at the position.

In the embodiment of the application, the scaled single-channel images are spliced into the original image through the splicing process, so that sufficient nonlinearity can be introduced, and the definition of the generated target image can be improved.

Further, the pixel point of the original image, the pixel value of which belongs to the first channel, is a first pixel point, the pixel point of the original image, the pixel value of which belongs to the second channel, is a second pixel point, and the pixel point of the original image, the pixel value of which belongs to the third channel, is a third pixel point; the generating module 704 is specifically configured to: traversing second pixel points and third pixel points in the original image, and determining an interpolation direction in a processing window range with each second pixel point or each third pixel point as a central point; calculating the pixel value of the second pixel point or the third pixel point on a first channel according to the interpolation direction and the pixel value of the pixel point related to the second pixel point or the third pixel point in the processing window range to obtain a first intermediate image; traversing a fourth pixel point corresponding to the second pixel point position and a fifth pixel point corresponding to the third pixel point position in the first intermediate image, and determining an interpolation direction within the processing window range by taking each fourth pixel point or each fifth pixel point as a central point; calculating the pixel value of the fourth pixel point on a third channel or the pixel value of the fifth pixel point on a second channel according to the interpolation direction and the pixel value of the pixel point related to the fourth pixel point or the fifth pixel point in the processing window range to obtain a second intermediate image; traversing sixth pixel points corresponding to the first pixel point positions in the second intermediate image, and determining an interpolation direction within the processing window range by taking each sixth pixel point as a central point; and calculating pixel values of the sixth pixel point on a second channel and a third channel according to the interpolation direction and the pixel value of the pixel point related to the sixth pixel point in the processing window range to obtain the target image.

In the embodiment of the application, the pixel points including the pixel values belonging to the single channel in the original image are traversed in sequence, and the pixel values of other channels are inserted, so that the multi-channel image can be restored according to the original image, and the definition of the generated target image can be improved because enough nonlinearity can be introduced in the insertion process; meanwhile, because a large amount of operation processes such as training and the like are not needed, the calculation amount of the whole process can be reduced.

Further, the generating module 704 is further configured to: calculating the horizontal direction gradient and the vertical direction gradient in the processing window range; and determining the direction corresponding to the smaller value of the horizontal direction gradient and the vertical direction gradient as the interpolation direction.

Further, the generating module 704 is further configured to: calculating two diagonal direction gradients in the processing window range; and determining the direction corresponding to the smaller value of the two diagonal direction gradients as the interpolation direction.

Further, the scaling module 702 is specifically configured to: calculating a zoom rate according to the target resolution, the initial resolution and the total number of the single-channel images; and carrying out scaling processing on the single-channel image according to the scaling rate to obtain the scaled image.

In the embodiment of the present application, during the process of scaling the single-channel image, a corresponding scaling rate may be calculated according to the target resolution and the initial resolution, so as to appropriately scale the single-channel image according to the scaling rate.

Further, when the scaling rate is greater than 1, the scaling module 702 is further configured to: for at least one image unit in the single-channel image, if the total number M of pixel points to be inserted supports uniform interpolation, uniformly inserting the M pixel points to be inserted between every two original pixel points in the image unit; if the total number M of the pixel points to be inserted does not support uniform interpolation, uniformly inserting N pixel points to be inserted between every two original pixel points in the image unit; wherein N is a positive integer smaller than M, and M-N +1 is smaller than the number of original pixel points in the image unit; after the N to-be-inserted pixel points are inserted, calculating a to-be-inserted pixel value between every two adjacent pixel points in the image unit; and taking the pixel value to be inserted corresponding to the minimum M-N distances in the calculated distances as the pixel points of the M-N pixel points to be inserted, and inserting the M-N pixel points to be inserted between the pixel points corresponding to the minimum M-N distances.

In the embodiment of the application, in the case that the single-channel image needs to be enlarged, the single-channel image may be scaled in an interpolation manner. When the pixel value corresponding to the pixel point to be inserted is calculated, the loss of definition in the interpolation process can be reduced according to the pixel values of two adjacent pixel points and the target definition; in addition, for the pixel points to be inserted which cannot have uniform difference values, the pixel points can be distributed on the broken line or around the broken line as much as possible to increase the overall variance.

Further, when the scaling rate is greater than 1, the scaling module 702 is further configured to: inserting pixel points among the pixel points on the single-channel image; the pixel value of the inserted pixel point is obtained through calculation according to the pixel values of two pixel points adjacent to the inserted pixel point and the target definition, and the pixel value closer to the central point of the image unit in the single-channel image occupies a larger proportion in the calculation process.

Further, the image unit includes at least one of the following objects: a first row, a first column, a last row, and a last column in the single-channel image.

Referring to fig. 8, fig. 8 is a block diagram of an electronic device according to an embodiment of the present disclosure, where the electronic device 800 includes: at least one processor 801, at least one communication interface 802, at least one memory 803, and at least one communication bus 804. Wherein the communication bus 804 is used for implementing direct connection communication of these components, the communication interface 802 is used for communicating signaling or data with other node devices, and the memory 803 stores machine readable instructions executable by the processor 801. When the electronic device 800 is in operation, the processor 801 communicates with the memory 803 via the communication bus 804, and the machine-readable instructions, when called by the processor 801, perform the image processing methods described above.

For example, the processor 801 of the embodiment of the present application may read the computer program from the memory 803 through the communication bus 804 and execute the computer program to implement the following method: step S101: a plurality of single-channel images contained in an input image are acquired. Step S102: and respectively carrying out scaling processing on the plurality of single-channel images according to the target resolution and the initial resolution corresponding to the input image to obtain a plurality of corresponding scaled images. Step S103: and splicing the plurality of zoomed images to obtain a single-channel original image. Wherein the resolution of the original image is the target resolution. Step S104: and calculating the pixel values of the pixel points on other channels according to the channel to which the pixel value of each pixel point in the original image belongs and the pixel value of the pixel point related to the pixel point, so as to obtain the target image with the same number of channels as the input image.

The processor 801 may be an integrated circuit chip having signal processing capabilities. The Processor 801 may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), application Specific Integrated Circuits (ASICs), field-Programmable Gate arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components. Which may implement or perform the various methods, steps, and logic blocks disclosed in the embodiments of the present application. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The Memory 803 may include, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read Only Memory (PROM), an Erasable Read Only Memory (EPROM), an electrically Erasable Read Only Memory (EEPROM), and the like.

It will be appreciated that the configuration shown in fig. 8 is merely illustrative and that electronic device 800 may include more or fewer components than shown in fig. 8 or have a different configuration than shown in fig. 8. The components shown in fig. 8 may be implemented in hardware, software, or a combination thereof. In this embodiment, the electronic device 800 may be, but is not limited to, an entity device such as a desktop, a laptop, a smart phone, an intelligent wearable device, and a vehicle-mounted device, and may also be a virtual device such as a virtual machine. In addition, the electronic device 800 is not necessarily a single device, but may also be a combination of multiple devices, such as a server cluster, and the like.

Embodiments of the present application further provide a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions, which when executed by a computer, the computer is capable of performing the steps of the image processing method in the above embodiments, for example, including: acquiring a plurality of single-channel images contained in an input image; respectively carrying out scaling processing on the multiple single-channel images according to a target resolution and an initial resolution corresponding to the input image to obtain multiple corresponding scaled images; wherein the target resolution is greater than the initial resolution; splicing the multiple zoomed images to obtain a single-channel original image; wherein the resolution of the original image is the target resolution; and calculating the pixel values of the pixel points on other channels according to the channel to which the pixel value of each pixel point in the original image belongs and the pixel value of the pixel point related to the pixel point, so as to obtain the target image with the same number of channels as the input image.

The embodiments of the present application also disclose a computer program product, which includes a computer program stored on a non-transitory computer readable storage medium, the computer program including program instructions, when the program instructions are executed by a computer, the computer can execute the method provided by the above method embodiments, for example, the method includes: acquiring a plurality of single-channel images contained in an input image; respectively carrying out scaling processing on the multiple single-channel images according to a target resolution and an initial resolution corresponding to the input image to obtain multiple corresponding scaled images; wherein the target resolution is greater than the initial resolution; splicing the multiple zoomed images to obtain a single-channel original image; wherein the resolution of the original image is the target resolution; and calculating the pixel values of the pixel points on other channels according to the channel to which the pixel value of each pixel point in the original image belongs and the pixel value of the pixel point related to the pixel point, so as to obtain the target image with the same number of channels as the input image.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described apparatus embodiments are merely illustrative, and for example, the division of the units into only one type of logical function may be implemented in other ways, and for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.

In addition, units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

Furthermore, the functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

It should be noted that the functions, if implemented in the form of software functional modules and sold or used as independent products, may be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, an optical disk, or other various media capable of storing program codes.

In this document, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions.

The above description is only an example of the present application and is not intended to limit the scope of the present application, and various modifications and changes may be made to the present application by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An image processing method, comprising:

acquiring a plurality of single-channel images contained in an input image;

respectively carrying out scaling processing on the multiple single-channel images according to a target resolution and an initial resolution corresponding to the input image to obtain multiple corresponding scaled images; wherein the target resolution is greater than the initial resolution;

splicing the multiple zoomed images to obtain a single-channel original image; wherein the resolution of the original image is the target resolution;

and calculating the pixel values of the pixel points on other channels according to the channel to which the pixel value of each pixel point in the original image belongs and the pixel value of the pixel point related to the pixel point, so as to obtain the target image with the same number of channels as the input image.

2. The image processing method according to claim 1, wherein the channels include a first-type channel, a second-type channel, and a third-type channel, and the plurality of single-channel images include two first images corresponding to the first-type channel, one second image corresponding to the second-type channel, and one third image corresponding to the third-type channel.

3. The image processing method according to claim 2, wherein said stitching the multiple scaled images to obtain a single-channel original image comprises:

splicing pixel points at the same position in the zoomed first image, the zoomed second image and the zoomed third image into pixel blocks at corresponding positions in the original image; and each pixel block comprises four pixel points, two pixel points on one diagonal line in each pixel block are pixel points of the zoomed second image and the zoomed third image at the position respectively, and two pixel points on the other diagonal line are pixel points of the zoomed first image at the position.

4. The image processing method according to claim 3, wherein the pixel point in the original image whose pixel value belongs to the first channel is a first pixel point, the pixel point in the original image whose pixel value belongs to the second channel is a second pixel point, and the pixel point in the original image whose pixel value belongs to the third channel is a third pixel point;

the method for calculating the pixel values of the pixel points on other channels according to the channel to which the pixel value of each pixel point in the original image belongs and the pixel value of the pixel point related to the pixel point to obtain the target image with the same number of channels as the input image comprises the following steps:

traversing second pixel points and third pixel points in the original image, and determining an interpolation direction in a processing window range with each second pixel point or each third pixel point as a central point;

calculating the pixel value of the second pixel point or the third pixel point on a first channel according to the interpolation direction and the pixel value of the pixel point related to the second pixel point or the third pixel point in the processing window range to obtain a first intermediate image;

traversing fourth pixel points corresponding to the second pixel point position and fifth pixel points corresponding to the third pixel point position in the first intermediate image, and determining an interpolation direction within the processing window range by taking each fourth pixel point or each fifth pixel point as a central point;

calculating the pixel value of the fourth pixel point on a third channel or the pixel value of the fifth pixel point on a second channel according to the interpolation direction and the pixel value of the pixel point related to the fourth pixel point or the fifth pixel point in the processing window range to obtain a second intermediate image;

traversing sixth pixel points corresponding to the first pixel point positions in the second intermediate image, and determining an interpolation direction within the processing window range by taking each sixth pixel point as a central point;

and calculating pixel values of the sixth pixel point on a second channel and a third channel according to the interpolation direction and the pixel value of the pixel point related to the sixth pixel point in the processing window range to obtain the target image.

5. The method of claim 4, wherein the traversing the second pixel points and the third pixel points in the original image and determining the interpolation direction within the processing window centered at each of the second pixel points or the third pixel points comprises:

calculating the horizontal direction gradient and the vertical direction gradient in the processing window range;

6. The method according to claim 4, wherein the traversing fourth pixel points corresponding to the second pixel point position and fifth pixel points corresponding to the third pixel point position in the first intermediate image and determining an interpolation direction within the processing window range with each of the fourth pixel points or the fifth pixel points as a center point comprises:

calculating two diagonal direction gradients in the processing window range;

7. The image processing method according to any one of claims 1 to 6, wherein the scaling the plurality of single-channel images according to the target resolution and the initial resolution corresponding to the input image to obtain a plurality of corresponding scaled images comprises:

calculating a zoom rate according to the target resolution, the initial resolution and the total number of the single-channel images;

and carrying out scaling processing on the single-channel image according to the scaling rate to obtain the scaled image.

8. The image processing method according to claim 7, wherein when the scaling rate is greater than 1, the scaling the single-channel image according to the scaling rate to obtain the scaled image includes:

for at least one image unit in the single-channel image, if the total number M of pixel points to be inserted supports uniform interpolation, uniformly inserting the M pixel points to be inserted between every two original pixel points in the image unit;

if the total number M of the pixel points to be inserted does not support uniform interpolation, uniformly inserting N pixel points to be inserted between every two original pixel points in the image unit; wherein N is a positive integer smaller than M, and M-N +1 is smaller than the number of original pixel points in the image unit;

after the N to-be-inserted pixel points are inserted, calculating a to-be-inserted pixel value between every two adjacent pixel points in the image unit;

and taking the pixel values to be inserted corresponding to the minimum M-N distances in the calculated distances as the pixels of the M-N pixels to be inserted, and inserting the M-N pixels to be inserted between the pixels corresponding to the minimum M-N distances.

9. The image processing method according to claim 7, wherein when the scaling rate is greater than 1, the scaling the single-channel image according to the scaling rate to obtain the scaled image includes:

inserting pixel points among the pixel points on the single-channel image; the pixel value of the inserted pixel point is obtained through calculation according to the pixel values of two pixel points adjacent to the inserted pixel point and the target definition, and the pixel value of the pixel point which is closer to the central point of the image unit in the single-channel image occupies a larger proportion in the calculation process.

10. The image processing method according to claim 8 or 9, wherein the image unit includes at least one of the following objects: a first row, a first column, a last row, and a last column in the single-channel image.

11. An electronic device, comprising: a processor, a memory, and a bus;

the processor and the memory are communicated with each other through the bus;

the memory stores program instructions executable by the processor, the program instructions being invoked by the processor to perform the image processing method of any of claims 1 to 10.

12. A non-transitory computer-readable storage medium storing computer instructions which, when executed by a computer, cause the computer to perform the image processing method according to any one of claims 1 to 10.

13. A computer program product comprising a computer program, characterized in that the computer program, when being executed by a processor, carries out the image processing method of any one of claims 1-10.