CN114549313A

CN114549313A - Image processing method, image processing device, electronic equipment and storage medium

Info

Publication number: CN114549313A
Application number: CN202210154606.5A
Authority: CN
Inventors: 梁树宇
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2022-02-18
Filing date: 2022-02-18
Publication date: 2022-05-27

Abstract

The disclosure provides an image processing method, and relates to the field of automatic driving, in particular to the field of deep learning. The specific implementation scheme is as follows: determining an ith image and an i +1 th image having a matching region from among the N images, where N is an integer greater than or equal to 2, and i is 1, … … N-1; determining a matching region between the ith image and the (i + 1) th image to obtain a first matching region in the ith image and a second matching region in the (i + 1) th image; determining a plurality of first matching sub-regions and a plurality of second matching sub-regions for the first matching region and the second matching region, respectively; determining a target matching image pair according to the difference between the images of the plurality of first matching sub-regions and the images of the plurality of second matching sub-regions; and processing at least one of the ith image and the (i + 1) th image according to the target matching image pair. The present disclosure also provides an image processing apparatus, an electronic device, and a storage medium.

Description

Image processing method, image processing device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of automated driving techniques, and more particularly, to deep learning techniques. More particularly, the present disclosure provides an image processing method, apparatus, electronic device, and storage medium.

Background

A plurality of capturing devices may be mounted on the vehicle to capture a plurality of images around the vehicle. A stitched image (e.g., an overhead view) stitched from multiple images may be presented to the driver to assist in driving.

Disclosure of Invention

The present disclosure provides an image processing method, apparatus, device, and storage medium.

According to an aspect of the present disclosure, there is provided an image processing method including: determining an ith image and an i +1 th image having a matching region from among the N images, where N is an integer greater than or equal to 2, and i is 1, … … N-1; determining a matching region between the ith image and the (i + 1) th image to obtain a first matching region in the ith image and a second matching region in the (i + 1) th image; determining a plurality of first matching sub-regions and a plurality of second matching sub-regions for the first matching region and the second matching region, respectively; determining a target matching image pair according to the difference between the images of the first matching sub-regions and the images of the second matching sub-regions; and processing at least one of the ith image and the (i + 1) th image according to the target matching image pair.

According to another aspect of the present disclosure, there is provided an image processing apparatus including: a first determining module, configured to determine an i-th image and an i + 1-th image having a matching region in N images, where N is an integer greater than or equal to 2, and i is 1, … … N-1; a second determining module, configured to determine a matching region between the ith image and the (i + 1) th image, to obtain a first matching region in the ith image and a second matching region in the (i + 1) th image; a third determining module, configured to determine a plurality of first matching sub-regions and a plurality of second matching sub-regions of the first matching region and the second matching region, respectively; a fourth determining module, configured to determine a target matching image pair according to a difference between the images of the plurality of first matching sub-regions and the images of the plurality of second matching sub-regions; and the processing module is used for processing at least one of the ith image and the (i + 1) th image according to the target matching image pair.

According to another aspect of the present disclosure, there is provided an electronic device including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method provided in accordance with the present disclosure.

According to another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to perform a method provided according to the present disclosure.

According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements a method provided according to the present disclosure.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

FIG. 1 is a schematic diagram of an exemplary application scenario of an image processing method and apparatus according to one embodiment of the present disclosure;

FIG. 2 is a flow diagram of an image processing method according to one embodiment of the present disclosure;

FIG. 3A is a schematic diagram of N images according to another embodiment of the present disclosure;

FIG. 3B is a schematic diagram of a first matching region of an ith image, according to one embodiment of the present disclosure;

FIG. 3C is a schematic diagram of a second matching region of an i +1 th image according to one embodiment of the present disclosure;

FIG. 4A is a schematic diagram of a first matching region of an ith image, according to another embodiment of the present disclosure;

FIG. 4B is a schematic diagram of a first matching subregion, according to another embodiment of the present disclosure;

FIG. 4C is a schematic diagram of a second matching region of an i +1 th image according to another embodiment of the present disclosure;

FIG. 4D is a schematic diagram of a second matching subregion, according to another embodiment of the present disclosure;

FIG. 5 is a flow diagram of an image processing method according to another embodiment of the present disclosure;

FIG. 6 is a flow diagram of an image processing method according to another embodiment of the present disclosure;

FIG. 7 is a block diagram of an image processing apparatus according to one embodiment of the present disclosure; and

fig. 8 is a block diagram of an electronic device to which an image processing method may be applied according to one embodiment of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

An acquisition device on a vehicle, such as an onboard camera, may acquire images from a variety of positions and angles. These images can be stitched into a stitched image to assist driving. Under the influence of environments at different positions, the contrast and brightness difference of images acquired by the cameras may be large, which results in uneven pictures of finally displayed spliced images.

Fig. 1 is a schematic view of an application scenario of an image processing method and apparatus according to an embodiment of the present disclosure.

As shown in fig. 1, 4 acquisition devices are loaded at corresponding positions on the vehicle 100. Each acquisition device may be used to acquire images. The 4 acquisition devices are respectively: the device comprises a collecting device 101 mounted on the head of the vehicle, a collecting device 102 mounted on the left side of the vehicle, a collecting device 103 mounted on the right side of the vehicle and a collecting device 104 mounted on the tail of the vehicle. The 4 acquisition devices can acquire images.

Fig. 2 is a flow diagram of an image processing method according to one embodiment of the present disclosure.

As shown in fig. 2, the method 200 may include operations S210 to S250.

In operation S210, an ith image and an (i + 1) th image having a matching region among the N images are determined.

For example, N is an integer of 2 or more, i is 1, … … N-1.

For example, the N images are simultaneously acquired by N acquisition devices loaded at corresponding positions on the vehicle, respectively. Taking N-4 as an example, the corresponding position on the vehicle includes: vehicle head, vehicle left side, vehicle tail and vehicle right side.

For example, taking i-1 as an example, the 1 st image may be captured by the above-mentioned capturing device 101 mounted on the head of the vehicle, and the 2 nd image may be captured by the above-mentioned capturing device 102 mounted on the left side of the vehicle.

In operation S220, a matching region between the ith image and the (i + 1) th image is determined, resulting in a first matching region in the ith image and a second matching region in the (i + 1) th image.

For example, the first matching region for image 1 may be the region located at the lower left corner of image 1 described above. The second matching region for the 2 nd image may be the region located in the upper right corner of the 2 nd image described above.

In operation S230, a plurality of first matching sub-areas and a plurality of second matching sub-areas of the first matching area and the second matching area, respectively, are determined.

For example, the plurality of matching sub-regions may be determined in various ways. In one example, the matching region may be divided into 4 matching sub-regions, each matching sub-region having an equal area.

In operation S240, a target matching image pair is determined according to a difference between the images of the plurality of first matching sub-regions and the images of the plurality of second matching sub-regions.

For example, the mean values of the pixels in the first and second matching sub-regions may be determined separately to determine the target matching image pair. In one example, a target matched image pair may be determined from the two matched sub-regions having the smallest difference between the mean values of the pixels.

In operation S250, at least one of the ith image and the (i + 1) th image is processed according to the target matching image pair.

For example, the Wallis filtering process may be performed on the ith image and the (i + 1) th image from the target matching image pair. For another example, the Wallis filtering process may be performed on one of the ith image and the (i + 1) th image from the target matching image pair.

It is understood that Wallis filtering is a filtering method for image enhancement, which can suppress noise while increasing the contrast of the original image, and can map the gray level mean and variance of the local image to the given gray level mean and variance values. The Wallis filtering process is a local image transformation, which can enhance the tiny information of the gray scale in the image. This characteristic makes the mean and variance of the gray levels of the respective regions of the image approximately equal, thereby realizing the inter-image uniformity processing (e.g., color-uniformizing processing).

By the embodiment of the disclosure, the color or brightness of the processed image is more uniform, smooth and natural, and the driving of the driver can be more efficiently assisted.

In some embodiments, determining a matching region between the ith image and the (i + 1) th image, and obtaining a first matching region in the ith image and a second matching region in the (i + 1) th image comprises: determining first world coordinate values of a plurality of pixels in an ith image; determining second world coordinate values of a plurality of pixels in the (i + 1) th image; determining a pixel region in the ith image, wherein the first world coordinate value is consistent with the second world coordinate value, as a first matching region; and determining a pixel region in the i +1 th image, in which the second world coordinate value is consistent with the first world coordinate value, as a second matching region. The following will be described in detail with reference to fig. 3A to 3B.

Fig. 3A is a schematic diagram of N images according to another embodiment of the present disclosure.

As shown in fig. 3A, a capture device 301 loaded on a vehicle 300 may be used to capture a1 st image 310. The capture device 302 on board the vehicle 300 may be used to capture the 2 nd image 320. The capture device 303 onboard the vehicle 300 may be used to capture the 3 rd image 330. The capture device 304 on board the vehicle 300 may be used to capture the 4 th image 340.

In one example, the vehicle 300 may be, for example, the vehicle 100 described above. The acquisition device 301 may be, for example, the acquisition device 101. The acquisition device 302 may be, for example, the acquisition device 102. The acquisition means 303 may be, for example, acquisition means 103. The acquisition device 304 may be, for example, the acquisition device 104.

For example, taking i as 1 as an example, the first world coordinate values of a plurality of pixels in the 1 st image may be determined, and the second world coordinate values of a plurality of pixels in the 2 nd image may also be determined. In one example, the acquisition device 301 may be a wide-angle camera. The first world coordinate values of the plurality of pixels in the 1 st image may be determined based on the world coordinates of the vehicle 300, the world coordinates of the capturing device 301, and the maximum wide angle of the capturing device 301. In a similar manner, second world coordinate values for a plurality of pixels in the 2 nd image may be determined.

Fig. 3B is a schematic diagram of a first matching region of an ith image according to one embodiment of the present disclosure.

For example, taking i ═ 1 as an example, for the 1 st image 310, the first matching region 311 may be determined from a pixel region in which the first world coordinate value in the 1 st image 310 coincides with the second world coordinate value in the 2 nd image 320. As shown in fig. 3B, the first matching region 311 is located at the lower left corner of the 1 st image 310.

Fig. 3C is a schematic diagram of a second matching region of an i +1 th image according to one embodiment of the present disclosure.

For example, taking i-1 as an example, for the 2 nd image 320, the second matching region 321 may be determined according to a pixel region in which the second world coordinate value in the 2 nd image 320 coincides with the first world coordinate value in the 1 st image 310. As shown in fig. 3C, the second matching region 321 is located in the upper right corner of the 2 nd image 320.

In some embodiments, as described above, taking i-2 as an example, the first matching region of the 2 nd image 320 is located at the lower right corner of the 2 nd image 320, and the second matching region of the 3 rd image 330 is located at the upper left corner of the 3 rd image 330.

In some embodiments, as described above, taking i-3 as an example, the first matching region of the 3 rd image 330 is located in the upper right corner of the 3 rd image 320, and the second matching region of the 4 th image 340 is located in the lower left corner of the 4 th image 340.

In some embodiments, the first matching region for image 4 340 is located in the upper left corner of image 4 340 and the second matching region for image 1 310 is located in the lower right corner of image 1 310.

It will be appreciated that from the 4 images, 4 first matching regions and 4 second matching regions may be determined.

In some embodiments, determining the plurality of first matching sub-regions and the plurality of second matching sub-regions for the first matching region and the second matching region, respectively, comprises: determining a first central sub-region and a plurality of first corner sub-regions of a first matching region, and a second central sub-region and a plurality of second corner sub-regions of a second matching region; and determining a first matching sub-region based on the first central sub-region and one of the first plurality of corner sub-regions, and determining a second matching sub-region based on the second central sub-region and one of the second plurality of corner sub-regions. The following will describe in detail with reference to fig. 4A to 4D.

Fig. 4A is a schematic diagram of a first matching region of an ith image according to another embodiment of the present disclosure.

As shown in fig. 4A, the first matching region 411 includes a first center sub-region 4111, a first corner sub-region 4112a, a first corner sub-region 4112b, a first corner sub-region 4112c, and a first corner sub-region 4112 d. In one example, the first matching region 411 may be, for example, the first matching region 311 described above.

Fig. 4B is a schematic diagram of a first matching sub-region according to another embodiment of the present disclosure.

For example, a first matching sub-region 4113a may be determined based on the first center sub-region 4111 and the first corner sub-region 4112 a.

Fig. 4C is a schematic diagram of a second matching region of an i +1 th image according to another embodiment of the present disclosure.

As shown in fig. 4C, the second matching region 421 includes a second center sub-region 4211, a second corner sub-region 4212a, a second corner sub-region 4212b, a second corner sub-region 4212C, and a second corner sub-region 4212 d. In one example, the second matching region 421 can be, for example, the second matching region 321 described above.

Fig. 4D is a schematic illustration of a second matching subregion in accordance with another embodiment of the present disclosure.

For example, a second matching sub-region 4213b may be determined from the second center sub-region 4211 and the second corner sub-region 4212 b.

The characteristics of the matching region can be better characterized by the region center, and in the embodiment of the disclosure, each matching sub-region comprises the region center, so that the processed image is more real.

In some embodiments, determining the target matching image pair from the differences between the images of the plurality of first matching sub-regions and the images of the plurality of second matching sub-regions from each other comprises: respectively determining a first mean value and a first variance value of the pixels in the first matching subareas and a second mean value and a second variance value of the pixels in the second matching subareas; and determining a mean difference between the first mean and the second mean and a ratio of the first variance to the second variance value. Determining a target first matching sub-region and a target second matching sub-region which are the smallest in difference according to the average value difference and the ratio; and determining a target matching image pair from the image of the target first matching sub-region and the image of the target second matching sub-region.

For example, taking i ═ 1 as an example, the difference Diff between the first matching sub-region and the second matching sub-region can be determined from the mean difference and the ratio by the following formula:

M_A1jis the first mean value, M_B2kIs the second mean value, S_A1jIs a first variance, S_B2kIs the second variance. In one example, the first mean value of the first matching sub-region 4113a described above is M_A11The first variance is S_A11. In one example, the second mean value of the second matching sub-region 4213b described above is M_B22The second variance is S_B22。

In one example, a plurality of differences may be determined by equation one above. If, among these differences, the difference between the above-described first matching sub-region 4113a and the above-described second matching sub-region 4213b is the smallest, the first matching sub-region 4113a may be taken as the target first matching sub-region, and the second matching sub-region 4213b may be taken as the target second matching sub-region. Next, the image of the target first matching sub-region and the image of the target second matching sub-region are determined as a target matching image pair a11_ B22.

As described above, from the 4 images, 4 first matching regions and 4 second matching regions can be determined. Further, a total of 4 target matching image pairs may be determined in a manner similar to the determination of target matching image pair a11_ B22.

Fig. 5 is a flowchart of an image processing method according to one embodiment of the present disclosure.

As shown in fig. 5, the method 550 may process at least one of the ith image and the (i + 1) th image according to the target matched image pair. The following description will be made in detail with reference to operations S551 to S553. In the present embodiment, the target matching image pair a11_ B22 described above is exemplified.

In operation S551, a reference mean value is determined according to the pixel mean value of the first matching image and the pixel mean value of the second matching image.

For example, the first matching image may be, for example, the image corresponding to the first matching sub-region 4113a described above, i.e., the image of the target first matching sub-region.

For example, the second matching image may be, for example, the image corresponding to the second matching sub-region 4213b described above, i.e., the image of the target second matching sub-region.

For example, the mean value between the pixel mean value of the first matching image and the pixel mean value of the second matching image may be taken as the reference mean value M_f. In one example, the first mean value M described above may be calculated_A11And the second mean value M described above_B22To determine a reference mean value M_f。

In operation S552, a reference variance value is determined according to a pixel variance value of the first matching image and a pixel variance value of the second matching image

For example, the mean value between the pixel variance value of the first matching image and the pixel variance value of the second matching image may be taken as the reference variance S_f. In one example, the first variance S described above may be calculated_A11And a second variance of S as described above_B22To determine a reference variance S_f。

At least one of the ith picture and the (i + 1) th picture is processed according to the reference mean value and the reference variance value in operation S553.

In the disclosed embodiment, at least one of the ith image and the (i + 1) th image may be processed using the following formula:

f(x，y)＝[g(x，y)-M_g]k + B (formula two)

B＝[b*M_f+(1-b)*M_g](formula three)

g (x, y) is the pixel value at (x, y) in the ith image or the (i + 1) th image, f (x, y) is the pixel value at (x, y) in the processed ith image or the processed (i + 1) th image, M_fIs a reference mean value, M_gIs the pixel mean value of the ith image or the (i + 1) th image, K is obtained according to the reference variance, b is a preset parameter, and both K and b are more than or equal to 0 and less than or equal to 1.

For example, K is derived from the reference variance by the following formula:

c is another preset parameter, c is greater than or equal to 0 and less than or equal to 1, S_gIs the pixel variance value of the ith image or the (i + 1) th image. By the formula, the previous image characteristics can be maintained, and the image is clearer.

For example, c may be a value of 0.25 to 0.9 inclusive, and b may be a value of 0.25 to 1.0 inclusive.

It will be appreciated that, with reference to the manner in which at least one of the i-th image and the i + 1-th image is processed from the target-matched image pair a11_ B22, at least one of the two images corresponding to that image pair may be processed from any of the 4 target-matched image pairs described above.

In some embodiments, after the 1 st and 2 nd images are processed, the 2 nd and 3 rd images may be processed in a similar manner. Next, the 3 rd image and the 4 th image may be processed in a similar manner. Finally, the 4 th image and the 1 st image may be processed in a similar manner to complete one round of forward image processing. For example, forward may refer to clockwise.

In some embodiments, after processing the 1 st and 2 nd images, the 4 th and 1 st images may be processed in a similar manner. Next, the 3 rd image and the 4 th image may be processed in a similar manner. Finally, the 2 nd and 3 rd images may be processed in a similar manner to complete a round of reverse image processing. For example, reverse may refer to a counterclockwise direction.

It is to be understood that a plurality of rounds of image processing may be performed such that the difference between the N images satisfies a predetermined condition.

It is understood that a plurality of rounds of forward image processing may be performed, a plurality of rounds of reverse image processing may be performed, or a plurality of rounds of forward image processing and a plurality of rounds of reverse image processing may be performed alternately. Through image processing of a plurality of rounds, the color or brightness of the processed image can be smoother or more natural.

Fig. 6 is a flowchart of an image processing method according to another embodiment of the present disclosure.

As shown in fig. 6, the method 660 may be performed after operation S553. Details will be described below in conjunction with operations S661 through S662.

In operation S661, N target matching image pairs of the N images are determined.

For example, taking N-4 as an example, a total of 4 target matching image pairs may be determined, as described above, in a manner similar to the determination of target matching image pair a11_ B22.

In operation S662, the N images are processed according to the pixel mean value and the pixel variance value of 2 × N target matching images among the N target matching image pairs.

In embodiments of the present disclosure, an average pixel mean and an average pixel variance value for 2 × N target matching images may be determined.

For example, the average pixel mean of the nth target matched image pair is determined

And average pixel variance value

In one example, N ═ 1, … N. In this embodiment, n is 1, 2, 3, 4.

In the embodiment of the present disclosure, the average pixel mean value and the average pixel variance value may be respectively used as a reference mean value and a reference variance value to process the N images.

For example, average pixel mean

And average pixel variance value

Respectively as the mean value and the reference square of the referenceAnd difference processing is carried out on the N images.

In one example, the average pixel mean is averaged

And average pixel variance value

And respectively serving as a reference mean value and a reference variance value, and simultaneously processing the 4 images by using the second formula and the third formula to obtain the processed 4 images.

It is to be appreciated that method 660 can be performed after one round of image processing as described above, as well as after a predetermined number of rounds of image processing. So that the color of the processed image is further smoothed or uniform.

In the embodiments described above, N ═ 4 is taken as an example. However, in the present disclosure, the value of N is not limited thereto. In other embodiments, N may be 2. In other embodiments, N may be 5. The image processing method described above can be applied as long as there is a matching region between the N images.

It should be noted that any of the above-mentioned capturing devices can capture RGB images. The N images described above may be grayscale images of the same channel in the N RGB images acquired by the N acquisition devices at the same time. After the above-mentioned image processing is performed on each channel grayscale image in the N RGB images, the grayscale images of 3 channels may be correspondingly fused to obtain the processed N RGB images.

Fig. 7 is a block diagram of an image processing apparatus according to one embodiment of the present disclosure.

As shown in fig. 7, the apparatus 700 may include a first determination module 710, a second determination module 720, a third determination module 730, a third determination module 740, and a processing module 750.

A first determining module 710, configured to determine an ith image and an (i + 1) th image having a matching region in the N images. N is an integer of 2 or more, i is 1, … … N-1.

A second determining module 720, configured to determine a matching region between the ith image and the (i + 1) th image, to obtain a first matching region in the ith image and a second matching region in the (i + 1) th image.

A third determining module 730, configured to determine a plurality of first matching sub-regions and a plurality of second matching sub-regions of the first matching region and the second matching region, respectively.

A fourth determining module 740, configured to determine a target matching image pair according to a difference between the images of the plurality of first matching sub-regions and the images of the plurality of second matching sub-regions.

And the processing module 750 is configured to process at least one of the ith image and the (i + 1) th image according to the target matching image pair.

In some embodiments, the third determining module comprises: a first determination submodule for determining a first central subregion and a plurality of first corner subregions of the first matching region, and a second central subregion and a plurality of second corner subregions of the second matching region; and a second determining submodule for determining one of said first matching sub-regions on the basis of said first central sub-region and one of said first corner sub-regions, and a third determining submodule for determining one of said second matching sub-regions on the basis of said second central sub-region and one of said second corner sub-regions.

In some embodiments, the fourth determining module comprises: a fourth determining submodule, configured to determine a first mean and a first variance of the pixels in the plurality of first matching sub-areas and a second mean and a second variance of the pixels in the plurality of second matching sub-areas, respectively; the fifth determination submodule is used for determining the mean difference between the first mean value and the second mean value and the ratio of the first variance to the second variance value; a sixth determining submodule, configured to determine, according to the average difference and the ratio, a target first matching sub-region and a target second matching sub-region that are the smallest in difference from each other; and a seventh determining sub-module, configured to determine the image of the target first matching sub-region and the image of the target second matching sub-region as a target matching image pair.

In some embodiments, the processing module comprises: the eighth determining submodule is used for determining a reference mean value according to the pixel mean value of the first matching image and the pixel mean value of the second matching image; a ninth determining submodule, configured to determine a reference variance value according to the pixel variance value of the first matching image and the pixel variance value of the second matching image; and a first processing sub-module, configured to process at least one of the ith image and the (i + 1) th image according to the reference mean value and the reference variance value.

In some embodiments, the first processing sub-module comprises: a first processing unit for processing at least one of the i-th image and the i + 1-th image using the following formula: f (x, y) ═ g (x, y) -M_g]*K+B；B＝[b*M_f+(1-b)*M_g](ii) a g (x, y) is the pixel value at (x, y) in the ith image or the i +1 th image, f (x, y) is the pixel value at (x, y) in the processed ith image or the processed i +1 th image, M_fIs the reference mean, M_gIs the pixel mean of the ith image or the (i + 1) th image, K is based on the reference variance S_fB is a preset parameter, and both K and b are more than or equal to 0 and less than or equal to 1.

In some embodiments of the present invention, the,

c is a preset parameter, c is greater than or equal to 0 and less than or equal to 1, S_fIs the reference variance value, S_gIs the pixel variance value of the ith image or the (i + 1) th image.

In some embodiments, further comprising: a tenth determination submodule for determining N target matching image pairs of the N images; and the second processing submodule is used for processing the N images according to the pixel mean value and the pixel variance value of 2 x N target matching images in the N target matching image pairs.

In some embodiments, the second processing sub-module comprises: a first determining unit, configured to determine an average pixel mean value and an average pixel variance value of the 2 × N target matching images; and the second processing unit is used for respectively taking the average pixel mean value and the average pixel variance value as the reference mean value and the reference variance value and processing the N images.

In some embodiments, the second processing submodule includes: a second determination unit for determining an average pixel mean of the nth target matching image pair

And average pixel variance value

A third processing unit for averaging the pixel mean values

And the average pixel variance value

And respectively taking the N images as the reference mean value and the reference variance value to process the N images.

In some embodiments, the second determining module comprises: an eleventh determining submodule for determining first world coordinate values of a plurality of pixels in the ith image; a twelfth determining submodule for determining second world coordinate values of a plurality of pixels in the i +1 th image; a thirteenth determining submodule for determining a pixel region in the ith image, in which the first world coordinate value is consistent with the second world coordinate value, as a first matching region; and a fourteenth determining submodule for determining a pixel region in the i +1 th image, in which the second world coordinate value coincides with the first world coordinate value, as a second matching region.

In some embodiments, the N images are simultaneously acquired by N acquisition devices loaded at corresponding locations on the vehicle, respectively.

In the technical scheme of the disclosure, the collection, storage, use, processing, transmission, provision, disclosure and other processing of the personal information of the related user are all in accordance with the regulations of related laws and regulations and do not violate the good customs of the public order.

The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.

FIG. 8 illustrates a schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 8, the apparatus 800 includes a computing unit 801 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM)802 or a computer program loaded from a storage unit 808 into a Random Access Memory (RAM) 803. In the RAM803, various programs and data required for the operation of the device 800 can also be stored. The calculation unit 801, the ROM 802, and the RAM803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.

A number of components in the device 800 are connected to the I/O interface 805, including: an input unit 806, such as a keyboard, a mouse, or the like; an output unit 807 such as various types of displays, speakers, and the like; a storage unit 808, such as a magnetic disk, optical disk, or the like; and a communication unit 809 such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.

Computing unit 801 may be a variety of general and/or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and the like. The calculation unit 801 executes the respective methods and processes described above, such as an image processing method. For example, in some embodiments, the image processing method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and/or installed onto device 800 via ROM 802 and/or communications unit 809. When the computer program is loaded into the RAM803 and executed by the computing unit 801, one or more steps of the image processing method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the image processing method by any other suitable means (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and the present disclosure is not limited herein.

The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. An image processing method, comprising:

determining an ith image and an i +1 th image having a matching region from among the N images, where N is an integer greater than or equal to 2, and i is 1, … … N-1;

determining a matching region between the ith image and the (i + 1) th image to obtain a first matching region in the ith image and a second matching region in the (i + 1) th image;

determining a plurality of first matching sub-regions and a plurality of second matching sub-regions for the first matching region and the second matching region, respectively;

determining a target matching image pair according to the difference between the images of the plurality of first matching sub-regions and the images of the plurality of second matching sub-regions; and

and processing at least one of the ith image and the (i + 1) th image according to the target matching image pair.

2. The method of claim 1, wherein the determining a plurality of first matching sub-regions and a plurality of second matching sub-regions for each of the first matching region and the second matching region comprises:

determining a first central subregion and a plurality of first corner subregions of the first matching region and a second central subregion and a plurality of second corner subregions of the second matching region; and

determining one of the first matching sub-regions based on the first central sub-region and one of the first plurality of corner sub-regions, and determining one of the second matching sub-regions based on the second central sub-region and one of the second plurality of corner sub-regions.

3. The method of claim 1, wherein determining a target matching image pair from differences between the images of the first and second plurality of matching sub-regions from each other comprises:

determining a first mean value and a first variance value of the pixels in the plurality of first matching sub-regions and a second mean value and a second variance value of the pixels in the plurality of second matching sub-regions, respectively; and

determining a mean difference between the first mean and the second mean and a ratio of the first variance to the second variance value;

determining a target first matching sub-region and a target second matching sub-region which are the smallest in difference according to the average value difference and the ratio; and

and determining the image of the target first matching subarea and the image of the target second matching subarea as a target matching image pair.

4. The method of one of claims 1-3, wherein the target matching image pair comprises a first matching image and a second matching image; processing at least one of the ith image and the (i + 1) th image according to the target matching image pair includes:

determining a reference mean value according to the pixel mean value of the first matching image and the pixel mean value of the second matching image;

determining a reference variance value according to the pixel variance value of the first matching image and the pixel variance value of the second matching image; and

and processing at least one of the ith image and the (i + 1) th image according to the reference mean value and the reference variance value.

5. The method of claim 4, wherein the processing at least one of the i-th image and the i + 1-th image according to the reference mean and the reference variance value comprises processing at least one of the i-th image and the i + 1-th image using the following formula:

f(x，y)＝[g(x，y)-M_g]*K+B

B＝[b*M_f+(1-b)*M_g]

g (x, y) is the pixel value at (x, y) in the ith image or the i +1 th image, f (x, y) is the pixel value at (x, y) in the processed ith image or the processed i +1 th image, M_fIs the reference mean, M_gIs the pixel mean of the ith image or the (i + 1) th image, and K is based on the reference variance S_fObtained, b is a predetermined parameter, KAnd b are both 0 or more and 1 or less.

6. The method of claim 5, wherein,

7. The method of claim 5, further comprising:

determining N target matched image pairs of the N images;

and processing the N images according to the pixel mean value and the pixel variance value of 2 x N target matching images in the N target matching image pairs.

8. The method of claim 7, wherein the processing the N images according to the pixel mean and pixel variance values of 2 x N target matched images of the N target matched image pairs comprises:

determining an average pixel mean and an average pixel variance value of the 2 x N target matching images;

and respectively taking the average pixel mean value and the average pixel variance value as the reference mean value and the reference variance value, and processing the N images.

9. The method of claim 7, wherein the processing the N images according to the pixel mean and pixel variance values of 2 x N target matched images of the N target matched image pairs comprises:

determining an average pixel mean for an nth target matched image pair

And average pixel variance value

Average pixel mean value

And average pixel variance value

10. The method of claim 1, wherein the determining a matching region between the ith image and the (i + 1) th image, resulting in a first matching region in the ith image and a second matching region in the (i + 1) th image comprises:

determining first world coordinate values of a plurality of pixels in the ith image;

determining second world coordinate values of a plurality of pixels in the (i + 1) th image;

determining a pixel region in the ith image, wherein the first world coordinate value is consistent with the second world coordinate value, as a first matching region; and

and determining a pixel region in the i +1 th image, wherein the second world coordinate value is consistent with the first world coordinate value, as a second matching region.

11. The method according to one of claims 1 to 10, wherein the N images are acquired simultaneously by N acquisition devices respectively loaded at corresponding positions on the vehicle.

12. An image processing apparatus comprising:

a first determining module, configured to determine an i-th image and an i + 1-th image having a matching region in N images, where N is an integer greater than or equal to 2, and i is 1, … … N-1;

a second determining module, configured to determine a matching region between the ith image and the (i + 1) th image, to obtain a first matching region in the ith image and a second matching region in the (i + 1) th image;

a third determining module, configured to determine a plurality of first matching sub-regions and a plurality of second matching sub-regions of the first matching region and the second matching region, respectively;

a fourth determining module, configured to determine a target matching image pair according to a difference between the images of the plurality of first matching sub-regions and the images of the plurality of second matching sub-regions; and

and the processing module is used for processing at least one of the ith image and the (i + 1) th image according to the target matching image pair.

13. The apparatus of claim 12, wherein the third determining means comprises:

a first determination submodule for determining a first central subregion and a plurality of first corner subregions of the first matching region, and a second central subregion and a plurality of second corner subregions of the second matching region; and

a second determination submodule for determining one of said first matching sub-areas on the basis of said first central sub-area and one of a plurality of first corner sub-areas, an

A third determination submodule for determining one of said second matching sub-regions from said second central sub-region and one of a plurality of second corner sub-regions.

14. The apparatus of claim 12, wherein the fourth determining means comprises:

a fourth determining submodule, configured to determine a first mean value and a first variance value of the pixels in the first matching sub-regions and a second mean value and a second variance value of the pixels in the second matching sub-regions, respectively; and

a fifth determining submodule, configured to determine a mean difference between the first mean and the second mean and a ratio of the first variance to the second variance;

a sixth determining submodule, configured to determine, according to the average difference and the ratio, a target first matching sub-region and a target second matching sub-region that are the smallest in difference from each other; and

and the seventh determining sub-module is used for determining the image of the target first matching sub-region and the image of the target second matching sub-region as a target matching image pair.

15. The apparatus according to one of claims 12-14, wherein the processing module comprises:

the eighth determining submodule is used for determining a reference mean value according to the pixel mean value of the first matching image and the pixel mean value of the second matching image;

a ninth determining submodule, configured to determine a reference variance value according to the pixel variance value of the first matching image and the pixel variance value of the second matching image; and

a first processing sub-module, configured to process at least one of the ith image and the (i + 1) th image according to the reference mean value and the reference variance value.

16. The apparatus of claim 15, wherein the first processing sub-module comprises: a first processing unit for processing at least one of the i-th image and the i + 1-th image using the following formula:

f(x，y)＝[g(x，y)-M_g]*K+B

B＝[b*M_f+(1-b)*M_g]

g (x, y) is the pixel value at (x, y) in the ith image or the i +1 th image, f (x, y) is the pixel value at (x, y) in the processed ith image or the processed i +1 th image, M_fIs the reference mean, M_gIs the pixel mean of the ith image or the (i + 1) th image, and K is based on the reference variance S_fAnd b is a preset parameter,k and b are both 0-1 inclusive.

17. The apparatus of claim 16, wherein,

18. The apparatus of claim 16, further comprising:

a tenth determination submodule for determining N target matching image pairs of the N images;

and the second processing submodule is used for processing the N images according to the pixel mean value and the pixel variance value of 2 x N target matching images in the N target matching image pairs.

19. The apparatus of claim 18, wherein the second processing sub-module comprises:

a first determining unit, configured to determine an average pixel mean value and an average pixel variance value of the 2 × N target matching images;

and the second processing unit is used for respectively taking the average pixel mean value and the average pixel variance value as the reference mean value and the reference variance value and processing the N images.

20. The apparatus of claim 18, wherein the second processing sub-module comprises:

a second determination unit for determining an average pixel mean of the nth target matching image pair

And the average pixel variance value

A third processing unit for averaging the pixel mean values

And average pixel variance value

21. The apparatus of claim 12, wherein the second determining means comprises:

an eleventh determining submodule for determining first world coordinate values of a plurality of pixels in the ith image;

a twelfth determining submodule for determining second world coordinate values of a plurality of pixels in the i +1 th image;

a thirteenth determining submodule, configured to determine, as a first matching region, a pixel region in the ith image where the first world coordinate value is consistent with the second world coordinate value; and

a fourteenth determining submodule, configured to determine, as a second matching region, a pixel region in the i +1 th image where the second world coordinate value coincides with the first world coordinate value.

22. The apparatus of one of claims 12-21, wherein the N images are acquired simultaneously by N acquisition devices loaded at corresponding locations on the vehicle, respectively.

23. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 11.

24. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1 to 11.

25. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 11.