WO2023225825A1

WO2023225825A1 - Position difference graph generation method and apparatus, electronic device, chip, and medium

Info

Publication number: WO2023225825A1
Application number: PCT/CN2022/094569
Authority: WO
Inventors: 李超; 胡毅
Original assignee: 上海玄戒技术有限公司
Priority date: 2022-05-23
Filing date: 2022-05-23
Publication date: 2023-11-30
Also published as: CN116438568A

Abstract

The present disclosure relates to a position difference graph generation method and apparatus, an electronic device, a chip, and a medium. The method comprises: performing position difference calculation on a first image and a second image to obtain an initial position difference graph, the initial position difference graph comprising at least one hole pixel point having an unknown position difference value; and performing superpixel division on the initial position difference graph, each superpixel obtained by division comprising a plurality of pixel points, and complementing the position difference value of the corresponding hole pixel point on the basis of the position difference values of other pixel points in the superpixel where each hole pixel point is located.

Description

Position difference map generation method and device, electronic equipment, chip and medium

Technical field

The present disclosure relates to the field of image processing, and in particular, to a position difference map generation method and device, electronic equipment, chips and media.

Background technique

In order to improve the quality of image works, most electronic devices nowadays usually need to first obtain a position difference map used to characterize the position relationship between various pixels of different images when shooting images or videos, so as to based on the position difference. Adjust the corresponding image.

For example, in a multi-camera scenario, it is necessary to obtain a disparity map between images captured by different cameras, so as to perform image fusion on multiple images captured based on the disparity map. For another example, when shooting a moving object, the optical flow map between different images taken at different times can be obtained to predict the movement of the moving object based on the optical flow map, and then adjust the image based on the predicted movement. Make adjustments.

Contents of the invention

In view of this, the present disclosure provides a position difference map generation method and device, electronic equipment, chips and media, which can accurately generate a position difference map between two images.

According to a first aspect of the present disclosure, a method for generating a location difference map is provided, including:

Perform dislocation calculation on the first image and the second image to obtain an initial position difference map; the initial position difference map contains at least one hole pixel with an unknown position difference value;

The initial position difference map is divided into super pixels. Each super pixel obtained by the division contains multiple pixels, and the corresponding holes are classified based on the position difference values of other pixels in the super pixel where each hole pixel is located. Complete the pixel value of the pixel.

According to a second aspect of the present disclosure, a location difference map generating device is provided, including:

The calculation unit performs disparity calculation on the first image and the second image to obtain an initial position difference map; the initial position difference map contains at least one hole pixel with an unknown position difference value;

The dividing unit divides the initial position difference map into super pixels. Each super pixel obtained by the division contains multiple pixel points, and the position difference value pair is based on the position difference value of other pixel points in the super pixel where each hole pixel point is located. The pixel value of the corresponding hole pixel is completed.

According to a third aspect of the present disclosure, a position difference map generation method is provided, which is applied to an image processor and includes:

Receive the first image and the second image generated by the image sensor; the first image and the second image are captured by the camera to which the image sensor belongs, aiming at the same subject;

Perform dislocation calculation on the first image and the second image to obtain an initial position difference map, where the initial position difference map contains at least one hole pixel with an unknown position difference value;

The initial position difference map is divided into super pixels. Each super pixel obtained by the division contains multiple pixels, and the corresponding holes are classified based on the position difference values of other pixels in the super pixel where each hole pixel is located. The position difference value of the pixel is completed.

According to a fourth aspect of the present disclosure, an electronic device is provided, including:

processor;

Memory used to store instructions executable by the processor;

Wherein, the processor implements the method described in the first aspect by running the executable instructions.

According to a fifth aspect of the present disclosure, a computer-readable storage medium is provided, computer instructions are stored thereon, and when the instructions are executed by a processor, the steps of the method described in the first aspect are implemented.

In the technical solution of the present disclosure, after obtaining the first image and the second image, dislocation calculation can be performed on the two to obtain an initial position difference map. On this basis, the initial position difference map can be divided into super pixels, and the position difference values of the hole pixels contained in the super pixels can be completed based on the position difference values of each pixel in the divided super pixels. It should be understood that since the present disclosure completes the position difference values of the hole pixels contained in each of them based on the divided super pixels, it avoids the problem of hole pixels in the position difference map generated by the related technology, which affects the final imaging. .

Description of the drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings needed to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without exerting any creative effort.

Figure 1 is a flow chart of a method for generating a position difference map according to an exemplary embodiment of the present disclosure;

Figure 2 is a flow chart of a method for generating a disparity map according to an exemplary embodiment of the present disclosure;

FIG. 3 is a schematic diagram of an initial disparity map according to an exemplary embodiment of the present disclosure;

FIG. 4 is a schematic histogram diagram of an initial disparity map according to an exemplary embodiment of the present disclosure;

FIG. 5 is a schematic diagram of an adjusted initial disparity map according to an exemplary embodiment of the present disclosure;

Figure 6 is a schematic diagram of a super-pixel division according to an exemplary embodiment of the present disclosure;

FIG. 7 is a schematic diagram of a brightness diagram of a main image according to an exemplary embodiment of the present disclosure;

FIG. 8 is a schematic diagram after adjusting the pixel points included in the super pixel according to an exemplary embodiment of the present disclosure;

Figure 9 is a block diagram of a position difference map generating device according to an exemplary embodiment of the present disclosure;

Figure 10 is a block diagram of another location difference map generating device according to an exemplary embodiment of the present disclosure;

Figure 11 is a schematic structural diagram of an electronic device in an exemplary embodiment of the present disclosure.

Detailed ways

The technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only some of the embodiments of the present disclosure, rather than all of the embodiments. Based on the embodiments in this disclosure, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of this disclosure.

The terminology used in the embodiments of the present disclosure is for the purpose of describing specific embodiments only and is not intended to limit the embodiments of the present disclosure. As used in the embodiments of the present disclosure and the appended claims, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will also be understood that the term "and/or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, third, etc. may be used to describe various information in the embodiments of the present disclosure, the information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of the embodiments of the present disclosure, the first information may also be called second information, and similarly, the second information may also be called first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "when" or "in response to determining."

For the purpose of simplicity and ease of understanding, the terms used in this article are "greater than" or "less than", "higher than" or "lower than" when characterizing size relationships. But for those skilled in the art, it can be understood that: the term "greater than" also covers the meaning of "greater than or equal to", and "less than" also covers the meaning of "less than or equal to"; the term "higher than" covers the meaning of "higher than or equal to". "The meaning of "less than" also covers the meaning of "less than or equal to".

In order to improve the imaging effect of the captured images, current electronic devices, such as smartphones and tablets, usually capture multiple frames of images to adjust the images based on the position differences between different images to obtain the final image.

In the present disclosure, an image composed of position difference values between various pixel points of different images may be called a position difference map. In practical applications, the accuracy of the generated position difference map often determines the final imaging effect.

The two most typical position difference maps are: disparity map and optical flow map. Among them, the disparity map refers to the image used to represent the position difference between the images captured by different cameras; and the optical flow map refers to the image used to represent the position difference between the images captured by the moving object at different times. position difference between images. Below, taking the disparity map as an example, we will briefly introduce the method of generating disparity maps in related technologies:

In order to improve imaging quality, electronic devices are usually equipped with multiple cameras, so that when image capture is required, images of the same subject can be captured through multiple cameras. The imaging principle can be summarized as follows: image fusion of images captured by multiple cameras to supplement the details in the image, thereby improving the image quality.

However, in the actual image fusion process, since each camera is installed at different positions of the electronic device, there are certain differences in the captured images. Therefore, before performing image fusion, it is first necessary to determine the position difference of the same picture content in different images, so that image fusion can be performed based on the position difference. Only in this way can different images be accurately fused to achieve the purpose of improving image quality.

In the field of image processing, the position difference of the same picture content in different images is represented by the above-mentioned disparity map. For example, when dual cameras are used to capture images, disparity calculation can be performed based on the images captured by the two cameras, and we get Disparity map between two images. Since the position difference between the two cameras is fixed at the factory and is known, after obtaining the disparity map, based on the position difference between the two cameras and the disparity value of each pixel in the disparity map, The depth information of each pixel in the original image is obtained, and the two images are fused based on the depth information. Similar to the dual camera, when at least three cameras are used to capture images, the above operations can also be performed on any two captured images to obtain the depth information of the corresponding pixels, and then perform image fusion. Here No longer.

From the above introduction, it is easy to see that the accuracy of the disparity map determines the quality of the final image. In the related art, a disparity map is usually generated by performing disparity calculation on two captured images. The principle of disparity calculation is: match pixels representing the same picture content in two images, and use the distance between the two matched pixels as the disparity value of the corresponding pixel. To generate a disparity map in this way, it is necessary to accurately match the content of the two images in order to accurately generate the disparity map. However, in the actual matching application process, there is no guarantee that every pixel in the two images will be accurately matched. It often happens that individual pixels in one image cannot be matched to the corresponding pixels in the other image. (For example, the path between a certain camera and the subject is blocked by objects, etc.), resulting in the generated disparity map where the disparity values of some pixels are unknown. Pixels with unknown disparity values are also called hole pixels. point. Obviously, due to the existence of hole pixels, the disparity map is obviously inaccurate, which will inevitably lead to poor image quality of the image obtained by image fusion.

The method of generating an optical flow map in related technologies is similar to the method of generating a disparity map, except that at least two images used to generate the optical flow map are: images taken by the same camera at different times of the same subject, and Images taken by different cameras on the same subject. Correspondingly, there are hole pixels with unknown optical flow values in the optical flow map obtained by the related technology, which leads to poor quality of the adjusted image after any image is adjusted based on the optical flow map.

To this end, the present disclosure proposes a position difference map generation method to avoid the problem in related technologies that the final imaging based on the position difference map has poor quality due to the presence of hole pixels in the position difference map.

Figure 1 illustrates a method for generating a position difference map according to an exemplary embodiment of the present disclosure. As shown in Figure 1, the method may include the following steps:

Step 102: Calculate the disparity between the first image and the second image to obtain an initial position difference map; the initial position difference map contains at least one hole pixel with an unknown position difference value.

As can be seen from the above introduction, the reason why high-quality images cannot be obtained in related technologies is due to the existence of hole pixels in the position difference map obtained through dislocation calculation.

In view of this, after calculating the position difference between the two captured images, the present disclosure will further complete the position difference value operation of the hole pixels in the obtained initial position difference map to avoid the position difference value in the position difference map. There are hole pixels, which leads to the problem that the final imaging quality based on the position difference map is not high.

In the present disclosure, after the initial position difference map is generated, the initial position difference map can be divided into super pixels, where each super pixel obtained by the division includes multiple pixel points. On this basis, the position difference value of the hole pixel can be completed based on the position difference value of other pixels in the superpixel where each hole pixel is located.

It should be understood that there is a certain continuity in the pixel values of pixels that are relatively close in the image. Similarly, there is a certain degree of continuity in the position information of pixels that are closer in the image, and the position information determines the position difference value of the corresponding pixels, resulting in the position difference between the pixels that are closer in the image. There is also some continuity in the difference values. Therefore, based on the position difference values of pixels within the same super pixel, completing the position difference values of the hole pixels contained in it is equivalent to using the continuity of the position difference values between pixels within the super pixel to complete the hole The position difference value of pixels is completed with high accuracy.

It can be seen that by dividing the super pixels and completing the position difference values of the hole pixels in the corresponding super pixels based on the position difference values of the pixels in the divided super pixels, a more accurate position difference map can be obtained, thereby improving the The quality of the final fused image.

It should be noted that this disclosure can refer to any two captured images as the first image and the second image, and perform position difference calculation on the two to obtain an initial position difference map. Among them, any algorithm can be used to calculate the dislocation, for example, the BM (block matching, block matching) algorithm or the SGM (semi-global matching, semi-global matching algorithm) algorithm can be used. Of course, the above two algorithms are schematic. The specific algorithm used to calculate the dislocation can be determined by those skilled in the art according to actual needs, and this disclosure does not limit this.

In the present disclosure, since the camera is likely to produce certain distortion when capturing images, the first image and the second image can be dedistorted before performing disparity calculation. In practical applications, any de-distortion algorithm can be used to de-distort the first image and the second image, and the present disclosure does not limit this. In addition, after completing the dedistortion operation on the first image and the second image, the first image and the second image can also be image aligned to more accurately calculate the disparity of the images.

In the present disclosure, the above-mentioned hole pixels can be filled in various scenarios. For example, in a multi-camera scenario, different cameras can be used to capture images of the same subject to obtain a first image and a second image. In this case, the disparity calculation of the first image and the second image can be Parallax calculation, the calculated position difference map can be a parallax map; for another example, in a sports scene, the same camera can be used to capture images of the same subject at different times to obtain the first image and the second image. When , the position difference calculation performed on the first image and the second image may be an optical flow calculation, and the calculated position difference map may be an optical flow map. Of course, multi-camera scenes and sports scenes are schematic. The specific scene to which the technical solution of the present disclosure is applied can be determined by those skilled in the art according to actual needs, and the present disclosure does not limit this.

Step 104: Perform super-pixel division on the initial position difference map. Each super pixel obtained by division includes multiple pixel points, and the position difference value pair is based on the position difference value of other pixel points in the super pixel where each hole pixel point is located. The position difference value of the corresponding hole pixel is completed.

In this disclosure, the meaning of superpixel is: a set of pixels composed of multiple pixels. In other words, the process of superpixel division can also be regarded as: the process of assigning all pixel points in the initial position difference map to different pixel point sets.

In the present disclosure, the initial position difference map can be divided into super pixels according to a preset size, so that each divided super pixel contains a preset number of pixels. For example, superpixels can be divided with a size of "3*3" so that each divided superpixel contains 9 pixels. Of course, this example is only illustrative. The specific size used for super-pixel division can be determined by those skilled in the art according to actual needs, and this disclosure does not limit this.

As mentioned above, both the pixel value distribution of the image and the position difference value distribution of the position difference map show a certain degree of continuity, and the continuity of the two is usually similar. For example, when shooting a portrait, there is a certain continuity in the pixel values of various positions of a certain piece of skin on the person's face; and there is also a certain degree of continuity in the position information of each position of the skin. Since the position difference value is affected by the position information, Therefore, there is also continuity between position difference values. It can be seen that the distribution of pixel values can reflect the distribution of position difference values to a certain extent.

In view of this, after the superpixels are divided, the superpixels can also be adjusted based on the distribution of pixel values, so that there is a certain continuity between the pixels inside the adjusted superpixels, and thus the superpixels can be more accurately classified. Hole pixels inside the pixels are filled.

For example, the pixel value distribution of the first image can be obtained, and the pixel points included in each superpixel can be adjusted according to the pixel value distribution. It should be understood that the pixel value distribution can be used to characterize the flat area and the edge area in the first image, where the flat area refers to the area where the pixel value changes relatively gently, and the edge area refers to the area where the pixel value changes relatively rapidly. Area. The present disclosure can set a preset value as a standard for dividing flat areas and edge areas. Pixels whose pixel difference from its own neighborhood pixels is not greater than the preset value can be determined to belong to the flat area, and pixels that are different from its own neighborhood pixels can be determined to belong to the flat area. Pixels whose pixel difference value of neighboring pixels is greater than the preset value are determined to belong to the edge area. On this basis, the pixels included in the superpixel can be adjusted so that the pixels included in the adjusted superpixel belong to the same flat area or the same edge area.

It should be understood that when the pixel points included in a super pixel belong to the same flat area or the same edge area, it means that there is a certain continuity in the change in position difference value between each pixel point in the corresponding super pixel. At this time, the hole pixels contained in the super pixel are completed based on the position difference values of the pixels in the super pixels, and the position difference values completed for the hole pixels are more accurate.

In the present disclosure, a brightness map of the first image can be obtained as the above-mentioned pixel value distribution. On this basis, based on the brightness value of each pixel in the brightness map, the pixels contained in each divided super pixel can be adjusted, so that each pixel in the same super pixel after adjustment and its neighbor pixels The brightness difference of the points does not exceed the preset brightness value.

It should be understood that the brightness map represents the brightness distribution of the first image, which is equivalent to determining the distribution of position difference values based on the distribution of brightness values. Of course, in practical applications, since there is a certain similarity in the distribution of various pixel values, the distribution of position difference values can be determined through the distribution of other types of pixel values, such as RGB values.

It should be stated that in the actual disparity calculation process, one of the two images is used as the reference image, and "pixels in the other image that match the content of any pixel in the reference image are "The distance between " and any pixel is used as the position difference value of any pixel. In other words, the position difference value of any pixel in the initial position difference map is used to characterize the difference between the pixel at the same position in the reference image and "the pixel in another image that matches the content of the pixel at the same position." distance. Therefore, when adjusting the pixels included in the superpixel based on the brightness value distribution of the first image, the first image refers to the image that is used as the reference image when calculating the disparity. For example, in a multi-camera scenario, since the imaging quality of the main camera is higher, the image captured by the main camera is usually used as the reference image when calculating the disparity.

In the present disclosure, after the superpixels are divided, the positional difference values of the hole pixels contained in the superpixels can be completed based on the positional difference values of the pixels contained in the superpixels. This disclosure can use multiple methods to complete hole pixels.

In one embodiment, when completing any hole pixel, the position difference values of other pixels in the superpixel where the hole pixel is located can be obtained first, and the obtained position differences can be calculated. The calculated average value is used as the position difference value of any hole pixel.

In this embodiment, in order to more accurately complete the position difference value of the hole pixel point, weight values can also be added to the above-mentioned other pixel points. Then, the calculated average value of the position difference value of the other pixel points can be weighted average value. In this embodiment, weight values can be set for the above-mentioned other pixels in various ways. For example, the weight value can be set according to the distance between each pixel and the hole pixel, where the weight value of any pixel can be negatively correlated with the distance, that is, the weight of the pixel in the super pixel is closer to the hole pixel. The higher the value. It should be understood that due to the continuity of the position difference values, the position difference values of pixels that are closer to the hole pixel point are closer to the position difference value of the hole pixel point. Therefore, the weight of each pixel point is set in this way. value, the position difference value of the hole pixel can be determined more accurately. Of course, the method of setting the weight value for each pixel according to the distance from the hole pixel is only illustrative. Those skilled in the art can also use other methods to set the weight value for each pixel according to actual needs. For example, they can also set the weight value for each pixel according to the distance from the hole pixel. The distance from the center of the superpixel to which it belongs sets a weight value for each pixel, and the relationship between the two can also be a negative correlation, which is not limited by this disclosure.

In another embodiment, when completing the position difference value of any hole pixel, the position difference values of other pixels in the superpixel where the hole pixel is located can be obtained first, and each obtained position can be obtained. The median value of the difference value is used as the position difference value of any hole pixel.

Of course, the above examples are only illustrative. Specifically, how to complete the position difference value of the hole pixel point according to the position difference value of the pixel point in the super pixel can be set by those skilled in the art according to actual needs, and this disclosure does not limit this. .

In the present disclosure, the position difference values of pixels whose position difference values are not within the preset disparity range in the initial position difference map can also be adjusted to unknown, so that the corresponding pixel points are converted into hole pixel points. It should be understood that for two images taken of the same subject, since the position information of the area with the same picture content is almost the same, therefore, the position calculated based on the two pixels with the same picture content in the two images The difference value is usually not too large. If the position difference of a certain pixel is large and exceeds the preset range, it is likely that a matching error occurred when matching pixels based on the image content mentioned above. It can be seen that through the above method of converting pixels whose position difference values exceed the preset parallax range into hole pixels, the problem of position difference value calculation errors caused by inaccurate pixel matching can be effectively avoided.

It should be understood that in most cases, matching errors usually lead to the calculated position difference value being too large. Therefore, the preset parallax range is mostly the upper limit of the position difference value. When the position difference value exceeds this When the upper limit is reached, the corresponding pixels are converted into hole pixels. In a feasible implementation, a histogram of the initial position difference map can be generated, and the pixels in the histogram whose position difference value is higher than the preset position difference value can be converted into hole pixels.

In addition, since the number of pixels in the image is large, it can usually cover a larger number of position difference values. When the number of pixels with a position difference value of certain values is small, it is likely to be due to mismatching. caused. Therefore, the present disclosure can also determine pixels whose position difference value is smaller than the preset value as mismatched pixels. In this case, it is also possible to generate a histogram of the initial position difference map first, determine the number of values smaller than the preset value based on the histogram, and then convert the pixels with the position difference value to the hole pixels.

In the present disclosure, after completing position difference values of hole pixels based on super pixels, median filtering can also be performed on the first image to obtain the first image after removing noise, and based on the filtered first image The pixel value continuity of each included pixel point is determined, and then a secondary completion of the position difference value is performed on the position difference map completed by the position difference value based on the pixel value continuity. For example, when using the bilateral operator principle (FastBilateralSolver) to perform secondary completion of the position difference map, the first image obtained through filtering and the position difference map completed through the position difference value can be used as the input of the bilateral operator algorithm. Position difference map after secondary completion with output position difference value

It should be noted that the technical solution of the present disclosure can be applied to any type of electronic equipment. For example, the electronic equipment can be mobile terminals such as smartphones and tablet computers, or fixed terminals such as smart TVs and PCs (Personal Computers). terminal. Which type of electronic device is specifically used as the execution subject of the technical solution of the present disclosure can be determined by those skilled in the art according to actual needs, and the present disclosure does not limit this.

It should also be stated that the technical solution of the present disclosure can be implemented not only through software, but also through various entity components. For example, a distortion correction component, an image alignment component, a disparity calculation component, a superpixel adjustment component, a position difference value completion component, etc. can be deployed in an electronic device to implement each step in the technical solution of the present disclosure.

It can be seen from the above introduction that when the present disclosure obtains the first image and the second image obtained by photographing the same subject, the disparity calculation can be performed on the first image and the second image to obtain a pixel point containing at least one hole. Initial position difference map. On this basis, the present disclosure can further divide the initial position difference map into super pixels, and supplement the position difference value of the corresponding hole pixel point based on the position difference value of other pixel points in the super pixel where each hole pixel point is located. Complete.

It should be understood that there is a certain continuity in the distribution of position difference values. Therefore, it is more accurate to complete the position difference value of the hole pixel contained in the super pixel based on the position difference value of the pixels in the same super pixel, which is equivalent to the position difference value of the hole pixel based on the continuity characteristics of the position difference value. The difference value is used for completion, which has high accuracy and avoids the problem in related technologies that the final imaging quality based on the position difference map is not high due to the presence of empty pixels with unknown position difference values in the position difference map.

Furthermore, the pixel value distribution of the image is usually close to the position difference value distribution of its position difference map. Therefore, after the super pixels are obtained by dividing based on the preset size, the present disclosure can further obtain the pixel value distribution of the first image, and adjust the pixel points contained in the divided super pixels based on the pixel value distribution, so as to Make the pixels included in the adjusted superpixels be located in the same flat area or the same edge area. It should be understood that the position difference values of pixels located in the same flat area or the same edge area show a certain degree of continuity. Therefore, the pixels in the same super pixel are adjusted to the same flat area or the same edge area, and based on The position difference value of the pixels in the super pixel is used to complete the position difference value of the hole pixels in the super pixel, which can rely on its continuity characteristics to improve the accuracy of parallax completion of the hole pixels.

It should be understood that although it is pointed out above that the technical solution of the present disclosure is applicable to electronic equipment. However, in actual applications, it is the physical components contained in the electronic device that perform the multiple operations described above. For example, the above operations may be performed by a central processing unit of the electronic device, or may be performed by an image processor independently deployed in the electronic device.

Since the central processing unit usually has a high load and image processing tasks take up a lot of resources, in order to improve the efficiency of image processing, technicians usually deploy an independent image processor to complete image processing tasks.

In view of this, the present disclosure also proposes a disparity map generation method applied to an image processor. In this method, most operations are consistent with the disparity map generation method applied to electronic devices described above, except that the image processor is described as the execution subject. For relevant content, please refer to the above introduction and will not be repeated below.

In this disparity map generation method, the following steps may be included:

Step 1: Receive a first image generated by a first image sensor and a second image generated by a second image sensor; the first image and the second image are generated by the first camera to which the first image sensor belongs and the second image generated by the second image sensor. The second camera to which the second image sensor belongs takes pictures of the same subject.

In this embodiment, an electronic device may be equipped with an image sensor for image collection, so as to generate a first image and a second image corresponding to the same subject based on the collected raw data. On this basis, the image sensor can transmit the generated first image and the second image to the image processor, so that the image sensor performs position difference calculation on the first image and the second image to obtain a position difference map.

As mentioned above, since the technical solution of this embodiment is different from the position difference map generation method described above, only the execution subject is different. Therefore, this embodiment no longer performs disparity calculation, super-pixel division, and disparity value compensation. The congruent operations will not be described in detail. For relevant content, please refer to the introduction above.

As mentioned above, the present disclosure can be applied to both multi-camera scenes and sports scenes. Therefore, when this embodiment is applied to different scenarios, there are certain differences in the execution of this step. in,

In a multi-camera scenario, the electronic device may be equipped with a first camera and a second camera, where the first camera includes a first image sensor and the second camera includes a second image sensor. On this basis, the actual execution process of this step may be: image acquisition is performed through the first image sensor and the second image sensor respectively, so that the first image sensor and the second image sensor generate images with the same object based on the collected original data. The first image and the second image corresponding to the subject. On this basis, the first image sensor and the second image sensor can transmit the generated first image and the second image to the image processor, so that the image processor performs parallax calculation on the first image and the second image to obtain Disparity map.

It should be stated that in a multi-camera scenario, the disparity value in the disparity map represents the difference in depth information, that is, the position information mentioned above refers to the depth information, usually "from the camera to the corresponding object in the image" "distance" (mostly refers to the distance information in the direction of the camera axis). The disparity map is ultimately used for image fusion of the first image and the second image, which essentially improves the quality of the final imaging by improving the accuracy of image fusion.

In a sports scene, the electronic device can call the assembled image sensor to collect images twice at different times to obtain the first image and the second image. For example, the continuous shooting mode can be turned on to continuously shoot the subject, or Use video mode for video shooting. During this process, the subject and the electronic device may move relative to each other, so that the position information of the subject in the two images changes. On this basis, the image sensor can transmit the first image and the second image to the image processor, so that the image processor performs optical flow calculation on the first image and the second image to obtain an optical flow map.

It should be stated that in a sports scene, the optical flow value in the optical flow map represents the position difference at different times, that is, the difference in position information mentioned above is the displacement. The optical flow map is usually used to adjust a certain image. For example, based on the optical flow map and the image captured first, the image captured later is adjusted to eliminate afterimages, etc., thereby improving the quality of the final imaging. Of course, this example is only illustrative. The specific method of adjusting the image based on the optical flow map can be determined by those skilled in the art according to actual needs, and this embodiment does not limit this.

Step 2: Perform disparity calculation on the first image and the second image to obtain an initial disparity map. The initial disparity map contains at least one hole pixel with unknown disparity value.

Step 3: Perform super-pixel division on the initial disparity map. Each super-pixel obtained by the division contains multiple pixels, and the corresponding disparity values are based on the disparity values of other pixels in the super-pixel where each hole pixel is located. The disparity value of the hole pixel is completed.

It should be noted that the image processor in this embodiment can be mounted on different chips according to the actual situation. For example, it can be mounted on an ISP (Image Signal Processing) chip or an SoC (System on Chip) chip. , the specific chip to be mounted on can be determined by those skilled in the art according to actual needs, and this disclosure does not limit this.

FIG. 2 is a flowchart of a method for generating a disparity map according to an exemplary embodiment of the present disclosure. As shown in Figure 2, this method is applied to a smartphone equipped with at least two cameras and may include the following steps:

Step 201: Capture an image of the subject based on the main photography and the secondary photography.

In this embodiment, the smartphone can be equipped with a main camera with better imaging effect and a secondary camera with relatively poor imaging effect. Then, when the user takes a picture of the subject through the dual camera mode, the smartphone can simultaneously call the main camera and the secondary camera to take pictures of the subject to obtain the main camera image and the secondary camera image.

Step 202: Use the SGM algorithm to calculate parallax on the captured main image and secondary image.

In this embodiment, after obtaining the main image and the secondary image, disparity calculation can be performed on the two based on the preset SGM algorithm to obtain an initial disparity map. For example, the initial disparity map obtained through disparity calculation can be shown in Figure 3, in which the disparity values of most pixels are known, but the disparity values of some pixels are still unknown.

It should be noted that due to the high imaging quality of the main camera, the initial parallax map is mostly generated based on the main camera. In other words, the disparity value of any pixel in the generated initial disparity map is used to represent "the distance between a pixel at the same position in the main image and a pixel in the secondary image that matches the content of the pixel."

Step 203: Generate a histogram of the obtained initial disparity map.

As introduced above, disparity calculation actually calculates the distance between pixels with consistent content in the two images. The content matching between pixels directly determines the accuracy of disparity calculation. However, mismatching of pixels will inevitably occur in the actual matching process, resulting in inaccurate disparity values of corresponding pixels.

Therefore, in this embodiment, after the initial disparity map is generated, pixels containing inaccurate disparity values due to mismatching may be identified, and the identified pixels may be converted into hole pixels. On this basis, the pixel can be completed in the subsequent hole pixel completion operation. It is not difficult to see that this process is equivalent to correcting the pixels with inaccurate parallax values caused by mismatching.

In this embodiment, the histogram of the initial disparity map can be obtained to determine pixels with a disparity value higher than a preset value as pixels with inaccurate disparity values, and convert the pixels into hole pixels. For example, the histogram obtained based on the initial disparity map shown in Figure 3 can be as shown in Figure 4, that is, the number of pixels with disparity values of each value can be statistically obtained. Assuming that the preset upper limit of the disparity value is 8, pixels with a disparity value exceeding 8 can be converted into hole pixels, that is, the initial disparity map shown in Figure 3 is converted into the disparity map shown in Figure 5.

Of course, this example is only illustrative, and pixels with inaccurate disparity values may also be determined without relying on the preset value. For example, after counting the number of pixels with disparity values of each value, the value with the smallest number of pixels can be determined, and the pixels with the disparity value of this value can be converted into hole pixels. How to specifically determine pixels with inaccurate disparity values and convert them into hole pixels can be determined by those skilled in the art according to actual conditions, and this embodiment does not limit this.

Step 204: Convert pixels in the initial disparity map whose disparity value exceeds a preset value into hole pixels based on the histogram.

Step 205: Perform super-pixel division on the initial disparity map according to a preset size.

In this embodiment, the initial disparity map can be divided into super pixels according to a preset size. It should be stated that since the size of the initial disparity map is the same as that of the main image, and the pixels at the same position correspond to the same picture content, super-pixel division of the initial disparity map is equivalent to super-pixel division of the main image. , there are only differences in expression, but the actual meaning is the same.

Following the above example, the initial disparity map can be divided into super pixels with a size of "3*3" to obtain several super pixels containing 9 pixels as shown in Figure 6, such as super pixels A, B, and C. Among them, superpixel A contains hole pixel point a, superpixel B contains hole pixel points b1 and b2, and superpixel C contains hole pixel points c1 and c2.

Step 206: Obtain the brightness map of the main image.

Following the above example, assume that the obtained brightness map of the main camera image is as shown in Figure 7. Among them, the brightness difference between the circled pixels and their neighboring pixels is large and belongs to the edge area in the main camera image, while The brightness difference between other pixels and neighboring pixels is small and belongs to a flat area. Step 207: Adjust the pixels contained in the superpixel based on the brightness map.

In this embodiment, after obtaining the brightness map, the pixels contained in the super pixels can be further adjusted based on the brightness map. The adjustment standard is: as pointed out above, the pixels in the adjusted super pixels belong to the same flat area or the same edge area. Among them, the number of pixels contained in the superpixels before and after adjustment can be further restricted to be the same.

Following the above example, assume that the superpixels obtained by the division shown in Figure 6 are adjusted based on the brightness map shown in Figure 7 . Then, the superpixels obtained by adjusting superpixels A, B, and C respectively are superpixels A’, B’, and C’ as shown in Figure 8. Among them, the pixels included in superpixel A' belong to the flat area on the left side of the image; the pixels included in superpixel B' belong to the edge area circled in Figure 7; and superpixel C' belongs to the flat area in the upper right corner of the image. .

Step 208: Complete the hole pixels in the superpixel based on the adjusted disparity value of each pixel in the superpixel.

Following the above example, after obtaining the adjusted superpixels A', B', and C', the hole pixel a can be completed based on the disparity value of the pixel in the superpixel A', assuming that averaging is used To complete it, then the average disparity value of each pixel in super pixel A' is: (2*2+3*2+4+5+7*2)/8=4.125. Since the disparity value is usually an integer, 4 can be used as the disparity value of the hole pixel point a.

Other hole pixels, such as hole pixels b1, b2, c1, c2, can also be filled in a similar way until all hole pixels in the image are filled, and a more accurate secondary image can be obtained. Disparity map of the main image. It should be understood that the above examples only use superpixels A, B, and C as examples to introduce the technical solutions in this specification. The operation methods of other pixels are also similar and will not be described again here.

It should be noted that since the size of the brightness map, the disparity map, and the main camera image are also consistent, super-pixel division of the brightness map is also equivalent to super-pixel division of the disparity map. Therefore, in actual operation, the brightness map can also be divided into super pixels, and the divided super pixels in the brightness map can be adjusted. In this case, the order of

steps

205, 206, 207, and 208 can be adjusted. For example, the brightness map can be obtained first, and then the brightness map can be divided into super pixels based on the preset size, and then based on the brightness value of each pixel in the brightness map Adjust the divided superpixels. On this basis, the operation of completing the hole pixels based on the disparity value within the superpixel is performed.

It can be seen from the above technical solution that after acquiring the main camera image and the secondary camera image, the smartphone in this embodiment can perform disparity calculation on the two to obtain an initial disparity map between the two, and perform a disparity calculation on the initial disparity map. Superpixel partitioning. Furthermore, the divided superpixels can also be adjusted based on the brightness value of the main image, so that each pixel in the superpixel belongs to the same flat area or edge area. On this basis, the hole pixels contained in the super pixel are completed based on the disparity value of each pixel in the super pixel, which avoids the problem of image processing based on the disparity map due to the existence of hole pixels in the disparity map in related technologies. The problem of poor quality of the fused image.

FIG. 9 is a block diagram of a position difference map generating device according to an exemplary embodiment of the present disclosure. Referring to FIG. 9 , the device includes a computing unit 901 and a dividing unit 902 .

The calculation unit 901 performs dislocation calculation on the first image and the second image to obtain an initial position difference map; the initial position difference map contains at least one hole pixel with an unknown position difference value;

The dividing unit 902 performs super-pixel division on the initial position difference map. Each super pixel obtained by division includes multiple pixel points, and is based on the position difference values of other pixel points in the super pixel where each hole pixel point is located. Complete the position difference value of the corresponding hole pixel.

optional,

The position difference map is a disparity map, and the first image and the second image are captured by different cameras for the same subject; or,

The position difference map is an optical flow map, and the first image and the second image are obtained by shooting the same subject at different times with the same camera.

Optionally, the dividing unit 902 is further used for:

The initial position difference image is divided into super pixels according to a preset size, so that each divided super pixel contains a preset number of pixel points.

Optionally, the dividing unit 902 is further used for:

Obtain the position difference value of each other pixel point in the superpixel where any hole pixel is located, and calculate the average value of each obtained position difference value, so as to use the calculated average value as the any hole pixel point position difference value. Optionally, the dividing unit 902 is further used for:

Calculate the weighted average of the obtained position difference values; wherein, the weight value of any other pixel is negatively correlated with the distance between the any other pixel and any hole pixel.

Optionally, the dividing unit 902 is further used for:

The position difference value of each other pixel point in the super pixel where any hole pixel point is located is obtained, and the median value of the obtained position difference values is taken as the position difference value of any hole pixel point.

As shown in Figure 10, Figure 10 is a block diagram of another location difference map generating device according to an exemplary embodiment of the present disclosure. Based on the aforementioned embodiment shown in Figure 9, this embodiment also includes: a determination unit 903 , adjustment unit 904 and filtering unit 905.

Optional, also includes:

Determining unit 903 determines the distribution of pixel values in the first image. The distribution of pixel values is used to characterize the flat areas and edge areas in the first image; where the pixels in the flat area and their neighbors The pixel difference value of a pixel is not greater than the preset value, and the pixel difference value of a pixel in the edge area and its neighbor pixels is greater than the preset value;

The adjustment unit 904 adjusts the pixel points included in each super pixel based on the pixel value distribution, so that the adjusted pixel points in the same super pixel belong to the same flat area or the same edge area.

optional,

The determining unit 903 is further configured to: obtain the brightness map of the first image;

The adjustment unit 904 is further configured to: based on the brightness value of each pixel point in the brightness map, adjust the pixel points contained in each divided super pixel, so that each pixel point in the same super pixel after adjustment is the same as its neighbor. The brightness difference of domain pixels does not exceed the preset brightness value.

Optionally, the adjustment unit 904 is also used for:

In the initial position difference map, the position difference values of pixels whose position difference values are not within the preset parallax range are adjusted to unknown, so that the corresponding pixels are converted into hole pixels.

optional,

It also includes: a filtering unit 905 that performs median filtering on the first image to obtain the first image after removing noise;

The dividing unit 902 is also used to: determine the pixel value continuity of each included pixel point based on the filtered first image, and position the position difference map completed by the position difference value based on the determined pixel value continuity. Secondary completion of difference values.

As for the device embodiment, since it basically corresponds to the method embodiment, please refer to the partial description of the method embodiment for relevant details. The device embodiments described above are only illustrative. The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in One location, or it can be distributed across multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the disclosed solution. Persons of ordinary skill in the art can understand and implement the method without any creative effort.

Correspondingly, the present disclosure also provides a device for generating a location difference map, including: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to implement as described in any of the above embodiments A method for generating a position difference map. For example, the method may include: performing disparity calculation on the first image and the second image to obtain an initial position difference map; the initial position difference map contains at least one hole pixel with an unknown position difference value. ; Perform super-pixel division on the initial position difference map. Each super-pixel obtained by the division contains multiple pixels, and the corresponding corresponding pixels are calculated based on the position difference values of other pixels in the super-pixel where each hole pixel is located. The position difference value of the hole pixel is completed.

Correspondingly, the present disclosure also provides an electronic device. The electronic device includes a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be processed by one or more processors. Executing the one or more programs includes instructions for implementing the position difference map generation method as described in any of the above embodiments. For example, the method may include: performing disparity calculation on the first image and the second image to obtain An initial position difference map; the initial position difference map contains at least one hole pixel with an unknown position difference value; the initial position difference map is divided into super pixels, and each super pixel obtained by the division contains multiple pixels, And the position difference value of the corresponding hole pixel is completed based on the position difference value of other pixels in the superpixel where each hole pixel is located.

Correspondingly, the present disclosure also provides a chip, which includes one or more interface circuits and one or more processors; the interface circuit is used to receive signals from the memory of the electronic device and send signals to the processor. The signal includes a computer instruction stored in a memory; when the processor executes the computer instruction, the electronic device is caused to execute any of the position difference map generating methods described above.

FIG. 11 is a block diagram of a device 1100 for implementing a position difference map generation method according to an exemplary embodiment. For example, the device 1100 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like.

Referring to Figure 11, the device 1100 may include one or more of the following components: a processing component 1102, a memory 1104, a power supply component 1106, a multimedia component 1108, an audio component 1110, an input/output (I/O) interface 1112, a sensor component 1114, and communications component 1116.

Processing component 1102 generally controls the overall operations of device 1100, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 1102 may include one or more processors 1120 to execute instructions to complete all or part of the steps of the above method. Additionally, processing component 1102 may include one or more modules that facilitate interaction between processing component 1102 and other components. For example, processing component 1102 may include a multimedia module to facilitate interaction between multimedia component 1108 and processing component 1102.

Memory 1104 is configured to store various types of data to support operations at device 1100 . Examples of such data include instructions for any application or method operating on device 1100, contact data, phonebook data, messages, pictures, videos, etc. Memory 1104 may be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EEPROM), Programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.

Power supply component 1106 provides power to various components of device 1100 . Power supply components 1106 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 1100 .

Multimedia component 1108 includes a screen that provides an output interface between the device 1100 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide action. In some embodiments, multimedia component 1108 includes a front-facing camera and/or a rear-facing camera. When the device 1100 is in an operating mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data. Each front-facing camera and rear-facing camera can be a fixed optical lens system or have a focal length and optical zoom capabilities.

Audio component 1110 is configured to output and/or input audio signals. For example, audio component 1110 includes a microphone (MIC) configured to receive external audio signals when device 1100 is in operating modes, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 1104 or sent via communications component 1116 . In some embodiments, audio component 1110 also includes a speaker for outputting audio signals.

The I/O interface 1112 provides an interface between the processing component 1102 and a peripheral interface module. The peripheral interface module may be a keyboard, a click wheel, a button, etc. These buttons may include, but are not limited to: Home button, Volume buttons, Start button, and Lock button.

Sensor component 1114 includes one or more sensors for providing various aspects of status assessment for device 1100 . For example, the sensor component 1114 can detect the open/closed state of the device 1100, the relative positioning of components, such as the display and keypad of the device 1100, and the sensor component 1114 can also detect a change in position of the device 1100 or a component of the device 1100. , the presence or absence of user contact with device 1100 , device 1100 orientation or acceleration/deceleration and temperature changes of device 1100 . Sensor assembly 1114 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 1114 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 1114 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

Communications component 1116 is configured to facilitate wired or wireless communications between device 1100 and other devices. The device 1100 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, 4G LTE, 5G NR (New Radio), or a combination thereof. In one exemplary embodiment, the communication component 1116 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communications component 1116 also includes a near field communications (NFC) module to facilitate short-range communications. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

In an exemplary embodiment, apparatus 1100 may be configured by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable Gate array (FPGA), controller, microcontroller, microprocessor or other electronic components are implemented for executing the above method.

In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions, such as a memory 1104 including instructions, which are executable by the processor 1120 of the device 1100 to complete the above method is also provided. For example, the non-transitory computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

Other embodiments of the disclosure will be readily apparent to those skilled in the art from consideration of the specification and practice of the disclosure herein. The present disclosure is intended to cover any variations, uses, or adaptations of the disclosure that follow the general principles of the disclosure and include common common sense or customary technical means in the technical field that are not disclosed in the disclosure. . It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It is to be understood that the present disclosure is not limited to the precise structures described above and illustrated in the accompanying drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the disclosure is limited only by the appended claims.

The above are only preferred embodiments of the present disclosure and are not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included in this disclosure. within the scope of protection.

Claims

A method for generating a position difference map, which is characterized by including:

Perform dislocation calculation on the first image and the second image to obtain an initial position difference map; the initial position difference map contains at least one hole pixel with an unknown position difference value;

The initial position difference map is divided into super pixels. Each super pixel obtained by the division contains multiple pixels, and the corresponding holes are classified based on the position difference values of other pixels in the super pixel where each hole pixel is located. The position difference value of the pixel is completed.
The method according to claim 1, characterized in that:

The position difference map is a disparity map, and the first image and the second image are captured by different cameras for the same subject; or,

The position difference map is an optical flow map, and the first image and the second image are obtained by shooting the same subject at different times with the same camera.
The method according to claim 1, characterized in that said super-pixel division of the initial position difference map includes:

The initial position difference image is divided into super pixels according to a preset size, so that each divided super pixel contains a preset number of pixel points.
The method according to claim 1, further comprising:

Determine the distribution of pixel values in the first image. The distribution of pixel values is used to characterize the flat area and edge area in the first image; wherein, the pixels in the flat area and the pixels in its neighbor pixels are The difference is not greater than the preset value, and the pixel difference between the pixels in the edge area and its neighbor pixels is greater than the preset value;

The pixel points included in each super pixel are adjusted based on the pixel value distribution, so that the adjusted pixel points in the same super pixel belong to the same flat area or the same edge area.
The method according to claim 4, characterized in that:

Determining the distribution of pixel values in the first image includes: obtaining a brightness map of the first image;

Adjusting the pixel points included in each super pixel based on the pixel value distribution includes: adjusting the pixel points included in each divided super pixel based on the brightness value of each pixel point in the brightness map. , so that the adjusted brightness difference between each pixel in the same super pixel and its neighboring pixels does not exceed the preset brightness value.
The method according to claim 1, characterized in that, the position difference value of any hole pixel point is completed based on the position difference value of other pixel points in the super pixel where any hole pixel point is located, including: :

Obtain the position difference value of each other pixel point in the superpixel where any hole pixel is located, and calculate the average value of each obtained position difference value, so as to use the calculated average value as the any hole pixel point position difference value.
The method according to claim 6, characterized in that the calculation of the average value of each obtained position difference value includes:

Calculate the weighted average of the obtained position difference values; wherein, the weight value of any other pixel is negatively correlated with the distance between the any other pixel and any hole pixel.
The method according to claim 1, characterized in that, the position difference value of any hole pixel point is completed based on the position difference value of other pixel points in the super pixel where any hole pixel point is located, including: :

Obtain the position difference value of each other pixel point in the superpixel where any hole pixel is located, and take the median value of the obtained position difference values as the position difference value of any hole pixel point.
The method according to claim 1, further comprising:

In the initial position difference map, the position difference values of pixels whose position difference values are not within the preset parallax range are adjusted to unknown, so that the corresponding pixels are converted into hole pixels.
The method according to claim 1, further comprising:

Perform median filtering on the first image to obtain a first image after removing noise;

Based on the filtered first image, the pixel value continuity of each included pixel point is determined, and the position difference map is supplemented by the position difference value for a second time based on the determined pixel value continuity.
A device for generating a position difference map, which is characterized in that it includes:

The calculation unit performs disparity calculation on the first image and the second image to obtain an initial position difference map; the initial position difference map contains at least one hole pixel with an unknown position difference value;

The dividing unit divides the initial position difference map into super pixels. Each super pixel obtained by the division contains multiple pixel points, and the position difference value pair is based on the position difference value of other pixel points in the super pixel where each hole pixel point is located. The position difference value of the corresponding hole pixel is completed.
A position difference map generation method, characterized in that it is applied to an image processor and includes:

Receive the first image and the second image generated by the image sensor; the first image and the second image are captured by the camera to which the image sensor belongs, aiming at the same subject;

Perform dislocation calculation on the first image and the second image to obtain an initial position difference map, where the initial position difference map contains at least one hole pixel with an unknown position difference value;

The initial position difference map is divided into super pixels. Each super pixel obtained by the division contains multiple pixels, and the corresponding holes are classified based on the position difference values of other pixels in the super pixel where each hole pixel is located. The position difference value of the pixel is completed.
An electronic device, characterized by including:

processor;

Memory used to store instructions executable by the processor;

Wherein, the processor implements the method according to any one of claims 1-10 by running the executable instructions.
A computer-readable storage medium on which computer instructions are stored, characterized in that when the instructions are executed by a processor, the steps of the method according to any one of claims 1-10 are implemented.
A chip characterized by:

Comprising one or more interface circuits and one or more processors; the interface circuit is used to receive signals from a memory of an electronic device and send the signals to the processor, where the signals include computer instructions stored in the memory ; When the processor executes the computer instructions, the electronic device is caused to execute the position difference map generation method described in any one of claims 1 to 10 and claim 12.