CN116485859A - Window determination method and device, electronic equipment and storage medium - Google Patents

Abstract

The present disclosure relates to a method for determining a window, comprising: traversing a preset speckle image through a kth alternative window in the L alternative windows; wherein, the sizes of the alternative windows are different, k and L are positive integers, and k is more than or equal to 1 and less than or equal to L; determining information parameters of speckle in the kth alternative window when the kth alternative window is positioned in different areas in the preset speckle image, wherein the information parameters comprise: average information amount and/or information difference amount; determining a target window from the L candidate windows according to the information parameters; the target window is used for traversing the target speckle image to obtain target depth information.

Description

Window determination method and device, electronic equipment and storage medium

Technical Field

The disclosure relates to the field of information processing, and in particular relates to a method and a device for determining a window, electronic equipment and a storage medium.

Background

With the development of image technology, the quality of the image obtained when the image is acquired by the image acquisition module is higher and higher. More and more terminal devices have an image acquisition function, and images can be generated through an image acquisition module in the terminal device, so that image acquisition is realized. In order to obtain an image with higher quality, the image acquisition module may generate an image with depth information, for example, the image acquisition module performing imaging according to the structured light imaging principle may obtain depth information of a current shooting scene, and the image acquisition module may generate an image with depth information according to the depth information.

Disclosure of Invention

The disclosure provides a window determining method, a window determining device, electronic equipment and a storage medium.

In a first aspect of an embodiment of the present disclosure, a method for determining a window is provided, including: traversing a preset speckle image through a kth alternative window in the L alternative windows; wherein, the sizes of the alternative windows are different, k and L are positive integers, and k is more than or equal to 1 and less than or equal to L; determining information parameters of speckle in the kth alternative window when the kth alternative window is positioned in different areas in the preset speckle image, wherein the information parameters comprise: average information amount and/or information difference amount; determining a target window from the L candidate windows according to the information parameters; the target window is used for traversing the target speckle image to obtain target depth information.

In one embodiment, when traversing the preset speckle image, the kth of the candidate windows traverses N different regions in the preset speckle image; the method comprises the following steps: determining the probability density of the gray level m in the kth alternative window in the ith area according to the gray level of the pixel contained when the kth alternative window is positioned in the ith area; wherein i is more than or equal to 1 and N is more than or equal to N.

In one embodiment, the determining the information parameter of the speckle in the kth candidate window when the kth candidate window is located in a different region in the preset speckle image includes: and determining the average information amount according to the entropy of the probability density of each gray level corresponding to the kth candidate window in the N areas.

In one embodiment, the determining the information parameter of the speckle in the kth candidate window when the kth candidate window is located in a different region in the preset speckle image includes: determining the information difference according to the probability density of the gray level m in the kth alternative window when the kth alternative window is positioned in the ith area and the probability density of the gray level m in the kth alternative window when the kth alternative window is positioned in the jth area; the j-th area is an area except the i-th area in the N areas.

In one embodiment, when the information parameter includes an average information amount and an information difference amount, the determining, according to the information parameter, a target window from L candidate windows includes: determining a first reference value according to the average information quantity and a first weight of the average information quantity; determining a second reference value according to the information difference amount and a second weight of the information difference amount; and determining the target window according to the first reference value and the second reference value.

In one embodiment, the determining the target window according to the first reference value and the second reference value includes: and determining the target window by the candidate window with the largest sum of the first reference value and the second reference value.

In one embodiment, the determining, according to the information parameter, a target window from L candidate windows includes: and determining a target window from L candidate windows according to the number of different areas traversed by the kth candidate window in the target speckle image, the number of pixels contained in the kth candidate window, the average information quantity and the information difference quantity.

In a second aspect of the embodiments of the present disclosure, there is provided a window determining apparatus, including: the traversing module is used for traversing the preset speckle image through the kth alternative window in the L alternative windows; wherein the sizes of the alternative windows are different, k and L are positive integers, and k is smaller than or equal to L; the average information amount determining module is configured to determine information parameters of speckle in the kth candidate window when the kth candidate window is located in a different area in the preset speckle image, where the information parameters include: average information amount and/or information difference amount; the target window determining module is used for determining a target window from L candidate windows according to the information parameters; the target window is used for traversing the target speckle image to obtain target depth information.

In a third aspect of the disclosed embodiments, there is provided an electronic device, including:

a processor and a memory for storing executable instructions capable of executing on the processor, wherein: the processor is configured to execute the executable instructions that, when executed, perform the method of any of the embodiments described above.

In a fourth aspect of the disclosed embodiments, there is provided a non-transitory computer-readable storage medium having stored therein computer-executable instructions that, when executed by a processor, implement the method of any of the above embodiments.

The technical scheme provided by the embodiment of the disclosure can comprise the following beneficial effects:

according to the embodiment of the disclosure, the preset speckle image is traversed through the kth alternative window in the L alternative windows with different sizes, wherein k is the first window to the L window, namely, the preset speckle image is traversed through each window in the L windows respectively. And then determining information parameters of the speckle in the kth alternative window when the kth alternative window is positioned in different areas in the preset speckle image, wherein the information parameters comprise: average amount of information and/or amount of information difference. And determining a target window from the L candidate windows according to the information parameters corresponding to the determined windows, wherein the target window can be used for traversing the target speckle image to obtain target depth information.

The target window is determined according to the information parameters of the candidate window when the preset speckle image is traversed, so that the target window can be utilized to traverse the corresponding target speckle image in the actual shooting scene, the target depth information corresponding to the actual shooting scene is obtained, and the depth image and the like can be conveniently generated according to the depth information. By the method, a more accurate window can be obtained, so that more accurate depth information can be obtained, and the problem of low universality and accuracy caused by manually determining the window according to experience is solved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure.

FIG. 1 is a schematic diagram of a structured light imaging structure, according to an example embodiment;

FIG. 2 is a schematic diagram illustrating a specific application of a window according to an exemplary embodiment;

FIG. 3 is a schematic diagram illustrating one approach to depth information according to an example embodiment;

FIG. 4 is a flow diagram illustrating a method of determining a window according to an exemplary embodiment;

FIG. 5 is a schematic diagram illustrating one determination of a target window according to an example embodiment;

FIG. 6 is a schematic diagram of a window determining apparatus according to an exemplary embodiment;

fig. 7 is a block diagram of a terminal device, according to an example embodiment.

Detailed Description

Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus consistent with some aspects of the disclosure as detailed in the accompanying claims.

In an imaging system, imaging can be performed by a structured light imaging principle to obtain depth information. Referring to fig. 1, a schematic diagram of a structured light imaging structure includes a speckle transmitting end and a speckle receiving end, where the speckle transmitting end may be a speckle transmitter and the speckle receiving end may be an image sensor for image acquisition. The speckle transmitting end transmits speckle to a preset object, and the speckle receiving end receives the speckle reflected by the target object to obtain a speckle reference picture. The preset object may be a whiteboard, which may be an object different from that in the actual photographed scene, for obtaining the speckle reference pattern.

For example, the speckle emitter projects specially encoded speckle patterns onto the surface of the whiteboard, when the speckle patterns are reflected back from the surface of the object, the image sensor can acquire the deformed patterns along with different distances of the object to obtain speckle reference images, the speckle reference images are saved, and the patterns acquired by the image sensor can be used for determining depth information.

Then shooting the shot target object of the actual shooting scene by the method, transmitting the speckle pattern subjected to the special coding to the target object by the speckle transmitter, and acquiring the deformed pattern by the image sensor to obtain a speckle target image.

And then obtaining a depth image according to the speckle reference image and the speckle target image, and calculating the deformation of each pixel in the speckle reference image and the speckle target image through a structured light algorithm to obtain corresponding parallax, thereby further obtaining depth information. And matching the speckle reference image and the speckle target image in a window matching mode to obtain depth image information.

Referring to fig. 2, a schematic view of a specific application of a window is shown in fig. 2 (a), and fig. 2 (b) is another representation of fig. 2 (a). Referring to fig. 3, a schematic diagram of matching a speckle reference map and a speckle target map by means of window matching is shown, so as to obtain depth information. Part (a) of fig. 3 is a speckle reference diagram, and part (b) of fig. 3 is a speckle target diagram. The speckle can be a coded speckle after coding, and the window can decode the speckle target image acquired by the speckle receiver to obtain depth information.

With reference to fig. 2 and 3, after the speckle transmitter transmits random speckle, the speckle receiver acquires a speckle target image, and when the deformation of the very center pixel of the speckle target image is determined, the pixel is used as the center, and the speckle target image is matched with the corresponding position in the speckle target image through a window. For example, the window is an image block with 25×25 pixels, then based on the matching of the window, the pixel areas at the same position in the speckle reference image and the speckle target image, namely the area of the window in the speckle reference image and the area in the speckle target image, are matched through the window, and the parallax of the area where the window is located can be obtained through calculating the offset of the pixel information of the window in the two images, so that the corresponding depth information can be determined by combining the physical parameters of the image acquisition module again according to the triangulation principle.

In this process, the size of the window, i.e. the size of the image block, may affect the disparity and thus may affect the determined depth information. Under the general condition, the size of the window can be determined according to an empirical value or an experimental value, so that the obtained window is inaccurate in size and low in universality, thereby influencing the determination of depth information and reducing the accuracy of the depth information.

Referring to fig. 4, a flow chart of a method for determining a window according to the present disclosure is provided, where the method includes the following steps:

step S100, traversing a preset speckle image through a kth alternative window in L alternative windows; wherein, the sizes of the alternative windows are different, k and L are positive integers, and k is more than or equal to 1 and less than or equal to L.

Step S200, determining information parameters of the speckle in the kth candidate window when the kth candidate window is located in a different area in the preset speckle image, where the information parameters include: average amount of information and/or amount of information difference.

Step S300, determining a target window from L candidate windows according to the information parameters; the target window is used for traversing the target speckle image to obtain target depth information.

For step S100, determining L candidate windows, and then traversing the preset speckle image by using each window in the L windows, so as to obtain information parameters corresponding to each window, and determining a target window from the information parameters. The L windows may be L windows determined from a plurality of windows of different sizes, for example, L candidate windows determined from a history window that has been used before, and the L windows all have different sizes. The size of the window may be represented by pixels, for example, window 1 is 2 x 2 pixels in size, window 2 is 3*3 pixels in size, window 3 is 5*5 pixels in size, and so on.

In one embodiment, the L windows may also be arranged in a sequence to form a sequence of windows, where the windows in the sequence of windows are different in size. For example, set ^W ＝{W ₁ W ₂ ...... W _L }，Set ^W Representing a window sequence comprising L windows, W _L Indicating that the L-th window, i.e. the i-th window and the j-th window are different in size, W can be used _i ≠W _j I+.j. The value of L may be determined according to an actual application scenario, where L may be different in different application scenarios.

Traversing the preset speckle image through the kth alternative window, wherein k is more than or equal to 1 and less than or equal to L, and k is from the first alternative window to the kth alternative window. The kth alternative window may be W _k Indicating that the preset speckle image is traversed through the first window when k is equal to 1, through the second window when k is equal to 2, and through the L-th window when k is equal to L. Traversing the preset speckle image through the first to the L-th alternative windows respectively, and representing each given lower window in the L-th alternative windows by the k-th alternative window.

The preset speckle image may be a speckle image collected by the speckle receiver after the speckle emitter emits speckle to the preset object and reflected by the preset object. The distance between the speckle emitter and the preset object may be x, and the unit of x may be meters, decimeters, centimeters, etc., and the value of x may be determined according to an actual shooting scene, for example, 2 meters, 5 meters, 10 meters, 50 meters, 20 centimeters, 50 centimeters, etc. The preset object may be a whiteboard, or may be other preset objects determined according to actual needs, which is not limited herein.

In one embodiment, the preset speckle images traversed by the L alternative windows are the same preset speckle image, namely, the speckle images which are collected by the speckle receiver and reflected by the preset object after the speckle emitter generates speckle to the same preset object at the same distance x.

In another embodiment, the distance x between the speckle emitter and the preset object may be adjusted after the window of the L candidate windows is updated and/or after the actual usage requirement changes, such as after a shooting scene changes.

In one embodiment, when the candidate window traverses the preset speckle image, the traversal can be performed in a non-overlapping traversal mode, and the area in the preset speckle image covered by the candidate window each time is not overlapped.

When the candidate window traverses the preset speckle image, the candidate window traverses the preset speckle image according to the size of the candidate window, and when the size of the candidate window is different, the number of areas traversed by the candidate window to traverse the preset speckle image may be different. When the preset speckle image is unchanged, the number of different areas in the preset speckle traversed by the larger alternative window when the preset speckle image is traversed may be smaller, and the number of times of movement is smaller. The smaller candidate window may traverse a larger number of different regions in the preset speckle image than the number of times it is moved.

For example, the size of the preset speckle image is 1024×1440 pixels. When traversing the preset speckle image through the first alternative window with the size of 3*3, the number of times the first alternative window moves is U1, namely an area with the size of U1 first alternative windows in the preset speckle image. When traversing the preset speckle image through the second alternative window with the size of 5*5, the number of times the second alternative window moves is U2, namely an area with the size of U2 second alternative windows in the preset speckle image. Where U1 is greater than U2.

For step S200, when the candidate window traverses the preset speckle image, the candidate window needs to be moved in the preset speckle image to cover different areas of the preset speckle image. Taking the kth alternative window as an example for explanation, the information parameter corresponding to each alternative window in the L alternative windows can be determined through the step.

Determining information parameters of speckle in a kth alternative window when the kth alternative window is positioned in different areas in a preset speckle image, wherein the information parameters comprise: average amount of information and/or amount of information difference. When the kth alternative window is positioned in each area in the preset speckle image, a part of pixels in the preset speckle image are covered in the kth alternative window, and the information amount of the speckle in the kth alternative window can be determined according to the pixel information of the preset speckle image, for example, the information can be gray scale information of the pixels. The information amount of the speckle in the kth alternative window can represent the distribution situation of the speckle in the kth alternative window, and can be specifically determined according to the gray scale information of the pixels.

After determining that the kth alternative window is located in each region in the preset speckle image, the average information amount of the speckle in the kth alternative window can be determined after the kth alternative window traverses the preset speckle image. Also, the information difference between the information amounts of the speckle of the kth alternative window can be determined according to the information amounts of the speckle in the kth alternative window when the kth alternative window is positioned in each region in the preset speckle image and the information amounts of the speckle of the kth alternative window when the kth alternative window is positioned in different regions in the preset speckle image.

For example, when k is equal to 1, traversing a first alternative window of size 3*3 pixels by a preset speckle image, wherein the preset speckle image is 9*9 pixels, and the preset speckle image has no overlapping area of the first to ninth areas, and the size of the first alternative window is 9. When the first alternative window traverses the preset speckle image, 9 areas in the preset speckle image need to be traversed, the traversing sequence from the first area to the ninth area is not limited. And determining a first information parameter of speckle in the first alternative window after traversing the first region to the ninth region.

Specifically, the method comprises the following steps: determining a first information amount in the first alternative window when the first alternative window is in the first area, determining a second information amount … … in the first alternative window when the first alternative window is in the second area, determining a ninth information amount in the first alternative window when the first alternative window is in the ninth area, and determining an average information amount of speckle in the first alternative window when the first alternative window traverses the preset speckle image according to the nine information amounts. Meanwhile, the information difference amounts between the corresponding information amounts of the first alternative window at different areas, for example, the difference amounts between the first information amount and the second information amount to the ninth information amount, respectively, may also be determined.

Similarly, for the second of the L alternative windows … …, the L th alternative window, each alternative window may determine a corresponding amount of information.

For step S300, after determining the information parameters corresponding to each candidate window, the target window is determined from the L candidate windows according to the information parameters corresponding to each candidate window. In an actual application scene, the determined target window can be utilized to traverse the target speckle image received by the speckle receiver during shooting, so that corresponding target depth information is obtained.

For example, an alternative window with the largest sum of information parameters is determined, and then the alternative window is taken as a target window. Since the larger the average information amount in the alternative window, the more speckle information in the window is indicated, more speckle information can be obtained through the alternative window. The larger the information difference amount is, the larger the difference of the corresponding information amount is when the alternative window traverses different areas of the preset speckle image, so that the depth information can be better obtained. And combining the target window determined by the average information quantity and the information difference quantity, and obtaining more accurate depth information when traversing the target speckle image.

The target window is determined according to the information parameters of the candidate window when the preset speckle image is traversed, so that the target window can be utilized to traverse the corresponding target speckle image in the actual shooting scene, the target depth information corresponding to the actual shooting scene is obtained, and the depth image and the like can be conveniently generated according to the depth information. By the method, a more accurate window can be obtained, so that more accurate depth information can be obtained, and the problem of low universality and accuracy caused by manually determining the window according to experience is solved.

In another embodiment, the kth alternative window traverses N different regions in the preset speckle image while traversing the preset speckle image;

the method further comprises the steps of: determining the probability density of the gray level m in the kth alternative window in the ith area according to the gray level of the pixel contained when the kth alternative window is positioned in the ith area; wherein i is more than or equal to 1 and N is more than or equal to N.

When traversing the preset speckle image, the kth alternative window needs to traverse N different areas, namely the area with the size of the kth alternative window in the preset speckle image, and the traversing of the preset speckle image can be completed after the kth alternative window traverses the N different areas. For example, there are no overlapping areas in the preset speckle image having a total of 9 first candidate window sizes from the first area to the ninth area. The first candidate window needs to traverse 9 areas in the preset speckle image when traversing the preset speckle image.

When the kth alternative window traverses the ith area, the gray scale information of each pixel in the kth alternative window can be determined, and then the probability density of each gray scale can be determined according to the gray scale information of each pixel in the kth alternative window. For example, the probability density of each gray level within the 1 st candidate window in the 3 rd region is determined based on the gray level of the pixel included when the 1 st candidate window is located in the 3 rd region. In this embodiment, m represents each gray level, and the value of m may be determined according to actual requirements, for example, 0.ltoreq.m.ltoreq.255.

For example, the probability density of a pixel having a gray level of 0 is determined to be the first probability density, the second probability density … … of a pixel having a gray level of 1 is determined to be the second hundred fifty-six probability density of a pixel having a gray level of 255.

This allows determining the probability density of the gray scale of each pixel in the kth candidate window at each of the first through ninth regions.

The specific process of determining the gray-scale information and determining the probability density according to the gray-scale information is not described in detail herein, the gray-scale information may be determined by means of recognition or detection, and the probability density of the gray-scale may be determined by means of a probability density algorithm. The probability density of each gray level can also be determined in the form of a probability density histogram.

In another embodiment, step S200, determining the information parameters of the speckle in the kth candidate window when the kth candidate window is located in a different region in the preset speckle image, includes:

and determining the average information quantity according to the entropy of the probability density of each gray level corresponding to the kth candidate window in the N areas.

This embodiment is an embodiment of determining an average information amount. After determining the probability density of each gray level corresponding to the kth candidate window in each region, the entropy of the probability density of each gray level corresponding to the kth candidate window in each region may be determined, and then the average information amount of the speckle of the kth candidate window in each region may be determined according to the entropy of the probability density of each gray level corresponding to the kth candidate window in each region.

For example, the number of the cells to be processed,indicating that the kth alternative window is located in the ith region of the preset speckle pattern,/and->When the kth alternative window is positioned in the ith area in the preset speckle image, the probability density of the gray level m of the pixels in the kth alternative window can be determined through the formula (1), and when the kth alternative window is positioned in the ith area in the preset speckle image, the entropy of the probability density of the gray level of each pixel in the kth alternative window can be determined.

The entropy of the probability density of the gray scale of each pixel in the kth alternative window is obtained when the kth alternative window is positioned in the ith area in the preset speckle image.

Through the formula (1), when the kth alternative window is positioned in each region in the preset speckle image, the entropy of the probability density of the gray scale of each pixel in the kth alternative window can be determined, and the entropy of the probability density corresponding to N regions respectively. For example, when N is equal to 9, the entropy of probability densities corresponding to 9 regions where i is equal to 1 to i is equal to 9, respectively, may be determined.

And then determining the average information quantity according to the entropy of the probability density of each gray level corresponding to the kth candidate window in the N areas. The average information amount may be an arithmetic average of entropy of probability densities of respective gray scales corresponding to the kth candidate window in the N regions.

For example, the average information amount may be determined by formula (2).

E _k The average information amount of the speckle in the kth candidate window when the kth candidate window is in the N areas, namely the arithmetic average value of the entropy of the probability density of each gray level corresponding to the kth candidate window in the N areas.

And determining the average information quantity corresponding to each candidate window in the L candidate windows from the first candidate window to the L candidate window through the formula (1) and the formula (2).

In another embodiment, step S200, determining the information parameters of the speckle in the kth candidate window when the kth candidate window is located in a different region in the preset speckle image, includes:

and determining the information difference according to the probability density of the gray level m in the kth alternative window when the kth alternative window is positioned in the ith area and the probability density of the gray level m in the kth alternative window when the kth alternative window is positioned in the jth area. The j-th region is a region other than the i-th region among the N regions.

This embodiment is an embodiment of determining the amount of information difference. After determining the probability density of the gray level m in the kth alternative window when the kth alternative window is positioned in the ith area, determining the difference degree of the probability density of the gray level m in the kth alternative window when the kth alternative window is positioned in the ith area and the probability density of the gray level m in the kth alternative window when the kth alternative window is positioned in other areas according to the probability density of the gray level m in the kth alternative window when the kth alternative window is positioned in different areas. And then determining the average difference of the difference corresponding to each gray level according to the difference, and further determining the target difference, namely the information difference when i is different in value according to the average difference.

Of the N regions, regions other than the i-th region may be represented by the j-th region.

For example, the difference between the probability density of the gray level m in the kth candidate window when the kth candidate window is located in the ith area and the probability density of the gray level m in the kth candidate window when the kth candidate window is located in the other areas and the average difference between the differences corresponding to the respective gray levels may be determined according to the formula (3).

Representing a probability density indicating that the gray scale of a pixel within the kth candidate window is m when the kth candidate window is located in the ith region. />Representing the probability density that the gray scale of the pixel in the kth alternative window is m when the kth alternative window is positioned in the jth region. />The difference between the probability density of each gray level of the pixels in the kth alternative window when the kth alternative window is positioned in the ith area and the probability density of each gray level of the pixels in the kth alternative window when the kth alternative window is positioned in the jth area (namely other areas), namely cross entropy, is represented. />The average difference of the difference, i.e., the average value of the cross entropy, is represented.

The amount of information difference can be determined by the formula (4),

the average value of the cross entropy corresponding to the kth candidate window in the N areas is represented, namely the information difference amount.

For example, when i is equal to 1 and n is equal to 9, the average value of the cross entropy of the probability densities of the respective gray scales of the pixels in the kth alternative window in the first region and the probability densities of the respective gray scales of the pixels in the kth alternative window in the other eight regions can be determined by the formula (3). The average value of the cross entropy corresponding to the case where i is equal to 1 to 9 can be determined by the formula (4), and then the information difference amount is determined from the average value of the average values of 9 cross entropies. For example, i is equal to a first average value of the corresponding cross entropy when 1, i is equal to a second average value … … i is equal to 9, and then the information difference amount is determined from the nine average values.

For each candidate window, a corresponding amount of information difference may be determined.

In another embodiment, referring to fig. 5, which is a schematic diagram for determining a target window, when the information parameter includes an average information amount and an information difference amount, step S300 determines, according to the information parameter, the target window from L candidate windows, including:

step S301, determining a first reference value according to a first weight of the average information quantity and the average information quantity;

Step S302, determining a second reference value according to the information difference amount and a second weight of the information difference amount;

step S303, determining a target window according to the first reference value and the second reference value.

The average information quantity and the information difference quantity are respectively provided with a weight, a first reference value is determined according to the first weight of the average information quantity and the average information quantity, a second reference value is determined according to the second weight of the information difference quantity and the information difference quantity, and a target window is determined by combining the first reference value and the second reference value. The first weight of the average information amount and the second weight of the information difference amount may be determined according to actual requirements. For example, an alternative window where the sum of the first reference value and the second reference value is largest may be regarded as the target window.

The target window can be determined from multiple dimensions by combining the average information quantity and the information difference quantity, and the limitation caused by determining the target window according to a single index is reduced, so that the accuracy of determining the target window is improved, and the accuracy of depth information is improved.

In another embodiment, step S300, determining, according to the information parameter, the target window from the L candidate windows includes:

and determining a target window according to the first reference value, and determining the candidate window with the maximum first reference value as the target window.

And determining a target window according to the second reference value, and determining the candidate window with the maximum second reference value as the target window.

In another embodiment, step S300, determining, according to the information parameter, the target window from the L candidate windows includes:

and determining the target window from the L candidate windows according to the number of different areas traversed by the kth candidate window in the target speckle image, the number of pixels contained in the kth candidate window, the average information quantity and the information difference quantity.

And on the basis of the average information quantity and the information difference quantity, determining a target window by combining the number N of areas traversed by the preset speckle image when the candidate window traverses the preset speckle image and the number of pixels in the candidate window when each area. The target window may be determined by combining the average information amount and the information difference amount according to a product of the number N of areas traversed in the preset speckle image and the number of pixels in the candidate window at each area when the candidate window traverses the preset speckle image.

The product of the number of areas N traversed by the preset speckle image and the number of pixels in the preset speckle image at each area can be mapped to a range of 0 to 1 when the preset speckle image is traversed according to the alternative window, and the determination of the target window can be facilitated because the values of the average information amount and the information difference amount are in the range of 0 to 1.

When the size of the preset speckle image is unchanged, the number N of the areas traversed by the preset speckle image when the preset speckle image is traversed by the alternative window and the number of pixels in the alternative window when each area are affected by the operation amount (operation times) for obtaining the target depth information, namely the number N of the areas traversed by the preset speckle image and the number of pixels in the alternative window when the preset speckle image is traversed by the alternative window are affected by the size of the window, so that the size of the window is related to the operation amount, and the efficiency of shooting the image is affected.

Based on the average information quantity and the information difference quantity, the number N of the areas traversed by the preset speckle image when the preset speckle image is traversed by the alternative window and the number of pixels in the alternative window in each area are combined, so that a more matched target window can be comprehensively determined.

When the size of the preset speckle image and the size of the candidate window are known, the number of regions N traversed by the candidate window in traversing the preset speckle image and the number of pixels within the candidate window in each region may be determined.

The amount of computation of traversing the preset speckle image by the kth alternative window can be represented by formula (5):

Represents the kthC, traversing a preset speckle image by using an alternative window to obtain the calculation quantity of the average information quantity and the information difference quantity _k The calculated amount after normalization. />And C _k The mapping relation between the two can be determined according to actual requirements.

According to the embodiment, the operation amount corresponding to each candidate window can be determined.

And determining a third reference value according to the operand and the third weight of the operand, and determining a target window by combining the first reference value, the second reference value and the third reference value. For example, in combination with the sum of the first reference value, the second reference value and the third reference value, the candidate window with the largest sum value is determined as the target window.

For example, the sum of the first reference value, the second reference value, and the third reference value may be determined by formula (6).

S _k ＝w _E *E _k +w _D *D _k +w _c *C _k (6)

w _E Representing the average information quantity E corresponding to the kth alternative window _k W is as follows _D Representing the information difference D corresponding to the kth alternative window _k W is as follows _c Representing the operand C corresponding to the kth alternative window _k Third weight of S _k Representing the sum of the first, second and third reference values obtained through the kth alternative window.

In another embodiment, referring to fig. 6, there is a schematic structural diagram of a window determining apparatus, the apparatus including:

The traversing module 1 is used for traversing the preset speckle image through the kth alternative window in the L alternative windows; wherein the sizes of the alternative windows are different, k and L are positive integers, and k is smaller than or equal to L;

an information parameter determining module 2, configured to determine an information parameter of a speckle in the kth candidate window when the kth candidate window is located in a different area in the preset speckle image, where the information parameter includes: average information amount and/or information difference amount;

a target window determining module 3, configured to determine a target window from L candidate windows according to the information parameter; the target window is used for traversing the target speckle image to obtain target depth information.

In another embodiment, when traversing the preset speckle image, the kth of the candidate windows traverses N different regions in the preset speckle image;

the apparatus further comprises:

the probability density determining module is used for determining the probability density of m of the gray level in the kth alternative window in the ith area according to the gray level of the pixel contained when the kth alternative window is positioned in the ith area; wherein i is more than or equal to 1 and N is more than or equal to N.

In another embodiment, the information parameter determining module 2 is further configured to: and determining the average information amount according to the entropy of the probability density of each gray level corresponding to the kth candidate window in the N areas.

In another embodiment, the information parameter determining module 2 is further configured to: determining the information difference according to the probability density of the gray level m in the kth alternative window when the kth alternative window is positioned in the ith area and the probability density of the gray level m in the kth alternative window when the kth alternative window is positioned in the jth area;

the j-th area is an area except the i-th area in the N areas.

In another embodiment, the target window determining module 3 includes:

a first reference value determining unit configured to determine a first reference value according to the average information amount and a first weight of the average information amount;

a second reference value determining unit, configured to determine a second reference value according to the information difference amount and a second weight of the information difference amount;

and the target window determining unit is used for determining the target window according to the first reference value and the second reference value.

In another embodiment, the target window determining unit is specifically configured to: and determining the target window by the candidate window with the largest sum of the first reference value and the second reference value.

In another embodiment, the target window determination module 3 is further configured to:

and determining a target window from L candidate windows according to the number of different areas traversed by the kth candidate window in the target speckle image, the number of pixels contained in the kth candidate window, the average information quantity and the information difference quantity.

In another embodiment, there is also provided an electronic device including:

a processor and a memory for storing executable instructions capable of executing on the processor, wherein:

the processor is configured to execute the executable instructions that, when executed, perform the method of any of the embodiments described above.

In another embodiment, there is also provided a non-transitory computer readable storage medium having stored therein computer executable instructions that when executed by a processor implement the method of any of the above embodiments.

It should be noted that, the "first" and "second" in the embodiments of the present disclosure are merely for convenience of expression and distinction, and are not otherwise specifically meant.

Fig. 7 is a block diagram of a terminal device, according to an example embodiment. For example, the terminal device may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, an exercise device, a personal digital assistant, and the like.

Referring to fig. 7, the terminal device may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input/output (I/O) interface 812, a sensor component 814, and a communication component 816.

The processing component 802 generally controls overall operation of the terminal device, such as operations associated with presentation, telephone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the methods described above. Further, the processing component 802 can include one or more modules that facilitate interactions between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

The memory 804 is configured to store various types of data to support operations at the terminal device. Examples of such data include instructions for any application or method operating on the terminal device, contact data, phonebook data, messages, pictures, video, etc. The memory 804 may be implemented by any type or combination of volatile or nonvolatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.

The power component 806 provides power to the various components of the terminal device. The power components 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the terminal devices.

The multimedia component 808 includes a screen between the terminal device and the user that provides an output interface. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may sense not only the boundary of a touch or sliding action, but also the duration and pressure associated with the touch or sliding operation. In some embodiments, the multimedia component 808 includes a front camera and/or a rear camera. The front camera and/or the rear camera may receive external multimedia data when the terminal device is in an operation mode, such as a photographing mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive external audio signals when the terminal device is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 further includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be a keyboard, click wheel, buttons, etc. These buttons may include, but are not limited to: homepage button, volume button, start button, and lock button.

The sensor assembly 814 includes one or more sensors for providing status assessment of various aspects for the terminal device. For example, the sensor assembly 814 may detect an on/off state of the terminal device, a relative positioning of the assemblies, such as a display and keypad of the terminal device, the sensor assembly 814 may also detect a change in position of the terminal device or one of the assemblies of the terminal device, the presence or absence of user contact with the terminal device, an orientation or acceleration/deceleration of the terminal device, and a change in temperature of the terminal device. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscopic sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 816 is configured to facilitate communication between the terminal device and other devices, either wired or wireless. The terminal device may access a wireless network based on a communication standard, such as WiFi,4G or 5G, or a combination thereof. In one exemplary embodiment, the communication component 816 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short range communications. For example, the NFC module may be implemented based on a Radio Frequency Identification (RFID) technology, an infrared data association (IrDA) technology, an Ultra Wideband (UWB) technology, a Bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the terminal device may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic elements for executing the methods described above.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any adaptations, uses, or adaptations of the disclosure following the general principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It is to be understood that the present disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims (10)

1. A method of determining a window, comprising:

traversing a preset speckle image through a kth alternative window in the L alternative windows; wherein, the sizes of the alternative windows are different, k and L are positive integers, and k is more than or equal to 1 and less than or equal to L;

determining information parameters of speckle in the kth alternative window when the kth alternative window is positioned in different areas in the preset speckle image, wherein the information parameters comprise: average information amount and/or information difference amount;

determining a target window from the L candidate windows according to the information parameters; the target window is used for traversing the target speckle image to obtain target depth information.

2. The method of claim 1, wherein a kth of the candidate windows traverses N different regions in the preset speckle image when traversing the preset speckle image;

the method comprises the following steps:

Determining the probability density of the gray level m in the kth alternative window in the ith area according to the gray level of the pixel contained when the kth alternative window is positioned in the ith area; wherein i is more than or equal to 1 and N is more than or equal to N.

3. The method of claim 2, wherein determining the information parameter of the speckle in the kth candidate window when the kth candidate window is located in a different region in the preset speckle image comprises:

and determining the average information amount according to the entropy of the probability density of each gray level corresponding to the kth candidate window in the N areas.

4. The method of claim 2, wherein determining the information parameter of the speckle in the kth candidate window when the kth candidate window is located in a different region in the preset speckle image comprises:

determining the information difference according to the probability density of the gray level m in the kth alternative window when the kth alternative window is positioned in the ith area and the probability density of the gray level m in the kth alternative window when the kth alternative window is positioned in the jth area;

the j-th area is an area except the i-th area in the N areas.

5. The method according to claim 1, wherein when the information parameter includes an average information amount and an information difference amount, the determining, according to the information parameter, a target window from L candidate windows includes:

determining a first reference value according to the average information quantity and a first weight of the average information quantity;

determining a second reference value according to the information difference amount and a second weight of the information difference amount;

and determining the target window according to the first reference value and the second reference value.

6. The method of claim 5, wherein said determining said target window based on said first reference value and said second reference value comprises:

and determining the target window by the candidate window with the largest sum of the first reference value and the second reference value.

7. The method of claim 5, wherein determining a target window from the L candidate windows according to the information parameter comprises:

and determining a target window from L candidate windows according to the number of different areas traversed by the kth candidate window in the target speckle image, the number of pixels contained in the kth candidate window, the average information quantity and the information difference quantity.

8. A window determining apparatus, comprising:

the traversing module is used for traversing the preset speckle image through the kth alternative window in the L alternative windows; wherein the sizes of the alternative windows are different, k and L are positive integers, and k is smaller than or equal to L;

the information parameter determining module is configured to determine information parameters of speckle in the kth candidate window when the kth candidate window is located in a different area in the preset speckle image, where the information parameters include: average information amount and/or information difference amount;

the target window determining module is used for determining a target window from L candidate windows according to the information parameters; the target window is used for traversing the target speckle image to obtain target depth information.

9. An electronic device, comprising:

a processor and a memory for storing executable instructions capable of executing on the processor, wherein:

a processor for executing the executable instructions, which when executed perform the method of any of the preceding claims 1 to 7.

10. A non-transitory computer readable storage medium having stored therein computer executable instructions which when executed by a processor implement the method of any one of the preceding claims 1 to 7.