WO2022000454A1 - 图像处理方法、集成电路、装置、可移动平台及存储介质 - Google Patents

图像处理方法、集成电路、装置、可移动平台及存储介质 Download PDF

Info

Publication number
WO2022000454A1
WO2022000454A1 PCT/CN2020/100101 CN2020100101W WO2022000454A1 WO 2022000454 A1 WO2022000454 A1 WO 2022000454A1 CN 2020100101 W CN2020100101 W CN 2020100101W WO 2022000454 A1 WO2022000454 A1 WO 2022000454A1
Authority
WO
WIPO (PCT)
Prior art keywords
block
pixels
cost aggregation
area
aggregation result
Prior art date
Application number
PCT/CN2020/100101
Other languages
English (en)
French (fr)
Inventor
陈琳
李泽祥
高明明
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to PCT/CN2020/100101 priority Critical patent/WO2022000454A1/zh
Priority to CN202080005541.7A priority patent/CN114144765A/zh
Publication of WO2022000454A1 publication Critical patent/WO2022000454A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20021—Dividing image into blocks, subimages or windows

Definitions

  • the present application relates to the technical field of image processing, and in particular, to an image processing method, an integrated circuit, a device, a movable platform and a storage medium.
  • the binocular stereo matching technology uses different viewpoints to framing the same scene to match the corresponding points of the two images obtained, so as to obtain the corresponding point parallax, and then the depth information of the pixels can be calculated.
  • This technology can be applied to the fields of obstacle avoidance and 3D reconstruction of mobile platforms (eg, UAVs and unmanned vehicles).
  • the process of calculating depth information includes: matching cost calculation, cost aggregation, disparity calculation and optimization, and disparity refinement. Among them, for the step of cost aggregation, it takes a long time and the processing frame rate is low. Therefore, it is necessary to propose an improved solution.
  • Embodiments of the present application provide an image processing method, an integrated circuit, a device, a movable platform, and a storage medium, which are used to solve the problem that, for the steps of cost aggregation in the prior art, the processing sequence of pixels is required to be relatively high, which leads to the implementation of The problem is that the method is limited.
  • an embodiment of the present application provides an image processing method, wherein a target image in two images corresponding to different viewpoints is divided into multiple blocks with overlapping partial areas, and a first block in the multiple blocks
  • the overlapping area includes the first overlapping area corresponding to the first direction in the first block; the method includes:
  • a plurality of pixels located in the to-be-aggregated area of the first block are calculated A first cost aggregation result of the first pixel in the first direction.
  • an embodiment of the present application provides an image processing apparatus, in which a target image in two images corresponding to different viewpoints is divided into multiple blocks with overlapping partial areas, and a first block in the multiple blocks
  • the overlapping area includes a first overlapping area corresponding to the first direction in the first block
  • the device includes a memory and a processor
  • the memory for storing program codes
  • the processor calls the program code, and when the program code is executed, is configured to perform the following operations:
  • a plurality of pixels located in the to-be-aggregated area of the first block are calculated A first cost aggregation result of the first pixel in the first direction.
  • an embodiment of the present application provides an integrated circuit, in which a target image in two images corresponding to different viewpoints is divided into multiple blocks whose partial areas overlap, and a first block of the multiple blocks is The overlapping area includes a first overlapping area corresponding to the first direction in the first block; the circuit includes: a calculation unit; the calculation unit is used for:
  • a plurality of pixels located in the to-be-aggregated area of the first block are calculated A first cost aggregation result of the first pixel in the first direction.
  • an embodiment of the present application provides an image processing apparatus, including: the integrated circuit according to any one of the third aspect.
  • an embodiment of the present application provides a movable platform, the movable platform includes a body, a power system disposed on the body, a photographing device, and any one of the second or fourth aspects. the image processing device;
  • the photographing device is used for photographing during the flight of the movable platform to obtain two images corresponding to different viewpoints.
  • an embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and the computer program includes at least a piece of code, and the at least one piece of code can be executed by a computer to control the The computer performs the method according to the first aspect.
  • an embodiment of the present application provides a computer program, which, when the computer program is executed by a computer, is used to implement the method described in any one of the first aspect.
  • Embodiments of the present application provide an image processing method, an integrated circuit, a device, a removable platform, and a storage medium, which reduce the requirements for processing sequences, thereby reducing the limitations on the implementation of cost aggregation.
  • FIG. 1 is a schematic diagram of an application scenario of an image processing method provided by an embodiment of the present application.
  • FIG. 2 is a schematic diagram of 16 cost aggregation directions provided by an embodiment of the present application.
  • FIG. 3 is a schematic diagram of eight cost aggregation directions provided by an embodiment of the present application.
  • FIG. 4A is a schematic flowchart of an image processing method provided by an embodiment of the present application.
  • 4B is a schematic diagram of block division of a target image according to an embodiment of the present application.
  • FIG. 5 is a schematic flowchart of an image processing method provided by another embodiment of the present application.
  • FIG. 6 is a schematic diagram of the principle that the first overlapping area is an upper partial area provided by an embodiment of the present application.
  • FIG. 7 is a schematic diagram of the principle that the first overlapping area is a left partial area provided by an embodiment of the present application.
  • FIG. 8 is a schematic diagram of the principle that the first overlapping area is a lower partial area according to an embodiment of the present application.
  • FIG. 9 is a schematic diagram of the principle that the first overlapping area is a right partial area according to an embodiment of the present application.
  • FIG. 10 is a schematic diagram of a block provided by an embodiment of the present application.
  • FIG. 11 is a schematic diagram of a block provided by another embodiment of the present application.
  • 12A is a schematic diagram of block division of a target image provided by another embodiment of the present application.
  • 12B is a schematic diagram of block division of a target image provided by another embodiment of the present application.
  • FIG. 13 is a schematic diagram of a block provided by another embodiment of the present application.
  • FIG. 14 is a schematic diagram of the principle that some pixels include upper edge pixels according to an embodiment of the present application.
  • FIG. 15 is a schematic diagram of the principle that some pixels include left edge pixels according to an embodiment of the present application.
  • 16 is a schematic diagram of the principle that some pixels include lower edge pixels according to an embodiment of the present application.
  • 17 is a schematic diagram of the principle that some pixels include right edge pixels according to an embodiment of the present application.
  • 18 is a schematic diagram of some pixels in a to-be-aggregated area of the second block according to an embodiment of the application;
  • 19 is a schematic diagram of restoring the unidirectional cost aggregation results of other pixels based on the unidirectional cost aggregation results of some pixels according to an embodiment of the present application;
  • 20 is a schematic diagram of some pixels in a to-be-aggregated area of each block in a target image provided by an embodiment of the present application;
  • 21 is a schematic diagram of some edge pixels in a region to be aggregated in a target image corresponding to each block in a target image according to an embodiment of the present application;
  • FIG. 22 is a schematic flowchart of an image processing method provided by another embodiment of the present application.
  • FIG. 23 is a schematic flowchart of an image processing method provided by another embodiment of the present application.
  • FIG. 24 is a schematic diagram of the processing sequence of blocks by the first and second groups of computing units according to an embodiment of the application.
  • 25 is a schematic flowchart of an image processing method provided by another embodiment of the present application.
  • 26 is a schematic diagram of reading a first cost aggregation result from an internal memory according to an embodiment of the present application
  • 27A is a schematic diagram of accessing a first cost aggregation result in multiple consecutive clock cycles according to an embodiment of the present application
  • 27B is a schematic diagram of accessing a first cost aggregation result in multiple consecutive clock cycles according to another embodiment of the present application.
  • FIG. 28 is a schematic diagram of accessing a first cost aggregation result in consecutive multiple clock cycles according to another embodiment of the present application.
  • 29 is a schematic diagram of the processing sequence of blocks by the first and second groups of computing units according to an embodiment of the application.
  • FIG. 30 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application.
  • FIG. 31 is a schematic structural diagram of a movable platform according to an embodiment of the present application.
  • the image processing method provided in this embodiment of the present application can be applied to the image processing system 10 shown in FIG. 1 , where the image processing system 10 includes a photographing device 11 and an image processing device 12 .
  • the photographing device 11 can capture two images corresponding to different viewpoints.
  • the photographing device 11 can be, for example, a binocular camera.
  • the photographing device 11 can also be other types of devices (for example, a device including multiple cameras). ), which is not limited in this application.
  • the image processing device 12 may obtain two images corresponding to different viewpoints from the photographing device 11, and use the image processing method provided in the embodiment of the present application to process the target image in the two images.
  • the image processing device 12 may include an integrated circuit (Integrated Circuit, IC), and the integrated circuit may be, for example, an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a central processing unit (Central Processing Unit, CPU) and the like.
  • the photographing apparatus 11 and the image processing apparatus 12 may be integrated into the same device, which may be, for example, a movable platform.
  • the movable platform includes at least one of an unmanned aerial vehicle, a gimbal, an unmanned vehicle, and a mobile terminal.
  • the device may also be in other forms, which are not limited in this application.
  • the photographing apparatus 11 and the image processing apparatus 12 may be provided in different devices, for example, the photographing apparatus 11 may be provided in a movable platform, and the image processing apparatus 12 may be provided in a control terminal for controlling the movable platform
  • the photographing device 11 and the image processing device 12 may also be respectively disposed in other equipment, which is not limited in this application.
  • the image processing method provided in the embodiment of the present application is applied to the cost aggregation stage in which the depth information of the image is calculated by using the stereo matching technology, and the target image is the image for which the depth information needs to be calculated among the two images.
  • the main process of calculating image depth information includes:
  • the matching cost is also called the matching cost value.
  • the matching cost represents the similarity between two pixels.
  • Cost aggregation is used to use all pixels in the image corresponding to the entire scene to influence each single pixel to obtain a more accurate result than the matching cost.
  • Parallax calculation and optimization and parallax refinement are used to verify the calculation results and perform sub-pixel processing, etc.
  • the depth information of the target image can be calculated, and then the distance between an object and the photographing device can be further obtained according to the principle of triangulation. This distance is the depth information.
  • the embodiments of the present application specifically improve the method for performing cost aggregation, and the mathematical principle of cost aggregation will be introduced below.
  • C(p, D p ) is the matching cost of a certain pixel, and p is any pixel in the image.
  • the disparity difference is 1, and the range of Np is the adjacent area of pixel p.
  • ] is a penalty term when the disparity variation range in adjacent pixels is large (ie, the disparity difference value is greater than 1).
  • the computational complexity involved in the cost aggregation process is relatively high, and it is a non-deterministic polynomial (NP) problem with polynomial complexity.
  • cost aggregation can be performed from 16 directions, cost aggregation can also be performed from 8 directions, or cost aggregation can also be performed from other number of directions, which is not limited in this embodiment of the present application.
  • the cost aggregation in a certain direction can be achieved by the following formula:
  • the first term C(p, d) in the polynomial is the matching cost of pixel p.
  • the second item is the minimum value among the 4 items of data in the function min(), the 4 items of data include L r (pr,d), L r (pr,d-1)+P 1 , L r (pr,d) +1)+P 1 and min i L r (pr,i)+P 2 .
  • L r (pr, d) represents the cost aggregation result of the previous pixel of pixel p in the r direction under the disparity d.
  • L r (pr, d-1) represents the cost aggregation result of the previous pixel of pixel p in the r direction under the disparity d-1.
  • L r (pr, d+1) represents the cost aggregation result of the previous pixel of pixel p in the r direction under disparity d+1.
  • min i L r (pr,i) represents the minimum cost aggregation result of the previous pixel of pixel p in the r direction under the disparity i, where the value of i ranges from 0 to the maximum disparity value. .
  • P 1 and P 2 are penalty coefficients, and the farther the distance from the pixel p, the larger the penalty coefficient.
  • the third action min k L r (pr, k ) is to prevent L r (p, d) saturated, i.e. as to prevent the iteration L r (p, d) growing too large.
  • the cost aggregation result of a certain pixel p in a certain direction can be calculated (that is, the unidirectional cost aggregation result), and by using this formula repeatedly, the cost aggregation corresponding to a certain pixel p in all directions can be obtained.
  • the cost aggregation results corresponding to a certain pixel p in all directions are added together to obtain the final cost aggregation result of a certain pixel p.
  • the disparity d corresponding to the minimum final cost aggregation result is the final disparity value.
  • the present application proposes a new method for calculating cost aggregation results.
  • the entire target image is divided into multiple blocks with overlapping partial areas, and the overlapping area of the first block in the multiple blocks includes the target overlap. area, the target overlapping area includes the first overlapping area corresponding to the first direction, based on the first overlapping area of the first block and the matching cost of the pixels in the area to be aggregated, it is calculated that the pixels in the first block are in the first block.
  • the cost aggregation result in one direction realizes the calculation based on the matching cost of the pixels in the first block to obtain the first cost aggregation result of the pixels located in the to-be-aggregated area of the first block in the first direction, so that the difference between different blocks is achieved.
  • the calculations of the first cost aggregation results are independent of each other, and cost aggregation can no longer limit the processing order of pixels between different blocks, reducing the requirements for processing order, thereby reducing the restrictions on the implementation of cost aggregation. .
  • the overlapping area of the first block refers to the area in the first block that overlaps with other blocks
  • the area to be aggregated in the first block refers to the area in the first block where the cost aggregation result needs to be calculated
  • the target overlapping area of the block refers to a reference area in the first block that does not need to calculate the cost aggregation result but can be used to calculate the cost aggregation of the area to be aggregated.
  • FIG. 4A is a schematic flowchart of an image processing method provided by an embodiment of the present application, and the execution subject of this embodiment may be the image processing apparatus 12. As shown in FIG. 4A , the method of this embodiment may include:
  • Step 41 Obtain the matching cost of the pixels in the first block of the plurality of blocks.
  • the multiple blocks are multiple blocks overlapped by partial areas obtained by dividing the target image
  • the first block may be any one of the multiple blocks including the target overlapping area block.
  • the block 1 and the block 2 include the target overlapping area, wherein the target overlapping area of the block 1 is the largest in the block 1. Area 1 on the right, and the leftmost Area 2 in Block 2 overlaps with Area 1 in Block 1.
  • the target overlapping area of block 2 is the rightmost area 3 in block 2 , and the leftmost area 4 in block 3 overlaps with area 3 of block 2 . It should be noted that the target overlapping area in FIG. 4B is only an example.
  • the overlapping area between the first block and other blocks may include a target overlapping area and other overlapping areas.
  • the overlapping area of the first block refers to an area of the first block that overlaps with other blocks.
  • the overlapping area of block 2 includes area 2 and area 3 .
  • the target overlapping area of the first block refers to the cost aggregation of a plurality of pixels (hereinafter referred to as the first pixel) in the to-be-aggregated area for calculating the first block in the first block
  • the overlapping area of the results For example, in Figure 4B, Region 3 of the two overlapping regions of Block 2.
  • the to-be-aggregated area of the first block refers to an area in the first block where the cost aggregation result needs to be calculated.
  • the to-be-aggregated area of the first block may specifically be another area in the first block except the target overlapping area, and the to-be-aggregated area includes a non-overlapping area and other overlapping areas.
  • the to-be-aggregated area of block 2 includes area 2 and area 5 of block 2 , where area 2 is another overlapping area of block 2 , and area 5 is a non-overlapping area of block 2 .
  • other overlapping areas of the first block can be used to calculate the cost aggregation results of other blocks. For example, in FIG.
  • the block 2 if the area 2 of the block 2 overlaps the area 1 of the block 1, the block 2 The area 2 of can be used to calculate the cost aggregation result of block 1.
  • region 1 of block 1 can also be used to calculate the cost aggregation result of block 1.
  • the position of the target overlapping area of the first block in the first block corresponds to the direction used for aggregation (for convenience of description, hereinafter referred to as the first direction), the first The target overlapping area of a block includes the first overlapping area corresponding to the first direction in the first block. It can be understood that the first overlapping area in the first block is used for calculating the The overlapping area of the first cost aggregation results in the first direction of the plurality of first pixels in the to-be-aggregated area of the first block.
  • the first direction may be some or all directions among all directions that need to perform cost aggregation, for example, it may be four directions from L0_pd to L3_pd in FIG. 3 , or, for example, it may be L4_pd to L7_pd in FIG. 3 .
  • the first direction may also be in other forms, which are not limited in this application.
  • the first cost aggregation result refers to the result of cost aggregation of pixel matching costs in the first direction. It can be understood that, in the case where the first direction is multiple preset directions, the first cost aggregation result may be the sum of the cost aggregation results in each direction in the multiple first directions (ie, multiple preset directions).
  • the matching cost of the pixels located in the first block can be obtained by calculation, or the matching cost of the pixels located in the first block can be obtained by reading from the memory.
  • the matching cost of the pixels located in the first block can be obtained.
  • the matching cost of the pixels in the first block may be obtained in other ways, which is not limited in this application.
  • the matching cost of the pixel located in the first block includes the matching cost of the pixel located in the first overlapping area of the first block and the matching cost of the pixel located in the to-be-aggregated area of the first block. .
  • the matching cost of the pixels in the first overlapping area is only used to calculate the matching cost of the pixels in the to-be-aggregated area, and is not stored in the first area
  • the calculation process of the cost aggregation corresponding to the block calculates the matching cost of the pixels in the first overlapping area.
  • Step 42 based on the first overlapping area of the first block and the matching cost of the pixels in the to-be-aggregated area of the first block, calculate a plurality of first blocks located in the to-be-aggregated area of the first block.
  • the first block can be regarded as a sub-image, and then the number of pixels located in the to-be-aggregated area of the first block in the first direction can be calculated based on the aforementioned formula 2 for the sub-image. Aggregate results at a cost. Specifically, starting from the edge pixels in the first overlapping area of the first block, the matching cost of the pixels located in the first overlapping area of the first block and the to-be-aggregated area can be calculated sequentially along the first direction. The first cost aggregation result in one direction is to obtain the first cost aggregation result of the plurality of first pixels located in the to-be-aggregated area of the first block in the first direction.
  • the entire target image is divided into multiple blocks with overlapping partial areas, and the overlapping area of the first block in the multiple blocks includes the first overlap corresponding to the first direction.
  • area based on the matching cost of the pixels in the first overlapping area and/or the area to be aggregated in the first block, calculate the cost aggregation result of the pixels in the first block in the first direction, and realize the cost aggregation based on the first block
  • the matching cost of the middle pixel is calculated to obtain the first cost aggregation result of the pixel located in the to-be-aggregated area of the first block in the first direction, so that the calculation of the first cost aggregation result between different blocks is independent of each other, Cost aggregation can no longer limit the processing order of pixels between different blocks.
  • the processing order of pixels is relatively high. , which reduces the requirements for processing order, thereby reducing the restrictions on the implementation of cost aggregation.
  • FIG. 5 is a schematic flowchart of an image processing method provided by another embodiment of the present application. Based on the embodiment shown in FIG. 4A , this embodiment mainly describes that a first direction is one of all directions that need to perform cost aggregation. An alternative implementation for partial orientation cases. As shown in FIG. 5 , the method of this embodiment may include:
  • Step 51 Obtain the matching cost of the pixels located in the first block of the plurality of blocks.
  • step 51 is similar to step 41, and details are not repeated here.
  • Step 52 based on the first overlapping area of the first block and the matching cost of the pixels in the to-be-aggregated area of the first block, calculate and obtain a plurality of first blocks located in the to-be-aggregated area of the first block.
  • the first direction includes a downward direction
  • the first overlapping area includes an upper partial area of the first block.
  • the downward direction may be a vertical downward direction or an oblique downward direction. Taking the eight directions shown in FIG. 3 as an example, the vertical downward direction may be the L1_pd direction in FIG. 3 , and the oblique downward direction may be the L0_pd direction and the L2_pd direction in FIG. 3 .
  • the first overlapping area of the first block may specifically be the upper part area of the first block framed by the black and thick dashed line.
  • the aggregation area may be an area adjacent to the above-mentioned upper partial area in the first block framed by a thick black solid line.
  • the first block shown in FIG. 6 is a block that does not include the upper edge pixels of the target image among the multiple blocks, and the first block shown in FIG. 6 can be, for example, a block Block 4-Block 9 in 12B.
  • the first direction includes a rightward direction
  • the first overlapping area includes a left partial area of the first block.
  • the rightward direction may include a horizontal rightward direction and an oblique rightward direction.
  • the horizontal right direction may be, for example, the L3_pd direction in FIG. 3
  • the oblique right direction may be, for example, the L2_pd direction and the L4_pd direction in FIG. 3 .
  • the first overlapping area of the first block may specifically be the left part area of the first block framed by the black thick dashed line, and the waiting area of the first block Specifically, the aggregation area may be an area adjacent to the above-mentioned left partial area of the first block, which is framed by a thick solid line.
  • the first direction including the L3_pd direction as an example, for a certain row of pixels in the first block, starting from the pixel C0 along the L3_pd direction, the pixels C1, C2, C3, C4, C5, . . . ...C18 cost aggregation results in the L3_pd direction to get the first cost aggregation results for these pixels in the first direction.
  • the first block shown in FIG. 7 is a block that does not include the left edge pixels of the target image among the multiple blocks, and the first block shown in FIG. 7 can be, for example, a block Block 2, Block 3, Block 5, Block 6, Block 8 or Block 9 in 12B.
  • the first direction includes an upward direction
  • the first overlapping region includes a lower partial region of the first block.
  • the upward direction may include a vertical upward direction and an oblique upward direction. Taking the eight directions shown in FIG. 3 as an example, the vertical upward direction may be, for example, the L5_pd direction in FIG. 3
  • the oblique upward direction may be, for example, the L4_pd direction and the L6_pd direction in FIG. 3 .
  • the first overlapping area of the first block may specifically be the lower part of the first block framed by the black and thick dashed line, and the first block to be aggregated Specifically, the area may be an area adjacent to the above-mentioned lower part of the first block, which is framed by a thick solid line.
  • the first direction including the L5_pd direction as an example, for a certain column of pixels in the first block, starting from the pixel E0 along the L5_pd direction, the pixels E1, E2, E3, E4, E5, . . . ...E18 cost aggregation results in the L5_pd direction to get the first cost aggregation results for these pixels in the first direction.
  • the pixels F1, F2, F3 The cost aggregation results of F4, F5, and F7 in the L4_pd direction to obtain the first cost aggregation results of these pixels in the first direction.
  • the first block shown in FIG. 8 is a block that does not include the lower edge pixels of the target image among the multiple blocks, and the first block shown in FIG. 8 can be, for example, a block Block 1-Block 6 in 12B.
  • the first direction includes a leftward direction
  • the first overlapping area includes a right partial area of the first block.
  • the leftward direction may include a horizontal leftward direction and an oblique leftward direction. Taking the eight directions shown in FIG. 3 as an example, the horizontal left direction may be, for example, the L7_pd direction in FIG. 3 , and the oblique left direction may be, for example, the L0_pd direction and the L6_pd direction in FIG. 3 .
  • the first overlapping area of the first block may specifically be the right part area of the first block framed by the black and thick dashed line, and the waiting area of the first block Specifically, the aggregation area may be an area adjacent to the above-mentioned right partial area of the first block framed by a thick solid line.
  • the first direction including the L7_pd direction as an example, for a certain row of pixels in the first block, starting from the pixel G0 along the L7_pd direction, the pixels G1, G2, G3, G4, G5, . . . ...G18 cost aggregation results in the L7_pd direction to get the first cost aggregation results for these pixels in the first direction.
  • the pixels H1, H2, H3 The cost aggregation results of H4 and H5 in the L6_pd direction to obtain the first cost aggregation results of these pixels in the first direction.
  • the first block shown in FIG. 9 is a block that does not include the right edge pixel of the target image among the multiple blocks, and the first block shown in FIG. 9 can be, for example, a block Block 1, Block 2, Block 4, Block 5, Block 7 or Block 8 in 12B.
  • the number of rows or columns occupied by the first overlapping area in FIGS. 6-9 is only an example. It can be understood that, the more rows or columns occupied by the first overlapping area, the greater the amount of calculation, and the more accurate the first cost aggregation result obtained by calculation. The number of rows or columns occupied by the first overlapping area can be flexibly implemented according to requirements.
  • the first overlapping area of the first block may be a black and thick dashed frame.
  • the left part area and the upper part area of the first block, the to-be-aggregated area of the first block may specifically be the area adjacent to the left part area and the upper part area of the first block framed by a thick solid line.
  • the first overlapping area of the first block may be a black thick dashed frame.
  • the right partial area and the lower partial area of the first block, and the to-be-aggregated area of the first block may specifically be the area adjacent to the right partial area and the lower partial area of the first block framed by a thick black solid line.
  • the obtained 4 blocks can be as shown in FIG. 12A .
  • the four blocks are respectively block 1 to block 4, wherein the first block may be block 1 to block 3 of the four blocks.
  • the pixels on the right edge and the lower edge of block 4 in FIG. 12A are the edge pixels of the target image, when the first direction is from L4_pd to L7_pd, the cost aggregation result of the pixels in block 4 will not be affected. The effect of pixels in other blocks, so block 4 does not include the first overlapping area.
  • the target image is equally divided into 4 blocks with a size of 16 ⁇ 16 pixels of the area to be aggregated as an example, and in order to simplify the drawing, only block 1 is shown in FIG. 12A . of the first overlapping area.
  • the area corresponding to block 1 is the area to be aggregated in block 1, and its size is 16 ⁇ 16 pixels.
  • the right side and bottom of the area to be aggregated in block 1 are framed by black and thick dashed lines.
  • the area of is the first overlapping area of block 1.
  • the size of block 1 is (16+3) ⁇ (16+3) pixels.
  • the 3 in the first bracket in the above formula is used to represent the number of columns of pixels in the first overlapping area on the right side of the area to be aggregated in block 1 (the first small square in the figure represents one pixel)
  • the second The 3 in the brackets is used to represent the number of rows of pixels in the first overlapping area below the area to be aggregated in block 1 (the first small square in the figure represents one pixel).
  • the size of the to-be-aggregated area of block 2 is 16 ⁇ 16 pixels
  • the size of block 2 is 16 ⁇ (16+3) pixels, where 3 in parentheses is used to represent the first overlap below block 2 The number of rows of pixels in the region.
  • the size of the to-be-aggregated area of block 3 is 16 ⁇ 16 pixels, and the size of block 3 is (16+3) ⁇ 16 pixels, where 3 in parentheses is used to indicate the first overlapping area on the right side of block 3 The number of columns of pixels.
  • the obtained 9 blocks can be as shown in FIG. 12B .
  • the 9 blocks are respectively block 1 to block 9, wherein the first block may be block 1 to block 8 of the 9 blocks.
  • the cost aggregation result of the pixels in the block 9 will not be affected. The influence of pixels in other blocks, so block 9 does not include the first overlapping area.
  • the cost aggregation result of block 9 is calculated using a plurality of preset values.
  • the target image is equally divided into 9 blocks with a size of 16 ⁇ 16 pixels of the area to be aggregated as an example, and in order to simplify the drawing, only block 1 is shown in FIG. 12B . of the first overlapping area.
  • the area corresponding to block 1 is the area to be aggregated in block 1, and its size is 16 ⁇ 16 pixels.
  • the right side and bottom of the area to be aggregated in block 1 are framed by black and thick dashed lines.
  • the area of is the first overlapping area of block 1, and the size of block 1 is (16+3) ⁇ (16+3) pixels, where the 3 in the first bracket is used to indicate the right side of the area to be aggregated in block 1 The number of columns of pixels in the first overlapping area on the side (the first small square in the figure represents one pixel), and the 3 in the second bracket is used to indicate the pixel in the first overlapping area below the area to be aggregated in block 1 (the first small square in the figure represents a pixel).
  • the size of the to-be-aggregated area of block 2 is 16 ⁇ 16 pixels, and the size of block 2 is (16+3) ⁇ (16+3) pixels, where the first 3 is used to represent the to-be-aggregated area of block 2
  • the number of columns of pixels in the first overlapping area to the right of the aggregation area, and the 3 in the second bracket is used to indicate the number of rows of pixels in the first overlapping area below the area to be aggregated in block 2.
  • the size of the to-be-aggregated area of block 3 is 16 ⁇ 16 pixels, and the size of block 3 is 16 ⁇ (16+3) pixels, where 3 in parentheses is used to represent the pixels in the first overlapping area below block 3 number of rows.
  • the size of the area to be aggregated in block 4 is 16 ⁇ 16 pixels, and the size of block 4 is (16+3) ⁇ (16+3) pixels.
  • the number of columns of pixels in the first overlapping area to the right of the aggregation area, and the 3 in the second bracket is used to indicate the number of rows of pixels in the first overlapping area below the area to be aggregated in block 4.
  • the size of the area to be aggregated in block 5 is 16 ⁇ 16 pixels, and the size of block 5 is (16+3) ⁇ (16+3) pixels.
  • the number of columns of pixels in the first overlapping area to the right of the aggregation area, and the 3 in the second bracket is used to indicate the number of rows of pixels in the first overlapping area below the area to be aggregated in block 5.
  • the size of the to-be-aggregated area of block 6 is 16 ⁇ 16 pixels, and the size of block 6 is 16 ⁇ (16+3) pixels, where 3 in parentheses is used to represent the pixels in the first overlapping area below block 6 number of rows.
  • the size of the to-be-aggregated area of block 7 is 16 ⁇ 16 pixels, and the size of block 7 is (16+3) ⁇ 16 pixels, where 3 in parentheses is used to indicate the first overlapping area on the right side of block 7 The number of columns of pixels.
  • the size of the to-be-aggregated area of block 8 is 16 ⁇ 16 pixels, and the size of block 8 is (16+3) ⁇ 16 pixels, where 3 in parentheses is used to indicate the first overlapping area on the right side of block 8 The number of columns of pixels.
  • Step 53 Calculate and obtain a second cost aggregation result of a plurality of second pixels located in the to-be-aggregated area of the second block of the plurality of blocks in the second direction.
  • the second block may be any block including the target overlapping area among the plurality of blocks.
  • the second block and the first block may be the same block, or the second block and the first block may be different blocks. It should be noted that, there may be no sequence restriction between step 53 and step 52 .
  • the overlapping area between the second block and other blocks may include a target overlapping area and other overlapping areas.
  • the target overlapping area of the second block may refer to a plurality of pixels in the to-be-aggregated area used to calculate the second block in the second block (for ease of description, hereinafter referred to as the second pixel)
  • the other overlapping areas of the second block may refer to the cost aggregation results of multiple pixels in the to-be-aggregated areas of other blocks adjacent to the second block that are used to calculate and obtain.
  • the second block may be any one of block 1 to block 4 among the four blocks in FIG. 12A .
  • the second direction refers to a part of the directions that are different from the first direction among all the directions for which cost aggregation needs to be performed.
  • the first direction and the second direction may be opposite to each other.
  • the first direction is the four directions from L0_pd to L3_pd in FIG. 3
  • the second direction may be the four directions from L4_pd to L7_pd in FIG. 3
  • the first direction may be four directions from L4_pd to L7_pd in FIG. 3
  • the second direction may be four directions from L0_pd to L3_pd in FIG. 3 .
  • the first direction may also be in other forms, which are not limited in this application.
  • the second cost aggregation result refers to the result of cost aggregation of pixel matching costs in the second direction. It can be understood that, in the case where the second direction is multiple preset directions, the second cost aggregation result may be the sum of the cost aggregation results in each direction in the multiple second directions (ie, multiple preset directions).
  • the target overlapping area of the second block may include a second overlapping area corresponding to the second direction in the second block.
  • the calculation to obtain a second cost aggregation result of a plurality of second pixels located in the to-be-aggregated area of the second block in the second direction may specifically include: based on the first cost of the second block The matching cost of the two overlapping areas and the pixels in the to-be-aggregated area of the second block is calculated to obtain the second value of the second pixels in the second direction in the to-be-aggregated area of the second block. Cost aggregation result.
  • the method similar to that described in step 42 or step 52 can also be used to calculate the multiple pixels in the first The way the first cost in the direction aggregates the results.
  • the pixels included in block 5 can be as shown in FIG. 13 shown.
  • the first overlapping area of the block 5 can specifically be the right part area and the lower part area of the block 5 framed by the black thick dashed line
  • the second overlapping area of the block 5 can specifically be the black thick dashed line.
  • the left part area and the upper part area of block 5, and the to-be-aggregated area of block 5 may specifically be the area in the middle of the first block framed by a thick black solid line.
  • the pixel in the area to be aggregated in the first block is calculated and obtained in the first direction.
  • the cost aggregation result is calculated to obtain the second cost aggregation result of multiple second pixels located in the area to be aggregated in the second block in the second direction, and the cost aggregation in different directions for the blocks in the target image is realized , so that the final cost aggregation result of the pixel (ie, the following third cost aggregation result) can be obtained.
  • the calculation obtains the first number of the second pixels located in the to-be-aggregated area of the second block in the second direction.
  • Two-cost aggregation results which can specifically include:
  • a unidirectional cost aggregation result of some pixels in the plurality of second pixels in at least one of the plurality of preset directions is read from the memory, and based on the unidirectional cost aggregation of the partial pixels
  • the direction cost aggregation result is calculated to obtain second cost aggregation results of other pixels in the plurality of second pixels, and the unidirectional cost aggregation results of the partial pixels are obtained through previous processing and stored in the memory.
  • the unidirectional cost aggregation result refers to the cost aggregation result in a single direction.
  • the cost aggregation result of some pixels in the L0_pd direction of the plurality of second pixels may be read from the memory, and the plurality of second pixels may be read from the memory.
  • the cost aggregation result of some pixels in the L1_pd direction may be read from the memory, and the plurality of second pixels may be read from the memory.
  • the cost aggregation result of some pixels in the L1_pd direction are the cost aggregation results in different single directions respectively.
  • the previous processing can be done based on the aforementioned Equation 2. Taking the 8 directions shown in FIG. 3 as an example, assuming that the second direction includes 4 directions in the upper left part in FIG. 3 , namely L0_pd to L3_pd, it is possible to calculate a certain pixel p in the r direction based on the formula 2 given above. Calculate the unidirectional cost aggregation results of all pixels in the target image in the L0_pd to L3_pd directions respectively, and store the unidirectional cost aggregation results of at least one direction in L0_pd to L3_pd of some pixels in the memory. .
  • the unidirectional cost aggregation results of all pixels in the second direction in the to-be-aggregated area of the second block can be calculated and obtained, but some pixels in at least one second direction are selected. Save the unidirectional cost aggregation results on the Afterwards, based on the unidirectional cost aggregation results of some pixels in the to-be-aggregated area of the second block stored in the memory in at least one second direction, it is possible to restore other pixels in the to-be-aggregated area of the second block in each second block.
  • the unidirectional cost aggregation result in the direction (eg, the preset direction) is calculated based on the unidirectional cost aggregation results of other pixels in the second direction (eg, the preset direction) based on the area to be aggregated in the second block
  • the second cost aggregation result of other pixels in the to-be-aggregated area of the second block is calculated based on the unidirectional cost aggregation results of other pixels in the second direction (eg, the preset direction) based on the area to be aggregated in the second block.
  • the memory in order to reduce the occupation of the internal memory of the integrated circuit as much as possible, the memory may be an external memory of the integrated circuit.
  • the memory may also be other memory, which is not limited in this application.
  • a plurality of second pixels located in the to-be-aggregated area of the second block can be obtained by calculation based on the second cost aggregation results of some of the pixels in the multiple second pixels located in the to-be-aggregated area of the second block The theoretical basis for the second cost aggregation results of other pixels in .
  • Equation 2 it can be seen from Equation 2 that for the cost aggregation result of any pixel in the image in a single direction, it is determined by two parts. Specifically, the cost aggregation result includes the matching cost of the pixel and the cost aggregation result of the previous pixel in the direction. Based on this, taking the direction L0_pd as an example, Equation 2 can be rewritten as Equation 3 to describe the process of cost aggregation:
  • i is the current pixel in the image
  • C i represents the matching cost of the current pixel i
  • L0_pd i-1 represents the cost aggregation result of the previous pixel i-1 in the L0_pd direction
  • L0_pd i represents the current pixel i in the L0_pd direction
  • the cost aggregation result i is the current pixel in the image
  • C i represents the matching cost of the current pixel i
  • L0_pd i-1 represents the cost aggregation result of the previous pixel i-1 in the L0_pd direction
  • L0_pd i represents the current pixel i in the L0_pd direction
  • the cost aggregation result is the current pixel in the image
  • C i represents the matching cost of the current pixel i
  • L0_pd i-1 represents the cost aggregation result of the previous pixel i-1 in the L0_pd direction
  • L0_pd i represents
  • the cost aggregation result of a certain pixel p in a certain direction can be determined based on the cost aggregation result of the previous pixel of the pixel p in this direction.
  • the second cost aggregation results of some pixels in the plurality of second pixels in the area are calculated to obtain the second cost aggregation results of other pixels in the plurality of second pixels located in the to-be-aggregated area of the second block. For example, when at least one pixel in the plurality of second pixels is the previous pixel of other pixels in a certain direction, the second cost aggregation result of other pixels may be calculated according to the second cost aggregation result of the at least one pixel.
  • the second cost aggregation result of pixels in the second direction can be expressed as:
  • the second direction may be a plurality of preset directions, and when the plurality of preset directions include a downward direction, the part of the pixels may include an upper edge of the to-be-aggregated area of the second block of pixels.
  • the downward direction may include a vertical downward direction and an oblique downward direction. Taking the eight directions shown in FIG. 3 as an example, the vertical downward direction may be, for example, the L1_pd direction in FIG. 3 , and the oblique downward direction may be, for example, the L0_pd direction and the L2_pd direction in FIG. 3 .
  • the cost aggregation result of the pixel I in the L1_pd direction may be determined based on the cost aggregation result of the previous pixel (ie, the pixel I3) of the pixel I in the L1_pd direction in the L1_pd direction
  • the cost aggregation result of pixel I3 in the L1_pd direction can be determined based on the cost aggregation result of the previous pixel (ie pixel I2) of pixel I3 in the L1_pd direction in the L1_pd direction
  • the cost aggregation result of pixel I2 in the L1_pd direction can be based on
  • the cost aggregation result of the previous pixel in the L1_pd direction of the pixel I2 (ie, the pixel I1) in the L1_pd direction is determined, and the cost aggregation result of the pixel I1 in the L1_pd direction
  • the result of cost aggregation in the L1_pd direction of the pixel I00) on the upper edge of the block is determined. It can be seen that the cost aggregation results of the pixels on the upper edge of the area to be aggregated in the second block in the vertical downward direction can be used to determine the cost aggregation results of other pixels in the vertical downward direction. Therefore, in one embodiment, the cost aggregation result of the pixels in the upper edge row (for example, the row where the pixel I00 is located) of the area to be aggregated in the second block in the L1_pd direction is stored, so as to be used to determine other pixels in the L1_pd direction The cost aggregation result. Among them, other pixels are located inside the first block. That is, other pixels are not on the edge of the block.
  • the cost aggregation result of a certain pixel I in the L2_pd direction may be based on the cost in the L2_pd direction of the previous pixel (ie, pixel I6 ) of the pixel I in the L2_pd direction
  • the aggregation result is determined.
  • the cost aggregation result of pixel I6 in the L2_pd direction can be determined based on the cost aggregation result of the previous pixel (ie, pixel I5) of the pixel I6 in the L2_pd direction in the L2_pd direction, and the cost aggregation of the pixel I5 in the L2_pd direction.
  • the result can be determined based on the cost aggregation result of pixel I5 in the L2_pd direction (ie, pixel I4) in the L2_pd direction, and the cost aggregation result of pixel I4 in the L2_pd direction can be based on the pixel I4's previous pixel in the L2_pd direction. (ie, the pixel I01 on the upper edge of the block) is determined by the cost aggregation result in the L2_pd direction. It can be seen that the cost aggregation results of the pixels on the upper edge of the area to be aggregated in the second block in the oblique downward direction can be used to determine the cost aggregation results of other pixels in the oblique downward direction.
  • the cost aggregation result of the pixels in the upper edge row (for example, the row where the pixel I01 is located) of the area to be aggregated in the second block in the L2_pd direction is stored, so as to be used to determine other pixels in the L2_pd direction
  • the cost aggregation result is stored, so as to be used to determine other pixels in the L2_pd direction.
  • other pixels are located inside the first block. That is, other pixels are not on the edge of the block.
  • the cost aggregation result of the pixels of the upper edge row of the to-be-aggregated area of the second block in the L0_pd direction is stored, so as to be used to determine other pixels in the L0_pd direction
  • other pixels are located inside the first block. That is, other pixels are not on the edge of the block.
  • the second direction may be a plurality of preset directions, and when the plurality of preset directions include a rightward direction, the part of the pixels may include a leftward position in the to-be-aggregated area of the second block. edge pixels.
  • the rightward direction may include a horizontal rightward direction and an oblique rightward direction. Taking the eight directions shown in FIG. 3 as an example, the horizontal right direction may be, for example, the L3_pd direction in FIG. 3 , and the oblique right direction may be, for example, the L2_pd direction and the L4_pd direction in FIG. 3 .
  • the cost aggregation result of pixel J in the L3_pd direction may be determined based on the cost aggregation result of the previous pixel (ie, pixel J3) of the pixel J in the L3_pd direction in the L3_pd direction
  • the cost aggregation result of pixel J3 in the L3_pd direction can be determined based on the cost aggregation result of the previous pixel of pixel J3 in the L3_pd direction (ie, pixel J2) in the L3_pd direction
  • the cost aggregation result of pixel J2 in the L3_pd direction can be based on
  • the cost aggregation result of the pixel J2 in the L3_pd direction (ie, pixel J1) in the L3_pd direction is determined
  • the cost aggregation result of the pixel J1 in the L3_pd direction can be based on the pixel J1 in the
  • the result of cost aggregation in the L3_pd direction of the pixel J00) at the left edge of the block is determined. It can be seen that the cost aggregation result of the pixels on the left edge of the area to be aggregated in the second block in the horizontal right direction can be used to determine the cost aggregation results of other pixels in the horizontal right direction. Therefore, in one embodiment, the cost aggregation result of the pixels in the left edge row of the area to be aggregated in the second block (for example, the row where the pixel J00 is located) in the L3_pd direction is stored for determining other pixels in the L3_pd direction. The cost aggregation result. Among them, other pixels are located inside the first block. That is, other pixels are not on the edge of the block.
  • the cost aggregation result of a certain pixel J in the L2_pd direction may be based on the cost in the L2_pd direction of the previous pixel (ie, pixel J6 ) of the pixel J in the L2_pd direction
  • the aggregation result is determined.
  • the cost aggregation result of pixel J6 in the L2_pd direction can be determined based on the cost aggregation result of the previous pixel (ie, pixel J5) of pixel J6 in the L2_pd direction in the L2_pd direction, and the cost aggregation of pixel J5 in the L2_pd direction.
  • the result can be determined based on the cost aggregation result of pixel J5 in the L2_pd direction of the previous pixel (ie, pixel J4) in the L2_pd direction, and the cost aggregation result of pixel J4 in the L2_pd direction can be based on the previous pixel of pixel J4 in the L2_pd direction. (ie, the pixel J01 at the left edge of the block) is determined by the cost aggregation result in the L2_pd direction. It can be seen that the cost aggregation result of the pixels on the left edge of the area to be aggregated in the second block in the diagonal right direction can be used to determine the cost aggregation results of other pixels in the diagonal right direction.
  • the cost aggregation result of the pixels in the left edge row of the area to be aggregated in the second block (for example, the row where the pixel J01 is located) in the L2_pd direction is stored, so as to be used to determine other pixels in the L2_pd direction
  • the cost aggregation result is stored, so as to be used to determine other pixels in the L2_pd direction.
  • other pixels are located inside the first block. That is, other pixels are not on the edge of the block.
  • the cost aggregation result of the pixels of the left edge row of the area to be aggregated in the second block in the L4_pd direction is stored to be used to determine other pixels in the L4_pd direction.
  • other pixels are located inside the first block. That is, other pixels are not on the edge of the block.
  • the second direction may be a plurality of preset directions, and when the plurality of preset directions include an upward direction, the part of the pixels may include a lower edge of the to-be-aggregated area of the second block of pixels.
  • the upward direction may include a vertical upward direction and an oblique upward direction. Taking the eight directions shown in FIG. 3 as an example, the vertical upward direction may be, for example, the L5_pd direction in FIG. 3 , and the oblique upward direction may be, for example, the L4_pd direction and the L6_pd direction in FIG. 3 .
  • the cost aggregation result of the pixel K in the L5_pd direction may be determined based on the cost aggregation result of the previous pixel (ie, the pixel K3) of the pixel K in the L5_pd direction in the L5_pd direction
  • the cost aggregation result of pixel K3 in the L5_pd direction can be determined based on the cost aggregation result of the previous pixel of pixel K3 in the L5_pd direction (ie pixel K2) in the L5_pd direction
  • the cost aggregation result of pixel K2 in the L5_pd direction can be based on
  • the cost aggregation result of the previous pixel (ie pixel K1) in the L5_pd direction of the pixel K2 in the L5_pd direction is determined, and the cost aggregation result of the pixel K1 in the L5_pd direction can be
  • the result of cost aggregation in the L5_pd direction of the pixel K00) at the lower edge of the block is determined. It can be seen that the cost aggregation results of the pixels at the lower edge of the area to be aggregated in the second block in the vertical upward direction can be used to determine the cost aggregation results of other pixels in the vertical upward direction. Therefore, in one embodiment, the cost aggregation result of the pixels in the lower edge row of the area to be aggregated in the second block (for example, the row where the pixel K00 is located) in the L5_pd direction is stored for determining other pixels in the L5_pd direction. The cost aggregation result. Among them, other pixels are located inside the first block. That is, other pixels are not on the edge of the block.
  • the cost aggregation result of the pixel K in the L4_pd direction may be based on the cost aggregation result of the previous pixel (ie, the pixel K6 ) in the L4_pd direction of the pixel K in the L4_pd direction
  • the cost aggregation result of the pixel K6 in the L4_pd direction can be determined based on the cost aggregation result of the pixel K6 in the L4_pd direction (ie, the pixel K5) in the L4_pd direction
  • the cost aggregation result of the pixel K5 in the L4_pd direction can be It is determined based on the cost aggregation result of the previous pixel in the L4_pd direction of the pixel K5 (ie, the pixel K4) in the L4_pd direction, and the cost aggregation result of the pixel K4 in the L4
  • the previous pixel in the L4_pd direction i.e. The cost aggregation result of the pixel K01
  • the cost aggregation results of the pixels at the lower edge of the area to be aggregated in the second block in the oblique upward direction can be used to determine the cost aggregation results of other pixels in the oblique upward direction.
  • the cost aggregation result of the pixels in the lower edge row (for example, the row where the pixel K01 is located) of the area to be aggregated in the second block in the L4_pd direction is stored, so as to be used to determine other pixels in the L4_pd direction
  • the cost aggregation result is stored, so as to be used to determine other pixels in the L4_pd direction.
  • other pixels are located inside the first block. That is, other pixels are not on the edge of the block.
  • the cost aggregation result of the pixels in the L6_pd direction of the pixels in the lower edge row of the area to be aggregated in the second block is stored to be used to determine other pixels in the L6_pd direction.
  • other pixels are located inside the first block. That is, other pixels are not on the edge of the block.
  • the second direction may be a plurality of preset directions, and when the plurality of preset directions include a left direction, the part of the pixels including the second block is located at the right edge of the to-be-aggregated area of pixels.
  • the leftward direction may include a horizontal leftward direction and an oblique leftward direction. Taking the eight directions shown in FIG. 3 as an example, the horizontal left direction may be, for example, the L7_pd direction in FIG. 3 , and the oblique left direction may be, for example, the L0_pd direction and the L6_pd direction in FIG. 3 .
  • the cost aggregation result of a certain pixel L in the L7_pd direction may be based on the cost aggregation of the previous pixel (ie, pixel L3 ) of the pixel L in the L7_pd direction in the L7_pd direction.
  • the cost aggregation result of pixel L3 in the L7_pd direction can be determined based on the cost aggregation result of the previous pixel (ie pixel L2) of the pixel L3 in the L7_pd direction in the L7_pd direction, and the cost aggregation result of the pixel L2 in the L7_pd direction
  • the previous pixel in the L7_pd direction ( That is, the cost aggregation result of the pixel L00) on the right edge of the block in the direction of L7_pd is determined. It can be seen that the cost aggregation result of the pixels on the right edge of the area to be aggregated in the second block in the horizontal left direction can be used to determine the cost aggregation results of other pixels in the horizontal left direction. Therefore, in one embodiment, the cost aggregation result of the pixels in the right edge row of the area to be aggregated in the second block (for example, the row where the pixel L00 is located) in the L7_pd direction is stored to be used to determine other pixels in the L7_pd direction. The cost aggregation result. Among them, other pixels are located inside the first block. That is, other pixels are not on the edge of the block.
  • the cost aggregation result of a certain pixel L in the L6_pd direction may be based on the cost in the L6_pd direction of the previous pixel (ie, pixel L6 ) of the pixel L in the L6_pd direction
  • the aggregation result is determined.
  • the cost aggregation result of pixel L6 in the L6_pd direction can be determined based on the cost aggregation result of the previous pixel (ie, pixel L5) of the pixel L6 in the L6_pd direction in the L6_pd direction, and the cost aggregation of the pixel L5 in the L6_pd direction.
  • the result can be determined based on the cost aggregation result of pixel L5 in the L6_pd direction of the previous pixel (ie, pixel L4) in the L6_pd direction, and the cost aggregation result of pixel L4 in the L6_pd direction can be based on the pixel L4
  • the previous pixel in the L6_pd direction ie, the pixel L01 on the right edge of the block
  • the cost aggregation result of the pixels on the right edge of the area to be aggregated in the second block in the oblique left direction can be used to determine the cost aggregation results of other pixels in the oblique left direction.
  • the cost aggregation result of the pixels in the right edge row of the area to be aggregated in the second block (for example, the row where the pixel L01 is located) in the L6_pd direction is stored for determining other pixels in the L6_pd direction.
  • the cost aggregation result is located inside the first block. That is, other pixels are not on the edge of the block.
  • the cost aggregation result of the pixels in the right edge row of the area to be aggregated in the second block in the L0_pd direction is stored, so as to be used to determine other pixels in the L0_pd direction
  • other pixels are located inside the first block. That is, other pixels are not on the edge of the block.
  • the second direction is L0_pd to L3_pd in the upper left part of FIG. 3 as an example, as shown in FIG.
  • the partial pixels of the direction cost aggregation result may specifically include pixels located at the upper edge of the area to be aggregated in the second block indicated by a black box, and pixels located at the left edge of the area to be aggregated in the second block indicated by a gray box.
  • Other pixels in the to-be-aggregated area of the second block may specifically be pixels indicated by white boxes.
  • the different filling colors of the pixels in FIG. 18 indicate that the directions corresponding to the cost aggregation results that need to be stored in the memory are different.
  • the storage device needs to store the cost aggregation results in the three directions of L0_pd, L1_pd, and L2_pd respectively; for the gray frame in Figure 18
  • the pixel at the position is the pixel at the left edge of the area to be aggregated in the second block, and the storage device needs to store the cost aggregation results in the two directions of L2_pd and L3_pd respectively.
  • the pixel at the overlapping position of the gray box and the black box in Figure 18 i.e.
  • the storage device needs to store the cost aggregation results stored in the four directions of L0_pd, L1_pd, L2_pd and L3_pd respectively.
  • the pixel at the overlapping position of the gray frame and the black frame is the pixel at the upper edge and the left edge at the same time.
  • the unidirectional cost aggregation result in each second direction of other pixels in the to-be-aggregated area of the second block may be determined in a line-by-line or point-by-point manner. It should be noted that, in one embodiment, when the second direction is L0_pd ⁇ L7_pd, a method similar to the above-mentioned method may be used to obtain the cost aggregation result of the pixels located in the block in the second direction. For the sake of brevity, no further description is given.
  • the result of unidirectional cost aggregation in at least one second direction of the pixels on the upper edge and the left edge of the area to be aggregated in the second block can be performed in parallel row by row and column by row.
  • the unidirectional cost aggregation result in each second direction of other pixels in the to-be-aggregated area of the second block is determined in the manner of .
  • the unidirectional cost matching results of the pixels in the first row (1901) in the L0_pd, L1_pd, and L2_pd directions, respectively, and the matching cost of the pixels in the second row (1902) calculate the unidirectional matching cost results of the pixels in the second row in the L0_pd, L1_pd, and L2_pd directions respectively.
  • the unidirectional cost aggregation result of the pixels in the second column in the L3_pd direction can also be calculated from the unidirectional cost aggregation results of the pixels in the 1st column in the L3_pd direction and the matching cost of the pixels in the 2nd column.
  • the calculation processes of the unidirectional cost aggregation results in different directions from L0_pd to L3_pd are independent of each other, so the unidirectional cost aggregation results in different directions can be calculated in parallel.
  • the recovery of the unidirectional cost aggregation results of pixels in row 2 in the L0_pd, L1_pd, and L2_pd directions and the unidirectional cost aggregation results of pixels in column 2 in the L3_pd direction can be completed within one clock cycle.
  • the recovery calculation of the unidirectional cost aggregation results in each second direction of other pixels in the to-be-aggregated area of the second block can be completed in less time. .
  • the calculation speed is faster.
  • the point-by-point method when adopted, it can be determined in a point-by-point manner according to the unidirectional cost aggregation results of the pixels on the upper edge and the left edge of the to-be-aggregated area of the second block in at least one second direction. Unidirectional cost aggregation results of other pixels in the to-be-aggregated area of the second block in each second direction.
  • the unidirectional cost of the pixel at position (1,0) in the L3_pd direction can be calculated first.
  • the matching result calculates the unidirectional cost matching result of the pixel at position (1,1) in the L3_pd direction, and calculates the position through the unidirectional cost matching result of the pixel at position (0,1) in the L0_pd, L1_pd, and L2_pd directions respectively.
  • the addition result can be stored in the internal memory of the integrated circuit.
  • the addition result of the unidirectional cost matching results of some pixels and other pixels in the L0_pd to L3_pd direction in the to-be-aggregated area of the second block ie, the second cost aggregation result
  • the second cost aggregation result can finally be calculated, and the The two-cost aggregation results are stored in memory. Still taking the to-be-aggregated area of the second block shown in FIG.
  • one block of 16 ⁇ 16 ⁇ 16 can be opened in the internal memory of the integrated circuit Bit storage space for storing the second cost aggregation result of the second pixel in the to-be-aggregated area of the second block.
  • the point-by-point restoration method can directly restore the unidirectional cost matching results of a single second pixel in the L0_pd to L3_pd directions, so The unidirectional cost matching results in the four directions can be directly added to obtain a second cost aggregation result of a single second pixel, and the second cost aggregation result is stored in the internal memory of the integrated circuit.
  • the line-by-line restoration method it is impossible to restore all the unidirectional cost matching results of a single second pixel in the L0_pd to L3_pd directions at the same time.
  • the cost aggregation result of the pixels on the upper edge of the area to be aggregated in the second block in the downward direction is the default maximum value, so it is not necessary to store it in the storage device.
  • the cost aggregation result of the pixels at the left edge of the area to be aggregated in the second block in the right direction is the default maximum value , so it can be stored without needing to be stored in the storage device.
  • the result of cost aggregation in the upward direction of the pixels of the lower edge of the area to be aggregated in the second block is the default maximum value.
  • the cost aggregation result of the pixels on the right edge of the area to be aggregated in the second block in the left direction is the maximum by default value.
  • the above-mentioned default maximum value may not be stored in the same storage manner as the calculated cost aggregation result. For example, there is no need to store the default maximum value in the storage device corresponding to the calculated cost aggregation result.
  • the to-be-aggregated areas of the 9 blocks shown in FIG. 12B form a complete target image, and the second direction includes L0-pd to L3-pd, And the second block is any one of blocks 1 to 9 in FIG. 12B , then some of the pixels in the to-be-aggregated area of the second block may be the pixels in the black frame position and the pixels in the gray frame position in FIG. 20 . Referring to FIG.
  • the second direction includes L0-pd to L3-pd
  • some pixels in the to-be-aggregated area of the second block are pixels on the upper edge and left edge of the to-be-aggregated area of the second block
  • the unidirectional cost aggregation results of partial pixels in the L0-pd to L3-pd directions, respectively, should be stored in memory.
  • the pixels on the left edge of the areas to be aggregated in block 1, block 4, and block 7 are all pixels on the edge of the target image, the pixels on the left edge of the areas to be aggregated in blocks 1, 4, and 7 are The unidirectional cost aggregation results of pixels in the L0-pd to L3-pd directions can directly take the default value.
  • the pixels on the upper edge of the areas to be aggregated in block 1, block 2 and block 3 are all pixels on the edge of the target image, the pixels on the upper edge of the areas to be aggregated in blocks 1, 2 and 3 are respectively The unidirectional cost aggregation result in the L0-pd to L3-pd direction can directly take the default value.
  • the pixels on the left edge and the upper edge of the area to be aggregated in block 5, block 6, block 8 and block 9 are not the pixels of the edge of the target image, block 5, block 6, block 8
  • the unidirectional cost aggregation results of the pixels on the upper edge and the left edge of the area to be aggregated in block 9 respectively in the L0-pd to L3-pd directions cannot directly take the default value.
  • the target image is equally divided into 9 blocks with a size of 16 ⁇ 16 pixels of the area to be aggregated as an example.
  • the image processing method provided by the embodiment of the present application, by reading the single image of some pixels in at least one preset direction among the multiple preset directions of the multiple second pixels in the to-be-aggregated area of the second block from the memory
  • the direction cost aggregation result, and the second cost aggregation result of other pixels in the plurality of second pixels is calculated based on the unidirectional cost aggregation result of the part of the pixels, which realizes that some pixels in the area to be aggregated based on a block are at least
  • the unidirectional cost aggregation result in one direction is calculated to obtain the first cost aggregation result of other pixels in the to-be-aggregated area of the block.
  • the above method expands the range of methods for calculating the second cost aggregation result of pixels, and improves the calculation of the second cost aggregation result of pixels. flexibility.
  • the first cost aggregation result and the second cost aggregation result corresponding to each of the multiple blocks may be calculated according to a predetermined processing sequence, that is, there is a predetermined processing sequence among the multiple blocks , the second direction is a plurality of preset directions; the calculation obtains the second cost aggregation result of the plurality of second pixels located in the to-be-aggregated area of the second block of the plurality of blocks in the second direction, Specifically, it can include:
  • the second cost aggregation result refers to the result of cost aggregation of pixel matching costs in the second direction. It can be understood that, in the case where the second direction is a plurality of preset directions, the second cost aggregation result may be the sum of the unidirectional cost aggregation results in each direction among the plurality of preset directions. Taking the second direction as L4_pd to L7_pd shown in FIG. 3 as an example, the second cost aggregation result of a pixel in the to-be-aggregated area of the target block may be the unidirectional cost aggregation of the pixel at L4_pd, L5_pd, L6_pd and L7_pd respectively the sum of the results.
  • the target block is located before the second block, and the to-be-aggregated area of the target block is adjacent to the to-be-aggregated area of the second block.
  • the processing sequence of the nine blocks shown in 12B may be, for example, block 1 ⁇ block 2 ⁇ block 3 ⁇ block 4 ⁇ block 5 ⁇ block 6 ⁇ block 7 ⁇ block 8 ⁇ block 9.
  • the target block of the second block may be the block 1 .
  • the target blocks of the second block may be the block 2 and the block 4.
  • the plurality of preset directions include an upward direction
  • the partial edge pixels include pixels located at the lower edge of the to-be-aggregated area of the target block. pixel.
  • the second direction is a plurality of preset directions
  • the plurality of preset directions include a downward direction
  • the part of the edge pixels includes the upper part of the target block in the to-be-aggregated area edge pixels.
  • the plurality of preset directions include a leftward direction
  • the partial edge pixels include the target block in the area to be aggregated located on the right edge pixels.
  • the plurality of preset directions include a rightward direction, and the part of the edge pixels including the area to be aggregated in the target block are located on the left side edge pixels.
  • the processing sequence of 9 blocks can be block 1 ⁇ block 2 ⁇ block 3 ⁇ block 4 ⁇ block 5 ⁇ block 6 ⁇ block 7 ⁇ block 8 ⁇ block 9, then the second block
  • the target block of the block may be block 1-block 8 in FIG. 21 , and some edge pixels in the target block may be the pixels at the position of the black frame and the pixel at the position of the gray frame in FIG. 21 .
  • the unidirectional cost aggregation results of the right edge pixels on L2-pd and L3-pd in the to-be-aggregated area of block 1 can be used to calculate the second cost aggregation result of the pixels in the to-be-aggregated area of block 2
  • the unidirectional cost aggregation results of the lower edge pixels on L0-pd, L1-pd, and L2-pd in the to-be-aggregated area of block 1 can be used to calculate the second cost aggregation result of the pixels in the to-be-aggregated area of block 4.
  • the unidirectional cost aggregation results of the right edge pixels on L2-pd and L3-pd in the to-be-aggregated area of block 2 can be used to calculate the second cost aggregation result of the pixels in the to-be-aggregated area of block 3.
  • the unidirectional cost aggregation results of the lower edge pixels in the to-be-aggregated area on L0-pd, L1-pd, and L2-pd can be used to calculate the second cost aggregation result of the pixels in the to-be-aggregated area of block 5.
  • the unidirectional cost aggregation results of the lower edge pixels on L0-pd, L1-pd, and L2-pd in the to-be-aggregated area of block 3 can be used to calculate the second cost aggregation result of the pixels in the to-be-aggregated area of block 6.
  • the unidirectional cost aggregation results of the right edge pixels on L2-pd and L3-pd in the to-be-aggregated area of block 4 can be used to calculate the second cost aggregation result of the pixels in the to-be-aggregated area of block 5.
  • the unidirectional cost aggregation results of the lower edge pixels in the to-be-aggregated area on L0-pd, L1-pd, and L2-pd can be used to calculate the second cost aggregation result of the pixels in the to-be-aggregated area of block 7.
  • the unidirectional cost aggregation results of the right edge pixels on L2-pd and L3-pd in the to-be-aggregated area of block 5 can be used to calculate the second cost aggregation result of the pixels in the to-be-aggregated area of block 6.
  • the unidirectional cost aggregation result of the lower edge pixels in the area to be aggregated on L0-pd, L1-pd, and L2-pd can be used to calculate the second cost aggregation result of the pixels in the area to be aggregated in block 8.
  • the unidirectional cost aggregation results of the lower edge pixels on L0-pd, L1-pd, and L2-pd in the to-be-aggregated area of block 6 can be used to calculate the second cost aggregation result of the pixels in the to-be-aggregated area of block 9.
  • the unidirectional cost aggregation result of the right edge pixels in the area to be aggregated in block 7 on L2-pd and L3-pd can be used to calculate the second cost aggregation result of the pixels in the area to be aggregated in block 8.
  • the unidirectional cost aggregation result of the right edge pixels in the area to be aggregated in block 8 on L2-pd and L3-pd can be used to calculate the second cost aggregation result of the pixels in the area to be aggregated in block 9.
  • block 1 when block 1 is used as the second block, there are no other blocks before block 1 in the preset processing sequence, so the target block of block 1 is empty; block 2 is used as the second block Block, the preset processing sequence is located before block 2 and the area to be aggregated adjacent to the area to be aggregated of block 2 is block 1, so block 1 is the target block of block 2;
  • block 3 when block 3 is used as the second block, the block that is located before block 3 in the preset processing sequence and whose area to be aggregated is adjacent to the area to be aggregated of block 3 is block 2, so block 2 is block 3
  • block 5 when block 5 is used as the second block, in the preset processing sequence, the block that is located before block 5 and the area to be aggregated is adjacent to the area to be
  • the blocks that are located before block 8 in the preset processing sequence and whose area to be aggregated is adjacent to the area to be aggregated of block 8 are block 5 and block 7, so block 5 and block 7 is the target block of block 8;
  • block 9 is used as the second block, the block that is located before block 9 in the preset processing sequence and the area to be aggregated adjacent to the area to be aggregated of block 9 is the block 6 and block 8, so block 6 and block 8 are the target blocks of block 9.
  • the target image is equally divided into 9 blocks with a size of 16 ⁇ 16 pixels of the area to be aggregated as an example.
  • the target block includes a first target block that is adjacent to the second block in the predetermined processing sequence; the first target block
  • the unidirectional cost aggregation results of some edge pixels in the region to be aggregated in at least one of the second directions are stored in the internal memory of the integrated circuit.
  • the unidirectional cost aggregation result of some edge pixels in at least one of the second directions is calculated by the calculation unit in the integrated circuit.
  • the first target block may be the block 4 .
  • the block (ie the second block) needs to use the unidirectional cost aggregation results of some edge pixels in the to-be-aggregated area of the first target block in at least one second direction.
  • the unidirectional cost aggregation results of some edge pixels in the region in at least one second direction are stored in the internal memory, which facilitates data acquisition, reduces the occupation of bandwidth between the integrated circuit and the external memory, and reduces processing time.
  • the target block includes a second target block, the second target block is spaced from the second block in the predetermined processing sequence; the second target block is The unidirectional cost aggregation results of some edge pixels in the to-be-aggregated area in at least one of the second directions are stored in an external memory of the integrated circuit. Wherein, the unidirectional cost aggregation result of some edge pixels in at least one of the second directions is calculated by the calculation unit in the integrated circuit. Taking the second block as the block 5 in FIG. 12B as an example, the second target block may be the block 2 .
  • the second target block is spaced from the second block in the predetermined processing sequence, that is, it is not adjacent to the second block, that is, after calculating the second cost aggregation result of the second target block, after the second target block is In the next block adjacent to the two target blocks (that is, blocks other than the second block), it is not necessary to use some edge pixels in the to-be-aggregated area of the second target block in at least one second direction
  • the occupation of the internal memory can be reduced.
  • the result of unidirectional cost aggregation based on the partial edge pixels in the to-be-aggregated area of the target block in the multiple blocks in at least one preset direction among the multiple preset directions The second cost aggregation result of multiple second pixels located in the to-be-aggregated area of the second block in the second direction is obtained by calculating, so that in the process of calculating the second aggregation cost, the The unidirectional cost aggregation results of some edge pixels in the aggregation area in at least one preset direction, and the second cost aggregation results of the pixels in the to-be-aggregated areas of other blocks in the second direction are calculated, and the previous processing needs to be performed to obtain the results.
  • the unidirectional cost aggregation result of some pixels in at least one preset direction in the area to be aggregated in the second block so as to restore the unidirectional cost aggregation of other pixels in at least one preset direction in the area to be aggregated in the second block
  • the second cost aggregation result of the pixels in the second direction in the to-be-aggregated area of the second block is calculated, which saves the previous processing, reduces the amount of calculation, reduces the processing time, and improves the processing frame. Rate.
  • FIG. 22 is a schematic flowchart of an image processing method provided by another embodiment of the present application. On the basis of the previous embodiments of this embodiment, it is mainly described that an integrated unit in an integrated circuit is used to perform an image processing of the pixels in the to-be-aggregated area of the first block. An optional implementation of final cost aggregate computation. As shown in Figure 22, the method of this embodiment may include:
  • Step 221 using a set of calculation units in the integrated circuit, based on the matching cost of the pixels in the first overlapping area of the first block in the plurality of blocks and the to-be-aggregated area of the first block, calculate and obtain The first cost aggregation result of the plurality of first pixels in the to-be-aggregated area of the first block in the first direction.
  • Step 222 Using the set of calculation units, calculate and obtain the second cost aggregation result of the plurality of first pixels located in the to-be-aggregated area of the first block in the second direction.
  • step 221 is taken as an example before step 222 . It can be understood that, in other embodiments, step 221 may also be performed after step 222, that is, the second cost aggregation result is calculated first, and then the first cost aggregation result is calculated.
  • Step 223 Using the set of calculation units, based on the first cost aggregation result of the pixels in the to-be-aggregated area of the first block in the first direction and the second cost aggregation result in the second direction, calculate Obtain the third cost aggregation result of the pixels in the to-be-aggregated area of the first block.
  • the first cost aggregation result, the second cost aggregation result, and the third cost aggregation result may specifically include aggregation results of pixels under each parallax within the parallax range.
  • the first cost aggregation result of a pixel p under disparity d can be expressed as L first direction (p, d)
  • the second cost aggregation result of pixel p under disparity d can be expressed as L second direction (p, d)
  • the third cost aggregation result of pixel p under disparity d can be expressed as L first direction + second direction (p, d).
  • the third cost aggregation result of the plurality of second pixels may specifically include: for each pixel in the plurality of second pixels, comparing the first cost aggregation result of the pixel under each parallax with the pixel in each parallax.
  • the second cost aggregation results under the parallax are correspondingly added to obtain the third cost aggregation results of the pixels under each of the parallaxes.
  • the first cost aggregation result and the second cost aggregation result are intermediate results of cost aggregation
  • the third cost aggregation result is the final cost aggregation result.
  • the first direction is L0_pd-L3_pd
  • the second direction is L4_pd-L7_pd
  • L second direction (p , d) L L4_pd (p , d) + L L5_pd (p, d) + L L6_pd (p, d) + L L7_pd (p, d)
  • One direction + second direction ( pL, L0_pd (p,d)+L L
  • the storage manner of the third cost aggregation result can be flexibly implemented according to requirements.
  • the third cost aggregation result can be stored in an external memory such as DDR of the integrated circuit.
  • the final disparity may also be determined based on the third cost aggregate result.
  • the method of this embodiment may further include: taking the disparity with the smallest aggregation result of the pixels within the disparity range as the final disparity of the pixels. For example, assuming that the parallax d is equal to the parallax 10 in the range of 0 to 127, and the value of L in the first direction + the second direction (p, d) is the smallest, then 10 is the final parallax of the pixel p.
  • the second set of calculation units may be used to calculate the final disparity of the pixels.
  • the to-be-aggregated areas of the nine blocks shown in FIG. 12B constitute a complete target image
  • the first direction includes L4-pd to L7-pd
  • the second The directions include L0-pd to L3-pd
  • the processing order of the 9 blocks is block 1 ⁇ block 2 ⁇ block 3 ⁇ block 4 ⁇ block 5 ⁇ block 6 ⁇ block 7 ⁇ block 8 ⁇ Block 9.
  • this group of calculation units can be used to calculate first, and based on the matching cost of the pixels in the first overlapping area of block 1 and the area to be aggregated in block 1, a plurality of The aggregation result of the first cost of the pixel in the first direction is then calculated by using this group of computing units to obtain the aggregation result of the second cost of the pixels located in the area to be aggregated in block 1 in the second direction, and then using The set of calculation units, based on the aggregation result of the first cost of the pixels in the area to be aggregated in the block 1 in the first direction and the aggregation result of the second cost in the second direction, calculate the aggregation to be aggregated in the block 1 The third cost aggregation result for the pixels in the region.
  • this group of calculation units can be used first to calculate, based on the matching cost of the pixels in the first overlapping area of block 2 and the area to be aggregated in block 2, to calculate a plurality of pixels located in the area to be aggregated in block 2.
  • the aggregation result of the first cost in the first direction is then calculated by using the set of calculation units to obtain the aggregation result of the second cost of the pixels located in the area to be aggregated in block 2 in the second direction, and then using the A group calculation unit, based on the aggregation result of the first cost of the pixels in the area to be aggregated in block 2 in the first direction and the aggregation result of the second cost in the second direction, calculates the area to be aggregated in block 2
  • the third cost aggregation result for the pixels in . ..., and so on, until the third cost aggregation result of the pixels in the to-be-aggregated area of the last block (ie, block 9) is obtained by calculation.
  • a set of calculation units are used to first calculate the first cost aggregation result of the pixels in the to-be-aggregated area of the first block in the first direction, and then the set of calculation units is used to calculate the first cost aggregation result.
  • FIG. 23 is a schematic flowchart of an image processing method provided by another embodiment of the present application. Based on the previous embodiments of this embodiment, the calculation of the first cost of multiple first pixels in the to-be-aggregated area of the first block is mainly described. An optional implementation manner of calculating the aggregation result and calculating the second cost aggregation result of multiple second pixels in the to-be-aggregated area of the second block. As shown in Figure 23, the method of this embodiment may include:
  • Step 231 using the first group of calculation units, based on the matching cost of the pixels in the first overlapping area of the first block in the plurality of blocks and the to-be-aggregated area of the first block, calculate and obtain the location in the first area.
  • the plurality of blocks are overlapping blocks obtained by dividing the target image into partial regions.
  • the processing order may include a first processing order for the first group of computing units.
  • the multiple blocks can be specifically block 1 to block 9 in FIG. 12B as an example, and the first processing sequence between the 9 blocks can be, for example, as follows. Block 9 ⁇ Block 8 ⁇ Block 7 ⁇ Block 6 ⁇ Block 5 ⁇ Block 4 ⁇ Block 3 ⁇ Block 2 ⁇ Block 1, of course, in other embodiments, between multiple blocks
  • the first processing sequence can also be in other forms, which are not limited in this application.
  • the first group of calculation units is a group of calculation units in an integrated circuit, and the first group of calculation units can be used to calculate the first pixels in the to-be-aggregated area of the first block of the plurality of blocks on the first side. Aggregate the result with the first cost upwards.
  • Step 232 using the second group of calculation units to calculate and obtain the second cost aggregation result of the plurality of second pixels located in the to-be-aggregated area of the second block in the second direction.
  • the processing sequence among the multiple blocks may further include a second processing sequence for the second group of computing units.
  • the second processing sequence may be the same as the first processing sequence. Consistent, in another embodiment the second processing order may be inconsistent with the first processing order.
  • the second processing order among the 9 blocks can be, for example, block 1 ⁇ block 2 ⁇ block 3 ⁇ block 4 ⁇ Block 5 ⁇ Block 6 ⁇ Block 7 ⁇ Block 8 ⁇ Block 9, of course, in other embodiments, the second processing sequence between multiple blocks can also be in other forms, which is not covered in this application. Do limit.
  • the second group of computing units is another group of computing units in the integrated circuit, and the second group of computing units and the first group of computing units are two groups of computing units processed in parallel in the integrated circuit, namely the first computing unit and the second computing unit. Units can perform computations in parallel, which is beneficial to reduce computation time compared with serial computations.
  • the second group of calculation units may be configured to calculate the second cost aggregation result of the plurality of second pixels in the to-be-aggregated area of the second block of the plurality of blocks in the second direction.
  • the calculation is obtained.
  • the first cost aggregation result of the plurality of first pixels located in the to-be-aggregated area of the first block in the first direction using the second set of calculation units to calculate the multiple
  • the second cost aggregation result of the second pixel in the second direction wherein the first group of calculation units and the second group of calculation units are two groups of calculation units processed in parallel in the integrated circuit, which realizes the calculation of the calculation units located in the first block.
  • the second cost aggregation result of the second pixel in the to-be-aggregated area located in the second block in the second direction is calculated,
  • the calculation time can be reduced by almost half, and the frame rate can be nearly doubled, thereby reducing the processing time and improving the processing frame rate.
  • both the first cost aggregation result and the second cost aggregation result are the intermediate results of cost aggregation.
  • the first cost aggregation of the pixel p also needs to be aggregated.
  • the result is added to the second cost aggregation result for that pixel p. Therefore, in the case where the first processing order and the second processing order are inconsistent, it is necessary to store the intermediate result of cost aggregation of multiple blocks.
  • the first group of calculation units calculates and obtains the first cost aggregation result a1 of the plurality of pixels 4 located in the area to be aggregated in block 4
  • the second group of calculation units calculates and obtains the first cost aggregation result a1 of the plurality of pixels 1 located in the area to be aggregated in block 1.
  • the second cost aggregation result b1 needs to store the first cost aggregation result a1 and the second cost aggregation result b1.
  • the first group of calculation units calculates and obtains the first cost aggregation result a2 of a plurality of pixels 3 located in the to-be-aggregated area of block 3, and the second group of calculation units calculates to obtain the to-be-aggregated area located in block 2
  • the second cost aggregation result b2 of the plurality of pixels 2 in the middle needs to store the first cost aggregation result a2 and the second cost aggregation result b2.
  • the first group of calculation units calculates and obtains the first cost aggregation result a3 of a plurality of pixels 2 located in the to-be-aggregated area of block 2, and the second group of calculation units calculates to obtain the to-be-aggregated area located in block 3
  • the second cost aggregation results b2 of the pixels 2 in block 2 have been obtained in the previous period, after the first cost aggregation results of the pixels 2 in block 2 are calculated in the period 3, it can be directly based on the The second cost aggregation result b2 of the plurality of pixels 2 of block 2 calculated in period 2 and the first cost aggregation result a3 of the plurality of pixels 2 of block 2 calculated in period 3 are calculated to obtain the plurality of The final cost aggregation result of pixel 2, similarly, since the first cost aggregation result a2 of the plurality of pixels 3 in block 3 has been obtained in the previous period, the calculation of the plurality of pixels 3 in block 3 is performed in period 3.
  • the second cost aggregation result After the second cost aggregation result, it can be directly based on the first cost aggregation result a2 of the plurality of pixels 3 of the block 3 calculated in the period 2 and the calculation of the plurality of pixels 3 of the block 3 in the period 3.
  • the second cost aggregation result b3 is calculated to obtain the final cost aggregation result of the plurality of pixels 3, so that there is no need to store the first cost aggregation result a3 of the plurality of pixels 2 of block 2 and the plurality of pixels 3 of block 3 in the memory
  • the first group of calculation units calculates and obtains the first cost aggregation result a4 of the plurality of pixels 1 located in the area to be aggregated in block 1
  • the second group of calculation units calculates and obtains the to-be-aggregated results located in block 4
  • the first cost aggregation results of the plurality of pixels 1 in the to-be-aggregated area of block 1 are obtained by calculation in period 4 After a4, it can be directly based on the second cost aggregation result b1 of the plurality of pixels 1 in the to-be-aggregated area of the block 1 calculated in the period 1 and the calculation results of the plurality of pixels 1 in the block 1 in the period 4.
  • the first cost aggregation result a4 is calculated to obtain the final cost aggregation result of the plurality of pixels 1.
  • the first cost aggregation result a1 of the plurality of pixels 4 in block 4 has been obtained in the previous period, it is calculated in period 4.
  • the second cost aggregation result b4 of the plurality of pixels 4 in block 4 it can be directly calculated based on the first cost aggregation result a1 of the plurality of pixels 4 in block 4 calculated in period 1 and in period 4
  • the obtained second cost aggregation result b4 of the plurality of pixels 4 of the block 4 is calculated to obtain the final cost aggregation result of the plurality of pixels 4, so that there is no need to store the first cost of the plurality of pixels 1 of the block 1 in the memory.
  • the aggregated result a4 and the second cost aggregated result b4 of the plurality of pixels 4 of block 4 .
  • time period 1 to the time period 4 are four adjacent time periods from early to late. Pixel 1 to Pixel 4 are used to distinguish pixels in different blocks in Block 1 to Block 4.
  • a total of storage units with a specific size for storing 4 intermediate results are required, wherein a storage unit of a specific size is equal to the first cost aggregation result of each pixel in the first direction in the to-be-aggregated area of a block
  • the sum of the sizes of , or the sum of the sizes of the second cost aggregation results of each pixel in the to-be-aggregated area of a block in the second direction requires a relatively high storage space capacity. That is, in the case where the first processing order and the second processing order are inconsistent, many intermediate results need to be stored at the same time, and the capacity of the storage space is required to be high.
  • the first direction and the second direction are multiple preset directions, and the directions included in the first direction and the second direction are different.
  • the first processing sequence and the second processing sequence may be consistent, the first block and the second block may be adjacent in the processing sequence, and the second processing sequence may be adjacent.
  • a block may precede the first block.
  • both the first processing sequence and the second processing sequence may be block 1 ⁇ block 2 ⁇ block 3 ⁇ block 4.
  • the first processing The group computing unit calculates and obtains the first cost aggregation result a1' of the plurality of pixels located in the to-be-aggregated area of block 1, and at this time, the first cost aggregation result a1' needs to be stored.
  • the first group of calculation units calculates and obtains the first cost aggregation result a2' of multiple pixels located in the to-be-aggregated area of block 2
  • the second group of calculation units calculates and obtains the first cost aggregation result a2' located at the
  • the second cost aggregation result b1' of multiple pixels in the to-be-aggregated area of block 1 since the first cost aggregation result a1' of block 1 has been obtained in the previous period, block 1 can be directly determined during this period
  • the final cost aggregation result of the pixels in the middle there is no need to store the second cost aggregation result b1' and the first cost aggregation result a1'.
  • the first group of calculation units calculates and obtains the first cost aggregation result a3' of multiple pixels located in the to-be-aggregated area of block 3
  • the second group of calculation units calculates and obtains the first cost aggregation result a3' located at the The second cost aggregation result b2' of multiple pixels in the to-be-aggregated area of block 2 is obtained.
  • the first group of calculation units calculates and obtains the first cost aggregation result a4' of multiple pixels located in the area to be aggregated in block 4
  • the second group of calculation units calculates and obtains the first cost aggregation result a4' located at
  • the second cost aggregation result b3' of multiple pixels in the to-be-aggregated area of block 3 is obtained. Since the first cost aggregation result a3' of block 3 has been obtained in the previous period, block 3 can be directly determined during this period
  • the final cost aggregation result of the pixels in the to-be-aggregated area that is, there is no need to store the second cost aggregation result b3'.
  • the second group of calculation units calculates and obtains the second cost aggregation result b4' of the plurality of pixels located in the to-be-aggregated area of block 4.
  • the number of storage units with a specific size of intermediate results that need to be simultaneously stored in the storage device is less than 4. Therefore, the capacity requirement for storage space is small. For example, two storage units of a specific size may be required to store the intermediate results at the same time. Specifically, in the period from t1 seconds to t2 seconds, the first cost aggregation result a1' and the first cost aggregation result a2' may be stored.
  • the bit width of the cost aggregation result is 16 bits (bit) and the maximum disparity (disparity) is equal to 128, the method of storing two intermediate results of a specific size at the same time requires 4Mbit storage space.
  • Fig. 25 is a schematic flowchart of an image processing method provided by another embodiment of the present application. Based on the embodiment shown in Fig. 23, this embodiment mainly describes a method for determining the final cost aggregation result of a certain pixel p (that is, as follows An optional implementation of the third cost aggregation result). As shown in Figure 25, the method of this embodiment may include:
  • Step 251 using the first group of calculation units, based on the matching cost of the pixels in the first overlapping area of the first block in the plurality of blocks and the to-be-aggregated area of the first block, calculate and obtain the location in the first area.
  • the calculated first cost aggregation results of the plurality of first pixels in the first direction in the to-be-aggregated area of the first block may be stored in the internal memory of the integrated circuit or in the integrated circuit's internal memory. in external memory.
  • the internal memory of the integrated circuit can be, for example, a static random-access memory (Static Random-Access Memory, SRAM), and the external memory of the integrated circuit can be, for example, a double-rate synchronous dynamic random access memory (Double Data Rate Synchronous Dynamic Random Access Memory, DDR SDRAM) (abbreviated as DDR).
  • Step 252 using the second group of calculation units, to calculate and obtain the second cost aggregation result of the plurality of second pixels located in the to-be-aggregated area of the second block in the second direction, and to obtain the second cost aggregation result in the second direction based on the plurality of second pixels.
  • the first cost aggregation result in the first direction and the second cost aggregation result of the plurality of second pixels in the second direction are calculated to obtain the third cost aggregation result of the plurality of second pixels.
  • the second block is adjacent to the first block, and the second block is located before the first block in the processing sequence.
  • the first cost aggregation result of the plurality of second pixels located in the to-be-aggregated area of the second block in the first direction has been calculated.
  • the first cost aggregation result of multiple second pixels located in the to-be-aggregated area of the second block in the first direction may be calculated by the first group of calculation units, or may be obtained by all Other computing units other than the first group of computing units calculate and obtain a first cost aggregation result of a plurality of second pixels located in the to-be-aggregated area of the second block in the first direction.
  • the first cost aggregation result of the plurality of second pixels in the first direction in the to-be-aggregated area of the second block that has been calculated before it needs to be stored in the internal storage device of the integrated circuit or the in the external memory device of the integrated circuit.
  • a first cost aggregation result of a first pixel located in the to-be-aggregated area of the first block and a first cost aggregation of a second pixel located in the to-be-aggregated area of the second block The results are all stored in the internal memory of the integrated circuit.
  • the result of cost aggregation can share the same storage space in the internal memory.
  • a storage space in the internal memory can store the first cost aggregation result of the second pixel located in the to-be-aggregated area of the second block, and after the period of time and with the In another period of time adjacent to the period of time, the storage space in the internal memory may store the first cost aggregation result of the first pixel located in the to-be-aggregated area of the first block.
  • the intermediate results stored at the same time can be equal to a specific size, which is beneficial to reduce the occupation of the internal memory for further aggregated intermediate results, save the storage resources of the internal storage space, reduce the capacity requirement for the internal storage space, and save costs.
  • the size of a single block is 32 ⁇ 32 pixels
  • the bit width of the cost aggregation result is 16 bits and the maximum disparity is 128, the method of storing an intermediate result of a specific size at the same time requires 2Mbit storage space.
  • a single calculation of the first set of calculation units and the second set of calculation units each occupies a single clock cycle.
  • a single calculation may be performed for a single pixel at all parallaxes, or a single calculation may be performed for a single pixel at a partial parallax, or a single calculation may be performed for multiple pixels under all parallaxes.
  • the following method 1 or method 2 can be used to implement the first block in the to-be-aggregated area.
  • the first cost aggregation result of the first pixel and the first cost aggregation result of the second pixel located in the to-be-aggregated area of the second block share the same storage space in the internal memory.
  • the first data calculated by the first group of calculation units is stored in the internal memory in a storage space corresponding to the second data used by the second group of calculation units for calculation within the clock cycle.
  • the first data is data of the first cost aggregation result of the first pixel
  • the second data is the data of the first cost aggregation result of the second pixel.
  • the size of the data is related to the range targeted for a single calculation.
  • the size of the data may be the size of the first cost aggregation result of a single pixel under all parallaxes;
  • the size of the data may be the size of the first cost aggregation result of a single pixel under partial parallax; in the case where a single calculation may be performed for multiple pixels under all parallax , the size of the data may be the size of the first cost aggregation result of multiple pixels under all parallaxes.
  • the internal memory may include a first internal memory, the first internal memory may be a single-port internal memory, and the first internal memory is a single-port internal memory, which is beneficial to saving costs; or, The first internal memory may be another type of internal memory used as a single-port internal memory (for example, a dual-port internal memory).
  • the second data and the third data adjacent to the second data are read from the target storage space, and the first data calculated by the first group of computing units is cached.
  • the second data is used by the second group of calculation units to calculate the third cost aggregation result within the first clock cycle;
  • the third data is the data of the first cost aggregation result of the second pixel , and the third data is used by the second group of calculation units to calculate a third cost aggregation result in a second clock cycle;
  • the second clock cycle is the next clock cycle of the first clock cycle.
  • the second group of computing units can calculate the third cost in the adjacent first clock cycle and the second clock cycle.
  • the aggregation result of the first cost of the aggregation result that is, reading from the first internal memory every other clock cycle is beneficial to reduce the number of reads for the first internal memory and improve the reading efficiency.
  • data K1 for the first cost aggregation result of the second pixel calculated by the second group of calculation units in clock cycle 1 may be read from the first internal memory K1 , and the data K2 for the first cost aggregation result of the second pixel calculated by the second group of calculation units in clock cycle 2; after that, in clock cycle 3, can be read from the first internal memory for the first cost aggregation
  • the data K3 of the first cost aggregation result of the second pixel calculated by the two sets of calculation units in clock cycle 3 and the data K3 of the second pixel in the second block used by the second set of calculation units in clock cycle 4
  • the data K4 of the aggregated result of the first cost ..., and so on.
  • the second pixel is the pixel in the second block
  • clock cycle 1 and clock cycle 3 are the first clock cycle
  • clock cycle 2 and clock cycle 4 are the second clock cycle
  • data K1 and data K3 are the aforementioned
  • the second data, data K2 and data K4 are the aforementioned third data.
  • the method of this embodiment may further include: in the second clock cycle, writing the first data and the fourth data into the target storage space of the first internal memory; the first data Four data is the data of the first cost aggregation result of the first pixel calculated by the first group of calculation units in the second clock cycle. Therefore, by writing the first internal memory once in the second clock cycle, the first cost aggregation result calculated by the first group of computing units in the first clock cycle and the second clock cycle can be written into The first internal memory, that is, writing into the first internal memory every other clock cycle is beneficial to reduce the number of times of writing to the first internal memory and improve the reading efficiency.
  • the first cost for the second pixel calculated by the second group of calculation units in clock cycle 1 may be read from the first internal memory
  • the data K1 of the aggregation result, and the data K2 of the first cost aggregation result of the second pixel calculated by the first group of calculation units in clock cycle 2 the first group of calculation units calculated the first cost aggregation of the first pixel
  • the data K5 is cached, and the second group of calculation units calculates the third cost result based on the data K1.
  • the first group of calculation units calculates and obtains the data K6 of the first cost aggregation result of the first pixel, writes the data K5 and K6 into the target storage space originally occupied by the data K1 and the data K2, and,
  • the second group of calculation units calculates the third aggregate cost result based on the data K2; after that, in clock cycle 3, the third set of calculation units used for the calculation in clock cycle 3 of the second group of calculation units can be read from the first internal memory.
  • the first group of calculation units calculates and obtains the data K8 of the first cost aggregation result of the first pixel, writes the data K7 and K8 into the target storage space originally occupied by the data K3 and the data K4, and,
  • the second group of calculation units calculates the third cost result based on the data K4; , and so on.
  • the first pixel is the pixel in the first block
  • the second pixel is the pixel in the second block
  • the data K5 and the data K7 are the aforementioned first data
  • the data K1 and the data K3 are the aforementioned first data.
  • the second data, the data K2 and the data K4 are the aforementioned third data
  • the data K6 and the data K8 are the aforementioned fourth data.
  • the internal memory may include a first internal memory
  • the first internal memory may be a dual-port internal memory.
  • data K1 for the first cost aggregation result of the second pixel calculated by the second group of calculation units in clock cycle 1 may be read from the first internal memory K1 , and write the data K5 of the first cost aggregation result of the first pixel calculated by the first group of calculation units in the clock cycle 1 into the target storage space originally occupied by the data K1 in the first internal memory; after that, in the clock cycle 2 , the data K2 for the first cost aggregation result of the second pixel calculated by the second group of calculation units in clock cycle 2 can be read from the first internal memory, and the first group of calculation units in clock cycle 2.
  • the data K6 of the first cost aggregation result of the first pixel obtained by the internal calculation is written into the target storage space originally occupied by the data K2 in the first internal memory;
  • the data K7 is written into the target storage space originally occupied by the data K3 in the first internal memory; after that, in clock cycle 4, it can be read from the first internal memory for the second group of calculation units to perform in clock cycle 4.
  • the first pixel is the pixel in the first block
  • the second pixel is the pixel in the second block
  • clock cycle 1 and clock cycle 3 are the first clock cycle
  • clock cycle 2 and clock cycle 4 are For the second clock cycle, data K1 and data K3 are second data, data K2 and data K4 are third data, data K5 and data K7 are first data, and data K6 and data K8 are fourth data.
  • the bit width of the cost aggregation result is 16 bits
  • the size of the first cost aggregation result of the second pixel calculated by the second group of computing units in a single clock cycle is Dper ⁇ 16 (that is, the size of the second data is Dper ⁇ 16)
  • the size of the first cost aggregation result of the first pixel calculated by the first group of computing units is also Dper ⁇ 16 (that is, the size of the first data is also Dper ⁇ 16), so that the bit width of the first internal memory can be set For Dper ⁇ 16 ⁇ 2, the bit width is doubled, the depth is reduced by 50%, and the capacity is unchanged.
  • the first data calculated by the first group of calculation units in the second clock cycle is stored in the internal memory in the storage space corresponding to the second data used by the second group of calculation units for calculation in the first clock cycle;
  • the first data is the data of the first cost aggregation result of the first pixel
  • the second data is the data of the first cost aggregation result of the second pixel
  • the second clock cycle is the the next clock cycle of the first clock cycle.
  • the internal memory may include a first internal memory and a second internal memory.
  • the first internal memory and the second internal memory may be single-port internal memories, and the first internal memory and the second internal memory are both single-port internal memories, which is conducive to saving costs;
  • the internal memory and the second internal memory may be other types of internal memories used as single-port internal memories, and the method in this embodiment may further include: within a first clock cycle, The first data is written into the first internal memory, and the second data is read from the target storage space of the second internal memory; the second data is used for the second group of computing units in the The third cost aggregation result is calculated within the first clock cycle.
  • the method of this embodiment may further include: within the second clock cycle, reading third data from the first internal memory, and writing fourth data calculated by the second group of computing units into in the target storage space of the second internal memory.
  • the third data is the data of the first cost aggregation result of the second pixel, and the third data is used for the third cost result calculated by the second group of calculation units in the second clock cycle;
  • the fourth data is the data of the first cost aggregation result of the first pixel calculated by the first group of calculation units in the second clock cycle;
  • the second clock cycle is the first clock the next clock cycle of the cycle.
  • data K1 for the first cost aggregation result of the second pixel calculated by the second group of calculation units in clock cycle 1 may be read from the second internal memory K1 , and write the data K5 of the first cost aggregation result of the first pixel calculated by the first group of calculation units in the clock cycle 1 into the first internal memory; after that, in the clock cycle 2, can be retrieved from the first internal memory.
  • Read data K2 for the first cost aggregation result of the second pixel calculated by the second group of calculation units in clock cycle 2 and calculate the first group of calculation units in clock cycle 2 to obtain the first pixel of the first pixel.
  • the data K6 of the cost aggregation result is written into the target storage space originally occupied by the data K1 in the second internal memory; after that, in the clock cycle 3, it can be read from the second internal memory for the second group of calculation units in the clock cycle.
  • the data K3 of the first cost aggregation result of the second pixel that is calculated within 3 and the data K7 of the first cost aggregation result of the first pixel calculated by the first group of calculation units in clock cycle 3 is written into the first internal memory.
  • the first pixel is the pixel in the first block
  • the second pixel is the pixel in the second block
  • clock cycle 1 and clock cycle 3 are the first clock cycle
  • clock cycle 2 and clock cycle 4 are
  • data K1 and data K3 are second data
  • data K2 and data K4 are third data
  • data K5 and data K7 are first data
  • data K6 and data K8 are fourth data.
  • the first aggregation result of the plurality of pixels in the to-be-aggregated area of a specific block of the plurality of blocks in the first direction may be obtained in the process of the previous processing.
  • a block is the first of the plurality of blocks processed by the second set of computing units.
  • the first number of pixels in the to-be-aggregated area of block 1 in the first direction can be obtained in advance during the previous processing.
  • the result of cost aggregation refer to Figure 29: in a period of time t from t0 seconds to t1 seconds, the first group of calculation units can calculate the first number of pixels located in the to-be-aggregated area of block 2 in the first direction.
  • the second group of calculation units can calculate and obtain the second cost aggregation result of the plurality of pixels located in the area to be aggregated in block 1 in the second direction, and based on the result obtained in the previous processing process in block 1
  • the first cost aggregation result of the plurality of pixels in the to-be-aggregated area in the first direction is calculated, and the third cost aggregation result of the plurality of pixels located in the to-be-aggregated area of block 1 is obtained.
  • the first group of calculation units can calculate and obtain the first cost aggregation results of multiple pixels located in the to-be-aggregated area of block 3 in the first direction
  • the second group of calculation units can calculate
  • the unit can calculate and obtain the second cost aggregation result of multiple pixels located in the area to be aggregated in block 2 in the second direction, and calculate the obtained block 2 based on the first group of calculation units in the period from t0 seconds to t1 seconds
  • the first cost aggregation result of the plurality of pixels in the to-be-aggregated area in the first direction is calculated, and the third cost aggregation result of the plurality of pixels located in the to-be-aggregated area of block 2 is obtained.
  • the first group of calculation units can calculate and obtain the first cost aggregation results of multiple pixels located in the to-be-aggregated area of block 4 in the first direction
  • the second group of calculation units can calculate
  • the unit can calculate and obtain the second cost aggregation result of the plurality of pixels located in the area to be aggregated in block 3 in the second direction, and calculate the obtained block 3 based on the first group of calculation units in the period from t1 seconds to t2 seconds
  • the first cost aggregation result of the plurality of pixels in the to-be-aggregated area in the first direction is calculated, and the third cost aggregation result of the plurality of pixels located in the to-be-aggregated area of block 3 is obtained.
  • the second group of calculation units can calculate and obtain the second cost aggregation result of the plurality of pixels located in the to-be-aggregated area of block 4 in the second direction, and based on the first
  • the first cost aggregation result of the plurality of pixels in the to-be-aggregated area of block 4 in the first direction obtained by the group calculation unit in the period from t2 seconds to t3 seconds is obtained, and the result of the first cost aggregation in the to-be-aggregated area of block 4 is obtained by calculation.
  • the third cost aggregation result of multiple pixels Comparing Fig. 29 and Fig.
  • the first aggregation result of the plurality of pixels in the to-be-aggregated area of block 1 in the first direction is obtained in the previous processing process, and the block obtained by the calculation of the first group of calculation units is more
  • the first aggregation result of multiple pixels in the to-be-aggregated area of 1 in the first direction can shorten the processing time for the entire image, that is, shorten the time consumption from t5 seconds to t4 seconds.
  • the calculation is obtained.
  • the third cost aggregation result of a plurality of second pixels is obtained by calculation, wherein, the first block and the second block are adjacent and the second block is located before the first block, which realizes the calculation of the to-be-to-be-located block located in the first block.
  • FIG. 30 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application.
  • a target image in two images corresponding to different viewpoints is divided into multiple blocks with overlapping partial areas.
  • the overlapping area of the blocks includes a target overlapping area
  • the target overlapping area of the first block includes a first overlapping area corresponding to the first direction in the first block.
  • the apparatus 300 may include: a processor 301 and a memory 302 .
  • the memory 302 is used to store program codes
  • the processor 301 calls the program code, and when the program code is executed, is configured to perform the following operations:
  • the image processing apparatus provided in this embodiment can be used to execute the technical solutions of the foregoing method embodiments, and the implementation principles and technical effects thereof are similar to those of the method embodiments, and are not repeated here.
  • An embodiment of the present application further provides an integrated circuit, in which a target image in two images corresponding to different viewpoints is divided into multiple blocks whose partial areas overlap, and the overlapping area of the first block in the multiple blocks includes: the first overlapping area corresponding to the first direction in the first block; the circuit includes: a calculation unit; the calculation unit is used for:
  • Embodiments of the present application further provide an image processing apparatus, and the image processing apparatus may include the integrated circuit.
  • FIG. 31 is a schematic structural diagram of a movable platform provided by an embodiment of the application.
  • the movable platform 310 includes a body 311 , a power system 312 disposed on the body, a photographing device 313 and the aforementioned The image processing device 314 described in the embodiment;
  • the power system 312 for providing power to the movable platform
  • the photographing device 313 is used for photographing during the flight of the movable platform to obtain two images corresponding to different viewpoints.
  • the movable platform 310 may further include a gimbal 315 , and the photographing device 313 may be disposed on the body 311 through the gimbal 315 .
  • the movable platform may also include other elements or devices, which are not listed here.
  • Embodiments of the present application provide an image processing method, an integrated circuit, a device, a movable platform, and a storage medium.
  • the entire target image is divided into multiple blocks with overlapping partial areas, and the first block of the multiple blocks is divided into multiple blocks.
  • the overlapping area includes the target overlapping area, and the target overlapping area includes the first overlapping area corresponding to the first direction. Based on the first overlapping area of the first block and the matching cost of the pixels in the area to be aggregated, the first area is calculated.
  • the cost aggregation result of the pixels in the block in the first direction realizes the calculation based on the matching cost of the pixels in the first block to obtain the first cost aggregation result of the pixels located in the to-be-aggregated area of the first block in the first direction , so that the calculation of the first cost aggregation results between different blocks is independent of each other, and the cost aggregation can no longer limit the processing order of pixels between different blocks, and is determined sequentially along the first direction for the entire target image.
  • the cost aggregation result of each pixel in the entire image the requirement for the processing order of the pixels is relatively high, and the requirement for the processing order is reduced, thereby reducing the restrictions on the implementation of cost aggregation.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Image Processing (AREA)

Abstract

一种图像处理方法、集成电路、装置、可移动平台及存储介质。该方法包括：获取位于所述第一区块中像素的匹配代价；基于所述第一区块的第一重叠区域和所述第一区块的待聚合区域中像素的匹配代价，计算得到位于所述第一区块的待聚合区域中的多个第一像素在所述第一方向上的第一代价聚合结果。本申请降低了对于处理顺序的要求，从而减少了对于代价聚合的实现方式的限制。

Description

图像处理方法、集成电路、装置、可移动平台及存储介质 技术领域
本申请涉及图像处理技术领域，尤其涉及一种图像处理方法、集成电路、装置、可移动平台及存储介质。
背景技术
双目立体匹配技术利用不同视点对同一场景进行取景后所获得的两幅图像进行对应点匹配，从而获得对应点视差，进而可以计算得到像素的深度信息。此技术可以应用于可移动平台(例如，无人机和无人车)的避障以及三维重建等领域。一般来说，计算深度信息的过程包括：匹配代价计算、代价聚合、视差计算和优化、视差精细化。其中，对于代价聚合的步骤来说，耗时较长，处理帧率较低。因此，有必要提出一种改进的方案。
发明内容
本申请实施例提供一种图像处理方法、集成电路、装置、可移动平台及存储介质，用以解决现有技术中对于代价聚合的步骤来说，对像素的处理顺序要求较高，导致对实现方式的限制较大的问题。
第一方面，本申请实施例提供一种图像处理方法，对应不同视点的两幅图像中的目标图像被划分为部分区域重叠的多个区块，所述多个区块中的第一区块的重叠区域包括所述第一区块中与第一方向对应的第一重叠区域；所述方法包括：
获取位于所述第一区块中像素的匹配代价；
基于所述第一区块的所述第一重叠区域和所述第一区块的待聚合区域中像素的匹配代价，计算得到位于所述第一区块的所述待聚合区域中的多个第 一像素在所述第一方向上的第一代价聚合结果。
第二方面，本申请实施例提供一种图像处理装置，对应不同视点的两幅图像中的目标图像被划分为部分区域重叠的多个区块，所述多个区块中的第一区块的重叠区域包括所述第一区块中与第一方向对应的第一重叠区域；所述装置包括存储器和处理器；
所述存储器，用于存储程序代码；
所述处理器，调用所述程序代码，当程序代码被执行时，用于执行以下操作：
获取位于所述第一区块中像素的匹配代价；
基于所述第一区块的所述第一重叠区域和所述第一区块的待聚合区域中像素的匹配代价，计算得到位于所述第一区块的所述待聚合区域中的多个第一像素在所述第一方向上的第一代价聚合结果。
第三方面，本申请实施例提供一种集成电路，对应不同视点的两幅图像中的目标图像被划分为部分区域重叠的多个区块，所述多个区块中的第一区块的重叠区域包括所述第一区块中与第一方向对应的第一重叠区域；所述电路包括：计算单元；所述计算单元用于：
获取位于所述第一区块中像素的匹配代价；
基于所述第一区块的所述第一重叠区域和所述第一区块的待聚合区域中像素的匹配代价，计算得到位于所述第一区块的所述待聚合区域中的多个第一像素在所述第一方向上的第一代价聚合结果。
第四方面，本申请实施例提供一种图像处理装置，包括：第三方面任一项所述的集成电路。
第五方面，本申请实施例提供一种可移动平台，所述可移动平台包括机身、设置于所述机身上的动力系统、拍摄装置和第二方面或第四方面任一项所述的图像处理装置；
所述动力系统，用于为所述可移动平台提供动力；
所述拍摄装置，用于在所述可移动平台飞行过程中进行拍摄，以获得对应不同视点的两幅图像。
第六方面，本申请实施例提供一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，所述计算机程序包含至少一段代码，所述至少一段代码可由计算机执行，以控制所述计算机执行如第一方面所述的方法。
第七方面，本申请实施例提供一种计算机程序，当所述计算机程序被计算机执行时，用于实现如第一方面任一项所述的方法。
本申请实施例提供一种图像处理方法、集成电路、装置、可移动平台及存储介质，降低了对于处理顺序的要求，从而减少了对于代价聚合的实现方式的限制。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本申请实施例提供的图像处理方法的应用场景示意图；
图2为本申请一实施例提供的16个代价聚合方向示意图；
图3为本申请一实施例提供的8个代价聚合方向示意图；
图4A为本申请一实施例提供的图像处理方法的流程示意图；
图4B为本申请一实施例提供的目标图像的区块划分的示意图；
图5为本申请另一实施例提供的图像处理方法的流程示意图；
图6为本申请一实施例提供的第一重叠区域为上部分区域的原理的示意图；
图7为本申请一实施例提供的第一重叠区域为左部分区域的原理的示意图；
图8为本申请一实施例提供的第一重叠区域为下部分区域的原理的示意图；
图9为本申请一实施例提供的第一重叠区域为右部分区域的原理的示意图；
图10为本申请一实施例提供的区块的示意图；
图11为本申请另一实施例提供的区块的示意图；
图12A为本申请另一实施例提供的目标图像的区块划分的示意图；
图12B为本申请又一实施例提供的目标图像的区块划分的示意图；
图13为本申请又一实施例提供的区块的示意图；
图14为本申请一实施例提供的部分像素包括上边缘像素的原理的示意图；
图15为本申请一实施例提供的部分像素包括左边缘像素的原理的示意图；
图16为本申请一实施例提供的部分像素包括下边缘像素的原理的示意图；
图17为本申请一实施例提供的部分像素包括右边缘像素的原理的示意图；
图18为本申请一实施例提供的第二区块的待聚合区域的部分像素的示意图；
图19为本申请一实施例提供的基于部分像素的单方向代价聚合结果恢复其他像素的单方向代价聚合结果的示意图；
图20为本申请一实施例提供的目标图像中各区块的待聚合区域中部分像素的示意图；
图21为本申请一实施例提供的目标图像中各区块对应目标区块的待聚合区域中部分边缘像素的示意图；
图22为本申请又一实施例提供的图像处理方法的流程示意图；
图23为本申请又一实施例提供的图像处理方法的流程示意图；
图24为本申请一实施例提供的第一、第二组计算单元对于区块的处理顺序示意图；
图25为本申请又一实施例提供的图像处理方法的流程示意图；
图26为本申请一实施例提供的从内部存储器中读取第一代价聚合结果的示意图；
图27A为本申请一实施例提供的连续多个时钟周期内第一代价聚合结果的存取示意图；
图27B为本申请另一实施例提供的连续多个时钟周期内第一代价聚合结果的存取示意图；
图28为本申请又一实施例提供的连续多个时钟周期内第一代价聚合结果的存取示意图；
图29为本申请一实施例提供的第一、第二组计算单元对于区块的处理顺序示意图；
图30为本申请一实施例提供的图像处理装置的结构示意图；
图31为本申请一实施例提供的可移动平台的结构示意图。
具体实施方式
为使本申请实施例的目的、技术方案和优点更加清楚，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
本申请实施例提供的图像处理方法可以应用于如图1所示的图像处理系统10，图像处理系统10包括拍摄装置11和图像处理装置12。
拍摄装置11可以拍摄得到对应不同视点的两幅图像，拍摄装置11例如可以为双目摄像头，当然，在其他实施例中，拍摄装置11还可以为其他类型装置(例如，包含多个摄像头的装置)，本申请对此不做限定。
图像处理装置12可以从拍摄装置11获得对应不同视点的两幅图像，并针对两幅图像中的目标图像采用本申请实施例提供的图像处理方法进行处理。其中，图像处理装置12可以包括集成电路(Integrated Circuit，IC)，该集成电路例如可以为专用集成电路(Application Specific Integrated Circuit，ASIC)、中央处理器(Central Processing Unit，CPU)等。
可选的，拍摄装置11和图像处理装置12可以集成于同一设备，该设备例如可以为可移动平台。示例性的，所述可移动平台包括无人飞行器、云台、无人车、移动终端中的至少一种。当然，在其他实施例中该设备还可以为其他形式，本申请对此不做限定。
或者，可选的，拍摄装置11和图像处理装置12可以设置于不同的设备，例如，拍摄装置11可以设置于可移动平台中，图像处理装置12可以设置于用于控制可移动平台的控制终端中，当然，在其他实施例中，拍摄装置11和图像处理装置12还可以分别设置于其他设备中，本申请对此不做限定。
本申请实施例提供的图像处理方法是应用于利用立体匹配技术计算图像深度信息的代价聚合阶段，所述目标图像即为两幅图像中需要计算深度信息的图像。其中，计算图像深度信息的主要过程包括：
1、计算匹配代价(matching cost)。其中，匹配代价也称为匹配代价值。
2、代价聚合(cost aggregation)。
3、视差计算和优化(disparity computation/optimization)。
4、视差精细化(disparity refinement)。
其中，匹配代价表示两个像素之间的相似度。代价聚合是用于利用整个场景对应的图像中的所有像素对每一个单一像素施加影响，来获得一个较匹配代价更加精确的结果。视差计算和优化和视差精细化用于对计算结果进行校验以及进行亚像素处理等。通过上面列举的图像深度信息的计算过程，可以计算出目标图像的深度信息，然后根据三角测量原理可以进一步得到某一物体到拍摄装置之间的距离。此距离即为深度信息。本申请实施例具体针对进行代价聚合的方法进行改进，下面将介绍代价聚合的数学原理。
在数学上，代价聚合的过程可以看作是求取一个全局能量最小值：
Figure PCTCN2020100101-appb-000001
其中，C(p,D p)为某一像素的匹配代价，p为图像中的任一像素。P 1T[|D p-D q|＝1]为相邻像素中视差变化范围较小时的惩罚项，T[|D p-D q|＝1]表示相邻像素D p和D q之间的视差差值为1，Np范围为像素p的相邻区域。P 2T[|D p-D q＞1|]为相邻像素中视差变化范围较大(即，视差差值大于1)时的惩罚项。代价聚合过程涉及的计算复杂度较高，是一个多项式复杂程度的非确定性问题(Non-deterministic Polynomial，NP)问题。
为了降低代价聚合过程的计算复杂度，可以仅使用某些方向上的像素进行代价聚合。如图2和图3所示，依次是16个方向和8个方向示意图，图中的p为图像中某一像素。除了可以从16个方向进行代价聚合之外，还可以从8个方向进行代价聚合，或者也可以从其他数目的方向进行代价聚合，对此本申请实施例不做限定。
对于图像中的某一像素p，在某一方向上的代价聚合可以通过下面的公式实现：
Figure PCTCN2020100101-appb-000002
其中，多项式中的第一项C(p,d)为像素p的匹配代价。第二项为函数min()中的4项数据中的最小值，该4项数据包括L r(p-r,d)、L r(p-r,d-1)+P 1、 L r(p-r,d+1)+P 1和min iL r(p-r,i)+P 2。L r(p-r,d)表示像素p在r方向上的上一个像素在视差d下的代价聚合结果。L r(p-r,d-1)表示像素p在r方向上的上一个像素在视差d-1下的代价聚合结果。L r(p-r,d+1)表示像素p在r方向上的上一个像素在视差d+1下的代价聚合结果。min iL r(p-r,i)表示像素p在r方向上的上一个像素在视差i下的最小代价聚合结果，其中，i的取值范围为0到最大视差值之间的任一数值。P 1和P 2为惩罚系数，与像素p的距离越远，惩罚系数越大。第三项min kL r(p-r,k)的作用是防止L r(p,d)饱和，即防止随着迭代L r(p,d)的增长过大。
通过上面的公式2，可以计算某一像素p在某一方向上的代价聚合结果(即，单方向代价聚合结果)，重复使用该公式，可以得到某一像素p在所有方向上分别对应的代价聚合结果，进而将某一像素p在所有方向上分别对应的代价聚合结果进行相加，即可得到某一像素p的最终代价聚合结果。对于某一像素p来说，最小的最终代价聚合结果对应的视差d为最终的视差取值。
通过上面的公式2可以看出，随着惩罚项的不断累加，在r方向上与某一像素p距离较远的其他像素对该像素p在r方向上的代价聚合结果的影响比较弱。基于此，本申请提出一种新的计算代价聚合结果的方法，通过整张目标图像被划分为部分区域重叠的多个区块，多个区块中的第一区块的重叠区域包括目标重叠区域，所述目标重叠区域包括与第一方向对应的第一重叠区域，基于第一区块的第一重叠区域和待聚合区域中像素的匹配代价，计算得到第一区块中的像素在第一方向上的代价聚合结果，实现了基于第一区块中像素的匹配代价计算得到位于第一区块的待聚合区域中像素在第一方向上的第一代价聚合结果，使得不同区块之间第一代价聚合结果的计算之间是相互独立的，代价聚合可以不再限制不同区块之间像素的处理顺序，降低了对于处理顺序的要求，从而减少了对于代价聚合的实现方式的限制。其中，第一区块重叠区域是指第一区块中与其他区块相互重叠的区域，第一区块的待聚合区域是指第一区块中需要计算代价聚合结果的区域，第一区块的目标重叠区域是指第一区块中不需要计算代价聚合结果但是可以用于计算待聚合区域的代价聚合的参考区域。
下面结合附图，对本申请的一些实施方式作详细说明。在不冲突的情况下，下述的实施例及实施例中的特征可以相互组合。
图4A为本申请一实施例提供的图像处理方法的流程示意图，本实施例的 执行主体可以为图像处理装置12。如图4A所示，本实施例的方法可以包括：
步骤41，获取位于所述多个区块中的第一区块中像素的匹配代价。
本步骤中，所述多个区块是通过对目标图像划分得到的部分区域重叠的多个区块，所述第一区块可以为所述多个区块中的包括目标重叠区域的任意一个区块。以目标图像被划分为如图4B所示的3个区块为例，参考图4B，区块1和区块2包括目标重叠区域，其中，区块1的目标重叠区域为区块1中最右侧的区域1，且区块2中最左侧的区域2与区块1的区域1重叠。区块2的目标重叠区域为区块2中最右侧的区域3，且区块3中最左侧的区域4与区块2的区域3重叠。需要说明的是，图4B中的目标重叠区域仅为举例。
所述第一区块与其他区块之间的重叠区域可以包括目标重叠区域和其他重叠区域。其中，第一区块的重叠区域是指第一区块中与其他区块相互重叠的区域。例如，在图4B中，区块2的重叠区域包括区域2和区域3。所述第一区块的目标重叠区域是指所述第一区块中用于计算得到第一区块的待聚合区域中多个像素(为了便于描述，以下记为第一像素)的代价聚合结果的重叠区域。例如，在图4B中，区块2的两个重叠区域中的区域3。第一区块的待聚合区域是指第一区块中需要计算代价聚合结果的区域。第一区块的待聚合区域具体可以是第一区块中除目标重叠区域之外的其他区域，待聚合区域包括非重叠区域和其他重叠区域。例如，在图4B中，区块2的待聚合区域包括区块2的区域2和区域5，其中区域2为区块2的其他重叠区域，区域5为区块2的非重叠区域。需要说明的是，第一区块的其他重叠区域可以用于计算其他区块的代价聚合结果，例如在图4B中，区块2的区域2与区块1的区域1重叠的，区块2的区域2可以用于计算区块1的代价聚合结果。当然，区块1的区域1也可以用于计算区块1的代价聚合结果。
可以理解的是，所述第一区块的目标重叠区域在所述第一区块中的位置，与聚合所使用的方向(为了便于描述，以下记为第一方向)相对应，所述第一区块的目标重叠区域包括所述第一区块中与第一方向对应的第一重叠区域，可以理解的是，所述第一区块中的第一重叠区域是指用于计算所述第一区块的待聚合区域中的多个第一像素在第一方向上的第一代价聚合结果的重叠区域。
其中，所述第一方向可以是需要进行代价聚合的所有方向中的部分方向或全部方向，例如可以为图3中的L0_pd至L3_pd四个方向，或者，例如可以为 图3中的L4_pd至L7_pd四个方向，当然，在其他实施例中，第一方向还可以为其他形式，本申请对此不做限定。
第一代价聚合结果是指像素的匹配代价在第一方向上代价聚合的结果。可以理解的是，在第一方向为多个预设方向情况下，第一代价聚合结果可以为多个第一方向(即，多个预设方向)中各方向上代价聚合的结果之和。
示例性的，可以计算获得位于第一区块中像素的匹配代价，或者，可以通过从存储器中读取的方式获得位于第一区块中像素的匹配代价，当然，在其他实施例中，也可以通过其他方式获得第一区块中像素的匹配代价，本申请对此不做限定。
可以理解的是，位于第一区块中像素的匹配代价包括位于所述第一区块的第一重叠区域中像素的匹配代价以及位于所述第一区块的待聚合区域中像素的匹配代价。在一个实施方式中，在第一区块对应的代价聚合的计算过程中，第一重叠区域中的像素的匹配代价仅用于计算待聚合区域的像素的匹配代价，而不存储在第一区块对应的代价聚合的计算过程计算得到的第一重叠区域中的像素的匹配代价。
步骤42，基于所述第一区块的第一重叠区域和所述第一区块的待聚合区域中像素的匹配代价，计算得到位于所述第一区块的待聚合区域中的多个第一像素在第一方向上的第一代价聚合结果。
本步骤中，可以将第一区块视为一个子图像，进而可以针对该子图像基于前述的公式2计算得到位于第一区块的待聚合区域中的多个像素在第一方向上的第一代价聚合结果。具体的，可以从所述第一区块的第一重叠区域中的边缘像素开始，沿着第一方向依次计算位于第一区块的第一重叠区域和待聚合区域中像素的匹配代价在第一方向上的第一代价聚合结果，以得到位于第一区块的待聚合区域中的多个第一像素在第一方向上的第一代价聚合结果。
本申请实施例提供的图像处理方法，通过整张目标图像被划分为部分区域重叠的多个区块，多个区块中的第一区块的重叠区域包括与第一方向对应的第一重叠区域，基于第一区块的第一重叠区域和/或待聚合区域中像素的匹配代价，计算得到第一区块中的像素在第一方向上的代价聚合结果，实现了基于第一区块中像素的匹配代价计算得到位于第一区块的待聚合区域中像素在第一方向上的第一代价聚合结果，使得不同区块之间第一代价聚合结果的 计算之间是相互独立的，代价聚合可以不再限制不同区块之间像素的处理顺序，与针对整张目标图像沿着第一方向依次确定整张图像中各像素的代价聚合结果，对像素的处理顺序要求较高相比，降低了对于处理顺序的要求，从而减少了对于代价聚合的实现方式的限制。
图5为本申请另一实施例提供的图像处理方法的流程示意图，本实施例在图4A所示实施例的基础上，主要描述了一种第一方向是需要进行代价聚合的所有方向中的部分方向情况下的一种可选实现方式。如图5所示，本实施例的方法可以包括：
步骤51，获取位于所述多个区块中的第一区块中像素的匹配代价。
需要说明的是，步骤51与步骤41类似，在此不再赘述。
步骤52，基于所述第一区块的第一重叠区域和所述第一区块的待聚合区域中像素的匹配代价，计算得到位于所述第一区块的待聚合区域中的多个第一像素在第一方向上的第一代价聚合结果。
在一个实施例中，所述第一方向包括向下方向，所述第一重叠区域包括所述第一区块的上部分区域。其中，所述向下方向可以为垂直向下方向或者斜向下方向。以图3所示的8个方向为例，垂直向下方向可以为图3中的L1_pd方向，斜向下方向可以为图3中的L0_pd方向和L2_pd方向。
如图6所示，在第一方向包括向下方向情况下，第一区块的第一重叠区域具体可以为黑粗虚线框框出的第一区块的上部分区域，第一区块的待聚合区域具体可以为黑粗实线框框出的第一区块中与上述上部分区域相邻的区域。参考图6，以第一方向包括L1_pd方向为例，针对第一区块中的某一列像素，可以沿着L1_pd方向从像素A0开始，依次计算出像素A1、A2、A3、A4、A5、……A18在L1_pd方向上的代价聚合结果，以得到这些像素在第一方向上的第一代价聚合结果。
继续参考图6，以第一方向包括L2_pd方向为例，针对第一区块中的某一排L2_pd方向的像素，可以沿着L2_pd方向从像素B0开始，依次计算出像素B1、B2、B3、B4、B5、B6在L2_pd方向上的代价聚合结果，以得到这些像素在第一方向上的第一代价聚合结果。
需要说明的是，在一个实施方式中，图6所示的第一区块是多个区块中不包含目标图像上边缘像素的区块，图6所示的第一区块例如可以是图12B中的区块4-区块9。
在另一个实施例中，所述第一方向包括向右方向，所述第一重叠区域包括所述第一区块的左部分区域。其中，所述向右方向可以包括水平向右方向和斜向右方向。以图3所示的8个方向为例，水平向右方向例如可以为图3中的L3_pd方向，斜向右方向例如可以为图3中的L2_pd方向和L4_pd方向。
如图7所示，在第一方向包括向右方向情况下，第一区块的第一重叠区域具体可以为黑粗虚线框框出的第一区块的左部分区域，第一区块的待聚合区域具体可以为黑粗实线框框出的与第一区块上述左部分区域相邻的区域。参考图7，以第一方向包括L3_pd方向为例，针对第一区块中的某一行像素，可以沿着L3_pd方向从像素C0开始，依次计算出像素C1、C2、C3、C4、C5、……C18在L3_pd方向上的代价聚合结果，以得到这些像素在第一方向上的第一代价聚合结果。
继续参考图7，以第一方向包括L2_pd方向为例，针对第一区块中的某一排L2_pd方向的像素，可以沿着L2_pd方向从像素D0开始，依次计算出像素D1、D2、D3、D4、D5、D7在L2_pd方向上的代价聚合结果，以得到这些像素在第一方向上的第一代价聚合结果。
需要说明的是，在一个实施方式中，图7所示的第一区块是多个区块中不包含目标图像左边缘像素的区块，图7所示的第一区块例如可以是图12B中的区块2、区块3、区块5、区块6、区块8或区块9。
在又一个实施例中，所述第一方向包括向上方向，所述第一重叠区域包括所述第一区块的下部分区域。其中，所述向上方向可以包括垂直向上方向和斜向上方向。以图3所示的8个方向为例，垂直向上方向例如可以为图3中的L5_pd方向，斜向上方向例如可以为图3中的L4_pd方向和L6_pd方向。
如图8所示，在第一方向包括向上方向情况下，第一区块的第一重叠区域具体可以为黑粗虚线框框出的第一区块的下部分区域，第一区块的待聚合区域具体可以为黑粗实线框框出的与第一区块上述下部分区域相邻的区域。参考图8，以第一方向包括L5_pd方向为例，针对第一区块中的某一列像素，可以沿着L5_pd方向从像素E0开始，依次计算出像素E1、E2、E3、E4、E5、……E18在L5_pd方向上的代价聚合结果，以得到这些像素在第一方向上的第一代价聚合结果。
继续参考图8，以第一方向包括L4_pd方向为例，针对第一区块中的某一排L4_pd方向的像素，可以沿着L4_pd方向从像素F0开始，依次计算出像素F1、 F2、F3、F4、F5、F7在L4_pd方向上的代价聚合结果，以得到这些像素在第一方向上的第一代价聚合结果。
需要说明的是，在一个实施方式中，图8所示的第一区块是多个区块中不包含目标图像下边缘像素的区块，图8所示的第一区块例如可以是图12B中的区块1-区块6。
在又一个实施例中，所述第一方向包括向左方向，所述第一重叠区域包括所述第一区块的右部分区域。其中，所述向左方向可以包括水平向左方向和斜向左方向。以图3所示的8个方向为例，水平向左方向例如可以为图3中的L7_pd方向，斜向左方向例如可以为图3中的L0_pd方向和L6_pd方向。
如图9所示，在第一方向包括向左方向情况下，第一区块的第一重叠区域具体可以为黑粗虚线框框出的第一区块的右部分区域，第一区块的待聚合区域具体可以为黑粗实线框框出的与第一区块上述右部分区域相邻的区域。参考图9，以第一方向包括L7_pd方向为例，针对第一区块中的某一行像素，可以沿着L7_pd方向从像素G0开始，依次计算出像素G1、G2、G3、G4、G5、……G18在L7_pd方向上的代价聚合结果，以得到这些像素在第一方向上的第一代价聚合结果。
继续参考图9，以第一方向包括L6_pd方向为例，针对第一区块中的某一排L6_pd方向的像素，可以沿着L6_pd方向从像素H0开始，依次计算出像素H1、H2、H3、H4、H5在L6_pd方向上的代价聚合结果，以得到这些像素在第一方向上的第一代价聚合结果。
需要说明的是，在一个实施方式中，图9所示的第一区块是多个区块中不包含目标图像右边缘像素的区块，图9所示的第一区块例如可以是图12B中的区块1、区块2、区块4、区块5、区块7或区块8。
需要说明的是，图6-图9中一个小方格表示一个像素。
需要说明的是，图6-图9中第一重叠区域所占行数或列数仅为举例。可以理解的是，第一重叠区域所占行数或列数越多，计算量越大，计算得到的第一代价聚合结果越精确。第一重叠区域所占行数或列数，可以根据需求灵活实现。
基于前述图6至图9的分析可知，以第一方向为图3中左上部分的L0_pd至L3_pd为例，如图10所示，第一区块的第一重叠区域具体可以为黑粗虚线框框出的第一区块的左部分区域和上部分区域，第一区块的待聚合区域具体可以 为黑粗实线框框出的与第一区块左部分区域和上部分区域相邻的区域。
基于前述图6至图9的分析可知，以第一方向为图3中左上部分的L4_pd至L7_pd为例，如图11所示，第一区块的第一重叠区域具体可以为黑粗虚线框框出的第一区块的右部分区域和下部分区域，第一区块的待聚合区域具体可以为黑粗实线框框出的与第一区块右部分区域和下部分区域相邻的区域。
在图11的基础上，以目标图像为32×32像素的图像，且目标图像被划分为部分区域重叠的4个区块为例，所得到的4个区块可以如图12A所示。参考图12A，4个区块分别为区块1至区块4，其中第一区块可以为4个区块中的区块1-区块3。需要说明的是，由于图12A中区块4的右边缘和下边缘的像素是目标图像的边缘像素，在第一方向为L4_pd至L7_pd时，区块4中像素的代价聚合结果并不会受到其他区块中像素的影响，因此区块4中不包括第一重叠区域。
需要说明的是，图12A中以将目标图像等分为待聚合区域的大小均为16×16像素的4个区块为例，并且为了简化附图，图12A中仅示出区块1的第一重叠区域的。其中，区块1对应黑粗实线框框出的区域即为区块1的待聚合区域，其大小为16×16像素，区块1的待聚合区域的右侧以及下方由黑粗虚线框框出的区域为区块1的第一重叠区域。区块1的大小为(16+3)×(16+3)像素。其中，上式中第一个括号中的3用于表示区块1的待聚合区域右侧的第一重叠区域中像素的列数(图中的第一小方格代表一个像素)，第二个括号中的3用于表示区块1的待聚合区域下方的第一重叠区域中像素的行数(图中的第一小方格代表一个像素)。类似的，区块2的待聚合区域的大小为16×16像素，区块2的大小为16×(16+3)像素，其中括号中的3用于表示区块2的下方的第一重叠区域中像素的行数。区块3的待聚合区域的大小为16×16像素，区块3的大小为(16+3)×16像素，其中括号中的3用于表示区块3的右侧的第一重叠区域中像素的列数。
在图11的基础上，以目标图像为48×48像素的图像，且目标图像被划分为部分区域重叠的9个区块为例，所得到的9个区块可以如图12B所示。参考图12B，9个区块分别为区块1至区块9，其中第一区块可以为9个区块中的区块1-区块8。需要说明的是，由于图12B中区块9的右边缘和下边缘的像素是目标图像的边缘像素，在第一方向为L4_pd至L7_pd时，区块9中像素的代价聚合结果并不会受到其他区块中像素的影响，因此区块9中不包括第一重叠区域。在一实施方式中，利用多个预设值来计算区块9的代价聚合结果。
需要说明的是，图12B中以将目标图像等分为待聚合区域的大小均为16×16像素的9个区块为例，并且为了简化附图，图12B中仅示出区块1的第一重叠区域的。其中，区块1对应黑粗实线框框出的区域即为区块1的待聚合区域，其大小为16×16像素，区块1的待聚合区域的右侧以及下方由黑粗虚线框框出的区域为区块1的第一重叠区域，区块1的大小为(16+3)×(16+3)像素，其中第一个括号中的3用于表示区块1的待聚合区域右侧的第一重叠区域中像素的列数(图中的第一小方格代表一个像素)，第二个括号中的3用于表示区块1的待聚合区域下方的第一重叠区域中像素的行数(图中的第一小方格代表一个像素)。类似的，区块2的待聚合区域的大小为16×16像素，区块2的大小为(16+3)×(16+3)像素，其中第一个3用于表示区块2的待聚合区域右侧的第一重叠区域中像素的列数，第二个括号中的3用于表示区块2的待聚合区域下方的第一重叠区域中像素的行数。区块3的待聚合区域的大小为16×16像素，区块3的大小为16×(16+3)像素，其中括号中的3用于表示区块3的下方的第一重叠区域中像素的行数。区块4的待聚合区域的大小为16×16像素，区块4的大小为(16+3)×(16+3)像素，其中第一个括号中的3用于表示区块4的待聚合区域右侧的第一重叠区域中像素的列数，第二个括号中的3用于表示区块4的待聚合区域下方的第一重叠区域中像素的行数。区块5的待聚合区域的大小为16×16像素，区块5的大小为(16+3)×(16+3)像素，其中第一个括号中的3用于表示区块5的待聚合区域右侧的第一重叠区域中像素的列数，第二个括号中的3用于表示区块5的待聚合区域下方的第一重叠区域中像素的行数。区块6的待聚合区域的大小为16×16像素，区块6的大小为16×(16+3)像素，其中括号中的3用于表示区块6的下方的第一重叠区域中像素的行数。区块7的待聚合区域的大小为16×16像素，区块7的大小为(16+3)×16像素，其中括号中的3用于表示区块7的右侧的第一重叠区域中像素的列数。区块8的待聚合区域的大小为16×16像素，区块8的大小为(16+3)×16像素，其中括号中的3用于表示区块8的右侧的第一重叠区域中像素的列数。
步骤53，计算得到位于所述多个区块中第二区块的待聚合区域中的多个第二像素在第二方向上的第二代价聚合结果。
本步骤中，所述第二区块可以为所述多个区块中的包括目标重叠区域的任意一个区块。所述第二区块与所述第一区块可以为同一区块，或者，所述第二区块与所述第一区块可以为不同区块。需要说明的是，步骤53与步骤52 之间可以没有先后顺序限制。
类似于第一区块，所述第二区块与其他区块之间的重叠区域可以包括目标重叠区域和其他重叠区域。其中，所述第二区块的目标重叠区域可以是指所述第二区块中用于计算得到第二区块的待聚合区域中多个像素(为了便于描述，以下记为第二像素)的代价聚合结果的重叠区域。所述第二区块的其他重叠区域可以是指所述第二区块中用于计算得到与所述第二区块相邻的其他区块的待聚合区域中多个像素的代价聚合结果的重叠区域。所述第二区块可以为图12A中4个区块中的区块1-区块4中的任何一个。
所述第二方向是指需要进行代价聚合的所有方向中与第一方向不同的部分方向。可选的，为了简化实现，第一方向和第二方向可以对应相反，例如第一方向为图3中的L0_pd至L3_pd四个方向，第二方向可以为图3中的L4_pd至L7_pd四个方向，又例如，第一方向可以为图3中的L4_pd至L7_pd四个方向，第二方向可以为图3中L0_pd至L3_pd四个方向。当然，在其他实施例中，第一方向还可以为其他形式，本申请对此不做限定。
第二代价聚合结果是指像素的匹配代价在第二方向上代价聚合的结果。可以理解的是，在第二方向为多个预设方向情况下，第二代价聚合结果可以为多个第二方向(即，多个预设方向)中各方向上代价聚合的结果之和。
在一个实施例中，所述第二区块的目标重叠区域可以包括所述第二区块中与第二方向对应的第二重叠区域。基于此，所述计算得到位于所述第二区块的待聚合区域中的多个第二像素在第二方向上的第二代价聚合结果，具体可以包括：基于所述第二区块的第二重叠区域和所述第二区块的待聚合区域中像素的匹配代价，计算得到位于所述第二区块的待聚合区域中的多个第二像素在所述第二方向上的第二代价聚合结果。即，针对区块中待聚合区域中的多个第二像素在第二方向上的第二代价聚合结果的方式，也可以采用类似于步骤42或步骤52所述的计算多个像素在第一方向上的第一代价聚合结果的方式。具体方式参见前述实施例的相关描述，在此不再赘述。
对应于图12B，假设第二区块为图12B中的区块5，且第一方向为L0_pd至L3_pd四个方向，第二方向为L4_pd至L7_pd，则区块5包括的像素可以如图13所示。参考图13，区块5的第一重叠区域具体可以为黑粗虚线框框出的区块5的右部分区域和下部分区域，区块5的第二重叠区域具体可以为黑粗虚线框框出的区块5的左部分区域和上部分区域，区块5的待聚合区域具体可以为黑粗 实线框框出的第一区块中间的区域。
本申请实施例提供的图像处理方法，通过基于第一区块的第一重叠区域和待聚合区域中像素的匹配代价，计算得到第一区块的待聚合区域中的像素在第一方向上的代价聚合结果，计算得到位于第二区块的待聚合区域中的多个第二像素在第二方向上的第二代价聚合结果，实现了针对目标图像中的区块在不同方向上的代价聚合，从而能够获得像素的最终代价聚合结果(即如下的第三代价聚合结果)。
在另一个实施例中，所述第二方向为多个预设方向情况下，所述计算得到位于所述第二区块的待聚合区域中的多个第二像素在第二方向上的第二代价聚合结果，具体可以包括：
从存储器中读取所述多个第二像素中部分像素在所述多个预设方向中的至少一个所述预设方向上的单方向代价聚合结果，并基于所述部分像素的所述单方向代价聚合结果计算得到所述多个第二像素中其他像素的第二代价聚合结果，所述部分像素的单方向代价聚合结果是通过先前处理获得并存储在所述存储器中。
其中，所述单方向代价聚合结果是指在单个方向上的代价聚合结果。以至少一个所述预设方向为L0_pd方向和L1_pd方向为例，可以从存储器中读取多个第二像素中部分像素在L0_pd方向上的代价聚合结果，以及，从存储器中读取多个第二像素中部分像素在L1_pd方向上的代价聚合结果。其中，L0_pd方向和L1_pd方向分别为不同的单个方向上的代价聚合结果。
所述先前处理可以基于前述公式2完成。以图3所示的8个方向为例，假设第二方向包括图3中的左上部分的4个方向，即L0_pd至L3_pd，可以基于前述公式2给出的计算某一像素p在r方向上的单方向代价聚合结果，计算目标图像中所有像素分别在L0_pd至L3_pd方向上分别的单方向代价聚合结果，并将部分像素的L0_pd至L3_pd中至少一个方向的单方向代价聚合结果存储在存储器中。
可以看出，在先前处理过程中，第二区块的待聚合区域中所有像素在各第二方向上的单方向代价聚合结果均能够计算获得，只是选取了其中部分像素在至少一个第二方向上的单方向代价聚合结果进行保存。之后，可以基于存储器中保存的第二区块的待聚合区域中部分像素在至少一个第二方向上的单方向代价聚合结果，恢复得到第二区块的待聚合区域中其他像素在各个第 二方向(例如，预设方向)上的单方向代价聚合结果，以基于第二区块的待聚合区域中其他像素在各个第二方向(例如，预设方向)上的单方向代价聚合结果计算得到第二区块的待聚合区域中其他像素的第二代价聚合结果。
在一个实施例中，为了尽可能的减少对于集成电路的内部存储器的占用，所述存储器可以为所述集成电路的外部存储器。当然，在其他实施例中，所述存储器还可以为其他存储器，本申请对此不做限定。
这里，介绍一下可以基于位于第二区块的待聚合区域中的多个第二像素中部分像素的第二代价聚合结果，计算得到位于第二区块的待聚合区域中的多个第二像素中其他像素的第二代价聚合结果的理论依据。
通过公式2可以看出，对于图像中任一像素在单一方向上的代价聚合结果来说，其由两部分来决定。具体来说，该代价聚合结果包括该像素的匹配代价以及在该方向上的上一个像素的代价聚合结果。基于此，以方向L0_pd方向为例，可以将公式2重写为公式3来描述代价聚合的过程：
L0_pd i＝f(C i,L0_pd i-1)   (公式3)
其中，i为图像中的当前像素，C i表示当前像素i的匹配代价，L0_pd i-1表示在L0_pd方向上的上一个像素i-1的代价聚合结果，L0_pd i表示L0_pd方向的当前像素i的代价聚合结果。
通过公式3可以看出，某一像素p在某一方向上的代价聚合结果可以基于该像素p在该方向上的上一个像素的代价聚合结果确定，由此能够基于位于第二区块的待聚合区域中的多个第二像素中部分像素的第二代价聚合结果，计算得到位于第二区块的待聚合区域中的多个第二像素中其他像素的第二代价聚合结果。例如，当多个第二像素中的至少一个像素为其他像素在某一方向上的上一个像素时，可以依据该至少一个像素的第二代价聚合结果，来计算其他像素的第二代价聚合结果。
当将图3中左上部分的L0_pd至L3_pd中的任何一个方向作为第二方向时，像素在第二方向上的第二代价聚合结果可以表示为：
Figure PCTCN2020100101-appb-000003
在一个实施例中，第二方向可以为多个预设方向，当该多个预设方向包括向下方向时，所述部分像素可以包括所述第二区块的待聚合区域中位于上边缘的像素。其中，所述向下方向可以包括垂直向下方向和斜向下方向。以 图3所示的8个方向为例，垂直向下方向例如可以为图3中的L1_pd方向，斜向下方向例如可以为图3中的L0_pd方向和L2_pd方向。
参考图14，在第二方向包括L1_pd方向情况下，像素I在L1_pd方向上的代价聚合结果可以基于像素I在L1_pd方向上的上一像素(即像素I3)在L1_pd方向上的代价聚合结果确定，像素I3在L1_pd方向上的代价聚合结果可以基于像素I3在L1_pd方向上的上一像素(即像素I2)在L1_pd方向上的代价聚合结果确定，像素I2在L1_pd方向上的代价聚合结果可以基于像素I2在L1_pd方向上的上一像素(即像素I1)在L1_pd向上的代价聚合结果确定，而像素I1在L1_pd方向上的代价聚合结果可以基于像素I1在L1_pd方向上的上一像素(即区块上边缘的像素I00)在L1_pd方向上的代价聚合结果确定。可以看出，第二区块的待聚合区域上边缘的像素在垂直向下方向上的代价聚合结果可以用于确定其他像素在垂直向下方向上的代价聚合结果。因此，在一实施方式中，存储第二区块的待聚合区域上边缘行(例如，像素I00所在的行)的像素在L1_pd方向上的代价聚合结果，以用于确定其他像素在L1_pd方向上的代价聚合结果。其中，其他像素位于第一区块的内部。也就是说，其他像素不位于区块的边缘。
继续参考图14，在第二方向包括L2_pd方向情况下，某一像素I在L2_pd方向上的代价聚合结果可以基于像素I在L2_pd方向上的上一像素(即像素I6)在L2_pd方向上的代价聚合结果确定，像素I6在L2_pd方向上的代价聚合结果可以基于像素I6在L2_pd方向上的上一像素(即像素I5)在L2_pd方向上的代价聚合结果确定，像素I5在L2_pd方向上的代价聚合结果可以基于像素I5在L2_pd方向上的上一像素(即像素I4)在L2_pd向上的代价聚合结果确定，而像素I4在L2_pd方向上的代价聚合结果可以基于像素I4在L2_pd方向上的上一像素(即区块上边缘的像素I01)在L2_pd方向上的代价聚合结果确定。可以看出，第二区块的待聚合区域上边缘的像素在斜向下方向上的代价聚合结果可以用于确定其他像素在斜向下方向上的代价聚合结果。因此，在一实施方式中，存储第二区块的待聚合区域上边缘行(例如，像素I01所在的行)的像素在L2_pd方向上的代价聚合结果，以用于确定其他像素在L2_pd方向上的代价聚合结果。其中，其他像素位于第一区块的内部。也就是说，其他像素不位于区块的边缘。
基于与L2_pd方向类似的原理，针对L0_pd方向，在一实施方式中，存储第二区块的待聚合区域上边缘行的像素在L0_pd方向上的代价聚合结果，以用 于确定其他像素在L0_pd方向上的代价聚合结果。其中，其他像素位于第一区块的内部。也就是说，其他像素不位于区块的边缘。
在另一个实施例中，第二方向可以为多个预设方向，当该多个预设方向包括向右方向时，所述部分像素可以包括所述第二区块的待聚合区域中位于左边缘的像素。其中，所述向右方向可以包括水平向右方向和斜向右方向。以图3所示的8个方向为例，水平向右方向例如可以为图3中的L3_pd方向，斜向右方向例如可以为图3中的L2_pd方向和L4_pd方向。
参考图15，在第二方向包括L3_pd方向情况下，像素J在L3_pd方向上的代价聚合结果可以基于像素J在L3_pd方向上的上一像素(即像素J3)在L3_pd方向上的代价聚合结果确定，像素J3在L3_pd方向上的代价聚合结果可以基于像素J3在L3_pd方向上的上一像素(即像素J2)在L3_pd方向上的代价聚合结果确定，像素J2在L3_pd方向上的代价聚合结果可以基于像素J2在L3_pd方向上的上一像素(即像素J1)在L3_pd向上的代价聚合结果确定，而像素J1在L3_pd方向上的代价聚合结果可以基于像素J1在L3_pd方向上的上一像素(即区块左边缘的像素J00)在L3_pd方向上的代价聚合结果确定。可以看出，第二区块的待聚合区域左边缘的像素在水平向右方向上的代价聚合结果可以用于确定其他像素在水平向右方向上的代价聚合结果。因此，在一实施方式中，存储第二区块的待聚合区域左边缘行(例如，像素J00所在的行)的像素在L3_pd方向上的代价聚合结果，以用于确定其他像素在L3_pd方向上的代价聚合结果。其中，其他像素位于第一区块的内部。也就是说，其他像素不位于区块的边缘。
继续参考图15，在第二方向包括L2_pd方向情况下，某一像素J在L2_pd方向上的代价聚合结果可以基于像素J在L2_pd方向上的上一像素(即像素J6)在L2_pd方向上的代价聚合结果确定，像素J6在L2_pd方向上的代价聚合结果可以基于像素J6在L2_pd方向上的上一像素(即像素J5)在L2_pd方向上的代价聚合结果确定，像素J5在L2_pd方向上的代价聚合结果可以基于像素J5在L2_pd方向上的上一像素(即像素J4)在L2_pd向上的代价聚合结果确定，而像素J4在L2_pd方向上的代价聚合结果可以基于像素J4在L2_pd方向上的上一像素(即区块左边缘的像素J01)在L2_pd方向上的代价聚合结果确定。可以看出，第二区块的待聚合区域左边缘的像素在斜向右方向上的代价聚合结果可以用于确定其他像素在斜向右方向上的代价聚合结果。因此， 在一实施方式中，存储第二区块的待聚合区域左边缘行(例如，像素J01所在的行)的像素在L2_pd方向上的代价聚合结果，以用于确定其他像素在L2_pd方向上的代价聚合结果。其中，其他像素位于第一区块的内部。也就是说，其他像素不位于区块的边缘。
基于与L2_pd方向类似的原理，针对L4_pd方向，在一实施方式中，存储第二区块的待聚合区域左边缘行的像素在L4_pd方向上的代价聚合结果，以用于确定其他像素在L4_pd方向上的代价聚合结果。其中，其他像素位于第一区块的内部。也就是说，其他像素不位于区块的边缘。
在又一个实施例中，第二方向可以为多个预设方向，当该多个预设方向包括向上方向时，所述部分像素可以包括所述第二区块的待聚合区域中位于下边缘的像素。其中，所述向上方向可以包括垂直向上方向和斜向上方向。以图3所示的8个方向为例，垂直向上方向例如可以为图3中的L5_pd方向，斜向上方向例如可以为图3中的L4_pd方向和L6_pd方向。
参考图16，在第二方向包括L5_pd方向情况下，像素K在L5_pd方向上的代价聚合结果可以基于像素K在L5_pd方向上的上一像素(即像素K3)在L5_pd方向上的代价聚合结果确定，像素K3在L5_pd方向上的代价聚合结果可以基于像素K3在L5_pd方向上的上一像素(即像素K2)在L5_pd方向上的代价聚合结果确定，像素K2在L5_pd方向上的代价聚合结果可以基于像素K2在L5_pd方向上的上一像素(即像素K1)在L5_pd向上的代价聚合结果确定，而像素K1在L5_pd方向上的代价聚合结果可以基于像素K1在L5_pd方向上的上一像素(即区块下边缘的像素K00)在L5_pd方向上的代价聚合结果确定。可以看出，第二区块的待聚合区域下边缘的像素在垂直向上方向上的代价聚合结果可以用于确定其他像素在垂直向上方向上的代价聚合结果。因此，在一实施方式中，存储第二区块的待聚合区域下边缘行(例如，像素K00所在的行)的像素在L5_pd方向上的代价聚合结果，以用于确定其他像素在L5_pd方向上的代价聚合结果。其中，其他像素位于第一区块的内部。也就是说，其他像素不位于区块的边缘。
继续参考图16，在第二方向包括L4_pd方向情况下，像素K在L4_pd方向上的代价聚合结果可以基于像素K在L4_pd方向上的上一像素(即像素K6)在L4_pd方向上的代价聚合结果确定，像素K6在L4_pd方向上的代价聚合结果可以基于像素K6在L4_pd方向上的上一像素(即像素K5)在L4_pd方 向上的代价聚合结果确定，像素K5在L4_pd方向上的代价聚合结果可以基于像素K5在L4_pd方向上的上一像素(即像素K4)在L4_pd向上的代价聚合结果确定，而像素K4在L4_pd方向上的代价聚合结果可以基于像素K4在L4_pd方向上的上一像素(即区块下边缘的像素K01)在L4_pd方向上的代价聚合结果确定。可以看出，第二区块的待聚合区域下边缘的像素在斜向上方向上的代价聚合结果可以用于确定其他像素在斜向上方向上的代价聚合结果。因此，在一实施方式中，存储第二区块的待聚合区域下边缘行(例如，像素K01所在的行)的像素在L4_pd方向上的代价聚合结果，以用于确定其他像素在L4_pd方向上的代价聚合结果。其中，其他像素位于第一区块的内部。也就是说，其他像素不位于区块的边缘。
基于与L4_pd方向类似的原理，针对L6_pd方向，在一实施方式中，存储第二区块的待聚合区域下边缘行的像素在L6_pd方向上的代价聚合结果，以用于确定其他像素在L6_pd方向上的代价聚合结果。其中，其他像素位于第一区块的内部。也就是说，其他像素不位于区块的边缘。
在又一个实施例中，第二方向可以为多个预设方向，当该多个预设方向包括向左方向时，所述部分像素包括所述第二区块的待聚合区域中位于右边缘的像素。其中，所述向左方向可以包括水平向左方向和斜向左方向。以图3所示的8个方向为例，水平向左方向例如可以为图3中的L7_pd方向，斜向左方向例如可以为图3中的L0_pd方向和L6_pd方向。
参考图17，在第二方向包括L7_pd方向情况下，某一像素L在L7_pd方向上的代价聚合结果可以基于像素L在L7_pd方向上的上一像素(即像素L3)在L7_pd方向上的代价聚合结果确定，像素L3在L7_pd方向上的代价聚合结果可以基于像素L3在L7_pd方向上的上一像素(即像素L2)在L7_pd方向上的代价聚合结果确定，像素L2在L7_pd方向上的代价聚合结果可以基于像素L2在L7_pd方向上的上一像素(即像素L1)在L7_pd向上的代价聚合结果确定，而像素L1在L7_pd方向上的代价聚合结果可以基于像素L1在L7_pd方向上的上一像素(即区块右边缘的像素L00)在L7_pd方向上的代价聚合结果确定。可以看出，第二区块的待聚合区域右边缘的像素在水平向左方向上的代价聚合结果可以用于确定其他像素在水平向左方向上的代价聚合结果。因此，在一实施方式中，存储第二区块的待聚合区域右边缘行(例如，像素L00所在的行)的像素在L7_pd方向上的代价聚合结果，以用于确定其 他像素在L7_pd方向上的代价聚合结果。其中，其他像素位于第一区块的内部。也就是说，其他像素不位于区块的边缘。
继续参考图17，在第二方向包括L6_pd方向情况下，某一像素L在L6_pd方向上的代价聚合结果可以基于像素L在L6_pd方向上的上一像素(即像素L6)在L6_pd方向上的代价聚合结果确定，像素L6在L6_pd方向上的代价聚合结果可以基于像素L6在L6_pd方向上的上一像素(即像素L5)在L6_pd方向上的代价聚合结果确定，像素L5在L6_pd方向上的代价聚合结果可以基于像素L5在L6_pd方向上的上一像素(即像素L4)在L6_pd向上的代价聚合结果确定，而像素L4在L6_pd方向上的代价聚合结果可以基于像素L4在L6_pd方向上的上一像素(即区块右边缘的像素L01)在L6_pd方向上的代价聚合结果确定。可以看出，第二区块的待聚合区域右边缘的像素在斜向左方向上的代价聚合结果可以用于确定其他像素在斜向左方向上的代价聚合结果。因此，在一实施方式中，存储第二区块的待聚合区域右边缘行(例如，像素L01所在的行)的像素在L6_pd方向上的代价聚合结果，以用于确定其他像素在L6_pd方向上的代价聚合结果。其中，其他像素位于第一区块的内部。也就是说，其他像素不位于区块的边缘。
基于与L7_pd方向类似的原理，针对L0_pd方向，在一实施方式中，存储第二区块的待聚合区域右边缘行的像素在L0_pd方向上的代价聚合结果，以用于确定其他像素在L0_pd方向上的代价聚合结果。其中，其他像素位于第一区块的内部。也就是说，其他像素不位于区块的边缘。
基于前述图14至图17的分析可知，以第二方向为图3中左上部分的L0_pd至L3_pd为例，如图18所示，第二区块的待聚合区域中用于恢复其他像素的单方向代价聚合结果的部分像素具体可以包括黑色框表示的位于第二区块的待聚合区域上边缘的像素，以及灰色框表示的位于第二区块的待聚合区域左边缘的像素。第二区块的待聚合区域中的其他像素具体可以为白色框表示的像素。需要说明的是，图18中像素点不同的填充色，表示在存储器中需要存储的代价聚合结果对应的方向不同。针对图18中黑色框位置的像素即第二区块的待聚合区域上边缘的像素，存储设备中需要存储分别在L0_pd、L1_pd、L2_pd三个方向上的代价聚合结果；针对图18中灰色框位置的像素即第二区块的待聚合区域左边缘的像素，存储设备中需要存储分别在L2_pd、L3_pd两个方向上的代价聚合结果。针对图18中灰色框与黑色框交叠位置的 像素(即像素1801)，存储设备中需要存储分别存储在L0_pd、L1_pd、L2_pd和L3_pd四个方向上的代价聚合结果。其中，灰色框与黑色框交叠位置的像素为同时位于上边缘和左边缘的像素。
本申请实施例中，可以采用逐行或者逐点的方式来确定第二区块的待聚合区域中其他像素在各第二方向上的单方向代价聚合结果。需要说明的是，在一个实施方式中，当第二方向为L0_pd～L7_pd时，可以采用与上述方式相似的方式来获得位于区块内部的像素在第二方向上的代价聚合结果。为求简洁，不再赘述。
对应于图18，在采用逐行的方式时可以根据第二区块的待聚合区域中上边缘和左边缘的像素在至少一个第二方向上的单方向代价聚合结果，并行按照逐行和列的方式确定第二区块的待聚合区域中其他像素在各第二方向上的单方向代价聚合结果。
在实际应用中，参考图19，可以由第1行(1901)中的像素在L0_pd、L1_pd、L2_pd方向上分别的单方向代价匹配结果和第2行(1902)中像素的匹配代价(也即前文所述的matching cost)，分别计算第2行中的像素在L0_pd、L1_pd、L2_pd方向上分别的单方向匹配代价结果。还可以通过第1列中的像素在L3_pd方向上的单方向代价聚合结果和第2列中像素的匹配代价，计算第2列中像素在L3_pd方向上的单方向代价聚合结果。
L0_pd至L3_pd不同方向上的单方向代价聚合结果的计算过程之间是相互独立的，因此可以并行计算不同方向上的单方向代价聚合结果。例如，一个时钟周期内可以完成第2行中的像素分别在L0_pd、L1_pd、L2_pd方向上的单方向代价聚合结果以及第2列中的像素在L3_pd方向上的单方向代价聚合结果的恢复，对于图19所示的第二区块的待聚合区域来说，仅较少的时间即可完成第二区块的待聚合区域中其他像素在各个第二方向上的单方向代价聚合结果的恢复计算。对于逐行恢复其他像素的单方向代价聚合结果的方式来说，计算速度较快。
值得说明的是，以图19所示的第二区块的待聚合区域为例，假设每个单方向代价匹配结果的数据量为16比特，那么可以在集成电路的内部存储器中开辟4块16×16×16比特的存储空间，用于存储第二区块的待聚合区域中第二像素在L0_pd-L3_pd方向上的单方向代价匹配结果，其中4块16×16×16比特的存储空间对应于L0_pd-L3_pd 4个方向。
对应于图18，在采用逐点的方式时可以根据第二区块的待聚合区域中上边缘和左边缘的像素在至少一个第二方向上的单方向代价聚合结果，按照逐点的方式确定第二区块的待聚合区域中其他像素在各第二方向上的单方向代价聚合结果。
在实际应用中，参考图19，假设第二区块的待聚合区域左上角的像素的位置为(0,0)，可以先由位置(1,0)的像素在L3_pd方向上的单方向代价匹配结果计算出位置(1,1)的像素在L3_pd方向上的单方向代价匹配结果，通过位置(0,1)的像素分别在L0_pd、L1_pd、L2_pd方向上的单方向代价匹配结果计算出位置(1,1)的像素分别在L0_pd、L1_pd、L2_pd方向上的单方向代价匹配结果。在计算出位置(1,1)的像素分别在L0_pd至L3_pd方向上的单方向代价匹配结果之后，可以将它们相加。
可以理解的是，在计算出位置(1,1)的像素在L0_pd至L3_pd方向上的单方向代价匹配结果的相加结果之后，可以将该相加结果存储到集成电路的内部存储器中。通过这样的方式，最后能够计算出第二区块待聚合区域中部分像素和其他像素在L0_pd至L3_pd方向上的单方向代价匹配结果的相加结果(即第二代价聚合结果)，并将第二代价聚合结果存储到存储器中。仍以图19所示的第二区块的待聚合区域为例，假设每个单方向代价匹配结果的数据量为16比特，那么可以在集成电路的内部存储器中开辟1块16×16×16比特的存储空间，用于存储第二区块的待聚合区域中第二像素的第二代价聚合结果。相比于逐行恢复第二区块的单方向代价聚合结果的方式来说，采用逐点恢复的方式可以直接恢复出单个第二像素分别在L0_pd至L3_pd方向上的单方向代价匹配结果，因此可以直接将这4个方向上的单方向代价匹配结果相加得到单个第二像素的第二代价聚合结果，将第二代价聚合结果存储到集成电路的内部存储器上。而采用逐行恢复的方式，无法同时将单个第二像素分别在L0_pd至L3_pd方向上的单方向代价匹配结果都恢复出，因此需要将单个第二像素分别在L0_pd至L3_pd方向上的单方向代价匹配结果都存储在内部存储器中。从而，对于逐点恢复的方式来说，需要占用的存储空间较小。
需要说明的是，在第二区块的待聚合区域的上边缘的像素是目标图像的上边缘的像素情况下，第二区块的待聚合区域上边缘的像素在向下方向上的代价聚合结果是默认最大值，因此可以无需存储在存储设备中。在第二区块的待聚合区域的左边缘的像素是目标图像的左边缘的像素情况下，第二区块 的待聚合区域左边缘的像素在向右方向上的代价聚合结果是默认最大值，因此可以无需存储在存储设备中。在第二区块的待聚合区域的下边缘的像素是目标图像的下边缘的像素情况下，第二区块的待聚合区域下边缘的像素在向上方向上的代价聚合结果是默认最大值。在第二区块的待聚合区域的右边缘的像素是目标图像的右边缘的像素情况下，第二区块的待聚合区域的右边缘的像素在向左方向上的代价聚合结果是默认最大值。在一个实施方式中，上述默认最大值可以不采用与计算得到的代价聚合结果相同的存储方式进行存储。例如，无需将默认最大值存储在计算得到的代价聚合结果对应的存储设备中。
基于此，针对于图12B所示的9个区块，假设图12B所示的9个区块的待聚合区域组成了一张完整的目标图像，第二方向包括L0-pd至L3-pd，并且第二区块为图12B中的区块1～9中的任意一个，则第二区块的待聚合区域中的部分像素可以为图20中黑色框位置的像素和灰色框位置的像素。参考图20，由于第二方向包括L0-pd至L3-pd，因此理论上第二区块的待聚合区域中的部分像素为第二区块的待聚合区域上边缘和左边缘的像素，且应将部分像素分别在L0-pd至L3-pd方向上的单方向代价聚合结果保存在存储器中。然而，由于区块1、区块4和区块7的待聚合区域左边缘的像素均是目标图像的边缘的像素，因此区块1、区块4和区块7的待聚合区域左边缘的像素分别在L0-pd至L3-pd方向上的单方向代价聚合结果可以直接取默认值。由于区块1、区块2和区块3的待聚合区域上边缘的像素均是目标图像的边缘的像素，因此区块1、区块2和区块3的待聚合区域上边缘的像素分别在L0-pd至L3-pd方向上的单方向代价聚合结果可以直接取默认值。另外，由于区块5、区块6、区块8和区块9的待聚合区域左边缘和上边缘的像素均不是目标图像的边缘的像素，因此区块5、区块6、区块8和区块9的待聚合区域上边缘和左边缘的像素分别在L0-pd至L3-pd方向上的单方向代价聚合结果不可以直接取默认值。
需要说明的是，图20中以将目标图像等分为待聚合区域的大小均为16×16像素的9个区块为例。
本申请实施例提供的图像处理方法，通过从存储器中读取第二区块的待聚合区域中的多个第二像素中部分像素在多个预设方向中的至少一个预设方向上的单方向代价聚合结果，并基于该部分像素的单方向代价聚合结果计算得到该多个第二像素中其他像素的第二代价聚合结果，实现了基于一区块的 待聚合区域中的部分像素在至少一个方向上的单方向代价聚合结果，计算得到该区块的待聚合区域中其他像素的第一代价聚合结果的方式。与仅基于公式2计算得到区块的待聚合区域中像素的第二代价聚合结果相比，上述方法扩大了计算像素的第二代价聚合结果的方法范围，提高了计算像素的第二代价聚合结果的灵活性。
在又一个实施例中，可以根据预定的处理顺序来计算所述多个区块各自对应的第一代价聚合结果和第二代价聚合结果，即所述多个区块之间存在预定的处理顺序，第二方向为多个预设方向；所述计算得到位于所述多个区块中第二区块的待聚合区域中的多个第二像素在第二方向上的第二代价聚合结果，具体可以包括：
基于所述多个区块中目标区块的待聚合区域中的部分边缘像素在所述多个预设方向中的至少一个所述预设方向上的单方向代价聚合结果，计算得到位于所述多个区块中第二区块的待聚合区域中多个第二像素在第二方向上的第二代价聚合结果。
其中，第二代价聚合结果是指像素的匹配代价在第二方向上代价聚合的结果。可以理解的是，在第二方向为多个预设方向情况下，第二代价聚合结果可以为该多个预设方向中各方向上的单方向代价聚合的结果之和。以第二方向为图3所示的L4_pd至L7_pd为例，目标区块的待聚合区域中一像素的第二代价聚合结果可以为该像素在L4_pd、L5_pd、L6_pd以及L7_pd分别的单方向代价聚合结果的和。
所述预定的处理顺序中所述目标区块位于所述第二区块之前，且所述目标区块的待聚合区域与第二区块的待聚合区域位置相邻。
针对于图12B所示的9个区块，假设图12B所示的9个区块的待聚合区域组成了一张完整的目标图像，且第二方向包括L0-pd至L3-pd，则图12B所示的9个区块的处理顺序例如可以为区块1→区块2→区块3→区块4→区块5→区块6→区块7→区块8→区块9。基于此，以第二区块为图12B中的区块2为例，第二区块的目标区块可以为区块1。以第二区块为图12B中的区块5为例，第二区块的目标区块可以为区块2和区块4。
在一个实施例中，在第二方向为多个预设方向情况下，所述多个预设方向包括向上方向，所述部分边缘像素包括所述目标区块的待聚合区域中位于下边缘的像素。
在另一个实施例中，在第二方向为多个预设方向情况下，所述多个预设方向包括向下方向，所述部分边缘像素包括所述目标区块的待聚合区域中位于上边缘的像素。
在又一个实施例中，在第二方向为多个预设方向情况下，所述多个预设方向包括向左方向，所述部分边缘像素包括所述目标区块的待聚合区域中位于右边缘的像素。
在又一个实施例中，在第二方向为多个预设方向情况下，所述多个预设方向包括向右方向，所述部分边缘像素包括所述目标区块的待聚合区域中位于左边缘的像素。
需要说明的是，关于目标区块中的部分边缘像素与第二方向的关系，与图14-图17所示的第二区块的部分像素与第二方向的关系类似，在此不再赘述。
基于此，针对于图12B所示的9个区块，假设图12B所示的9个区块的待聚合区域组成了一张完整的目标图像，第二方向包括L0-pd至L3-pd，9个区块的处理顺序例如可以为区块1→区块2→区块3→区块4→区块5→区块6→区块7→区块8→区块9，则第二区块的目标区块可以为图21中的区块1-区块8，目标区块中的部分边缘像素可以为图21中黑色框位置的像素和灰色框位置的像素。参考图21，区块1的待聚合区域中右边缘像素在L2-pd、L3-pd上的单方向代价聚合结果可以用于计算区块2的待聚合区域中像素的第二代价聚合结果，区块1的待聚合区域中下边缘像素在L0-pd、L1-pd、L2-pd上的单方向代价聚合结果可以用于计算区块4的待聚合区域中像素的第二代价聚合结果。区块2的待聚合区域中右边缘像素在L2-pd、L3-pd上的单方向代价聚合结果可以用于计算区块3的待聚合区域中像素的第二代价聚合结果，区块2的待聚合区域中下边缘像素在L0-pd、L1-pd、L2-pd上的单方向代价聚合结果可以用于计算区块5的待聚合区域中像素的第二代价聚合结果。区块3的待聚合区域中下边缘像素在L0-pd、L1-pd、L2-pd上的单方向代价聚合结果可以用于计算区块6的待聚合区域中像素的第二代价聚合结果。区块4的待聚合区域中右边缘像素在L2-pd、L3-pd上的单方向代价聚合结果可以用于计算区块5的待聚合区域中像素的第二代价聚合结果，区块4的待聚合区域中下边缘像素在L0-pd、L1-pd、L2-pd上的单方向代价聚合结果可以用于计算区块7的待聚合区域中像素的第二代价聚合结果。区块5的待聚合区域中右边缘像素在L2-pd、L3-pd上的单方向代价聚合结果可以用于计算区块6的待聚合区域中像素的第二代价聚合结果，区 块5的待聚合区域中下边缘像素在L0-pd、L1-pd、L2-pd上的单方向代价聚合结果可以用于计算区块8的待聚合区域中像素的第二代价聚合结果。区块6的待聚合区域中下边缘像素在L0-pd、L1-pd、L2-pd上的单方向代价聚合结果可以用于计算区块9的待聚合区域中像素的第二代价聚合结果。区块7的待聚合区域中右边缘像素在L2-pd、L3-pd上的单方向代价聚合结果可以用于计算区块8的待聚合区域中像素的第二代价聚合结果。区块8的待聚合区域中右边缘像素在L2-pd、L3-pd上的单方向代价聚合结果可以用于计算区块9的待聚合区域中像素的第二代价聚合结果。
由此可以看出，区块1作为第二区块时，预设的处理顺序中区块1之前不存在其他区块，因此区块1的目标区块为空；区块2作为第二区块时，预设的处理顺序中位于区块2之前且待聚合区域与区块2的待聚合区域相邻的区块为区块1，因此区块1为区块2的目标区块；区块3作为第二区块时，预设的处理顺序中位于区块3之前且待聚合区域与区块3的待聚合区域相邻的区块为区块2，因此区块2为区块3的目标区块；区块4作为第二区块时，预设的处理顺序中位于区块4之前且待聚合区域与区块4的待聚合区域相邻的区块为区块1，因此区块1为区块4的目标区块；区块5作为第二区块时，预设的处理顺序中位于区块5之前且待聚合区域与区块5的待聚合区域相邻的区块为区块2和区块4，因此区块2和区块4为区块5的目标区块；区块6作为第二区块时，预设的处理顺序中位于区块6之前且待聚合区域与区块6的待聚合区域相邻的区块为区块3和区块5，因此区块3和区块5为区块6的目标区块；区块7作为第二区块时，预设的处理顺序中位于区块7之前且待聚合区域与区块7的待聚合区域相邻的区块为区块4，因此区块4为区块7的目标区块；区块8作为第二区块时，预设的处理顺序中位于区块8之前且待聚合区域与区块8的待聚合区域相邻的区块为区块5和区块7，因此区块5和区块7为区块8的目标区块；区块9作为第二区块时，预设的处理顺序中位于区块9之前且待聚合区域与区块9的待聚合区域相邻的区块为区块6和区块8，因此区块6和区块8为区块9的目标区块。
需要说明的是，图21中以将目标图像等分为待聚合区域的大小均为16×16像素的9个区块为例。
在一个实施例中，所述目标区块包括第一目标区块，在所述预定的处理顺序中所述第一目标区块与所述第二区块相邻；所述第一目标区块的待聚合区域中的部分边缘像素在至少一个所述第二方向上的单方向代价聚合结果存 储在集成电路的内部存储器中。其中，部分边缘像素在至少一个所述第二方向上的单方向代价聚合结果是由集成电路中的计算单元计算得到的。以第二区块为图12B中的区块5为例，第一目标区块可以为区块4。
由于在预定的处理顺序中第一目标区块是与第二区块相邻，即在计算第一目标区块的第二代价聚合结果之后，因此在与第一目标区块相邻的下一个区块(即第二区块)中需要使用第一目标区块的待聚合区域中的部分边缘像素在至少一个第二方向上的单方向代价聚合结果，通过将第一目标区块的待聚合区域中的部分边缘像素在至少一个第二方向上的单方向代价聚合结果存储在内部存储器，能够便于数据获取，减少对于集成电路与外部存储器之间带宽的占用，减少处理耗时。
在一个实施例中，所述目标区块包括第二目标区块，在所述预定的处理顺序中所述第二目标区块与所述第二区块间隔；所述第二目标区块的待聚合区域中的部分边缘像素在至少一个所述第二方向上的单方向代价聚合结果存储在所述集成电路的外部存储器中。其中，部分边缘像素在至少一个所述第二方向上的单方向代价聚合结果是由集成电路中的计算单元计算得到的。以第二区块为图12B中的区块5为例，第二目标区块可以为区块2。
由于在预定的处理顺序中第二目标区块是与第二区块间隔，即未与第二区块相邻，即在计算第二目标区块的第二代价聚合结果之后，因此在与第二目标区块相邻的下一个区块(即第二区块之外的其他区块)中并不需要使用第二目标区块的待聚合区域中的部分边缘像素在至少一个第二方向上的单方向代价聚合结果，通过将第二目标区块的待聚合区域中的部分边缘像素在至少一个第二方向上的单方向代价聚合结果存储在外部存储器，能够减少对于内部存储器的占用。
本申请实施例提供的图像处理方法，通过基于多个区块中目标区块的待聚合区域中的部分边缘像素在多个预设方向中的至少一个预设方向上的单方向代价聚合结果，计算得到位于第二区块的待聚合区域中多个第二像素在第二方向上的第二代价聚合结果，实现了在计算第二聚合代价的过程中，基于计算得到的之前区块的待聚合区域中部分边缘像素在至少一个预设方向上的单方向代价聚合结果，计算其他区块的待聚合区域中像素在第二方向上的第二代价聚合结果，与需要先进行先前处理获得以获得第二区块的待聚合区域中部分像素在至少一个预设方向上的单方向代价聚合结果，以恢复得到第二 区块的待聚合区域中其他像素在至少一个预设方向上的单方向代价聚合结果，从而计算得到二区块的待聚合区域中像素在第二方向上的第二代价聚合结果相比，省去了先前处理，减少了计算量，减少了处理时长，提高了处理帧率。
图22为本申请又一实施例提供的图像处理方法的流程示意图，本实施例前述实施例的基础上，主要描述了采用集成电路中的集成单元进行第一区块的待聚合区域中像素的最终代价聚合计算的一种可选实现方式。如图22所示，本实施例的方法可以包括：
步骤221，采用集成电路中的一组计算单元，基于多个区块中第一区块的第一重叠区域和所述第一区块的待聚合区域中像素的匹配代价，计算得到位于所述第一区块的待聚合区域中的多个第一像素在第一方向上的第一代价聚合结果。
需要说明的是，关于计算得到第一代价聚合结果的具体方式，可以参见前述实施例的相关描述，在此不再赘述。
步骤222，采用该组计算单元，计算得到位于所述第一区块的待聚合区域中的多个第一像素在第二方向上的第二代价聚合结果。
需要说明的是，本实施例中以步骤221在步骤222之前为例。可以理解的是，在其他实施例中，步骤221也可以在步骤222之后，即先计算第二代价聚合结果，再计算第一代价聚合结果。
需要说明的是，关于计算得到第一区块中像素的第二代价聚合结果的具体方式，与前述实施例所述的计算第二区块中像素的第二代价聚合结果的具体方式类似，在此不再赘述。
步骤223，采用该组计算单元，基于所述第一区块的待聚合区域中的像素在第一方向上的第一代价聚合结果以及在所述第二方向上的第二代价聚合结果，计算得到第一区块的待聚合区域中的像素的第三代价聚合结果。
本申请实施例中，所述第一代价聚合结果、所述第二代价聚合结果以及所述第三代价聚合结果具体可以包括像素在视差范围内各视差下的聚合结果。示例性的，某一像素p在视差d下的第一代价聚合结果可以表示为L 第一方向(p,d)，像素p在视差d下的第二代价聚合结果可以表示为L 第二方向(p,d)，像素p在视差d下的第三代价聚合结果可以表示为L 第一方向+第二方向(p,d)。
基于此，所述基于所述多个第二像素在第一方向上的第一代价聚合结果 和所述多个第二像素在所述第二方向上的第二代价聚合结果，计算得到所述多个第二像素的第三代价聚合结果，具体可以包括：针对于所述多个第二像素中的各像素，将所述像素在各视差下的第一代价聚合结果与所述像素在各所述视差下的第二代价聚合结果对应相加，得到所述像素在各所述视差下的第三代价聚合结果。需要说明的是，其中的第一代价聚合结果和第二代价聚合结果均为代价聚合的中间结果，第三代价聚合结果为最终代价聚合结果。
以视差范围为0到127，第一方向为L0_pd-L3_pd，第二方向为L4_pd-L7_pd为例，某一像素p在视差d下的第一代价聚合结果L 第一方向(p,d)＝L L0_pd(p,d)+L L1_pd(p,d)+L L2_pd(p,d)+L L3_pd(p,d)，像素p在视差d下的第二代价聚合结果L 第二方向(p,d)＝L L4_pd(p,d)+L L5_pd(p,d)+L L6_pd(p,d)+L L7_pd(p,d)，像素p在视差d下的第三代价聚合结果L 第一方向+第二方向(＝pL, L0_pd(p,d)+L L1_) pd(p,d)+L L2_pd(p,d)+L L3_pd(p,d)+L L4_pd(p,d)+L L5_pd(p,d)+L L6_pd(p,d)+L L7_pd(p,d)。其中，L Li_pd(p,d)，i等于0至7，可以表示像素p在方向Li_pd上的代价聚合。
需要说明的是，对于第三代价聚合结果的存储方式，可以根据需求灵活实现，在一个实施例中，可以将第三代价聚合结果存储在集成电路的外部存储器例如DDR中。
可选的，在计算获得第三聚合代价结果之后，还可以基于第三代价聚合结果确定最终视差。基于此，本实施例的方法还可以包括：将所述像素在所述视差范围内聚合结果最小的视差，作为所述像素的最终视差。例如，假设视差d等于0至127中视差10时，L 第一方向+第二方向(p,d)的值最小，则10即为像素p的最终视差。在一个实施例中，可以采用所述第二组计算单元计算像素的最终视差。
针对于图12B所示的9个区块，假设图12B所示的9个区块的待聚合区域组成了一张完整的目标图像，且第一方向包括L4-pd至L7-pd，第二方向包括L0-pd至L3-pd，9个区块的处理顺序为区块1→区块2→区块3→区块4→区块5→区块6→区块7→区块8→区块9。基于此假设，可以先采用该组计算单元计算，基于区块1的第一重叠区域和区块1的待聚合区域中像素的匹配代价，计算得到位于区块1的待聚合区域中的多个像素在第一方向上的第一代价聚合结果，然后采用该组计算单元计算，计算得到位于区块1的待聚合区域中的多个像素在 第二方向上的第二代价聚合结果，之后采用该组计算单元，基于区块1的待聚合区域中的像素在第一方向上的第一代价聚合结果以及在所述第二方向上的第二代价聚合结果，计算得到区块1的待聚合区域中的像素的第三代价聚合结果。进一步的，可以先采用该组计算单元计算，基于区块2的第一重叠区域和区块2的待聚合区域中像素的匹配代价，计算得到位于区块2的待聚合区域中的多个像素在第一方向上的第一代价聚合结果，然后采用该组计算单元计算，计算得到位于区块2的待聚合区域中的多个像素在第二方向上的第二代价聚合结果，之后采用该组计算单元，基于区块2的待聚合区域中的像素在第一方向上的第一代价聚合结果以及在所述第二方向上的第二代价聚合结果，计算得到区块2的待聚合区域中的像素的第三代价聚合结果。……，依次类推，直至计算得到最后一个区块(即区块9)区块的待聚合区域中的像素的第三代价聚合结果。
本申请实施例提供的图像处理方法，采用一组计算单元先计算第一区块的待聚合区域中的像素在第一方向上的第一代价聚合结果，再采用该组计算单元计算该第一区块的待聚合区域中的像素在第二方向上的第二代价聚合结果，最后采用该组计算单元计算该第一区块的待聚合区域中的像素的最终代价聚合结果，实现了先对一区块的待聚合区域中像素的匹配代价在部分方向上进行聚合，再对该区块的待聚合区域中图像中像素的匹配代价在另一部分方向上进行聚合的串行计算方式。
图23为本申请又一实施例提供的图像处理方法的流程示意图，本实施例前述实施例的基础上，主要描述了计算第一区块的待聚合区域中多个第一像素的第一代价聚合结果和计算第二区块的待聚合区域中多个第二像素的第二代价聚合结果计算的一种可选实现方式。如图23所示，本实施例的方法可以包括：
步骤231，采用第一组计算单元，基于多个区块中第一区块的第一重叠区域和所述第一区块的待聚合区域中像素的匹配代价，计算得到位于所述第一区块的待聚合区域中的多个第一像素在第一方向上的第一代价聚合结果。
本步骤中，所述多个区块是通过对目标图像划分得到的部分区域重叠的多个区块。所述多个区块之间存在一定的处理顺序。所述处理顺序可以包括针对第一组计算单元的第一处理顺序。以将目标图像划分为图12B所示的9个区块，多个区块具体可以为图12B中的区块1至区块9为例，9个区块之间第一 处理顺序例如可以为区块9→区块8→区块7→区块6→区块5→区块4→区块3→区块2→区块1，当然，在其他实施例中，多个区块之间的第一处理顺序还可以为其他形式，本申请对此不做限定。
所述第一组计算单元是集成电路中的一组计算单元，第一组计算单元可以用于计算多个区块中第一区块的待聚合区域中的多个第一像素在第一方向上的第一代价聚合结果。
需要说明的是，关于计算得到第一代价聚合结果的具体方式，可以参见前述实施例的相关描述，在此不再赘述。
步骤232，采用第二组计算单元，计算得到位于第二区块的待聚合区域中的多个第二像素在第二方向上的第二代价聚合结果。
本步骤中，所述多个区块之间的处理顺序还可以包括针对第二组计算单元的第二处理顺序，需要说明的是，在一个实施例中第二处理顺序可以与第一处理顺序一致，在另一个实施例中第二处理顺序可以与第一处理顺序不一致。同样，以将目标图像划分为如图12B所示的9个区块为例，9个区块之间第二处理顺序例如可以为区块1→区块2→区块3→区块4→区块5→区块6→区块7→区块8→区块9，当然，在其他实施例中，多个区块之间的第二处理顺序还可以为其他形式，本申请对此不做限定。
所述第二组计算单元是集成电路中的另一组计算单元，第二组计算单元和第一组计算单元是集成电路中并行处理的两组计算单元，即第一计算单元和第二计算单元可以并行进行计算，与串行计算相比，有利于减少计算耗时。第二组计算单元可以用于计算多个区块中第二区块的待聚合区域中的多个第二像素在第二方向上的第二代价聚合结果。
本申请实施例提供的图像处理方法，通过采用第一组计算单元，基于多个区块中第一区块的第一重叠区域和第一区块的待聚合区域中像素的匹配代价，计算得到位于第一区块的待聚合区域中的多个第一像素在第一方向上的第一代价聚合结果，采用第二组计算单元，计算得到位于第二区块的待聚合区域中的多个第二像素在第二方向上的第二代价聚合结果，其中，第一组计算单元和第二组计算单元是集成电路中并行处理的两组计算单元，实现了在计算位于第一区块的待聚合区域中的第一像素在第一方向上的第一代价聚合结果的同时，对位于第二区块的待聚合区域中第二像素在第二方向上的第二代价聚合结果进行计算，与先对一区块的待聚合区域中像素的匹配代价在部 分方向上进行聚合，再对该区块的待聚合区域中图像中像素的匹配代价在另一部分方向上进行聚合的串行计算方式相比，可减少几乎一半的计算时间，提升近一倍的帧率，从而减少了处理耗时，提高了处理帧率。
可以理解的是，第一代价聚合结果和第二代价聚合结果均是代价聚合的中间结果，为了获得目标图像中某一像素p的最终代价聚合结果，还需要将该像素p的第一代价聚合结果和该像素p的第二代价聚合结果进行相加。因此，在第一处理顺序和第二处理顺序不一致的情况下，需要存储多个区块的代价聚合的中间结果。
以第一处理顺序为区块4→区块3→区块2→区块1，第二处理顺序为区块1→区块2→区块3→区块4为例：在时段1内，第一组计算单元计算得到位于区块4的待聚合区域中多个像素4的第一代价聚合结果a1，第二组计算单元计算得到位于区块1的待聚合区域的多个像素1的第二代价聚合结果b1，需要存储第一代价聚合结果a1和第二代价聚合结果b1。之后，在时段2内，第一组计算单元计算得到位于区块3的待聚合区域中多个像素3的第一代价聚合结果a2，第二组计算单元计算得到位于区块2的待聚合区域中多个像素2的第二代价聚合结果b2，需要存储第一代价聚合结果a2和第二代价聚合结果b2。然后，在时段3内，第一组计算单元计算得到位于区块2的待聚合区域中多个像素2的第一代价聚合结果a3，第二组计算单元计算得到位于区块3的待聚合区域中多个像素3的第二代价聚合结果b3。另外，由于区块2的多个像素2的第二代价聚合结果b2在之前时段已获得，因此时段3内在计算得到区块2的多个像素2的第一代价聚合结果之后，可以直接基于在时段2内计算得到的区块2的该多个像素2的第二代价聚合结果b2和时段3内计算得到的区块2的该多个像素2的第一代价聚合结果a3计算得到该多个像素2的最终代价聚合结果，类似的，由于区块3的该多个像素3的第一代价聚合结果a2在之前时段已获得，因此在时段3计算得到区块3的该多个像素3的第二代价聚合结果之后，可以直接基于在时段2内计算得到的区块3的该多个像素3的第一代价聚合结果a2和时段3内计算得到的区块3的该多个像素3的第二代价聚合结果b3计算得到该多个像素3的最终代价聚合结果，从而无需在存储器中存储区块2的该多个像素2第一代价聚合结果a3和区块3的该多个像素3的第二代价聚合结果b3。最后，在时段4内，第一组计算单元计算得到位于区块1的待聚合区域中该多个像素1的第一代价聚合结果a4，第二组计算单元计算得到位于区块4的待聚合区域中该多个像素4 的第二代价聚合结果b4。另外，由于区块1的该多个像素1的第二代价聚合结果b1在之前时段已获得，因此时段4内在计算得到区块1的待聚合区域中该多个像素1的第一代价聚合结果a4之后，可以直接基于在时段1内计算得到的区块1的待聚合区域中该多个像素1的第二代价聚合结果b1和时段4内计算得到的区块1的该多个像素1的第一代价聚合结果a4计算得到该多个像素1的最终代价聚合结果，类似的，由于区块4的该多个像素4的第一代价聚合结果a1在之前时段已获得，因此在时段4计算得到区块4的该多个像素4的第二代价聚合结果b4之后，可以直接基于在时段1内计算得到的区块4的该多个像素4的第一代价聚合结果a1和时段4内计算得到的区块4的该多个像素4的第二代价聚合结果b4计算得到该多个像素4的最终代价聚合结果，从而无需在存储器中存储区块1的该多个像素1的第一代价聚合结果a4和区块4的该多个像素4的第二代价聚合结果b4。
需要说明的是，时段1至时段4是相邻且时间由早至晚的4个时段。像素1至像素4是为了区分区块1-区块4中不同区块的像素。
可以看出，在时段1至时段4的处理过程中，存在存储设备中需要同时存储多个像素4的第一代价聚合结果a1、多个像素3的第一代价聚合结果a2、多个像素1的第二代价聚合结果b1和多个像素2的第二代价聚合结果b2。因此，一共需要用于存储4个中间结果的、具有特定大小的存储单元，其中，一个特定大小的存储单元等于一个区块的待聚合区域中各像素在第一方向下的第一代价聚合结果的大小之和，或者等于一个区块的待聚合区域中各像素在第二方向下的第二代价聚合结果的大小之和，对存储空间的容量要求较高。即，在第一处理顺序和第二处理顺序不一致的情况下，需要同时存储较多的中间结果，对存储空间的容量要求较高。其中，第一方向和第二方向为多个预设方向，并且第一方向和第二方向包括的方向不相同。
为了降低对于存储空间的容量要求，第一处理顺序和第二处理顺序可以一致，在所述处理顺序中所述第一区块和所述第二区块可以相邻，且处理顺序中第二区块可以位于第一区块之前。例如，第一处理顺序和第二处理顺序均可以为区块1→区块2→区块3→区块4，参考图24：在时间t由t0秒至t1秒的一段时间内，第一组计算单元计算得到位于区块1的待聚合区域中多个像素的第一代价聚合结果a1’，此时，第一代价聚合结果a1’需要被存储。在时间t由t1秒至t2秒的一段时间内，第一组计算单元计算得到位于区块2的待聚合区域 中多个像素的第一代价聚合结果a2’，第二组计算单元计算得到位于区块1的待聚合区域中多个像素的第二代价聚合结果b1’，由于区块1的第一代价聚合结果a1’在之前时段已获得，因此在该时段内可以直接确定出区块1中像素的最终代价聚合结果，无需存储对第二代价聚合结果b1’和第一代价聚合结果a1’。另外，由于在该时间段内，不能得到区块2的最终代价聚合结果，而仅得到区块2的第一代价聚合结果a2’。因此，需要存储第一代价聚合结果a2’。在时间t由t2秒至t3秒的一段时间内，第一组计算单元计算得到位于区块3的待聚合区域中多个像素的第一代价聚合结果a3’，第二组计算单元计算得到位于区块2的待聚合区域中多个像素的第二代价聚合结果b2’，由于区块2的第一代价聚合结果a2’在之前时段已获得，因此在该时段内可以直接确定出区块2的待聚合区域中像素的最终代价聚合结果，无需存储第二代价聚合结果b2’。另外，由于在该时间段内，不能得到区块3的待聚合区域中像素的最终代价聚合结果，而仅得到区块3的第一代价聚合结果a3’，因此需要存储第一代价聚合结果a3’。在时间t由t3秒至t4秒的一段时间内，第一组计算单元计算得到位于区块4的待聚合区域中多个像素的第一代价聚合结果a4’，第二组计算单元计算得到位于区块3的待聚合区域中多个像素的第二代价聚合结果b3’，由于区块3的第一代价聚合结果a3’在之前时段已获得，因此在该时段内可以直接确定出区块3的待聚合区域中像素的最终代价聚合结果，即无需存储第二代价聚合结果b3’。另外，由于在该时间段内，不能得到区块4的待聚合区域中像素的最终代价聚合结果，而仅得到区块4的第一代价聚合结果a4’。因此，需要存储第一代价聚合结果a4’。在时间t由t4秒至t5秒的一段时间内，第二组计算单元计算得到位于区块4的待聚合区域中多个像素的第二代价聚合结果b4’，由于区块4的第一代价聚合结果a4’在之前时段已获得，因此在该时段内可以直接确定出区块4的待聚合区域中像素的最终代价聚合结果，无需存储第二代价聚合结果b4’。
可以看出，在t0秒至t5秒的处理过程中，存储设备中需要同时存储的中间结果的具有特定大小的存储单元的个数小于4。因此，对存储空间的容量要求较小。例如，可以需要同时存储中间结果的2个特定大小的存储单元，具体的，在由t1秒到t2秒的时段，可以存储第一代价聚合结果a1’和第一代价聚合结果a2’。在由t2秒到t3秒的时段，由于第一代价聚合结果a1’已用于计算最终代价聚合结果，因此只需要同时存储第一代价聚合结果a2’和第一代价聚合结 果a3’。在由t3秒到t4秒的时段，由于第一代价聚合结果a2’已用于计算最终代价聚合结果，因此只需要同时存储第一代价聚合结果a3’和第一代价结果a4’。假设单个区块的大小为32×32像素，代价聚合结果的位宽为16比特(bit)且最大视差(disparity)等于128，则同时存储2个特定大小的中间结果的方式，需要4Mbit的存储空间。
图25为本申请又一实施例提供的图像处理方法的流程示意图，本实施例在图23所示实施例的基础上，主要描述了一种确定某一像素p的最终代价聚合结果(即如下的第三代价聚合结果)的一种可选实现方式。如图25所示，本实施例的方法可以包括：
步骤251，采用第一组计算单元，基于多个区块中第一区块的第一重叠区域和所述第一区块的待聚合区域中像素的匹配代价，计算得到位于所述第一区块的待聚合区域中的多个第一像素在第一方向上的第一代价聚合结果。
本步骤中，计算得到的位于第一区块的待聚合区域中多个第一像素在第一方向上的第一代价聚合结果，可以存储在所述集成电路的内部存储器或所述集成电路的外部存储器中。其中，集成电路的内部存储器例如可以为静态随机存取存储器(Static Random-Access Memory，SRAM)，集成电路的外部存储器例如可以为双倍速率同步动态随机存储器(Double Data Rate Synchronous Dynamic Random Access Memory，DDR SDRAM)(简称为DDR)。
需要说明的是，关于采用第一组计算单元计算得到位于第一区块的待聚合区域中多个像素的第一代价聚合结果的具体方式，可以参见前述实施例的相关描述，在此不再赘述。
步骤252，采用第二组计算单元，计算得到位于第二区块的待聚合区域中的多个第二像素在第二方向上的第二代价聚合结果，以及基于所述多个第二像素在第一方向上的第一代价聚合结果和所述多个第二像素在所述第二方向上的第二代价聚合结果，计算得到所述多个第二像素的第三代价聚合结果。
本步骤中，所述第二区块与所述第一区块相邻，且处理顺序中所述第二区块位于所述第一区块之前。关于采用第二组计算单元计算得到位于第二区块的待聚合区域中多个像素的第二代价聚合结果的具体方式，可以参见前述实施例的相关描述，在此不再赘述。
需要说明的是，在执行步骤252之前，已计算得到位于所述第二区块的待聚合区域中多个第二像素在第一方向上的第一代价聚合结果。可选的，可以 是由所述第一组计算单元计算得到位于所述第二区块的待聚合区域中多个第二像素在第一方向上的第一代价聚合结果，或者，可以由所述第一组计算单元之外的其他计算单元计算得到位于所述第二区块的待聚合区域中多个第二像素在第一方向上的第一代价聚合结果。
对于之前已计算得到的位于所述第二区块的待聚合区域中多个第二像素在第一方向上的第一代价聚合结果，需要先存储在所述集成电路的内部存储设备或所述集成电路的外部存储设备中。
在一个实施例中，位于所述第一区块的待聚合区域中的第一像素的第一代价聚合结果和位于所述第二区块的待聚合区域中的第二像素的第一代价聚合结果均存储在所述集成电路的内部存储器中。通过将位于第一区块的待聚合区域中多个第一像素的第一代价聚合结果和位于第二区块的待聚合区域中的多个第二像素的第一代价聚合结果均存储在内部存储器中，与存储在外部存储器相比，有利于提高第一代价聚合结果的存取效率，减少对于集成电路与外部存储器之间带宽的占用。
可选的，位于所述第一区块的待聚合区域中的多个第一像素的第一代价聚合结果和位于所述第二区块的待聚合区域中的多个第二像素的第一代价聚合结果，可以共用所述内部存储器中的同一块存储空间。具体的，在一段时间内，内部存储器中的一存储空间中可以存储位于所述第二区块的待聚合区域中的第二像素的第一代价聚合结果，在与该段时间之后且与该段时间相邻的另一段时间内，内部存储器中的该存储空间中可以存储位于所述第一区块的待聚合区域中的第一像素的第一代价聚合结果。通过位于第一区块的待聚合区域中像素的第一代价聚合结果和位于第二区块的待聚合区域中像素的第一代价聚合结果，共用内部存储器中的同一块存储空间，能够实现需要同时存储的中间结果可以等于1个特定大小，有利于减小进一步聚合的中间结果对于内部存储器的占用，节省内部存储空间的存储资源，降低对于内部存储空间的容量要求，节省成本。假设单个区块的大小为32×32像素，代价聚合结果的位宽为16比特(bit)且最大视差(disparity)等于128，则同时存储1个特定大小的中间结果的方式，需要2Mbit的存储空间。
在一个实施例中，所述第一组计算单元和所述第二组计算单元的单次计算均占用单个时钟周期。示例性的，单次计算可以针对单个像素在全部视差进行计算，或者，单次计算可以针对单个像素在部分视差进行计算，或者， 单次计算可以针对多个像素在全部视差下进行计算。
可选的，在第一组计算单元和第二组计算单元的单次计算均占用单个时钟周期情况下，可以采用如下方式一或方式二实现位于所述第一区块的待聚合区域中的第一像素的第一代价聚合结果和位于所述第二区块的待聚合区域中的第二像素的第一代价聚合结果，共用所述内部存储器中的同一块存储空间。
方式一
一个时钟周期内，所述第一组计算单元计算得到的第一数据，被存入所述内部存储器中该时钟周期内所述第二组计算单元计算使用的第二数据对应的存储空间。其中，所述第一数据为所述第一像素的第一代价聚合结果的数据，所述第二数据为所述第二像素的第一代价聚合结果的数据。由此，可以实现一个时钟周期内，计算得到的第一像素的第一代价聚合结果，占用该时钟周期内计算使用的第二像素的第一代价聚合结果。
需要说明的是，所述数据的大小与单次计算针对的范围相关。示例性的，在单次计算可以针对单个像素在全部视差进行计算的情况下，所述数据的大小可以为单个像素在全部视差下的第一代价聚合结果的大小；在单次计算可以针对单个像素在部分视差进行计算的情况下，所述数据的大小可以为单个像素在部分视差下的第一代价聚合结果的大小；在单次计算可以针对多个像素在全部视差下进行计算的情况下，所述数据的大小可以为多个像素在全部视差下的第一代价聚合结果的大小。
在一个实施例中，所述内部存储器可以包括第一内部存储器，所述第一内部存储器可以为单端口内部存储器，通过所述第一内部存储器是单端口内部存储器，有利于节省成本；或者，第一内部存储器可以为作为单端口内部存储器使用的其他类型内部存储器(例如，双端口内部存储器)，本实施例的方法还可以包括：在第一时钟周期内，从所述第一内部存储器的目标存储空间中读取所述第二数据以及与所述第二数据相邻的第三数据，并缓存所述第一组计算单元计算得到的所述第一数据。其中，所述第二数据用于所述第二组计算单元在所述第一时钟周期内计算第三代价聚合结果；所述第三数据为所述第二像素的第一代价聚合结果的数据，所述第三数据用于所述第二组计算单元在第二时钟周期内计算第三代价聚合结果；所述第二时钟周期为所述第一时钟周期的下一个时钟周期。
基于此，可以实现通过在第一时钟周期内单次读取第一内部存储器，可以读取到用于第二组计算单元在相邻的第一时钟周期和第二时钟周期内计算第三代价聚合结果的第一代价聚合结果，即每隔一个时钟周期从第一内部存储器中进行一次读取，有利于减少针对第一内部存储器的读取次数，提高读取效率。
例如，如图26所示，在时钟周期1内，可以从第一内部存储器中读取用于第二组计算单元在时钟周期1内进行计算的第二像素的第一代价聚合结果的数据K1，以及用于第二组计算单元在时钟周期2内进行计算的第二像素的第一代价聚合结果的数据K2；之后，在时钟周期3内，可以从第一内部存储器中读取用于第二组计算单元在时钟周期3内进行计算的第二像素的第一代价聚合结果的数据K3，以及用于第二组计算单元在时钟周期4内进行计算的第二区块中第二像素的第一代价聚合结果的数据K4；……，依次类推。其中，第二像素即为第二区块中的像素，时钟周期1、时钟周期3即为第一时钟周期，时钟周期2、时钟周期4即为第二时钟周期，数据K1和数据K3为前述的第二数据，数据K2和数据K4为前述的第三数据。
进一步的，本实施例的方法还可以包括：在所述第二时钟周期内，将所述第一数据和第四数据写入所述第一内部存储器的所述目标存储空间中；所述第四数据为在所述第二时钟周期内所述第一组计算单元计算得到的所述第一像素的第一代价聚合结果的数据。由此，可以实现通过在第二时钟周期内单次写入第一内部存储器，可以将第一组计算单元在第一时钟周期内和第二时钟周期内计算得到的第一代价聚合结果写入第一内部存储器，即每隔一个时钟周期向第一内部存储器中进行一次写入，有利于减少针对第一内部存储器的写入次数，提高读取效率。另外，通过上述在第一时钟周期内读取两个数据，在第二时钟周期内写入两个数据的方式，使得无需占用用于计算单元计算代价聚合结果之外的其他时钟周期进行数据读写，确保第一组计算单元和第二组计算单元在每个时钟周期均能够进行计算，有利于提高处理效率，减少处理时长。
例如，参考图27A，在图26的基础上，在时钟周期1内，可以从第一内部存储器中读取用于第二组计算单元在时钟周期1内进行计算的第二像素的第一代价聚合结果的数据K1，以及用于第一组计算单元在时钟周期2内进行计算的第二像素的第一代价聚合结果的数据K2，第一组计算单元计算得到第 一像素的第一代价聚合结果的数据K5，将数据K5进行缓存，并且，第二组计算单元基于数据K1进行第三代价结果的计算。之后，在时钟周期2内，第一组计算单元计算得到第一像素的第一代价聚合结果的数据K6，将数据K5和K6写入原本由数据K1和数据K2占用的目标存储空间，并且，第二组计算单元基于数据K2进行第三聚合代价结果的计算；之后，在时钟周期3内，可以从第一内部存储器中读取用于第二组计算单元在时钟周期3内进行计算的第二像素的第一代价聚合结果的数据K3，以及用于第一组计算单元在时钟周期4内进行计算的第二像素的第一代价聚合结果的数据K4，第一组计算单元计算得到第一像素的第一代价聚合结果的数据K7，将数据K7进行缓存，并且，第二组计算单元基于数据K3进行第三代价结果的计算。之后，在时钟周期4内，第一组计算单元计算得到第一像素的第一代价聚合结果的数据K8，将数据K7和K8写入原本由数据K3和数据K4占用的目标存储空间，并且，第二组计算单元基于数据K4进行第三代价结果的计算；……，依次类推。其中，第一像素即为第一区块中的像素，第二像素即为第二区块中的像素，数据K5和数据K7即为前述的第一数据，数据K1和数据K3即为前述的第二数据，数据K2和数据K4即为前述的第三数据，数据K6和数据K8即为前述的第四数据。
然而，本申请并非限于此。在另一实施方式中，所述内部存储器可以包括一个第一内部存储器，且所述第一内部存储器可以为双端口内部存储器。例如，如图27B所示，在时钟周期1内，可以从第一内部存储器中读取用于第二组计算单元在时钟周期1内进行计算的第二像素的第一代价聚合结果的数据K1，并将第一组计算单元在时钟周期1内计算得到第一像素的第一代价聚合结果的数据K5写入第一内部存储器中原本由数据K1占用的目标存储空间；之后，在时钟周期2内，可以从第一内部存储器中读取用于第二组计算单元在时钟周期2内进行计算的第二像素的第一代价聚合结果的数据K2，并将第一组计算单元在时钟周期2内计算得到第一像素的第一代价聚合结果的数据K6写入第一内部存储器中原本由数据K2占用的目标存储空间；之后，在时钟周期3内，可以从第一内部存储器中读取用于第二组计算单元在时钟周期3内进行计算的第二像素的第一代价聚合结果的数据K3，并将第一组计算单元在时钟周期3内计算得到第一像素的第一代价聚合结果的数据K7写入第一内部存储器中原本由数据K3占用的目标存储空间；之后，在时钟周期4内，可以从第一内 部存储器中读取用于第二组计算单元在时钟周期4内进行计算的第二像素的第一代价聚合结果的数据K4，并将第一组计算单元在时钟周期4内计算得到第一像素的第一代价聚合结果的数据K8写入第一内部存储器中原本由数据K4占用的目标存储空间；……，依次类推。其中，第一像素即为第一区块中的像素，第二像素即为第二区块中的像素，时钟周期1、时钟周期3即为第一时钟周期，时钟周期2、时钟周期4即为第二时钟周期，数据K1和数据K3即为第二数据，数据K2和数据K4即为第三数据，数据K5和数据K7即为第一数据，数据K6和数据K8即为第四数据。
需要说明的是，为了便于表示在同一时钟周期内读取和写入的数据占第一内部存储器的同一存储空间，图9b中第一内部存储器中存储的灰色线条填充表示的数据表示在对应时钟周期需要读取的数据。
在单次计算可以针对单个像素在部分视差进行计算的情况下，假设单个区块的大小为32×32像素，最大视差等于128，单个时钟周期计算的视差数量为Dper，Dper小于128，第一代价聚合结果的位宽为16bit，则单个时钟周期内第二组计算单元计算消耗的第二像素的第一代价聚合结果的大小为Dper×16(即第二数据的大小为Dper×16)，第一组计算单元计算得到的第一像素的第一代价聚合结果的大小也为Dper×16(即第一数据的大小也为Dper×16)，由此可以将第一内部存储器的位宽设置为Dper×16×2，位宽增加一倍，深度减小50％，容量不变。
方式二
第二时钟周期内所述第一组计算单元计算得到的第一数据，被存入所述内部存储器中第一时钟周期内所述第二组计算单元计算使用的第二数据对应的存储空间；其中，所述第一数据为所述第一像素的第一代价聚合结果的数据，所述第二数据为所述第二像素的第一代价聚合结果的数据；所述第二时钟周期为所述第一时钟周期的下一个时钟周期。由此，可以实现一时钟周期内计算得到的第一像素的第一代价聚合结果，占用相邻时钟周期内计算使用的第二像素的第一代价聚合结果。需要说明的是，关于数据的具体说明可以参见前述方式一中的相关描述，在此不再赘述。
在一个实施例中，所述内部存储器可以包括第一内部存储器和第二内部存储器。所述第一内部存储器和所述第二内部存储器可以为单端口内部存储器，通过所述第一内部存储器和所述第二内部存储器均是单端口内部存储器， 有利于节省成本；或者，第一内部存储器和第二内部存储器可以为作为单端口内部存储器使用的其他类型内部存储器，本实施例的方法还可以包括：在第一时钟周期内，将所述第一组计算单元计算得到的所述第一数据写入所述第一内部存储器中，并从所述第二内部存储器的目标存储空间中读取所述第二数据；所述第二数据用于所述第二组计算单元在所述第一时钟周期内计算第三代价聚合结果。
基于此，可以实现在一个时钟周期内从一个内部存储器中读取到用于第二组计算单元在该周期内进行计算的第二像素的第一代价聚合结果，并将第一计算单元在该周期计算得到的第一像素的第一代价聚合结果存储至另一内部存储器中，即在同一时钟周期内分别针对不同的内部存储器进行读或写，有利于降低对于内部存储器的端口数量要求，降低成本。另外，能够使得针对第一代价聚合结果的读或写，无需占用用于计算单元计算代价聚合结果之外的其他时钟周期进行数据读写，确保第一组计算单元和第二组计算单元在每个时钟周期均能够进行计算，有利于提高处理效率，减少处理时长。
进一步的，本实施例的方法还可以包括：在第二时钟周期内，从所述第一内部存储器中读取第三数据，并将所述第二组计算单元计算得到的第四数据写入所述第二内部存储器的所述目标存储空间中。其中，所述第三数据为所述第二像素的第一代价聚合结果的数据，所述第三数据用于所述第二组计算单元在第二时钟周期内计算得到的第三代价结果；所述第四数据为在所述第二时钟周期内所述第一组计算单元计算得到的所述第一像素的第一代价聚合结果的数据；所述第二时钟周期为所述第一时钟周期的下一个时钟周期。由此，可以实现第一像素的第一代价聚合结果和第二像素的第一代价聚合结果，共用两个内部存储器的存储空间。
例如，如图28所示，在时钟周期1内，可以从第二内部存储器中读取用于第二组计算单元在时钟周期1内进行计算的第二像素的第一代价聚合结果的数据K1，并将第一组计算单元在时钟周期1内计算得到第一像素的第一代价聚合结果的数据K5写入第一内部存储器中；之后，在时钟周期2内，可以从第一内部存储器中读取用于第二组计算单元在时钟周期2内进行计算的第二像素的第一代价聚合结果的数据K2，并将第一组计算单元在时钟周期2内计算得到第一像素的第一代价聚合结果的数据K6写入第二内部存储器中原本由数据K1占用的目标存储空间；之后，在时钟周期3内，可以从第二内部存储器中读 取用于第二组计算单元在时钟周期3内进行计算的第二像素的第一代价聚合结果的数据K3，并将第一组计算单元在时钟周期3内计算得到第一像素的第一代价聚合结果的数据K7写入第一内部存储器中原本由数据K2占用的目标存储空间；之后，在时钟周期4内，可以从第一内部存储器中读取用于第二组计算单元在时钟周期4内进行计算的第二像素的第一代价聚合结果的数据K4，并将第一组计算单元在时钟周期4内计算得到第一像素的第一代价聚合结果的数据K8写入第二内部存储器中原本由数据K3占用的目标存储空间；……，依次类推。其中，第一像素即为第一区块中的像素，第二像素即为第二区块中的像素，时钟周期1、时钟周期3即为第一时钟周期，时钟周期2、时钟周期4即为第二时钟周期，数据K1和数据K3即为第二数据，数据K2和数据K4即为第三数据，数据K5和数据K7即为第一数据，数据K6和数据K8即为第四数据。
可选的，所述多个区块中特定区块的待聚合区域中的多个像素在第一方向上的第一聚合结果，可以是在所述先前处理的过程中获得，所述特定区块为所述多个区块中经所述第二组计算单元第一个处理的区块。通过在所述先前处理的过程中计算获得所述特定区块的待聚合区域中的多个像素在第一方向上的第一聚合结果，可以减少第一计算单元的需要处理的区块的数量，有利于减少处理时长，提高处理帧率。
假设处理顺序为前述的区块1→区块2→区块3→区块4，在先前处理的过程中可以预先获得区块1的待聚合区域中多个像素在第一方向上的第一代价聚合结果，参考图29：在时间t由t0秒至t1秒的一段时间内，第一组计算单元可以计算得到位于区块2的待聚合区域中多个像素在第一方向上的第一代价聚合结果，第二组计算单元可以计算得到位于区块1的待聚合区域中多个像素在第二方向上的第二代价聚合结果，并基于在先前处理的过程中获得的位于区块1的待聚合区域中多个像素在第一方向上的第一代价聚合结果，计算得到位于区块1的待聚合区域中的多个像素的第三代价聚合结果。在时间t由t1秒至t2秒的一段时间内，第一组计算单元可以计算得到位于区块3的待聚合区域中多个像素在第一方向上的第一代价聚合结果，第二组计算单元可以计算得到位于区块2的待聚合区域中多个像素在第二方向上的第二代价聚合结果，并基于第一组计算单元在t0秒至t1秒的时段内计算获得的区块2的待聚合区域中多个像素在第一方向上的第一代价聚合结果，计算得到位于区块2的待聚合区域中的多个像素的第三代价聚合结果。在时间t由t2秒至t3秒的一段时间内，第一组 计算单元可以计算得到位于区块4的待聚合区域中多个像素在第一方向上的第一代价聚合结果，第二组计算单元可以计算得到位于区块3的待聚合区域中多个像素在第二方向上的第二代价聚合结果，并基于第一组计算单元在t1秒至t2秒的时段内计算获得的区块3的待聚合区域中多个像素在第一方向上的第一代价聚合结果，计算得到位于区块3的待聚合区域中的多个像素的第三代价聚合结果。在时间t由t3秒至t4秒的一段时间内，第二组计算单元可以计算得到位于区块4的待聚合区域中多个像素在第二方向上的第二代价聚合结果，并基于第一组计算单元在t2秒至t3秒的时段内计算获得的区块4的待聚合区域中多个像素在第一方向上的第一代价聚合结果，计算得到位于区块4的待聚合区域中的多个像素的第三代价聚合结果。对比图29和图24可以看出，在先前处理过程中获得区块1的待聚合区域中的多个像素在第一方向上的第一聚合结果，较由第一组计算单元计算得到区块1的待聚合区域中的多个像素在第一方向上的第一聚合结果，可以缩短针对整个图像进行处理的耗时，即将耗时由t5秒缩短至t4秒。
需要说明的是，关于基于第一聚合代价结果和第二聚合代价结果，计算得到第三代价聚合结果的具体方式，可以参见前述实施例的相关描述，在此不再赘述。
本申请实施例提供的图像处理方法，通过采用第一组计算单元，基于多个区块中第一区块的第一重叠区域和第一区块的待聚合区域中像素的匹配代价，计算得到位于第一区块的待聚合区域中的多个第一像素在第一方向上的第一代价聚合结果，采用第二组计算单元，计算得到位于第二区块的待聚合区域中的多个第二像素在第二方向上的第二代价聚合结果，以及多个第二像素在第一方向上的第一代价聚合结果和多个第二像素在第二方向上的第二代价聚合结果，计算得到多个第二像素的第三代价聚合结果，其中，第一区块和第二区块相邻且第二区块位于第一区块之前，实现了在计算位于第一区块的待聚合区域中的第一像素在第一方向上的第一代价聚合结果的同时，对位于第二区块的待聚合区域中第二像素在第二方向上的第二代价聚合结果以及第二像素的最终代价聚合结果进行计算，有利于减少处理耗时，提高处理帧率，并且能够减少需要存储的中间数据，降低对于存储器的容量要求，有利于节省成本。
图30为本申请一实施例提供的图像处理装置的结构示意图，对应不同视 点的两幅图像中的目标图像被划分为部分区域重叠的多个区块，所述多个区块中的第一区块的重叠区域包括目标重叠区域，所述第一区块的目标重叠区域包括所述第一区块中与第一方向对应的第一重叠区域。如图30所示，该装置300可以包括：处理器301和存储器302。
所述存储器302，用于存储程序代码；
所述处理器301，调用所述程序代码，当程序代码被执行时，用于执行以下操作：
获取位于所述第一区块中像素的匹配代价；
基于所述第一区块的第一重叠区域和所述第一区块的待聚合区域中像素的匹配代价，计算得到位于所述第一区块的待聚合区域中的多个第一像素在所述第一方向上的第一代价聚合结果。
本实施例提供的图像处理装置，可以用于执行前述方法实施例的技术方案，其实现原理和技术效果与方法实施例类似，在此不再赘述。
本申请实施例还提供一种集成电路，对应不同视点的两幅图像中的目标图像被划分为部分区域重叠的多个区块，所述多个区块中的第一区块的重叠区域包括所述第一区块中与第一方向对应的第一重叠区域；所述电路包括：计算单元；所述计算单元用于：
获取位于所述第一区块中像素的匹配代价；
基于所述第一区块的第一重叠区域和所述第一区块的待聚合区域中像素的匹配代价，计算得到位于所述第一区块的待聚合区域中的多个第一像素在所述第一方向上的第一代价聚合结果。
本申请实施例还提供一种图像处理装置，该图像处理装置可以包括所述集成电路。
图31为本申请一实施例提供的可移动平台的结构示意图，如图31所示，该可移动平台310包括机身311、设置于所述机身上的动力系统312、拍摄装置313和前述实施例所述的图像处理装置314；
所述动力系统312，用于为所述可移动平台提供动力；
所述拍摄装置313，用于在所述可移动平台飞行过程中进行拍摄，以获得对应不同视点的两幅图像。
可选的，可移动平台310还可以包括云台315，拍摄装置313可以通过云台315设置在机身311上。当然，可移动平台除上述列出装置外，还可以包 括其他元件或装置，这里不一一例举。
本申请实施例提供一种图像处理方法、集成电路、装置、可移动平台及存储介质，通过整张目标图像被划分为部分区域重叠的多个区块，多个区块中的第一区块的重叠区域包括目标重叠区域，所述目标重叠区域包括与第一方向对应的第一重叠区域，基于第一区块的第一重叠区域和待聚合区域中像素的匹配代价，计算得到第一区块中的像素在第一方向上的代价聚合结果，实现了基于第一区块中像素的匹配代价计算得到位于第一区块的待聚合区域中像素在第一方向上的第一代价聚合结果，使得不同区块之间第一代价聚合结果的计算之间是相互独立的，代价聚合可以不再限制不同区块之间像素的处理顺序，与针对整张目标图像沿着第一方向依次确定整张图像中各像素的代价聚合结果，对像素的处理顺序要求较高相比，降低了对于处理顺序的要求，从而减少了对于代价聚合的实现方式的限制。
本领域普通技术人员可以理解：实现上述各方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成。前述的程序可以存储于一计算机可读取存储介质中。该程序在执行时，执行包括上述各方法实施例的步骤；而前述的存储介质包括：ROM、RAM、磁碟或者光盘等各种可以存储程序代码的介质。
最后应说明的是：以上各实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述各实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分或者全部技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的范围。

Claims (99)

  1. 一种图像处理方法，其特征在于，对应不同视点的两幅图像中的目标图像被划分为部分区域重叠的多个区块，所述多个区块中的第一区块的重叠区域包括所述第一区块中与第一方向对应的第一重叠区域；所述方法包括：
    获取位于所述第一区块中像素的匹配代价；
    基于所述第一区块的所述第一重叠区域和所述第一区块的待聚合区域中像素的匹配代价，计算得到位于所述第一区块的所述待聚合区域中的多个第一像素在所述第一方向上的第一代价聚合结果。
  2. 根据权利要求1所述的方法，其特征在于，所述第一方向为多个预设方向，所述多个预设方向包括向上方向，所述第一重叠区域包括所述第一区块的下部分区域。
  3. 根据权利要求1所述的方法，其特征在于，所述第一方向为多个预设方向，所述多个预设方向包括向下方向，所述第一重叠区域包括所述第一区块的上部分区域。
  4. 根据权利要求1所述的方法，其特征在于，所述第一方向为多个预设方向，所述多个预设方向包括向左方向，所述第一重叠区域包括所述第一区块的右部分区域。
  5. 根据权利要求1所述的方法，其特征在于，所述第一方向为多个预设方向，所述多个预设方向包括向右方向，所述第一重叠区域包括所述第一区块的左部分区域。
  6. 根据权利要求1-5任一项所述的方法，其特征在于，所述方法还包括：
    计算得到位于所述多个区块中第二区块的待聚合区域中的多个第二像素在第二方向上的第二代价聚合结果。
  7. 根据权利要求6所述的方法，其特征在于，所述第二区块的重叠区域包括所述第二区块中与所述第二方向对应的第二重叠区域；所述计算得到位于所述多个区块中第二区块的待聚合区域中的多个第二像素在第二方向上的第二代价聚合结果，包括：
    基于所述第二区块的所述第二重叠区域和所述第二区块的待聚合区域中像素的匹配代价，计算得到位于所述第二区块的所述待聚合区域中的多个第二像素在第二方向上的第二代价聚合结果。
  8. 根据权利要求6所述的方法，其特征在于，所述第二方向为多个预设 方向；所述计算得到位于所述多个区块中第二区块的待聚合区域中的多个第二像素在第二方向上的第二代价聚合结果，包括：
    从存储器中读取所述多个第二像素中部分像素在所述多个预设方向中的至少一个所述预设方向上的单方向代价聚合结果，并基于所述部分像素的所述单方向代价聚合结果计算得到所述多个第二像素中其他像素的第二代价聚合结果，所述部分像素的所述单方向代价聚合结果是通过先前处理获得并存储在所述存储器中。
  9. 根据权利要求8所述的方法，其特征在于，所述多个预设方向包括向下方向，所述部分像素包括所述第二区块的所述待聚合区域中位于上边缘的像素。
  10. 根据权利要求8所述的方法，其特征在于，所述多个预设方向包括向右方向，所述部分像素包括所述第二区块的所述待聚合区域中位于左边缘的像素。
  11. 根据权利要求8所述的方法，其特征在于，所述多个预设方向包括向上方向，所述部分像素包括所述第二区块的所述待聚合区域中位于下边缘的像素。
  12. 根据权利要求8所述的方法，其特征在于，所述多个预设方向包括向左方向，所述部分像素包括所述第二区块的所述待聚合区域中位于右边缘的像素。
  13. 根据权利要求6所述的方法，其特征在于，根据预定的处理顺序来计算所述多个区块各自对应的第一代价聚合结果和第二代价聚合结果，所述第二方向为多个预设方向；所述计算得到位于所述多个区块中第二区块的待聚合区域中的多个第二像素在第二方向上的第二代价聚合结果，包括：
    基于所述多个区块中目标区块的待聚合区域中的部分边缘像素在所述多个预设方向中的至少一个所述预设方向上的单方向代价聚合结果，计算得到位于所述多个区块中第二区块的待聚合区域中多个第二像素在第二方向上的第二代价聚合结果；
    所述预定的处理顺序中所述目标区块位于所述第二区块之前，且所述目标区块的所述待聚合区域与所述第二区块的所述待聚合区域相邻。
  14. 根据权利要求13所述的方法，其特征在于，所述多个预设方向包括向上方向，所述部分边缘像素包括所述目标区块的所述待聚合区域中位于下 边缘的像素。
  15. 根据权利要求13所述的方法，其特征在于，所述多个预设方向包括向下方向，所述部分边缘像素包括所述目标区块的所述待聚合区域中位于上边缘的像素。
  16. 根据权利要求13所述的方法，其特征在于，所述多个预设方向包括向左方向，所述部分边缘像素包括所述目标区块的所述待聚合区域中位于右边缘的像素。
  17. 根据权利要求13所述的方法，其特征在于，所述多个预设方向包括向右方向，所述部分边缘像素包括所述目标区块的所述待聚合区域中位于左边缘的像素。
  18. 根据权利要求13-17任一项所述的方法，其特征在于，所述目标区块包括第一目标区块，在所述预定的处理顺序中所述第一目标区块与所述第二区块相邻；所述第一目标区块的所述待聚合区域中的部分边缘像素在至少一个所述预设方向上的单方向代价聚合结果存储在内部存储器中。
  19. 根据权利要求13-17任一项所述的方法，其特征在于，所述目标区块包括第二目标区块，在所述预定的处理顺序中所述第二区块与所述第二区块间隔；所述第二目标区块的所述待聚合区域中的部分边缘像素在至少一个所述预设方向上的单方向代价聚合结果存储在外部存储器中。
  20. 根据权利要求6-19任一项所述的方法，其特征在于，所述第一区块和所述第二区块是同一区块。
  21. 根据权利要求6-19任一项所述的方法，其特征在于，所述多个区块之间存在一定的处理顺序；
    采用第一组计算单元，基于所述第一区块的所述第一重叠区域和所述第一区块的待聚合区域中像素的匹配代价，计算得到位于所述第一区块的所述待聚合区域中的多个第一像素在所述第一方向上的第一代价聚合结果；
    采用第二组计算单元，计算得到位于所述多个区块中第二区块的待聚合区域中的多个第二像素在第二方向上的第二代价聚合结果；
    其中，所述第一组计算单元和所述第二组计算单元是集成电路中并行处理的两组计算单元。
  22. 根据权利要求21所述的方法，其特征在于，在所述处理顺序中所述第一区块和所述第二区块相邻，且所述处理顺序中所述第二区块位于所述第 一区块之前。
  23. 根据权利要求22所述的方法，其特征在于，所述方法还包括：
    采用所述第二组计算单元，基于所述多个第二像素在所述第一方向上的第一代价聚合结果和所述多个第二像素在所述第二方向上的第二代价聚合结果，计算得到所述多个第二像素的第三代价聚合结果。
  24. 根据权利要求23所述的方法，其特征在于，位于所述第一区块的所述待聚合区域中的所述第一像素的第一代价聚合结果和位于所述第二区块的所述待聚合区域中的所述第二像素的第一代价聚合结果均存储在内部存储器中。
  25. 根据权利要求24所述的方法，其特征在于，位于所述第一区块的所述待聚合区域中的所述第一像素的第一代价聚合结果和位于所述第二区块的所述待聚合区域中的所述第二像素的第一代价聚合结果，共用所述内部存储器中的同一块存储空间。
  26. 根据权利要求25所述的方法，其特征在于，所述第一组计算单元和所述第二组计算单元的单次计算均占用单个时钟周期；
    一时钟周期内所述第一组计算单元计算得到的第一数据，被存入所述内部存储器中该时钟周期内所述第二组计算单元计算使用的第二数据对应的存储空间；其中，所述第一数据为所述第一像素的第一代价聚合结果的数据，所述第二数据为所述第二像素的第一代价聚合结果的数据。
  27. 根据权利要求26所述的方法，其特征在于，所述内部存储器包括第一内部存储器；所述方法还包括：
    在第一时钟周期内，从所述第一内部存储器的目标存储空间中读取所述第二数据以及与所述第二数据相邻的第三数据，并缓存所述第一组计算单元计算得到的所述第一数据；
    其中，所述第二数据用于所述第二组计算单元在所述第一时钟周期内计算第三代价聚合结果；所述第三数据为所述第二像素的第一代价聚合结果的数据，所述第三数据用于所述第二组计算单元在第二时钟周期内计算第三代价聚合结果；所述第二时钟周期为所述第一时钟周期的下一个时钟周期。
  28. 根据权利要求27所述的方法，其特征在于，所述方法还包括：
    在所述第二时钟周期内，将所述第一数据和第四数据写入所述第一内部存储器的所述目标存储空间中；所述第四数据为在所述第二时钟周期内所述 第一组计算单元计算得到的所述第一像素的第一代价聚合结果的数据。
  29. 根据权利要求25所述的方法，其特征在于，所述第一组计算单元和所述第二组计算单元的单次计算均占用单个时钟周期；
    第二时钟周期内所述第一组计算单元计算得到的第一数据，被存入所述内部存储器中第一时钟周期内所述第二组计算单元计算使用的第二数据对应的存储空间；其中，所述第一数据为所述第一像素的第一代价聚合结果的数据，所述第二数据为所述第二像素的第一代价聚合结果的数据；所述第二时钟周期为所述第一时钟周期的下一个时钟周期。
  30. 根据权利要求29所述的方法，其特征在于，所述内部存储器包括第一内部存储器和第二内部存储器；所述方法还包括：
    在第一时钟周期内，将所述第一组计算单元计算得到的所述第一数据写入所述第一内部存储器中，并从所述第二内部存储器的目标存储空间中读取所述第二数据；所述第二数据用于所述第二组计算单元在所述第一时钟周期内计算第三代价聚合结果。
  31. 根据权利要求30所述的方法，其特征在于，所述方法还包括：
    在第二时钟周期内，从所述第一内部存储器中读取第三数据，并将所述第二组计算单元计算得到的第四数据写入所述第二内部存储器的所述目标存储空间中；
    其中，所述第三数据为所述第二像素的第一代价聚合结果的数据，所述第三数据用于所述第二组计算单元在第二时钟周期内计算得到的第三代价结果；所述第四数据为在所述第二时钟周期内所述第一组计算单元计算得到的所述第一像素的第一代价聚合结果的数据。
  32. 根据权利要求26-31任一项所述的方法，其特征在于，所述数据的大小包括下述中的任意一种：
    单个像素在全部视差下的第一代价聚合结果的大小，单个像素在部分视差下的第一代价聚合结果的大小，多个像素在全部视差下的第一代价聚合结果的大小。
  33. 根据权利要求23-32任一项所述的方法，其特征在于，所述第一代价聚合结果、所述第二代价聚合结果以及所述第三代价聚合结果包括像素在视差范围内各视差下的聚合结果。
  34. 根据权利要求33所述的方法，其特征在于，所述基于所述多个第二 像素在所述第一方向上的第一代价聚合结果和所述多个第二像素在所述第二方向上的第二代价聚合结果，计算得到所述多个第二像素的第三代价聚合结果，包括：
    针对于所述多个第二像素中的各像素，将所述像素在各视差下的第一代价聚合结果与所述像素在各所述视差下的第二代价聚合结果对应相加，得到所述像素在各所述视差下的第三代价聚合结果。
  35. 根据权利要求34所述的方法，其特征在于，所述方法还包括：
    将所述像素在所述视差范围内聚合结果最小的视差，作为所述像素的最终视差。
  36. 根据权利要求1所述的方法，其特征在于，采用集成电路中的一组计算单元，基于所述第一区块的所述第一重叠区域和所述第一区块的所述待聚合区域中像素的匹配代价，计算得到位于所述第一区块的所述待聚合区域中的多个第一像素在所述第一方向上的第一代价聚合结果；
    所述方法还包括：采用该组计算单元，计算得到位于所述第一区块的所述待聚合区域中的所述多个第一像素在第二方向上的第二代价聚合结果。
  37. 根据权利要求36所述的方法，其特征在于，所述方法还包括：
    采用该组计算单元，基于所述多个第一像素在所述第一方向上的第一代价聚合结果和所述多个第一像素在所述第二方向上的第二代价聚合结果，计算得到所述多个第一像素的第三代价聚合结果。
  38. 根据权利要求6-37任一项所述的方法，其特征在于，所述第一方向和所述第二方向对应相反。
  39. 一种图像处理装置，其特征在于，对应不同视点的两幅图像中的目标图像被划分为部分区域重叠的多个区块，所述多个区块中的第一区块的重叠区域包括所述第一区块中与第一方向对应的第一重叠区域；所述装置包括存储器和处理器；
    所述存储器，用于存储程序代码；
    所述处理器，调用所述程序代码，当程序代码被执行时，用于执行以下操作：
    获取位于所述第一区块中像素的匹配代价；
    基于所述第一区块的所述第一重叠区域和所述第一区块的待聚合区域中像素的匹配代价，计算得到位于所述第一区块的所述待聚合区域中的多个第 一像素在所述第一方向上的第一代价聚合结果。
  40. 根据权利要求39所述的装置，其特征在于，所述第一方向为多个预设方向，所述多个预设方向包括向上方向，所述第一重叠区域包括所述第一区块的下部分区域。
  41. 根据权利要求39所述的装置，其特征在于，所述第一方向为多个预设方向，所述多个预设方向包括向下方向，所述第一重叠区域包括所述第一区块的上部分区域。
  42. 根据权利要求39所述的装置，其特征在于，所述第一方向为多个预设方向，所述多个预设方向包括向左方向，所述第一重叠区域包括所述第一区块的右部分区域。
  43. 根据权利要求39所述的装置，其特征在于，所述第一方向为多个预设方向，所述多个预设方向包括向右方向，所述第一重叠区域包括所述第一区块的左部分区域。
  44. 根据权利要求39-43任一项所述的装置，其特征在于，所述处理器还用于：
    计算得到位于所述多个区块中第二区块的待聚合区域中的多个第二像素在第二方向上的第二代价聚合结果。
  45. 根据权利要求44所述的装置，其特征在于，所述第二区块的重叠区域包括所述第二区块中与所述第二方向对应的第二重叠区域；所述处理器用于计算得到位于所述多个区块中第二区块的待聚合区域中的多个第二像素在第二方向上的第二代价聚合结果，具体包括：
    基于所述第二区块的所述第二重叠区域和所述第二区块的待聚合区域中像素的匹配代价，计算得到位于所述第二区块的所述待聚合区域中的多个第二像素在第二方向第二代价聚合结果。
  46. 根据权利要求44所述的装置，其特征在于，所述第二方向为多个预设方向；所述处理器用于计算得到位于所述多个区块中第二区块的待聚合区域中的多个第二像素在第二方向上的第二代价聚合结果，包括：
    从存储器中读取所述多个第二像素中部分像素在所述多个预设方向中的至少一个所述预设方向上的单方向代价聚合结果，并基于所述部分像素的所述单方向代价聚合结果计算得到所述多个第二像素中其他像素的第二代价聚合结果，所述部分像素的所述单方向代价聚合结果是通过先前处理获得并存 储在所述存储器中。
  47. 根据权利要求46所述的装置，其特征在于，所述多个预设方向包括向下方向，所述部分像素包括所述第二区块的所述待聚合区域中位于上边缘的像素。
  48. 根据权利要求46所述的装置，其特征在于，所述多个预设方向包括向右方向，所述部分像素包括所述第二区块的所述待聚合区域中位于左边缘的像素。
  49. 根据权利要求46所述的装置，其特征在于，所述多个预设方向包括向上方向，所述部分像素包括所述第二区块的所述待聚合区域中位于下边缘的像素。
  50. 根据权利要求46所述的装置，其特征在于，所述多个预设方向包括向左方向，所述部分像素包括所述第二区块的所述待聚合区域中位于右边缘的像素。
  51. 根据权利要求44所述的装置，其特征在于，根据预定的处理顺序来计算所述多个区块各自对应的第一代价聚合结果和第二代价聚合结果，所述第二方向为多个预设方向；所述处理器用于计算得到位于所述多个区块中第二区块的待聚合区域中的多个第二像素在第二方向上的第二代价聚合结果，包括：
    基于所述多个区块中目标区块的待聚合区域中的部分边缘像素在所述多个预设方向中的至少一个所述预设方向上的单方向代价聚合结果，计算得到位于所述多个区块中第二区块的待聚合区域中多个第二像素在第二方向上的第二代价聚合结果；
    所述预定的处理顺序中所述目标区块位于所述第二区块之前，且所述目标区块的所述待聚合区域与所述第二区块的所述待聚合区域位置相邻。
  52. 根据权利要求51所述的装置，其特征在于，所述多个预设方向包括向上方向，所述部分边缘像素包括所述目标区块的所述待聚合区域中位于下边缘的像素。
  53. 根据权利要求51所述的装置，其特征在于，所述多个预设方向包括向下方向，所述部分边缘像素包括所述目标区块的所述待聚合区域中位于上边缘的像素。
  54. 根据权利要求51所述的装置，其特征在于，所述多个预设方向包括 向左方向，所述部分边缘像素包括所述目标区块的所述待聚合区域中位于右边缘的像素。
  55. 根据权利要求51所述的装置，其特征在于，所述多个预设方向包括向右方向，所述部分边缘像素包括所述目标区块的所述待聚合区域中位于左边缘的像素。
  56. 根据权利要求51-55任一项所述的装置，其特征在于，所述目标区块包括第一目标区块，在所述预定的处理顺序中所述第一目标区块与所述第二区块相邻；所述第一目标区块的所述待聚合区域中的部分边缘像素在至少一个所述预设方向上的单方向代价聚合结果存储在内部存储器中。
  57. 根据权利要求51-55任一项所述的装置，其特征在于，所述目标区块包括第二目标区块，在所述预定的处理顺序中所述第二目标区块与所述第二区块间隔；所述第二目标区块的所述待聚合区域中的部分边缘像素在至少一个所述预设方向上的单方向代价聚合结果存储在外部存储器中。
  58. 根据权利要求44-57任一项所述的装置，其特征在于，所述第一方向和所述第二方向对应相反。
  59. 一种集成电路，其特征在于，对应不同视点的两幅图像中的目标图像被划分为部分区域重叠的多个区块，所述多个区块中的第一区块的重叠区域包括所述第一区块中与第一方向对应的第一重叠区域；所述电路包括：计算单元；所述计算单元用于：
    获取位于所述第一区块中像素的匹配代价；
    基于所述第一区块的所述第一重叠区域和所述第一区块的待聚合区域中像素的匹配代价，计算得到位于所述第一区块的所述待聚合区域中的多个第一像素在所述第一方向上的第一代价聚合结果。
  60. 根据权利要求59所述的集成电路，其特征在于，所述第一方向为多个预设方向，所述多个预设方向包括向上方向，所述第一重叠区域包括所述第一区块的下部分区域。
  61. 根据权利要求59所述的集成电路，其特征在于，所述第一方向为多个预设方向，所述多个预设方向包括向下方向，所述第一重叠区域包括所述第一区块的上部分区域。
  62. 根据权利要求59所述的集成电路，其特征在于，所述第一方向为多个预设方向，所述多个预设方向包括向左方向，所述第一重叠区域包括所述 第一区块的右部分区域。
  63. 根据权利要求59所述的集成电路，其特征在于，所述第一方向为多个预设方向，所述多个预设方向包括向右方向，所述第一重叠区域包括所述第一区块的左部分区域。
  64. 根据权利要求59-62任一项所述的集成电路，其特征在于，所述计算单元还用于：
    计算得到位于所述多个区块中第二区块的待聚合区域中的多个第二像素在第二方向上的第二代价聚合结果。
  65. 根据权利要求64所述的集成电路，其特征在于，所述第二区块的重叠区域包括所述第二区块中与所述第二方向对应的第二重叠区域；所述计算单元，用于计算得到位于所述多个区块中第二区块的待聚合区域中的多个第二像素在第二方向上的第二代价聚合结果，具体包括：
    基于所述第二区块的所述第二重叠区域和所述第二区块的待聚合区域中像素的匹配代价，计算得到位于所述第二区块的所述待聚合区域中的多个第二像素在所述第二方向上的第二代价聚合结果。
  66. 根据权利要求64所述的集成电路，其特征在于，所述第二方向为多个预设方向；所述计算单元用于计算得到位于所述多个区块中第二区块的待聚合区域中的多个第二像素在第二方向上的第二代价聚合结果，包括：
    从存储器中读取所述多个第二像素中部分像素在所述多个预设方向中的至少一个所述预设方向上的单方向代价聚合结果，并基于所述部分像素的所述单方向代价聚合结果计算得到所述多个第二像素中其他像素的第二代价聚合结果，所述部分像素的所述单方向代价聚合结果是通过先前处理获得并存储在所述存储器中。
  67. 根据权利要求66所述的集成电路，其特征在于，所述多个预设方向包括向下方向，所述部分像素包括所述第二区块的所述待聚合区域中位于上边缘的像素。
  68. 根据权利要求66所述的集成电路，其特征在于，所述多个预设方向包括向右方向，所述部分像素包括所述第二区块的所述待聚合区域中位于左边缘的像素。
  69. 根据权利要求66所述的集成电路，其特征在于，所述多个预设方向包括向上方向，所述部分像素包括所述第二区块的所述待聚合区域中位于下 边缘的像素。
  70. 根据权利要求66所述的集成电路，其特征在于，所述多个预设方向包括向左方向，所述部分像素包括所述第二区块的所述待聚合区域中位于右边缘的像素。
  71. 根据权利要求64所述的集成电路，其特征在于，根据预定的处理顺序来计算所述多个区块各自对应的第一代价聚合结果和第二代价聚合结果，所述第二方向为多个预设方向；所述计算单元，用于计算得到位于所述多个区块中第二区块的待聚合区域中的多个第二像素在第二方向上的第二代价聚合结果，包括：
    基于所述多个区块中目标区块的待聚合区域中的部分边缘像素在所述多个预设方向中的至少一个所述预设方向上的单方向代价聚合结果，计算得到位于所述多个区块中第二区块的待聚合区域中多个第二像素在第二方向上的第二代价聚合结果；
    所述预定的处理顺序中所述目标区块位于所述第二区块之前，且所述目标区块的所述待聚合区域与所述第二区块的所述待聚合区域位置相邻。
  72. 根据权利要求71所述的集成电路，其特征在于，所述多个预设方向包括向上方向，所述部分边缘像素包括所述目标区块的所述待聚合区域中位于下边缘的像素。
  73. 根据权利要求71所述的集成电路，其特征在于，所述多个预设方向包括向下方向，所述部分边缘像素包括所述目标区块的所述待聚合区域中位于上边缘的像素。
  74. 根据权利要求71所述的集成电路，其特征在于，所述多个预设方向包括向左方向，所述部分边缘像素包括所述目标区块的所述待聚合区域中位于右边缘的像素。
  75. 根据权利要求71所述的集成电路，其特征在于，所述多个预设方向包括向右方向，所述部分边缘像素包括所述目标区块的所述待聚合区域中位于左边缘的像素。
  76. 根据权利要求71-75任一项所述的集成电路，其特征在于，所述目标区块包括第一目标区块，在所述预定的处理顺序中所述第一目标区块与所述第二区块相邻；所述第一目标区块的所述待聚合区域中的部分边缘像素在至少一个所述预设方向上的单方向代价聚合结果存储在内部存储器中。
  77. 根据权利要求71-75任一项所述的集成电路，其特征在于，所述目标区块包括第二目标区块，在所述预定的处理顺序中所述第二区块与所述第二区块间隔；所述第二目标区块的所述待聚合区域中的部分边缘像素在至少一个所述预设方向上的单方向代价聚合结果存储在外部存储器中。
  78. 根据权利要求64-77任一项所述的集成电路，其特征在于，所述第一区块和所述第二区块是同一区块。
  79. 根据权利要求64-77任一项所述的集成电路，其特征在于，所述多个区块之间存在一定的处理顺序；
    所述集成电路中的第一组计算单元，基于所述第一区块的所述第一重叠区域和所述第一区块的待聚合区域中像素的匹配代价，计算得到位于所述第一区块的所述待聚合区域中的多个第一像素在第一方向上的第一代价聚合结果；
    所述集成电路中的第二组计算单元，计算得到位于所述多个区块中第二区块的待聚合区域中的多个第二像素在第二方向上的第二代价聚合结果；
    其中，所述第一组计算单元和所述第二组计算单元是集成电路中并行处理的两组计算单元。
  80. 根据权利要求79所述的集成电路，其特征在于，在所述处理顺序中所述第一区块和所述第二区块相邻，且所述处理顺序中所述第二区块位于所述第一区块之前。
  81. 根据权利要求80所述的集成电路，其特征在于，所述第二组计算单元，还用于：
    基于所述多个第二像素在所述第一方向上的第一代价聚合结果和所述多个第二像素在所述第二方向上的第二代价聚合结果，计算得到所述多个第二像素的第三代价聚合结果。
  82. 根据权利要求81所述的集成电路，其特征在于，位于所述第一区块的所述待聚合区域中的所述第一像素的第一代价聚合结果和位于所述第二区块的所述待聚合区域中的所述第二像素的第一代价聚合结果均存储在内部存储器中。
  83. 根据权利要求82所述的集成电路，其特征在于，位于所述第一区块的所述待聚合区域中的所述第一像素的第一代价聚合结果和位于所述第二区块的所述待聚合区域中的所述第二像素的第一代价聚合结果，共用所述内部 存储器中的同一块存储空间。
  84. 根据权利要求83所述的集成电路，其特征在于，所述第一组计算单元和所述第二组计算单元的单次计算均占用单个时钟周期；
    一时钟周期内所述第一组计算单元计算得到的第一数据，被存入所述内部存储器中该时钟周期内所述第二组计算单元计算使用的第二数据对应的存储空间；其中，所述第一数据为所述第一像素的第一代价聚合结果的数据，所述第二数据为所述第二像素的第一代价聚合结果的数据。
  85. 根据权利要求81所述的集成电路，其特征在于，所述内部存储器包括第一内部存储器；所述计算单元，还用于：
    在第一时钟周期内，从所述第一内部存储器的目标存储空间中读取所述第二数据以及与所述第二数据相邻的第三数据，并缓存所述第一组计算单元计算得到的所述第一数据；
    其中，所述第二数据用于所述第二组计算单元在所述第一时钟周期内计算第三代价聚合结果；所述第三数据为所述第二像素的第一代价聚合结果的数据，所述第三数据用于所述第二组计算单元在第二时钟周期内计算第三代价聚合结果；所述第二时钟周期为所述第一时钟周期的下一个时钟周期。
  86. 根据权利要求85所述的集成电路，其特征在于，所述计算单元，还用于：
    在所述第二时钟周期内，将所述第一数据和第四数据写入所述第一内部存储器的所述目标存储空间中；所述第四数据为在所述第二时钟周期内所述第一组计算单元计算得到的所述第一像素的第一代价聚合结果的数据。
  87. 根据权利要求83所述的集成电路，其特征在于，所述第一组计算单元和所述第二组计算单元的单次计算均占用单个时钟周期；
    第二时钟周期内所述第一组计算单元计算得到的第一数据，被存入所述内部存储器中第一时钟周期内所述第二组计算单元计算使用的第二数据对应的存储空间；其中，所述第一数据为所述第一像素的第一代价聚合结果的数据，所述第二数据为所述第二像素的第一代价聚合结果的数据；所述第二时钟周期为所述第一时钟周期的下一个时钟周期。
  88. 根据权利要求87所述的集成电路，其特征在于，所述内部存储器包括第一内部存储器和第二内部存储器；所述计算单元还用于：
    在第一时钟周期内，将所述第一组计算单元计算得到的所述第一数据写 入所述第一内部存储器中，并从所述第二内部存储器的目标存储空间中读取所述第二数据；所述第二数据用于所述第二组计算单元在所述第一时钟周期内计算第三代价聚合结果。
  89. 根据权利要求88所述的集成电路，其特征在于，所述计算单元还用于：
    在第二时钟周期内，从所述第一内部存储器中读取第三数据，并将所述第二组计算单元计算得到的第四数据写入所述第二内部存储器的所述目标存储空间中；
    其中，所述第三数据为所述第二像素的第一代价聚合结果的数据，所述第三数据用于所述第二组计算单元在第二时钟周期内计算得到的第三代价结果；所述第四数据为在所述第二时钟周期内所述第一组计算单元计算得到的所述第一像素的第一代价聚合结果的数据。
  90. 根据权利要求84-89任一项所述的集成电路，其特征在于，所述数据的大小包括下述中的任意一种：
    单个像素在全部视差下的第一代价聚合结果的大小，单个像素在部分视差下的第一代价聚合结果的大小，多个像素在全部视差下的第一代价聚合结果的大小。
  91. 根据权利要求78、81-90任一项所述的集成电路，其特征在于，所述第一代价聚合结果、所述第二代价聚合结果以及所述第三代价聚合结果包括像素在视差范围内各视差下的聚合结果。
  92. 根据权利要求91所述的集成电路，其特征在于，所述计算单元用于基于所述多个第二像素在所述第一方向上的第一代价聚合结果和所述多个第二像素在所述第二方向上的第二代价聚合结果，计算得到所述多个第二像素的第三代价聚合结果，具体包括：
    针对于所述多个第二像素中的各像素，将所述像素在各视差下的第一代价聚合结果与所述像素在各所述视差下的第二代价聚合结果对应相加，得到所述像素在各所述视差下的第三代价聚合结果。
  93. 根据权利要求92所述的集成电路，其特征在于，所述计算单元还用于：
    将所述像素在所述视差范围内聚合结果最小的视差，作为所述像素的最终视差。
  94. 根据权利要求59所述的集成电路，其特征在于，所述集成电路中的一组计算单元，用于基于所述第一区块的所述第一重叠区域和所述第一区块的待聚合区域中像素的匹配代价，计算得到位于所述第一区块的所述待聚合区域中的多个第一像素在所述第一方向上的第一代价聚合结果；
    所述该组计算单元，还用于计算得到位于所述第一区块的所述待聚合区域中的所述多个第一像素在第二方向上的第二代价聚合结果。
  95. 根据权利要求94所述的集成电路，其特征在于，所述该组计算单元，还用于：
    采用该组计算单元，基于所述多个第一像素在所述第一方向上的第一代价聚合结果和所述多个第一像素在所述第二方向上的第二代价聚合结果，计算得到所述多个第一像素的第三代价聚合结果。
  96. 根据权利要求64-95任一项所述的集成电路，其特征在于，所述第一方向和所述第二方向对应相反。
  97. 一种图像处理装置，其特征在于，包括：权利要求59-96任一项所述的集成电路。
  98. 一种可移动平台，其特征在于，所述可移动平台包括机身、设置于所述机身上的动力系统、拍摄装置和权利要求39-58、97任一项所述的图像处理装置；
    所述动力系统，用于为所述可移动平台提供动力；
    所述拍摄装置，用于在所述可移动平台飞行过程中进行拍摄，以获得对应不同视点的两幅图像。
  99. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有计算机程序，所述计算机程序包含至少一段代码，所述至少一段代码可由计算机执行，以控制所述计算机执行如权利要求1-38任一项所述的方法。
PCT/CN2020/100101 2020-07-03 2020-07-03 图像处理方法、集成电路、装置、可移动平台及存储介质 WO2022000454A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2020/100101 WO2022000454A1 (zh) 2020-07-03 2020-07-03 图像处理方法、集成电路、装置、可移动平台及存储介质
CN202080005541.7A CN114144765A (zh) 2020-07-03 2020-07-03 图像处理方法、集成电路、装置、可移动平台及存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/100101 WO2022000454A1 (zh) 2020-07-03 2020-07-03 图像处理方法、集成电路、装置、可移动平台及存储介质

Publications (1)

Publication Number Publication Date
WO2022000454A1 true WO2022000454A1 (zh) 2022-01-06

Family

ID=79317231

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/100101 WO2022000454A1 (zh) 2020-07-03 2020-07-03 图像处理方法、集成电路、装置、可移动平台及存储介质

Country Status (2)

Country Link
CN (1) CN114144765A (zh)
WO (1) WO2022000454A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104091339A (zh) * 2014-07-17 2014-10-08 清华大学深圳研究生院 一种图像快速立体匹配方法及装置
CN105528785A (zh) * 2015-12-03 2016-04-27 河北工业大学 一种双目视觉图像立体匹配方法
CN107220994A (zh) * 2017-06-01 2017-09-29 成都通甲优博科技有限责任公司 一种立体匹配的方法及系统
CN107220997A (zh) * 2017-05-22 2017-09-29 成都通甲优博科技有限责任公司 一种立体匹配方法及系统

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104091339A (zh) * 2014-07-17 2014-10-08 清华大学深圳研究生院 一种图像快速立体匹配方法及装置
CN105528785A (zh) * 2015-12-03 2016-04-27 河北工业大学 一种双目视觉图像立体匹配方法
CN107220997A (zh) * 2017-05-22 2017-09-29 成都通甲优博科技有限责任公司 一种立体匹配方法及系统
CN107220994A (zh) * 2017-06-01 2017-09-29 成都通甲优博科技有限责任公司 一种立体匹配的方法及系统

Also Published As

Publication number Publication date
CN114144765A (zh) 2022-03-04

Similar Documents

Publication Publication Date Title
US10356385B2 (en) Method and device for stereo images processing
CN109743562B (zh) 基于Census算法的匹配代价计算电路结构及其工作方法
CA2929403C (en) Multi-dimensional sliding window operation for a vector processor
CN107909537B (zh) 一种基于卷积神经网络的图像处理方法及移动终端
US20230025068A1 (en) Hybrid machine learning architecture with neural processing unit and compute-in-memory processing elements
US20190228502A1 (en) Method and system for correcting a distorted input image
Li et al. High throughput hardware architecture for accurate semi-global matching
US9153064B2 (en) Grouping pixels to be textured
WO2021088569A1 (en) Convolution method and device, electronic device
CN106952304A (zh) 一种利用视频序列帧间相关性的深度图像计算方法
CN114169514B (zh) 一种卷积硬件加速方法及卷积硬件加速电路
CN109447239B (zh) 一种基于arm的嵌入式卷积神经网络加速方法
CN102999885A (zh) Retinex视频增强算法亮度平均值确定方法与装置
WO2022000454A1 (zh) 图像处理方法、集成电路、装置、可移动平台及存储介质
WO2022000458A1 (zh) 图像深度信息确定方法、装置、设备和存储介质
US20140369562A1 (en) Image processor
US20210241408A1 (en) Data processing method and sensor device for performing the same
US20230252600A1 (en) Image size adjustment structure, adjustment method, and image scaling method and device based on streaming architecture
WO2022000456A1 (zh) 图像处理方法、集成电路、装置及设备
US20220121551A1 (en) Method and device for calculating runtime of neural network on processor
WO2023103551A1 (zh) 图像数据处理方法、装置、设备及存储介质
US10235763B2 (en) Determining optical flow
CN107622037A (zh) 一种提高图形处理单元的矩阵乘计算性能的方法和装置
CN113344765A (zh) 一种频域天文图像目标检测方法及系统
RU168781U1 (ru) Устройство обработки стереоизображений

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20942602

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20942602

Country of ref document: EP

Kind code of ref document: A1