WO2022241729A1

WO2022241729A1 - Image processing method and apparatus, and movable platform and storage medium

Info

Publication number: WO2022241729A1
Application number: PCT/CN2021/094965
Authority: WO
Inventors: 高明明; 张清顺; 赵文扬; 杨康; 陈琳; 林蔓虹
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2021-05-20
Filing date: 2021-05-20
Publication date: 2022-11-24

Abstract

An image processing method and apparatus, and a movable platform and a storage medium. The method comprises: reading, from a memory, a first image block of a reference image and a second image block of an image to be processed; and performing image registration by using the first image block and the second image block, wherein the reference image is divided into a plurality of image blocks according to the size of the first image block, and the reference image is continuously stored in the memory in a unit of image block, and/or the image to be processed is divided into a plurality of image blocks according to the size of the second image block, and the image to be processed is continuously stored in the memory in a unit of image block. By means of the embodiments, the improvement of the read efficiency of a memory is facilitated.

Description

Image processing method, device, movable platform and storage medium

technical field

The present application relates to the technical field of image processing, and in particular, relates to an image processing method, device, removable platform, and storage medium.

Background technique

In the field of computer vision, image registration methods are used in object detection, object tracking, motion estimation, video analysis, object location, model reconstruction, remote sensing data analysis, and aerial reconnaissance. In the process of image registration, it is necessary to read the image blocks of the reference image and the image blocks of the image to be processed from the memory for processing. Since the images are usually stored in the order from left to right and from top to bottom in the memory, then The storage addresses of the image blocks are usually discontinuous, so multiple read instructions need to be used to read the image block, resulting in the problem of low reading efficiency for the memory.

Contents of the invention

In view of this, one of the objectives of the present application is to provide an image processing method, device, removable platform and storage medium.

In the first aspect, the embodiment of the present application provides an image processing method, including:

reading the first image block of the reference image and the second image block of the image to be processed from the memory;

performing image registration using the first image block and the second image block;

Wherein, the reference image is divided into multiple image blocks according to the size of the first image block, and the reference image is continuously stored in the memory in units of image blocks; and/or, the image to be processed is The size of the second image block is divided into a plurality of image blocks, and the image to be processed is continuously stored in the memory in units of image blocks.

In a second aspect, the embodiment of the present application provides an image processing device, including a memory, a data loading module, and a processor;

The memory is used to store reference images and images to be processed;

The data loading module is used to read the first image block of the reference image and the second image block of the image to be processed from the memory; wherein, the reference image is divided into the size of the first image block A plurality of image blocks, the reference image is continuously stored in the memory in units of image blocks; and/or, the image to be processed is divided into a plurality of image blocks according to the size of the second image block, so The image to be processed is continuously stored in the memory in units of image blocks;

The processor is configured to perform image registration using the first image block and the second image block.

In a third aspect, the embodiment of the present application provides a mobile platform, including:

body;

a power system mounted on the fuselage for powering the movable platform; and,

The image processing device according to the second aspect.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed by the image processing device, the computer program described in the first aspect is implemented. image processing method.

In the image processing method, device, mobile platform, and storage medium provided by the embodiments of the present application, the reference image stored in the memory is divided into multiple image blocks according to the size of the first image block, and the reference image is in The memory is continuously stored in units of image blocks, and/or, the image to be processed stored in the memory is divided into multiple image blocks by the size of the second image block, and the image to be processed is stored in the memory in the form of Image blocks are continuously stored in units; when the first image block of the reference image and the second image block of the image to be processed are read from the memory for image registration, since the reference image and/or the image to be processed are stored as image blocks in the memory Stored as a unit, and can also be read in units of image blocks when reading, then fewer read instructions can be used to read the first image block and/or the second image block, which is conducive to improving the storage capacity of the memory. read efficiency.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings that need to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can also be obtained based on these drawings without any creative effort.

FIG. 1 is a schematic diagram of a reference image provided by an embodiment of the present application;

FIG. 2 is a schematic diagram of image registration using the optical flow method provided by an embodiment of the present application;

Fig. 3 is a schematic diagram of an application scenario provided by an embodiment of the present application;

FIG. 4 is a schematic flowchart of an image processing method provided by an embodiment of the present application;

Figure 5 and Figure 6 are different schematic diagrams of acquiring and storing image blocks provided by an embodiment of the present application;

FIG. 7 is a schematic diagram of data read using 16 burst signals (burst length is 4) provided by an embodiment of the present application;

Fig. 8 is a schematic diagram of two situations of reading image blocks provided by an embodiment of the present application;

Figure 9, Figure 10 and Figure 11 are schematic diagrams of different numbers of image blocks including pixels of the first image block provided by an embodiment of the present application;

Figure 12 is a schematic diagram of a second buffer provided by an embodiment of the present application;

Fig. 13 is a schematic flowchart of another image processing method provided by an embodiment of the present application;

Fig. 14, Fig. 15, Fig. 16, Fig. 17 and Fig. 18 are schematic diagrams of different structures of an image processing device provided by an embodiment of the present application;

Fig. 19 is a schematic structural diagram of a mobile platform provided by an embodiment of the present application.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the application with reference to the drawings in the embodiments of the application. Apparently, the described embodiments are only some of the embodiments of the application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

In the related art, in the process of image registration, it is necessary to read the image blocks of the reference image and the image blocks of the image to be processed from the memory for processing, and the storage addresses of the image blocks are usually discontinuous, so multiple read instructions need to be used to read the image block, resulting in the problem of low reading efficiency for the memory. For example, for the image shown in Figure 1, assuming that a grid in the figure represents a pixel, the image shown in Figure 1 is usually stored continuously in the memory in the order of raster scanning from left to right and from top to bottom, and in image registration In the process, usually only the image block data in the image needs to be read. Assume that the image block data to be read is the gray part in Figure 1. Obviously, the storage addresses of the image block data are discontinuous, and at least three read Instructions, such as 3 burst signals, the first burst signal is used to read

pixels

1, 2, 3; the second burst signal is used to read pixels 4, 5, 6; the third burst signal Signaling is used to read pixels 7, 8, and 9, so as to obtain image block data of the gray part, and there is a problem of low reading efficiency of the memory.

Aiming at the problems in the related art, the embodiment of the present application provides an image registration method, the reference image stored in the memory is divided into multiple image blocks according to the size of the first image block, and the reference image is in the The memory is continuously stored in units of image blocks, and/or, the image to be processed stored in the memory is divided into multiple image blocks with the size of the second image block, and the image to be processed is stored in the memory as an image Blocks are continuously stored in units; then when reading the first image block of the reference image and the second image block of the image to be processed from the memory for image registration, since the reference image and/or the image to be processed are based on image blocks in the memory The unit is stored, and when reading, it can also be read in units of image blocks, so that fewer read instructions can be used to read the first image block and/or the second image block, which is conducive to improving the readability of the memory. Take efficiency.

Among them, image registration can be applied to target detection, target tracking, motion estimation, video analysis, target positioning, model reconstruction, remote sensing data analysis, aerial reconnaissance, etc.

In an exemplary embodiment, for example, in a target tracking scenario, target tracking has a wide range of applications in fields such as visual navigation, robotics, intelligent transportation, or public security. Among them, an important method in the target tracking process is the optical flow method. Simply put, optical flow is the instantaneous two-dimensional velocity field generated by the movement of pixels on the observed imaging plane of a spatially moving object, including the object's apparent structure and dynamic behavior. Important information; it uses the changes of pixels in the image sequence in the time domain and the correlation between adjacent frames to find the corresponding relationship between the previous frame and the current frame, so as to calculate the object's distance between adjacent frames. Motion information, so as to realize target tracking according to the motion information of objects between adjacent frames. Of course, the optical flow method can also be applied to scenarios such as target positioning, and the above is only for illustration and does not constitute a limitation to the present application.

For example, for a continuous video frame sequence, after extracting the key points used to describe the foreground target in the previous frame (for example, it can be randomly generated, or the feature points can be used as key points), you can find the key points that appeared in the previous frame. The best position of the point in the current frame, for example, please refer to Figure 2, find the position of the key point u in the image I in the image J, so as to obtain the position coordinates of the foreground target in the current frame; iteratively for the video frame sequence Goal tracking can be realized. Among them, there are many kinds of optical flow methods, such as KLT algorithm and LK algorithm.

In the process of using the optical flow method for target tracking, for example, please refer to Figure 2, when looking for the position of the key point u in the image I in the image J, by reading the first position of the key point u in the image I from the memory The first image block and the second image block in the image J are iteratively calculated to solve the velocity vector of the key point u with respect to the image J, and then determine the position of the key point u in the image J. Among them, in the first calculation process, the second image block of the key point u in image J can be obtained according to the position of the first image block at the same position in image J, and then according to the distance between the first image block and the second image block Solve the velocity vector of the key point u with respect to the image J, and then determine the position of the key point u in the image J; in the subsequent iterative calculation process, the key point u can be re-determined in the image J according to the velocity vector solved last time According to the re-determined position of the key point u in image J, the second image block in image J is obtained, and the re-acquired second image block is used for iterative calculation with the first image block until the predetermined value is met. Set the iteration condition position. It can be understood that the preset iteration condition may be specifically set according to an actual application scenario.

Wherein, for the image I (hereinafter referred to as the reference image), the image I can be divided into a plurality of image blocks according to the size of the first image block, and the first image is continuously stored in the memory in units of image blocks; for For the image J (hereinafter referred to as the image to be processed), the image J may also be divided into multiple image blocks according to the size of the second image block, and the second image blocks are continuously stored in the memory. Subsequently, when reading the first image block of image I and the second image block of image J, the reading can be performed in units of image blocks, which can significantly improve the reading efficiency.

In another exemplary embodiment, for example, in scenarios such as target detection and target positioning, the general flow of another image registration method is as follows: two images are respectively acquired (for example, one is a reference image, and the other is a reference image). is the key point of the image to be processed (for example, it can be randomly generated, or the feature point can be used as the key point); the image block of the key point is read from the memory address where the two images are stored in the memory, and the similarity measurement is carried out Determine the similarity between the image blocks of the two images to determine the matching key points; then obtain the image space coordinate transformation parameters through the matching key points; finally perform target detection or target positioning based on the image space coordinate transformation parameters. Wherein, for the storage method of the above-mentioned two images in the memory, the image can be divided into several blocks according to the size of the image block, and stored continuously in the memory in units of image blocks, and subsequently read The image blocks of the key points can be read in units of image blocks, which can significantly improve the reading efficiency.

In some embodiments, the image processing method in the embodiment of the present application may be applied to an image processing device.

In a possible implementation, the image processing device may be an electronic device with data processing capabilities, such as a computer, a server, a cloud server or a terminal, a mobile platform (such as an unmanned aerial vehicle, an unmanned vehicle, or a mobile robot) and many more.

In another possible implementation, the image processing device may also be a computer chip or an integrated circuit with data processing capabilities, such as a central processing unit (Central Processing Unit, CPU), a digital signal processor (Digital Signal Processor, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC) or off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA), etc. Wherein, the image processing device can be installed on an electronic device, such as the image processing device can be installed on a mobile platform such as an unmanned aerial vehicle, an unmanned vehicle or a mobile robot; or the image processing device can also be installed on a handheld PTZ.

In an exemplary application scenario, the image processing device may be an unmanned aerial vehicle or the image processing device may be a chip or an integrated circuit installed in an unmanned aerial vehicle. Referring to FIG. 3 , for example, the unmanned aerial vehicle 100 can realize the function of following and photographing a target object or the function of target positioning through the image processing device.

For example, for the following shooting function of the target object, the UAV 100 is equipped with an imaging device 200, through which multiple images of the target object are collected, and then the image processing device can be used to perform the The image processing method sequentially performs image registration on the current frame (reference image) and the next frame (image to be processed), and automatically tracks the movement change process of the target object, so that the unmanned aerial vehicle 100 can detect the target object according to the result of image registration. Perform follow-up shooting.

For example, for the target positioning function, the imaging device 200 is used to collect multiple images of the unmanned aerial vehicle 100 during flight, and then the image processing device can be used to perform the image processing method, and the current frame (refer to image) and the next frame (image to be processed) for image registration to locate the position of the target in different images, and then assist in determining the position of the UAV 100 according to the position of the located target.

Next, the image processing method provided by the embodiment of the present application will be described: Please refer to FIG. 4, which is a schematic flow chart of an image processing method provided by the embodiment of the present application. The method can be executed by an image processing device. Methods include:

In step S101, a first image block of a reference image and a second image block of an image to be processed are read from a memory. Wherein, the reference image is divided into multiple image blocks according to the size of the first image block, and the reference image is continuously stored in the memory in units of image blocks; and/or, the image to be processed is The size of the second image block is divided into a plurality of image blocks, and the image to be processed is continuously stored in the memory in units of image blocks.

In step S102, image registration is performed using the first image block and the second image block.

In some embodiments, the reference image and/or the image to be processed are continuously stored in the memory in units of image blocks, and the storage addresses of the pixels in each image block in the memory are also consecutive , when reading the first image block or the second image block, it can be read from the memory in units of image blocks, which is conducive to improving the reading efficiency of the memory, thereby improving the efficiency of image registration .

In an example, for the reference image or the image to be processed, please refer to FIG. 5. Taking the reference image as an example, the reference image is divided into It is divided into a plurality of image blocks, and then stored continuously in the memory in units of image blocks, and the storage addresses of the pixels in each image block in the memory are also continuous. The same processing method is also used for the image to be processed, the image to be processed is divided into multiple image blocks according to the size of the second image block, and then stored continuously in the memory in units of image blocks, and for each The storage addresses of the pixels in the image block in the memory are also continuous. Exemplarily, the size of the first image block and the size of the second image block are the same, of course, in some embodiments, the size of the first image block and the size of the second image block can also be Different, this embodiment does not impose any limitation on this.

The process of storing the reference image in the memory in the form of image blocks is described here: after the reference image is acquired, please refer to FIG. In the first buffer zone of the row, each cache row is used to cache a row of pixels of the reference image; then, for each image block of the reference image, according to the size of the first image block, the The target cache line in the first cache line reads the specified pixel, and stores the specified pixel continuously in the memory. In FIG. 6 , the size of the first image block is 16*16 as an example.

In some embodiments, the size of the first image block includes width and height; the number of the target cache line is determined according to one of the height and width; the number of specified pixels is determined according to the height and width The other one is determined; for example, the size of the first image block is 16*32, the data of the target cache line is 16, and the specified number of pixels is 32, that is, for each image block, the image processing device needs to start from 16 32 pixels are read from the specified positions in each target cache line, and then 16*32 pixels are continuously stored in the memory. Assuming that a pixel row has 64 pixels, for the first image block, the first to 32nd pixels need to be read, and for the second image block, the 33rd to 64th pixels need to be read.

In some embodiments, the number of cache lines included in the first buffer is at least the number of the target cache lines. For example, the size of the first image block is 16*32, the data of the target cache line is 16, and the number of cache lines included in the first buffer zone is at least 16.

The process of storing the image to be processed in the memory in the form of image blocks is described here: after acquiring the image to be processed, the image processing device may cache the image to be processed in In the first buffer, each cache line is used to cache a row of pixels of the image to be processed; then, for each image block of the image to be processed, according to the size of the second image block from the multiple The target cache line in one cache line reads the specified pixel, and stores the specified pixel in the memory continuously.

In some embodiments, the size of the second image block includes width and height; the number of the target cache line is determined according to one of the height and width; the number of specified pixels is determined according to the height and width The other one is determined; for example, the size of the second image block is 16*32, the data of the target cache line is 16, and the specified number of pixels is 32, that is, for each image block, the image processing device needs to start from 16 32 pixels are read from the specified positions in each target cache line, and then 16*32 pixels are continuously stored in the memory.

In some embodiments, the number of the first buffers is at least two; the reference image is buffered in one of the first buffers; the image to be processed is buffered in the other first buffer. In other embodiments, in order to save cache resources, the number of the first buffer may be one; for example, the first buffer may be used to process the reference image, and then the image to be processed may be processed using the first buffer; Alternatively, the first buffer zone may be used to process the image to be processed, and then the reference image may be processed using the first buffer zone.

In some embodiments, in order to improve the accuracy of image registration, an image pyramid of a reference image and an image pyramid of an image to be processed are usually constructed for image registration. The image pyramid is a series of images arranged in a pyramid shape with gradually reduced resolution and derived from the same original image. For each image in the image pyramid, the above method can be used to divide it into multiple image blocks and store the multiple image blocks continuously in the memory, and the storage addresses of the pixels in the image block in the memory are also continuous .

Exemplarily, the first image block of the reference image and the second image block of the image to be processed may be read from the memory in a burst continuous read manner.

In the related art, for the image stored in the memory from top to bottom and from left to right, in the actual reading process, the pixel bit width of the image is P=8bit, the bit width of the memory M=128bit, That is, each storage unit in the memory can store 16 (16=128/8) pixels, and the burst continuous reading method is adopted. Assuming that the burst length of the burst signal L is 4, it means that the continuous pixel can be read. For the data stored in 4 storage units, at least 64 pixels (4*16=64) can be read in each read operation, assuming that the image block data size to be read is 16*16, as shown in Figure 7, The gray part in Figure 7 is the image block data required this time. A burst signal (burst length is 4) can read a row of 64 pixel data, and the image block data size is 16*16, so 16 burst signals are required. (burst length is 4) read 64*16 data as shown in FIG. 2 . Also referring to Fig. 8, for the image stored in the memory from top to bottom and from left to right, there are two situations when reading a 16*16 image block data (here with 16*16 image block For example, the same situation exists for image blocks of other sizes): (1) A row of data of image block data can be read using a burst signal (burst length is 4), then for 16*16 image block data needs 16 burst signals (burst length is 4) to read; (2) one line of image block data needs to use two burst signals (burst length is 4) to read, then for 16*16 image Block data requires 16 burst signals (burst length 8) to read.

And in the present application, since the image (here, the reference image is taken as an example) is continuously stored in the memory in units of image blocks, and the storage addresses of the pixels in the image blocks in the memory are continuous, then when reading a 16 * There are the following three situations in the first image block of 16: (1) Please refer to Figure 9, the data of the first image block is all in one image block, then only one burst signal (burst length is 16) is required Read, the required read instruction is less; (2) Please refer to Figure 10, the data of the first image block is in two image blocks, then only two burst signals (burst length is 16) can be read Fetch, the required read instruction is less; (3) Please refer to Figure 11, the data of the first image block is in four image blocks, then need 4 burst signals (burst length is 16) to read , although the data volume read by 4 burst signals (burst length is 16) is the same as the data volume read by 16 burst signals (burst length is 4) in the case (1) in the embodiment shown in Figure 8, However, since the burst length of the burst signal in this application is larger, the greater the burst length, the higher the memory access efficiency.

In some embodiments, during the image registration process, the image processing device may use the first burst signal to read the first image block of the reference image from the memory, and use the second burst signal to read the first image block from the memory. Get the second image block of the image to be processed. Wherein, there are three situations as shown in Fig. 9, Fig. 10 and Fig. 11 when reading the first image block and/or the second image block, it can be said that the first burst signal and/or the second burst signal The numbers are all less than or equal to 4. The quantity of the first burst signal is determined according to the number of image blocks containing the pixels of the first image block (such as in FIG. 10, if the number of image blocks containing the pixels of the first image block is 2, then 2 second a burst signal); and/or, the number of the second burst signal is determined according to the number of image blocks including the pixels of the second image block.

In the case that the reference image is continuously stored in the memory in units of image blocks, and the pixels in the image blocks are also stored continuously, the first burst signal can be used to read in units of image blocks, and a first burst The signal is used to read the data of an image block, and the burst length of the first burst signal can be determined according to the number of pixels contained in the first image block, the pixel bit width and the bit width of the memory; wherein, The pixel bit width and the bit width of the memory determine the number of pixels that can be stored in a storage unit in the memory, then the number of pixels contained in the first image block and the number of pixels that can be stored in a storage unit can be determined for storage The number of storage units for the first image block, the number of storage units for storing the first image block is the burst length of the first burst signal.

Similarly, when the image to be processed is stored continuously in the memory in units of image blocks, and the pixels in the image blocks are also stored continuously, the second burst signal can be used to read in units of image blocks, one The second burst signal is used to read the data of an image block, and the signal length of the second burst signal can be determined according to the number of pixels contained in the second image block, the pixel bit width and the bit width of the memory ; Wherein, the pixel bit width and the bit width of the memory determine the number of pixels that can be stored in a storage unit in the memory, then the number of pixels included in the second image block and the number of pixels that can be stored in a storage unit can be determined The number of storage units used to store the second image block is the burst length of the second burst signal.

In some embodiments, the first image block and the second image block are determined according to the key points extracted from the reference image, and then according to the image configuration of the first image block and the second image block A standard process to determine the position of key points in the image to be processed. Wherein, the key points can be determined randomly from the reference image, or can be determined according to feature points or feature information extracted from the reference image, and the feature points or feature information include but not limited to corner points, edge points, height Curvature points, contours, intersections, line segments, closed boundaries, or centers of gravity, etc. Exemplarily, the first image block may be an area of a preset size centered on the key point in the reference image; and after estimating the position of the key point in the image to be processed, the The second image block may be an area of a preset size centered on the key point in the image to be processed.

In some other embodiments, the first image block can be determined according to the first feature point extracted from the reference image, and the second image block can be determined according to the second feature point extracted from the image to be processed, and then according to The image registration process of the first image block and the second image block is used to obtain the relative pose and the like between the reference image and the image to be processed. Exemplarily, the first image block is an area of a preset size centered on the first feature point in the reference image, and the second image block is an area of preset size centered on the second feature point in the image to be processed Area.

In some embodiments, considering that in the embodiment shown in FIG. 7 , more data than image blocks may be read each time an image block is read, and irrelevant data will be discarded, thereby causing waste The situation of memory bandwidth; And in some cases, such as in the embodiment as shown in Figure 2, find the key point u in image I in the position in image J, by being to read key point u in image from memory The first image block in I and the second image block in image J are iteratively calculated to solve the velocity vector of key point u with respect to image J, and then determine the position of key point u in image J, wherein the iterative process requires Read the second image block from the memory multiple times, each time when reading the second image block may read more data than the second image block, and irrelevant data will be discarded, resulting in a larger Data waste. Therefore, for the above problem, when reading the first image block, the reference image area including the first image block can be read from the memory, and the reference image area except the first image block The data is used for the next image registration process; when reading the second image block, the image area to be processed including the second image block can also be read from the memory, and the image area to be processed Data other than the second image block are used for the next image registration process.

Wherein, the reference image area and/or the image area to be processed can be cached in the second buffer, and in this embodiment, when reading the first image block and/or the second image block, it can read Image blocks with more data and/or read more data than the second image block are cached, and the cached data can be directly read and used later, which can effectively reduce the access efficiency to the memory and improve the data reading efficiency. At the same time It also avoids wasting redundant data that is read.

In a possible implementation manner, the reference image area and the image area to be processed share a second buffer. For the embodiment shown in Figure 2, the image registration process is to find the position of the key point u in the image I (reference image) in the image J (image to be processed), by reading the key point u from the memory The first image block in image I and the second image block in image J are iteratively calculated to solve the velocity vector of key point u with respect to image J, and then determine the position of key point u in image J, where the iterative The process needs to read the second image block from the memory multiple times, so it is more urgent to cache the second image block. Therefore, in the second buffer zone, the buffer capacity of the buffer interval corresponding to the reference image area and/or the buffer capacity of the buffer interval corresponding to the image area to be processed are based on the number of key points extracted from the reference image Determine; wherein, the buffer capacity of the buffer interval corresponding to the reference image area is positively correlated with the number of key points; and/or, the buffer capacity of the buffer interval corresponding to the image area to be processed is related to the number of key points The number is negatively correlated; that is, the smaller the number of key points extracted from the reference image, the smaller the buffer capacity of the buffer interval corresponding to the reference image area, and the smaller the buffer capacity of the buffer interval corresponding to the image area to be processed bigger. Exemplarily, in the second buffer zone, the buffering capacity of the buffering section corresponding to the reference image area is smaller than the buffering capacity of the buffering area corresponding to the image area to be processed, so as to realize effective buffering of the second image block .

In another possible implementation manner, there are at least two second buffers; the reference image area is cached in one of the second buffers; the image area to be processed is cached in another second buffer. in the buffer. Exemplarily, considering that it is more urgent to cache the second image block in the embodiment shown in FIG. Cache capacity of the second buffer corresponding to the image to be processed.

In some embodiments, during the process of reading the reference image area including the first image block from the memory, the image processing device first determines whether the first image block is cached in the second buffer ; Wherein, the reference image area cached in the second buffer zone corresponds to a first storage identifier, and the first storage identifier is used to indicate the position of the reference image area in the reference image; the The image processing device may determine whether the first image block related to the key point is cached in the second buffer according to the key point extracted from the reference image and the first storage identifier.

If the first image block is cached in the second buffer, the first image block is read from the second buffer without reading from a memory, which is beneficial to improve data reading efficiency. If the first image block is not cached in the second buffer, read a reference image area including the first image block from the memory, and simultaneously generate a reference image including the first image block The first storage ID for the region. If part of the data of the first image block is buffered in the second buffer, read the part of the data from the second buffer and read from the memory including the data except the part of the data the reference image area of other data, and simultaneously generate the first storage identifier of the reference image area including other data except the partial data.

Exemplarily, for the case where key points are adjacent, take Figure 12 as an example, when the coordinates of point A and point B are very close, the required data is cached in the second buffer, and at this time, for point B, there will be no It is necessary to read the image block data about point B from the memory, which reduces the number of accesses to the memory, and is also conducive to improving the efficiency of data reading, while avoiding the waste of redundant data read.

In some embodiments, during the process of reading the image region to be processed including the second image block from the memory, the image processing device first determines whether the second image is cached in the second buffer block; wherein, the image region to be processed cached in the second buffer zone corresponds to a second storage identifier, and the second storage identifier is used to indicate the location of the image region to be processed in the image to be processed location; the image processing apparatus may determine whether the second image block related to the key point is cached in the second buffer according to the key point extracted from the reference image and the second storage identifier.

If the second image block is cached in the second buffer, the second image block is read from the second buffer without reading from a memory, which is beneficial to improve data reading efficiency. If the second image block is not cached in the second buffer, read the image area to be processed including the second image block from the memory, and generate an image area to be processed including the second image block at the same time The second storage ID of the image region. If part of the data of the second image block is cached in the second buffer, read the part of the data from the second buffer and read from the memory including the parts except the part of the data The image area to be processed of other data, and simultaneously generate a second storage identifier of the image area to be processed including other data except the part of data. In this embodiment, more data is read and cached in the second buffer during the reading process, which can effectively reduce the number of accesses to the memory, and is also conducive to improving the efficiency of data reading, while avoiding the waste of redundant data read .

In some embodiments, the data volume of the image area to be processed and/or the reference image area can be determined according to the bit width of the line connected to the memory, such as the image area to be processed and/or the The amount of data in the reference image area is an integer multiple of the bit width of the line connected to the memory, that is, the amount of data read each time is just an integer multiple of the maximum transmission capacity of the line connected to the memory, which is conducive to improving Data transfer efficiency.

In some embodiments, please refer to FIG. 13 , the embodiment of the present application also provides another image processing method, which can be executed by an image processing device, and the method includes:

In step S201, the reference image area including the first image block and the image area to be processed including the second image block are read from the memory; wherein, the reference image area excludes the first image block The data other than the second image block is used for the next image registration process; the data other than the second image block in the image area to be processed is used for the next image registration process.

In step S202, image registration is performed using the first image block and the second image block.

In this embodiment, when reading the first image block or the second image block, more data than the first image block and/or more data than the second image block can be read and cached, and can be directly read later The cached data is used, which can effectively reduce the access efficiency to the memory, improve the data reading efficiency, and avoid the waste of redundant data read.

Correspondingly, referring to FIG. 14 , the present application also provides an image processing device, including a memory 10 , a data loading module 20 and a processor 30 .

The memory 10 is used for storing reference images and images to be processed.

The data loading module 20 is used to read the first image block of the reference image and the second image block of the image to be processed from the memory 10; wherein, the reference image is stored with the size of the first image block Cutting into multiple image blocks, the reference image is continuously stored in the memory 10 in units of image blocks; and/or, the image to be processed is cut into multiple images with the size of the second image block The image to be processed is continuously stored in the memory 10 in units of image blocks.

The processor 30 is configured to perform image registration using the first image block and the second image block.

Wherein, the memory 10 may include at least one type of storage medium, and the storage medium includes a flash memory, a hard disk, a multimedia card, a card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access Memory (SRAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Programmable Read Only Memory (PROM), Magnetic Memory, Magnetic Disk, Optical Disk, etc. The storage 10 may be an internal storage unit of the device, such as a hard disk or a memory of the device. The memory 10 can also be an external storage device of the device, such as a plug-in hard disk equipped on the device, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash memory card (Flash Card) and the like. Further, the memory 10 may also include both an internal storage unit of the device and an external storage device. The memory 10 is used for storing reference images and images to be processed. The memory 10 can also be used to temporarily store data that has been output or will be output.

In an embodiment, in the reference image and/or the image to be processed, pixels in each image block have consecutive storage addresses in the memory 10 .

In an embodiment, please refer to FIG. 15 , the device further includes a first conversion module 40 .

The first conversion module 40 is configured to: after acquiring the reference image, cache the reference image in the first buffer 11 including a plurality of cache lines, each cache line is used to cache the reference image A row of pixels of the image; for each image block of the reference image, read specified pixels from the target cache line in the plurality of cache lines according to the size of the first image block, and place the specified pixel in the stored continuously in the memory 10.

In an embodiment, please refer to FIG. 15 , the device further includes a second conversion module 50 .

The second converting module 50 is used for: buffering the image to be processed in the first buffer 11 comprising a plurality of cache lines, each of which is used to cache a row of pixels of the image to be processed; For each image block of the image to be processed, according to the size of the second image block, a specified pixel is read from a target cache line in the plurality of cache lines, and the specified pixels are continuously stored in the memory 10 storage.

In an embodiment, the size of the first image block and the second image block are the same; the size of the first image block and the second image block includes width and height; the number of the target cache line It is determined according to one of the height and width; the number of specified pixels is determined according to the other of the height and width.

In one embodiment, the number of the first buffers 11 is at least two; the reference image is buffered in one of the first buffers 11; the image to be processed is buffered in the other first buffer 11 .

Referring to FIG. 15 , the first conversion module 40 includes at least one first buffer 11 , and the second conversion module 50 also includes at least one first buffer 11 .

In an embodiment, the data loading module 20 is specifically configured to read the first image block of the reference image and the second image block of the image to be processed from the memory 10 in a burst continuous reading manner.

In an embodiment, the data loading module 20 is specifically configured to: use the first burst signal to read the first image block of the reference image from the memory 10, and use the second burst signal to read the first image block of the reference image from the memory 10. Get the second image block of the image to be processed.

In an embodiment, the burst length of the first burst signal is determined according to the number of pixels contained in the first image block, the pixel bit width and the bit width of the memory 10; and/or, the second The burst length of the burst signal is determined according to the number of pixels included in the second image block, the pixel bit width and the bit width of the memory 10 .

In an embodiment, the number of the first burst signal and/or the second burst signal is less than or equal to four.

In an embodiment, the number of the first burst signals is determined according to the number of image blocks including the pixels of the first image block; and/or, the number of the second burst signals is determined according to the number of image blocks including the pixels of the first image block The image block number of pixels of the second image block is determined.

In an embodiment, referring to FIG. 16 , the first image block and the second image block are determined according to key points extracted from the reference image. Please refer to FIG. 16 , the data loading module 20 includes a first loading module 21 and a second loading module 22 . The first loading module 21 is used to read the first image block from the memory 10 according to the coordinates of key points extracted from the reference image; the second loading module 22 is used to read the first image block from the The coordinates of the key points extracted from the reference image or the coordinates of the key points after iterative calculation by the processor 30 are read from the memory 10 to read the second image block.

In an embodiment, the data loading module 20 is specifically configured to: read the reference image area including the first image block and/or the image area to be processed including the second image block from the memory 10; Wherein, the data in the reference image area except for the first image block is used for the next image registration process; the data in the image area to be processed except for the second image block is used for the next One-time image registration process.

In an embodiment, please refer to FIG. 17 , the data loading module 20 includes a second buffer 12 , and the reference image area and/or the image area to be processed are buffered in the second buffer 12 .

In an embodiment, in the second buffer zone 12, the buffer capacity of the buffer interval corresponding to the reference image area and/or the buffer capacity of the buffer interval corresponding to the image area to be processed are based on the data extracted from the reference image. The number of keypoints is determined.

In an embodiment, the buffer capacity of the buffer interval corresponding to the reference image area is positively correlated with the number of key points; and/or, the buffer capacity of the buffer interval corresponding to the image area to be processed is related to the key The number of points is negatively correlated.

In an embodiment, in the second buffer zone 12, the buffer capacity of the buffer interval corresponding to the reference image area is smaller than the buffer capacity of the buffer interval corresponding to the image area to be processed.

In one embodiment, referring to FIG. 18, there are at least two second buffers 12; the reference image area is cached in one of the second buffers 12; the image area to be processed is cached in another a second buffer 12.

In one embodiment, the data loading module 20 is specifically configured to: determine whether the first image block is cached in the second buffer 12; if the second buffer 12 has the first image block cached, An image block, read the first image block from the second buffer 12; if the first image block is not cached in the second buffer 12, read from the memory 10 including The reference image area of the first image block; if the second buffer 12 is buffered with partial data of the first image block, read the partial data from the second buffer 12 and read the partial data from the second buffer 12 A reference image area including data other than the partial data is read from the memory 10.

In an embodiment, the reference image area cached in the second buffer 12 corresponds to a first storage identifier, and the first storage identifier is used to indicate the location of the reference image area in the reference image. Location.

The data loading module 20 is also used for: according to the key points extracted from the reference image and the first storage identifier, determine whether the first key point related to the key point is cached in the second buffer 12. Image blocks.

In one embodiment, the data loading module 20 is specifically configured to: determine whether the second image block is cached in the second buffer 12; if the second image block is cached in the second buffer 12, Two image blocks, read the second image block from the second buffer 12; if the second image block is not cached in the second buffer 12, read from the memory 10 including The to-be-processed image area of the second image block; if the second buffer 12 is buffered with part of the data of the second image block, read the part of the data from the second buffer 12 and read the data from the second buffer 12 The image area to be processed including other data except the partial data is read from the memory 10 .

In an embodiment, the image region to be processed cached in the second buffer 12 corresponds to a second storage identifier, and the second storage identifier is used to indicate that the image region to be processed is in the position in the image.

The data loading module 20 is specifically configured to: according to the key points extracted from the reference image and the second storage identifier, determine whether the second buffer 12 has cached the second data about the key points. Image blocks.

As for the device embodiment, since it basically corresponds to the method embodiment, for related parts, please refer to the part description of the method embodiment. The device embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in One place, or it can be distributed to multiple network elements. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. It can be understood and implemented by those skilled in the art without creative effort.

Various implementations described herein can be implemented using a computer readable medium such as computer software, hardware, or any combination thereof. For hardware implementation, the embodiments described herein can be implemented by using Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays ( FPGA), processors, controllers, microcontrollers, microprocessors, electronic units designed to perform the functions described herein. For software implementation, an embodiment such as a procedure or a function may be implemented with a separate software module that allows at least one function or operation to be performed. The software codes can be implemented by a software application (or program) written in any suitable programming language, stored in memory and executed by a controller.

In one embodiment, please refer to FIG. 19. When the image processing device is a chip or an integrated circuit with data processing capability, the image processing device can be installed on a movable platform, and this embodiment also provides a A kind of movable platform, described movable platform comprises:

fuselage 01;

The power system 02 is installed on the fuselage 01 and is used to provide power for the movable platform; and the above-mentioned image processing device 03 .

In one embodiment, the mobile platform includes but is not limited to unmanned aerial vehicles, unmanned vehicles, unmanned ships or mobile robots.

In one embodiment, the movable platform further includes an imaging device.

In an exemplary embodiment, there is also provided a non-transitory computer-readable storage medium including instructions, such as a memory including instructions, which are executable by a processor of an apparatus to perform the above method. For example, the non-transitory computer readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.

A non-transitory computer-readable storage medium, enabling the terminal to execute the above method when instructions in the storage medium are executed by a processor of the terminal. It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is a relationship between these entities or operations. any such actual relationship or order exists between them. The term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements but also other elements not expressly listed elements, or also elements inherent in such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.

The methods and devices provided by the embodiments of the present application have been described in detail above. The principles and implementation methods of the present application have been explained by using specific examples in this paper. The descriptions of the above embodiments are only used to help understand the methods and methods of the present application. core idea; at the same time, for those of ordinary skill in the art, according to the idea of this application, there will be changes in the specific implementation and application scope. In summary, the content of this specification should not be construed as limiting the application .

Claims

An image processing method, characterized in that, comprising:

reading the first image block of the reference image and the second image block of the image to be processed from the memory;

performing image registration using the first image block and the second image block;

Wherein, the reference image is divided into multiple image blocks according to the size of the first image block, and the reference image is continuously stored in the memory in units of image blocks; and/or, the image to be processed is The size of the second image block is divided into a plurality of image blocks, and the image to be processed is continuously stored in the memory in units of image blocks.
The method according to claim 1, characterized in that, in the reference image and/or the image to be processed, the storage addresses of the pixels in each image block in the memory are continuous.
The method according to claim 1 or 2, characterized in that the method further comprises:

After acquiring the reference image, buffering the reference image in a first buffer comprising a plurality of cache lines, each of which is used to cache a row of pixels of the reference image;

For each image block of the reference image, read specified pixels from a target cache line in the multiple cache lines according to the size of the first image block, and store the specified pixels continuously in the memory .
The method according to claim 1 or 2, characterized in that the method further comprises:

Cache the image to be processed in a first buffer comprising a plurality of cache lines, each of which is used to cache a row of pixels of the image to be processed;

For each image block of the image to be processed, read specified pixels from a target cache line in the multiple cache lines according to the size of the second image block, and store the specified pixels consecutively in the memory storage.
The method according to claim 3 or 4, wherein the first image block and the second image block have the same size; the sizes of the first image block and the second image block include width and high;

The number of the target cache lines is determined according to one of the height and width; the number of specified pixels is determined according to the other of the height and width.
The method according to claim 3 or 4, wherein the number of the first buffer zones is at least two;

The reference image is cached in one of the first buffers; the image to be processed is cached in the other first buffer.
The method according to claim 1, characterized in that, the first image block of the reference image and the second image block of the image to be processed are read from a memory in a burst continuous reading manner.
The method according to claim 7, wherein said reading the first image block of the reference image and the second image block of the image to be processed from the memory comprises:

Using the first burst signal to read the first image block of the reference image from the memory, and using the second burst signal to read the second image block of the image to be processed from the memory.
The method according to claim 8, wherein the burst length of the first burst signal is determined according to the number of pixels contained in the first image block, the pixel bit width and the bit width of the memory; and/ Or, the burst length of the second burst signal is determined according to the number of pixels included in the second image block, the pixel bit width, and the bit width of the memory.
The method according to claim 8, wherein the number of the first burst signal and/or the second burst signal is less than or equal to four.
The method according to claim 8 or 10, wherein the number of the first burst signals is determined according to the number of image blocks including pixels of the first image block;

And/or, the quantity of the second burst signal is determined according to the number of image blocks including the pixels of the second image block.
The method according to claim 1, wherein the first image block and the second image block are determined according to key points extracted from the reference image.
The method according to claim 1, wherein reading the first image block of the reference image and the second image block of the image to be processed from the memory comprises:

reading a reference image area including the first image block and/or an image area to be processed including the second image block from a memory;

Wherein, the data in the reference image area except for the first image block is used for the next image registration process; the data in the image area to be processed except for the second image block is used for the next One-time image registration process.
The method according to claim 13, wherein the reference image area and/or the image area to be processed are cached in a second buffer.
The method according to claim 14, characterized in that, in the second buffer, the buffer capacity of the buffer interval corresponding to the reference image area and/or the buffer capacity of the buffer interval corresponding to the image area to be processed Both are determined according to the number of keypoints extracted from the reference image.
The method according to claim 15, wherein the buffer capacity of the buffer interval corresponding to the reference image area is positively correlated with the number of key points;

And/or, the buffer capacity of the buffer interval corresponding to the image area to be processed has a negative correlation with the number of key points.
The method according to any one of claims 14 to 16, characterized in that, in the second buffer, the buffer capacity of the buffer interval corresponding to the reference image area is smaller than that of the buffer interval corresponding to the image area to be processed cache capacity.
The method according to claim 14, wherein the number of the second buffer zone is at least two;

The reference image area is cached in one of the second buffers; the image area to be processed is cached in the other second buffer.
The method according to claim 14, wherein the reading the reference image area including the first image block from the memory comprises:

determining whether the first image block is cached in the second buffer;

If the first image block is cached in the second buffer, read the first image block from the second buffer;

If the first image block is not cached in the second buffer, read a reference image area including the first image block from the memory;

If part of the data of the first image block is buffered in the second buffer, read the part of the data from the second buffer and read from the memory including the data except the part of the data Reference image area for other data.
The method according to claim 19, wherein the reference image area cached in the second buffer zone corresponds to a first storage identifier, and the first storage identifier is used to indicate that the reference image area is in a location in said reference image;

The determining whether the first image block is cached in the second buffer includes:

According to the key point extracted from the reference image and the first storage identifier, determine whether the first image block related to the key point is cached in the second buffer.
The method according to claim 14, wherein the reading the image area to be processed including the second image block from the memory comprises:

determining whether the second image block is cached in the second buffer;

If the second image block is cached in the second buffer, read the second image block from the second buffer;

If the second image block is not cached in the second buffer, read the image area to be processed including the second image block from the memory;

If part of the data of the second image block is cached in the second buffer, read the part of the data from the second buffer and read from the memory including the parts except the part of the data The image area to be processed for other data.
The method according to claim 21, wherein the region of the image to be processed cached in the second buffer zone corresponds to a second storage identifier, and the second storage identifier is used to indicate the image to be processed The position of the region in the image to be processed;

The determining whether the second image block is cached in the second buffer includes:

Determine whether the second image block related to the key point is cached in the second buffer according to the key point extracted from the reference image and the second storage identifier.
An image processing device, characterized in that it includes a memory, a data loading module, and a processor;

The memory is used to store reference images and images to be processed;

The data loading module is used to read the first image block of the reference image and the second image block of the image to be processed from the memory; wherein, the reference image is divided into the size of the first image block A plurality of image blocks, the reference image is continuously stored in the memory in units of image blocks; and/or, the image to be processed is divided into a plurality of image blocks according to the size of the second image block, so The image to be processed is continuously stored in the memory in units of image blocks;

The processor is configured to perform image registration using the first image block and the second image block.
The device according to claim 23, characterized in that, in the reference image and/or the image to be processed, the storage addresses of the pixels in each image block in the memory are continuous.
The device according to claim 23 or 24, wherein the device further comprises a first conversion module;

The first conversion module is configured to: after acquiring the reference image, cache the reference image in a first buffer including a plurality of cache lines, each of the cache lines is used to cache the reference image A row of pixels; for each image block of the reference image, read specified pixels from the target cache line in the plurality of cache lines according to the size of the first image block, and store the specified pixel in the memory in continuous storage.
The device according to claim 23 or 24, wherein the device further comprises a second conversion module;

The second conversion module is configured to: cache the image to be processed in a first buffer including a plurality of cache lines, each cache line is used to cache a row of pixels of the image to be processed; for the For each image block of the image to be processed, read specified pixels from a target cache line in the plurality of cache lines according to the size of the second image block, and store the specified pixels continuously in the memory.
The device according to claim 25 or 26, wherein the first image block and the second image block have the same size; the sizes of the first image block and the second image block include width and high;

The number of the target cache lines is determined according to one of the height and width; the number of specified pixels is determined according to the other of the height and width.
The device according to claim 25 or 26, wherein the number of the first buffer zones is at least two;

The reference image is cached in one of the first buffers; the image to be processed is cached in the other first buffer.
The device according to claim 23, wherein the data loading module is specifically configured to read the first image block of the reference image and the image block of the image to be processed from the memory in a burst continuous read mode. the second image block.
The device according to claim 29, wherein the data loading module is specifically configured to: use the first burst signal to read the first image block of the reference image from the memory, and use the second burst signal The second image block of the image to be processed is read from the memory.
The device according to claim 30, wherein the burst length of the first burst signal is determined according to the number of pixels included in the first image block, the pixel bit width, and the bit width of the memory;

And/or, the burst length of the second burst signal is determined according to the number of pixels contained in the second image block, the pixel bit width, and the bit width of the memory.
The device according to claim 30, wherein the number of the first burst signal and/or the second burst signal is less than or equal to four.
The device according to claim 30 or 32, wherein the number of the first burst signals is determined according to the number of image blocks including the pixels of the first image block;

And/or, the quantity of the second burst signal is determined according to the number of image blocks including the pixels of the second image block.
The device according to claim 23, wherein the first image block and the second image block are determined according to key points extracted from the reference image.
The device according to claim 23, wherein the data loading module is specifically configured to: read the reference image area including the first image block and/or the reference image area including the second image block from the memory. The image area to be processed; wherein, the data in the reference image area except the first image block is used for the next image registration process; the data in the image area to be processed except the second image block The data is used for the next image registration process.
The device according to claim 35, wherein the reference image area and/or the image area to be processed are cached in a second buffer.
The device according to claim 36, wherein in the second buffer, the buffer capacity of the buffer interval corresponding to the reference image area and/or the buffer capacity of the buffer interval corresponding to the image area to be processed Both are determined according to the number of keypoints extracted from the reference image.
The device according to claim 37, wherein the buffer capacity of the buffer interval corresponding to the reference image area is positively correlated with the number of key points;

And/or, the buffer capacity of the buffer interval corresponding to the image area to be processed has a negative correlation with the number of key points.
The device according to any one of claims 36 to 38, wherein, in the second buffer zone, the buffer capacity of the buffer zone corresponding to the reference image area is smaller than the buffer zone corresponding to the image area to be processed cache capacity.
The device according to claim 36, wherein there are at least two second buffer zones;

The reference image area is cached in one of the second buffers; the image area to be processed is cached in the other second buffer.
The device according to claim 36, wherein the data loading module is specifically used for:

determining whether the first image block is cached in the second buffer;

If the first image block is cached in the second buffer, read the first image block from the second buffer;

If the first image block is not cached in the second buffer, read a reference image area including the first image block from the memory;

If part of the data of the first image block is buffered in the second buffer, read the part of the data from the second buffer and read from the memory including the data except the part of the data Reference image area for other data.
The device according to claim 41, wherein the reference image region cached in the second buffer zone corresponds to a first storage identifier, and the first storage identifier is used to indicate that the reference image region is in a location in said reference image;

The data loading module is further configured to: determine whether the first image block related to the key point is cached in the second buffer according to the key point extracted from the reference image and the first storage identifier .
The device according to claim 36, wherein the data loading module is specifically used for:

determining whether the second image block is cached in the second buffer;

If the second image block is cached in the second buffer, read the second image block from the second buffer;

If the second image block is not cached in the second buffer, read the image area to be processed including the second image block from the memory;

If part of the data of the second image block is cached in the second buffer, read the part of the data from the second buffer and read from the memory including the parts except the part of the data The image area to be processed for other data.
The device according to claim 43, wherein the region of the image to be processed cached in the second buffer zone corresponds to a second storage identifier, and the second storage identifier is used to indicate the image to be processed The position of the region in the image to be processed;

The data loading module is specifically configured to: determine whether the second image block related to the key point is cached in the second buffer according to the key point extracted from the reference image and the second storage identifier .
A mobile platform, characterized in that it comprises:

body;

a power system mounted on the fuselage for powering the movable platform; and,

The image processing device according to any one of claims 23 to 44.
A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, and when the computer program is executed by the image processing device, the image according to any one of claims 1 to 22 is realized Approach.