WO2022188020A1 - 图像处理方法、装置、可移动平台及存储介质 - Google Patents

图像处理方法、装置、可移动平台及存储介质 Download PDF

Info

Publication number
WO2022188020A1
WO2022188020A1 PCT/CN2021/079684 CN2021079684W WO2022188020A1 WO 2022188020 A1 WO2022188020 A1 WO 2022188020A1 CN 2021079684 W CN2021079684 W CN 2021079684W WO 2022188020 A1 WO2022188020 A1 WO 2022188020A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
reference image
memory
storage address
read
Prior art date
Application number
PCT/CN2021/079684
Other languages
English (en)
French (fr)
Inventor
高明明
张清顺
陈琳
林蔓虹
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to PCT/CN2021/079684 priority Critical patent/WO2022188020A1/zh
Publication of WO2022188020A1 publication Critical patent/WO2022188020A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis

Definitions

  • the present application relates to the technical field of image processing, and in particular, to an image processing method, device, movable platform and storage medium.
  • image registration methods are used in target detection, moving target tracking, motion estimation, video analysis, target localization, model reconstruction, remote sensing data analysis, and aerial reconnaissance.
  • the general process of one of the image registration methods is as follows: feature extraction is performed on two images to obtain feature information; image block data related to the feature information is read from the storage address where the two images are stored in the memory, The similarity measure determines the similarity between the image blocks of the two images to determine the matching feature information; then obtains the image space coordinate transformation parameters through the matching feature information; finally, image registration is performed based on the image space coordinate transformation parameters.
  • the image block data of the image needs to be repeatedly read from the memory for many times. Since the storage addresses of the image block data are discontinuous, multiple A read instruction is used to read the image block data, which causes the problem of low read efficiency for the memory; and in the actual process, since the bit width of the memory is usually larger than the pixel bit width of the image, a storage unit in the memory The data of multiple pixels can be stored, and the data read from the memory is read in units of storage units, so that when reading the required image block data from the memory, it may be necessary to read more data than the image block data. , resulting in a large bandwidth loss.
  • one of the objectives of the present application is to provide an image processing method, device, movable platform and storage medium.
  • an embodiment of the present application provides an image processing method, including:
  • the image blocks in the reference image are read from the consecutive first storage addresses of the memory; the image blocks in the reference image are written in advance from the second storage address of the memory to the first storage address. in a storage address; the image blocks are stored discontinuously in the second storage address;
  • Image registration is performed on the image blocks in the reference image and the image to be processed.
  • an embodiment of the present application provides an image processing apparatus, including a data loading module, a memory, and a processor;
  • the memory for storing the image block in the reference image and the image to be processed
  • the data loading module is configured to read the image blocks in the reference image from the continuous first storage address of the memory in a burst continuous reading mode, and read the to-be-processed image from the memory;
  • the image blocks in the reference image are written in the first storage address from the second storage address of the memory in advance; the image blocks are stored discontinuously in the second storage address;
  • the processor is configured to perform image registration between the image blocks in the reference image and the image to be processed.
  • an embodiment of the present application provides a movable platform, including:
  • a power system mounted on the fuselage for powering the movable platform
  • the image processing apparatus according to the second aspect.
  • an embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and the computer program implements the first aspect when executed by the image processing apparatus image processing method.
  • a burst continuous reading method is used to read image blocks in a reference image from a continuous first storage address in a memory; the The image blocks in the reference image are written in advance from the second storage address of the memory into the first storage address; the image blocks are stored discontinuously in the second storage address; and then the reference image is stored in the first storage address.
  • the image block is registered with the image to be processed.
  • the image block data of the reference image that needs to be used for image registration is re-stored, and stored in a continuous first storage address. When the image block of the reference image needs to be read later, the burst data can be used.
  • the continuous reading method is used to read the relevant image block data from the continuous first storage address at one time, which is conducive to improving the reading efficiency of the memory and further shortening the image registration time; and because the image blocks are continuous at the first storage address.
  • other irrelevant data can also be reduced or not required to be read, which is beneficial to saving the bandwidth of the memory.
  • FIG. 1 is a schematic diagram of a reference image provided by an embodiment of the present application.
  • burst length is 4
  • burst length is 4
  • FIG. 3 is a schematic diagram of optical flow tracking provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of an application scenario provided by an embodiment of the present application.
  • FIG. 5 is a schematic flowchart of an image processing method provided by an embodiment of the present application.
  • FIG. 6 is a schematic diagram of storage of an image block at a first storage address provided by an embodiment of the present application
  • FIG. 7 and 8 are schematic diagrams of different structures of an image processing apparatus provided by an embodiment of the present application.
  • FIG. 9 is a schematic structural diagram of a movable platform provided by an embodiment of the present application.
  • the memory when an image needs to be separately registered with other different images, it is necessary to repeatedly read the image block data of the image from the memory, and the memory usually takes the whole image data as a unit. storage, and the image block is only a small part of the image, although the entire image data is stored in a continuous address, the storage address of the image block data is usually discontinuous, which requires the use of multiple read instructions from different storage
  • the image block data is read by address, which causes the problem of low reading efficiency of the memory; and in the actual process, because the bit width of the memory is usually larger than the pixel bit width of the image, one storage unit in the memory can store multiple pixels.
  • the data read from the memory is read in units of storage units, so that when reading the required image block data from the memory, it may be necessary to read more data than the image block data, resulting in larger bandwidth loss.
  • the image shown in Figure 1 is usually stored in the memory continuously in raster order from left to right and from top to bottom, while In the process of image registration, it is usually only necessary to read the image block data in the image.
  • the storage address of the image block data is discontinuous, at least need to use 3 read commands, such as 3 burst signals, the first burst signal is used to read pixels 1, 2, and 3; the second burst signal is used to read pixels 4, 5, and 6; The third burst signal is used to read pixels 7, 8, and 9, so as to obtain the image block data in the gray part, and there is a problem that the reading efficiency of the memory is low.
  • 3 read commands such as 3 burst signals
  • a row of 64 pixel data can be read, and the image block data size is 16*16, then 16 burst signals (burst length 4) are required to read the 64*16 data as shown in Figure 2, that is More data than the image block data is read, resulting in a waste of memory bandwidth, and the read data cannot be used directly, and further processing is required to obtain 16*16 image block
  • an embodiment of the present application provides an image processing method, which adopts a burst continuous reading method to read image blocks in a reference image from consecutive first storage addresses in a memory; the image blocks in the reference image are Writing into the first storage address from the second storage address of the memory in advance; the image blocks are stored discontinuously in the second storage address; and then the image blocks in the reference image and the image to be processed are stored Perform image registration.
  • the image block data of the reference image that needs to be used for image registration is re-stored, and stored in a continuous first storage address. When the image block of the reference image needs to be read later, the burst data can be used.
  • the continuous reading method is used to read the relevant image block data from the continuous first storage address at one time, which is conducive to improving the reading efficiency of the memory and further shortening the image registration time; and because the image blocks are continuous at the first storage address.
  • other irrelevant data can also be reduced or not required to be read, which is beneficial to saving the bandwidth of the memory.
  • image registration can be applied to target detection, moving target tracking, motion estimation, video analysis, target positioning, model reconstruction, remote sensing data analysis, aerial reconnaissance and so on.
  • the KLT optical flow tracking algorithm can be used to implement the moving target tracking process, see Figure 3, the KLT optical flow tracking algorithm determines the feature point u in the image I in the image J by determining the feature point u. position, so as to realize the moving target tracking process. In this process, it is necessary to read the image block data centered on the feature point u from the position where the image I is stored in the memory, and use the image block data and the image J to perform KLT optical flow tracking algorithm processing, so as to determine the feature point u in the image. position in J.
  • the image block data needs multiple read instructions to read, resulting in the problem of low memory read efficiency; and in the actual process, because the bit width of the memory is usually larger than the pixel bit width of the image, a storage unit in the memory
  • the data of multiple pixels can be stored, and the data read from the memory is read in units of storage units, so that when reading the required image block data from the memory, it may be necessary to read more data than the image block data.
  • the image processing method can be used to re-store the image block data about the feature point u in the image I, and store it in a continuous first storage address.
  • the image block data can be read from the continuous first storage address at one time in a burst continuous reading mode, which is beneficial to improve the reading efficiency of the memory, thereby further shortening the image registration time, and can effectively save the bandwidth of the memory .
  • the image processing methods in the embodiments of the present application may be applied to an image processing apparatus.
  • the image processing apparatus may be an electronic device with data processing capability, such as a computer, a server, a cloud server or a terminal, a movable platform (such as an unmanned aerial vehicle, an unmanned vehicle or a mobile robot) and many more.
  • a computer such as a computer, a server, a cloud server or a terminal
  • a movable platform such as an unmanned aerial vehicle, an unmanned vehicle or a mobile robot
  • the image processing device may also be a computer chip or integrated circuit with data processing capability, such as a central processing unit (Central Processing Unit, CPU), a digital signal processor (Digital Signal Processor, DSP), Application Specific Integrated Circuit (ASIC) or off-the-shelf Programmable Gate Array (Field-Programmable Gate Array, FPGA), etc.
  • the image processing apparatus may be installed on an electronic device, for example, the image processing apparatus may be installed on a movable platform such as an unmanned aerial vehicle, an unmanned vehicle, or a mobile robot.
  • the image processing apparatus may be an unmanned aerial vehicle or the image processing apparatus may be a chip or an integrated circuit installed in an unmanned aerial vehicle.
  • the unmanned aerial vehicle 100 can realize the following shooting function of the target object through the image processing device.
  • the drone is installed with an imaging device 200, and a plurality of images of the target object are collected by the imaging device 200, and then the image processing device can be used to perform the image processing method, and the plurality of images can be processed.
  • Image registration is performed to automatically track the movement change process of the target object, so that the unmanned aerial vehicle 100 can follow and shoot the target object according to the result of the image registration.
  • the KLT optical flow tracking algorithm can be used to implement the target object tracking process.
  • Other target tracking algorithms can also be used, which is not limited in this embodiment.
  • the KLT optical flow tracking algorithm realizes the moving target tracking process by determining the position of the feature point u in the image I in the image J.
  • the drone may determine a reference image (ie, image I) from a plurality of images captured by the imaging device 200, and then acquire the reference image from the second storage address in the memory where the reference image is stored.
  • the image block data is re-stored to the continuous first storage address, that is, the image block data about the feature point u in the image I is re-stored, and stored in the continuous first storage address, and the image block data needs to be used subsequently.
  • the image block data can be read from the consecutive first storage addresses at one time by means of burst continuous reading, which is beneficial to improve the reading efficiency of the memory, thereby further shortening the image registration time; then use the image block data Image registration is performed with other images to be processed, and the movement change process of the target object is automatically tracked, so that the UAV 100 can also follow and shoot the target object according to the result of the image registration.
  • FIG. 5 is a schematic flowchart of an image processing method provided by the embodiment of the present application.
  • the method may be executed by an image processing apparatus. Methods include:
  • step S101 the image blocks in the reference image are read from the consecutive first storage addresses of the memory in a burst continuous reading manner; the image blocks in the reference image are pre-fetched from the second storage address of the memory Write into the first storage address; the image blocks are stored discontinuously in the second storage address.
  • step S102 image registration is performed on the image blocks in the reference image and the image to be processed.
  • the image block data that needs to be reused in this image may be repeated. Re-storage, the image block data is stored in the unit of image block to the continuous first storage address, and then when the image block of the reference image is read from the memory, it can be read from the continuous first storage address. Reading at the second storage address improves the reading efficiency of the memory, thereby further shortening the time for image registration.
  • a reference image in a real-time image acquisition scenario, can be determined at a specified time interval, for example, a reference image can be determined every 1 second, the reference image A is selected in the first second, and the subsequent acquisition is to be processed. The image is registered with the reference image A, and the reference image B is selected in the second second, and the subsequently collected to-be-processed image is registered with the reference image B, and so on.
  • the reference image when a reference image is selected from multiple images, the reference image can be selected according to image information, the image information can be used to reflect the sharpness of the image, and the image information includes but is not limited to: signal-to-noise ratio, image gradient, local variance, or mean square error (Mean Square Error, MSE), etc., the one with the most image information among the multiple images to be selected may be used as the reference image.
  • MSE mean square error
  • the embodiments of the present application do not impose any restrictions on the specific method of acquiring image information, and specific selections can be made according to actual application scenarios.
  • the image information is image gradient information
  • the Brenner gradient function and the Tenengrad gradient function can be used.
  • Laplacian gradient function or energy gradient function to obtain the image gradient information of the image.
  • the image blocks in the reference image are determined based on feature information extracted from the reference image.
  • the feature information includes position information of at least one of the following features in the reference image: corner points, edge points, high curvature points, contours, intersection points, line segments, closed boundaries, and center of gravity.
  • the image processing apparatus may The image block related to the feature point is acquired from the second storage location where the reference image is stored in the memory according to the position information of the feature point.
  • the image block may be the pixel at the location of the feature point and its neighbors. domain pixels, etc.
  • the image block may be stored in a continuous first storage address of the memory.
  • whether to perform the image processing method of this embodiment may be determined by setting an image processing mode.
  • the image processing mode includes a first mode, the first mode instructs to reuse the reference image, and the image processing apparatus performs the image processing of this embodiment when the current image processing mode is in the first mode The method; if the current image processing mode is not in the first mode, read the image block in the reference image from the second storage address where the reference image is stored according to the processing method of the related art.
  • the opening manner of the first mode may be specifically set according to the actual application scenario, which is not limited in this embodiment.
  • the first mode may be turned on based on a user instruction.
  • the first mode may also be adaptively enabled based on state information of other devices that cooperate with the image processing device.
  • the image processing device may respond to the The unmanned aerial vehicle is performing the target object following function, and the first mode is turned on.
  • the image The processing device may read an image block in the reference image from the second storage address in the memory for storing the reference image, so as to perform image registration on the image block with the image to be processed, and The image processing apparatus may write the image blocks read from the second storage address into consecutive first storage addresses, so that the image of the reference image may be quickly read from the first storage address subsequently Block data for image registration with other images to be processed.
  • the image block data of the reference image is processed in units of image blocks. Re-storing is convenient for subsequent quick reading of image blocks in the reference image, and is also conducive to improving image registration efficiency.
  • the image block of the reference image when the image block of the reference image is stored in the first storage address, the image block may be processed in the order in which the pixels in the image block are processed in the image registration process. If the pixels in the image block are written into the first storage address, after the image block is subsequently read from the first storage address, no other processing process is required, and the image processing device can Each pixel in the image block is processed, so as to further improve the image registration efficiency.
  • the sequence of processing each pixel in the image block during the image registration process is 1 ⁇ 2 ⁇ 3 ⁇ 4 ⁇ 5 ⁇ 6 ⁇ 7 ⁇ 8 ⁇ 9, the pixels in the image block can be written into the first storage address in this order.
  • the storage method is shown in Figure 6. The pixels in the image block follow the processing order in the image registration process. Continuous storage at the first storage address.
  • the reference images are usually processed in the order in which the pixels in the image blocks are processed in the image registration process.
  • the pixels in the image block of the reference image are output to the processor in the image processing device for image registration, in other words, in the process of storing the image block of the reference image to the first storage address, it can be output to The sequence in which the processor in the image processing device performs image registration, writes the pixels in the image block into the first storage address, and after the image block is subsequently read from the first storage address, there is no need to do so.
  • the image can be directly output to the processor in the image processing device for image registration, which is beneficial to further improve the image registration efficiency.
  • the image processing apparatus may read the image block in the reference image from the first storage address.
  • the first storage address is stored in units of image blocks and the storage addresses of the image blocks are continuous, compared to reading image blocks from the second storage address, the memory bandwidth is saved and the read rate is improved. Take efficiency.
  • whether the reference image is used for the first time may be determined according to an external instruction (for example, the external instruction may indicate the number of times of use of the reference image), for example, if the external instruction indicates that the number of times of use of the reference image is 0, It is determined that the reference image is used for the first time, otherwise it is determined that the reference image is not used for the first time.
  • an external instruction for example, the external instruction may indicate the number of times of use of the reference image
  • the image processing apparatus since the image processing apparatus reads the image blocks in the reference image according to the feature information of the reference image, after the image processing apparatus acquires the feature information of the reference image , it can be compared with the feature information obtained in the last image registration process, if it is inconsistent, it is determined that the reference image is used for the first time, and if it is consistent, it is determined that the reference image is not used for the first time.
  • a data selector may be used in the image processing apparatus to determine whether to read the image block data of the reference image from the first storage address or to read the image of the reference image from the second storage address block data.
  • FIG. 7 shows a structural diagram of the image processing apparatus.
  • the image processing apparatus includes a memory 10 , a data loading module 20 and a processor 30 .
  • the memory 10 includes a first storage address, a second storage address and a third storage address, the first storage address is used to store image block data in the reference image, the second storage address is used to store the reference image, and the first storage address is used to store the image block data in the reference image.
  • Three storage addresses are used to store images to be processed.
  • the data loading module 20 includes a first loading unit 21 , a second loading unit 22 , a data selector 23 and a writing unit 24 . In the image registration process, it is assumed that the current image processing mode is the first mode.
  • the first loading unit 21 passes the data selector 23 according to the feature information of the reference image.
  • the image block of the reference image is read from the second storage address;
  • the second loading unit 22 reads the to-be-processed image from the third storage location where the to-be-processed image is stored according to the feature information of the reference image and the writing unit 24 writes the pixels in the image block of the reference image into the first image block according to the sequence of processing the pixels in the image block of the reference image in the image registration process.
  • the processor 30 is configured to perform image registration between the image blocks in the reference image and the image blocks of the image to be processed.
  • the image blocks of the reference image that need to be reused are re-stored, so that the image blocks in the reference image can be quickly read later, and it is also beneficial to improve the image registration efficiency.
  • the first loading unit 21 passes the data selector 23 according to the feature information of the reference image.
  • the image block of the reference image is read from the first storage address;
  • the second loading unit 22 reads the to-be-processed image from the third storage location where the to-be-processed image is stored according to the feature information of the reference image the image block;
  • the processor 30 is configured to perform image registration between the image block in the reference image and the image block of the image to be processed.
  • the image processing apparatus may The feature information generates a second burst signal, and then uses the second burst signal to read the image block in the reference image from the second storage address of the memory, and compares the image block of the reference image with the to-be-processed image block. Image registration is performed on the image, and image blocks in the reference image are written to a first memory address. Among them, since the image blocks of the reference image are not stored continuously at the second storage address, multiple second burst signals are required. For example, in the embodiment shown in FIG. 2 , 16 second burst signals are required, and the second The length of the burst signal is 4.
  • the image processing apparatus may generate a first burst signal according to feature information extracted from the reference image , and then use the first burst signal to read the image block in the reference image from the first storage address in the memory.
  • the image blocks in the reference image are stored continuously at the first storage address, only one first burst signal is needed to read the image block data from the first storage address.
  • the number of one burst signal is less than the number of the second burst signal, which effectively improves the access efficiency to the memory.
  • the burst length of the first burst signal can be based on the number of pixels corresponding to the image block, the data amount of each pixel, and the memory.
  • the unit storage capacity is determined. In an example, set the burst length of the first burst signal to be L, the number of pixels corresponding to the image block to be N, the data amount (ie data bit width) of each pixel to be P bits, and the memory
  • the unit storage capacity (that is, the data bit width of each storage unit) is M bit, then For example, in the embodiment shown in FIG. 2 , one first burst signal is required, and the length of the first burst signal is 16. By comparison, it can be seen that the burst length of the first burst signal is greater than the burst length of the second burst signal, and the greater the burst length, the higher the access efficiency to the memory.
  • the image processing apparatus may also be set to only perform the image processing method of this embodiment, that is, to reuse the reference image by default, without setting an image processing mode.
  • the image processing apparatus reads the reference image from the second storage address in the memory for storing the reference image. image block, to perform image registration on the image block with the image to be processed, and write the image block read from the second storage address into the first storage address; if the reference image If it is not used for the first time, the image processing apparatus reads the image block in the reference image from the first storage address, which can effectively save the bandwidth of the memory and improve the reading performance compared with reading the image block from the second storage address. Take efficiency.
  • an embodiment of the present application further provides an image processing apparatus, and the apparatus includes a data loading module 20 , a memory 10 and a processor 30 ;
  • the memory 10 is used for storing the image blocks in the reference image and the image to be processed.
  • the data loading module 20 is configured to read the image blocks in the reference image from the consecutive first storage addresses of the memory 10 in a burst continuous reading mode, and read the to-be-processed data from the memory 10 image; the image blocks in the reference image are written into the first storage address from the second storage address of the memory 10 in advance; the image blocks are stored discontinuously in the second storage address.
  • the processor 30 is configured to perform image registration between the image blocks in the reference image and the image to be processed.
  • the memory 10 may include at least one type of storage medium, and the storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory, etc. Memory (SRAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Programmable Read Only Memory (PROM), magnetic memory, magnetic disks, optical disks, and the like.
  • the memory 10 may be an internal storage unit of the device, such as a hard disk or a memory of the device.
  • the memory 10 can also be an external storage device of the device, such as a pluggable hard disk equipped on the device, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash memory card (Flash Card) and the like. Further, the memory 10 may also include both an internal storage unit of the apparatus and an external storage device. The memory 10 is used to store reference images and images to be processed. The memory 10 may also be used to temporarily store data that has been output or is to be output.
  • a pluggable hard disk equipped on the device such as a pluggable hard disk equipped on the device, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash memory card (Flash Card) and the like. Further, the memory 10 may also include both an internal storage unit of the apparatus and an external storage device. The memory 10 is used to store reference images and images to be processed. The memory 10 may also be used to temporarily store data that has been output or is to be output
  • the data loading module 20 further includes a data selector 23 and a writing unit 24 .
  • the data loading module 20 is further configured to: if it is detected that the current image processing mode indicates that the reference image is to be reused and the reference image is used for the first time, use the data selector 23 to retrieve the image from the memory 10 . reading the image block in the reference image at the second storage address where the reference image is stored; and, by the writing unit 24, reading the image block from the second storage address into the first storage address.
  • the writing unit 24 is specifically configured to: write the pixels in the image block into the first storage address according to the processing order of the pixels in the image block in the image registration process middle.
  • the data loading module 20 is further configured to: if it is detected that the current image processing mode indicates that the reference image is to be reused and the reference image is not used for the first time, use the data selector from The image block in the reference image is read at the first storage address.
  • the image blocks in the reference image are determined according to feature information extracted from the reference image.
  • the feature information includes position information of at least one of the following features in the reference image: corner points, edge points, high curvature points, contours, intersection points, line segments, closed boundaries, and center of gravity.
  • the data loading module 20 is specifically configured to: if it is detected that the current image processing mode indicates that the reference image is to be reused, and the reference image is not used for the first time, according to the data from the reference image
  • the extracted feature information generates a first burst signal; the image block in the reference image is read from the first storage address in the memory 10 using the first burst signal.
  • the burst length of the first burst signal is determined according to the number of pixels corresponding to the image block, the data amount of each pixel, and the unit storage capacity of the memory 10 .
  • the number of the first burst signal is one.
  • the data loading module 20 is specifically configured to: if it is detected that the current image processing mode indicates that the reference image is to be reused, and the reference image is used for the first time, according to the data extracted from the reference image generate a second burst signal; use the second burst signal to read the image block in the reference image from the second storage address of the memory 10; The block is written to the first memory address.
  • the number of the first burst signals is less than the number of the second burst signals.
  • the burst length of the first burst signal is greater than the burst length of the second burst signal.
  • the data loading module 20 includes a first loading unit 21 , a second loading unit 22 , a data selector 23 and a writing unit 24 .
  • the current image processing mode is the first mode.
  • the first loading unit 21 If the reference image is used for the first time, indicating that the first storage address has not yet stored the image block of the reference image, the first loading unit 21 generates a second burst signal according to the feature information of the reference image, and uses The second burst signal reads the image block of the reference image from the second storage address through the data selector 23; The image block of the to-be-processed image is read at the third storage location of the processed image; and, the writing unit 24, according to the sequence of processing each pixel in the image block of the reference image in the image registration process, writes The pixels in the image blocks of the reference image are written into the first storage address; the processor 30 is configured to perform image registration between the image blocks in the reference image and the image blocks of the image to be processed.
  • the image blocks of the reference image that need to be reused are re-stored, so that the image blocks in the reference image can be quickly read later, and it is also beneficial to improve the image registration efficiency.
  • the first loading unit 21 If the reference image is not used for the first time, indicating that the first storage address has stored the image block of the reference image, the first loading unit 21 generates a first burst signal according to the feature information of the reference image, and then Using the first burst signal and through the data selector 23 to read the image block of the reference image from the first storage address; the second loading unit 22 from the storage
  • the image blocks of the to-be-processed image are read from the third storage location of the to-be-processed image; the processor 30 is configured to perform image registration between the image blocks in the reference image and the image blocks of the to-be-processed image.
  • the bandwidth of the memory 10 is saved and the bandwidth of the memory 10 is improved. read efficiency.
  • the various embodiments described herein can be implemented using computer readable media such as computer software, hardware, or any combination thereof.
  • the embodiments described herein can be implemented using application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays ( FPGA), processors, controllers, microcontrollers, microprocessors, electronic units designed to perform the functions described herein are implemented.
  • ASICs application specific integrated circuits
  • DSPs digital signal processors
  • DSPDs digital signal processing devices
  • PLDs programmable logic devices
  • FPGA field programmable gate arrays
  • processors controllers, microcontrollers, microprocessors, electronic units designed to perform the functions described herein are implemented.
  • embodiments such as procedures or functions may be implemented with separate software modules that allow the performance of at least one function or operation.
  • the software codes may be implemented by a software application (or program) written in any suitable programming language, which may be stored in
  • the image processing device when the image processing device is a chip or integrated circuit with data processing capability, the image processing device can be installed on a movable platform, then this embodiment also provides a A movable platform, the movable platform includes:
  • the power system 02 is installed on the fuselage 01 to provide power for the movable platform; and the above-mentioned image processing device 03 .
  • the movable platform includes, but is not limited to, an unmanned aerial vehicle, an unmanned vehicle, an unmanned vessel, or a mobile robot.
  • the movable platform further includes an imaging device.
  • non-transitory computer-readable storage medium such as a memory including instructions, executable by a processor of an apparatus to perform the above-described method.
  • the non-transitory computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.
  • a non-transitory computer-readable storage medium when the instructions in the storage medium are executed by the processor of the terminal, enable the terminal to execute the above method.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Processing (AREA)

Abstract

一种图像处理方法、装置、可移动平台及存储介质，所述方法包括：采用突发连续读取方式，从存储器连续的第一存储地址处读取参考图像中的图像块；所述参考图像中的图像块被预先从所述存储器的第二存储地址写入所述第一存储地址中；所述图像块在所述第二存储地址中不连续存储；将所述参考图像中的图像块与待处理图像进行图像配准。本实施例有利于提高参考图像的图像块的读取效率，进一步缩短图像配准的时间。

Description

图像处理方法、装置、可移动平台及存储介质 技术领域
本申请涉及图像处理技术领域，具体而言，涉及一种图像处理方法、装置、可移动平台及存储介质。
背景技术
在计算机视觉领域，图像配准方法应用于目标检测、运动目标跟踪、运动估计、视频分析、目标定位、模型重建、遥感数据分析以及航空侦察等方面。其中一种图像配准方法的大致流程如下：分别对两幅图像进行特征提取得到特征信息；从存储器存储两幅图像的存储地址处读取与所述特征信息相关的图像块数据，通过进行相似性度量确定两幅图像的图像块之间的相似性，以确定匹配的特征信息；然后通过匹配的特征信息得到图像空间坐标变换参数；最后基于图像空间坐标变换参数进行图像配准。
其中，当某一图像需要与其他不同的多张图像分别进行图像配准时，需要从存储器中多次重复读取该图像的图像块数据，由于图像块数据的存储地址不连续，从而需要使用多个读取指令来读取该图像块数据，造成针对于存储器的读取效率较低的问题；而且在实际过程中，由于存储器的位宽通常大于图像的像素位宽，存储器中的一个存储单元可以存储多个像素的数据，而从存储器中读取数据是以存储单元为单位读取的，使得从存储器读取需要的图像块数据时，可能需要读取比该图像块数据更多的数据，从而造成较大的带宽损失。
发明内容
有鉴于此，本申请的目的之一是提供一种图像处理方法、装置、可移动平台及存储介质。
第一方面，本申请实施例提供了一种图像处理方法，包括：
采用突发连续读取方式，从存储器连续的第一存储地址处读取参考图像中的图像 块；所述参考图像中的图像块被预先从所述存储器的第二存储地址写入所述第一存储地址中；所述图像块在所述第二存储地址中不连续存储；
将所述参考图像中的图像块与待处理图像进行图像配准。
第二方面，本申请实施例提供了一种图像处理装置，包括数据载入模块、存储器和处理器；
所述存储器，用于存储参考图像中的图像块以及待处理图像；
所述数据载入模块，用于采用突发连续读取方式，从存储器连续的第一存储地址处读取参考图像中的图像块，以及从所述存储器中读取所述待处理图像；所述参考图像中的图像块被预先从所述存储器的第二存储地址写入所述第一存储地址中；所述图像块在所述第二存储地址中不连续存储；
所述处理器，用于将所述参考图像中的图像块与待处理图像进行图像配准。
第三方面，本申请实施例提供了一种可移动平台，包括：
机身；
动力系统，安装于所述机身上，用于为所述可移动平台提供动力；以及，
如第二方面所述的图像处理装置。
第四方面，本申请实施例提供了一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，所述计算机程序被所述图像处理装置执行时实现如第一方面所述的图像处理方法。
本申请实施例所提供的一种图像处理方法、装置、可移动平台及存储介质，采用突发连续读取方式，从存储器连续的第一存储地址处读取参考图像中的图像块；所述参考图像中的图像块被预先从所述存储器的第二存储地址写入所述第一存储地址中；所述图像块在所述第二存储地址中不连续存储；然后将所述参考图像中的图像块与待处理图像进行图像配准。本实施例中，将需要用于图像配准的参考图像的图像块数据进行重新存储，将其存储至连续的第一存储地址中，后续需要读取该参考图像的图像块时，可以采用突发连续读取方式从连续的第一存储地址处一次读取相关图像块数据，从而有利于提高存储器的读取效率，进一步缩短图像配准的时间；并且由于图像块在第一存储地址处连续存储，在从第一存储地址处读取该图像块数据时也可以减少或者无需读取到其他无关数据，有利于节省存储器的带宽。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使 用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1是本申请一个实施例提供的参考图像的示意图；
图2是本申请一个实施例提供的使用16个burst信号(突发长度为4)读取的数据的示意图；
图3是本申请一个实施例提供的光流跟踪的示意图；
图4是本申请一个实施例提供的一种应用场景示意图；
图5是本申请一个实施例提供的一种图像处理方法的流程示意图；
图6是本申请一个实施例提供的图像块在第一存储地址处的存储示意图；
图7以及图8是本申请一个实施例提供的图像处理装置的不同结构示意图；
图9是本申请一个实施例提供的一种可移动平台的结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
相关技术中，当某一图像需要与其他不同的多张图像分别进行图像配准时，需要从存储器中多次重复读取该图像的图像块数据，而存储器中通常以整幅图像数据为单位连续存储，而图像块只是图像中的一小部分，虽然整幅图像数据存储在连续的地址中，但图像块数据的存储地址通常是不连续的，从而需要使用多个读取指令从不同的存储地址读取所述图像块数据，造成存储器的读取效率较低的问题；而且在实际过程中，由于存储器的位宽通常大于图像的像素位宽，存储器中的一个存储单元可以存储多个像素的数据，而从存储器中读取数据是以存储单元为单位读取的，使得从存储器读取需要的图像块数据时，可能需要读取比该图像块数据更多的数据，从而造成较大的带宽损失。
在一个例子中，比如如图1所示的图像，假设图中一个格子表示一个像素，图1所示的图像在存储器中通常以从左到右及从上到下的光栅顺序连续存储，而在图像配准过程中，通常只需要读取图像中的图像块数据，假设需要读取的图像块数据为图1 灰色部分，显然，该图像块数据的存储地址是不连续的，至少需要使用3个读取指令，比如3个突发(burst)信号，第1个突发信号用于读取像素1,2,3；第2个突发信号用于读取像素4,5,6；第3个突发信号用于读取像素7,8,9，从而获得灰色部分的图像块数据，存在存储器的读取效率较低的问题。
在另一个例子中，在实际的读取过程中，设图像数据的像素位宽P＝8bit，存储器的数据位宽M＝128bit，即存储器中的每个存储单元可以存储16个像素，采用突发连续读取方式，假设突发(burst)信号L的突发长度为4，表示可以读取连续的4个存储单元中存储的数据，则每次读取操作至少可以读取64个像素(4*16＝64)，假设需要读取的图像块数据大小为16*16，如图2所示，图2中的灰色部分为本次需要的图像块数据，一个burst信号(突发长度为4)可以读取一行64个像素的数据，图像块数据大小为16*16，则需要16个burst信号(突发长度为4)读取到如图2所示的64*16的数据，即读取了比该图像块数据更多的数据，从而造成浪费存储器带宽的情况，且读取的数据并不能直接使用，还需要进一步处理以便获取16*16的图像块数据，操作较为繁琐；同时，为了避免这种带宽损失，burst信号的突发长度也只能取较小的值(比如取4)，存在存储器的读取效率较低的问题。
基于此，本申请实施例提供了一种图像处理方法，采用突发连续读取方式，从存储器连续的第一存储地址处读取参考图像中的图像块；所述参考图像中的图像块被预先从所述存储器的第二存储地址写入所述第一存储地址中；所述图像块在所述第二存储地址中不连续存储；然后将所述参考图像中的图像块与待处理图像进行图像配准。本实施例中，将需要用于图像配准的参考图像的图像块数据进行重新存储，将其存储至连续的第一存储地址中，后续需要读取该参考图像的图像块时，可以采用突发连续读取方式从连续的第一存储地址处一次读取相关图像块数据，从而有利于提高存储器的读取效率，进一步缩短图像配准的时间；并且由于图像块在第一存储地址处连续存储，在从第一存储地址处读取该图像块数据时也可以减少或者无需读取到其他无关数据，有利于节省存储器的带宽。
其中，图像配准可以应用于目标检测、运动目标跟踪、运动估计、视频分析、目标定位、模型重建、遥感数据分析、航空侦察等方面。
示例性的，比如在运动目标跟踪场景中，比如可以使用KLT光流跟踪算法来实现运动目标跟踪过程，请参阅图3，KLT光流跟踪算法通过确定图像I中的特征点u在图像J中的位置，从而实现运动目标跟踪过程。在这个过程中，需要从存储器中存储图像I的位置读取以特征点u为中心的图像块数据，使用该图像块数据与图像J进行 KLT光流跟踪算法处理，从而确定特征点u在图像J中的位置。考虑到当图像I需要与其他不同的图像J进行多次KLT计算时，需要多次重复从存储器中读取图像I中有关于特征点u的图像块数据，而图像块数据的存储地址通常不连续，该图像块数据需要多个读取指令来读取，造成存储器读取效率低的问题；而且在实际过程中，由于存储器的位宽通常大于图像的像素位宽，存储器中的一个存储单元可以存储多个像素的数据，而从存储器中读取数据是以存储单元为单位读取的，使得从存储器读取需要的图像块数据时，可能需要读取比该图像块数据更多的数据(比如需要的图像块数据一部分存储在一个存储单元中，另一部分存储在另一个存储单元中，则需要读取2个存储单元中的数据才能获取需要的图像块数据)，从而造成较大的带宽损失。因此，可以使用本申请实施例提供的图像处理方法，将图像I中有关于特征点u的图像块数据重新存储，将其存储至连续的第一存储地址中，后续需要使用该图像块数据时，可以采用突发连续读取方式从连续的第一存储地址处一次读取该图像块数据，有利于提高存储器的读取效率，从而进一步缩短图像配准的时间，并且能够有效节省存储器的带宽。
其中，本申请实施例的所述图像处理方法可以应用于图像处理装置中。
在一种可能的实现方式中，所述图像处理装置可以是具有数据处理能力的电子设备，如电脑、服务器、云端服务器或者终端、可移动平台(例如无人飞行器、无人驾驶车辆或者移动机器人等)等。
在另一种可能的实现方式中，所述图像处理装置也可以是具有数据处理能力的计算机芯片或者集成电路，例如中央处理单元(Central Processing Unit，CPU)、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)或者现成可编程门阵列(Field-Programmable Gate Array，FPGA)等。其中，所述图像处理装置可安装于电子设备上，如所述图像处理装置可以安装在无人飞行器、无人驾驶车辆或者移动机器人等可移动平台上。
在一示例性的应用场景中，所述图像处理装置可以是无人飞行器或者所述图像处理装置可以是安装于无人机中的芯片或者集成电路。请参阅图4，所述无人飞行器100可以通过所述图像处理装置来实现目标对象跟随拍摄功能。所述无人机安装有成像装置200，通过所述成像装置200来采集针对于目标对象的多张图像，进而可以使用所述图像处理装置来执行所述图像处理方法，对所述多张图像进行图像配准，自动跟踪目标对象的运动变化过程，实现无人飞行器100可以根据图像配准的结果对目标对象进行跟随拍摄。
在图4所示的场景中，比如可以使用KLT光流跟踪算法来实现目标对象跟踪过程， 当然，也可以使用其他目标跟踪算法，本实施例对此不做任何限制。KLT光流跟踪算法通过确定图像I中的特征点u在图像J中的位置，从而实现运动目标跟踪过程。在这个过程中，所述无人机可以从成像装置200拍摄的多张图像中确定参考图像(即图像I)，然后从存储器中存储该参考图像的第二存储地址处获取所述参考图像的图像块数据并重新存储至连续的第一存储地址处，即将图像I中有关于特征点u的图像块数据重新存储，将其存储至连续的第一存储地址中，后续需要使用该图像块数据时，可以采用突发连续读取方式从连续的第一存储地址处一次读取该图像块数据，有利于提高存储器的读取效率，从而进一步缩短图像配准的时间；然后使用该图像块数据与其他待处理图像进行图像配准，自动跟踪目标对象的运动变化过程，使得所述无人飞行器100也可以根据图像配准的结果对目标对象进行跟随拍摄。
接下来对本申请实施例提供的图像处理方法进行说明：请参阅图5，图5为本申请实施例提供的一种图像处理方法的流程示意图，所述方法可以由图像处理装置来执行，所述方法包括：
在步骤S101中，采用突发连续读取方式，从存储器连续的第一存储地址处读取参考图像中的图像块；所述参考图像中的图像块被预先从所述存储器的第二存储地址写入所述第一存储地址中；所述图像块在所述第二存储地址中不连续存储。
在步骤S102中，将所述参考图像中的图像块与待处理图像进行图像配准。
在一些实施例中，如果某一图像需要与其他多张图像重复进行图像配准，则可以将该张图像(本实施例中以参考图像来说明该图像)中需要重复使用的图像块数据进行重新存储，以图像块为单位将图像块数据存储至连续的第一存储地址处，后续在从存储器读取参考图像的图像块时可以从连续的第一存储地址处读取，相较于从第二存储地址处读取，提高了存储器的读取效率，从而进一步缩短图像配准的时间。
可以理解的是，本实施例对于所述参考图像的选取方式不做任何限制，可依据实际应用场景进行具体设置。
在一个例子中，在实时的图像采集场景中，可以每隔指定时间确定一张参考图像，比如可以每隔1秒确定一张参考图像，在第1秒选择参考图像A，后续采集的待处理图像与参考图像A进行图像配准，在第2秒选择参考图像B，后续采集的待处理图像与参考图像B进行图像配准，依次类推。
在另一个例子中，在从多张图像中选择参考图像时，可以根据图像信息来选择参考图像，所述图像信息可以用来反映图像的清晰度，所述图像信息包括但不限于：信噪比、图像梯度、局部方差或者均方误差(Mean Square Error，MSE)等，可以将待 选取的多张图像中图像信息最多的一个作为所述参考图像。可以理解的是，本申请实施例对于获取图像信息的具体方式不做任何限制，可依据实际应用场景进行具体选择，例如所述图像信息为图像梯度信息时，可以通过Brenner梯度函数、Tenengrad梯度函数、Laplacian梯度函数或者能量梯度函数等方式来获取图像的图像梯度信息。
在一些实施例中，所述参考图像中的图像块根据从所述参考图像中提取的特征信息所确定。示例性的，所述特征信息包括以下至少一种特征在所述参考图像中的位置信息：角点、边缘点、高曲率点、轮廓、交点、线段、封闭边界、重心。
在一个例子中，以特征点(比如角点、边缘点、高曲率点等)为例进行说明，在对所述参考图像进行特征提取获取特征点及其位置信息之后，所述图像处理装置可以根据所述特征点的位置信息从存储器存储所述参考图像的第二存储位置处获取与所述特征点相关的图像块，比如所述图像块可以是所述特征点所在位置的像素及其邻域像素等。可选的，在所述参考图像需要被重复使用的情况下，可以将所述图像块存储至存储器的连续的第一存储地址处。
在一些实施例中，为了兼顾重复使用参考图像的图像处理过程和非重复使用参考图像的图像处理过程，可以通过设置图像处理模式来确定是否进行本实施例的所述图像处理方法。所述图像处理模式包括第一模式，所述第一模式指示重复使用所述参考图像，所述图像处理装置在当前的图像处理模式处于第一模式的情况下进行本实施例的所述图像处理方法；如果当前的图像处理模式不处于第一模式，则根据相关技术的处理方式从存储参考图像的第二存储地址处读取参考图像中的图像块。本实施例通过设置图像处理模式，可以兼顾两种图像处理过程，具有广泛的适用性。
当然，所述第一模式的开启方式可依据实际应用场景进行具体设置，本实施例对此不做任何限制。示例性的，可以基于用户指令开启所述第一模式。示例性的，也可以基于与所述图像处理装置配合的其他装置的状态信息来自适应开启所述第一模式，比如在图4所示的实施例中，所述图像处理装置可以响应于所述无人飞行器正在执行的目标对象跟随功能，开启所述第一模式。
在一实施例中，如果当前的图像处理模式处于第一模式、且所述参考图像为首次使用，此时第一存储地址处并未存储有所述参考图像的图像块数据，则所述图像处理装置可以从所述存储器中用于存储所述参考图像的所述第二存储地址处读取所述参考图像中的图像块，以将所述图像块与待处理图像进行图像配准，并且所述图像处理装置可以将从所述第二存储地址处读取的所述图像块写入连续的第一存储地址中，以便后续可以从第一存储地址处快速读取所述参考图像的图像块数据，与其他待处理图像 进行图像配准。本实施例中，为了在重复使用所述参考图像的情况下能够节省存储器带宽以及提高读取效率，在首次使用所述参考图像的情况下，将参考图像的图像块数据以图像块为单元进行重新存储，方便后续能够快速读取所述参考图像中的图像块，也有利于提高图像配准效率。
为了进一步提高图像配准效率，在将所述参考图像的图像块存储至第一存储地址时，可以按照图像配准过程中对所述图像块中各像素进行处理的顺序，将所述图像块中的像素写入所述第一存储地址中，则后续在从第一存储地址读取该图像块之后，无需再进行其他处理过程，所述图像处理装置即可按照读取的顺序对所述图像块中各像素进行处理，从而有利于进一步提高图像配准效率。
在一个例子中，假设针对于图1所述的图像块，照图像配准过程中对所述图像块中各像素进行处理的顺序为1→2→3→4→5→6→7→8→9，则可以按照该顺序将所述图像块中的像素写入所述第一存储地址中，其存储方式如图6所示，该图像块中的像素按照图像配准过程中的处理顺序在第一存储地址处连续存储。
从另一个角度考虑，为了提高图像配准效率，在读取所述参考图像的图像块之后，通常会按照图像配准过程中对所述图像块中各像素进行处理的顺序将所述参考图像的图像块中的像素输出给所述图像处理装置中的处理器进行图像配准，则换句话说，在将所述参考图像的图像块存储至第一存储地址的过程中，可以按照输出给所述图像处理装置中的处理器进行图像配准的顺序，将所述图像块中的像素写入所述第一存储地址中，则后续在从第一存储地址读取该图像块之后，无需再进行其他处理过程，即可直接输出给所述图像处理装置中的处理器进行图像配准，从而有利于进一步提高图像配准效率。
如果检测到当前的图像处理模式指示重复使用所述参考图像(即当前的处理模式处于第一模式)、且所述参考图像为非首次使用，表明第一存储地址处已经存储有所述参考图像的图像块，则所述图像处理装置可以从所述第一存储地址处读取所述参考图像中的图像块。本实施例中，由于在所述第一存储地址处是以图像块为单位进行存储且图像块的存储地址连续，相对于从第二存储地址读取图像块，节省了存储器带宽以及提高了读取效率。
可以理解的是，本实施例对于所述参考图像是否为首次使用的确定方式不做任何限制，可依据实际应用场景进行具体设置。在一个例子中，可以根据外部指令(比如该外部指令可以指示所述参考图像的使用次数)确定所述参考图像是否为首次使用，比如如果所述外部指令指示该参考图像的使用次数为0，确定所述参考图像为首次使 用，否则确定所述参考图像为非首次使用。在另一个例子中，由于所述图像处理装置是根据所述参考图像的特征信息来读取所述参考图像中的图像块，则所述图像处理装置在获取到所述参考图像的特征信息之后，可以将其与上一次图像配准过程中获取的特征信息进行比对，如果不一致，确定所述参考图像为首次使用，如果一致则确定所述参考图像为非首次使用。
在一示例性的实施例中，所述图像处理装置中可以使用数据选择器来确定是从第一存储地址处读取参考图像的图像块数据还是从第二存储地址处读取参考图像的图像块数据。
示例性的，请参阅图7，图7示出了所述图像处理装置的一种结构图，所述图像处理装置中包括有存储器10，数据载入模块20以及处理器30。其中，所述存储器10包括第一存储地址、第二存储地址以及第三存储地址，第一存储地址用于存储参考图像中的图像块数据，第二存储地址用于存储所述参考图像，第三存储地址用于存储待处理图像。其中，所述数据载入模块20包括有第一载入单元21、第二载入单元22、数据选择器23以及写入单元24。在图像配准过程中，假设当前的图像处理模式为第一模式。
如果所述参考图像为首次使用，表明第一存储地址还未存储有该参考图像的图像块，则所述第一载入单元21根据所述参考图像的特征信息，通过所述数据选择器23从第二存储地址处读取所述参考图像的图像块；所述第二载入单元22根据所述参考图像的特征信息从存储待处理图像的第三存储位置处读取所述待处理图像的图像块；并且，所述写入单元24按照图像配准过程中对所述参考图像的图像块中各像素进行处理的顺序，将所述参考图像的图像块中的像素写入所述第一存储地址中；所述处理器30用于将所述参考图像中的图像块与待处理图像的图像块进行图像配准。本实施例中，将需要重复使用的所述参考图像的图像块进行重新存储，方便后续能够快速读取所述参考图像中的图像块，也有利于提高图像配准效率。
如果所述参考图像为非首次使用，表明第一存储地址已存储有该参考图像的图像块，则所述第一载入单元21根据所述参考图像的特征信息，通过所述数据选择器23从第一存储地址处读取所述参考图像的图像块；所述第二载入单元22根据所述参考图像的特征信息从存储待处理图像的第三存储位置处读取所述待处理图像的图像块；所述处理器30用于将所述参考图像中的图像块与待处理图像的图像块进行图像配准。本实施例中，由于在所述第一存储地址处是以图像块为单位进行存储且图像块的存储地址连续，相对于从第二存储地址读取图像块，节省了存储器10带宽以及提高了读取效 率。
在一种可能的实现方式中，如果检测到当前的图像处理模式指示重复使用所述参考图像、且所述参考图像为非首次使用，所述图像处理装置可以根据从所述参考图像中提取的特征信息生成第二突发信号，然后使用所述第二突发信号从所述存储器的第二存储地址处读取所述参考图像中的图像块，将所述参考图像的图像块与待处理图像进行图像配准，以及将所述参考图像中的图像块写入第一存储地址中。其中，由于参考图像的图像块在第二存储地址处不连续存储，则需要多个第二突发信号，比如如图2所示的实施例中，需要16个第二突发信号，第二突发信号的长度为4。
如果检测到当前的图像处理模式指示重复使用所述参考图像、且所述参考图像为非首次使用，则所述图像处理装置可以根据从所述参考图像中提取的特征信息生成第一突发信号，然后使用所述第一突发信号从存储器中的所述第一存储地址处读取所述参考图像中的图像块。其中，由于参考图像中的图像块在第一存储地址处连续存储，因此只需要一个第一突发信号即可从第一存储地址处读取到所述图像块数据，对比可知，所述第一突发信号的数量少于所述第二突发信号的数量，有效提高了对存储器的访问效率。
并且由于参考图像中的图像块在第一存储地址处连续存储，则所述第一突发信号的突发长度可以根据所述图像块对应的像素数量、每个像素的数据量以及所述存储器的单位存储量确定。在一个例子中，设所述第一突发信号的突发长度为L，所述图像块对应的像素数量为N，每个像素的数据量(即数据位宽)为P bit，所述存储器的单位存储量(即每个存储单元的数据位宽)为M bit，则
Figure PCTCN2021079684-appb-000001
比如如图2所示的实施例中，需要1个第一突发信号，该第一突发信号的长度为16。对比可知，所述第一突发信号的突发长度大于所述第二突发信号的突发长度，突发长度越大，针对于存储器的访问效率越高。
在另一些实施例中，所述图像处理装置也可以被设置为只进行本实施例的所述图像处理方法，即默认重复使用所述参考图像，无需设置图像处理模式。在图像配准过程中，如果所述参考图像为首次使用，则所述图像处理装置从所述存储器中用于存储所述参考图像的所述第二存储地址处读取所述参考图像中的图像块，以将所述图像块与待处理图像进行图像配准，以及将从所述第二存储地址处读取的所述图像块写入所述第一存储地址中；如果所述参考图像为非首次使用，则所述图像处理装置从所述第一存储地址处读取所述参考图像中的图像块，相对于从第二存储地址读取图像块能够有效节省存储器的带宽以及提高读取效率。
相应的，请参阅图8，本申请实施例还提供了一种图像处理装置，所述装置包括数据载入模块20、存储器10和处理器30；
所述存储器10，用于存储参考图像中的图像块以及待处理图像。
所述数据载入模块20，用于采用突发连续读取方式，从存储器10连续的第一存储地址处读取参考图像中的图像块，以及从所述存储器10中读取所述待处理图像；所述参考图像中的图像块被预先从所述存储器10的第二存储地址写入所述第一存储地址中；所述图像块在所述第二存储地址中不连续存储。
所述处理器30，用于将所述参考图像中的图像块与待处理图像进行图像配准。
其中，所述存储器10可以包括至少一种类型的存储介质，存储介质包括闪存、硬盘、多媒体卡、卡型存储器(例如，SD或DX存储器等等)、随机访问存储器(RAM)、静态随机访问存储器(SRAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、可编程只读存储器(PROM)、磁性存储器、磁盘、光盘等等。存储器10可以是装置的内部存储单元，例如装置的硬盘或内存。存储器10也可以是装置的外部存储设备，例如装置上配备的插接式硬盘，智能存储卡(Smart Media Card,SMC)，安全数字(Secure Digital,SD)卡，闪存卡(Flash Card)等。进一步地，存储器10还可以既包括装置的内部存储单元也包括外部存储设备。存储器10用于存储参考图像和待处理图像。存储器10还可以用于暂时地存储已经输出或者将要输出的数据。
在一实施例中，请参阅图7，所述数据载入模块20还包括数据选择器23和写入单元24。
所述数据载入模块20还用于：如果检测到当前的图像处理模式指示重复使用所述参考图像、且所述参考图像为首次使用，使用所述数据选择器23从所述存储器10中用于存储所述参考图像的所述第二存储地址处读取所述参考图像中的图像块；以及，通过所述写入单元24将从所述第二存储地址处读取的所述图像块写入所述第一存储地址中。
在一实施例中，所述写入单元24具体用于：按照图像配准过程中对所述图像块中各像素的处理顺序，将所述图像块中的像素写入所述第一存储地址中。
在一实施例中，所述数据载入模块20还用于：如果检测到当前的图像处理模式指示重复使用所述参考图像、且所述参考图像为非首次使用，使用所述数据选择器从所述第一存储地址处读取所述参考图像中的图像块。
在一实施例中，所述参考图像中的图像块根据从所述参考图像中提取的特征信息 所确定。
在一实施例中，所述特征信息包括以下至少一种特征在所述参考图像中的位置信息：角点、边缘点、高曲率点、轮廓、交点、线段、封闭边界、重心。
在一实施例中，所述数据载入模块20具体用于：如果检测到当前的图像处理模式指示重复使用所述参考图像、且所述参考图像为非首次使用，根据从所述参考图像中提取的特征信息生成第一突发信号；使用所述第一突发信号从存储器10中的所述第一存储地址处读取所述参考图像中的图像块。
在一实施例中，所述第一突发信号的突发长度根据所述图像块对应的像素数量、每个像素的数据量以及所述存储器10的单位存储量确定。
在一实施例中，所述第一突发信号的数量为1个。
在一实施例中，所述数据载入模块20具体用于：如果检测到当前的图像处理模式指示重复使用所述参考图像、且所述参考图像为首次使用，根据从所述参考图像中提取的特征信息生成第二突发信号；使用所述第二突发信号从所述存储器10的第二存储地址处读取所述参考图像中的图像块；以及，将所述参考图像中的图像块写入第一存储地址中。
在一实施例中，所述第一突发信号的数量少于所述第二突发信号的数量。
在一实施例中，所述第一突发信号的突发长度大于所述第二突发信号的突发长度。
示例性的，请参阅图7，所述数据载入模块20包括有第一载入单元21、第二载入单元22、数据选择器23以及写入单元24。在图像配准过程中，假设当前的图像处理模式为第一模式。
如果所述参考图像为首次使用，表明第一存储地址还未存储有该参考图像的图像块，则所述第一载入单元21根据所述参考图像的特征信息生成第二突发信号，使用所述第二突发信号并通过所述数据选择器23从第二存储地址处读取所述参考图像的图像块；所述第二载入单元22根据所述参考图像的特征信息从存储待处理图像的第三存储位置处读取所述待处理图像的图像块；并且，所述写入单元24按照图像配准过程中对所述参考图像的图像块中各像素进行处理的顺序，将所述参考图像的图像块中的像素写入所述第一存储地址中；所述处理器30用于将所述参考图像中的图像块与待处理图像的图像块进行图像配准。本实施例中，将需要重复使用的所述参考图像的图像块进行重新存储，方便后续能够快速读取所述参考图像中的图像块，也有利于提高图像配准效率。
如果所述参考图像为非首次使用，表明第一存储地址已存储有该参考图像的图像 块，则所述第一载入单元21根据所述参考图像的特征信息生成第一突发信号，然后使用所述第一突发信号并通过所述数据选择器23从第一存储地址处读取所述参考图像的图像块；所述第二载入单元22根据所述参考图像的特征信息从存储待处理图像的第三存储位置处读取所述待处理图像的图像块；所述处理器30用于将所述参考图像中的图像块与待处理图像的图像块进行图像配准。本实施例中，由于在所述第一存储地址处是以图像块为单位进行存储且图像块的存储地址连续，相对于从第二存储地址读取图像块，节省了存储器10带宽以及提高了读取效率。
对于装置实施例而言，由于其基本对应于方法实施例，所以相关之处参见方法实施例的部分说明即可。以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。本领域普通技术人员在不付出创造性劳动的情况下，即可以理解并实施。
这里描述的各种实施方式可以使用例如计算机软件、硬件或其任何组合的计算机可读介质来实施。对于硬件实施，这里描述的实施方式可以通过使用特定用途集成电路(ASIC)、数字信号处理器(DSP)、数字信号处理装置(DSPD)、可编程逻辑装置(PLD)、现场可编程门阵列(FPGA)、处理器、控制器、微控制器、微处理器、被设计为执行这里描述的功能的电子单元中的至少一种来实施。对于软件实施，诸如过程或功能的实施方式可以与允许执行至少一种功能或操作的单独的软件模块来实施。软件代码可以由以任何适当的编程语言编写的软件应用程序(或程序)来实施，软件代码可以存储在存储器中并且由控制器执行。
在一实施例中，请参阅图9，当所述图像处理装置为具有数据处理能力的芯片或者集成电路时，所述图像处理装置可以安装于可移动平台上，则本实施例还提供了一种可移动平台，所述可移动平台包括：
机身01；
动力系统02，安装于所述机身01上，用于为所述可移动平台提供动力；以及，上述的图像处理装置03。
在一实施例中，所述可移动平台包括但不限于无人飞行器、无人驾驶车辆，无人驾驶船只或者移动机器人等。
在一实施例中，所述可移动平台还包括有成像装置。
在示例性实施例中，还提供了一种包括指令的非临时性计算机可读存储介质，例 如包括指令的存储器，上述指令可由装置的处理器执行以完成上述方法。例如，非临时性计算机可读存储介质可以是ROM、随机存取存储器(RAM)、CD-ROM、磁带、软盘和光数据存储设备等。
一种非临时性计算机可读存储介质，当存储介质中的指令由终端的处理器执行时，使得终端能够执行上述方法。
需要说明的是，在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
以上对本申请实施例所提供的方法和装置进行了详细介绍，本文中应用了具体个例对本申请的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本申请的方法及其核心思想；同时，对于本领域的一般技术人员，依据本申请的思想，在具体实施方式及应用范围上均会有改变之处，综上所述，本说明书内容不应理解为对本申请的限制。

Claims (26)

  1. 一种图像处理方法，其特征在于，包括：
    采用突发连续读取方式，从存储器连续的第一存储地址处读取参考图像中的图像块；所述参考图像中的图像块被预先从所述存储器的第二存储地址写入所述第一存储地址中；所述图像块在所述第二存储地址中不连续存储；
    将所述参考图像中的图像块与待处理图像进行图像配准。
  2. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    如果检测到当前的图像处理模式指示重复使用所述参考图像、且所述参考图像为首次使用，从所述存储器中用于存储所述参考图像的所述第二存储地址处读取所述参考图像中的图像块，以将所述图像块与待处理图像进行图像配准；
    以及，将从所述第二存储地址处读取的所述图像块写入所述第一存储地址中。
  3. 根据权利要求2所述的方法，其特征在于，所述将从所述第二存储地址处读取的所述图像块写入所述第一存储地址中，包括：
    按照图像配准过程中对所述图像块中各像素进行处理的顺序，将所述图像块中的像素写入所述第一存储地址中。
  4. 根据权利要求1或2所述的方法，其特征在于，所述从存储器中连续的第一存储地址处读取参考图像中的图像块，包括：
    如果检测到当前的图像处理模式指示重复使用所述参考图像、且所述参考图像为非首次使用，从所述第一存储地址处读取所述参考图像中的图像块。
  5. 根据权利要求1所述的方法，其特征在于，所述参考图像中的图像块根据从所述参考图像中提取的特征信息所确定。
  6. 根据权利要求5所述的方法，其特征在于，所述特征信息包括以下至少一种特征在所述参考图像中的位置信息：角点、边缘点、高曲率点、轮廓、交点、线段、封闭边界、重心。
  7. 根据权利要求1所述的方法，其特征在于，所述采用突发连续读取方式，从存储器中连续的第一存储地址处读取参考图像中的图像块，包括：
    如果检测到当前的图像处理模式指示重复使用所述参考图像、且所述参考图像为非首次使用，根据从所述参考图像中提取的特征信息生成第一突发信号；
    使用所述第一突发信号从存储器中的所述第一存储地址处读取所述参考图像中的图像块。
  8. 根据权利要求7所述的方法，其特征在于，所述第一突发信号的突发长度根据 所述图像块对应的像素数量、每个像素的数据量以及所述存储器的单位存储量确定。
  9. 根据权利要求7所述的方法，其特征在于，所述第一突发信号的数量为1个。
  10. 根据权利要求7所述的方法，其特征在于，还包括：
    如果检测到当前的图像处理模式指示重复使用所述参考图像、且所述参考图像为首次使用，根据从所述参考图像中提取的特征信息生成第二突发信号；
    使用所述第二突发信号从所述存储器的第二存储地址处读取所述参考图像中的图像块；
    以及，将所述参考图像中的图像块写入第一存储地址中。
  11. 根据权利要求10所述的方法，其特征在于，所述第一突发信号的数量少于所述第二突发信号的数量。
  12. 根据权利要求10所述的方法，其特征在于，所述第一突发信号的突发长度大于所述第二突发信号的突发长度。
  13. 一种图像处理装置，其特征在于，包括数据载入模块、存储器和处理器；
    所述存储器，用于存储参考图像中的图像块以及待处理图像；
    所述数据载入模块，用于采用突发连续读取方式，从存储器连续的第一存储地址处读取参考图像中的图像块，以及从所述存储器中读取所述待处理图像；所述参考图像中的图像块被预先从所述存储器的第二存储地址写入所述第一存储地址中；所述图像块在所述第二存储地址中不连续存储；
    所述处理器，用于将所述参考图像中的图像块与待处理图像进行图像配准。
  14. 根据权利要求13所述的装置，其特征在于，所述数据载入模块还包括数据选择器和写入单元；
    所述数据载入模块还用于：如果检测到当前的图像处理模式指示重复使用所述参考图像、且所述参考图像为首次使用，使用所述数据选择器从所述存储器中用于存储所述参考图像的所述第二存储地址处读取所述参考图像中的图像块；以及，通过所述写入单元将从所述第二存储地址处读取的所述图像块写入所述第一存储地址中。
  15. 根据权利要求14所述的装置，其特征在于，所述写入单元具体用于：按照图像配准过程中对所述图像块中各像素的处理顺序，将所述图像块中的像素写入所述第一存储地址中。
  16. 根据权利要求14所述的装置，其特征在于，所述数据载入模块还用于：如果检测到当前的图像处理模式指示重复使用所述参考图像、且所述参考图像为非首次使 用，使用所述数据选择器从所述第一存储地址处读取所述参考图像中的图像块。
  17. 根据权利要求13所述的装置，其特征在于，所述参考图像中的图像块根据从所述参考图像中提取的特征信息所确定。
  18. 根据权利要求17所述的装置，其特征在于，所述特征信息包括以下至少一种特征在所述参考图像中的位置信息：角点、边缘点、高曲率点、轮廓、交点、线段、封闭边界、重心。
  19. 根据权利要求13所述的装置，其特征在于，所述数据载入模块具体用于：
    如果检测到当前的图像处理模式指示重复使用所述参考图像、且所述参考图像为非首次使用，根据从所述参考图像中提取的特征信息生成第一突发信号；
    使用所述第一突发信号从存储器中的所述第一存储地址处读取所述参考图像中的图像块。
  20. 根据权利要求19所述的装置，其特征在于，所述第一突发信号的突发长度根据所述图像块对应的像素数量、每个像素的数据量以及所述存储器的单位存储量确定。
  21. 根据权利要求19所述的装置，其特征在于，所述第一突发信号的数量为1个。
  22. 根据权利要求19所述的装置，其特征在于，所述数据载入模块具体用于：
    如果检测到当前的图像处理模式指示重复使用所述参考图像、且所述参考图像为首次使用，根据从所述参考图像中提取的特征信息生成第二突发信号；
    使用所述第二突发信号从所述存储器的第二存储地址处读取所述参考图像中的图像块；
    以及，将所述参考图像中的图像块写入第一存储地址中。
  23. 根据权利要求22所述的装置，其特征在于，所述第一突发信号的数量少于所述第二突发信号的数量。
  24. 根据权利要求22所述的装置，其特征在于，所述第一突发信号的突发长度大于所述第二突发信号的突发长度。
  25. 一种可移动平台，其特征在于，包括：
    机身；
    动力系统，安装于所述机身上，用于为所述可移动平台提供动力；以及，
    如权利要求13至24任意一项所述的图像处理装置。
  26. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有计算机程序，所述计算机程序被所述图像处理装置执行时实现如权利要求1至12任意一项所述的图像处理方法。
PCT/CN2021/079684 2021-03-09 2021-03-09 图像处理方法、装置、可移动平台及存储介质 WO2022188020A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/079684 WO2022188020A1 (zh) 2021-03-09 2021-03-09 图像处理方法、装置、可移动平台及存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/079684 WO2022188020A1 (zh) 2021-03-09 2021-03-09 图像处理方法、装置、可移动平台及存储介质

Publications (1)

Publication Number Publication Date
WO2022188020A1 true WO2022188020A1 (zh) 2022-09-15

Family

ID=83227338

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/079684 WO2022188020A1 (zh) 2021-03-09 2021-03-09 图像处理方法、装置、可移动平台及存储介质

Country Status (1)

Country Link
WO (1) WO2022188020A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115314644A (zh) * 2022-09-16 2022-11-08 广州市保伦电子有限公司 一种基于fpga的视频数据处理系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102625086A (zh) * 2012-03-26 2012-08-01 深圳英飞拓科技股份有限公司 一种用于高清数字矩阵的ddr2存储方法和系统
CN103295222A (zh) * 2013-02-21 2013-09-11 南京理工大学 红外图像配准实现方法及其装置
CN104052954A (zh) * 2013-03-15 2014-09-17 索尼公司 图像处理设备及图像处理方法
CN105376551A (zh) * 2015-10-30 2016-03-02 杭州立体世界科技有限公司 一种基于立体影视播放装置的视频流实时翻转方法及系统
CN111052742A (zh) * 2017-09-30 2020-04-21 深圳市大疆创新科技有限公司 图像处理

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102625086A (zh) * 2012-03-26 2012-08-01 深圳英飞拓科技股份有限公司 一种用于高清数字矩阵的ddr2存储方法和系统
CN103295222A (zh) * 2013-02-21 2013-09-11 南京理工大学 红外图像配准实现方法及其装置
CN104052954A (zh) * 2013-03-15 2014-09-17 索尼公司 图像处理设备及图像处理方法
CN105376551A (zh) * 2015-10-30 2016-03-02 杭州立体世界科技有限公司 一种基于立体影视播放装置的视频流实时翻转方法及系统
CN111052742A (zh) * 2017-09-30 2020-04-21 深圳市大疆创新科技有限公司 图像处理

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115314644A (zh) * 2022-09-16 2022-11-08 广州市保伦电子有限公司 一种基于fpga的视频数据处理系统

Similar Documents

Publication Publication Date Title
US10453204B2 (en) Image alignment for burst mode images
CN109242913B (zh) 采集器相对参数的标定方法、装置、设备和介质
US20160267349A1 (en) Methods and systems for generating enhanced images using multi-frame processing
WO2017156243A1 (en) Deep-learning based feature mining for 2.5d sensing image search
US9122960B2 (en) Patch size adaptation for image enhancement
WO2014014687A1 (en) Identifying a maximally stable extremal region by skipping comparison of pixels in the region
CN110659658B (zh) 一种目标检测方法及其装置
CN103279952A (zh) 一种目标跟踪方法和装置
CN107749069B (zh) 图像处理方法、电子设备和图像处理系统
US11657485B2 (en) Method for expanding image depth and electronic device
CN111507340B (zh) 一种基于三维点云数据的目标点云数据提取方法
CN111046746A (zh) 一种车牌检测方法及装置
WO2022188020A1 (zh) 图像处理方法、装置、可移动平台及存储介质
CN112418243A (zh) 特征提取方法、装置及电子设备
WO2021056501A1 (zh) 提取特征点的方法、可移动平台及存储介质
CN112215036B (zh) 跨镜追踪方法、装置、设备及存储介质
CN109816588B (zh) 行车轨迹线的记录方法、装置及设备
CN112329729B (zh) 小目标船只检测方法、装置及电子设备
US20230069608A1 (en) Object Tracking Apparatus and Method
CN113721240B (zh) 一种目标关联方法、装置、电子设备及存储介质
WO2022241729A1 (zh) 一种图像处理方法、装置、可移动平台及存储介质
CN113869163A (zh) 目标跟踪方法、装置、电子设备及存储介质
WO2020037564A1 (zh) 图像处理方法、装置、设备及存储介质
JP2021047827A (ja) デバイス、システム、制御方法、及びプログラム
CN112037258B (zh) 目标跟踪方法、装置、设备和存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21929502

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21929502

Country of ref document: EP

Kind code of ref document: A1