WO2019041271A1

WO2019041271A1 - Image processing method, integrated circuit, processor, system and movable device

Info

Publication number: WO2019041271A1
Application number: PCT/CN2017/100061
Authority: WO
Inventors: 高明明; 杨康; 赵尧
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2017-08-31
Filing date: 2017-08-31
Publication date: 2019-03-07
Also published as: CN108701348A

Abstract

Disclosed are an image processing method, an integrated circuit, a processor, a computer system, and a movable device. Said method comprises: acquiring a first data block corresponding to a first feature point of a first image; storing the first data block in a continuous first storage space in a memory, and recording the correlation between the first feature point and the first storage space. The technical solutions of the embodiments of the present invention can improve the processing efficiency.

Description

Method, integrated circuit, processor, system, and mobile device for processing images

Copyright statement

The disclosure of this patent document contains material that is subject to copyright protection. This copyright is the property of the copyright holder. The copyright owner has no objection to the reproduction of the patent document or the patent disclosure in the official records and files of the Patent and Trademark Office.

Technical field

The present invention relates to the field of information technology and, more particularly, to a method of processing an image, an integrated circuit, a processor, a computer system, and a removable device.

Background technique

Currently, in the design of a Field-Programmable Gate Array (FPGA) or an Application Specific Integrated Circuit (ASIC), a memory such as a Double Data Rate Synchronous Dynamic Random (Double Data Rate Synchronous Dynamic Random) The bandwidth of Access Memory DDRSDRAM is increasingly becoming a bottleneck in system design. Therefore, it is beneficial to improve the performance of the system by reducing the bandwidth of each device to access the memory as much as possible.

For example, when processing an image, the image processing algorithm may need to read the image data from the memory. If random access is used, the bandwidth of the memory may occupy a lot, affecting the processing efficiency, and affecting the preemption of other devices on the system. The bandwidth of the memory, which in turn affects the performance of the entire system.

Therefore, how to improve the processing efficiency has become an urgent technical problem in chip design.

Summary of the invention

Embodiments of the present invention provide a method, an integrated circuit, a processor, a computer system, and a mobile device for processing an image, which can improve processing efficiency.

A first aspect provides a method for processing an image, including: acquiring a first data block corresponding to a first feature point of a first image; storing the first data block in a first storage space in a memory, and Recording a correspondence between the first feature point and the first storage space.

In a second aspect, an integrated circuit is provided, including: a processing unit and a cache; wherein the processing unit is configured to acquire, from the cache, a first data block corresponding to a first feature point of the first image; The first data block is stored in the first storage space in the memory, and the corresponding relationship between the first feature point and the first storage space is recorded.

In a third aspect, a processor is provided, comprising the integrated circuit of the second aspect described above.

In a fourth aspect, a computer system is provided, comprising the integrated circuit of the second aspect or the processor of the third aspect, and a memory.

In a fifth aspect, a computer system is provided, comprising: a memory for storing computer executable instructions; a processor for accessing the memory and executing the computer executable instructions to perform the method of the first aspect above The operation in .

According to a sixth aspect, a mobile device is provided, comprising: the integrated circuit of the second aspect; or the processor of the third aspect; or the computer system of the fourth or fifth aspect.

In a seventh aspect, a computer storage medium is provided having stored therein program code, the program code being operative to indicate a method of performing the first aspect described above.

In the technical solution of the embodiment of the present invention, the data blocks corresponding to the feature points are continuously stored in the memory, and the corresponding relationship between the feature points and the storage space is recorded. In this way, when the data block corresponding to the feature point needs to be read, the data block can be continuously read from the storage space according to the correspondence, and the reading efficiency can be improved with respect to the random access reading, thereby improving the processing efficiency.

DRAWINGS

FIG. 1 is an architectural diagram of a technical solution to which an embodiment of the present invention is applied.

2 is a schematic architectural diagram of a mobile device according to an embodiment of the present invention.

FIG. 3 is a schematic flowchart of a method for processing an image according to an embodiment of the present invention.

4 is a schematic block diagram of an integrated circuit in accordance with an embodiment of the present invention.

Figure 5 is a schematic block diagram of a computer system in accordance with one embodiment of the present invention.

Figure 6 is a schematic block diagram of a computer system in accordance with another embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be described below with reference to the accompanying drawings.

It should be understood that the specific examples herein are merely intended to provide a better understanding of the embodiments of the invention.

It should also be understood that in various embodiments of the present invention, the size of the sequence numbers of the processes does not imply a sequence of executions, and the order of execution of the processes should be determined by its function and internal logic, and should not be The implementation process of the embodiments of the present invention constitutes any limitation.

It should be understood that the various embodiments described in the specification may be implemented separately or in combination, and the embodiments of the present invention are not limited thereto.

The technical solution of the embodiment of the present invention can be applied to various image processing algorithms, such as the Kanade-Lucas-Tomasi (KLT) algorithm, the friction algorithm, or the orb algorithm, but the embodiment of the present invention is not limited thereto.

Image processing algorithms involve the processing of feature points when processing images. The technical solution of the embodiment of the present invention stores the data block corresponding to the feature point into a continuous storage space in the memory, and records the correspondence between the feature point and the storage space, so that the data block can be continuously read from the memory to improve the reading. Efficiency, which can improve processing efficiency.

In some possible designs, the manner in which the feature points are obtained may be determined by an image processing algorithm, such as a harris corner algorithm, a FAST (features from accelerated segment test) algorithm to detect feature points, etc., which is not limited in this embodiment of the present invention. .

In some possible designs, the data block of the feature point may include data corresponding to a minimum pixel area of a pixel of the image selected by the image processing algorithm for the feature point. That is to say, the data block of the feature point includes data corresponding to all the pixels required to process the feature point. The minimum pixel area may be a minimum rectangular pixel area of a pixel required to process the feature point, and the minimum rectangular pixel area is determined according to an image processing algorithm and a predetermined value, for example, when processing the KLT algorithm, according to accuracy requirements You can choose different sizes of square areas such as 3*3, 5*5, etc., or different sizes of rectangular areas such as 3*5, 5*3, etc. Of course, the minimum pixel area can also be a non-rectangular area, for example, it can be The set of pixels whose distance from the feature point is N may also be a discrete set of pixels, which is not limited herein.

As shown in FIG. 1, system 100 includes a processor 110 and an external memory 120. In some embodiments, the processor 110 can be a processor in a computing device or a processor in a mobile device (eg, a drone). The processor 110 can be any type of processor, which is not limited in this embodiment of the present invention. In some embodiments, the processor 110 can be comprised of an internal memory 112 and a processing unit 114. The internal memory 112 may be a cache, for example, a line buffer, but this is not limited by the embodiment of the present invention. In some embodiments, the processor 110 can be implemented by circuitry. In some embodiments, the processor 110 can be a general purpose processor. In some embodiments, the processor 110 can be a chip, such as an FPGA or ASIC or the like. External storage The memory 120 may be a memory, for example, a DDR SDRAM, but the embodiment of the present invention is not limited thereto.

The external memory 120 can store all of the pixels of the image. The internal memory 112 can store a portion of the pixels of the image. In some embodiments, the processor 110 reads a portion of the pixels of the image from the external memory 120 into the internal memory 112 for processing when processing the image.

In some designs, the mobile device, which may also be referred to as a mobile device, may adopt the technical solution of the embodiment of the present invention. The mobile device may be a drone, an unmanned boat, an autonomous vehicle or a robot, etc., but the embodiment of the present invention is not limited thereto.

2 is a schematic architectural diagram of a removable device 200 in accordance with an embodiment of the present invention.

As shown in FIG. 2, the mobile device 200 can include a power system 210, a control system 220, a sensing system 230, and a processing system 240.

Power system 210 is used to power the mobile device 200.

Taking the drone as an example, the power system of the drone may include an electronic governor (referred to as an electric current), a propeller, and a motor corresponding to the propeller. The motor is connected between the electronic governor and the propeller, and the motor and the propeller are disposed on the corresponding arm; the electronic governor is used for receiving the driving signal generated by the control system, and providing driving current to the motor according to the driving signal to control the motor Rotating speed. The motor is used to drive the propeller to rotate to power the drone's flight.

The sensing system 230 can be used to measure attitude information of the mobile device 200, that is, position information and state information of the mobile device 200 in space, for example, three-dimensional position, three-dimensional angle, three-dimensional velocity, three-dimensional acceleration, three-dimensional angular velocity, and the like. The sensing system 230 may include, for example, at least one of a gyroscope, an electronic compass, an Inertial Measurement Unit (IMU), a vision sensor, a Global Positioning System (GPS), a barometer, an airspeed meter, and the like. Kind.

In an embodiment of the invention, sensing system 230 can also be used to acquire images, i.e., sensing system 230 includes sensors for acquiring images, such as cameras and the like.

Control system 220 is used to control the movement of mobile device 200. The control system 220 can control the mobile device 200 in accordance with program instructions that are set in advance. For example, the control system 220 can control the movement of the mobile device 200 based on the attitude information of the mobile device 200 measured by the sensing system 230. Control system 220 can also control mobile device 200 based on control signals from the remote control. For example, for a drone, the control system 220 can be a flight control system (flying control) or a control circuit in a flight control.

Processing system 240 can process the images acquired by sensing system 230. For example, processing system 240 can be an Image Signal Processing (ISP) type of chip.

Processing system 240 may be system 100 of FIG. 1, or processing system 240 may include system 100 of FIG.

It should be understood that the above-mentioned division and naming of the components of the mobile device 200 are merely exemplary and should not be construed as limiting the embodiments of the present invention.

It should also be understood that the removable device 200 may also include other components not shown in FIG. 2, which are not limited by the embodiments of the present invention.

FIG. 3 shows a schematic flow diagram of a method 300 of processing an image in accordance with one embodiment of the present invention. The method 300 can be performed by the system 100 or processor 110 shown in FIG. 1; or by the removable device 200 shown in FIG. 2. In particular, when executed by the removable device 200, it can be performed by the processing system 240 of FIG.

310. Acquire a first data block corresponding to the first feature point of the first image.

The feature point is an object of image processing. When processing an image, for a feature point, multiple pixels associated with the feature point may be needed, for example, multiple pixels near the feature point. In the embodiment of the present invention, the data block including the plurality of pixels is a data block corresponding to the feature point.

Optionally, in an embodiment of the present invention, the first data block includes a minimum rectangular pixel area in which an image processing algorithm selects pixels of the first image for the first feature point.

The smallest rectangular pixel area is also referred to as a window that selects pixels. The first data block may be the smallest rectangular pixel area, or may be larger than the minimum rectangular pixel area, that is, the first data block includes all pixels required by the image processing algorithm to process the first feature point.

Optionally, in an embodiment of the present invention, the plurality of rows of pixels of the first image may be read, and the first data block corresponding to the first feature point in the plurality of rows of pixels is obtained.

Specifically, in the process of reading an image, multiple rows of pixels of the image are read into the cache at a time, and in this process, corresponding data blocks in the plurality of rows of pixels are acquired for subsequent operations; and then the image is read. One or more rows of pixels overwrite the data that first entered the cache, get the corresponding block of the newly read multi-line pixels, and so on, until the entire image is processed.

In the embodiment of the present invention, the first image may be an original image; or may be an intermediate image in an image processing process, for example, the first image may be a layer in an image pyramid, in particular, may be in a pyramid image. The bottom layer image.

For example, for the KLT algorithm, it is generally necessary to construct an image of the image during processing. pyramid. The first image may be a layer in the image pyramid. In this case, since the image data needs to be read when the image pyramid is generated, the read image data may simultaneously store the data block corresponding to the feature point according to the above method. Obtaining the data block while obtaining the pyramid image. Further, the feature points between each layer of the pyramid have a specific relationship, and the feature points of other layers of the pyramid image can be obtained according to the feature points of one layer, in the acquisition part Or in the process of the entire pyramid, the data blocks corresponding to the feature points of different layers of the pyramid can be notified. And there is no need to add extra read time, this method is not only for the KLT algorithm, but also for all image processing.

The first data block is stored in the first storage space in the memory, and the corresponding relationship between the first feature point and the first storage space is recorded.

In the embodiment of the present invention, the data blocks corresponding to the feature points are continuously stored in the memory, that is, one data block is stored in a continuous storage space, and the corresponding relationship between the feature points and the storage space is recorded. In this way, when the data block corresponding to a certain feature point needs to be read, the data block can be continuously read from the storage space according to the correspondence relationship, and the reading efficiency can be improved compared to the random access reading, thereby improving the data block. Processing efficiency.

Optionally, in an embodiment of the present invention, the correspondence may be a correspondence between the first feature point and a starting address of the first storage space.

Thus, when processing the first feature point, the first data block can be continuously read from the start address.

It should be understood that the corresponding relationship may also be a corresponding relationship of other forms, which is not limited by the embodiment of the present invention.

Optionally, in an embodiment of the present invention, the data blocks corresponding to the multiple feature points of the first image may be sequentially stored into the memory.

Specifically, for a plurality of feature points of the first image, in a sequence, for example, a raster scan order, and a raster is used for performing image processing on the feature point, a window is required for framed partial pixels for calculation, that is, First, the horizontal and vertical sequences are sequentially stored with corresponding data blocks, wherein each data block is stored in a continuous storage space in the memory. It should be understood that the foregoing sequence may also be other sequences, which are not limited by the embodiments of the present invention.

Alternatively, the data blocks of the plurality of feature points may occupy a contiguous storage space in the memory, that is, the plurality of data blocks may be stored adjacent to each other, but the embodiment of the present invention is not limited thereto.

Optionally, in an embodiment of the present invention, when the first feature point needs to be processed, The first data block is read from the first storage space according to the correspondence.

Since the data blocks of the feature points are continuously stored in the memory, when the image is processed by the algorithm, when the data block of a certain feature point needs to be read according to the processing order, the storage of the data block can be determined according to the correspondence relationship of the records. An address from which the data block is continuously read according to the storage address.

Optionally, in an embodiment of the present invention, determining, according to a feature point processing sequence of the second image, that the first feature point needs to be processed; and reading the first data from the first storage space according to the correspondence relationship Piece.

Specifically, the feature point processing order, that is, the order in which the feature points are selected, may be the feature point processing order of the first image, or may be the feature point processing order of other images.

For example, for the KLT algorithm, in the feature point matching processing of the I image and the J image, the first image and the second image respectively represent corresponding layers in the pyramid of the I image and the J image. In the processing of the layer, the feature points may be selected in accordance with the feature point processing order of the first image, or the feature points may be selected in the feature point processing order of the second image. If the feature point is selected according to the feature point processing order of the second image, for example, the selected feature point corresponds to the first feature point in the first image, when the first data block in the first image needs to be read, The storage address of the first data block may be determined according to the corresponding relationship of the record, and the first data block is continuously read according to the storage address.

Optionally, for the second image, the foregoing solution may also be used to store the data block of the feature point. Optionally, the feature point processing order of the second image may be the same as the storage order, such that if the feature points are selected according to the feature point processing order of the second image, each data block of the second image may be read in a storage order. For the data block in the first image, it can be read as described above.

It should be understood that the second image may be stored in the memory in other manners, which is not limited by the embodiment of the present invention. Regardless of the manner in which the second image is stored, the processing sequence may be determined according to the second image. When it is determined that the feature points of the first image need to be processed according to the processing order, the corresponding feature points of the first image are read in the foregoing manner. data block.

It should also be understood that the first image and the second image are also not limited to the KLT algorithm. For other algorithms, the processing sequence may also be determined according to the second image. When it is determined that the feature points of the first image need to be processed according to the processing order, the data blocks of the corresponding feature points of the first image are read in the foregoing manner.

Optionally, the feature point data blocks may not be stored for the feature points in the second image, but are sequentially sorted, for example, from left to right and top to bottom, so that when the second image enters the cache, The feature points in all the second images may be traversed in the process of reading from top to bottom, and then the feature points in all the second images traversed are according to the storage address of the corresponding first image feature points. The data block corresponding to the first feature point is read for calculation.

Optionally, the second image may be continuously stored in the storage space according to the foregoing manner, and may be read in the storage order after the reading.

Optionally, the storage of the data block corresponding to the first image feature point may also be used for other calculations, such as orb, brief calculation, and the like.

The technical solution of the embodiment of the present invention can reduce the number of memory accesses and improve the reading efficiency and processing efficiency by continuously storing the data blocks corresponding to the feature points in the memory. In addition, the bandwidth of the memory occupied by other devices can be reduced. Impact, thereby improving the performance of the entire system.

The technical solution of the embodiment of the present invention is described in detail below by taking the KLT algorithm as an example. It is to be understood that the scope of the present invention is not limited by the scope of the embodiments of the invention.

The pseudo code for tracking feature points in the KLT algorithm is as follows:

For a point u in Image I, find corresponding location v in Image J (for point u of image I, find the corresponding position v in image J)

Build pyramids for Images I and J (building pyramids for images I and J):

And

Initialization of pyramidal guess:

For L=L _m down to 0with step of -1 (L starts from L _m in order -1 until 0)

Identify pixel location of point uin image I ^L (determine the pixel position of point u in image I ^L ):

u ^L =[p _x p _y ] ^T =u/2 ^L

^{Calculate derivative of I L with respect to} x ( I ^L calculates the derivative of x):

^{Calculate derivative of I L with respect to} y ( I ^L calculates the derivative of y):

Calculate spatial gradient matrix G:

Initialization of iterative L-K optical flow computation:

For k=1to K with step of 1(or until‖η ^k ‖<accuracy threshold) (k is incremented by 1 from 1 to K, or until ‖η ^k ‖<precision threshold)

Calculate image difference:

Calculate image mismatch vector:

Calculate optical flow using L-K optical flow computation (calculation of optical flow using L-K optical flow calculation):

Obtain the pixel displacement guess for next iteration: Obtain pixel displacement guess for the next iteration:

End of for-loop on k (for the end of the for loop of k)

Identify optical flow at Layer L: Determine the optical flow on layer L:

Identify pyramidal guess for next Layer L-1 (determine pyramid guess for the next layer L-1):

End of for-loop on L (for the end of the L for loop)

Identify final optical flow vector:

d=g ⁰ +d ⁰

Identify location of point on J: Determine the location of the point on J:

v=u+d

Solution:The corresponding point is at location v on image J (answer: position v of the corresponding point on image J)

As can be seen from the above, the key implementation steps of the KLT algorithm are two for loops, one for loop to implement G calculation, and another for loop to implement d ^L calculation, thereby performing the next layer of KLT algorithm operation. Calculate G and δI _k ,

Both need to read the data block. When the number of images increases, that is, the image size becomes larger, the reading of the memory is a bottleneck of system performance.

In the embodiment of the present invention, in the process of constructing the pyramid I or during the reading of the I data, the required data blocks are sequentially stored into the memory, wherein each data block is stored into a continuous storage in the memory. In space. Alternatively, a similar process is performed for the image J.

For example, during the creation of the pyramid data, the original data needs to be read in order, so a line buffer can be used. For example, the line buffer can cache 20 lines of data according to the position of the known feature points. The corresponding data block, for example, the pixel of the pixel area of 19×19 is read from the line buffer, and then written into the memory in order, and the starting address of the data block corresponding to the feature point is written in the memory. This scheme can be used for the storage of data blocks of each layer of the pyramid. Alternatively, if a layer of a pyramid is small in size, it can be stored in the cache without being stored in memory.

When performing the KLT algorithm calculation, for each layer, the feature points may be sequentially processed in the order of the feature points in the layer of the image J. In this way, the data blocks in the layer of the image J can be read in the storage order, and the data blocks in the layer of the corresponding image I can determine the storage address according to the correspondence relationship of the records, and realize the reading of the data block according to the storage address. . The data blocks in the layer of the image J can also be read in the reading order of the image J, that is, read from left to right and from top to bottom. For example, there are multiple feature points on the image J. The first feature point from top to bottom from left to right is the feature point 1J, and the feature point 1I in the corresponding image I, when it is required to calculate the G of the feature point 1I in the layer of the image I, according to the feature point corresponding to the feature point The starting address in memory, you can continuously read out the corresponding data block, but then perform subsequent operations (such as the second for loop).

In this way, the data blocks in the respective layers of the image I and the image J can be read faster, thereby improving the processing efficiency of the algorithm.

The method of processing an image of an embodiment of the present invention has been described in detail above, and an integrated circuit, a processor, a computer system, and a mobile device of an embodiment of the present invention will be described below. It should be understood that the integrated circuit, the processor, the computer system, and the mobile device of the embodiments of the present invention may perform the foregoing various methods of the embodiments of the present invention, that is, the specific working processes of the following various products, and may refer to the foregoing method embodiments. Corresponding process.

FIG. 4 shows a schematic block diagram of an integrated circuit 400 in accordance with an embodiment of the present invention. As shown in FIG. 4, the integrated circuit 400 can include a processing unit 410 and a cache 420.

The processing unit 410 is configured to acquire, from the cache 420, a first data block corresponding to the first feature point of the first image;

Storing the first data block into a continuous first storage space in the memory, and recording the first A correspondence between the feature point and the first storage space.

Optionally, in an embodiment of the present invention, the processing unit 410 is configured to:

The plurality of rows of pixels of the first image are read into the buffer 420, and the first data block corresponding to the first feature point is obtained from the cache 420.

Optionally, in an embodiment of the present invention, the correspondence is a correspondence between the first feature point and a start address of the first storage space.

The data blocks corresponding to the plurality of feature points of the first image are sequentially stored in the memory 420.

Optionally, in one embodiment of the invention, the sequence is a raster scan order.

Optionally, in an embodiment of the present invention, the first image is a layer in an image pyramid;

The processing unit 410 is configured to:

The first data block is obtained when the image pyramid is generated.

Optionally, in an embodiment of the present invention, the processing unit 410 is further configured to:

When the first feature point needs to be processed, the first data block is read from the first storage space according to the correspondence.

Determining that the first feature point needs to be processed according to a feature point processing sequence of the second image;

The first data block is read from the first storage space according to the correspondence.

Optionally, in an embodiment of the invention, the integrated circuit 400 can be an FPGA or an ASIC.

It should be understood that in the integrated circuit of the various embodiments of the present invention described above, the control unit may be implemented by a circuit, which may be a unified circuit or a circuit composed of several circuits. The specific implementation form of the circuit is not limited in the embodiment of the present invention.

It should also be understood that the various units or modules in the embodiments of the present invention may be integrated in one chip or distributed in different chips.

The embodiment of the present invention further provides a processor, which may include the integrated circuit of the various embodiments of the present invention described above.

The embodiment of the invention further provides a computer system, which can include the processor of the embodiment of the invention described above, and a memory.

FIG. 5 shows a schematic block diagram of a computer system 500 in accordance with another embodiment of the present invention.

As shown in FIG. 5, the computer system 500 can include the integrated circuit 400 of the foregoing embodiments of the present invention; and a memory 510.

FIG. 6 shows a schematic block diagram of a computer system 600 in accordance with yet another embodiment of the present invention.

As shown in FIG. 6, the computer system 600 can include a processor 610 and a memory 620.

It should be understood that the computer system may also include components that are generally included in other computer systems, such as input and output devices, communication interfaces, and the like, which are not limited by the embodiments of the present invention.

Memory 620 is used to store computer executable instructions.

The memory 620 may be various types of memory, for example, may include a high speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory, which is implemented by the present invention. This example is not limited to this.

The processor 610 is configured to access the memory 620 and execute the computer executable instructions to perform the operations in the method of processing images of the various embodiments of the present invention described above.

The processor 610 may include a microprocessor, a field-programmable gate array (FPGA), a central processing unit (CPU), a graphics processing unit (GPU), etc., and is implemented by the present invention. This example is not limited to this.

Embodiments of the present invention also provide a removable device that can include an integrated circuit, processor, or computer system in accordance with various embodiments of the present invention described above.

An integrated circuit, a processor, a computer system, and a removable device according to embodiments of the present invention may correspond to an execution body of a method of processing an image of an embodiment of the present invention, and each module in an integrated circuit, a processor, a computer system, and a removable device The above and other operations and/or functions are respectively implemented in order to implement the corresponding processes of the foregoing various methods, and are not described herein for brevity.

The embodiment of the invention further provides a computer storage medium, wherein the computer storage medium stores program code, and the program code can be used to indicate a method for processing an image of the embodiment of the invention.

It should be understood that in the embodiment of the present invention, the term "and/or" is merely an association relationship describing an associated object, indicating that there may be three relationships. For example, A and/or B may indicate that A exists separately, and A and B exist simultaneously, and B cases exist alone. In addition, the character "/" in this article generally indicates that the contextual object is an "or" relationship.

Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both, for clarity of hardware and software. Interchangeability, the composition and steps of the various examples have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods for implementing the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the present invention.

A person skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the system, the device and the unit described above can refer to the corresponding process in the foregoing method embodiment, and details are not described herein again.

In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, or an electrical, mechanical or other form of connection.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the embodiments of the present invention.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention contributes in essence or to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium. Including a number of instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to perform all of the methods described in various embodiments of the present invention Or part of the steps. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like, which can store program codes. .

The above is only the specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any equivalent person can be easily conceived within the technical scope of the present invention by any person skilled in the art. Modifications or substitutions are intended to be included within the scope of the invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

A method for processing an image, comprising:

Obtaining a first data block corresponding to the first feature point of the first image;

And storing the first data block into a continuous first storage space in the memory, and recording a correspondence between the first feature point and the first storage space.
The method according to claim 1, wherein the acquiring the first data block corresponding to the first feature point of the first image comprises:

Reading a plurality of rows of pixels of the first image, and acquiring the first data block corresponding to the first feature point in the plurality of rows of pixels.
The method according to claim 1 or 2, wherein the correspondence relationship is a correspondence between the first feature point and a start address of the first storage space.
The method according to any one of claims 1 to 3, wherein the storing the first data block into a continuous first storage space in the memory comprises:

Data blocks corresponding to the plurality of feature points of the first image are sequentially stored into the memory.
The method of claim 4 wherein said order is a raster scan order.
The method according to any one of claims 1 to 5, wherein the first image is a layer in an image pyramid;

The acquiring the first data block corresponding to the first feature point of the first image includes:

The first data block is acquired when the image pyramid is generated.
The method according to any one of claims 1 to 6, wherein the first data block comprises a minimum rectangular pixel area in which an image processing algorithm selects pixels of the first image for the first feature point.
The method according to any one of claims 1 to 7, wherein the method further comprises:

When the first feature point needs to be processed, the first data block is read from the first storage space according to the corresponding relationship.
The method according to any one of claims 1 to 7, wherein the method further comprises:

Determining that the first feature point needs to be processed according to a feature point processing sequence of the second image;

Reading the first data block from the first storage space according to the correspondence.
An integrated circuit, comprising: a processing unit and a cache;

The processing unit is configured to acquire, from the cache, a first data block corresponding to a first feature point of the first image;

And storing the first data block into a continuous first storage space in the memory, and recording a correspondence between the first feature point and the first storage space.
The integrated circuit of claim 10 wherein said processing unit is operative to:

Reading a plurality of rows of pixels of the first image into the cache, and acquiring, from the cache, the first data block corresponding to the first feature point in the plurality of rows of pixels.
The integrated circuit according to claim 10 or 11, wherein the correspondence relationship is a correspondence between the first feature point and a start address of the first storage space.
The integrated circuit according to any one of claims 10 to 12, wherein the processing unit is configured to:

Data blocks corresponding to the plurality of feature points of the first image are sequentially stored into the memory.
The integrated circuit of claim 13 wherein said sequence is a raster scan order.
The integrated circuit according to any one of claims 10 to 14, wherein the first image is a layer in an image pyramid;

The processing unit is used to:

The first data block is acquired when the image pyramid is generated.
The integrated circuit according to any one of claims 10 to 15, wherein the first data block comprises a minimum rectangular pixel area in which an image processing algorithm selects pixels of the first image for the first feature point.
The integrated circuit according to any one of claims 10 to 16, wherein the processing unit is further configured to:

When the first feature point needs to be processed, the first data block is read from the first storage space according to the corresponding relationship.
The integrated circuit according to any one of claims 10 to 16, wherein the processing unit is further configured to:

Determining that the first feature point needs to be processed according to a feature point processing sequence of the second image;

Reading the first data block from the first storage space according to the correspondence.
A processor, comprising an integrated circuit according to any one of claims 10 to 18.
A computer system, comprising:

An integrated circuit according to any one of claims 10 to 18 or a processor according to claim 19;

RAM.
A computer system, comprising:

a memory for storing computer executable instructions;

A processor for accessing the memory and executing the computer executable instructions to perform the operations of the method of any one of claims 1 to 9.
A mobile device, comprising:

An integrated circuit according to any one of claims 10 to 18; or

The processor of claim 19; or

A computer system according to claim 20 or 21.