WO2019019013A1 - 处理图像的方法、芯片、处理器、系统和可移动设备 - Google Patents

处理图像的方法、芯片、处理器、系统和可移动设备 Download PDF

Info

Publication number
WO2019019013A1
WO2019019013A1 PCT/CN2017/094289 CN2017094289W WO2019019013A1 WO 2019019013 A1 WO2019019013 A1 WO 2019019013A1 CN 2017094289 W CN2017094289 W CN 2017094289W WO 2019019013 A1 WO2019019013 A1 WO 2019019013A1
Authority
WO
WIPO (PCT)
Prior art keywords
pixels
rectangular pixel
algorithm
rows
pixel area
Prior art date
Application number
PCT/CN2017/094289
Other languages
English (en)
French (fr)
Inventor
解进
高明明
杨康
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to CN201780004676.XA priority Critical patent/CN108513670A/zh
Priority to PCT/CN2017/094289 priority patent/WO2019019013A1/zh
Publication of WO2019019013A1 publication Critical patent/WO2019019013A1/zh
Priority to US16/728,832 priority patent/US20200134771A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/0007—Image acquisition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/60—Memory management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/28—Indexing scheme for image data processing or generation, in general involving image processing hardware

Definitions

  • the present invention relates to the field of information technology and, more particularly, to a method, a chip, a processor, a computer system, and a removable device for processing an image.
  • the main power consumption and area are all random access memory ( Generated by Random Access Memory, RAM). If the area of the RAM is small, the power consumption and area of the corresponding system will be small.
  • RAM Random Access Memory
  • Embodiments of the present invention provide a method, a chip, a processor, a computer system, and a mobile device for processing an image, which can improve processing efficiency.
  • a method for processing an image comprising: acquiring a plurality of rows of pixels of an image according to a first rectangular pixel region; storing the plurality of rows of pixels in a plurality of row buffers, wherein the plurality of rows Each row of pixels in the row of pixels is stored in one of the storage spaces of the plurality of row buffers, the number of bits of the block in one row buffer being less than the number of columns of the image.
  • a second aspect provides a method for processing an image, including: reading image pixels in a corresponding row buffer of a plurality of row buffers according to a second rectangular pixel region, and selecting a corresponding pixel to output to a module of the second algorithm,
  • the second rectangular pixel area is a minimum rectangular pixel area of the second algorithm selected pixel; the read pixel is registered in a register, and the pixel registered in the register is repeatedly registered according to the first rectangular pixel area. Splicing and selecting a corresponding pixel output to the module of the first algorithm, wherein the first rectangular pixel region is a minimum rectangular pixel region of the first algorithm selected pixel, and the first rectangular pixel region covers the second A rectangular pixel area.
  • a third aspect provides a chip, including: a control unit and a plurality of line buffers; wherein the control unit is configured to acquire a plurality of rows of pixels of an image according to the first rectangular pixel region; and to display the plurality of rows of pixels Storing in blocks into the plurality of row buffers, wherein each row of pixels of the plurality of rows of pixels is stored in one of the plurality of row buffer storage spaces, the bits of the block in a row buffer The number is smaller than the number of columns of the image.
  • a fourth aspect provides a chip, including: a control unit, a plurality of row buffers, a register, a module of the first algorithm, and a module of the second algorithm; wherein the control unit is configured to use the second rectangular pixel region according to the second rectangular pixel region, Reading image pixels in a corresponding row buffer of the plurality of row buffers, and selecting corresponding pixels to output to a module of the second algorithm, wherein the second rectangular pixel region is a pixel selected by the second algorithm a minimum rectangular pixel area; and registering the read pixel into the register, splicing the plurality of registered pixels in the register according to the first rectangular pixel area, and selecting a corresponding pixel to output to the first algorithm And a module, wherein the first rectangular pixel area is a minimum rectangular pixel area of the first algorithm selected pixel, and the first rectangular pixel area covers the second rectangular pixel area.
  • a processor comprising the chip of the third or fourth aspect described above.
  • a computer system comprising: a memory for storing computer executable instructions; a processor for accessing the memory, and executing the computer executable instructions to perform the first or second The operation in the aspect of the method.
  • the seventh aspect provides a mobile device, comprising: the chip of the third or fourth aspect; or the processor of the above fifth aspect; or the computer system of the sixth aspect.
  • a computer storage medium having stored therein program code, the program code being operative to indicate a method of performing the first or second aspect described above.
  • multiple rows of pixels are stored in blocks according to a rectangular pixel area.
  • the number of bits in each line buffer of each block is smaller than the number of columns in the image, so that the number of columns of pixels read from the line buffer is smaller than the number of columns of the image, which can improve the reading speed, thereby improving processing efficiency.
  • FIG. 1 is an architectural diagram of a technical solution to which an embodiment of the present invention is applied.
  • FIG. 2 is a processing architecture diagram of a technical solution according to an embodiment of the present invention.
  • FIG. 3 is a schematic structural diagram of a mobile device according to an embodiment of the present invention.
  • FIG. 4 is a schematic flow chart of a method of processing an image according to an embodiment of the present invention.
  • FIG. 5 is a schematic diagram of a storage design of a line buffer according to an embodiment of the present invention.
  • FIG. 6 is a schematic flowchart of a method of processing an image according to another embodiment of the present invention.
  • Figure 7 is a schematic block diagram of a chip in accordance with one embodiment of the present invention.
  • Figure 8 is a schematic block diagram of a chip in accordance with another embodiment of the present invention.
  • Figure 9 is a schematic block diagram of a chip in accordance with still another embodiment of the present invention.
  • Figure 10 is a schematic block diagram of a computer system in accordance with an embodiment of the present invention.
  • the size of the sequence numbers of the processes does not imply a sequence of executions, and the order of execution of the processes should be determined by its function and internal logic, and should not be construed as an embodiment of the present invention.
  • the implementation process constitutes any limitation.
  • the technical solution of the embodiment of the present invention can be applied to various algorithms, such as the harris algorithm or the census algorithm, but the embodiment of the present invention is not limited thereto.
  • the technical solution of the embodiment of the present invention stores multiple rows of pixels of an image into a line buffer in blocks, which can improve the speed of reading pixels from the line buffer, thereby improving processing efficiency.
  • the technical solution of the embodiment of the present invention can enable multiple algorithms to share the line buffer, so that it is no longer necessary to provide a separate cache for each algorithm, so as to reduce the consumption of the storage resources.
  • the rectangular pixel area may be the smallest rectangular pixel area in which the algorithm selects pixels, that is, a window in which pixels may be selected by the algorithm. It should be understood that if the window of the algorithm selection pixel is not a rectangular area, the rectangular pixel area is the smallest rectangular pixel area covering the window.
  • the number of rows and the number of columns of the rectangular pixel area may be the same or different.
  • the rectangular pixel area may be a 3x3 pixel area; for the census algorithm, the rectangular pixel area may be a 13x13 pixel area.
  • FIG. 1 is an architectural diagram of a technical solution to which an embodiment of the present invention is applied.
  • system 100 can receive data to be processed 102, process data to be processed 102, generate processed data 108, and output processed data 108.
  • components in system 100 may be implemented by one or more processors, which may be processors in a computing device or processors in a mobile device (eg, a drone).
  • the processor may be any type of processor, which is not limited in this embodiment of the present invention.
  • the processor may be a chip comprised of a cache and processing circuitry (which may also be referred to as a processing unit or control unit).
  • one or more memories may also be included in system 100.
  • the memory can be used to store instructions and data, such as computer executable instructions to implement the technical solution of the embodiments of the present invention, data to be processed 102, processed data 108, and the like.
  • the memory can include a cache or memory.
  • the memory may be any kind of memory, which is not limited in this embodiment of the present invention.
  • the data to be processed 102 may include data of an image, or other similar multimedia data.
  • the data to be processed 102 may include sensory data from sensors, which may be visual sensors (eg, cameras, infrared sensors), near field sensors (eg, ultrasonic sensors, radar), position sensors, and the like.
  • the pending data 102 may include information from a user, such as biometric information, which may include facial features, fingerprint scans, retinal scans, DNA sampling, and the like.
  • biometric information which may include facial features, fingerprint scans, retinal scans, DNA sampling, and the like.
  • the data of the image is taken as an example, but the present invention is not limited thereto.
  • FIG. 2 is a diagram showing a processing architecture of a technical solution of an embodiment of the present invention.
  • a plurality of rows of pixels of an image are input into a line buffer, and the row buffer is used to store data (pixels) of the maximum number of rows to be stored in at least one algorithm calculation process.
  • the first algorithm in FIG. 2 may be a census algorithm
  • the second algorithm may be a harris algorithm, but the embodiment of the present invention is not limited thereto.
  • the harris algorithm needs to store 3 rows of data
  • the census algorithm needs to store 12 rows of data
  • the row cache is the storage space of the image's 12 rows of data.
  • multiple algorithms share the line buffer, so that it is no longer necessary to provide a separate line buffer for each algorithm, so that the purpose of reducing the consumption of the storage resources can be achieved.
  • a mobile device which may also be referred to as a mobile device, may process data using the technical solution of the embodiments of the present invention.
  • the mobile device may be a drone, an unmanned boat, an autonomous vehicle or a robot, etc., but the embodiment of the present invention is not limited thereto.
  • FIG. 3 is a schematic architectural diagram of a removable device 300 according to an embodiment of the present invention.
  • the mobile device 300 can include a power system 310, a control system 320, a sensing system 330, and a processing system 340.
  • Power system 310 is used to power the mobile device 300.
  • the power system of the drone may include an electronic governor (referred to as an electric current), a propeller, and a motor corresponding to the propeller.
  • the motor is connected between the electronic governor and the propeller, and the motor and the propeller are disposed on the corresponding arm; the electronic governor is used for receiving the driving signal generated by the control system, and providing driving current to the motor according to the driving signal to control the motor Rotating speed.
  • the motor is used to drive the propeller to rotate to power the drone's flight.
  • the sensing system 330 can be used to measure attitude information of the mobile device 300, that is, position information and state information of the mobile device 300 in space, such as three-dimensional position, three-dimensional angle, three-dimensional velocity, three-dimensional acceleration, and three-dimensional angular velocity.
  • the sensing system 330 may include, for example, at least one of a gyroscope, an electronic compass, an Inertial Measurement Unit (IMU), a vision sensor, a Global Positioning System (GPS), a barometer, an airspeed meter, and the like.
  • IMU Inertial Measurement Unit
  • GPS Global Positioning System
  • barometer an airspeed meter
  • sensing system 330 can also be used to acquire images, i.e., sensing system 330 includes sensors for acquiring images, such as cameras and the like.
  • Control system 320 is used to control the movement of mobile device 300.
  • the control system 320 can control the mobile device 300 in accordance with program instructions that are set in advance.
  • control system 320 can control the movement of mobile device 300 based on the attitude information of mobile device 300 as measured by sensing system 330.
  • Control system 320 can also control mobile device 300 based on control signals from the remote control.
  • the control system 320 can be a flight control system (flying control) or a control circuit in a flight control.
  • Processing system 340 can process the images acquired by sensing system 330.
  • processing system 340 can be an Image Signal Processing (ISP) type of chip.
  • ISP Image Signal Processing
  • Processing system 340 can be system 100 in FIG. 1, or processing system 340 can include The system 100 of Figure 1 is included.
  • removable device 300 may also include other components not shown in FIG. 3, which are not limited by the embodiments of the present invention.
  • FIG. 4 shows a schematic flow diagram of a method 400 of processing an image in accordance with one embodiment of the present invention.
  • the method 400 can be performed by the system 100 shown in FIG. 1; or by the removable device 300 shown in FIG. In particular, when executed by the removable device 300, it can be performed by the processing system 340 of FIG.
  • image pixels for subsequent algorithm processing are acquired according to the first rectangular pixel region.
  • the pixel can be a pixel output by a pre-stage algorithm module or a pixel in memory.
  • the number of rows of the acquired pixels may be not less than a difference between the number of rows of the first rectangular pixel region and a preset value.
  • the preset value can be 1 or other values.
  • the number of rows of the acquired pixels may be equal to a difference between the number of rows of the first rectangular pixel region and a preset value.
  • the number of rows of the plurality of rows of pixels is equal to one less than the number of rows of the first rectangular pixel region. For example, if the first rectangular pixel area is a 13x13 pixel area, the number of lines of the plurality of lines of pixels may be 12.
  • the plurality of rows of pixels are then stored in a line buffer for algorithm processing.
  • multiple rows of pixels are stored in blocks in a plurality of row buffers.
  • multiple rows of pixels are no longer stored in rows, but are stored in blocks in a plurality of row buffers, where each row of pixels is stored in one of the plurality of row buffers. That is, each of the plurality of rows of pixels acquiring the image occupies one block in the storage space of the plurality of line buffers instead of occupying one line buffer.
  • the number of bits per block in a row buffer is less than the number of columns in the image.
  • the number of columns of pixels read from the line buffer is smaller than the number of columns of the image, so that the reading speed can be increased, and the processing efficiency can be improved.
  • a requirement of a plurality of rectangular pixel regions may be considered.
  • the plurality of rows of pixels may be stored in blocks in a plurality of line buffers according to the first rectangular pixel area and the second rectangular pixel area.
  • the plurality of rows of pixels are stored in a plurality of row buffers in blocks according to the second rectangular pixel region and the first rectangular pixel region. That is, two rectangular pixel regions are considered when storing.
  • the number of rows of the second rectangular pixel area is smaller than the number of rows of the first rectangular pixel area. That is to say, in the embodiment of the present invention, pixels are acquired according to a rectangular pixel area having a large number of rows, and pixels are stored in a line buffer according to a rectangular pixel area having a large number of rows and a rectangular pixel area having a small number of rows.
  • the number of blocks on one column of the plurality of row buffers is equal to the number of rows of the second rectangular pixel region. That is, the number of blocks in the vertical direction of the storage space of the plurality of line buffers is equal to the number of lines of the second rectangular pixel area. For example, if the second rectangular pixel area is a 3x3 pixel area, the number of blocks in the vertical direction may be three. It should be understood that the above is only a preferred embodiment, and the relationship between the number of blocks in the vertical direction of the storage space and the number of rows of the second rectangular pixel area is not limited.
  • the number of bits (ie, the horizontal width of the block) of one block in one of the plurality of line buffers is not less than the number of columns of the second rectangular pixel area.
  • the horizontal width of one block is not less than 3.
  • the product of the number of blocks in one column of the plurality of row buffers (i.e., in the vertical direction of the storage space of the plurality of row buffers) and the number of blocks on a row buffer (i.e., in the lateral direction of the storage space of the plurality of row buffers) Not less than the difference between the number of rows of the first rectangular pixel area and the preset value. That is, the product of the number of blocks in the vertical direction of the storage space and the number of blocks in the horizontal direction is not smaller than the value associated with the number of rows of the first rectangular pixel area.
  • the value associated with the number of rows of the first rectangular pixel region is a difference between the number of rows of the first rectangular pixel region and a preset value, and the preset value may be 1 or other values.
  • the first rectangular pixel area is a pixel area of 13x13
  • the product of the number of blocks in the vertical direction of the storage space and the number of blocks on the horizontal direction is not less than 12.
  • the product of the number of blocks on one column of the plurality of row buffers and the number of blocks on one row buffer is equal to the number of rows of the first rectangular pixel region minus one. That is, the product of the number of blocks in the vertical direction of the storage space and the number of blocks in the horizontal direction is equal to the number of rows of the first rectangular pixel area minus one. For example, if the first rectangular pixel area is a pixel area of 13x13, the product of the number of blocks in the vertical direction of the storage space and the number of blocks on the horizontal direction is equal to 12.
  • the plurality of rows of pixels are from top to bottom, from left to right
  • the right is sequentially stored in a plurality of blocks in the storage space of the plurality of line buffers. That is to say, the blocks corresponding to the pixels of each row are arranged in order from top to bottom and from left to right.
  • the storage order of the blocks may also be changed, for example, may be changed to be arranged in order from left to right and from top to bottom, which is not limited by the embodiment of the present invention.
  • Figure 5 is a diagram showing the memory design of a line buffer in accordance with one embodiment of the present invention.
  • the first rectangular pixel area is a pixel area of 13x13
  • the second rectangular pixel area is a pixel area of 3x3.
  • 12 lines of pixels of the image can be read.
  • each row of pixels is stored in one of the plurality of row buffer storage spaces.
  • rows 0 to 11 respectively represent 12 rows of pixels of an image
  • each row of pixels is stored in one block
  • blocks corresponding to 12 rows of pixels are arranged in order from top to bottom and left to right.
  • the number of bits in a block in a row buffer is 4, which is greater than the number of columns in the second rectangular pixel area.
  • the line buffer can have a bit width of 16, and the number of blocks on a line buffer (horizontal) is 4.
  • the number of blocks (in the vertical direction) on one column of the plurality of line buffers is 3, which is equal to the number of lines of the second rectangular pixel area, and the product of the number of blocks in the vertical direction and the number of blocks in the horizontal direction is 12.
  • FIG. 5 is only an example, and other ways of storing in a block may be used.
  • the horizontal and vertical widths of each block may also be changed, which is not limited by the embodiment of the present invention.
  • the first rectangular pixel area is a minimum rectangular pixel area of a plurality of algorithms selected by a first algorithm; and the second rectangular pixel area is selected by a second algorithm of the multiple algorithms.
  • the first rectangular pixel area may cover the second rectangular pixel area.
  • the first rectangular pixel region and the second rectangular pixel region may be the largest and smallest of the smallest rectangular pixel regions of the plurality of algorithm selection pixels.
  • the largest and the smallest can be considered, which allows multiple algorithms to share the row buffer.
  • the present invention is not limited to the number of rectangular pixel regions considered in the memory design.
  • one rectangular pixel region for example, the above-mentioned largest one
  • a plurality of rectangular pixel regions may also be considered (for example, the above maximum And the smallest).
  • the pixels in the corresponding row buffers of the plurality of row buffers may be read according to the second rectangular pixel region, and the corresponding pixels are selected to be output to the module of the second algorithm.
  • the rectangular algorithm corresponding to the second algorithm is the smallest, and the required pixels are the least, so that the corresponding pixels can be acquired at one time and output to the module of the second algorithm.
  • the second algorithm is the harris algorithm
  • the rectangular pixel area is a 3x3 pixel area. It takes 3 rows and 3 columns of pixels.
  • Figure 5 suppose that it is the first point. It only needs to read the address 0, address 80, and address 160 of the storage space, and then select the data of row 3, row 1, and row 2 of 3 rows and 3 columns. Using register storage, select output 3x3 and send it to the harris algorithm for processing.
  • the read pixel may be registered in a register, and the pixels registered in the register are spliced according to the first rectangular pixel area, and the corresponding pixel is selected and output to the first algorithm. Module.
  • the rectangular area corresponding to the first algorithm is the largest, so the pixels required by the first algorithm can be obtained by splicing the pixels registered in the register, and then output to the module of the first algorithm.
  • the required pixels can be obtained by splicing the pixels registered in the register multiple times.
  • the first algorithm is taken as an example, but is not limited thereto.
  • the rectangular pixel area is a pixel area of 5x5 or 7x7, and each row requires 5 rows, 5 columns, or 7 rows and 7 columns of pixels.
  • the data that has been read out in this part is registered using a register.
  • the second algorithm (harris algorithm) reads address 1, address 81, address 161
  • the data stored in the previous register is compared with the data that has been read. After splicing and selection, the pixels required by the census algorithm can be obtained, and the census algorithm can be processed.
  • the address 3 the address 83, and the address 163 need to be read before the calculation can be performed.
  • 12 rows of pixels can be cached by themselves (including: address 0-2, address 80-82, address 160-162), the 13th row of pixels is derived from the input data, and the subsequent operations are analogous, that is, the larger processing
  • the algorithm needs to process the data in the stored data and the read data including the data required by the algorithm.
  • the plurality of rows of pixels may be acquired by a first input first output (FIFO) manner.
  • FIFO first input first output
  • multiple rows of pixels are stored in a row buffer according to a rectangular pixel area, and the number of bits in each row buffer of one block is smaller than the number of columns of the image, so that pixels are read from the line buffer.
  • the number of columns is smaller than the number of columns of the image, and the reading speed can be increased, so that the processing efficiency can be improved.
  • Pixels are stored in block buffers in blocks, which allows multiple algorithms to share row buffers. This eliminates the need to provide separate row buffers for each algorithm, thereby reducing storage resource consumption.
  • the embodiment of the present invention is not limited thereto. That is to say, the storage design scheme in the foregoing embodiment can be implemented separately, and does not depend on whether it is shared or not. In addition, the technical solution of sharing the line buffer in the embodiment of the present invention can also be implemented separately, and does not depend on the foregoing embodiment. Storage design. Based on this, the embodiment of the present invention further provides another method for processing an image, which is described below in conjunction with FIG. 6. It should be understood that some specific descriptions of the method shown in FIG. 6 may refer to the foregoing embodiments, and are not described herein again for brevity.
  • FIG. 6 shows a schematic flow diagram of a method 600 of processing an image in accordance with another embodiment of the present invention. As shown in FIG. 6, the method 600 includes:
  • a plurality of algorithms can share a line buffer, wherein, for an algorithm with a small rectangular pixel area, a pixel required for processing can be read once from the line buffer, and a larger algorithm for a rectangular pixel area is used.
  • a pixel required for processing can be read once from the line buffer, and a larger algorithm for a rectangular pixel area is used.
  • the pixels registered in the register are spliced together to obtain the pixels required for one processing.
  • FIG. 7 shows a schematic block diagram of a chip 700 in accordance with an embodiment of the present invention.
  • the chip 700 can include a control unit 710 and a plurality of row buffers 720.
  • the control unit 710 is configured to acquire a plurality of rows of pixels of an image according to the first rectangular pixel region;
  • the plurality of rows of pixels are stored in blocks into the plurality of row buffers 720, wherein each row of the plurality of rows of pixels is stored in a block in a storage space of the plurality of row buffers 720, the block being in a row buffer
  • the number of bits is less than the number of columns in the image.
  • the number of rows of the plurality of rows of pixels is not less than a difference between the number of rows of the first rectangular pixel region and a preset value.
  • the number of rows of the plurality of rows of pixels is equal to the number of rows of the first rectangular pixel region minus one.
  • the number of the blocks in a column of the plurality of row buffers is equal to the number of rows of the second rectangular pixel region, wherein the number of rows of the second rectangular pixel region is smaller than the first The number of rows in the rectangular pixel area.
  • the number of bits of the block in one of the plurality of row buffers is not less than the number of columns of the second rectangular pixel region.
  • the first rectangular pixel area is a minimum rectangular pixel area of a plurality of algorithms selected by a first algorithm; and the second rectangular pixel area is selected by a second algorithm of the multiple algorithms. a minimum rectangular pixel area of the pixel, wherein the first rectangular pixel area encompasses the second rectangular pixel area.
  • the chip 700 further includes a module 730 of the second algorithm.
  • the control unit 710 is further configured to read pixels in the corresponding row buffers of the plurality of row buffers 720 according to the second rectangular pixel region, and select corresponding pixels to output to the module 730 of the second algorithm.
  • the chip 700 further includes a register 740 and a module 750 of the first algorithm.
  • the control unit 710 is further configured to register the read pixel into the register 740, splicing the pixels registered in the register 740 according to the first rectangular pixel area, and selecting corresponding pixels to output to the first Module 750 of the algorithm.
  • the first algorithm is a census algorithm
  • the second algorithm is a harris algorithm
  • control unit 710 is configured to: display the multi-line image
  • the blocks are sequentially stored from top to bottom and left to right in a plurality of blocks in the storage space of the plurality of line buffers 720.
  • the chip 700 further includes:
  • the first in first out FIFO unit 760 wherein the control unit 710 is configured to acquire pixels of the image through the FIFO unit 760.
  • the chip 710 can be an FPGA or an ASIC.
  • FIG. 9 shows a schematic block diagram of a chip 900 in accordance with another embodiment of the present invention.
  • the chip 900 can include a control unit 910, a plurality of row buffers 920, registers 930, a module 940 of the first algorithm, and a module 950 of the second algorithm.
  • the control unit 910 is configured to read image pixels in the corresponding row buffers of the plurality of row buffers 920 according to the second rectangular pixel region, and select corresponding pixels to output to the module 950 of the second algorithm, where the a second rectangular pixel region is a minimum rectangular pixel region in which the second algorithm selects a pixel;
  • control unit may be implemented by a circuit, which may be a unified circuit or a circuit composed of several circuits.
  • the specific implementation form of the circuit is not limited in the embodiment of the present invention.
  • the embodiment of the present invention further provides a processor, which may include the chip of the various embodiments of the present invention described above.
  • FIG. 10 shows a schematic block diagram of a computer system 1000 in accordance with an embodiment of the present invention.
  • the computer system 1000 can include a processor 1010 and a memory 1020.
  • the computer system 1000 may also include components that are generally included in other computer systems, such as input and output devices, communication interfaces, and the like, which are not limited by the embodiments of the present invention.
  • Memory 1020 is for storing computer executable instructions.
  • the memory 1020 may be various kinds of memories, for example, may include a high speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory, which is implemented by the present invention. This example is not limited to this.
  • RAM high speed random access memory
  • non-volatile memory such as at least one disk memory
  • the processor 1010 is configured to access the memory 1020 and execute the computer executable instructions to perform the operations in the method of processing images of the various embodiments of the present invention described above.
  • the processor 1010 may include a microprocessor, a Field-Programmable Gate Array (FPGA), a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), etc., and is implemented by the present invention. This example is not limited to this.
  • Embodiments of the present invention also provide a removable device, which may include the chip, processor or computer system of the various embodiments of the present invention described above.
  • the chip, the processor, the computer system, and the removable device of the embodiments of the present invention may correspond to an execution body of the method of processing an image of the embodiment of the present invention, and the above-described modules of the chip, the processor, the computer system, and the movable device And other operations and/or functions, respectively, in order to implement the corresponding processes of the foregoing various methods, for brevity, no further details are provided herein.
  • the embodiment of the invention further provides a computer storage medium, wherein the computer storage medium stores program code, and the program code can be used to indicate a method for transmitting the encoded data according to the embodiment of the invention.
  • the term "and/or” is merely an association relationship describing an associated object, indicating that there may be three relationships.
  • a and/or B may indicate that A exists separately, and A and B exist simultaneously, and B cases exist alone.
  • the character "/" in this article generally indicates that the contextual object is an "or" relationship.
  • the disclosed systems, devices, and methods may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the unit is only a logical function division.
  • there may be another division manner for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, or an electrical, mechanical or other form of connection.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the embodiments of the present invention.
  • each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.
  • the integrated unit if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium.
  • the technical solution of the present invention contributes in essence or to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium.
  • a number of instructions are included to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention.
  • the foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like, which can store program codes. .

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Image Input (AREA)

Abstract

公开了一种处理图像的方法、芯片、处理器、计算机系统和可移动设备。该方法包括：根据第一矩形像素区域，获取图像的多行像素；将该多行像素按块存储到多个行缓存中，其中，该多行像素中每行像素存储到该多个行缓存的存储空间中的一个块中，该块在一个行缓存中的位数小于该图像的列数。本发明实施例的技术方案，能够提高处理效率。

Description

处理图像的方法、芯片、处理器、系统和可移动设备
版权申明
本专利文件披露的内容包含受版权保护的材料。该版权为版权所有人所有。版权所有人不反对任何人复制专利与商标局的官方记录和档案中所存在的该专利文件或者该专利披露。
技术领域
本发明涉及信息技术领域，并且更具体地，涉及一种处理图像的方法、芯片、处理器、计算机系统和可移动设备。
背景技术
目前在现场可编程门阵列(Field－Programmable Gate Array，FPGA)，即或专用集成电路(Application Specific Integrated Circuit，ASIC)等芯片的设计实现上，主要功耗以及面积都是由随机存取存储器(Random Access Memory，RAM)产生的。如果RAM的面积小，那么对应系统的功耗以及面积都会小。
现有技术中，通常将多种算法，例如harris算法以及census算法分别实现，即，给每种算法提供单独的缓存。在采用算法处理图像时，需要从相应的缓存中读取图像的多行像素，并选择相应的像素。由于像素从缓存中是整行读取的，因此会影响读取速度，尤其是当图片的大小越大时，对处理效率的影响会更大。
因此，如何提高处理效率成为芯片设计中一个亟待解决的技术问题。
发明内容
本发明实施例提供了一种处理图像的方法、芯片、处理器、计算机系统和可移动设备，能够提高处理效率。
第一方面，提供了一种处理图像的方法，包括：根据第一矩形像素区域，获取图像的多行像素；将所述多行像素按块存储到多个行缓存中，其中，所述多行像素中每行像素存储到所述多个行缓存的存储空间中的一个块中，所述块在一个行缓存中的位数小于所述图像的列数。
第二方面，提供了一种处理图像的方法，包括：根据第二矩形像素区域，读取多个行缓存中相应行缓存中的图像像素，并选择相应的像素输出到第二算法的模块，其中，所述第二矩形像素区域为所述第二算法选取像素的最小矩形像素区域；将读取的像素寄存到寄存器中，根据第一矩形像素区域，将所述寄存器中多次寄存的像素拼接，并选择相应的像素输出到第一算法的模块，其中，所述第一矩形像素区域为所述第一算法选取像素的最小矩形像素区域，所述第一矩形像素区域涵盖所述第二矩形像素区域。
第三方面，提供了一种芯片，包括：控制单元和多个行缓存；其中，所述控制单元，用于根据第一矩形像素区域，获取图像的多行像素；以及将所述多行像素按块存储到所述多个行缓存中，其中，所述多行像素中每行像素存储到所述多个行缓存的存储空间中的一个块中，所述块在一个行缓存中的位数小于所述图像的列数。
第四方面，提供了一种芯片，包括：控制单元、多个行缓存、寄存器、第一算法的模块和第二算法的模块；其中，所述控制单元，用于根据第二矩形像素区域，读取所述多个行缓存中相应行缓存中的图像像素，并选择相应的像素输出到所述第二算法的模块，其中，所述第二矩形像素区域为所述第二算法选取像素的最小矩形像素区域；以及将读取的像素寄存到所述寄存器中，根据第一矩形像素区域，将所述寄存器中多次寄存的像素拼接，并选择相应的像素输出到所述第一算法的模块，其中，所述第一矩形像素区域为所述第一算法选取像素的最小矩形像素区域，所述第一矩形像素区域涵盖所述第二矩形像素区域。
第五方面，提供了一种处理器，其特征在于，包括上述第三或第四方面的芯片。
第六方面，提供了一种计算机系统，包括：存储器，用于存储计算机可执行指令；处理器，用于访问所述存储器，并执行所述计算机可执行指令，以进行上述第一或第二方面的方法中的操作。
第七方面，提供了一种可移动设备，包括：上述第三或第四方面的芯片；或者，上述第五方面的处理器；或者，上述第六方面的计算机系统。
第八方面，提供了一种计算机存储介质，该计算机存储介质中存储有程序代码，该程序代码可以用于指示执行上述第一或第二方面的方法。
本发明实施例的技术方案，根据矩形像素区域将多行像素按块存储到 行缓存中，每个块在一个行缓存中的位数小于图像的列数，这样，从行缓存中读取像素的列数小于图像的列数，可以提高读取速度，从而能够提高处理效率。
附图说明
图1是应用本发明实施例的技术方案的架构图。
图2是本发明实施例的技术方案的处理架构图。
图3是本发明实施例的可移动设备的示意性架构图。
图4是本发明一个实施例的处理图像的方法的示意性流程图。
图5是本发明实施例的行缓存的存储设计的示意图。
图6是本发明另一个实施例的处理图像的方法的示意性流程图。
图7是本发明一个实施例的芯片的示意性框图。
图8是本发明另一个实施例的芯片的示意性框图。
图9是本发明又一个实施例的芯片的示意性框图。
图10是本发明实施例的计算机系统的示意性框图。
具体实施方式
下面将结合附图，对本发明实施例中的技术方案进行描述。
应理解，本文中的具体的例子只是为了帮助本领域技术人员更好地理解本发明实施例，而非限制本发明实施例的范围。
还应理解，在本发明的各种实施例中，各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本发明实施例的实施过程构成任何限定。
还应理解，本说明书中描述的各种实施方式，既可以单独实施，也可以组合实施，本发明实施例对此并不限定。
本发明实施例的技术方案可以应用于各种算法，例如harris算法或者census算法等，但本发明实施例对此并不限定。
本发明实施例的技术方案将图像的多行像素按块存储到行缓存中，可以提高从行缓存中读取像素的速度，从而能够提高处理效率。
进一步地，本发明实施例的技术方案可以使多种算法共用行缓存，从而不再需要给每种算法提供单独的缓存，达到降低存储资源的消耗的目的。
在本发明各种实施例中，矩形像素区域可以为算法选取像素的最小矩形像素区域，即可以为算法选取像素的窗口。应理解，若算法选取像素的窗口不为矩形区域，则矩形像素区域为涵盖该窗口的最小矩形像素区域。矩形像素区域的行数和列数可以相同也可以不同。例如，对于harris算法，矩形像素区域可以为3x3的像素区域；对于census算法，矩形像素区域可以为13x13的像素区域。
图1是应用本发明实施例的技术方案的架构图。
如图1所示，系统100可以接收待处理数据102，对待处理数据102进行处理，产生处理后数据108，并输出处理后数据108。在一些实施例中，系统100中的部件可以由一个或多个处理器实现，该处理器可以是计算设备中的处理器，也可以是可移动设备(例如无人机)中的处理器。该处理器可以为任意种类的处理器，本发明实施例对此不做限定。在一些实施例中，该处理器可以为由缓存和处理电路(也可以称为处理单元或控制单元)组成的芯片。在一些实施例中，系统100中还可以包括一个或多个存储器。该存储器可用于存储指令和数据，例如，实现本发明实施例的技术方案的计算机可执行指令，待处理数据102、处理后数据108等。例如，该存储器可以包括缓存或内存。该存储器可以为任意种类的存储器，本发明实施例对此也不做限定。
待处理数据102可以包括图像的数据，或者其他类似的多媒体数据。在一些情况下，待处理数据102可以包括来自传感器的传感数据，该传感器可以为视觉传感器(例如，相机、红外传感器)，近场传感器(例如，超声波传感器、雷达)，位置传感器等。在一些情况下，待处理数据102可以包括来自用户的信息，例如，生物信息，该生物信息可以包括面部特征，指纹扫描，视网膜扫描，DNA采样等。在本发明各种实施例中，以图像的数据为例进行描述，但本发明并不限于此。
图2示出了本发明一个实施例的技术方案的处理架构图。如图2所示，图像的多行像素输入到行缓存(line buffer)中，行缓存用于存储至少一种算法计算过程中，所需要存储的最大行数的数据(像素)。例如，图2中第一算法可以为census算法，第二算法可以为harris算法，但本发明实施例对此并不限定。harris算法计算需要存储3行数据，census算法计算需要存储12行的数据，那么行缓存就为图像的12行数据的存储空间。也就是说，在本 发明实施例中，多种算法共用行缓存，这样，不再需要给每种算法提供单独的行缓存，从而可以达到降低存储资源的消耗的目的。
在一些设计中，可移动设备，也可以称为移动设备，可以采用本发明实施例的技术方案处理数据。该可移动设备可以是无人机、无人驾驶船、自动驾驶车辆或机器人等，但本发明实施例对此并不限定。
图3是本发明实施例的可移动设备300的示意性架构图。
如图3所示，可移动设备300可以包括动力系统310、控制系统320、传感系统330和处理系统340。
动力系统310用于为该可移动设备300提供动力。
以无人机为例，无人机的动力系统可以包括电子调速器(简称为电调)、螺旋桨以及与螺旋桨相对应的电机。电机连接在电子调速器与螺旋桨之间，电机和螺旋桨设置在对应的机臂上；电子调速器用于接收控制系统产生的驱动信号，并根据驱动信号提供驱动电流给电机，以控制电机的转速。电机用于驱动螺旋桨旋转，从而为无人机的飞行提供动力。
传感系统330可以用于测量可移动设备300的姿态信息，即可移动设备300在空间的位置信息和状态信息，例如，三维位置、三维角度、三维速度、三维加速度和三维角速度等。传感系统330例如可以包括陀螺仪、电子罗盘、惯性测量单元(Inertial Measurement Unit，IMU)、视觉传感器、全球定位系统(Global Positioning System，GPS)、气压计、空速计等传感器中的至少一种。
在本发明实施例中，传感系统330还可用于采集图像，即传感系统330包括用于采集图像的传感器，例如相机等。
控制系统320用于控制可移动设备300的移动。控制系统320可以按照预先设置的程序指令对可移动设备300进行控制。例如，控制系统320可以根据传感系统330测量的可移动设备300的姿态信息控制可移动设备300的移动。控制系统320也可以根据来自遥控器的控制信号对可移动设备300进行控制。例如，对于无人机，控制系统320可以为飞行控制系统(飞控)，或者为飞控中的控制电路。
处理系统340可以处理传感系统330采集的图像。例如，处理系统340可以为图像信号处理(Image Signal Processing,ISP)类芯片。
处理系统340可以为图1中的系统100，或者，处理系统340可以包 括图1中的系统100。
应理解，上述对于可移动设备300的各组成部件的划分和命名仅仅是示例性的，并不应理解为对本发明实施例的限制。
还应理解，可移动设备300还可以包括图3中未示出的其他部件，本发明实施例对此并不限定。
图4示出了本发明一个实施例的处理图像的方法400的示意性流程图。该方法400可以由图1所示的系统100执行；或者由图3所示的可移动设备300执行。具体地，在由可移动设备300执行时，可以由图3中的处理系统340执行。
410，根据第一矩形像素区域，获取图像的多行像素。
在本发明实施例中，根据第一矩形像素区域，获取后续用于算法处理的图像像素。该像素可以是前级算法模块输出的像素或者是内存中的像素。
可选地，获取的像素的行数可以不小于该第一矩形像素区域的行数与预设值的差值。该预设值可以为1，也可以为其他数值。可选地，获取的像素的行数可以等于该第一矩形像素区域的行数与预设值的差值。可选地，该多行像素的行数等于该第一矩形像素区域的行数减一。例如，若该第一矩形像素区域为13x13像素区域，则该多行像素的行数可以为12。该多行像素接下来被存储到行缓存中，以用于算法处理。
420，将该多行像素按块存储到多个行缓存中，该多行像素中每行像素存储到该多个行缓存的存储空间中的一个块中，该块在一个行缓存中的位数小于该图像的列数。
在本发明实施例中，将多行像素按块(block)存储到多个行缓存中。换句话说，多行像素不再按行存储，而是按块存储到多个行缓存的存储空间中，其中，每行像素存储到该多个行缓存的存储空间中的一个块中。即，获取图像的多行像素中的每行像素占据多个行缓存的存储空间中的一个块，而不是占据一个行缓存。每个块在一个行缓存中的位数小于图像的列数。这样，从行缓存中读取像素的列数小于图像的列数，因此可以提高读取速度，从而能够提高处理效率。
在本发明实施例中，可选地，在将多行像素按块存储到行缓存中时，可以考虑多个矩形像素区域的要求。例如，可以根据第一矩形像素区域和第二矩形像素区域，将该多行像素按块存储到多个行缓存中。
具体而言，在根据第一矩形像素区域，获取多行像素后，根据第二矩形像素区域和第一矩形像素区域，将该多行像素按块存储到多个行缓存中。即，在存储时，考虑两个矩形像素区域。可选地，该第二矩形像素区域的行数小于该第一矩形像素区域的行数。也就是说，在本发明实施例中，根据行数较大的矩形像素区域获取像素，再同时根据行数较大的矩形像素区域和行数较小的矩形像素区域将像素存储到行缓存。
可选地，在本发明一个实施例中，在该多个行缓存的一列上块的数量等于该第二矩形像素区域的行数。也就是说，该多个行缓存的存储空间的纵向上块的数量等于该第二矩形像素区域的行数。例如，若该第二矩形像素区域为3x3的像素区域，则纵向上块的数量可以为3。应理解，以上只是优选的实施方案，本发明对存储空间的纵向上块的数量与第二矩形像素区域的行数的关系并不限定。
可选地，在本发明一个实施例中，一个块在多个行缓存中的一个行缓存中的位数(即块的横向宽度)不小于该第二矩形像素区域的列数。例如，若该第二矩形像素区域为3x3的像素区域，则一个块的横向宽度不小于3。应理解，以上只是优选的实施方案，本发明对块的横向宽度与第二矩形像素区域的列数的关系并不限定。
在该多个行缓存的一列上(即该多个行缓存的存储空间的纵向上)块的数量与一个行缓存上(即该多个行缓存的存储空间的横向上)块的数量的乘积不小于该第一矩形像素区域的行数与预设值的差值。也就是说，存储空间的纵向上块的数量与横向上块的数量的乘积不小于与该第一矩形像素区域的行数关联的值。与该第一矩形像素区域的行数关联的值为该第一矩形像素区域的行数与预设值的差值，该预设值可以为1，也可以为其他数值。例如，若该第一矩形像素区域为13x13的像素区域，则存储空间的纵向上块的数量与横向上块的数量的乘积不小于12。
可选地，在该多个行缓存的一列上块的数量与一个行缓存上块的数量的乘积等于该第一矩形像素区域的行数减一。也就是说，存储空间的纵向上块的数量与横向上块的数量的乘积等于该第一矩形像素区域的行数减一。例如，若该第一矩形像素区域为13x13的像素区域，则存储空间的纵向上块的数量与横向上块的数量的乘积等于12。
可选地，在本发明一个实施例中，该多行像素按块从上到下，从左到 右依次存储到该多个行缓存的存储空间中的多个块中。也就是说，各行像素对应的块按照从上到下，从左到右的顺序排列。
应理解，块的存储顺序也可以变换，例如，可以变换为按照从左到右，从上到下的顺序排列，本发明实施例对此并不限定。
图5示出了本发明一个实施例的行缓存的存储设计的示意图。
对应于图5，第一矩形像素区域为13x13的像素区域，第二矩形像素区域为3x3的像素区域。根据第一矩形像素区域，可以读取图像的12行像素。该12行像素中，每行像素存储到多个行缓存的存储空间中的一个块中。如图5所示，行0至行11分别表示图像的12行像素，每行像素存储到一个块中，12行像素对应的块按照从上到下，从左到右的顺序排列。一个块在一个行缓存中的位数为4，大于第二矩形像素区域的列数。行缓存的位宽可以为16，一个行缓存上(横向上)块的数量为4。在该多个行缓存的一列上(纵向上)块的数量为3，等于第二矩形像素区域的行数，纵向上块的数量与横向上块的数量的乘积为12。
应理解，图5仅是一种示例，还可以采用其他按块存储的方式，另外，每个块的横向和纵向宽度也可以变换，本发明实施例对此并不限定。
可选地，在本发明一个实施例中，该第一矩形像素区域为多种算法中第一算法选取像素的最小矩形像素区域；该第二矩形像素区域为该多种算法中第二算法选取像素的最小矩形像素区域。该第一矩形像素区域可以涵盖该第二矩形像素区域。
可选地，上述第一矩形像素区域和第二矩形像素区域可以为多种算法选取像素的最小矩形像素区域中的最大者和最小者。在存储设计中，可以考虑该最大者和该最小者，这样可以使得多种算法共用行缓存。
应理解，本发明对存储设计中考虑的矩形像素区域的个数并不限定，换句话说，可以考虑一个矩形像素区域(例如上述最大者)，也可以考虑多个矩形像素区域(例如上述最大者和最小者)。
可选地，对于第二算法，可以根据该第二矩形像素区域，读取该多个行缓存中相应行缓存中的像素，并选择相应的像素输出到该第二算法的模块。
第二算法对应的矩形像素区域最小，所需要的像素最少，因此可以一次性地获取相应像素，输出到第二算法的模块。
假设第二算法为harris算法，矩形像素区域为3x3的像素区域，即需 要3行3列的像素。以图5为例，假设是第一个点，那只需要读取存储空间的地址0，地址80，地址160，就可以从中选择出行0，行1，行2的3行3列的数据。利用寄存器存储，选择输出3x3，送给harris算法进行处理。
可选地，对于第一算法，可以将读取的像素寄存到寄存器中，根据该第一矩形像素区域，将该寄存器中多次寄存的像素拼接，并选择相应的像素输出到该第一算法的模块。
第一算法对应的矩形像素区域最大，因此可以通过将寄存器中多次寄存的像素拼接，得到第一算法需要的像素，再输出到第一算法的模块。
应理解，对于矩形像素区域介于第二算法和第一算法中间的其他算法，都可以通过将寄存器中多次寄存的像素拼接，得到所需要的像素。本发明实施例中仅以第一算法为例进行说明，但并不限于此。
假设第一算法为census算法，矩形像素区域为5x5或7x7的像素区域，每次处理需要5行5列或者7行7列的像素。以图5为例，先读取地址0，地址80，地址160，可以看出这个时候读取出12行的数据，每行4个像素，但这对于cenus算法(5x5或7x7)还是不够的，这部分已经读出的数据使用寄存器寄存，当第二算法(harris算法)读取到地址1、地址81，地址161的时候，将之前寄存器存储的数据，与目前已经读取的数据，进行拼接、选择，就可以得到census算法所需要的像素，从而进行census算法的处理，如果矩形像素区域为13x13的像素区域，则还需要读取到地址3、地址83、地址163时才能够进行计算，其中12行像素可以取自行缓存(包括：地址0-2、地址80-82、地址160-162)，第13行像素来源于输入数据，后续操作以此类推，即处理的较大的算法，需要在存储的数据和读取的数据包括了算法需要的数据，就可以开始算法的处理过程。
可选地，在本发明一个实施例中，可以通过先进先出(First Input First Output，FIFO)方式获取该多行像素。利用FIFO，在接受到后级的反压时，不再读取数据，从而控制数据读取的时序。
本发明实施例的技术方案，根据矩形像素区域将多行像素按块存储到行缓存中，每个块在一个行缓存中的位数小于图像的列数，这样，从行缓存中读取像素的列数小于图像的列数，可以提高读取速度，从而能够提高处理效率。
进一步地，本发明实施例的技术方案，根据多种矩形像素区域将多行 像素按块存储到行缓存中，可以使得多种算法共用行缓存，这样，不再需要给每种算法提供单独的行缓存，从而能够降低存储资源的消耗。
以上描述了本发明实施例的行缓存的存储设计的技术方案，但本发明实施例并不限于此。也就是说，前述实施例中的存储设计方案可以单独实施，并不依赖于是否共用；另外，本发明实施例中共用行缓存的技术方案也可以单独实施，并不依赖于前述实施例中的存储设计方案。基于此，本发明实施例又提供了另一种处理图像的方法，下面结合图6进行描述。应理解，图6所示方法中的一些具体的描述可以参考前述实施例，以下为了简洁，不再赘述。
图6示出了本发明另一个实施例的处理图像的方法600的示意性流程图。如图6所示，该方法600包括：
610，根据第二矩形像素区域，读取多个行缓存中相应行缓存中的图像像素，并选择相应的像素输出到第二算法的模块，其中，该第二矩形像素区域为该第二算法选取像素的最小矩形像素区域；
620，将读取的像素寄存到寄存器中，根据第一矩形像素区域，将该寄存器中多次寄存的像素拼接，并选择相应的像素输出到第一算法的模块，其中，该第一矩形像素区域为该第一算法选取像素的最小矩形像素区域，该第一矩形像素区域涵盖该第二矩形像素区域。
在本发明实施例中，多种算法可以共用行缓存，其中，对于矩形像素区域较小的算法，可以从行缓存中一次性读取一次处理所需要的像素，对于矩形像素区域较大的算法，可以通过将读取的像素寄存到寄存器中，将寄存器中多次寄存的像素拼接，得到一次处理所需要的像素。通过上述方案，可以实现多种算法共用行缓存，这样，不再需要给每种算法提供单独的行缓存，从而能够降低存储资源的消耗。
上文详细描述了本发明实施例的处理图像的方法，下面将描述本发明实施例的芯片、处理器、计算机系统和可移动设备。应理解，本发明实施例的芯片、处理器、计算机系统和可移动设备可以执行前述本发明实施例的各种方法，即以下各种产品的具体工作过程，可以参考前述方法实施例中的对应过程。
图7示出了本发明实施例的芯片700的示意性框图。如图7所示，该芯片700可以包括：控制单元710和多个行缓存720。
该控制单元710，用于根据第一矩形像素区域，获取图像的多行像素；以及
将该多行像素按块存储到该多个行缓存720中，其中该多行像素中每行像素存储到该多个行缓存720的存储空间中的一个块中，该块在一个行缓存中的位数小于该图像的列数。
可选地，在本发明一个实施例中，该多行像素的行数不小于该第一矩形像素区域的行数与预设值的差值。
可选地，在本发明一个实施例中，该多行像素的行数等于该第一矩形像素区域的行数减一。
可选地，在本发明一个实施例中，在该多个行缓存的一列上该块的数量等于第二矩形像素区域的行数，其中，该第二矩形像素区域的行数小于该第一矩形像素区域的行数。
可选地，在本发明一个实施例中，该块在该多个行缓存中的一个行缓存中的位数不小于该第二矩形像素区域的列数。
可选地，在本发明一个实施例中，该第一矩形像素区域为多种算法中第一算法选取像素的最小矩形像素区域；该第二矩形像素区域为该多种算法中第二算法选取像素的最小矩形像素区域，其中，该第一矩形像素区域涵盖该第二矩形像素区域。
可选地，在本发明一个实施例中，如图8所示，该芯片700还包括该第二算法的模块730。
该控制单元710还用于，根据该第二矩形像素区域，读取该多个行缓存720中相应行缓存中的像素，并选择相应的像素输出到该第二算法的模块730。
可选地，在本发明一个实施例中，如图8所示，该芯片700还包括寄存器740和该第一算法的模块750。
该控制单元710还用于，将读取的像素寄存到该寄存器740中，根据该第一矩形像素区域，将该寄存器740中多次寄存的像素拼接，并选择相应的像素输出到该第一算法的模块750。
可选地，在本发明一个实施例中，该第一算法为census算法，该第二算法为harris算法。
可选地，在本发明一个实施例中，该控制单元710用于，将该多行像 素按块从上到下，从左到右依次存储到该多个行缓存720的存储空间中的多个块中。
可选地，在本发明一个实施例中，如图8所示，该芯片700还包括：
先进先出FIFO单元760，其中，该控制单元710用于通过该FIFO单元760获取该图像的像素。
可选地，在本发明一个实施例中，该芯片710可以为FPGA或ASIC。
图9示出了本发明另一实施例的芯片900的示意性框图。如图9所示，该芯片900可以包括：控制单元910、多个行缓存920、寄存器930、第一算法的模块940和第二算法的模块950。
该控制单元910，用于根据第二矩形像素区域，读取该多个行缓存920中相应行缓存中的图像像素，并选择相应的像素输出到该第二算法的模块950，其中，该第二矩形像素区域为该第二算法选取像素的最小矩形像素区域；以及
将读取的像素寄存到该寄存器930中，根据第一矩形像素区域，将该寄存器930中多次寄存的像素拼接，并选择相应的像素输出到该第一算法的模块940，其中，该第一矩形像素区域为该第一算法选取像素的最小矩形像素区域，该第一矩形像素区域涵盖该第二矩形像素区域。
应理解，在上述本发明各种实施例的芯片中，控制单元可以由电路实现，该电路可以是统一的一个电路，也可以是由几个电路组成的电路。本发明实施例对电路的具体实现形式不做限定。
还应理解，本发明实施例中的各种单元或模块可以集成在一个芯片中，也可以分布在不同的芯片中。
本发明实施例还提供了一种处理器，该处理器可以包括上述本发明各种实施例的芯片。
图10示出了本发明实施例的计算机系统1000的示意性框图。
如图10所示，该计算机系统1000可以包括处理器1010和存储器1020。
应理解，该计算机系统1000还可以包括其他计算机系统中通常所包括的部件，例如，输入输出设备、通信接口等，本发明实施例对此并不限定。
存储器1020用于存储计算机可执行指令。
存储器1020可以是各种种类的存储器，例如可以包括高速随机存取存储器(Random Access Memory，RAM)，还可以包括非不稳定的存储器(non-volatile memory)，例如至少一个磁盘存储器，本发明实施例对此并不限定。
处理器1010用于访问该存储器1020，并执行该计算机可执行指令，以进行上述本发明各种实施例的处理图像的方法中的操作。
处理器1010可以包括微处理器，现场可编程门阵列(Field－Programmable Gate Array，FPGA)，中央处理器(Central Processing unit，CPU)，图形处理器(Graphics Processing Unit，GPU)等，本发明实施例对此并不限定。
本发明实施例还提供了一种可移动设备，该可移动设备可以包括上述本发明各种实施例的芯片、处理器或者计算机系统。
本发明实施例的芯片、处理器、计算机系统和可移动设备可对应于本发明实施例的处理图像的方法的执行主体，并且芯片、处理器、计算机系统和可移动设备中的各个模块的上述和其它操作和/或功能分别为了实现前述各个方法的相应流程，为了简洁，在此不再赘述。
本发明实施例还提供了一种计算机存储介质，该计算机存储介质中存储有程序代码，该程序代码可以用于指示执行上述本发明实施例的传输编码数据的方法。
应理解，在本发明实施例中，术语“和/或”仅仅是一种描述关联对象的关联关系，表示可以存在三种关系。例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中字符“/”，一般表示前后关联对象是一种“或”的关系。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、计算机软件或者二者的结合来实现，为了清楚地说明硬件和软件的可互换性，在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本发明的范围。
所属领域的技术人员可以清楚地了解到，为了描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另外，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口、装置或单元的间接耦合或通信连接，也可以是电的，机械的或其它的形式连接。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本发明实施例方案的目的。
另外，在本发明各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以是两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分，或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本发明各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本发明的具体实施方式，但本发明的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，可轻易想到各种等效的修改或替换，这些修改或替换都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应以权利要求的保护范围为准。

Claims (27)

  1. 一种处理图像的方法，其特征在于，包括：
    根据第一矩形像素区域，获取图像的多行像素；
    将所述多行像素按块存储到多个行缓存中，其中，所述多行像素中每行像素存储到所述多个行缓存的存储空间中的一个块中，所述块在一个行缓存中的位数小于所述图像的列数。
  2. 根据权利要求1所述的方法，其特征在于，所述多行像素的行数不小于所述第一矩形像素区域的行数与预设值的差值。
  3. 根据权利要求1或2所述的方法，其特征在于，所述多行像素的行数等于所述第一矩形像素区域的行数减一。
  4. 根据权利要求1至3任一项所述的方法，其特征在于，在所述多个行缓存的一列上所述块的数量等于第二矩形像素区域的行数，其中，所述第二矩形像素区域的行数小于所述第一矩形像素区域的行数。
  5. 根据权利要求4所述的方法，其特征在于，所述块在所述多个行缓存中的一个行缓存中的位数不小于所述第二矩形像素区域的列数。
  6. 根据权利要求4或5所述的方法，其特征在于，所述第一矩形像素区域为多种算法中第一算法选取像素的最小矩形像素区域；所述第二矩形像素区域为所述多种算法中第二算法选取像素的最小矩形像素区域，其中，所述第一矩形像素区域涵盖所述第二矩形像素区域。
  7. 根据权利要求6所述的方法，其特征在于，所述方法还包括：
    根据所述第二矩形像素区域，读取所述多个行缓存中相应行缓存中的像素，并选择相应的像素输出到所述第二算法的模块。
  8. 根据权利要求7所述的方法，其特征在于，所述方法还包括：
    将读取的像素寄存到寄存器中，根据所述第一矩形像素区域，将所述寄存器中多次寄存的像素拼接，并选择相应的像素输出到所述第一算法的模块。
  9. 根据权利要求6所述的方法，其特征在于，所述第一算法为census算法，所述第二算法为harris算法。
  10. 根据权利要求1所述的方法，其特征在于，所述多行像素按块从上到下，从左到右依次存储到所述多个行缓存的存储空间中的多个块中。
  11. 根据权利要求1所述的方法，其特征在于，所述获取图像的多行像素，包括：
    通过先进先出FIFO方式获取所述多行像素。
  12. 一种处理图像的方法，其特征在于，包括：
    根据第二矩形像素区域，读取多个行缓存中相应行缓存中的图像像素，并选择相应的像素输出到第二算法的模块，其中，所述第二矩形像素区域为所述第二算法选取像素的最小矩形像素区域；
    将读取的像素寄存到寄存器中，根据第一矩形像素区域，将所述寄存器中多次寄存的像素拼接，并选择相应的像素输出到第一算法的模块，其中，所述第一矩形像素区域为所述第一算法选取像素的最小矩形像素区域，所述第一矩形像素区域涵盖所述第二矩形像素区域。
  13. 一种芯片，其特征在于，包括：控制单元和多个行缓存；
    其中，所述控制单元，用于根据第一矩形像素区域，获取图像的多行像素；以及
    将所述多行像素按块存储到所述多个行缓存中，其中，所述多行像素中每行像素存储到所述多个行缓存的存储空间中的一个块中，所述块在一个行缓存中的位数小于所述图像的列数。
  14. 根据权利要求13所述的芯片，其特征在于，所述多行像素的行数不小于所述第一矩形像素区域的行数与预设值的差值。
  15. 根据权利要求13或14所述的芯片，其特征在于，所述多行像素的行数等于所述第一矩形像素区域的行数减一。
  16. 根据权利要求13至15任一项所述的芯片，其特征在于，在所述多个行缓存的一列上所述块的数量等于第二矩形像素区域的行数，其中，所述第二矩形像素区域的行数小于所述第一矩形像素区域的行数。
  17. 根据权利要求16所述的芯片，其特征在于，所述块在所述多个行缓存中的一个行缓存中的位数不小于所述第二矩形像素区域的列数。
  18. 根据权利要求16或17所述的芯片，其特征在于，所述第一矩形像素区域为多种算法中第一算法选取像素的最小矩形像素区域；所述第二矩形像素区域为所述多种算法中第二算法选取像素的最小矩形像素区域，其中，所述第一矩形像素区域涵盖所述第二矩形像素区域。
  19. 根据权利要求18所述的芯片，其特征在于，所述芯片还包括所述第二算法的模块；
    所述控制单元还用于，根据所述第二矩形像素区域，读取所述多个行缓 存中相应行缓存中的像素，并选择相应的像素输出到所述第二算法的模块。
  20. 根据权利要求19所述的芯片，其特征在于，所述芯片还包括寄存器和所述第一算法的模块；
    所述控制单元还用于，将读取的像素寄存到所述寄存器中，根据所述第一矩形像素区域，将所述寄存器中多次寄存的像素拼接，并选择相应的像素输出到所述第一算法的模块。
  21. 根据权利要求18所述的芯片，其特征在于，所述第一算法为census算法，所述第二算法为harris算法。
  22. 根据权利要求13所述的芯片，其特征在于，所述控制单元用于，将所述多行像素按块从上到下，从左到右依次存储到所述多个行缓存的存储空间中的多个块中。
  23. 根据权利要求13所述的芯片，其特征在于，所述芯片还包括：
    先进先出FIFO单元，其中，所述控制单元用于通过所述FIFO单元获取所述图像的像素。
  24. 一种芯片，其特征在于，包括：控制单元、多个行缓存、寄存器、第一算法的模块和第二算法的模块；
    其中，所述控制单元，用于根据第二矩形像素区域，读取所述多个行缓存中相应行缓存中的图像像素，并选择相应的像素输出到所述第二算法的模块，其中，所述第二矩形像素区域为所述第二算法选取像素的最小矩形像素区域；以及
    将读取的像素寄存到所述寄存器中，根据第一矩形像素区域，将所述寄存器中多次寄存的像素拼接，并选择相应的像素输出到所述第一算法的模块，其中，所述第一矩形像素区域为所述第一算法选取像素的最小矩形像素区域，所述第一矩形像素区域涵盖所述第二矩形像素区域。
  25. 一种处理器，其特征在于，包括根据权利要求13至24中任一项所述的芯片。
  26. 一种计算机系统，其特征在于，包括：
    存储器，用于存储计算机可执行指令；
    处理器，用于访问所述存储器，并执行所述计算机可执行指令，以进行根据权利要求1至12中任一项所述的方法中的操作。
  27. 一种可移动设备，其特征在于，包括：
    根据权利要求13至24中任一项所述的芯片；或者，
    根据权利要求25所述的处理器；或者，
    根据权利要求26所述的计算机系统。
PCT/CN2017/094289 2017-07-25 2017-07-25 处理图像的方法、芯片、处理器、系统和可移动设备 WO2019019013A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CN201780004676.XA CN108513670A (zh) 2017-07-25 2017-07-25 处理图像的方法、芯片、处理器、系统和可移动设备
PCT/CN2017/094289 WO2019019013A1 (zh) 2017-07-25 2017-07-25 处理图像的方法、芯片、处理器、系统和可移动设备
US16/728,832 US20200134771A1 (en) 2017-07-25 2019-12-27 Image processing method, chip, processor, system, and mobile device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/094289 WO2019019013A1 (zh) 2017-07-25 2017-07-25 处理图像的方法、芯片、处理器、系统和可移动设备

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/728,832 Continuation US20200134771A1 (en) 2017-07-25 2019-12-27 Image processing method, chip, processor, system, and mobile device

Publications (1)

Publication Number Publication Date
WO2019019013A1 true WO2019019013A1 (zh) 2019-01-31

Family

ID=63375168

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/094289 WO2019019013A1 (zh) 2017-07-25 2017-07-25 处理图像的方法、芯片、处理器、系统和可移动设备

Country Status (3)

Country Link
US (1) US20200134771A1 (zh)
CN (1) CN108513670A (zh)
WO (1) WO2019019013A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020107319A1 (zh) * 2018-11-29 2020-06-04 深圳市大疆创新科技有限公司 图像处理方法和设备，以及视频处理器
CN114168524B (zh) * 2021-12-07 2023-10-20 平头哥(上海)半导体技术有限公司 行缓存单元、加速单元、片上系统和行缓存配置方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH08191425A (ja) * 1995-01-09 1996-07-23 Mitsubishi Electric Corp メモリ制御装置
CN101022553A (zh) * 2007-03-28 2007-08-22 华为技术有限公司 图像数据的存取和解码方法及解码装置
CN101340580A (zh) * 2008-08-15 2009-01-07 上海龙晶微电子有限公司 视频硬件解码器的片外动态存储器的地址映射方法
US20100254618A1 (en) * 2009-04-01 2010-10-07 Yu-Min Chen Method for Accessing Image Data and Related Apparatus

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105809663A (zh) * 2014-12-31 2016-07-27 中国移动通信集团公司 一种获取图像局部区域的方法、装置和电子设备
CN105976304B (zh) * 2016-05-30 2019-05-10 北京奇艺世纪科技有限公司 一种图像水印嵌入、检测方法及装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH08191425A (ja) * 1995-01-09 1996-07-23 Mitsubishi Electric Corp メモリ制御装置
CN101022553A (zh) * 2007-03-28 2007-08-22 华为技术有限公司 图像数据的存取和解码方法及解码装置
CN101340580A (zh) * 2008-08-15 2009-01-07 上海龙晶微电子有限公司 视频硬件解码器的片外动态存储器的地址映射方法
US20100254618A1 (en) * 2009-04-01 2010-10-07 Yu-Min Chen Method for Accessing Image Data and Related Apparatus

Also Published As

Publication number Publication date
CN108513670A (zh) 2018-09-07
US20200134771A1 (en) 2020-04-30

Similar Documents

Publication Publication Date Title
US20210133093A1 (en) Data access method, processor, computer system, and mobile device
KR102499396B1 (ko) 뉴럴 네트워크 장치 및 뉴럴 네트워크 장치의 동작 방법
US20190347501A1 (en) Method of analyzing objects in images recored by a camera of a head mounted device
CN113874870A (zh) 基于图像的定位
WO2019104638A1 (zh) 神经网络处理的方法、装置、加速器、系统和可移动设备
US10776943B2 (en) System and method for 3D association of detected objects
US20190392556A1 (en) Method, chip, processor, computer system, and mobile device for image processing
US20170345197A1 (en) Display control method and display control device
US20160360185A1 (en) Three-dimensional imaging sensor calibration
US20220180472A1 (en) Application processor including reconfigurable scaler and devices including the processor
US20200134771A1 (en) Image processing method, chip, processor, system, and mobile device
EP4307219A1 (en) Three-dimensional target detection method and apparatus
CN115039015A (zh) 位姿跟踪方法、可穿戴设备、移动设备以及存储介质
EP3772035B1 (en) Adaptive sub-tiles for distortion correction in vision-based assistance systems and methods
JP2022520517A (ja) 人工現実システムにおいて平面を識別すること
CN115668374A (zh) 存储器中的运动传感器
WO2019041271A1 (zh) 处理图像的方法、集成电路、处理器、系统和可移动设备
WO2021102946A1 (zh) 计算装置、方法、处理器和可移动设备
WO2020155044A1 (zh) 卷积计算的装置、方法、处理器和可移动设备
EP3474224B1 (en) Graphics processing method and device
KR20200063017A (ko) 영상 처리를 위한 전자 장치 및 영상 처리 방법
CN113888552A (zh) 图像处理方法和设备
CN107111862B (zh) 低功率dma标记
CN115223110B (zh) 目标检测方法、装置、计算机设备和存储介质
WO2020073164A1 (zh) 数据存储的装置、方法、处理器和可移动设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17918852

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17918852

Country of ref document: EP

Kind code of ref document: A1