WO2023169369A1 - 一种行人重识别方法、系统、装置、设备及介质 - Google Patents

一种行人重识别方法、系统、装置、设备及介质 Download PDF

Info

Publication number
WO2023169369A1
WO2023169369A1 PCT/CN2023/079895 CN2023079895W WO2023169369A1 WO 2023169369 A1 WO2023169369 A1 WO 2023169369A1 CN 2023079895 W CN2023079895 W CN 2023079895W WO 2023169369 A1 WO2023169369 A1 WO 2023169369A1
Authority
WO
WIPO (PCT)
Prior art keywords
pedestrian
pooling
image features
image
memory
Prior art date
Application number
PCT/CN2023/079895
Other languages
English (en)
French (fr)
Inventor
杨宏斌
董刚
刘海威
蒋东东
曹其春
梁玲燕
晁银银
胡克坤
王斌强
尹文枫
Original Assignee
浪潮(北京)电子信息产业有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 浪潮(北京)电子信息产业有限公司 filed Critical 浪潮(北京)电子信息产业有限公司
Publication of WO2023169369A1 publication Critical patent/WO2023169369A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Definitions

  • the present application relates to the field of image processing technology, and more specifically, to a pedestrian re-identification method, system, device, equipment and non-volatile readable storage medium.
  • Pedestrian re-identification also called pedestrian re-identification, is a technology that uses computer vision technology to determine whether a specific pedestrian exists in an image or video sequence. Given a monitored pedestrian image, retrieve images of the pedestrian across devices. It is designed to make up for the visual limitations of current fixed cameras, and can be combined with pedestrian detection/pedestrian tracking technology, and can be widely used in intelligent video surveillance, intelligent security and other fields.
  • This application is to provide a pedestrian re-identification method, which can solve the technical problem of how to improve the applicability of pedestrian re-identification to a certain extent.
  • This application also provides a pedestrian re-identification system, device, equipment and non-volatile readable storage medium.
  • a pedestrian re-identification method applied to heterogeneous computing devices including:
  • image features include image features of the target pedestrian image to be recognized
  • the pedestrian re-identification result corresponding to the target pedestrian image is determined based on the pooling processing result and the convolution processing result.
  • the image features are pooled according to the instruction sequence to obtain the pooling results, including:
  • the data corresponding to the image features are stored separately in three first-in first-out memories.
  • the output end of the third first-in first-out memory is the same as the first first-in first-out memory.
  • the input terminals of the memory are connected;
  • the depth of the first first-in-first-out memory is the width of the feature map of the target pedestrian image
  • the second and third FIFO memories are twice as deep as they are wide.
  • the method further includes:
  • the pedestrian re-identification results are transmitted to a preset database to determine the target pedestrian information based on the pedestrian re-identification results and the person information stored in the preset database.
  • the image features, quantization parameters, instruction sequence and filter coefficients, pooling the image features according to the instruction sequence, and before obtaining the pooling processing result it also includes:
  • image features, quantization parameters, instruction sequences and filter coefficients are stored in memory, including:
  • a pedestrian re-identification system applied to heterogeneous computing devices including:
  • the first acquisition module is used to acquire image features, quantization parameters, instruction sequences and filter coefficients.
  • the image features include image features of the target pedestrian image to be recognized;
  • the first pooling module is used to pool the image features according to the instruction sequence to obtain the pooling processing results
  • the first convolution module is used to perform convolution processing on the image features according to the instruction sequence and based on the quantization parameters and filter coefficients to obtain the convolution processing results;
  • the first determination module is used to determine the pedestrian re-identification result corresponding to the target pedestrian image based on the pooling processing result and the convolution processing result.
  • a pedestrian re-identification device includes heterogeneous computing equipment
  • Heterogeneous computing equipment includes: PCIE controller; a crossbar switch matrix connected to the PCIE controller; a scheduling core connected to the crossbar switch matrix; a memory connected to the crossbar switch matrix; and a pool connected to the scheduling core, switch crossbar matrix, and memory respectively.
  • transformation operator a convolution operator connected to the scheduling core, switch cross matrix and memory respectively;
  • the PCIE controller is used to store image features, quantization parameters, instruction sequences and filter coefficients into the memory through the cross-switch matrix.
  • the image features include the image features of the target pedestrian image to be recognized;
  • Scheduling core used to control the work of pooling operators and convolution operators based on instruction sequences
  • the convolution operator is used to convolve image features based on quantization parameters and filter coefficients.
  • a pedestrian re-identification device including:
  • Memory used to store computer programs
  • the processor is used to implement the steps of any of the above person re-identification methods when executing a computer program.
  • a non-volatile readable storage medium A computer program is stored in the non-volatile readable storage medium. When the computer program is executed by a processor, the steps of any of the above person re-identification methods are implemented.
  • This application provides a pedestrian re-identification method, which is applied to heterogeneous computing devices to obtain image features, quantified parameters, instruction sequences and filter coefficients; the image features include image features of the target pedestrian image to be recognized; the image features are processed according to the instruction sequence Perform pooling processing to obtain the pooling processing results; perform image processing according to the instruction sequence and based on the quantization parameters and filter coefficients. The features are convolved to obtain the convolution processing result; the pedestrian re-identification result corresponding to the target pedestrian image is determined based on the pooling processing result and the convolution processing result.
  • This application implements a pedestrian re-identification network with the help of heterogeneous computing equipment, which can improve computing efficiency, high energy efficiency, flexibility, and good applicability.
  • the pedestrian re-identification system, device, equipment and non-volatile readable storage medium provided by this application also solve the corresponding technical problems.
  • Figure 1 is a flow chart of a pedestrian re-identification method provided by an embodiment of the present application
  • Figure 2 is a schematic structural diagram of a pedestrian re-identification system provided by an embodiment of the present application.
  • Figure 3 is a first structural schematic diagram of a pedestrian re-identification device provided by an embodiment of the present application.
  • Figure 4 is a schematic diagram of the pooling unit with a 3x3 structure
  • Figure 5 is a schematic diagram of the pooling unit with a 2x2 structure
  • Figure 6 is a second structural schematic diagram of a pedestrian re-identification device provided by an embodiment of the present application.
  • Figure 7 is a schematic structural diagram of the convolution unit
  • Figure 8 is a schematic diagram of the use of the pedestrian re-identification device in this application.
  • Figure 9 is a schematic structural diagram of a pedestrian re-identification device provided by an embodiment of the present application.
  • Figure 10 is another structural schematic diagram of a pedestrian re-identification device provided by an embodiment of the present application.
  • Figure 1 is a flow chart of a pedestrian re-identification method provided by an embodiment of the present application.
  • Some embodiments of the present application provide a method for pedestrian re-identification, which is applied to heterogeneous computing devices and may include the following steps:
  • Step S101 Obtain image features, quantization parameters, instruction sequences and filter coefficients; the image features include image features of the target pedestrian image to be recognized.
  • heterogeneous computing devices can first obtain image features, quantified parameters, instruction sequences, and filter coefficients for subsequent pedestrian re-identification based on image features, quantified parameters, instruction sequences, and filter coefficients; and the image features include targets to be identified.
  • image features include targets to be identified.
  • the specific contents of the image characteristics of the pedestrian image, quantization parameters, instruction sequences and filter coefficients can be determined according to actual needs, and are not specifically limited here in this application.
  • the heterogeneous computing device obtains image features, quantized parameters, instruction sequences, and filter coefficients
  • the image features, quantized parameters, and The number, instruction sequence and filter coefficients are stored in the memory.
  • the image features, quantization parameters, instruction sequences and filter coefficients can also be stored in different memories.
  • Step S102 Perform pooling processing on the image features according to the instruction sequence to obtain the pooling processing result.
  • heterogeneous computing devices after heterogeneous computing devices obtain image features, quantization parameters, instruction sequences, and filter coefficients, they can pool the image features according to the instruction sequence to obtain pooling processing results.
  • pooling image features according to instruction sequences to obtain pooling results in order to speed up the processing rate, when the pooling format is 3*3, it can be based on the instruction sequence.
  • the data corresponding to the image features are separately stored in three first-in first-out memories, where the output terminal of the third first-in first-out memory is connected to the input terminal of the first first-in first-out memory; for the three first-in first-out memories
  • the data in are pooled and the pooling results are obtained.
  • the parameters of the three FIFO memories can be determined according to actual needs.
  • the depth of the first FIFO memory can be the width of the feature map of the target pedestrian image
  • the second and third FIFO memory can be the width of the feature map of the target pedestrian image.
  • the depth of the out-of-memory memory can be twice the width, etc.
  • first-in first-out memories are used to store data for calculation, which decouples the data loading side and the data calculation side, so that the data loading module only needs to determine the dissatisfaction flag of the first-in first-out memory. to sequentially read the external memory for loading. Since the data is stored continuously, it only needs to be read in a streaming manner. There is no need to determine the address.
  • the output side of the FIFO memory determines the not-empty flag of each FIFO memory to output the data.
  • the computing module is a pipeline calculation; in this way, the data input and output ports can be fixed, without using a port selector, and external data only needs to be read once.
  • the cache reuses data according to the pooling mode, which can significantly simplify the system design and improve system performance.
  • Step S103 Convolve the image features according to the instruction sequence and based on the quantization parameters and filter coefficients to obtain the convolution processing result.
  • heterogeneous computing devices perform pooling processing on image features according to the instruction sequence. After obtaining the pooling processing results, they can perform convolution processing on the image features according to the instruction sequence and based on quantization parameters and filter coefficients to obtain the convolution process result.
  • Step S104 Determine the pedestrian re-identification result corresponding to the target pedestrian image based on the pooling processing result and the convolution processing result.
  • the heterogeneous computing device can determine the pedestrian re-identification result corresponding to the target pedestrian image based on the pooling processing results and the convolution processing results. Specifically, it can first be based on the pooling processing results and the convolution processing results. The pedestrian characteristics carried by the target pedestrian image are determined based on the results of the processing and convolution processing, and then the corresponding pedestrian re-identification results are determined based on the pedestrian characteristics.
  • This application does not make specific limitations here.
  • the heterogeneous computing device determines the pedestrian re-identification results corresponding to the target pedestrian image based on the pooling processing results and the convolution processing results, it can also transmit the pedestrian re-identification results to the preset database to use the pedestrian re-identification results based on the pedestrian re-identification results. and the person information stored in the preset database to determine the target pedestrian information, so as to obtain as much pedestrian information as possible with the help of the preset database, so as to facilitate users to use the target pedestrian information to analyze the target pedestrian image.
  • This application provides a pedestrian re-identification method, which is applied to heterogeneous computing devices to obtain image features, quantitative parameters, Instruction sequence and filter coefficient; the image features include the image features of the target pedestrian image to be recognized; the image features are pooled according to the instruction sequence to obtain the pooling processing result; the image features are processed according to the instruction sequence and based on the quantization parameters and filter coefficients. Convolution processing is performed to obtain the convolution processing result; the pedestrian re-identification result corresponding to the target pedestrian image is determined based on the pooling processing result and the convolution processing result.
  • This application implements a pedestrian re-identification network with the help of heterogeneous computing equipment, which can improve computing efficiency, high energy efficiency, flexibility, and good applicability.
  • Figure 2 is a schematic structural diagram of a pedestrian re-identification system provided by an embodiment of the present application.
  • Some embodiments of the present application provide a pedestrian re-identification system, which is applied to heterogeneous computing devices and may include:
  • the first acquisition module 101 is used to acquire image features, quantization parameters, instruction sequences and filter coefficients.
  • the image features include image features of the target pedestrian image to be recognized;
  • the first pooling module 102 is used to perform pooling processing on image features according to the instruction sequence to obtain pooling processing results;
  • the first convolution module 103 is used to perform convolution processing on the image features according to the instruction sequence and based on the quantization parameters and filter coefficients to obtain the convolution processing results;
  • the first determination module 104 is configured to determine the pedestrian re-identification result corresponding to the target pedestrian image based on the pooling processing result and the convolution processing result.
  • the first pooling module may include:
  • the first storage unit is used to store the data corresponding to the image features into three first-in-first-out memories based on the instruction sequence when the pooling format is 3*3. Among them, the output end of the third first-in-first-out memory Connected to the input of the first first-in-first-out memory;
  • the first pooling unit is used to pool the data in the three first-in-first-out memories to obtain the pooling processing result.
  • Some embodiments of the present application provide a pedestrian re-identification system, which is applied to heterogeneous computing devices.
  • the depth of the first first-in first-out memory is the width of the feature map of the target pedestrian image, and the second and third first-in first-out memories are The memory depth is twice the width.
  • Some embodiments of the present application provide a pedestrian re-identification system, which is applied to heterogeneous computing devices and may also include:
  • the first transmission module is used for the first determination module to determine the pedestrian re-identification result corresponding to the target pedestrian image based on the pooling processing result and the convolution processing result, and then transmit the pedestrian re-identification result to the preset database, so as to based on the pedestrian re-identification result and The person information stored in the default database determines the target pedestrian information.
  • Some embodiments of the present application provide a pedestrian re-identification system, which is applied to heterogeneous computing devices and may also include:
  • the first storage module is used for the first acquisition module to obtain image features, quantization parameters, instruction sequences and filter coefficients.
  • the first pooling module performs pooling processing on the image features according to the instruction sequence. Before obtaining the pooling processing result, the image Features, quantization parameters, instruction sequences and filter coefficients are stored in memory.
  • the first storage module may include:
  • the second storage unit is used to store image features, quantization parameters, instruction sequences and filter coefficients into different memories middle.
  • FIG. 3 is a first structural schematic diagram of a pedestrian re-identification device provided by an embodiment of the present application.
  • a pedestrian re-identification device provided by some embodiments of the present application includes heterogeneous computing equipment
  • Heterogeneous computing equipment includes: PCIE (peripheral component interconnect express, high-speed serial computer expansion bus standard) controller; a crossbar connected to the PCIE controller; a scheduling core connected to the crossbar matrix; and a crossbar matrix
  • PCIE peripheral component interconnect express, high-speed serial computer expansion bus standard
  • the crossbar connected to the PCIE controller
  • a scheduling core connected to the crossbar matrix
  • a crossbar matrix The connected memory;
  • the pooling unit (Pooling Unit) connected to the scheduling core, the switching cross matrix and the memory respectively;
  • the convolution unit Convolution Unit connected to the scheduling core, the switching cross matrix and the memory respectively;
  • the PCIE controller is used to store image features, quantization parameters, instruction sequences and filter coefficients into the memory through the cross-switch matrix.
  • the image features include the image features of the target pedestrian image to be recognized;
  • Scheduling core used to control the work of pooling operators and convolution operators based on instruction sequences
  • the convolution operator is used to convolve image features based on quantization parameters and filter coefficients.
  • the structures of the convolution operator and pooling operator can be determined according to actual needs. For example, considering that the 3x3 part of the depth-separable convolution does not have channel accumulation, a feature map corresponds to a 3x3kernel, so the filtering switch The frequency is low, and 1x1 convolution has channel accumulation. In order to reduce the cache of intermediate results, the calculation of each channel of a set of data is given priority. This will involve frequent switching of the filtering of each channel, and the two are output to the PE array in the convolution.
  • the bandwidth is different, so the filter cache is divided into two sets of caches, 1x1 and 3x3, that is, the filter coefficients can be the parameters of the 1x1 filter and the parameters of the 3x3 filter; and the convolution operator, pooling operator, memory, scheduling core, etc.
  • the quantity can also be determined according to actual needs, and is not specifically limited in this application.
  • the method and capacity of caching image features, quantization parameters, instruction sequences and filter coefficients in the PCIE controller can also be determined according to actual needs. For example, a cache capacity of 512K can be used to cache image features, and a cache capacity of 2MB can be used to cache 1x1 filter coefficients, 64KB cache space is used to cache 3x3 filter coefficients, 8K cache capacity is used to cache instruction sequences, 1M cache capacity is used to cache quantization parameters, etc.
  • This application does not make specific limitations here.
  • the operating principle of the person re-identification network on heterogeneous computing devices in this application can be referred to the existing technology, and will not be described again here.
  • other devices that implement corresponding functions can also be added to the pedestrian re-identification device provided in this application as needed, which will not be described in detail here.
  • the framework of the heterogeneous computing device in this application can be flexibly selected according to actual needs.
  • the framework of the heterogeneous computing device can be FPGA (Field Programmable Gate Array, Field Programmable Logic Gate Array), etc.
  • the type and structure of the instruction sequence can be flexibly determined according to actual needs.
  • the convolution instruction format can be as shown in Table 1
  • the pooling instruction format can be as shown in Table 2
  • the parameter definitions in the instructions can be as follows.
  • the instruction sequence can be as shown in Table 4, etc.
  • This application provides a pedestrian re-identification device, which includes heterogeneous computing equipment that implements a pedestrian re-identification network; the heterogeneous computing equipment includes: a PCIE controller; a cross-switch matrix connected to the PCIE controller; and a dispatcher connected to the cross-switch matrix core; a memory connected to the crossbar matrix; a pooling operator connected to the scheduling core, the switch crossmatrix and the memory respectively; a convolution operator connected to the scheduling core, the switch crossmatrix and the memory respectively; a PCIE controller for The image features, quantization parameters, instruction sequences and filter coefficients are stored in the memory through the cross switch matrix.
  • the image features include the image features of the target pedestrian image to be recognized; the scheduling core is used to control the pooling operator and convolution operation based on the instruction sequence.
  • This application implements a pedestrian re-identification network with the help of heterogeneous computing equipment, which can improve computing efficiency, high energy efficiency, flexibility, and good applicability.
  • the PCIE controller in order to facilitate data interaction between the PCIE controller and the crossbar switch matrix, can access the line through the register (Register Ctrl), the interrupt interface (Interrupt Ctrl) and the direct memory (DMA Transfer) and so on are connected to the crossbar switch matrix.
  • the memory may include a first memory that stores image features, a second memory that stores filter coefficients, and a third memory that stores quantization parameters. ; And a set of memories is used to store input data or output data of a network layer, which can reduce the use of on-chip cache.
  • pooling and residual results in the person re-identification network will be used as input data after several layers of convolution calculations, additional caches need to be used for temporary storage; in addition, the pooling and convolution of each layer All calculations such as product require ping-pong.
  • the storage structure is used to access data, that is, data is read from one set of caches, and after the calculation is completed, the results are written to another set of caches to prevent the source data from being overwritten. This requires multiple sets of caches for data flow at each layer.
  • the memory can include ultra random access memory (Ultra Random Access Memory, URAM), etc.
  • URAM Ultra Random Access Memory
  • Figure 4 is a schematic diagram of a pooling unit with a 3x3 structure
  • Figure 5 is a schematic diagram of a pooling unit with a 2x2 structure.
  • the pooling operator when the structure of the pooling operator is 3*3, the pooling operator includes 3 First Input First Output (FIFO) memories that store data to be calculated. ), the output terminal of the third FIFO memory is connected to the input terminal of the first FIFO memory, and the depth of the first FIFO memory is the width of the feature map of the target pedestrian image, and the second and The depth of the third first-in first-out memory is twice the width.
  • FIFO First Input First Output
  • the read port on the left side of Figure 4 reads the feature map data sequentially and loads the three first-in first-out memories in sequence, as shown in the figure As shown on the left side of 4, when loading into the FIFO memory of the third row, that is, when the third row has at least 1 data, the FIFO memory of the three rows starts to output in parallel, and three data are output in one clock cycle.
  • the maximum pool Choose to enter the 3-input pipeline calculation module as shown on the right side of Figure 4. First, the sum or maximum value of the 3 values is obtained through the addition tree or comparison tree structure, and then the sum or maximum value of the 3 values is obtained through the accumulator or circular comparator. It takes 3 clock cycles to get the final result.
  • the data of the third row is sent back to the first row.
  • the data loading module continuously loads two or three rows, while outputting one row. After loading two rows, the data can be continuously sent to the corresponding computing module to improve system performance.
  • the pooling operator When the structure of the pooling unit is 2*2, the pooling operator includes two first-in first-out memories that store the data to be calculated. At this time, you only need to load the data into two first-in first-out memories.
  • first-in When loading into the second row, first-in When the first-in-first-out memory is used, that is, when there is at least 1 data in the second row, the two rows of first-in first-out memory begin to output in parallel, and two data are output in one clock cycle.
  • the maximum pooling or average pooling selection enter as shown in Figure 5 2-input pipeline calculation module; first, the sum or maximum value of the two values is obtained through the addition tree or comparison tree structure, and then the final result is obtained through the accumulator or circular comparator in 2 clock cycles.
  • FIG. 6 is a second structural schematic diagram of a pedestrian re-identification device provided by an embodiment of the present application.
  • a database connected to heterogeneous computing devices is used to store character information.
  • the pedestrian calculation results can be matched with the database to obtain the character information of the target pedestrian.
  • the pedestrian re-identification device in order to facilitate the user to learn the character information of the target pedestrian, may also include a display for displaying the character information corresponding to the pedestrian calculation result in the database.
  • the pedestrian calculation result is also Computational results of target pedestrian images by heterogeneous computing devices.
  • a pedestrian re-identification device provided by this application may also include: a graphics processor connected to the heterogeneous computing equipment for processing pre-processed pedestrian images based on the YOLO network.
  • Perform target detection to obtain the detection results used to determine the target pedestrian image.
  • the target pedestrian image can be obtained by performing human frame preprocessing on the detection results, such as image extraction, cropping, etc.
  • a pedestrian re-identification device may also include: a multi-channel camera connected to a graphics processor for shooting the original pedestrian image.
  • the original pedestrian image is Decode and preprocess the pedestrian image. For example, opencv decodes the original pedestrian image according to a fixed frame rate, and performs size adjustment, pixel adjustment, etc. to obtain the preprocessed pedestrian image.
  • the structure of the convolution unit in this application can be determined according to actual needs.
  • its structure can be as shown in Figure 7.
  • the convolution unit uses a PE array, multiple channels are calculated in parallel, and there are residuals in the structure in turn. , quantization and activation modules. Specifically, you only need to select the corresponding calculation module according to whether there are residual and activation operations, and you can get the final calculation result.
  • the filter loading signal is output to the filter cache so that the corresponding filter can be loaded according to the determined timing to ensure the efficient operation of the PE array.
  • the structure of the convolution unit is simple and efficient, which reduces repeated memory access by multiple modules, simplifies signal interaction control, and improves overall computing efficiency.
  • FIG 8 is a schematic diagram of the use of the pedestrian re-identification device in this application.
  • the use process of the pedestrian re-identification device in this application is now described in combination with the host computer used to manage and control the pedestrian re-identification device, such as a server, which may include the following steps:
  • Step S201 Install the multi-channel camera, GPU computing card, and heterogeneous computing accelerator card on the workstation, and connect the monitor output;
  • Step S202 The host computer writes the filter parameters to the on-chip URAM cache through the PCIE interface;
  • Step S203 The host computer writes the quantization parameters to the on-chip URAM cache through the PCIE interface;
  • Step S204 The host computer writes the sequence of instructions to be executed into the on-chip URAM cache through the PCIE interface;
  • Step S205 The multi-channel camera collects image frames at a fixed frame rate according to real-time requirements to obtain original pedestrian images
  • Step S206 The software calls the opencv image processing function to decode and preprocess the original pedestrian image to obtain the preprocessed pedestrian image;
  • Step S207 The software sends the processed image to the YOLO network on the GPU for target detection and obtains the detection result;
  • Step S208 The software performs human frame preprocessing on the detection results to obtain the target pedestrian image
  • Step S209 The host computer writes the image feature data of the target pedestrian image to the on-chip URAM cache through the PCIE interface;
  • Step S210 The host computer sends a command to the on-chip scheduling core through the PCIE interface to start execution;
  • Step S211 Schedule the kernel to fetch and decode, and sequentially generate the control and parameter signals of each layer to the convolution unit, pooling unit and other computing components;
  • Step S212 The computing component sequentially completes fetching numbers from the cache, calculating, and writing the results back to the cache;
  • Step S213 After the calculations of all layers in the network are completed, obtain the calculation results of a set of images;
  • Step S214 The scheduling core generates a PCIE interrupt signal to the host computer, and the host computer retrieves the calculation result after receiving it;
  • Step S215 The software queries and matches the calculation results in the database, and outputs the person-related information on the display;
  • Step S216 Repeat the steps of collecting image frames at a fixed frame rate by the multi-channel camera according to real-time requirements and subsequent steps until the pedestrian re-identification task is completed.
  • This application also provides a pedestrian re-identification device and a non-volatile readable storage medium, both of which have the corresponding effects of a pedestrian re-identification method provided by some embodiments of this application.
  • FIG. 9 is a schematic structural diagram of a pedestrian re-identification device provided by an embodiment of the present application.
  • a pedestrian re-identification device provided by some embodiments of the present application includes a memory 201 and a processor 202.
  • a computer program is stored in the memory 201.
  • the processor 202 executes the computer program, it implements the pedestrian re-identification method described in any of the above embodiments. step.
  • another pedestrian re-identification device may also include: an input port 203 connected to the processor 202 for transmitting commands input from the outside to the processor 202; and the processor 202
  • the connected display unit 204 is used to display the processing results of the processor 202 to the outside world;
  • the communication module 205 connected to the processor 202 is used to implement communication between the pedestrian re-identification device and the outside world.
  • the display unit 204 can be a display panel, a laser scanning display, etc.; the communication methods used by the communication module 205 include but are not limited to mobile high-definition link technology (HML), universal serial bus (USB), high-definition multimedia interface (HDMI), Wireless connection: wireless fidelity technology (WiFi), Bluetooth communication technology, low-power Bluetooth communication technology, communication technology based on IEEE802.11s.
  • HML mobile high-definition link technology
  • USB universal serial bus
  • HDMI high-definition multimedia interface
  • WiFi wireless fidelity technology
  • Bluetooth communication technology low-power Bluetooth communication technology
  • Some embodiments of the present application provide a non-volatile readable storage medium.
  • the non-volatile readable storage medium stores a computer program.
  • pedestrian re-identification is implemented as described in any of the above embodiments. Method steps.
  • non-volatile readable storage media involved in this application include random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROM, or any other form of storage media known in the technical field.
  • RAM random access memory
  • ROM read-only memory
  • electrically programmable ROM electrically erasable programmable ROM
  • registers hard disks, removable disks, CD-ROM, or any other form of storage media known in the technical field.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Processing (AREA)
  • Traffic Control Systems (AREA)

Abstract

本申请公开了一种行人重识别方法、系统、装置、设备及计算机可读存储介质，应用于异构计算设备，获取图像特征、量化参数、指令序列和滤波系数；图像特征包括待识别的目标行人图像的图像特征；按照指令序列对图像特征进行池化处理，得到池化处理结果；按照指令序列并基于量化参数和滤波系数对图像特征进行卷积处理，得到卷积处理结果；基于池化处理结果和卷积处理结果确定目标行人图像对应的行人重识别结果。本申请借助异构计算设备实现了行人重识别网络，可以提高计算效率，能效比及灵活性高，适用性好。本申请提供的一种行人重识别系统、装置、设备及计算机可读存储介质也解决了相应技术问题。

Description

一种行人重识别方法、系统、装置、设备及介质
相关申请的交叉引用
本申请要求于2022年3月11日提交中国专利局，申请号为202210242524.6，申请名称为“一种行人重识别方法、系统、装置、设备及计算机介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像处理技术领域，更具体地说，涉及一种行人重识别方法、系统、装置、设备及非易失性可读存储介质。
背景技术
行人重识别也称行人再识别，是利用计算机视觉技术判断图像或者视频序列中是否存在特定行人的技术。给定一个监控行人图像，检索跨设备下的该行人图像。旨在弥补目前固定的摄像头的视觉局限，并可与行人检测/行人跟踪技术相结合，可广泛应用于智能视频监控、智能安保等领域。
现有实现行人重识别的方法有：在GPU(graphics processing unit，图形处理器)平台上运行所有的过程，包括图像采集、后面处理流程等，但该方法的计算效率低、能效比低且灵活性差，适用性差。
发明内容
本申请的目的是提供一种行人重识别方法，其能在一定程度上解决如何提高行人重识别的适用性的技术问题。本申请还提供了一种行人重识别系统、装置、设备及非易失性可读存储介质。
为了实现上述目的，本申请提供如下技术方案：
一种行人重识别方法，应用于异构计算设备，包括：
获取图像特征、量化参数、指令序列和滤波系数；图像特征包括待识别的目标行人图像的图像特征；
按照指令序列对图像特征进行池化处理，得到池化处理结果；
按照指令序列并基于量化参数和滤波系数对图像特征进行卷积处理，得到卷积处理结果；
基于池化处理结果和卷积处理结果确定目标行人图像对应的行人重识别结果。
在一些实施例中的，按照指令序列对图像特征进行池化处理，得到池化处理结果，包括：
当池化格式为3*3时，基于指令序列，将图像特征对应的数据分开存入三个先入先出存储器中，其中，第三个先入先出存储器的输出端与第一个先入先出存储器的输入端相连接；
对三个先入先出存储器中的数据进行池化处理，得到池化处理结果。
在一些实施例中的，第一个先入先出存储器的深度为目标行人图像的特征图的宽度，第 二个及第三个先入先出存储器的深度为两倍的宽度。
在一些实施例中的，基于池化处理结果和卷积处理结果确定目标行人图像对应的行人重识别结果之后，还包括：
传输行人重识别结果至预设数据库，以基于行人重识别结果及预设数据库中存储的人物信息确定目标行人信息。
在一些实施例中的，获取图像特征、量化参数、指令序列和滤波系数之后，按照指令序列对图像特征进行池化处理，得到池化处理结果之前，还包括：
将图像特征、量化参数、指令序列和滤波系数存入存储器。
在一些实施例中的，将图像特征、量化参数、指令序列和滤波系数存入存储器，包括：
将图像特征、量化参数、指令序列和滤波系数存入不同的存储器中。
一种行人重识别系统，应用于异构计算设备，包括：
第一获取模块，用于获取图像特征、量化参数、指令序列和滤波系数，图像特征包括待识别的目标行人图像的图像特征；
第一池化模块，用于按照指令序列对图像特征进行池化处理，得到池化处理结果；
第一卷积模块，用于按照指令序列并基于量化参数和滤波系数对图像特征进行卷积处理，得到卷积处理结果；
第一确定模块，用于基于池化处理结果和卷积处理结果确定目标行人图像对应的行人重识别结果。
一种行人重识别装置，包括异构计算设备；
异构计算设备包括：PCIE控制器；与PCIE控制器连接的交叉开关矩阵；与交叉开关矩阵连接的调度核；与交叉开关矩阵连接的存储器；分别与调度核、开关交叉矩阵及存储器连接的池化运算器；分别与调度核、开关交叉矩阵及存储器连接的卷积运算器；
PCIE控制器，用于通过交叉开关矩阵将图像特征、量化参数、指令序列及滤波系数存入存储器，图像特征包括待识别的目标行人图像的图像特征；
调度核，用于基于指令序列控制池化运算器、卷积运算器的工作；
池化运算器，用于对图像特征进行池化处理；
卷积运算器，用于基于量化参数、滤波系数对图像特征进行卷积处理。
一种行人重识别设备，包括：
存储器，用于存储计算机程序；
处理器，用于执行计算机程序时实现如上任一行人重识别方法的步骤。
一种非易失性可读存储介质，非易失性可读存储介质中存储有计算机程序，计算机程序被处理器执行时实现如上任一行人重识别方法的步骤。
本申请提供的一种行人重识别方法，应用于异构计算设备，获取图像特征、量化参数、指令序列和滤波系数；图像特征包括待识别的目标行人图像的图像特征；按照指令序列对图像特征进行池化处理，得到池化处理结果；按照指令序列并基于量化参数和滤波系数对图像 特征进行卷积处理，得到卷积处理结果；基于池化处理结果和卷积处理结果确定目标行人图像对应的行人重识别结果。本申请借助异构计算设备实现了行人重识别网络，可以提高计算效率，能效比及灵活性高，适用性好。本申请提供的一种行人重识别系统、装置、设备及非易失性可读存储介质也解决了相应技术问题。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据提供的附图获得其他的附图。
图1为本申请实施例提供的一种行人重识别方法的流程图；
图2为本申请实施例提供的一种行人重识别系统的结构示意图；
图3为本申请实施例提供的一种行人重识别装置的第一结构示意图；
图4为3x3结构的池化单元示意图；
图5为2x2结构的池化单元示意图；
图6为本申请实施例提供的一种行人重识别装置的第二结构示意图；
图7为卷积单元的结构示意图；
图8为本申请中行人重识别装置的使用示意图；
图9为本申请实施例提供的一种行人重识别设备的结构示意图；
图10为本申请实施例提供的一种行人重识别设备的另一结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
请参阅图1，图1为本申请实施例提供的一种行人重识别方法的流程图。
本申请一些实施例提供的一种行人重识别方法，应用于异构计算设备，可以包括以下步骤：
步骤S101：获取图像特征、量化参数、指令序列和滤波系数；图像特征包括待识别的目标行人图像的图像特征。
实际应用中，异构计算设备可以先获取图像特征、量化参数、指令序列和滤波系数，以便后续基于图像特征、量化参数、指令序列和滤波系数进行行人重识别；且图像特征包括待识别的目标行人图像的图像特征，量化参数、指令序列和滤波系数的具体内容可以根据实际需要确定，本申请在此不做具体限定。
具体应用场景中，异构计算设备在获取图像特征、量化参数、指令序列和滤波系数之后，为了便于应用图像特征、量化参数、指令序列和滤波系数，还可以将图像特征、量化参 数、指令序列和滤波系数存入存储器。具体的，为了便于区分图像特征、量化参数、指令序列和滤波系数，还可以将图像特征、量化参数、指令序列和滤波系数存入不同的存储器中等。
步骤S102：按照指令序列对图像特征进行池化处理，得到池化处理结果。
实际应用中，异构计算设备在获取图像特征、量化参数、指令序列和滤波系数之后，便可以按照指令序列对图像特征进行池化处理，得到池化处理结果。
具体应用场景中，异构计算设备在按照指令序列对图像特征进行池化处理，得到池化处理结果的过程中，为了加快处理速率，当池化格式为3*3时，可以基于指令序列，将图像特征对应的数据分开存入三个先入先出存储器中，其中，第三个先入先出存储器的输出端与第一个先入先出存储器的输入端相连接；对三个先入先出存储器中的数据进行池化处理，得到池化处理结果。且具体应用场景中，三个先入先出存储器的参数可以根据实际需要确定，比如第一个先入先出存储器的深度可以为目标行人图像的特征图的宽度，第二个及第三个先入先出存储器的深度可以为两倍的宽度等。
需要说明的是，在一些实施例中，借助不同的先入先出存储器存储用于计算的数据，解耦合了数据加载侧和数据计算侧，使得数据加载模块只需要判断先入先出存储器的不满标志来依次读取外存进行加载，由于数据是连续存放，只需要流式读取即可，不需要判断地址，先入先出存储器输出侧判断各先入先出存储器的不空标志来进行输出，数据计算模块是流水线计算；这样可以数据输入输出的端口固定，无需使用端口选择器，且外部数据只需读取一次，缓存根据池化模式进行数据重用，可以明显简化系统设计，提高系统性能。
步骤S103：按照指令序列并基于量化参数和滤波系数对图像特征进行卷积处理，得到卷积处理结果。
实际应用中，异构计算设备在按照指令序列对图像特征进行池化处理，得到池化处理结果之后，便可以按照指令序列并基于量化参数和滤波系数对图像特征进行卷积处理，得到卷积处理结果。
步骤S104：基于池化处理结果和卷积处理结果确定目标行人图像对应的行人重识别结果。
实际应用中，异构计算设备得到池化处理结果和卷积处理结果之后，便可以基于池化处理结果和卷积处理结果确定目标行人图像对应的行人重识别结果，具体的，可以先基于池化处理结果和卷积处理结果确定目标行人图像携带的行人特征，再基于该行人特征确定对应的行人重识别结果等，本申请在此不做具体限定。
具体应用场景中，异构计算设备在基于池化处理结果和卷积处理结果确定目标行人图像对应的行人重识别结果之后，还可以传输行人重识别结果至预设数据库，以基于行人重识别结果及预设数据库中存储的人物信息确定目标行人信息，以借助预设数据库尽可能多的获取行人信息，方便用户等使用目标行人信息对目标行人图像进行分析。
本申请提供的一种行人重识别方法，应用于异构计算设备，获取图像特征、量化参数、 指令序列和滤波系数；图像特征包括待识别的目标行人图像的图像特征；按照指令序列对图像特征进行池化处理，得到池化处理结果；按照指令序列并基于量化参数和滤波系数对图像特征进行卷积处理，得到卷积处理结果；基于池化处理结果和卷积处理结果确定目标行人图像对应的行人重识别结果。本申请借助异构计算设备实现了行人重识别网络，可以提高计算效率，能效比及灵活性高，适用性好。
请参阅图2，图2为本申请实施例提供的一种行人重识别系统的结构示意图。
本申请一些实施例提供的一种行人重识别系统，应用于异构计算设备，可以包括：
第一获取模块101，用于获取图像特征、量化参数、指令序列和滤波系数，图像特征包括待识别的目标行人图像的图像特征；
第一池化模块102，用于按照指令序列对图像特征进行池化处理，得到池化处理结果；
第一卷积模块103，用于按照指令序列并基于量化参数和滤波系数对图像特征进行卷积处理，得到卷积处理结果；
第一确定模块104，用于基于池化处理结果和卷积处理结果确定目标行人图像对应的行人重识别结果。
本申请一些实施例提供的一种行人重识别系统，应用于异构计算设备，第一池化模块可以包括：
第一存储单元，用于当池化格式为3*3时，基于指令序列，将图像特征对应的数据分开存入三个先入先出存储器中，其中，第三个先入先出存储器的输出端与第一个先入先出存储器的输入端相连接；
第一池化单元，用于对三个先入先出存储器中的数据进行池化处理，得到池化处理结果。
本申请一些实施例提供的一种行人重识别系统，应用于异构计算设备，第一个先入先出存储器的深度为目标行人图像的特征图的宽度，第二个及第三个先入先出存储器的深度为两倍的宽度。
本申请一些实施例提供的一种行人重识别系统，应用于异构计算设备，还可以包括：
第一传输模块，用于第一确定模块基于池化处理结果和卷积处理结果确定目标行人图像对应的行人重识别结果之后，传输行人重识别结果至预设数据库，以基于行人重识别结果及预设数据库中存储的人物信息确定目标行人信息。
本申请一些实施例提供的一种行人重识别系统，应用于异构计算设备，还可以包括：
第一存储模块，用于第一获取模块获取图像特征、量化参数、指令序列和滤波系数之后，第一池化模块按照指令序列对图像特征进行池化处理，得到池化处理结果之前，将图像特征、量化参数、指令序列和滤波系数存入存储器。
本申请一些实施例提供的一种行人重识别系统，应用于异构计算设备，第一存储模块可以包括：
第二存储单元，用于将图像特征、量化参数、指令序列和滤波系数存入不同的存储器 中。
请参阅图3，图3为本申请实施例提供的一种行人重识别装置的第一结构示意图。
本申请一些实施例提供的一种行人重识别装置，包括异构计算设备；
异构计算设备包括：PCIE(peripheral component interconnect express，高速串行计算机扩展总线标准)控制器；与PCIE控制器连接的交叉开关矩阵(crossbar)；与交叉开关矩阵连接的调度核；与交叉开关矩阵连接的存储器；分别与调度核、开关交叉矩阵及存储器连接的池化运算器(Pooling Unit)；分别与调度核、开关交叉矩阵及存储器连接的卷积运算器(Convolution Unit)；
PCIE控制器，用于通过交叉开关矩阵将图像特征、量化参数、指令序列及滤波系数存入存储器，图像特征包括待识别的目标行人图像的图像特征；
调度核，用于基于指令序列控制池化运算器、卷积运算器的工作；
池化运算器，用于对图像特征进行池化处理；
卷积运算器，用于基于量化参数、滤波系数对图像特征进行卷积处理。
实际应用中，卷积运算器、池化运算器的结构等均可以根据实际需要确定，比如考虑到深度可分离卷积的3x3部分没有通道累加，一张特征图对应一个3x3kernel，因此滤波的切换频率较低，而1x1卷积有通道累加，为减少中间结果的缓存，优先进行一组数据的各通道计算，这样会涉及到各通道的滤波频繁切换，两者输出给卷积中PE阵列的带宽不同，因此将滤波缓存分为1x1和3x3两组缓存，也即滤波系数可以为1x1滤波器的参数、3x3滤波器的参数；且卷积运算器、池化运算器、存储器、调度核等的数量也可以根据实际需要确定，本申请在此不做具体限定。
需要说明的是，PCIE控制器中缓存图像特征、量化参数、指令序列及滤波系数的方式及容量等也可以根据实际需要确定，比如可以用512K的缓存容量缓存图像特征，用2MB的缓存容量缓存1x1滤波系数，用64KB的缓存空间缓存3x3滤波系数，用8K的缓存容量缓存指令序列，用1M的缓存容量缓存量化参数等，本申请在此不做具体限定。此外，本申请中异构计算设备上行人重识别网络的运行原理可以参阅现有技术，在此不再赘述。此外，还可以根据需要在本申请提供的行人重识别装置中添加实现相应功能的其他器件，本申请在此不再赘述。
需要说明的是，本申请中异构计算设备的框架可以根据实际需要灵活选择，比如异构计算设备的框架可以为FPGA(Field Programmable Gate Array，现场可编程逻辑门阵列)等。
具体应用场景中，指令序列的类型及结构等可以根据实际需要灵活确定，比如卷积指令格式可以如表1所示，池化指令格式可以如表2所示，且指令中的参数定义可以如表3所示，指令序列可以如表4所示等。
表1卷积指令格式
表2池化指令格式
表3参数定义表
表4指令序列示意表
本申请提供的一种行人重识别装置，包括实现行人重识别网络的异构计算设备；异构计算设备包括：PCIE控制器；与PCIE控制器连接的交叉开关矩阵；与交叉开关矩阵连接的调度核；与交叉开关矩阵连接的存储器；分别与调度核、开关交叉矩阵及存储器连接的池化运算器；分别与调度核、开关交叉矩阵及存储器连接的卷积运算器；PCIE控制器，用于通过交叉开关矩阵将图像特征、量化参数、指令序列及滤波系数存入存储器，图像特征包括待识别的目标行人图像的图像特征；调度核，用于基于指令序列控制池化运算器、卷积运算器的工作；池化运算器，用于对图像特征进行池化处理；卷积运算器，用于基于量化参数、滤波系数对图像特征进行卷积处理。本申请借助异构计算设备实现了行人重识别网络，可以提高计算效率，能效比及灵活性高，适用性好。
本申请一些实施例提供的一种行人重识别装置中，为了便于PCIE控制器与交叉开关矩阵进行数据交互，PCIE控制器可以通过寄存器(Register Ctrl)、中断接口(Interrupt Ctrl)及直接存储器访问线路(DMA Transfer)等与交叉开关矩阵相连接。
本申请一些实施例提供的一种行人重识别装置中，为了便于存储计算所需的数据，存储器可以包括存储图像特征的第一存储器，存储滤波系数的第二存储器，存储量化参数的第三存储器；且一组存储器用于存储一个网络层的输入数据或输出数据，这样可以减少片上缓存使用。
需要说明的是，由于行人重识别网络中的池化和残差结果会用作若干层卷积计算后的输入数据，因此需要使用额外的缓存来进行暂存；另外每层的池化、卷积等计算都需要乒乓缓 存结构来存取数据，即从一组缓存读取数据，计算完成后将结果写入另外一组缓存，以防止源数据被覆盖，这样便需要多组缓存来进行各层数据流转。
具体应用场景中，为了便于存储数据，存储器可以包括超级随机访问存储器(Ultra Random Access Memory，URAM)等。
请参阅图4和图5，图4为3x3结构的池化单元示意图，图5为2x2结构的池化单元示意图。
本申请一些实施例提供的一种行人重识别装置中，池化运算器的结构为3*3时，池化运算器包括3个存储待计算数据的先入先出存储器(First Input First Output，FIFO)，第三个先入先出存储器的输出端与第一个先入先出存储器的输入端相连接，且第一个先入先出存储器的深度为目标行人图像的特征图的宽度，第二个及第三个先入先出存储器的深度为两倍的宽度，当启动池化运算器的计算后，图4左侧读端口按顺序读取特征图数据，依次加载3个先入先出存储器，如图4左侧所示，当加载到第3行先入先出存储器时，即第三行至少有1个数据时，三行先入先出存储器开始并行输出，一个时钟周期输出三个数据，根据最大池化或平均池化选择进入如图4右侧所示的3输入流水线计算模块；首先3个值通过加法树或比较树结构得到3个值的和或最大值，继而通过累加器或循环比较器用3个时钟周期得到最终结果，在并行输出进行计算的同时，如图4左侧所示，第三行的数据回送给第一行，数据加载模块连续加载二、三行，在输出一行的同时要加载完两行，则数据可以不间断送给相应计算模块，提高系统性能。
池化单元的结构为2*2时，池化运算器包括2个存储待计算数据的先入先出存储器，此时只需加载数据到两个先入先出存储器中，当加载到第二行先入先出存储器时，即第二行至少有1个数据时，两行先入先出存储器开始并行输出，一个时钟周期输出两个数据，根据最大池化或平均池化选择进入如图5所示的2输入流水线计算模块；首先2个值通过加法树或比较树结构得到2个值的和或最大值，继而通过累加器或循环比较器用2个时钟周期得到最终结果。
请参阅图6，图6为本申请实施例提供的一种行人重识别装置的第二结构示意图。
本申请一些实施例提供的一种行人重识别装置，还可以包括：
与异构计算设备连接的数据库，用于存储人物信息，相应的，可以将行人计算结果与数据库进行匹配，得到目标行人的人物信息等。
具体应用场景中，为了便于用户获知目标行人的人物信息，本申请提供的一种行人重识别装置中还可以包括显示器，用于显示数据库中与行人计算结果对应的人物信息，行人计算结果也即异构计算设备对目标行人图像的计算结果。
具体应用场景中，为了便于异构计算设备进行行人识别，还可以借助已有的神经网络预先对行人图像进行目标检测得到检测结果，再基于检测结果确定目标行人图像，并将目标行人图像输入给异构计算设备进行处理，相应的，本申请提供的一种行人重识别装置中还可以包括：与异构计算设备连接的图形处理器，用于基于YOLO网络对预处理后的行人图像进 行目标检测，得到用于确定目标行人图像的检测结果，具体的，可以通过对检测结果进行人框预处理，比如经过图像提取、裁剪等来得到目标行人图像。
具体应用场景中，为了便于进行行人重识别，本申请提供的一种行人重识别装置中还可以包括：与图形处理器连接的多路摄像头，用于拍摄原始的行人图像，此时，对原始的行人图像进行解码及预处理，比如按照固定帧率对原始的行人图像进行opencv解码，并进行大小调整、像素调整后等，便可以得到预处理后的行人图像。
需要说明的是，本申请中卷积单元的结构可以根据实际需要确定，比如其结构可以如图7所示，此时卷积单元使用PE阵列，多个通道并行计算，结构中依次有残差、量化和激活模块，具体的，只需根据是否有残差和激活操作，选择通过相应的计算模块，便可以得到最终的计算结果。且在此过程中Feature缓存输出给PE阵列的同时，输出filter加载信号给filter缓存，以便按照确定的时序加载相对应的filter，保证PE阵列高效运行。此外，不难看出，卷积单元的结构简洁高效，减少了多个模块重复访存的同时，简化了信号交互控制，提高了整体计算效率。
请参阅图8，图8为本申请中行人重识别装置的使用示意图。为了便于理解本申请提供的行人重识别装置，现结合用于管控行人重识别装置的上位机，比如服务器等，来对本申请中行人重识别装置的使用过程进行描述，其可以包括以下步骤：
步骤S201：将多路摄像头、GPU计算卡、异构计算加速卡安装到工作站上，连接显示器输出；
步骤S202：上位机通过PCIE接口将滤波器参数写到片上URAM缓存；
步骤S203：上位机通过PCIE接口将量化参数写到片上URAM缓存；
步骤S204：上位机通过PCIE接口将待执行的指令序列写到片上URAM缓存；
步骤S205：多路摄像头根据实时性要求按固定帧率采集图像帧，得到原始的行人图像；
步骤S206：软件调用opencv图像处理函数对原始的行人图像进行解码和预处理，得到预处理后的行人图像；
步骤S207：软件将处理后图像送入GPU上的YOLO网络进行目标检测，得到检测结果；
步骤S208：软件对检测结果进行人框预处理，得到目标行人图像；
步骤S209：上位机通过PCIE接口将目标行人图像的图像特征数据写到片上URAM缓存；
步骤S210：上位机通过PCIE接口发命令给片上调度核开始执行；
步骤S211：调度核取指、译码，依次生成每层的控制和参数信号给卷积单元、池化单元等计算部件；
步骤S212：计算部件依次完成从缓存取数、计算和将结果写回缓存；
步骤S213：待网络中所有层计算完成后，得到一组图像的计算结果；
步骤S214：调度核生成PCIE中断信号给上位机，上位机收到后取回计算结果；
步骤S215：软件将计算结果在数据库进行查询匹配，在显示器上输出显示人物相关信息；
步骤S216：重复执行多路摄像头根据实时性要求按固定帧率采集图像帧及之后的步骤，直到行人重识别任务完成。
本申请还提供了一种行人重识别设备及非易失性可读存储介质，其均具有本申请一些实施例提供的一种行人重识别方法具有的对应效果。请参阅图9，图9为本申请实施例提供的一种行人重识别设备的结构示意图。
本申请一些实施例提供的一种行人重识别设备，包括存储器201和处理器202，存储器201中存储有计算机程序，处理器202执行计算机程序时实现如上任一实施例所描述行人重识别方法的步骤。
请参阅图10，本申请一些实施例提供的另一种行人重识别设备中还可以包括：与处理器202连接的输入端口203，用于传输外界输入的命令至处理器202；与处理器202连接的显示单元204，用于显示处理器202的处理结果至外界；与处理器202连接的通信模块205，用于实现行人重识别设备与外界的通信。显示单元204可以为显示面板、激光扫描使显示器等；通信模块205所采用的通信方式包括但不局限于移动高清链接技术(HML)、通用串行总线(USB)、高清多媒体接口(HDMI)、无线连接：无线保真技术(WiFi)、蓝牙通信技术、低功耗蓝牙通信技术、基于IEEE802.11s的通信技术。
本申请一些实施例提供的一种非易失性可读存储介质，非易失性可读存储介质中存储有计算机程序，计算机程序被处理器执行时实现如上任一实施例所描述行人重识别方法的步骤。
本申请所涉及的非易失性可读存储介质包括随机存储器(RAM)、内存、只读存储器(ROM)、电可编程ROM、电可擦除可编程ROM、寄存器、硬盘、可移动磁盘、CD-ROM、或技术领域内所公知的任意其它形式的存储介质。
需要说明的是，在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
对所公开的实施例的上述说明，使本领域技术人员能够实现或使用本申请。对这些实施例的多种修改对本领域技术人员来说将是显而易见的，本文中所定义的一般原理可以在不脱离本申请的精神或范围的情况下，在其它实施例中实现。因此，本申请将不会被限制于本文所示的这些实施例，而是要符合与本文所公开的原理和新颖特点相一致的最宽的范围。

Claims (20)

  1. 一种行人重识别方法，其特征在于，应用于异构计算设备，包括：
    获取图像特征、量化参数、指令序列和滤波系数；所述图像特征包括待识别的目标行人图像的图像特征；
    按照所述指令序列对所述图像特征进行池化处理，得到池化处理结果；
    按照所述指令序列并基于所述量化参数和所述滤波系数对所述图像特征进行卷积处理，得到卷积处理结果；
    基于所述池化处理结果和所述卷积处理结果确定所述目标行人图像对应的行人重识别结果。
  2. 根据权利要求1所述的方法，其特征在于，所述按照所述指令序列对所述图像特征进行池化处理，得到池化处理结果，包括：
    当池化格式为3*3时，基于所述指令序列，将所述图像特征对应的数据分开存入三个先入先出存储器中，其中，第三个所述先入先出存储器的输出端与第一个所述先入先出存储器的输入端相连接；
    对三个所述先入先出存储器中的数据进行池化处理，得到所述池化处理结果。
  3. 根据权利要求2所述的方法，其特征在于，第一个所述先入先出存储器的深度为所述目标行人图像的特征图的宽度，第二个及第三个所述先入先出存储器的深度为两倍的所述宽度。
  4. 根据权利要求2所述的方法，其特征在于，所述对三个所述先入先出存储器中的数据进行池化处理，得到所述池化处理结果的步骤，包括：
    基于所述先入先出存储器的不满标志依次读取外存进行加载，对三个所述先入先出存储器中的数据进行池化处理，得到所述池化处理结果。
  5. 根据权利要求1所述的方法，其特征在于，所述基于所述池化处理结果和所述卷积处理结果确定所述目标行人图像对应的行人重识别结果的步骤，包括：
    基于所述池化处理结果和所述卷积处理结果确定目标行人图像携带的行人特征；
    基于所述行人特征确定对应的行人重识别结果。
  6. 根据权利要求1至5任一项所述的方法，其特征在于，所述基于所述池化处理结果和所述卷积处理结果确定所述目标行人图像对应的行人重识别结果之后，还包括：
    传输所述行人重识别结果至预设数据库，以基于所述行人重识别结果及所述预设数据库中存储的人物信息确定目标行人信息。
  7. 根据权利要求6所述的方法，其特征在于，所述获取图像特征、量化参数、指令序列和滤波系数之后，所述按照所述指令序列对所述图像特征进行池化处理，得到池化处理结果之前，还包括：
    将所述图像特征、所述量化参数、所述指令序列和所述滤波系数存入存储器。
  8. 根据权利要求7所述的方法，其特征在于，所述将所述图像特征、所述量化参数、所述指令序列和所述滤波系数存入存储器，包括：
    将所述图像特征、所述量化参数、所述指令序列和所述滤波系数存入不同的存储器中。
  9. 一种行人重识别系统，其特征在于，应用于异构计算设备，包括：
    第一获取模块，用于获取图像特征、量化参数、指令序列和滤波系数，所述图像特征包括待识别的目标行人图像的图像特征；
    第一池化模块，用于按照所述指令序列对所述图像特征进行池化处理，得到池化处理结果；
    第一卷积模块，用于按照所述指令序列并基于所述量化参数和所述滤波系数对所述图像特征进行卷积处理，得到卷积处理结果；
    第一确定模块，用于基于所述池化处理结果和所述卷积处理结果确定所述目标行人图像对应的行人重识别结果。
  10. 一种行人重识别装置，其特征在于，包括异构计算设备；
    所述异构计算设备包括：PCIE控制器；与所述PCIE控制器连接的交叉开关矩阵；与所述交叉开关矩阵连接的调度核；与所述交叉开关矩阵连接的存储器；分别与所述调度核、所述开关交叉矩阵及所述存储器连接的池化运算器；分别与所述调度核、所述开关交叉矩阵及所述存储器连接的卷积运算器；
    所述PCIE控制器，用于通过所述交叉开关矩阵将图像特征、量化参数、指令序列及滤波系数存入所述存储器，所述图像特征包括待识别的目标行人图像的图像特征；
    所述调度核，用于基于所述指令序列控制所述池化运算器、所述卷积运算器的工作；
    所述池化运算器，用于对所述图像特征进行池化处理；
    所述卷积运算器，用于基于所述量化参数、所述滤波系数对所述图像特征进行卷积处理。
  11. 根据权利要求10所述的装置，其特征在于，所述滤波系数为1x1滤波器的参数或3x3滤波器的参数。
  12. 根据权利要求11所述的装置，其特征在于，所述PCIE控制器用512K的缓存容量缓存所述图像特征，用2MB的缓存容量缓存1x1滤波系数，用64KB的缓存空间缓存3x3滤波系数，用8K的缓存容量缓存指令序列，用1M的缓存容量缓存量化参数。
  13. 根据权利要求10所述的装置，其特征在于，所述池化运算器为3*3时，所述池化运算器包括三个存储待计算数据的先入先出存储器，第三个先入先出存储器的输出端与第一个先入先出存储器的输入端相连接，且第一个先入先出存储器的深度为目标行人图像的特征图的宽度，第二个及第三个先入先出存储器的深度为两倍的宽度。
  14. 根据权利要求13所述的装置，其特征在于，当启动所述池化运算器的计算后，按顺序读取特征图数据，依次加载到三个所述先入先出存储器，当加载到所述第三个先入先出存储器时，所述三个先入先出存储器开始并行输出，一个时钟周期输出三个数据，根据最大池化或平均池化选择进入流水线计算模块。
  15. 根据权利要求10所述的装置，其特征在于，所述池化运算器的结构为2*2时，所述池化运算器包括2个存储待计算数据的先入先出存储器。
  16. 根据权利要求15所述的装置，其特征在于，将特征图数据依次加载到两个所述先入先出存储器中，当加载到第二个先入先出存储器时，两个先入先出存储器开始并行输出，一个时钟周期输出两个数据，根据最大池化或平均池化选择进入流水线计算模块。
  17. 根据权利要求10所述的装置，其特征在于，所述行人重识别装置中还包括：与所述异构计算设备连接的图形处理器，用于基于YOLO网络对预处理后的行人图像进行目标检测，得到用于确定目标行人图像的检测结果。
  18. 根据权利要求10所述的装置，其特征在于，所述行人重识别装置中还包括：与图形处理器连接的多路摄像头，用于拍摄原始的行人图像，对所述原始的行人图像进行解码及预处理，得到预处理后的行人图像。
  19. 一种行人重识别设备，其特征在于，包括：
    存储器，用于存储计算机程序；
    处理器，用于执行所述计算机程序时实现如权利要求1至8任一项所述行人重识别方法的步骤。
  20. 一种非易失性可读存储介质，其特征在于，所述计算机可读存储介质中存储有计算机程序，所述计算机程序被处理器执行时实现如权利要求1至8任一项所述行人重识别方法的步骤。
PCT/CN2023/079895 2022-03-11 2023-03-06 一种行人重识别方法、系统、装置、设备及介质 WO2023169369A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210242524.6 2022-03-11
CN202210242524.6A CN114581952A (zh) 2022-03-11 2022-03-11 一种行人重识别方法、系统、装置、设备及计算机介质

Publications (1)

Publication Number Publication Date
WO2023169369A1 true WO2023169369A1 (zh) 2023-09-14

Family

ID=81779569

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/079895 WO2023169369A1 (zh) 2022-03-11 2023-03-06 一种行人重识别方法、系统、装置、设备及介质

Country Status (2)

Country Link
CN (1) CN114581952A (zh)
WO (1) WO2023169369A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114581952A (zh) * 2022-03-11 2022-06-03 浪潮(北京)电子信息产业有限公司 一种行人重识别方法、系统、装置、设备及计算机介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200342271A1 (en) * 2019-04-29 2020-10-29 Beijing Baidu Netcom Science And Technology Co., Ltd. Pedestrian re-identification method, computer device and readable medium
CN112149694A (zh) * 2020-08-28 2020-12-29 特斯联科技集团有限公司 一种基于卷积神经网络池化模块的图像处理方法、系统、存储介质及终端
CN112183295A (zh) * 2020-09-23 2021-01-05 上海眼控科技股份有限公司 行人重识别方法、装置、计算机设备及存储介质
CN112801008A (zh) * 2021-02-05 2021-05-14 电子科技大学中山学院 行人重识别方法、装置、电子设备及可读存储介质
CN113705329A (zh) * 2021-07-07 2021-11-26 浙江大华技术股份有限公司 重识别方法、目标重识别网络的训练方法及相关设备
CN114581952A (zh) * 2022-03-11 2022-06-03 浪潮(北京)电子信息产业有限公司 一种行人重识别方法、系统、装置、设备及计算机介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200342271A1 (en) * 2019-04-29 2020-10-29 Beijing Baidu Netcom Science And Technology Co., Ltd. Pedestrian re-identification method, computer device and readable medium
CN112149694A (zh) * 2020-08-28 2020-12-29 特斯联科技集团有限公司 一种基于卷积神经网络池化模块的图像处理方法、系统、存储介质及终端
CN112183295A (zh) * 2020-09-23 2021-01-05 上海眼控科技股份有限公司 行人重识别方法、装置、计算机设备及存储介质
CN112801008A (zh) * 2021-02-05 2021-05-14 电子科技大学中山学院 行人重识别方法、装置、电子设备及可读存储介质
CN113705329A (zh) * 2021-07-07 2021-11-26 浙江大华技术股份有限公司 重识别方法、目标重识别网络的训练方法及相关设备
CN114581952A (zh) * 2022-03-11 2022-06-03 浪潮(北京)电子信息产业有限公司 一种行人重识别方法、系统、装置、设备及计算机介质

Also Published As

Publication number Publication date
CN114581952A (zh) 2022-06-03

Similar Documents

Publication Publication Date Title
CN108416327B (zh) 一种目标检测方法、装置、计算机设备及可读存储介质
CN108509272B (zh) 将gpu显存纹理拷贝到系统内存的方法、装置及电子设备
WO2023169369A1 (zh) 一种行人重识别方法、系统、装置、设备及介质
CN110073653A (zh) 视频图像数据传输方法、系统、存储介质及程序产品
US11314457B2 (en) Data processing method for data format conversion, apparatus, device, and system, storage medium, and program product
CN109359556B (zh) 一种基于低功耗嵌入式平台的人脸检测方法及系统
US20220114120A1 (en) Image processing accelerator
CN111984189A (zh) 神经网络计算装置和数据读取、数据存储方法及相关设备
CN111738917A (zh) 一种图片伸缩方法、装置、设备和存储介质
CN111523652A (zh) 处理器及其数据处理方法、摄像装置
CN112614041A (zh) 稀疏渲染的数据驱动方法及装置、存储介质、电子装置
CN111428740A (zh) 网络翻拍照片的检测方法、装置、计算机设备及存储介质
CN113438474B (zh) 一种摄像头模组测试装置及其控制方法
CN112235598B (zh) 一种视频结构化处理方法、装置及终端设备
US20070076963A1 (en) Image transmission mechanism and method for implementing the same
US9019284B2 (en) Input output connector for accessing graphics fixed function units in a software-defined pipeline and a method of operating a pipeline
US20200202479A1 (en) Method and Apparatus for Processing a Video Frame
CN113079342A (zh) 基于高分辨率影像装置的目标追踪方法及系统
CN108021393B (zh) 一种计算方法及相关产品
WO2020107267A1 (zh) 一种图像特征点匹配方法及装置
WO2023273515A1 (zh) 目标检测方法、装置、电子设备和存储介质
CN113128673B (zh) 数据处理方法、存储介质、神经网络处理器及电子设备
WO2021031154A1 (zh) 神经网络的特征图装载方法及设备
Sulzbachner et al. An optimized silicon retina stereo matching algorithm using time-space correlation
CN113538237A (zh) 一种图像拼接系统、方法及电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23765947

Country of ref document: EP

Kind code of ref document: A1