CN113505766B

CN113505766B - Image target detection method and device, electronic equipment and storage medium

Info

Publication number: CN113505766B
Application number: CN202111057802.2A
Authority: CN
Inventors: 张羽; 马雷; 朱耀宇; 郑雅菁; 曹岗; 黄铁军
Original assignee: Beijing Zhiyuan Artificial Intelligence Research Institute
Current assignee: Beijing Zhiyuan Artificial Intelligence Research Institute
Priority date: 2021-09-09
Filing date: 2021-09-09
Publication date: 2022-01-04
Anticipated expiration: 2041-09-09
Also published as: CN113505766A

Abstract

The application discloses an image target detection method and device, electronic equipment and a storage medium. The image target detection method comprises the following steps: dividing a pulse array representing an image into a plurality of image blocks, the pulse array comprising zero and non-zero elements having row and column coordinates; respectively merging the adjacent nonzero elements in each row of each image block into a nonzero element block; establishing an adjacent edge by taking each non-zero element block as a node according to the relation of column coordinates between the non-zero element blocks of adjacent rows; and combining the non-zero element blocks connected through the adjacent edges to generate a target frame containing a target. The image target detection method has the advantages of high speed and high energy efficiency, the data volume can be greatly reduced by adopting a non-zero element merging method, the processing speed is accelerated through the circuit structure design of parallel processing, the energy consumption requirement is reduced, and the processing speed is further improved.

Description

Image target detection method and device, electronic equipment and storage medium

Technical Field

The present application relates to the field of image processing technologies, and in particular, to a method and an apparatus for detecting an image target for a sparse pulse image or a binary image, an electronic device, and a storage medium.

Background

High speed and low power consumption are important requirements for scene intelligent visual detection (such as aerospace, guidance and guidance, military reconnaissance and other scenes). The artificial neural network has high power consumption and large model, and is not suitable for the scenes. The brain-like neuron simulates the brain working mechanism, has the advantages of small model, low power consumption, high speed and the like, is expected to replace an artificial neuron, becomes the basis of third-generation artificial intelligence, and solves the problems of high computing power, power consumption, high cost and the like of the conventional artificial intelligence technology. From the aspect of method flow, the existing computer vision processing method has high computational complexity, large calculated amount, high power consumption of a processor depending on operation, and no realization of parallelization acceleration, and is difficult to meet the application requirements of tasks such as aerospace, military reconnaissance, scientific observation and the like.

Those skilled in the art are constantly engaged in optimizing computer vision processing methods to achieve the effects of increasing processing speed, reducing energy consumption, etc.

Disclosure of Invention

The application aims to provide an image target detection method and device, electronic equipment and a storage medium. The following presents a simplified summary in order to provide a basic understanding of some aspects of the disclosed embodiments. This summary is not an extensive overview and is intended to neither identify key/critical elements nor delineate the scope of such embodiments. Its sole purpose is to present some concepts in a simplified form as a prelude to the more detailed description that is presented later.

According to an aspect of an embodiment of the present application, there is provided an image target detection method, including:

dividing a pulse array representing an image into a plurality of image blocks, the pulse array comprising zero and non-zero elements having row and column coordinates;

respectively merging the adjacent nonzero elements in each row of each image block into a nonzero element block;

establishing an adjacent edge by taking each non-zero element block as a node according to the relation of column coordinates between the non-zero element blocks of adjacent rows;

and combining the non-zero element blocks connected through the adjacent edges to generate a target frame containing a target.

Further, the merging the adjacent non-zero elements in each row of each of the image blocks into one non-zero element block respectively includes:

recording the line number, the starting point and the end point of the non-zero element block;

the row number of the non-zero element block is the row coordinate of the adjacent non-zero element, the starting point is the column coordinate of the first non-zero element in the adjacent non-zero element, the end point is the column coordinate of the last non-zero element in the adjacent non-zero element, and the end point is larger than the starting point.

Further, the establishing an adjacent edge by using each non-zero element block as a node according to the relation of the column coordinates between the non-zero element blocks of adjacent rows includes:

searching two non-zero element blocks positioned in adjacent rows according to the row number of the non-zero element block;

judging whether the intervals of the column coordinates of the two non-zero element blocks are overlapped or not; the interval of the column coordinates of the non-zero element block is an interval taking the starting point of the non-zero element block and the end point of the non-zero element block as end points at two ends;

if so, establishing an adjacent edge between the two non-zero element blocks;

otherwise, no adjacent edge is established between the two non-zero element blocks.

Further, the two non-zero element blocks include a first non-zero element block and a second non-zero element block, and determining whether there is an overlap between intervals of column coordinates of the two non-zero element blocks includes:

judging whether the end point of the first non-zero element block is greater than or equal to the start point of the second non-zero element block, and the end point of the second non-zero element block is greater than or equal to the start point of the first non-zero element block;

if yes, judging that the superposition exists;

if not, it is determined that there is no overlap.

According to another aspect of the embodiments of the present application, there is provided an image object detecting apparatus including:

a dividing module for dividing a pulse array representing an image into a plurality of image blocks, the pulse array comprising zero elements and non-zero elements having row and column coordinates;

a nonzero element merging module, configured to merge immediately adjacent nonzero elements in each row of each image block into a nonzero element block;

the establishing module is used for establishing an adjacent edge by taking each non-zero element block as a node according to the relation of the column coordinates between the non-zero element blocks of adjacent rows;

and the node merging module is used for merging the nonzero element blocks connected through the adjacent edges to generate a target frame containing a target.

Further, the row coordinate includes a row number, the column coordinate includes a start point and an end point of a non-zero element block, the number of the non-zero element merging modules is multiple, each non-zero element merging module includes a pulse array buffer, a first group of registers, a second group of registers, a first control logic submodule, a non-zero element block memory, a non-zero element merging circuit, a column address counter, a row address counter and a non-zero element block count memory of each row, and the first group of registers and the second group of registers respectively include one or more registers;

the pulse array buffer is used for storing one image block;

the first control logic sub-module is used for reading data of the pulse array buffer along the column direction of the pulse array and storing the data into the first group of registers and the second group of registers which are connected in sequence according to a pipeline mode;

the non-zero element merging circuit is used for storing the identification, the line number and the starting point of the non-zero element block into the non-zero element block memory when the first group of registers are at a high level and the second group of registers are at a low level; storing an end point of the non-zero element block in the non-zero element block memory when the first set of registers is low and the second set of registers is high;

the first control logic sub-module is further configured to control the column address counter to increment by one each time one datum is read; and writing the information in the row address counter and the number of the non-zero element blocks of the row into the non-zero element block counting memory every time the numerical value of the column address counter reaches a threshold value, and controlling the row address counter to increase by one.

Further, the establishing module comprises a judging circuit and an adjacent edge generating circuit; the judging circuit is used for judging whether the condition for generating the adjacent edge is met or not, the adjacent edge generating circuit is used for generating the adjacent edge when the condition for generating the adjacent edge is met, and the adjacent edge is established by taking each non-zero element block as a node.

Further, the discrimination circuit comprises a second control logic submodule, a third register, a fourth register, a first identifier generator, a second identifier generator, a fifth register, a sixth register, a seventh register, an eighth register, a first comparator and a second comparator;

the second control logic sub-module is configured to read the number of non-zero element blocks in two adjacent rows from the non-zero element block count memory, and send the number of non-zero element blocks to the third register and the fourth register respectively;

the first identifier generator and the second identifier generator respectively generate identifiers of two non-zero element blocks to be distinguished according to information stored in the third register and the fourth register, and store the identifiers in the fifth register and the sixth register;

the second control logic sub-module is further configured to read the non-zero element block memory according to the identifiers stored in the fifth register and the sixth register to obtain parameters of a non-zero element block to be determined, and store the parameters in the seventh register and the eighth register, respectively, where the parameters of the non-zero element block include column coordinates of the non-zero element block;

the first comparator and the second comparator are used for judging whether the intervals of the column coordinates of the two non-zero element blocks are overlapped according to the parameters of the non-zero element blocks stored in the seventh register and the eighth register, and if the intervals of the column coordinates of the two non-zero element blocks are overlapped, a starting signal for starting the adjacent edge generating circuit is generated; the interval of the column coordinates of the non-zero element block is an interval taking the starting point of the non-zero element block and the end point of the non-zero element block as end points at two ends.

Furthermore, the adjacent edge generating circuit comprises a third control logic submodule, a vertex buffer, an edge buffer, a starting point buffer, a ninth register, a tenth register, an eleventh register, a twelfth register and a thirteenth register;

every time the judging circuit generates a starting signal, the value of the tenth register is added with one on the basis of the value of the ninth register, and then the value of the ninth register is added with two;

the eleventh register and the twelfth register are used for buffering starting signals so as to enable reading and writing of the vertex buffer, the edge buffer and the starting point buffer;

the third control logic sub-module is configured to, when the eleventh register caches the start signal, write the value of the sixth register into the vertex buffer by using the value of the ninth register as a write address, read the signal in the start buffer by using the value of the fifth register as a read address and write the signal into the edge buffer by using the value of the ninth register as a write address, and write the value of the ninth register into the start buffer by using the value of the fifth register as a write address; and when the twelfth register caches the start signal, writing the value of the fifth register into the vertex buffer by taking the value of the tenth register as a write address, reading the signal in the start buffer by taking the value of the sixth register as a read address, writing the signal into the edge buffer by taking the value of the tenth buffer as a write address, and writing the value of the tenth register into the start buffer by taking the value of the sixth register as a write address.

Further, the node merging module comprises a fourth control logic submodule, a node queue buffer, a new node register, a merging submodule and a merging node buffer;

the fourth control logic sub-module is used for reading parameters of one node from the non-zero element block memory, wherein the parameters comprise an identification, a line number, a starting point and an end point;

retrieving the starting point buffer, the vertex buffer and the edge buffer according to the identifier of the node, acquiring the identifier of another node and storing the identifier of the other node in the node queue buffer; acquiring the parameters of the other node from a non-zero element block memory according to the identifier of the other node, and storing the parameters into the new node register;

and the merging submodule merges the nodes in the new node register and the read nodes and writes parameters of the merged nodes into the merged node buffer, or merges the nodes in the new node register and the nodes in the merged node buffer and updates the merged node buffer by using the parameters of the merged nodes.

According to another aspect of the embodiments of the present application, there is provided an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to implement any one of the image target detection methods described above.

According to another aspect of embodiments of the present application, there is provided a computer-readable storage medium having stored thereon a computer program, which is executed by a processor, to implement the image object detection method of any one of the above.

The technical scheme provided by one aspect of the embodiment of the application can have the following beneficial effects:

the image target detection method provided by the embodiment of the application has the advantages of high speed and high energy efficiency, and the data volume can be greatly reduced by adopting the nonzero element combination method, and repeated reading, judgment and other operations of meaningless elements are avoided, so that the processing speed is accelerated, the energy consumption requirement is reduced, and the processing speed is further improved.

Additional features and advantages of the application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the embodiments of the application, or may be learned by the practice of the embodiments. The objectives and other advantages of the application may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the present application, and other drawings can be obtained by those skilled in the art without creative efforts.

FIG. 1 shows a flow diagram of an image target detection method of one embodiment of the present application;

FIG. 2 illustrates a parallel processing flow diagram for non-zero element merging in one embodiment of the present application;

FIG. 3 shows a flow diagram of an image target detection method of another embodiment of the present application;

FIG. 4 illustrates a non-zero element merging diagram in one embodiment of the present application;

FIG. 5(a) shows a schematic diagram of two rows of non-zero element blocks in one embodiment of the present application;

FIG. 5(b) shows a graphical model diagram formed by the non-zero element blocks of FIG. 5 (a);

FIG. 6 is a block diagram of an image target detection apparatus according to an embodiment of the present application;

FIG. 7 illustrates a node generation circuit diagram of one embodiment of the present application;

FIG. 8 illustrates a neighboring edge discrimination circuit diagram according to an embodiment of the present application;

FIG. 9 illustrates a adjacency edge generation circuit diagram of one embodiment of the present application;

FIG. 10 illustrates a neighbor node merge circuit diagram for one embodiment of the present application;

FIG. 11 shows a block diagram of an electronic device of an embodiment of the present application;

FIG. 12 shows a computer-readable storage medium schematic of an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is further described with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

It will be understood by those within the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

In images acquired in certain application scenarios, such as aerospace, guidance and guidance, or military reconnaissance, the effective targets (real targets) are fewer in the pulse array and contain less noise. The pulse data can be processed in a blocking and parallel mode, and therefore the moving target can be detected quickly.

Based on this, as shown in fig. 1, an embodiment of the present application proposes an image target detection method, including:

s10, dividing the pulse array representing the image into a plurality of image blocks.

Wherein the pulse array includes zero and non-zero elements having row and column coordinates. Specifically, the pulse array is an array composed of 0 and 1, where when the pulse vision type sensor collects light, a pulse is represented by 1 if the pulse is generated, and a pulse is represented by 0 if the pulse is not generated, thereby configuring the pulse array.

Each divided image block comprises a part of the pulse array, optionally, each image block comprises one or more rows of the pulse array. In the example shown in fig. 2, after the pulse array enters the detection system, the pulse array is divided into 25 image blocks, and the image blocks are stored in 25 image block buffers respectively.

And S20, respectively merging the adjacent nonzero elements in each row of each image block into a nonzero element block.

And after the combined non-zero element blocks, namely nodes are combined into the non-zero element blocks, the information of the non-zero element blocks, such as identity, row coordinates, column coordinates, area and the like, is recorded. The row coordinates include a row number and the column coordinates include a start point and an end point of a non-zero element block.

Optionally, the merging operations described above are performed in parallel. As illustrated in fig. 2, each image block buffer is connected to one node generation unit and one buffer unit, so as to perform non-zero element merging on 25 image blocks in parallel, and store the merged node information in the corresponding buffer unit. Types of cache memory units include, but are not limited to, on-chip memory (RAM), off-chip static memory (SRAM), and off-chip dynamic memory (DRAM), among others.

In some embodiments, the input pulse array has a large number of elemental values labeled "0" (i.e., the corresponding region is not part of the moving object), few non-zero elements, and some spatial continuity (since the size of the moving object is typically greater than 1 pixel). Based on this feature, the present embodiment proposes a non-zero element merging method, which merges, for a row of a pulse array, immediately adjacent non-zero elements into a set (i.e., a non-zero element block), and records five parameter data of a row number, a start point, an end point, an identity (which may also be referred to as a "sequence number") and an area of the set. Taking fig. 4 as an example, the line number of the first block is "1", the starting point is "2", the end point is "3", the serial number is "1", and the area is "2"; the line number of the second block (the part of three continuous '1') is '1', the starting point is '7', the end point is '9', the serial number is '2', and the area is '3'; the line number of the fourth block is "1", the starting point is "17", the end point is "18", the area is "2", and the serial number is "4"; and so on. The optimization method reduces the storage space of effective information and is beneficial to reducing the storage resource consumption of a hardware circuit. The original pulse target search algorithm combines non-zero element areas through 8 neighborhood search, so that a target area is obtained. The nonzero element combination method avoids element-by-element processing of a large number of nonzero elements, and converts the element-by-element processing into processing of a nonzero area (namely a target component node), so that the calculation complexity of subsequent processing can be reduced.

And S30, establishing an adjacent edge by taking each non-zero element block as a node according to the relation of the column coordinates between the non-zero element blocks of adjacent rows.

In certain embodiments, comprising:

searching two non-zero element blocks positioned in adjacent rows according to the row coordinates of the non-zero element blocks;

if so, establishing an adjacent edge between the two non-zero element blocks;

In some embodiments, the two non-zero element blocks include a first non-zero element block and a second non-zero element block, and determining whether there is an overlap between intervals of column coordinates of the two non-zero element blocks includes:

if yes, judging that the superposition exists;

if not, it is determined that there is no overlap.

In some embodiments, the non-zero element block is obtained after merging of adjacent non-zero elements of each row. According to the method, the adjacency graph model is established through a breadth-first search algorithm, and the adjacency edges are generated. The vertices (i.e., nodes) of the adjacency graph model are the merged non-zero element blocks described above. And if the interval of the column coordinates of the non-zero element blocks between the adjacent rows is overlapped, establishing an adjacent edge, otherwise, not establishing the adjacent edge. Fig. 4 is an example of this method. In FIG. 5(a), r1 overlaps the interval of the column coordinates of r4, thus creating an abutting edge, as shown in FIG. 5 (b). Similarly, in fig. 5(a), r2 overlaps with the interval of the column coordinate of r5, r5 overlaps with the interval of the column coordinate of r3, and r3 overlaps with the interval of the column coordinate of r6, so that adjacent edges are respectively established, as shown in the adjacent graph model shown in fig. 5 (b). And there are no overlapping interval portions of column coordinates between r1 and r5, r6, r2 and r4, r6, r3 and r4, and therefore there are no adjoining sides. Optionally, for convenience of circuit implementation, the adjacent edges have directivity, and two adjacent edges are established between each pair of nodes, for example, two adjacent edges are established between r1 and r4, one is r1 — > r4, and the other is r4 — > r 1.

And S40, combining the nonzero element blocks connected through the adjacent edges to generate a target frame containing a target.

The position of the object can be estimated from the adjacent edges. After the adjacency graph model is established, the nonzero element blocks connected by the adjacency edges are merged, and the target area can be obtained. For example, after merging by adjacent edges, fig. 5(b) has two targets in common, one being a target made of r1 and r4, and the other being a target made of r2, r3, r5, and r 6.

Compared with the prior art, the method provided by the embodiment of the application has the advantages of high speed and high energy efficiency. The non-zero element merging method provided by the scheme can greatly reduce the data volume under the sparse condition, avoids repeated reading and judgment and other operations on meaningless elements, accelerates the processing speed and also reduces the energy consumption requirement. In addition, the method of the embodiment of the application adopts a parallel computing method of the non-zero element block, and further improves the processing speed.

Fig. 3 shows a flow of an image target detection method according to another embodiment of the present application. The pulse array generated for the image containing the letter "PKU" is divided into a plurality of image blocks to be respectively cached, non-zero element combination is performed in parallel for each image block to obtain a plurality of non-zero element blocks (namely target component nodes), information such as positions, areas, identity identifications and the like of the non-zero element blocks is cached, then edges (short for adjacent edges) of adjacent nodes are generated and stored through width-first search, nodes connected with the adjacent edges are combined to generate a target frame, and information of the detected target can be obtained, as shown in fig. 3, the letters "P", "K" and "U" are respectively framed out.

Referring to fig. 6, another embodiment of the present application provides an image object detecting apparatus including:

a dividing module 601 for dividing a pulse array representing an image into a plurality of image blocks, the pulse array comprising zero elements and non-zero elements having row coordinates and column coordinates;

a nonzero element merging module 602, configured to merge immediately adjacent nonzero elements in each row of each image block into a nonzero element block;

an establishing module 603, configured to establish an adjacent edge with each non-zero element block as a node according to a relation of column coordinates between the non-zero element blocks in adjacent rows;

and a node merging module 604, configured to merge the non-zero element blocks connected by the adjacent edges to generate a target frame including a target.

In some embodiments, the row coordinate includes a row number, the column coordinate includes a start point and an end point of a non-zero element block, the non-zero element merging module is plural, each non-zero element merging module includes a pulse array buffer, a first group of registers, a second group of registers, a first control logic submodule, a non-zero element block memory, a non-zero element merging circuit, a column address counter, a row address counter, and a non-zero element block count memory per row, the first group of registers and the second group of registers include one or more registers, respectively;

the pulse array buffer is used for storing one image block;

Then the value of the column address counter returns to the initial value, and the above processing is continued until all the data of the pulse array buffer is processed.

In some embodiments, the non-zero element merge module 602 is the node generation circuit shown in fig. 7, and the node generation circuit includes a pulse array buffer 701, a first control logic submodule 702, a first set of registers FF _ D1, a second set of registers FF _ D2, a non-zero element merge circuit 703, a column address counter Count _ w, a row address counter Count _ h, a column address total memory 705, an equal comparator 704, a non-zero element block memory 706, and a non-zero element block Count memory 707.

As an example, the non-zero element merging circuit 703 includes a register FF _ end, a register FF _ start, and a counter, and the input terminal of the not gate 1 is connected to the FF _ D1, and the output terminal of the FF _ D2 is connected to the input terminals of the registers FF _ start and FF _ end through the not gate and the and gate. The output end of the register FF _ start is connected with the counter and the register for recording node _ h, node _ minw and node _ id, and the output end of the register FF _ end is connected with the register for recording node _ maxw and node _ area.

The first control logic sub-module 702 acts on the other sub-modules. For clarity of drawing, connection lines between the control logic circuit and other sub-modules are omitted. The working process of the non-zero element merging module 602 is as follows: the pulse array buffer 701 stores pulse data of one image block, for example, 10 rows and 400 columns, and each data (0 or 1) corresponds to 1 bit position. The initial value of the column address counter Count _ w is 1, the initial value of the row address counter Count _ h is the row number of the first row of the pulse array of the image block, and the initial value of the node counter is 0. Column address total memory 705 is used to store the total number of columns in the burst array, e.g., 400.

The control logic circuit reads the data of the pulse array buffer and stores the data into the registers FF _ D1 and FF _ D2 according to a pipeline mode, if the data is 1, the register is at a high level, and if the data is 0, the register is at a low level. When the FF _ D1 is at a high level and the FF _ D2 is at a low level, the non-zero element is shown to appear, a non-zero element merging circuit is started, a line number (node _ h), a column coordinate (node _ minw) and an identity (node _ id) at the moment are recorded and stored in a non-zero element block memory, the line number is the numerical value of a line address counter Count _ h, and the identity is the numerical value of a node counter. When FF _ D1 is low and FF _ D2 is high, indicating that a non-zero element block is finished, the column coordinate (node _ maxw) and the area (node _ area) at this time are recorded, and since the processing is performed by row, the area of the non-zero element at this time is the length of the column coordinate of the non-zero element block, that is, the difference between the two column coordinates node _ maxw and node _ minw, is input to the non-zero element block memory, and the node counter is incremented, and then the non-zero element merging circuit 703 is turned off. Thus, a merged non-zero element block is obtained, whose identity is node _ id, line number is node _ h, starting point is node _ minw, ending point is node _ maxw, and area is node _ area. When both FF _ D1 and FF _ D2 are high or both are low, non-zero element merge circuit 703 is not enabled.

The first control logic sub-module 702 reads the pulse data of each row along the column direction (i.e. reading row by row), and the column address counter Count _ w is incremented by one every time one bit is read. The equal comparator 704 compares the values of the column address counter Count _ w and the column address total memory 705, and when the value of the column address counter Count _ w is equal to the value of the column address total memory 705, it indicates that a row of data is processed, and the number of the row of non-zero element blocks can be obtained according to the counter value, and the control logic writes the row number in the row address counter Count _ h and the number of the row of non-zero element blocks into the non-zero element block Count memory. Where the row address counter is the address signal, the number of non-zero element blocks is the data signal, and the result of the comparison of Count _ w with the column address total memory 705 is the enable signal. When a row of pulse data is processed, the value of the row address counter is incremented, the value of the column address counter returns to the initial value, and the data processing of the next row is continued until all the data in the pulse array buffer 701 is processed. It is understood that each image block is processed in parallel by using the same circuit as described above, and the processing results are stored in the corresponding non-zero element block memory and non-zero element block count memory, respectively.

Optionally, the first control logic sub-module 702 is further configured to, after generating all non-zero element blocks, adjust the identities of all nodes in a traversal manner such that the identities of all nodes continuously change.

It should be noted that the circuit of fig. 7 is merely one implementation of a circuit architecture in which the non-zero elements generate the target component nodes. In practical applications, in order to filter noise points of small blocks, the FF _ D1 and the FF _ D2 respectively include a plurality of registers connected in series, and the output levels of the FF _ D1 and the FF _ D2 are determined based on the levels of the registers in which a preset ratio is set, to enable the non-zero element combining circuit, so that a plurality of pulse data, not one pulse data, can be used as a judgment condition. For example, the FF _ D1 and the FF _ D2 respectively include 3 registers connected in sequence, and output is performed according to the level of at least 2 registers, and if 2 registers of the 3 registers of the FF _ D1 are at high level and 1 register is at low level, the output of the FF _ D1 is at high level, so that the influence of noise of a single pixel point can be avoided.

In some embodiments, the establishing module comprises a discrimination circuit and an adjacent edge generating circuit; the judging circuit is used for judging whether the condition for generating the adjacent edge is met or not, the adjacent edge generating circuit is used for generating the adjacent edge when the condition for generating the adjacent edge is met, and the adjacent edge is established by taking each non-zero element block as a node.

In some embodiments, the discrimination circuit includes a second control logic submodule, a third register, a fourth register, a first identifier generator, a second identifier generator, a fifth register, a sixth register, a seventh register, an eighth register, a first comparator, and a second comparator;

the second control logic sub-module is configured to read the number of non-zero element blocks in two adjacent rows from the non-zero element block count memory, and send the number of non-zero element blocks to the third register and the fourth register respectively; the number of the non-zero element blocks of the upper line in the two adjacent lines is sent to a third register, and the number of the non-zero element blocks of the lower line is sent to a fourth register;

the first identification generator and the second identification generator respectively generate identifications of two non-zero element blocks to be distinguished according to the information in the third register and the fourth register, and store the identifications in the fifth register and the sixth register; the first identification generator generates an identification of a non-zero element block of a previous row according to information in the third register and stores the identification in the fifth register, and the second identification generator generates an identification of a non-zero element block of a next row according to information in the fourth register and stores the identification in the sixth register;

the second control logic sub-module is further configured to read the non-zero element block memory according to the identifiers stored in the fifth register and the sixth register to obtain parameters of a non-zero element block to be determined, and store the parameters in the seventh register and the eighth register, respectively, where the parameters of the non-zero element block include column coordinates of the non-zero element block; the seventh register stores the parameters of the non-zero element block of the previous row, and the eighth register stores the parameters of the non-zero element block of the next row;

Optionally, the discrimination circuit further includes a third comparator, configured to compare parameters of the two non-zero element blocks stored in the seventh register and the eighth register, and input a comparison result to the first identifier generator and the second identifier generator. And the first identifier generator and the second identifier generator respectively generate identifiers of two non-zero element blocks to be distinguished according to the line numbers of two adjacent lines, the information in the third register and the fourth register and the comparison result of the third comparator, and store the identifiers in the fifth register and the sixth register.

As shown in fig. 8, the second control logic sub-module 801 of the discrimination circuit reads the number of non-zero element blocks of two adjacent rows from the non-zero element block count memory 707, sequentially feeds the number into the fourth register FF _ RN1 and the third register FF _ RN2, the first flag generator 802 generates a flag of the first non-zero element block in the previous row based on the row number of the previous row and information in the FF _ RN2, stores the flag into the fifth register ID _ UP, the second flag generator 803 generates a flag of the first non-zero element block in the next row based on the row number of the next row and information in the FF _ RN1, stores the sixth register ID _ DN, for example, the number of non-zero element blocks of row 1 is 3, the number of non-zero element blocks of row 2 is 2, the first non-zero element block of row 1 is identified as 1 in register ID _ UP, the first non-zero element block of row 2 is identified as 4 in register ID _ DN.

The second control logic sub-module 801 reads the non-zero element block memory according to the identifier in the registers ID _ UP and ID _ DN to obtain the information of the start point, end point, area, etc. of the corresponding non-zero element block, and stores the information in the registers N _ UP and N _ DN, respectively. The key of the generation of the adjacent edge is to determine whether the column coordinates of the two nodes overlap, and in this embodiment, two comparators are used to determine this condition. One of the inputs of the comparator 804 is a column coordinate maximum value of N _ UP (N _ UP _ MAXW) and a column coordinate minimum value of N _ DN (N _ DN _ MINW); the inputs of the other comparator 805 are the maximum value of column coordinates of N _ DN (N _ DN _ MAXW) and the minimum value of column coordinates of N _ UP (N _ UP _ MINW). The E _ ST signal is pulled high to enable the adjacent edge generating circuit only when the conditions of N _ UP _ MAXW ≧ N _ DN _ MINW and N _ DN _ MAXW ≧ N _ UP _ MINW are simultaneously satisfied.

Subsequently, the comparator 806 compares N _ UP _ MAXW and N _ DN _ MAXW, and inputs the comparison result to the first and second flag generators 802 and 803 to obtain the flag numbers of the next pair of non-zero nodes. For example, when N _ DN _ MAXW is larger than or equal to N _ UP _ MAXW, the value of ID _ UP is added by one, the value of ID _ DN is unchanged, and the non-zero element block of the previous row is read again and stored into N _ UP; and when the N _ UP _ MAXW is larger than or equal to the N _ DN _ MAXW, adding one to the ID _ DN, keeping the ID _ UP unchanged, and reading the next row of non-zero element blocks again and storing the non-zero element blocks in the N _ DN. Therefore, the non-zero element blocks in two adjacent rows can be compared one by one until the judgment of all the non-zero element blocks in the two rows is completed.

And when the two adjacent rows are processed, storing the value of the next row in FF _ RN2, taking out the number of the non-zero element blocks of the next row, storing the number in FF _ RN1, and continuously judging the non-zero element blocks of the next two adjacent rows until the processing of all the adjacent rows of the pulse array is completed. It is understood that the discrimination circuit is connected to the non-zero element block count memory and the non-zero element block memory corresponding to each image block, and is capable of handling discrimination of nodes divided into adjacent rows of different image blocks.

The adjacent edge generating circuit comprises a third control logic submodule 901, a vertex buffer E _ TO, an edge buffer E _ NX, a start buffer E _ Head, a ninth register E _ CNT1, a tenth register E _ CNT2, an eleventh register E _ EN1 and a twelfth register E _ EN 2;

the value of the tenth register E _ CNT2 is incremented by one on the basis of the value of the ninth register and then incremented by two on the basis of the value of the ninth register E _ CNT1 each time the enable signal E _ ST generated by the decision circuit is received. The initial values in the start buffer E _ Head and the edge buffer E _ NX are both 0.

When the eleventh register E _ EN1 buffers the enable signal, the third control logic sub-module 901 writes the value of the sixth register ID _ DN into the vertex buffer E _ TO using the value of the ninth register E _ CNT1 as a write address, reads the signal in the start buffer E _ Head using the value of the fifth register ID _ UP as a read address, writes the signal into the edge buffer E _ NX using the value of the ninth register E _ CNT1 as a write address, and writes the value of the ninth register E _ CNT1 into the start buffer E _ Head using the value of the fifth register ID _ UP as a write address; when the twelfth register E _ EN2 buffers the enable signal, a value of the fifth register ID _ UP is written into the vertex buffer E _ TO using a value of the tenth register E _ CNT2 as a write address, a signal in the start buffer E _ Head is read using a value of the sixth register ID _ DN as a read address and written into the side buffer E _ NX using a value of the tenth register E _ CNT2 as a write address, and then a value of the tenth register E _ CNT2 is written into the start buffer E _ Head using a value of the sixth register ID _ DN as a write address.

As shown in fig. 9. The sub-module mainly comprises three buffers: vertex buffer (E _ TO) adjacent TO the edge, next edge buffer (E _ NX) adjacent TO the edge and start buffer (E _ Head) of the spanning tree model adjacent TO the edge. The initial value of the register E _ CNT1 is-1, and when the E _ ST signal is asserted, the register E _ CNT1 adds "1" to the E _ CNT2 register, and then the E _ CNT1 adds 2 by itself. The E _ EN1 buffers the E _ ST signal for enabling reading and writing of three buffers (E _ Head, E _ NX, E _ TO). Wherein, writing E _ TO is ID _ DN signal, and writing address is E _ CNT1 signal; e _ NX writes a signal read from E _ Head, writes E _ NX at E _ CNT1, and reads E _ Head at ID _ UP; e _ CNT1 is then written into the E _ Head buffer at the ID _ UP address. Similar TO E _ EN1, E _ EN2 is also the write enable signals for E _ Head, E _ NX, and E _ TO. For ease of illustration, fig. 9 depicts these three memories twice. E _ EN2 writes ID _ UP TO the address indicated by E _ CNT2 in E _ TO; reading the address data indicated by the ID _ DN in the E _ Head buffer and writing the address data into the address indicated by the E _ CNT2 in the E _ NX; finally, the E _ CNT2 is written to the E _ Head at the address indicated by the ID _ DN. And repeating the steps until all the adjacent edges are generated.

In some embodiments, the node merge module includes a fourth control logic submodule, a node queue cache, a new node register, a merge submodule, and a merge node cache.

retrieving the starting point buffer, the vertex buffer and the edge buffer according to the identifier of the node, acquiring the identifier of another node and storing the identifier of the other node in the node queue buffer;

acquiring the parameters of the other node from a non-zero element block memory according to the identifier of the other node, and storing the parameters into the new node register;

Optionally, the module includes a node counter that increments by one each time a node is read from the non-zero element block memory.

The fourth control logic sub-module is further configured to, when the identifier of the node is the same as the value of the node counter, retrieve the start-point buffer, the vertex buffer, and the edge buffer according to the identifier of the node, obtain an identifier of another node and store the identifier in the node queue buffer, obtain a parameter of the another node from the non-zero element block memory according to the identifier of the another node, store the parameter in the new node register, and modify the identifier of the another node in the non-zero element block memory to the current value of the node counter.

Fig. 10 shows a specific embodiment of the node merging module, and after all the adjacent edges are generated, a bounding box (bounding box) of the area where the target is located is generated by the adjacent node merging module shown in fig. 10. The fourth control logic sub-module 1001 reads a parameter of a node from the non-zero element block memory 706 and sends the parameter TO the node register node, where the parameter of the node includes information such as an identifier, a start point, an end point, and a size, and if a value in the node _ num is equal TO the node identifier, which indicates that the node has not been merged, the node identifier is used as an address TO read the start point buffer E _ Head, the value read from the E _ Head is used as an address TO read data of the edge buffer E _ TO (i.e., an identifier of another node having an adjacent edge), and the data is written into the node queue buffer 1002, and the value is used as an address TO read the non-zero element block memory 706, so as TO obtain a parameter of another node, and the parameter is written into the new node buffer new _ node. And judging whether the identifier in the new _ node is equal to the identifier in the node, if not, updating the identifier of the node corresponding to the identifier in the new _ node in the non-zero element block memory 706 to the identifier in the node, namely unifying the identifiers of the two merged nodes to be the same identifier. And simultaneously, the merging submodule executes merging operation, frames (bounding boxes) of two nodes in the new _ node and the node are merged into a whole, and parameters of the merged node comprise an identifier, a minimum row number, a maximum row number, a minimum column coordinate, a maximum column coordinate and the like of the merged node and are stored in the merged _ node buffer.

Meanwhile, the value read from E _ Head is used as an address, the value of E _ NX is read, and it is determined whether or not "0". If the value is '0', the end point of the adjacent edge is reached, and the loop is ended; if the value is not 0, the value is taken as an address, E _ TO and E _ NX are continuously read, data of E _ TO is written into the node queue buffer 1002, the non-zero element block memory 706 is read by taking the value as the address, parameters of another node are obtained, the parameters are written into a new node buffer new _ node, and the identifier of the node corresponding TO the identifier in the new _ node in the non-zero element block memory 706 is updated TO the identifier in the node, that is, the identifiers of the three nodes which are combined are unified into the same identifier. And simultaneously executing a merging operation by the merging submodule, merging the borders (bounding boxes) of the two nodes in the new _ node buffer and the merged _ node buffer into a whole, and storing the merged box in the merged _ node buffer. That is, when a new node is read into the new _ node buffer, the new _ node buffer is merged with the node in the merged _ node buffer (instead of the node buffer). In the case where the value read by E _ NX is not 0, the above steps are repeated, thereby completing the merging of all nodes connected by adjacent edges until the loop exits.

The process of the merging operation is as follows: acquiring a minimum column coordinate (recorded as nn _ xmin) and a maximum column coordinate (recorded as nn _ xmax), a minimum row number (recorded as nn _ ymin) and a maximum row number (recorded as nn _ ymax) of a node to be merged; and acquiring a minimum column coordinate (recorded as nr _ xmin) and a maximum column coordinate (recorded as nr _ xmax), a minimum row number (recorded as nr _ ymin) and a maximum row number (recorded as nr _ ymax) of another node to be merged. The minimum value of the column coordinate of the merged node is the minimum value of nn _ xmin and nr _ xmin, and the maximum value of the column coordinate is the maximum value of nn _ xmax and nr _ xmax; the minimum value of the row number is the minimum value of nn _ ymin and nr _ ymin, and the maximum value of the row number is the maximum value of nn _ ymax and nr _ ymax, so that the framed area is the target area.

When the value read out by E _ NX is 0 and all the identifiers in the node queue buffer 1002 are processed, the fourth control logic sub-module 1001 writes the value of the merged _ node buffer into the inspection target memory 1003 to obtain an inspection target. At this time, node _ num is incremented by one, and then the next node is read from the non-zero element block memory 706 by using node _ num as the address, and the above processing is continued. For the nodes which have already been processed, as described above, since the identifiers thereof have been unified into smaller identifiers, the value in node _ num is not equal to the node identifier, and no processing will be performed, and node _ num continues to read the next node since adding one. Thus, after traversing all the non-zero element blocks, all the detected target areas are obtained.

It should be noted that, the operation of reading the node identifiers from E _ Head, E _ T0 and E _ NX and the merge operation may be processed in parallel, the read node identifiers are sequentially stored in the node queue buffer 1002, and then the fourth control logic sub-module 1001 sequentially reads the node identifiers from the node queue buffer 1002 to perform the merge operation, thereby further improving the processing efficiency.

According to the image target detection device provided by the embodiment of the application, the circuit expands the steps executed in series by software into the parallel circuit, the operation is convenient in a pipeline mode, the processing efficiency is improved, the circuit does not need an operating system, and does not need to adopt numerous functional components such as a general processor, and the like, so that the effect of low energy consumption is realized.

Another embodiment of the present application provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the image target detection method of any one of the above embodiments. As shown in fig. 11, the electronic device 10 may include: the system comprises a processor 100, a memory 101, a bus 102 and a communication interface 103, wherein the processor 100, the communication interface 103 and the memory 101 are connected through the bus 102; the memory 101 stores a computer program that can be executed on the processor 100, and the processor 100 executes the computer program to perform the method provided by any of the foregoing embodiments of the present application.

The Memory 101 may include a high-speed Random Access Memory (RAM), and may also include a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. The communication connection between the network element of the system and at least one other network element is realized through at least one communication interface 103 (which may be wired or wireless), and the internet, a wide area network, a local network, a metropolitan area network, and the like can be used.

The bus 102 may be an ISA bus, PCI bus, EISA bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory 101 is used for storing a program, and the processor 100 executes the program after receiving an execution instruction, and the method disclosed in any of the foregoing embodiments of the present application may be applied to the processor 100, or implemented by the processor 100.

Processor 100 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in the form of software in the processor 100. The Processor 100 may be a general-purpose Processor, and may include a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in the memory 101, and the processor 100 reads the information in the memory 101 and completes the steps of the method in combination with the hardware.

The electronic device provided by the embodiment of the application and the method provided by the embodiment of the application have the same inventive concept and have the same beneficial effects as the method adopted, operated or realized by the electronic device.

Another embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, the program being executed by a processor to implement the image object detection method of any of the above embodiments. Referring to fig. 12, a computer readable storage medium is shown as an optical disc 20, on which a computer program (i.e. a program product) is stored, which when executed by a processor, performs the method provided by any of the preceding embodiments.

It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, a phase change memory (PRAM), a Static Random Access Memory (SRAM), a Dynamic Random Access Memory (DRAM), other types of Random Access Memories (RAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a flash memory, or other optical and magnetic storage media, which are not described in detail herein.

The computer-readable storage medium provided by the above-mentioned embodiments of the present application and the method provided by the embodiments of the present application have the same advantages as the method adopted, executed or implemented by the application program stored in the computer-readable storage medium.

It should be noted that:

the algorithms and displays presented herein are not inherently related to any particular computer, virtual machine, or other apparatus. Various general purpose devices may also be used with the examples based on this disclosure. The required structure for constructing such a device will be apparent from the description above. In addition, this application is not directed to any particular programming language. It will be appreciated that a variety of programming languages may be used to implement the teachings of the present application as described herein, and any descriptions of specific languages are provided above to disclose the best modes of the present application.

It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and may be performed in other orders unless explicitly stated herein. Moreover, at least a portion of the steps in the flow chart of the figure may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

The above-mentioned embodiments only express the embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. An image object detection method, comprising:

combining the non-zero element blocks connected through the adjacent edges to generate a target frame containing a target;

the respectively merging the adjacent non-zero elements in each row of each image block into one non-zero element block comprises:

the row number of the non-zero element block is the row coordinate of the adjacent non-zero element, the starting point is the column coordinate of the first non-zero element in the adjacent non-zero element, the end point is the column coordinate of the last non-zero element in the adjacent non-zero element, and the end point is larger than the starting point;

establishing an adjacent edge by taking each non-zero element block as a node according to the relation of column coordinates between the non-zero element blocks of adjacent rows, comprising:

if so, establishing an adjacent edge between the two non-zero element blocks;

2. The image object detection method according to claim 1, wherein the two non-zero element blocks include a first non-zero element block and a second non-zero element block, and the determining whether there is an overlap between intervals of column coordinates of the two non-zero element blocks includes:

if yes, judging that the superposition exists;

if not, it is determined that there is no overlap.

3. An image object detecting apparatus, characterized by comprising:

the node merging module is used for merging the nonzero element blocks connected through the adjacent edges to generate a target frame containing a target;

if so, establishing an adjacent edge between the two non-zero element blocks;

4. The image object detection device according to claim 3, wherein the row coordinate includes a row number, the column coordinate includes a start point and an end point of a non-zero element block, the non-zero element merging module is plural, each non-zero element merging module includes a pulse array buffer, a first group of registers, a second group of registers, a first control logic sub-module, a non-zero element block memory, a non-zero element merging circuit, a column address counter, a row address counter, and a non-zero element block count memory per row, and the first group of registers and the second group of registers include one or more registers, respectively;

the pulse array buffer is used for storing one image block;

5. The image target detection device according to claim 4, wherein the establishing module comprises a discrimination circuit and an adjacent edge generating circuit; the judging circuit is used for judging whether the condition for generating the adjacent edge is met or not, the adjacent edge generating circuit is used for generating the adjacent edge when the condition for generating the adjacent edge is met, and the adjacent edge is established by taking each non-zero element block as a node.

6. The image object detecting device according to claim 5, wherein the discriminating circuit includes a second control logic sub-module, a third register, a fourth register, a first flag generator, a second flag generator, a fifth register, a sixth register, a seventh register, an eighth register, a first comparator and a second comparator;

7. The image object detection device of claim 6, wherein the adjacent edge generation circuit comprises a third control logic sub-module, a vertex buffer, an edge buffer, a start buffer, a ninth register, a tenth register, an eleventh register, a twelfth register, and a thirteenth register;

8. The image object detection device according to claim 7, wherein the node merging module includes a fourth control logic sub-module, a node queue buffer, a new node register, a merging sub-module, and a merged node buffer;

9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executing the program to implement the method of any one of claims 1-2.

10. A computer-readable storage medium, on which a computer program is stored, characterized in that the program is executed by a processor to implement the method according to any of claims 1-2.