CN113723318A - Method and device for determining shielding state of target object and monitoring equipment - Google Patents
Method and device for determining shielding state of target object and monitoring equipment Download PDFInfo
- Publication number
- CN113723318A CN113723318A CN202111023064.XA CN202111023064A CN113723318A CN 113723318 A CN113723318 A CN 113723318A CN 202111023064 A CN202111023064 A CN 202111023064A CN 113723318 A CN113723318 A CN 113723318A
- Authority
- CN
- China
- Prior art keywords
- preset
- region
- determining
- probability
- background
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000012544 monitoring process Methods 0.000 title abstract description 6
- 238000001514 detection method Methods 0.000 claims abstract description 16
- 238000004422 calculation algorithm Methods 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 6
- 230000004927 fusion Effects 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 4
- 238000012549 training Methods 0.000 claims description 4
- 238000012806 monitoring device Methods 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 abstract description 10
- 238000004458 analytical method Methods 0.000 description 10
- 238000013527 convolutional neural network Methods 0.000 description 8
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000000903 blocking effect Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000005034 decoration Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method, a device and monitoring equipment for determining the shielding state of a target object, wherein the method for determining the shielding state of the target object comprises the following steps: carrying out object detection on a target image of a target scene, and determining an object area in the target image; determining at least one preset sub-region corresponding to the object region; and determining whether the target object in the object region is shielded or not according to the historical shielding probability corresponding to each preset sub-region in at least one preset sub-region, wherein the historical shielding probability is based on the probability that the historical target in the corresponding preset sub-region is shielded. The method for determining the shielding state of the target object solves the problems of low speed and poor precision when the shielding condition of the target object in the image is judged in the related technology.
Description
Technical Field
The invention relates to the field of image processing, in particular to a method and a device for determining a shielding state of a target object and monitoring equipment.
Background
In the field of intelligent security, various targets in a monitoring video are analyzed, for example, attribute analysis of pedestrians, vehicles and non-motor vehicles is generally performed in video structuring, or re-identification of pedestrians and non-motor vehicles is performed. However, the quality of the targets in the natural scene is very different, some targets are seriously shielded, and if the targets are directly input into the subsequent analysis module for analysis, the accuracy of the analysis is difficult to ensure, and even an analysis result completely opposite to the actual result can be generated. Therefore, in order to ensure the accuracy of the recognition analysis, it is necessary to determine whether the target is occluded during the analysis.
In the related technology, the scheme for judging the target shielding condition in the image generally has great dependence on equipment calculation power, and has low analysis speed and poor accuracy.
In view of the above problems, no effective solution has been proposed.
The above information disclosed in the background section is only for enhancement of understanding of the background of the technology described herein. The background art may therefore contain certain information that does not form the known prior art to those skilled in the art.
Disclosure of Invention
The embodiment of the invention provides a method, a device and monitoring equipment for determining a shielding state of a target object, which are used for at least solving the problems of low speed and poor precision when the shielding condition of the target object in an image is judged in the related art.
In order to achieve the above object, according to a first aspect of embodiments of the present invention, there is provided a method of determining an occlusion state of a target object, including: carrying out object detection on a target image of a target scene, and determining an object area in the target image; determining at least one preset sub-region corresponding to the object region; and determining whether the target object in the object region is shielded or not according to the historical shielding probability corresponding to each preset sub-region in at least one preset sub-region, wherein the historical shielding probability is based on the probability that the historical target in the corresponding preset sub-region is shielded.
Further, determining whether the target object in the object region is occluded according to the historical occlusion probability corresponding to each preset sub-region in the at least one preset sub-region, includes: determining whether the object part of the target object, which is positioned in each preset subarea, is shielded or not according to the historical shielding probability of each preset subarea; and if the object part positioned in at least one preset subarea is blocked, determining that the target object is blocked.
Further, determining whether the object part located in each preset sub-region in the target object is occluded according to the historical occlusion probability of each preset sub-region, includes: the following operations are respectively carried out for each preset subarea: and if the historical shielding probability corresponding to one preset subarea in each preset subarea is greater than or equal to the first preset probability, determining that the part of the target object located in one preset subarea is shielded.
Further, still include: if the historical shielding probability corresponding to one preset sub-region in each preset sub-region is smaller than or equal to a second preset probability, determining that the part, located in the preset sub-region, of the target object is not shielded, wherein the second preset probability is smaller than the first preset probability.
Further, still include: if the historical shielding probability of one preset subregion in each preset subregion is larger than the second preset probability and smaller than the first preset probability, at least inputting the part of the target image corresponding to the preset subregion into a shielding condition judgment model obtained by pre-training, and acquiring the shielding condition of the part of the target object located in the preset subregion.
Further, before performing object detection on a target image of a target scene and determining an object region in the target image, the method further includes: acquiring a background image of a target scene; determining whether a background structure corresponding to each preset subregion in the background image is a bottom background, wherein the bottom background is a background structure which can not shield the target object; and under the condition that the background structure corresponding to the corresponding preset subarea is the bottom background, determining that the historical shielding probability of the preset subarea is 0.
Further, in a case that the background structure corresponding to the corresponding preset sub-region is not the bottom layer background, the method further includes: acquiring a multi-frame historical image of a target scene, wherein the frame number of the historical image is greater than a preset frame number; inputting multi-frame historical images into a pre-trained shielding condition judgment model, and acquiring a plurality of historical shielding results of a preset subregion; and calculating the historical shielding probability of the preset subarea according to the plurality of historical shielding results.
Further, determining whether a background structure corresponding to each preset subregion in the background image is a bottom-layer background, including: determining the background structure classification of each pixel point of the background image; and determining whether the background structure corresponding to the corresponding preset subarea in the background image is the bottom background or not according to the background structure classification of a plurality of pixel points of the background image corresponding to the preset subareas.
Further, determining whether a background structure corresponding to a corresponding preset subregion in the background image is a bottom background according to the background structure classification of a plurality of pixel points corresponding to each preset subregion of the background image, including: calculating the occupation ratio of pixel points belonging to a bottom background structure in a plurality of pixel points corresponding to a preset subregion; determining that a background structure corresponding to a preset subregion in a background image is a bottom background under the condition that the occupation ratio of pixel points belonging to the bottom background structure in a plurality of pixel points corresponding to the preset subregion is greater than or equal to the preset occupation ratio; and under the condition that the occupation ratio of the pixel points belonging to the bottom background structure in the plurality of pixel points corresponding to the preset subarea is smaller than the preset occupation ratio, determining that the background structure corresponding to the preset subarea in the background image is not the bottom background.
Further, acquiring a background image of the target scene, comprising: acquiring a multi-frame scene image of a target scene; and obtaining a background image by adopting an image fusion algorithm according to the multi-frame scene image.
According to a second aspect of the embodiments of the present invention, there is provided an apparatus for determining an occlusion state of a target object, including: the detection unit is used for carrying out object detection on a target image of a target scene and determining an object area in the target image; the first determining unit is used for determining at least one preset sub-area corresponding to the object area; and the second determining unit is used for determining whether the target object in the object region is occluded according to the historical occlusion probability corresponding to each preset sub-region in at least one preset sub-region, wherein the historical occlusion probability is based on the probability that the historical target in the corresponding preset sub-region is occluded.
According to a third aspect of the embodiments of the present invention, there is provided a non-volatile storage medium, the non-volatile storage medium includes a stored program, wherein, when the program is executed, a device in which the non-volatile storage medium is controlled is used for executing the above method for determining the shielding state of the target object.
According to a fourth aspect of the embodiments of the present invention, there is provided a processor for executing a program, where the program executes the method for determining the occlusion state of the target object.
According to a fifth aspect of the embodiments of the present invention, there is provided a monitoring device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the above method for determining an occlusion state of a target object when executing the computer program.
The method for determining the shielding state of the target object by applying the technical scheme of the invention comprises the following steps: carrying out object detection on a target image of a target scene, and determining an object area in the target image; determining at least one preset sub-region corresponding to the object region; and determining whether the target object in the object region is shielded or not according to the historical shielding probability corresponding to each preset sub-region in at least one preset sub-region, wherein the historical shielding probability is based on the probability that the historical target in the corresponding preset sub-region is shielded. Therefore, by combining probability analysis, whether the target object is shielded or not is determined at least according to the historical shielding probability of at least one preset subregion where the target object is located. When the judgment method is used for judging the shielding condition of the preset subarea with high or low shielding probability in the target image, compared with the image identification technology in the related technology, the judgment method has higher judgment precision, thereby solving the problems of low speed and poor precision when the shielding condition of the target in the image is judged in the related technology.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
FIG. 1 is a schematic flow chart diagram of an alternative embodiment of a method of determining a target object occlusion state according to the present invention;
FIG. 2 is a schematic view of an alternative embodiment of an apparatus for determining the occlusion status of a target object according to the present invention;
FIG. 3 is a schematic flow chart diagram illustrating an alternative embodiment of a method for determining a target object occlusion status according to the present invention;
FIG. 4 is a flowchart illustrating the process of determining the historical occlusion probability tables of the grids of the target scene according to the method for determining the occlusion state of the target object of the present invention.
Detailed Description
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It will be understood that when an element such as a layer, film, region, or substrate is referred to as being "on" another element, it can be directly on the other element or intervening elements may also be present. Also, in the specification and claims, when an element is described as being "connected" to another element, the element may be "directly connected" to the other element or "connected" to the other element through a third element.
Fig. 1 is a method for determining an occlusion state of a target object according to an embodiment of the present invention, as shown in fig. 1, the method includes the following steps:
step S102, acquiring a target image of a target scene;
step S104, determining one or more preset sub-regions where the target object in the target image is located;
step S106, determining whether the target object is shielded or not according to at least the historical shielding probability of one or more preset sub-regions, wherein the historical shielding probability is the probability that the historical target in the corresponding preset sub-region is shielded by other objects.
The method for determining the shielding state of the target object by adopting the scheme comprises the following steps: carrying out object detection on a target image of a target scene, and determining an object area in the target image; determining at least one preset sub-region corresponding to the object region; and determining whether the target object in the object region is shielded or not according to the historical shielding probability corresponding to each preset sub-region in at least one preset sub-region, wherein the historical shielding probability is based on the probability that the historical target in the corresponding preset sub-region is shielded. Therefore, by combining probability analysis, whether the target object is shielded or not is determined at least according to the historical shielding probability of at least one preset subregion where the target object is located. When the judgment method is used for judging the shielding condition of the preset subarea with high or low shielding probability in the target image, compared with the image identification technology in the related technology, the judgment method has higher judgment precision, thereby solving the problems of low speed and poor precision when the shielding condition of the target in the image is judged in the related technology.
It should be noted that the preset sub-regions are formed by being divided in advance according to the target scene, and the shape of each preset sub-region may be any shape.
Specifically, determining whether the target object in the object region is occluded according to the historical occlusion probability corresponding to each preset sub-region in the at least one preset sub-region includes: determining whether the object part of the target object, which is positioned in each preset subarea, is shielded or not according to the historical shielding probability of each preset subarea; and if the object part positioned in at least one preset subarea is blocked, determining that the target object is blocked.
And if the parts of the target object, which are positioned in all the preset sub-areas, are not shielded, determining that the target object is not shielded.
The historical shielding probability of each preset sub-region, namely the probability that the historical target in each preset sub-region is shielded by other objects, can store the historical statistical result, so that the historical shielding probability of each preset sub-region can be conveniently determined in the follow-up process.
In a specific case, the target object may occupy one or more predetermined sub-areas:
when the target object occupies a preset subregion, if the part of the target object, which is positioned in the preset subregion, is determined to be shielded according to the historical shielding probability of the preset subregion, the target object can be determined to be shielded certainly; and if the part of the target object, which is positioned in the preset subarea, is determined not to be occluded according to the historical occlusion probability of the preset subarea, the target object can be determined not to be occluded.
When the target object occupies a plurality of preset sub-areas, whether the part of the target object located in each preset sub-area is shielded or not can be determined according to the historical shielding probability of each preset sub-area, and if the part of the target object located in each preset sub-area is shielded, the target object is shielded certainly; otherwise, if the portions of the target object located in all the preset sub-areas are not occluded, it may be determined that the target object is not occluded.
Determining whether the object part positioned in each preset subarea in the target object is shielded or not according to the historical shielding probability of each preset subarea, wherein the method comprises the following steps: the following operations are respectively carried out for each preset subarea: and if the historical shielding probability corresponding to one preset subarea in each preset subarea is greater than or equal to the first preset probability, determining that the part of the target object located in one preset subarea is shielded. If the historical shielding probability corresponding to one preset sub-region in each preset sub-region is smaller than or equal to a second preset probability, determining that the part, located in the preset sub-region, of the target object is not shielded, wherein the second preset probability is smaller than the first preset probability.
In specific implementation, the first preset probability and the second preset probability may be flexibly selected according to actual conditions or requirements, for example, in this embodiment, the first preset probability may be 80% or 90%, at this time, if the historical blocking probability of the corresponding preset sub-region reaches the first preset probability, a portion of the target object located in the preset sub-region is very large and may be blocked, and at this time, the determination that the target object is blocked is made, which may be faster and more reliable in accuracy than other image recognition algorithms. Similarly, in this embodiment, the second preset probability may be 10% or 20%, and when the historical occlusion probability of the corresponding preset sub-region is less than or equal to the second preset probability, the probability that the part of the target object located in the preset sub-region is occluded is also small, and at this time, the determination that the part of the target object located in the preset sub-region is not occluded is made, which may be faster and more reliable than the speed and accuracy of using other image recognition algorithms.
If the historical shielding probability of one preset subregion in each preset subregion is larger than the second preset probability and smaller than the first preset probability, at least inputting the part of the target image corresponding to the preset subregion into a shielding condition judgment model obtained by pre-training, and acquiring the shielding condition of the part of the target object located in the preset subregion.
When the historical shielding probability of the corresponding preset subarea is between the first preset probability and the second preset probability, an accurate shielding condition result is obtained through the difficulty in passing the probability, at the moment, at least the part of the target image corresponding to the preset subarea is input into the shielding condition judgment model, and a more reliable judgment result is favorably obtained. Thus, by combining probability judgment and model judgment, the judgment speed can be effectively improved, and the judgment precision for various conditions can be ensured.
Specifically, before performing object detection on a target image of a target scene and determining an object region in the target image, the method further includes: acquiring a background image of a target scene; determining whether a background structure corresponding to each preset subregion in the background image is a bottom background, wherein the bottom background is a background structure which can not shield the target object; and under the condition that the background structure corresponding to the corresponding preset subarea is the bottom background, determining that the historical shielding probability of the preset subarea is 0.
That is to say, whether the background structure corresponding to each preset subregion in the background image is the bottom background is determined, if so, the background structure corresponding to the preset subregion cannot shield the target object, and at this time, the historical shielding probability of the preset subregion is directly determined to be 0, so that the result is reliable, and the historical shielding probability does not need to be calculated by performing a large amount of data analysis on the region subsequently. The bottom background is a background structure which cannot shield the target object, for example, background structures of sky, road and other types, and when a part of the target object appears in the preset sub-regions, background shielding can be definitely not generated, so that the judgment can be made quickly and accurately. However, if the determination is performed by using a CNN (convolutional neural network determination model) or other methods, the speed is slow, and the accuracy of the obtained result is poor when the CNN effect is poor.
Specifically, in the case that the background structure corresponding to the corresponding preset sub-region is not the bottom-layer background, the method further includes: acquiring a multi-frame historical image of a target scene, wherein the frame number of the historical image is greater than a preset frame number; inputting multi-frame historical images into a pre-trained shielding condition judgment model, and acquiring a plurality of historical shielding results of a preset subregion; and calculating the historical shielding probability of the preset subarea according to the plurality of historical shielding results.
Under the condition that a background structure corresponding to the corresponding preset subarea is not a bottom background, a shielding condition judgment model (CNN) is adopted to analyze the multi-frame historical images to obtain a plurality of historical shielding results, and then the historical shielding probability of the preset subarea can be calculated by calculating the ratio of the total number of the shielding results to the total number of all the historical shielding results. It can be understood that, when the total number of the historical occlusion results is small, the calculated historical occlusion probability may have a large error and a poor reliability, and by controlling the number of frames of the historical image to be greater than the preset number of frames, the historical occlusion probability region is favorably stabilized and reliable, and when the historical occlusion probability region is specifically implemented, the specific value of the preset number of frames can be flexibly selected.
Taking the example of dividing the image into a plurality of grid structures, at this time, each grid corresponds to each preset sub-region, and statistics is performed on probability information that the target object in each grid is shielded by the background, for example, if one target object occupies A, B, C three grids, the target counts of A, B, C three grids are respectively added with 1, and if the target object is judged according to CNN, if the part of the target object located in the corresponding grid is shielded by the background, the target object in the grid is shielded by the background by + 1; if the CNN judges that the part of the target object, which is positioned in the corresponding grid, is not shielded, the background shielding count of the target object in the grid is unchanged. And when the occurrence number of the target objects exceeds a certain confidence threshold N, counting the proportion of background occlusion of the target objects in the grid as the probability of the background occlusion of the target objects in the grid.
Specifically, determining whether a background structure corresponding to each preset subregion in the background image is a bottom background includes: determining the background structure classification of each pixel point of the background image; and determining whether the background structure corresponding to the corresponding preset subarea in the background image is the bottom background or not according to the background structure classification of a plurality of pixel points of the background image corresponding to the preset subareas.
That is to say, each pixel point of the background image is classified, the background structure to which the pixel point belongs is determined, and then whether the background structure corresponding to each preset subregion is the bottom background is determined according to the classification condition of a plurality of pixel points in each preset subregion.
Specifically, determining whether a background structure corresponding to a corresponding preset subregion in the background image is a bottom background according to the background structure classification of a plurality of pixel points corresponding to each preset subregion of the background image includes: calculating the occupation ratio of pixel points belonging to a bottom background structure in a plurality of pixel points corresponding to a preset subregion; determining that a background structure corresponding to a preset subregion in a background image is a bottom background under the condition that the occupation ratio of pixel points belonging to the bottom background structure in a plurality of pixel points corresponding to the preset subregion is greater than or equal to the preset occupation ratio; and under the condition that the occupation ratio of the pixel points belonging to the bottom background structure in the plurality of pixel points corresponding to the preset subarea is smaller than the preset occupation ratio, determining that the background structure corresponding to the preset subarea in the background image is not the bottom background.
Acquiring a background image of a target scene, comprising: acquiring a multi-frame scene image of a target scene; and obtaining a background image by adopting an image fusion algorithm according to the multi-frame scene image. Therefore, by adopting the image fusion algorithm, the foreground target in the image can be effectively removed, a complete background image is obtained, and the accuracy of the final judgment result is ensured.
As shown in fig. 3 and 4, fig. 3 is a schematic diagram of a method for determining an occlusion state of a target object according to a more specific embodiment of the present invention, in a specific implementation, a type of each preset sub-region is a grid, that is, a target image is divided into a plurality of grids, after the target image is obtained, a grid covered by the target image is determined, then a historical occlusion probability of each grid is determined according to each grid historical occlusion probability table of a target scene, according to a magnitude relationship between the historical occlusion probability of each grid and a first preset probability and a second preset probability, if the historical occlusion probability is greater than or equal to the first preset probability, a portion of the target object located in the grid is determined to be occluded, if the historical occlusion probability of each grid is less than the second preset probability, a portion of the target object located in the grid is determined to be not occluded, and if the historical occlusion probability is between the first preset probability and the second preset probability, and operating a pre-trained occlusion condition judgment model (CNN) to determine whether the part of the target object located in the grid is occluded. Fig. 4 is a specific embodiment of obtaining a historical occlusion probability table of each mesh of a target scene, and as shown in fig. 4, when the specific embodiment is implemented, a multi-frame image is fused by using an image fusion algorithm to remove a foreground target, a background image is obtained, the background image is divided into a plurality of meshes, then a scene segmentation algorithm is used to determine a background structure classification to which each pixel of the background image belongs, according to the background structure classification of each pixel, whether each mesh is a bottom background can be determined, if the corresponding mesh is the bottom background, the historical occlusion probability of the mesh is determined to be 0, otherwise, an occlusion condition judgment model (CNN) is run for the multi-frame historical image, and then the historical occlusion probability of the mesh is counted, and the historical occlusion probability table of each mesh of the target scene can be obtained by summarizing the historical occlusion probability table of each mesh.
Secondly, as shown in fig. 2, an embodiment of the present invention further provides an apparatus for determining an occlusion state of a target object, including: the detection unit is used for carrying out object detection on a target image of a target scene and determining an object area in the target image; the first determining unit is used for determining at least one preset sub-area corresponding to the object area; and the second determining unit is used for determining whether the target object in the object region is occluded according to the historical occlusion probability corresponding to each preset sub-region in at least one preset sub-region, wherein the historical occlusion probability is based on the probability that the historical target in the corresponding preset sub-region is occluded.
The second determination unit includes a first determination module and a second determination module: the first determining module is used for determining whether the object part positioned in each preset subarea in the target object is shielded or not according to the historical shielding probability of each preset subarea; the second determining module is used for determining that the target object is occluded if the object part located in the at least one preset sub-area is occluded.
Specifically, the first determining module is configured to: the following operations are respectively carried out for each preset subarea: and if the historical shielding probability corresponding to one preset subarea in each preset subarea is greater than or equal to the first preset probability, determining that the part of the target object located in one preset subarea is shielded.
The first determining module is further configured to: if the historical shielding probability corresponding to one preset sub-region in each preset sub-region is smaller than or equal to a second preset probability, determining that the part, located in the preset sub-region, of the target object is not shielded, wherein the second preset probability is smaller than the first preset probability.
Specifically, the first determining module is further configured to: if the historical shielding probability of one preset subregion in each preset subregion is larger than the second preset probability and smaller than the first preset probability, at least inputting the part of the target image corresponding to the preset subregion into a shielding condition judgment model obtained by pre-training, and acquiring the shielding condition of the part of the target object located in the preset subregion.
The device further comprises a first obtaining unit, a third determining unit and a fourth determining unit: the first acquisition unit is used for acquiring a background image of a target scene before object detection is carried out on the target image of the target scene and an object area in the target image is determined; the third determining unit is used for determining whether a background structure corresponding to each preset subregion in the background image is a bottom background, and the bottom background is a background structure which can not shield the target object; the fourth determining unit is configured to determine that the historical occlusion probability of the preset sub-region is 0 when the background structure corresponding to the corresponding preset sub-region is the bottom-layer background.
Specifically, under the condition that the background structure corresponding to the corresponding preset sub-region is not the bottom layer background, the device further comprises a second obtaining unit and an input unit: the second acquisition unit is used for acquiring multi-frame historical images of the target scene, and the frame number of the historical images is greater than the preset frame number; the input unit is used for inputting multi-frame historical images into a pre-trained shielding condition judgment model and acquiring a plurality of historical shielding results of a preset subregion; and calculating the historical shielding probability of the preset subarea according to the plurality of historical shielding results.
The third determining unit is configured to include: determining the background structure classification of each pixel point of the background image; and determining whether the background structure corresponding to the corresponding preset subarea in the background image is the bottom background or not according to the background structure classification of a plurality of pixel points of the background image corresponding to the preset subareas.
Specifically, the third determination unit is configured to: calculating the occupation ratio of pixel points belonging to a bottom background structure in a plurality of pixel points corresponding to a preset subregion; determining that a background structure corresponding to a preset subregion in a background image is a bottom background under the condition that the occupation ratio of pixel points belonging to the bottom background structure in a plurality of pixel points corresponding to the preset subregion is greater than or equal to the preset occupation ratio; and under the condition that the occupation ratio of the pixel points belonging to the bottom background structure in the plurality of pixel points corresponding to the preset subarea is smaller than the preset occupation ratio, determining that the background structure corresponding to the preset subarea in the background image is not the bottom background.
The first acquisition unit is used for: acquiring a multi-frame scene image of a target scene; and obtaining a background image by adopting an image fusion algorithm according to the multi-frame scene image.
In addition, the embodiment of the invention also provides a nonvolatile storage medium, the nonvolatile storage medium comprises a stored program, and when the program runs, the device where the nonvolatile storage medium is located is controlled to execute the method for determining the shielding state of the target object.
The embodiment of the present invention further provides a processor, where the processor is configured to execute a program, where the program executes the method for determining the occlusion state of the target object when running.
Finally, an embodiment of the present invention also provides a monitoring device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the method for determining the blocking status of the target object is implemented.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments. Moreover, the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer-executable instructions, and while a logical order is illustrated in the flowcharts, in some cases, the steps illustrated or described may be performed in an order different than here.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units may be a logical division, and in actual implementation, there may be another division, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.
Claims (14)
1. A method of determining a target object occlusion state, comprising:
carrying out object detection on a target image of a target scene, and determining an object area in the target image;
determining at least one preset sub-region corresponding to the object region;
and determining whether the target object in the object region is occluded according to the historical occlusion probability corresponding to each preset sub-region in the at least one preset sub-region, wherein the historical occlusion probability is based on the probability that the historical target in the corresponding preset sub-region is occluded.
2. The method according to claim 1, wherein determining whether the target object in the object region is occluded according to a historical occlusion probability corresponding to each preset sub-region in the at least one preset sub-region comprises:
determining whether an object part positioned in each preset subarea in the target object is shielded or not according to the historical shielding probability of each preset subarea;
and if the object part positioned in at least one preset subarea is blocked, determining that the target object is blocked.
3. The method according to claim 2, wherein determining whether the object portion of the target object located in each of the preset sub-regions is occluded according to the historical occlusion probability of each of the preset sub-regions comprises: the following operations are respectively carried out for each preset subarea:
if the historical shielding probability corresponding to one preset sub-region in each preset sub-region is greater than or equal to a first preset probability, it is determined that the part, located in the preset sub-region, of the target object is shielded.
4. The method for determining the occlusion state of a target object according to claim 3, further comprising:
if the historical shielding probability corresponding to one preset subregion in each preset subregion is smaller than or equal to a second preset probability, determining that the part, located in the preset subregion, of the target object is not shielded, wherein the second preset probability is smaller than the first preset probability.
5. The method for determining the occlusion state of a target object according to claim 4, further comprising:
if the historical shielding probability of one preset sub-region in each preset sub-region is larger than the second preset probability and smaller than the first preset probability, inputting at least a part of the target image corresponding to the preset sub-region into a shielding condition judgment model obtained through pre-training, and acquiring the shielding condition of the part of the target object located in the preset sub-region.
6. The method for determining the occlusion status of a target object according to any one of claims 1 to 5, wherein before performing object detection on a target image of a target scene and determining an object region in the target image, the method further comprises:
acquiring a background image of the target scene;
determining whether a background structure corresponding to each preset sub-region in the background image is a bottom background, wherein the bottom background is a background structure which cannot shield the target object;
and determining that the historical shielding probability of the preset sub-region is 0 under the condition that the corresponding background structure of the preset sub-region is the bottom background.
7. The method according to claim 6, wherein in case that the background structure corresponding to the corresponding preset sub-region is not an underlying background, the method further comprises:
acquiring a multi-frame historical image of the target scene, wherein the frame number of the historical image is greater than a preset frame number;
inputting a plurality of frames of historical images into a pre-trained shielding condition judgment model to obtain a plurality of historical shielding results of the preset sub-region;
and calculating the historical occlusion probability of the preset sub-region according to a plurality of historical occlusion results.
8. The method according to claim 6, wherein determining whether the background structure corresponding to each of the preset sub-regions in the background image is an underlying background comprises:
determining the background structure classification of each pixel point of the background image;
and determining whether the background structure corresponding to the corresponding preset subarea in the background image is a bottom background or not according to the background structure classification of the pixel points of the background image corresponding to the preset subareas.
9. The method according to claim 8, wherein determining whether a background structure in the background image corresponding to the corresponding predetermined sub-region is an underlying background according to the background structure classification of the plurality of pixel points in the background image corresponding to the respective predetermined sub-regions comprises:
calculating the occupation ratio of the pixel points belonging to the bottom background structure in the plurality of pixel points corresponding to the preset subarea;
determining that a background structure corresponding to the preset subregion in the background image is a bottom background under the condition that the occupation ratio of the pixel points belonging to the bottom background structure in the plurality of pixel points corresponding to the preset subregion is greater than or equal to a preset occupation ratio;
and under the condition that the occupation ratio of the pixel points belonging to the bottom background structure in the plurality of pixel points corresponding to the preset subarea is smaller than the preset occupation ratio, determining that the background structure corresponding to the preset subarea in the background image is not the bottom background.
10. The method according to claim 6, wherein obtaining the background image of the target scene comprises:
acquiring a multi-frame scene image of the target scene;
and obtaining the background image by adopting an image fusion algorithm according to the plurality of frames of scene images.
11. An apparatus for determining an occlusion state of a target object, comprising:
the device comprises a detection unit, a processing unit and a processing unit, wherein the detection unit is used for carrying out object detection on a target image of a target scene and determining an object area in the target image;
the first determining unit is used for determining at least one preset sub-area corresponding to the object area;
a second determining unit, configured to determine whether the target object in the object region is occluded according to a historical occlusion probability corresponding to each preset sub-region in the at least one preset sub-region, where the historical occlusion probability is based on a probability that a historical target in the corresponding preset sub-region is occluded.
12. A non-volatile storage medium, comprising a stored program, wherein when the program runs, a device in which the non-volatile storage medium is located is controlled to execute the method for determining the shielding status of the target object according to any one of claims 1 to 10.
13. A processor, characterized in that the processor is configured to run a program, wherein the program is run to perform the method of determining an occlusion state of a target object according to any one of claims 1 to 10.
14. A monitoring device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the method of determining a target object occlusion state of any of claims 1 to 10 when executing the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111023064.XA CN113723318A (en) | 2021-09-01 | 2021-09-01 | Method and device for determining shielding state of target object and monitoring equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111023064.XA CN113723318A (en) | 2021-09-01 | 2021-09-01 | Method and device for determining shielding state of target object and monitoring equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113723318A true CN113723318A (en) | 2021-11-30 |
Family
ID=78680714
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111023064.XA Pending CN113723318A (en) | 2021-09-01 | 2021-09-01 | Method and device for determining shielding state of target object and monitoring equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113723318A (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110222764A (en) * | 2019-06-10 | 2019-09-10 | 中南民族大学 | Shelter target detection method, system, equipment and storage medium |
| US20200034657A1 (en) * | 2017-07-27 | 2020-01-30 | Tencent Technology (Shenzhen) Company Limited | Method and apparatus for occlusion detection on target object, electronic device, and storage medium |
| US20200034959A1 (en) * | 2018-07-24 | 2020-01-30 | The Regents Of The University Of Michigan | Detection Of Near-Field Occlusions In Images |
| CN111523480A (en) * | 2020-04-24 | 2020-08-11 | 北京嘀嘀无限科技发展有限公司 | Method and device for detecting face obstruction, electronic equipment and storage medium |
| CN112733802A (en) * | 2021-01-25 | 2021-04-30 | 腾讯科技（深圳）有限公司 | Image occlusion detection method and device, electronic equipment and storage medium |
| CN112927178A (en) * | 2019-11-21 | 2021-06-08 | 中移物联网有限公司 | Occlusion detection method, occlusion detection device, electronic device, and storage medium |
| CN112967351A (en) * | 2021-03-05 | 2021-06-15 | 北京字跳网络技术有限公司 | Image generation method and device, electronic equipment and storage medium |
| CN113313189A (en) * | 2021-06-11 | 2021-08-27 | 上海高德威智能交通系统有限公司 | Shielding detection method and device and electronic equipment |
-
2021
- 2021-09-01 CN CN202111023064.XA patent/CN113723318A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200034657A1 (en) * | 2017-07-27 | 2020-01-30 | Tencent Technology (Shenzhen) Company Limited | Method and apparatus for occlusion detection on target object, electronic device, and storage medium |
| US20200034959A1 (en) * | 2018-07-24 | 2020-01-30 | The Regents Of The University Of Michigan | Detection Of Near-Field Occlusions In Images |
| CN110222764A (en) * | 2019-06-10 | 2019-09-10 | 中南民族大学 | Shelter target detection method, system, equipment and storage medium |
| CN112927178A (en) * | 2019-11-21 | 2021-06-08 | 中移物联网有限公司 | Occlusion detection method, occlusion detection device, electronic device, and storage medium |
| CN111523480A (en) * | 2020-04-24 | 2020-08-11 | 北京嘀嘀无限科技发展有限公司 | Method and device for detecting face obstruction, electronic equipment and storage medium |
| CN112733802A (en) * | 2021-01-25 | 2021-04-30 | 腾讯科技（深圳）有限公司 | Image occlusion detection method and device, electronic equipment and storage medium |
| CN112967351A (en) * | 2021-03-05 | 2021-06-15 | 北京字跳网络技术有限公司 | Image generation method and device, electronic equipment and storage medium |
| CN113313189A (en) * | 2021-06-11 | 2021-08-27 | 上海高德威智能交通系统有限公司 | Shielding detection method and device and electronic equipment |
Non-Patent Citations (2)
| Title |
|---|
| 储;朱陶;缪君;江烂达;: "基于遮挡检测和时空上下文信息的目标跟踪算法", 模式识别与人工智能, no. 08, 15 August 2017 (2017-08-15) * |
| 韩梅;纪明;史志富;肖君;刘亚琴;: "一种实时检测遮挡阴影的算法", 应用光学, no. 05, 15 September 2009 (2009-09-15) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110879951B (en) | Motion foreground detection method and device | |
| JP6733397B2 (en) | Leftover object detection device, method and system | |
| CN111010590A (en) | Video clipping method and device | |
| CN111914665B (en) | Face shielding detection method, device, equipment and storage medium | |
| CN110458790B (en) | Image detection method and device and computer storage medium | |
| CN111080654B (en) | Image lesion region segmentation method and device and server | |
| CN112417955B (en) | Method and device for processing tour inspection video stream | |
| CN108961316B (en) | Image processing method and device and server | |
| CN111932545A (en) | Image processing method, target counting method and related device thereof | |
| CN109934072B (en) | Personnel counting method and device | |
| CN111192241A (en) | Quality evaluation method and device of face image and computer storage medium | |
| CN111160107B (en) | Dynamic region detection method based on feature matching | |
| CN111159150A (en) | Data expansion method and device | |
| CN113674317A (en) | Vehicle tracking method and device of high-order video | |
| Lamba et al. | Segmentation of crowd flow by trajectory clustering in active contours | |
| CN110765875B (en) | Method, equipment and device for detecting boundary of traffic target | |
| CN113723318A (en) | Method and device for determining shielding state of target object and monitoring equipment | |
| CN111429487A (en) | Sticky foreground segmentation method and device for depth image | |
| JP2024516642A (en) | Behavior detection method, electronic device and computer-readable storage medium | |
| CN115546256A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| Dimiccoli et al. | Hierarchical region-based representation for segmentation and filtering with depth in single images | |
| CN110782425A (en) | Image processing method, image processing device and electronic equipment | |
| CN104754248A (en) | Method and device for acquiring target snapshot | |
| CN114549884A (en) | Abnormal image detection method, device, equipment and medium | |
| Oiwa et al. | Tracking with probabilistic background model by density forests |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |