CN115393791A - Foreign matter detection method, device and computer readable storage medium - Google Patents
Foreign matter detection method, device and computer readable storage medium Download PDFInfo
- Publication number
- CN115393791A CN115393791A CN202210965625.6A CN202210965625A CN115393791A CN 115393791 A CN115393791 A CN 115393791A CN 202210965625 A CN202210965625 A CN 202210965625A CN 115393791 A CN115393791 A CN 115393791A
- Authority
- CN
- China
- Prior art keywords
- target object
- target
- foreign
- detection
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 169
- 230000004044 response Effects 0.000 claims abstract description 25
- 238000000034 method Methods 0.000 claims abstract description 23
- 241000282693 Cercopithecidae Species 0.000 claims description 33
- 238000013507 mapping Methods 0.000 claims description 16
- 206010070245 Foreign body Diseases 0.000 claims 15
- 230000000875 corresponding effect Effects 0.000 description 36
- 238000012549 training Methods 0.000 description 7
- 238000012545 processing Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 230000010365 information processing Effects 0.000 description 4
- 241001465754 Metazoa Species 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000013475 authorization Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 239000003245 coal Substances 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000007477 logistic regression Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a foreign object detection method, a device and a computer readable storage medium, the method comprises: acquiring a target image; the target area of the target image comprises target equipment and a target object; in response to the target area including the target object, determining a degree of overlap between the target device, the target object, and the target object; responding to the overlapping degree meeting the overlapping requirement, and acquiring depth information of the target equipment, the target object and the target object; obtaining an initial foreign matter carrying result based on the depth information of the target equipment, the target object and the target object; determining a final foreign object carrying result of the target object based on the initial foreign object carrying result; and the final foreign matter carrying result is used for indicating whether the target object carries the target object when the target object is on the target device. Through the mode, the foreign matter detection accuracy can be improved.
Description
Technical Field
The present application relates to the field of image processing technologies, and in particular, to a method and an apparatus for detecting a foreign object, and a computer-readable storage medium.
Background
In the existing life, when some specific devices bear users, the users cannot carry prohibited articles which are definitely prohibited from carrying. If, monkey car etc. the monkey car is colliery a common simple and easy vehicle in the pit, and the auxiliary transport workman in mainly used underground mine in case the user carries and is definitely carried the prohibited articles that carry and take the monkey car, can increase the load of monkey car, and can reduce the life of monkey car, and the rope of transport monkey car can be pressed down to more serious condition.
Disclosure of Invention
The technical problem mainly solved by the application is to provide a foreign matter detection method, a device and a computer readable storage medium, which can improve the accuracy of foreign matter detection.
In order to solve the technical problem, the application adopts a technical scheme that: provided is a foreign matter detection method including: acquiring a target image; the target area of the target image comprises target equipment and a target object; in response to the target area including the target object, determining a degree of overlap between the target device, the target object, and the target object; responding to the overlapping degree meeting the overlapping requirement, and acquiring depth information of the target equipment, the target object and the target object; obtaining an initial foreign matter carrying result based on the depth information of the target equipment, the target object and the target object; determining a final foreign matter carrying result of the target object based on the initial foreign matter carrying result; and the final foreign matter carrying result is used for indicating whether the target object carries the target object when the target object is on the target device.
Wherein determining the degree of overlap between the target device, the target object and the target object comprises: respectively acquiring a first detection frame of target equipment, a second detection frame of a target object and a third detection frame of the target object; determining the degree of overlap among the target device, the target object and the target object based on the degree of overlap among the first detection frame, the second detection frame and the third detection frame; and the overlapping degree among the first detection frame, the second detection frame and the third detection frame is the intersection ratio of the first detection frame, the second detection frame and the third detection frame.
The acquiring of the depth information of the target device, the target object and the target object includes: respectively obtaining a first height of target equipment, a second height of a target object, a third height of the target object and a depth mapping model; inputting the first height, the second height and the third height into a depth mapping model respectively to obtain a first distance between the target equipment and the image acquisition device, a second distance between the target object and the image acquisition device and a third distance between the target object and the image acquisition device respectively; a first distance between the target device and the image acquisition apparatus is used as depth information of the target device, a second distance between the target object and the image acquisition apparatus is used as depth information of the target object, and a third distance between the target object and the image acquisition apparatus is used as depth information of the target object.
Wherein, based on the depth information of the target device, the target object and the target object, obtaining an initial foreign object carrying result comprises: and determining that the target object carries the target object when the target object is on the target device in response to the fact that the absolute value of the distance difference between any two of the target device, the target object and the target object is less than or equal to the distance difference threshold.
Wherein determining a final foreign object carrying result of the target object based on the initial foreign object carrying result comprises: and determining a final foreign matter carrying result of the target object based on the initial foreign matter carrying results corresponding to the preset number of historical frames in response to that the initial foreign matter carrying result corresponding to the current frame is that the target object carries the target object when the target object is on the target device.
Wherein, based on the initial foreign object carrying result corresponding to the preset number of historical frames, determining the final foreign object carrying result of the target object, comprising: in response to the fact that the initial foreign matter carrying results corresponding to the preset number of historical frames are all target objects carried when the target objects are on the target equipment, determining that the final foreign matter carrying results are target objects carried when the target objects are on the target equipment; or, in response to the fact that the number of frames carrying the target object when the initial foreign object carrying result in the preset number of historical frames is that the target object is on the target device meets a preset proportion, determining that the final foreign object carrying result is that the target object carries the target object when the target object is on the target device.
After determining a final foreign object carrying result of the target object based on the initial foreign object carrying results corresponding to the preset number of historical frames, the foreign object detection method further includes: and responding to the fact that the target object carries the target object when the target object is determined to be on the target device, and sending prompt information for prompting the target object to carry the target object to the user terminal.
Wherein, before determining the degree of overlap between the target device, the target object and the target object in response to the target area of the target image further including the target object, the foreign object detection method further includes: determining a detection area in the target area; the detection area comprises target equipment and a target object; in response to the target area further including the target object, determining a degree of overlap between the target device, the target object, and the target object, including: in response to the detection area in the target area further comprising the target object, determining a degree of overlap between the target device, the target object and the target object; and/or the target device comprises a monkey vehicle and the target object comprises a person.
In order to solve the above technical problem, another technical solution adopted by the present application is: the foreign matter detection device comprises a memory and a processor, wherein the memory stores program instructions, and the processor is used for executing the program instructions to realize the foreign matter detection method.
In order to solve the above technical problem, another technical solution adopted by the present application is: there is provided a computer-readable storage medium for storing program instructions that can be executed to implement the foreign object detection method described above.
According to the technical scheme, when the fact that the overlapping degree of the target device, the target object and the target object meets the overlapping requirement is determined, whether the target object carries the target object or not is determined based on the depth information of the target device, the target object and the target object. Therefore, when the overlapping degree of the target device, the target object and the target object is determined to meet the overlapping requirement, whether the target object carries the target object when the target object is on the target device is further determined according to the depth information of the target device, the target object and the target object, the occurrence of false detection caused by dislocation overlapping caused by the depth of field of the image is reduced, and the accuracy of foreign matter detection is improved.
Drawings
Fig. 1 is a schematic flowchart of an embodiment of a foreign object detection method provided in the present application;
FIG. 2 is a schematic flow chart illustrating another embodiment of a foreign object detection method provided herein;
FIG. 3 is a schematic diagram of an embodiment of heights of a target object at different distances from an image capture device according to the present disclosure;
FIG. 4 is a schematic flowchart illustrating an embodiment of step S12 shown in FIG. 1;
FIG. 5 is a flowchart illustrating an embodiment of step S13 shown in FIG. 1;
FIG. 6 is a schematic structural diagram of an embodiment of a foreign object detection apparatus provided in the present application;
FIG. 7 is a schematic structural diagram of an embodiment of a computer-readable storage medium provided in the present application.
Detailed Description
The following describes in detail the embodiments of the present application with reference to the drawings attached hereto.
In the following description, for purposes of explanation and not limitation, specific details are set forth such as particular system structures, interfaces, techniques, etc. in order to provide a thorough understanding of the present application.
The term "and/or" herein is merely an association describing an associated object, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship. Further, the term "plurality" herein means two or more than two. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of a, B, and C, and may mean including any one or more elements selected from the group consisting of a, B, and C.
Referring to fig. 1, fig. 1 is a schematic flow chart of an embodiment of a foreign object detection method provided in the present application. It should be noted that, if the result is substantially the same, the flow sequence shown in fig. 1 is not limited in this embodiment. As shown in fig. 1, the present embodiment includes:
step S11: and acquiring a target image.
The method of the embodiment is used for detecting foreign matters in a target image to determine whether a target object carries the target object when the target object is on a target device, wherein the foreign matter detection is detection of whether the target object carries the target object when the target object is on the target device. In an embodiment, the target image may be any image that needs to be subjected to foreign object detection, and may be obtained from a local storage or a cloud storage. It is to be understood that, in other embodiments, the target image may also be acquired by the image acquisition device in real time, and is not limited herein.
The target area of the target image comprises the target device and the target object. The foreign object detection is performed on the target area of the target image, so as to determine whether the target object included in the target area of the target image carries the target object when the target object is on the target device included in the target area of the target image, so that the target device and the target object included in the target area of the target image are the subjects of the foreign object detection, the foreign object detection is only required when the target device and the target object are included in the target area of the target image at the same time, and the foreign object detection is not required when the target area of the target image only includes one of the target device and the target object, or when the target device and the target object are not included in the target area of the target image, that is, the subject of the foreign object detection is not included in the target area of the target image.
In an embodiment, as shown in fig. 2, fig. 2 is a schematic flowchart of another example of the foreign object detection method provided in this application, where the target device is a monkey vehicle, the target object is a person, and then foreign object detection is performed on a target area including a target image of the monkey vehicle and the person to determine whether the person carries the target object when the person takes the monkey vehicle. The specific type of the target object is not limited, and the target object can be specifically set according to actual use requirements. For example, the target object is prohibited to carry when taking a monkey vehicle, the monkey vehicle is a common simple vehicle under a coal mine, is mainly used for assisting transportation workers of underground miners, and aims to shorten the time consumed by the miners in the process of going up and down the well, reduce the consumption of the miners in the process of going up and down the well and improve the effective working time of the miners; however, when a miner gets on or off duty, the miner may carry prohibited articles when taking the monkey vehicle, so that the load of the monkey vehicle is increased, the service life of the monkey vehicle is also shortened, and in severe cases, a rope for transporting the monkey vehicle may be broken; so, through the follow-up foreign matter detection to the target area who contains monkey car and people to and confirm in time whether the people carries forbidden contraband that carries when taking the monkey, on the one hand, can in time manage and control the action of taking the monkey car carrying contraband, to the use of normalized miner to the monkey car, on the other hand reduces the load of monkey car, has improved the life of monkey car.
It is understood that in other embodiments, the target device may also be an automobile, an elevator, a bicycle, a loading vehicle, etc., and is not specifically limited herein. It is understood that in other embodiments, the target object may be an animal or other object, and is not limited in particular.
Step S12: in response to the target area of the target image including the target object, a degree of overlap between the target device, the target object, and the target object is determined.
Since the target object itself does not exist when the target area of the target image does not include the target object, and it is more unlikely that the target object carries the target object when the target device is located on the target device, in this embodiment, the foreign object detection is performed on the target area of the target image in response to the target area of the target image including the target object. That is, when the detection determines that the target object is included in the target region of the target image, the foreign object detection is performed on the target region of the target image.
In addition, when the target device, the target object and the target object included in the target area of the target image overlap with each other, there is a certain probability that the target object carries the target object when the target object is on the target device. Therefore, in the present embodiment, when the target area of the target image includes the target object, the foreign object detection performed on the target area of the target image is specifically to determine the degree of overlap between the target device, the target object, and the target object. That is to say, when it is detected that the target area of the target image includes the target object, the degree of overlap between the target device, the target object, and the target object is determined, so as to determine whether the target object carries the target object on the target device subsequently according to the degree of overlap between the target device, the target object, and the target object.
In one embodiment, the degree of overlap between the target device, the target object, and the target object may be determined by the degree of overlap between the target object, the target device, and the detection frame corresponding to the target object. It is to be understood that, in other embodiments, the degree of overlap between the target device, the target object, and the target object may also be determined directly through the areas of the target device, the target object, and the target object themselves, or may also be determined in other manners, and is not particularly limited herein.
In order to improve the accuracy of the final foreign object carrying result obtained subsequently, in one embodiment, before determining the degree of overlap among the target device, the target object and the target object in response to the target area of the target image including the target object, determining a detection area in the target area, wherein the detection area includes the target device and the target object; at this time, in response to the detection area of the target area including the target object, a degree of overlap between the target device, the target object, and the target object is determined. That is, the detection region is defined, the size of the detection region is smaller than that of the target region, and the foreign object detection is performed on the detection region in the target region, so that the accuracy of the foreign object detection can be improved.
Step S13: and responding to the overlapping degree meeting the overlapping requirement, and acquiring the depth information of the target equipment, the target object and the target object.
In the plane coordinate system, the categories at different positions may be misjudged to be correlated due to the fact that the image depth of field may cause misalignment and overlapping, that is, a phenomenon that a target object is on the target device and carries the target object occurs. Therefore, in the present embodiment, the depth information of the target device, the target object, and the target object is acquired in response to the degree of overlap satisfying the overlap requirement. That is to say, when it is determined that the degree of overlap between the target device, the target object, and the target object satisfies the overlap requirement, that is, when it is suspected that the target object carries the target object on the target device, it is necessary to further perform foreign object detection on the target area of the target image, specifically, it is further determined whether the target object carries the target object on the target device according to the depth information of the target device, the target object, and the target object, so as to improve the accuracy of a final foreign object carrying result and reduce the occurrence of false detection.
The overlapping requirement is not limited, and the overlapping requirement can be specifically set according to the actual use requirement. In one embodiment, the degree of overlap between the target device, the target object, and the target object is an intersection ratio of corresponding areas of the target device, the target object, and the corresponding overlap requirement may be that the intersection ratio is greater than or equal to an overlap threshold, for example, the overlap threshold is 0.6, 0.7, and the like. For example, taking the target device as a monkey vehicle, the target object as a miner, the target object as a shovel, the overlapping degree of the target device, the target object and the target object as the intersection ratio of the target device, the target object and the target object corresponding to the area, and the overlapping requirement as the intersection ratio being greater than or equal to 0.6 as an example; when the intersection ratio of the areas corresponding to the monkey vehicle, the miner and the shovel is determined to be 0.7, the intersection ratio of the areas corresponding to the monkey vehicle, the miner and the shovel is more than 0.6, namely the overlapping requirement is met, the suspected miner is indicated to carry the shovel to take the monkey vehicle, and at the moment, in order to further determine whether the miner carries the shovel to take the monkey vehicle, the depth information of the monkey vehicle, the miner and the shovel is respectively obtained; and when the intersection ratio of the areas corresponding to the monkey vehicle, the miner and the shovel is determined to be 0.3, the intersection ratio of the areas corresponding to the monkey vehicle, the miner and the shovel is less than 0.6, namely the overlapping requirement is not met, and the fact that the miner does not carry the shovel to take the monkey vehicle or the fact that any two or three of the monkey vehicle, the miner and the shovel do not have the association relationship is indicated.
Since the corresponding heights of the reference objects (e.g., the target device, the target object, etc.) are different at different distances from the image capturing device, in an embodiment, the depth information includes the distance from the image capturing device, and the depth information of the target device, the target object, and the target object can be determined according to the specific height information of the target device, the target object, and the target object in the target image, that is, the distance between the target device, the target object, and the target object and the image capturing device.
Since the shapes of the target device, the target object, and the target object are different, in a specific embodiment, the distances from the target device, the target object, and the target object to the image capture device may be determined by the height information of the detection frames having regular shapes corresponding to the target device, the target object, and the target object. For example, as shown in fig. 3, fig. 3 is a schematic view of an embodiment of heights of a target object corresponding to different distances from an image acquisition device, where the target object is a miner, and the height of a detection frame of the miner is H 3 In the meantime, the distance between the miners and the image acquisition device (i.e., the camera in fig. 2) is S 3 (ii) a The height of the detection frame of the miner is H 2 The distance between the miner and the image acquisition device is S 2 (ii) a The height of the detection frame of the miner is H 1 The distance between the miner and the image acquisition device is S 1 。
Step S14: and obtaining an initial foreign matter carrying result based on the depth information of the target equipment, the target object and the target object.
In the present embodiment, the initial foreign object carrying result is obtained based on the depth information of the target device, the target object, and the target object. When the target object is suspected to be carried on the target device based on the overlapping degree of the target device, the target object and the target object, whether the target object carries the target object or not is further determined based on the depth information of the target device, the target object and the target object, so that an initial foreign object carrying result is obtained, namely whether the target object carries the target object or not is primarily determined, the situation that false detection is caused due to dislocation overlapping caused by depth of field of an image is reduced, and the accuracy of foreign object detection is improved.
In an embodiment, the depth information includes a distance from the image capturing device, and the determining to obtain the initial foreign object carrying result based on the depth information of the target device, the target object, and the target object specifically includes: and determining that the target object carries the target object when the target object is on the target device in response to that the absolute values of the distance difference between any two of the target device, the target object and the target object are less than or equal to a distance difference threshold. That is, when the absolute value of the distance difference between any two of the target device, the target object, and the target object is less than or equal to the distance difference threshold, it indicates that there is actually an overlap between any two of the target device, the target object, and the target object, that is, there is a correlation between any two of the target device, the target object, and the target object, rather than a misalignment overlap caused by the depth of field of the image, and therefore it can be determined that the target object carries the target object when the target device is on the target device; when the absolute value of the distance difference between any two of the target device, the target object and the target object is greater than the difference threshold, it is indicated that the fact that the distance difference between the two corresponding to the distance difference greater than the difference threshold is dislocation overlap caused by the depth of field of the image, but the fact that the distance difference between the two is actually overlapped is not true, namely the distance difference between the two is not actually related. The specific formula of the absolute value of the distance difference between any two of the target device, the target object and the target object is as follows: s dis ＝|S a -S b |
Wherein S is dis An absolute value representing a difference in distance between any two of the target device, the target object, and the target object; s a Represents any one of a target device, a target object, and a target object; s b Indicating a difference from S in the target device, target object and target object a Any of the above.
Step S15: based on the initial foreign object carrying result, a final foreign object carrying result of the target object is determined.
In order to improve the accuracy of foreign object detection, in the present embodiment, the final foreign object carrying result of the target object is further determined based on the initial foreign object carrying result.
In one embodiment, the initial foreign object carrying result may be directly used as the final foreign object carrying result of the target object. In other embodiments, the final foreign object carrying result of the target object may also be determined based on the initial foreign object carrying results corresponding to the preset number of historical frames; that is to say, the initial foreign object carrying results corresponding to the preset number of historical image frames are combined to finally determine whether the target object carries the target object when the target object is on the target device, so that the accuracy of foreign object detection is improved. The preset number is not limited, and the preset number can be specifically set according to actual use requirements.
In a specific embodiment, in response to that the initial foreign object carrying results corresponding to the preset number of history frames are all target objects carried when the target object is on the target device, it is determined that the final foreign object carrying result is that the target object carries the target object when the target object is on the target device. That is to say, when the initial foreign object carrying results corresponding to the preset number of history frames are that the target object carries the target object when the target object is on the target device, it is determined that the target object carries the target object when the target object is on the target device. In other specific embodiments, in response to that the number of frames in the preset number of history frames, in which the initial foreign object carrying result is that the target object carries the target object when the target object is on the target device, satisfies a preset ratio, it is determined that the final foreign object carrying result is that the target object carries the target object when the target object is on the target device. That is to say, when the initial foreign object carrying result in the initial foreign object carrying results of the preset number of history frames is that the number of frames carrying the target object when the target object is on the target device satisfies the preset proportion, it is determined that the target object carries the target object when the target object is on the target device. The size of the preset proportion is not limited, and the preset proportion can be specifically set according to actual use requirements. For example, taking the preset proportion as 80% and the preset number of image frames as 10 frames as an example, if the initial foreign matter carrying result in the initial foreign matter carrying result corresponding to the 10 frame image frames is that the number of frames carrying the target object when the target object is on the target device is 9 frames, the proportion of the number of frames carrying the target object when the target object is on the target device is 90%, and if the proportion is greater than the preset proportion, the preset proportion is satisfied, and it is determined that the target object carries the target object when the target object is on the target device.
In an embodiment, the initial foreign object carrying results of a preset number of history image frames are the initial foreign object carrying results of consecutive history image frames, that is, the foreign object detection is performed on each image frame acquired by the image acquisition device, so as to improve the accuracy of the foreign object detection. In order to improve the efficiency of foreign object detection and reduce the amount of calculation, in other embodiments, the initial foreign object carrying results of a preset number of historical image frames are the initial foreign object carrying results of consecutive and spaced image frames, that is, frame skipping foreign object detection is performed on the image frames acquired by the image acquisition device.
In an embodiment, after a final foreign object carrying result of a target object is determined based on an initial foreign object carrying result corresponding to a preset number of historical frames, in response to that the final foreign object carrying result is that the target object carries the target object when being on a target device, prompt information for prompting that the target object carries the target object when being on the target device is sent to a user terminal, so that a user can monitor and process the behavior of the target object in time. In other embodiments, after determining that the target object carries the target object while being on the target device, the alarm information may be sent to control the alarm to sound an alarm or control the alarm to flash, and to inform the target object that the current use of the target device is abnormal.
In the above embodiment, when it is determined that the degree of overlap between the target device, the target object, and the target object satisfies the overlap requirement, it is determined whether the target object carries the target object when the target object is on the target device based on the depth information of the target device, the target object, and the target object. Therefore, when the overlapping degree of the target device, the target object and the target object is determined to meet the overlapping requirement, whether the target object carries the target object when the target object is on the target device is further determined according to the depth information of the target device, the target object and the target object, the occurrence of false detection caused by dislocation overlapping caused by the depth of field of the image is reduced, and the accuracy of foreign matter detection is improved.
Referring to fig. 4, fig. 4 is a schematic flowchart illustrating an embodiment of step S12 shown in fig. 1. It should be noted that, if the result is substantially the same, the flow sequence shown in fig. 4 is not limited in this embodiment. As shown in fig. 4, in this embodiment, the specifically includes, by the degree of overlap between the target object, the target device, and the detection frame corresponding to the target object:
step S121: and respectively acquiring a first detection frame of the target device, a second detection frame of the target object and a third detection frame of the target object.
In this embodiment, the first detection frame of the target device, the second detection frame of the target object, and the third detection frame of the target object are respectively obtained, so that the overlapping degree between the target device, the target object, and the target object is determined according to the overlapping degree between the first detection frame of the target device, the second detection frame of the target object, and the third detection frame of the target object in the following.
In one embodiment, the target image may be detected by using a target detection algorithm to obtain a first detection frame of the target device, a second detection frame of the target object, and a third detection frame of the target object.
It is to be understood that, in other embodiments, the target image may also be processed by using the target detection model to obtain the first detection frame of the target device, the second detection frame of the target object, and the third detection frame of the target object, which is not limited herein. The training process of the target detection model specifically comprises the following steps: firstly, obtaining a sample image, and carrying out data annotation on the sample image by using a data annotation tool; the sample image is marked with a sample device, a sample object and a label of the sample object, the label corresponding to the sample device is set to be 0, the label corresponding to the sample object is set to be 1, the label corresponding to the sample object is set to be 2, 3, 4, 5, 6 \8230, the structure is 8230, and the like, and is determined according to the type of the sample object; in addition, a first sample detection frame of the sample equipment, a second sample detection frame of the sample object, a third sample detection frame of the sample object and parameter information of each sample detection frame are marked on the sample image, and the parameter information comprises the central position of the sample detection frame and the width and the height of the sample detection frame; for example, the annotation data record corresponding to the sample device on the a sample image is {0, x, y, w, h }, where 0 represents that the piece of annotation data record is the annotation data record of the sample device, { x, y } represents the center position of the first sample detection box of the sample device, and { w, h } represents the width and height of the first sample detection box of the sample device. Secondly, training the target detection model by using each sample image in the marked sample image set until the training is converged to finish the training of the target detection model.
Step S122: and determining the overlapping degree among the target equipment, the target object and the target object based on the overlapping degree among the first detection frame, the second detection frame and the third detection frame.
In the present embodiment, the degree of overlap between the target device, the target object, and the target object is determined based on the degree of overlap between the first detection frame, the second detection frame, and the third detection frame. Because the shapes of the target device, the target object and the target object are different, the calculation of the overlapping degree of the target device, the target object and the target object is complex and the calculation amount is large directly based on the corresponding areas of the target device, the target object and the target object, and the foreign matter detection efficiency can be reduced; and because the first detection frame is the minimum circumscribed rectangle of the target device, the second detection frame is the minimum circumscribed rectangle of the target object, and the third detection frame is the minimum circumscribed rectangle of the target object, and the shapes of the circumscribed rectangles corresponding to the target device, the target object and the target object are regular, the overlapping degree among the target device, the target object and the target object is determined based on the overlapping degree among the first detection frame, the second detection frame and the third detection frame, so that the calculation amount is reduced, and the efficiency of determining the overlapping degree among the target device, the target object and the target object is improved.
In one embodiment, the degree of overlap between the first detection frame of the target device, the second detection frame of the target object, and the third detection frame of the target object is directly set as the degree of overlap between the target device, the target object, and the target object. In a specific embodiment, the degree of overlap between the first detection frame of the target device, the second detection frame of the target object, and the third detection frame of the target object is an intersection ratio of the first detection frame of the target device, the second detection frame of the target object, and the third detection frame of the target object. It is to be understood that, in other specific embodiments, the degree of overlap between the first detection frame of the target device, the second detection frame of the target object, and the third detection frame of the target object may also be calculated by other manners, which is not limited herein. The specific formula for calculating the intersection ratio of the first detection frame of the target device, the second detection frame of the target object and the third detection frame of the target object is as follows:
the IOU represents the intersection ratio of a first detection frame of the target equipment, a second detection frame of the target object and a third detection frame of the target object, and the value of the IOU is generally a value between 0 and 1; r c A first detection box representing a target device; r p A second detection box representing a target object; r is s A third detection frame representing the target. It should be noted that the larger the IOU is, the higher the coincidence degree between the target device, the target object, and the target object is, that is, it indicates that there is a high probability that the target object carries the target object when the target device is on the target device.
Referring to fig. 5, fig. 5 is a schematic flowchart illustrating an embodiment of step S13 shown in fig. 1. It should be noted that, if the result is substantially the same, the flow sequence shown in fig. 5 is not limited in this embodiment. As shown in fig. 5, in this embodiment, the depth information is a distance from the image capturing apparatus, and the determining the distances from the target device, the target object, and the target object to the image capturing apparatus according to the specific height information of the target device, the target object, and the target object in the target image specifically includes:
step S131: and respectively acquiring a first height of the target device, a second height of the target object and a third height of the target object.
In this embodiment, a first height of the target device, a second height of the target object, and a third height of the target object are obtained, respectively. Since the shapes of the target device, the target object and the target object are different and irregular, in one embodiment, the height information of the detection frame with a regular shape corresponding to the target device, the target object and the target object can be used as the first height of the target device, the second height of the target object and the third height of the target object; at this time, the first height of the target device, the second height of the target object, and the third height of the target object are specifically heights of the corresponding detection frames.
Step S132: and acquiring a depth mapping model.
In this embodiment, a depth mapping model is obtained, where the depth mapping model is a logistic regression algorithm model, and specifically is a linear relationship model between the height of a reference object (e.g., a target device, a target object, or a target object) and the distance between the reference object and an image acquisition device.
Taking the height of the detection frame of the reference object actually required to be obtained as an example, the training process of the depth mapping model specifically comprises the following steps: firstly, obtaining a sample image, and carrying out data annotation on the sample image by using a data annotation tool; the sample image is marked with a sample device, a sample object and a label of the sample object, the label corresponding to the sample device is set to be 0, the label corresponding to the sample object is set to be 1, the label corresponding to the sample object is set to be 2, 3, 4, 5, 6 \8230, the structure is 8230, and the like, and is determined according to the type of the sample object; the height of the sample detection frame of the sample device, the distance between the sample device and the image capturing device, the height of the sample detection frame of the sample object, the distance between the sample object and the image capturing device, the height of the sample detection frame of the sample object, and the distance between the sample object and the image capturing device are marked on the sample image. Secondly, training the depth mapping model by using each sample image in the marked sample image set until the depth mapping model is converged so as to complete the training of the depth mapping model. It should be noted that, the heights of different types and the corresponding relationships between the heights of the different types and the distances between the heights of the different types and the image acquisition devices are different, and in the application process of the subsequent depth mapping model, the specific type of the reference object is determined to determine the linear relationship between the height of the reference object and the distance between the reference object and the image acquisition device, and then the distance between the reference object and the image acquisition device is accurately determined according to the height information of the reference object.
Step S133: and respectively inputting the first height, the second height and the third height into the depth mapping model to respectively obtain a first distance between the target equipment and the image acquisition device, a second distance between the target object and the image acquisition device and a third distance between the target object and the image acquisition device.
In this embodiment, the first height, the second height, and the third height are respectively input into the depth mapping model to respectively obtain a first distance between the target device and the image capturing apparatus, a second distance between the target object and the image capturing apparatus, and a third distance between the target object and the image capturing apparatus. Specifically, the first height of the target device, the second height of the target object and the third height of the target object are input into the depth mapping model, and since the target device, the target object and the target object are in different categories, the depth mapping model respectively determines a linear relationship between the height of the target device and a distance between the target device and the image acquisition device, a linear relationship between the height of the target object and a distance between the target object and the image acquisition device, and then determines and obtains a distance between the target device and the image acquisition device according to the relationship between the height and the distance between the target object and the image acquisition device.
The first distance between the target device and the image acquisition device is used as the depth information of the target device, the second distance between the target object and the image acquisition device is used as the depth information of the target object, and the third distance between the target object and the image acquisition device is used as the depth information of the target object.
Referring to fig. 6, fig. 6 is a schematic structural diagram of an embodiment of a foreign object detection apparatus provided in the present application. The foreign object detection apparatus 60 comprises a memory 61 and a processor 62 coupled to each other, the processor 62 being configured to execute program instructions stored in the memory 61 to implement the steps of any of the above-described embodiments of the foreign object detection method. In one particular implementation scenario, the foreign object detection device 60 may include, but is not limited to: a microcomputer, a server, and in addition, the foreign object detection device 60 may further include a mobile device such as a notebook computer, a tablet computer, and the like, which is not limited herein.
Specifically, the processor 62 is configured to control itself and the memory 61 to implement the steps of any of the above-described foreign object detection method embodiments. Processor 62 may also be referred to as a CPU (Central Processing Unit). The processor 62 may be an integrated circuit chip having signal processing capabilities. The Processor 62 may also be a general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. In addition, the processor 62 may be collectively implemented by an integrated circuit chip.
Referring to fig. 7, fig. 7 is a schematic structural diagram of an embodiment of a computer-readable storage medium provided in the present application. The computer readable storage medium 70 of the embodiments of the present application stores program instructions 71, and the program instructions 71, when executed, implement the methods provided by any of the embodiments of the foreign object detection method of the present application and any non-conflicting combinations. The program instructions 71 may form a program file stored in the computer readable storage medium 70 in the form of a software product, so as to enable a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the methods according to the embodiments of the present application. And the aforementioned computer-readable storage medium 70 includes: various media capable of storing program codes, such as a usb disk, a portable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, or terminal devices such as a computer, a server, a mobile phone, and a tablet.
If the technical scheme of the application relates to personal information, a product applying the technical scheme of the application clearly informs personal information processing rules before processing the personal information, and obtains personal independent consent. If the technical scheme of the application relates to sensitive personal information, a product applying the technical scheme of the application obtains individual consent before processing the sensitive personal information, and simultaneously meets the requirement of 'express consent'. For example, at a personal information collection device such as a camera, a clear and significant identifier is set to inform that the personal information collection range is entered, the personal information is collected, and if the person voluntarily enters the collection range, the person is regarded as agreeing to collect the personal information; or on the device for processing the personal information, under the condition of informing the personal information processing rule by using obvious identification/information, obtaining personal authorization by modes of popping window information or asking a person to upload personal information of the person by himself, and the like; the personal information processing rule may include information such as a personal information processor, a personal information processing purpose, a processing method, and a type of personal information to be processed.
The above description is only for the purpose of illustrating embodiments of the present application and is not intended to limit the scope of the present application, and all modifications of equivalent structures and equivalent processes, which are made by the contents of the specification and the drawings of the present application or are directly or indirectly applied to other related technical fields, are also included in the scope of the present application.
Claims (10)
1. A foreign object detection method, characterized in that the method comprises:
acquiring a target image; wherein the target area of the target image comprises a target device and a target object;
in response to the target area further comprising a target object, determining a degree of overlap between the target device, the target object, and the target object;
responding to the overlapping degree meeting the overlapping requirement, and acquiring depth information of the target equipment, the target object and the target object;
obtaining an initial foreign matter carrying result based on the depth information of the target equipment, the target object and the target object;
determining a final foreign object carrying result of the target object based on the initial foreign object carrying result; wherein the final foreign object carrying result is used for indicating whether the target object carries the target object when the target object is on the target device.
2. The method of claim 1, wherein the determining a degree of overlap between the target device, the target object, and the target object comprises:
respectively acquiring a first detection frame of the target device, a second detection frame of the target object and a third detection frame of the target object;
determining a degree of overlap between the target device, the target object, and the target object based on a degree of overlap between the first detection frame, the second detection frame, and the third detection frame; wherein the overlapping degree among the first detection frame, the second detection frame and the third detection frame is the intersection ratio of the first detection frame, the second detection frame and the third detection frame.
3. The method of claim 1, wherein the obtaining depth information of the target device, the target object, and the target object comprises:
respectively obtaining a first height of the target device, a second height of the target object, a third height of the target object and a depth mapping model;
inputting the first height, the second height and the third height into the depth mapping model respectively to obtain a first distance between the target device and the image acquisition device, a second distance between the target object and the image acquisition device and a third distance between the target object and the image acquisition device respectively; wherein a first distance between the target device and the image acquisition apparatus is used as the depth information of the target device, a second distance between the target object and the image acquisition apparatus is used as the depth information of the target object, and a third distance between the target object and the image acquisition apparatus is used as the depth information of the target object.
4. The method of claim 3, wherein the deriving an initial foreign object carrying result based on depth information of the target device, the target object, and the target object comprises:
and determining that the target object carries the target object when the target object is on the target device in response to that the absolute values of the distance differences between any two of the target device, the target object and the target object are less than or equal to a preset distance difference threshold.
5. The method of claim 1, wherein said determining a final foreign-object-entrainment result for the target object based on the initial foreign-object-entrainment result comprises:
and in response to that the initial foreign matter carrying result corresponding to the current frame is that the target object carries the target object when the target object is on the target device, determining a final foreign matter carrying result of the target object based on the initial foreign matter carrying results corresponding to a preset number of historical frames.
6. The method according to claim 5, wherein the determining a final foreign object carrying result of the target object based on the initial foreign object carrying results corresponding to a preset number of historical frames comprises: in response to that the initial foreign matter carrying results corresponding to the preset number of historical frames are that the target object carries the target object when the target object is on the target device, determining that the final foreign matter carrying result is that the target object carries the target object when the target object is on the target device; or,
and determining that the final foreign matter carrying result is that the target object carries the target object when the target object is on the target device in response to that the number of frames, carrying the target object when the initial foreign matter carrying result in the preset number of historical frames is that the target object carries the target object when the target object is on the target device, meets a preset proportion.
7. The method of claim 5, wherein after determining a final foreign object carrying result of the target object based on the initial foreign object carrying result corresponding to a preset number of history frames, the method further comprises:
and responding to the fact that the target object carries the target object when the target object is determined to be on the target device, and sending prompt information for prompting that the target object carries the target object to a user terminal.
8. The method of claim 1, wherein prior to the determining that the target region responsive to the target image further includes a target object, the method further comprises:
determining a detection area in the target area; wherein the detection area comprises the target device and the target object;
the determining a degree of overlap between the target device, the target object, and the target object in response to the target region further including a target object, includes:
determining a degree of overlap between the target device, the target object, and the target object in response to a detection region of the target region further including the target object; and/or the presence of a gas in the gas,
the target device comprises a monkey vehicle, and the target object comprises a person.
9. A foreign object detection apparatus, characterized in that the foreign object detection apparatus comprises a memory storing program instructions and a processor for executing the program instructions to implement the foreign object detection method according to any one of claims 1 to 8.
10. A computer-readable storage medium for storing program instructions executable to implement the foreign object detection method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210965625.6A CN115393791A (en) | 2022-08-11 | 2022-08-11 | Foreign matter detection method, device and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210965625.6A CN115393791A (en) | 2022-08-11 | 2022-08-11 | Foreign matter detection method, device and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115393791A true CN115393791A (en) | 2022-11-25 |
Family
ID=84118826
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210965625.6A Pending CN115393791A (en) | 2022-08-11 | 2022-08-11 | Foreign matter detection method, device and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115393791A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117894211A (en) * | 2023-12-28 | 2024-04-16 | 长沙北斗产业安全技术研究院股份有限公司 | Unmanned aerial vehicle countering identification method and system based on flight track |
-
2022
- 2022-08-11 CN CN202210965625.6A patent/CN115393791A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117894211A (en) * | 2023-12-28 | 2024-04-16 | 长沙北斗产业安全技术研究院股份有限公司 | Unmanned aerial vehicle countering identification method and system based on flight track |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| German et al. | Rapid entropy-based detection and properties measurement of concrete spalling with machine vision for post-earthquake safety assessments | |
| CN110807429A (en) | Construction safety detection method and system based on tiny-YOLOv3 | |
| CN110889339B (en) | Head and shoulder detection-based dangerous area grading early warning method and system | |
| CN113688965B (en) | Automatic storage code scanning detection method and cargo management system | |
| CN115393791A (en) | Foreign matter detection method, device and computer readable storage medium | |
| Kim et al. | Sequential pattern learning of visual features and operation cycles for vision-based action recognition of earthmoving excavators | |
| CN112991807B (en) | Parking space state detection method, device, equipment and storage medium | |
| CN114648748A (en) | Motor vehicle illegal parking intelligent identification method and system based on deep learning | |
| CN114241370A (en) | Intrusion identification method and device based on digital twin transformer substation and computer equipment | |
| US10140555B2 (en) | Processing system, processing method, and recording medium | |
| CN115995056A (en) | Automatic bridge disease identification method based on deep learning | |
| CN113807228A (en) | Parking event prompting method and device, electronic equipment and storage medium | |
| CN110969898A (en) | Ship height detection method, device and system | |
| CN110634120B (en) | Vehicle damage judgment method and device | |
| CN115298705A (en) | License plate recognition method and device, electronic equipment and storage medium | |
| CN112607542B (en) | Elevator control method, elevator control device, computer equipment and storage medium | |
| CN115311630A (en) | Method and device for generating distinguishing threshold, training target recognition model and recognizing target | |
| CN114841255A (en) | Detection model training method, device, equipment, storage medium and program product | |
| US20230072596A1 (en) | Human body detection device | |
| CN114332695A (en) | Method and device for identifying opening and closing of elevator door and storage medium | |
| CN115713758B (en) | Carriage identification method, system, device and storage medium | |
| CN111709360A (en) | Safety rope wearing identification method and system | |
| CN112818847A (en) | Vehicle detection method, device, computer equipment and storage medium | |
| CN116881831B (en) | Road identification method, system and storage medium based on remote sensing technology | |
| WO2023188292A1 (en) | Detection device, detection system, detection method, and non-transitory computer-readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |