CN111914659A - Article detection method, device, equipment and medium - Google Patents
Article detection method, device, equipment and medium Download PDFInfo
- Publication number
- CN111914659A CN111914659A CN202010641007.7A CN202010641007A CN111914659A CN 111914659 A CN111914659 A CN 111914659A CN 202010641007 A CN202010641007 A CN 202010641007A CN 111914659 A CN111914659 A CN 111914659A
- Authority
- CN
- China
- Prior art keywords
- illegal
- image
- determining
- moving
- item
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 68
- 238000003062 neural network model Methods 0.000 claims abstract description 61
- 238000000034 method Methods 0.000 claims abstract description 49
- 238000004590 computer program Methods 0.000 claims description 14
- 238000012549 training Methods 0.000 claims description 14
- 230000008569 process Effects 0.000 abstract description 22
- 238000010586 diagram Methods 0.000 description 13
- 238000004891 communication Methods 0.000 description 8
- 238000012545 processing Methods 0.000 description 6
- 238000012986 modification Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000013256 Gubra-Amylin NASH model Methods 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000009194 climbing Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000011895 specific detection Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/12—Acquisition of 3D measurements of objects
Abstract
The invention discloses a method, a device, equipment and a medium for detecting articles, which are used for solving the problems of missing detection and false detection of the existing article detection and incapability of accurately identifying large and small pieces of luggage. The method comprises the following steps: the method comprises the steps of determining a moving object and the size of the moving object in an image according to each frame of image collected by a binocular camera, determining whether the moving object is a first illegal object according to the size of the moving object, obtaining a second illegal object existing in each frame of image through a trained neural network model, and controlling an alarm device to give an alarm when a target illegal object determined according to the first illegal object and the second illegal object enters a preset area. According to the invention, the size of the moving object in the image acquired by the binocular camera is detected, and the object is detected by the neural network model, so that the target illegal object is determined, the phenomena of missing detection and false detection in the detection process are effectively avoided, and the accuracy of illegal object detection is further improved.
Description
Technical Field
The invention relates to the technical field of image processing and intelligent monitoring, in particular to a method, a device, equipment and a medium for detecting articles.
Background
With the continuous development of society, the living standard of people is gradually improved, and the construction of public transportation places such as stations, subways and the like which are convenient for people to go out is also gradually increased. In order to meet the requirements of daily shopping of people, staircase facilities are mostly provided in places such as stations, subways and shopping malls to facilitate people to go out for shopping, however, accidents are frequently caused by illegal staircase climbing of baby carriages, large luggage, shopping carts and the like, and some people still have a lucky psychology although security reminding marks are arranged in shopping malls and supermarkets. The potential safety hazard existing on the escalators of the baby carriage, the shopping cart and the large luggage is mainly that the escalators have a certain gradient, the escalator operation has certain inertia, if illegal articles such as the baby carriage are detected and identified at the escalator entrance, when the possibility that the illegal articles exist on the escalators is found, the escalator operation is stopped, and the dangerous events are prevented and reduced to a certain extent.
In the prior art, whether an illegal article exists is determined through article detection, and a specific detection method comprises the following steps:
the method comprises the steps of obtaining an image to be identified generated by scanning an object to be detected by a security inspection machine, converting red, green and blue (RGB) values of pixel points in a target area image into HSV values, converting the target area image into a gray-scale image, and inputting the gray-scale image into an object detection model to identify whether each object to be detected belongs to an illegal object. Because the method adopts color features and carries out recognition based on the gray level image and the BP neural network model, false detection and missing detection are easy to occur based on the training method, and the method can not recognize large pieces of luggage.
Identifying based on a generative countermeasure network (GAN) model, wherein the GAN model comprises: and segmenting the network model and judging the network model to obtain first image characteristic information and second image characteristic information, wherein the judging network model obtains a judging result based on the first image characteristic information, the second image characteristic information and the article image. Because the method adopts the simple neural network to learn and recognize, the actual size of the target cannot be accurately estimated, the size information of the illegal object is not fully utilized, and the size of the luggage cannot be distinguished while the false detection is effectively filtered.
Therefore, when detecting an illegal article, the conventional technology has risks of missing detection and false detection to a certain extent, and can not effectively identify a large piece of luggage with risks.
Disclosure of Invention
The invention provides a method, a device, equipment and a medium for detecting an article, which are used for solving the problems that illegal articles are easy to miss detection and false detection and large luggage cannot be identified in the prior art.
In a first aspect, the present invention provides a method of item detection, the method comprising:
determining moving objects in the images according to each frame of image acquired by the binocular camera, and determining the size of each moving object; determining each first offending item according to the determined size of each moving item;
acquiring a second illegal article existing in any input frame of image through the trained neural network model;
and if the target illegal object determined according to the first illegal object and the second illegal object is identified to be in a preset area, controlling an alarm device to give an alarm.
Further, the determining the size of each moving object comprises:
and determining the size of the image moving object according to the physical distance between each pixel point in the image acquired by the corrected binocular camera and the height from the ground.
Further, the determining each first offending item based on the determined size of each moving item includes:
and for each moving object, if the volume of the moving object is larger than a preset volume threshold value, determining that the moving object is a first illegal object.
Further, the determining each first offending item based on the determined size of each moving item includes:
and for each moving article, if the height of the moving article is within any preset height range of the illegal articles, determining that the moving article is the first illegal article.
Further, the neural network model is trained by:
in a second aspect, the present invention also provides an article detection apparatus, the apparatus comprising:
the determining module is used for determining moving objects in the images according to each frame of image acquired by the binocular camera and determining the size of each moving object; determining each first offending item according to the determined size of each moving item;
the recognition module is used for acquiring a second illegal article existing in any input frame of image through the trained neural network model;
and the control module is used for controlling an alarm device to give an alarm if the target illegal object determined according to the first illegal object and the second illegal object is identified to be in a preset area.
Further, the determining module is specifically configured to determine the size of the moving object of the image according to the physical distance between each pixel point in the image acquired by the corrected binocular camera and the height from the ground.
Further, the determining module is specifically configured to, for each moving object, determine that the moving object is a first illegal object if the volume of the moving object is greater than a preset volume threshold.
Further, the determining module is specifically configured to determine, for each moving object, that the moving object is the first illegal object if the height of the moving object is within any preset height range of the illegal object.
Further, the device also comprises:
the training module is used for acquiring any sample image in a sample set, first position information of each illegal item contained in the sample image and first identification information of the illegal item contained in the first position information; inputting the sample image into an original neural network model, and acquiring second position information of each illegal item contained in the sample image and second identification information of the illegal item contained in the second position information; and training the original neural network model according to the first position information, the second position information, the first identification information and the corresponding second identification information.
In a third aspect, the present invention also provides an electronic device, which at least comprises a processor and a memory, wherein the processor is configured to implement the steps of the article detection method according to any one of the above when executing the computer program stored in the memory.
In a fourth aspect, the present invention also provides a computer-readable storage medium storing a computer program which, when executed by a processor, implements the steps of the article detection method as described in any one of the above.
The embodiment of the invention provides a method, a device, equipment and a medium for detecting articles, wherein the method comprises the following steps: the method comprises the steps of determining a moving object and the size of the moving object in an image according to each frame of image collected by a binocular camera, determining whether the moving object is a first illegal object according to the size of the moving object, obtaining a second illegal object existing in each frame of image through a trained neural network model, and controlling an alarm device to give an alarm when a target illegal object determined according to the first illegal object and the second illegal object enters a preset area. According to the embodiment of the invention, the size of the moving object in the image acquired by the binocular camera is judged, and the object is detected by the neural network model, so that the target illegal object is determined, the illegal object can be effectively detected, the phenomena of missing detection and false detection in the detection process are avoided, and the accuracy of illegal object detection is further improved.
Drawings
In order to more clearly illustrate the technical solution of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on the drawings without creative efforts.
Fig. 1 is a schematic process diagram of an article detection method according to an embodiment of the present invention;
FIG. 2 is a detailed flow chart of article detection provided by an embodiment of the present invention;
fig. 3 is a schematic structural diagram of an article detection device according to an embodiment of the present invention;
fig. 4 is an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the present invention will be described in further detail with reference to the accompanying drawings, and it is apparent that the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The embodiment of the invention provides an article detection method, device, equipment and medium, which can avoid false detection and missing detection of illegal articles and improve the accuracy of article detection.
Example 1:
fig. 1 is a schematic diagram of an article detection process provided in an embodiment of the present invention, where the process includes the following steps:
s101: according to each frame of image collected by the binocular camera, determining moving objects in the image, determining the size of each moving object, and determining each first illegal object according to the determined size of each moving object.
The article detection provided by the embodiment of the invention is applied to electronic equipment, and the electronic equipment can be image acquisition equipment, a PC (personal computer) or a server and other intelligent equipment.
In order to improve the accuracy of illegal article detection, a binocular camera is adopted for image acquisition in the embodiment of the invention, and in order to determine the size of an article from an image acquired by the binocular camera, the binocular camera is calibrated and corrected firstly.
Before calibrating and correcting the binocular camera, firstly fixing the binocular camera, calibrating the binocular camera after fixing the binocular camera, acquiring left and right parallax images and point cloud information of the binocular camera after calibrating the binocular camera, and then correcting the binocular camera. The specific process of calibrating and correcting the binocular camera belongs to the prior art, and particularly aims to realize that each pixel point in a left image acquired by the calibrated and corrected binocular camera corresponds to each pixel point in a right image.
In the actual detection process, the binocular camera collects each frame of image according to a set time interval, and according to the position of an article between every two adjacent frames in the image, whether the article is a moving article can be determined aiming at each frame of image collected by the binocular camera. The specific process for determining the moving object belongs to the prior art, and is not described in detail in the embodiment of the invention.
After the moving object is detected, the distance between each pixel point in the image and the binocular camera is determined, and the actual height represented by each pixel point is determined, so that when the moving object exists in the image, the size of the moving object in the image can be determined.
In the embodiment of the invention, in order to effectively identify illegal articles such as large luggage and the like, an article size threshold is preset, and if the size of the moving article exceeds the preset size threshold, the moving article is judged to be the first illegal article.
S102: and acquiring a second illegal article existing in any input frame of image through the trained neural network model.
In order to further improve the accuracy of article detection, aiming at each frame of image collected by the binocular camera, each frame of image collected by the binocular camera is input into the neural network model, and the neural network model is adopted to detect illegal articles in the input image. The neural network model is trained in advance, so that the positions and the types of illegal articles existing in the images can be identified, and non-illegal articles, such as pedestrians and the like, appearing in the images can also be identified. In the embodiment of the invention, the illegal object is generally an object such as a baby carriage, a shopping cart, a large piece of luggage and the like.
The image input into the neural network model may be a left image acquired by a binocular camera or a right image acquired by the binocular camera, specifically, when the neural network model is trained, which of the left and right images is adopted, and when the neural network model is detected, the corresponding image is adopted. For example, when the neural network model is trained, a left image is used, when the neural network model is detected, the left image collected by the binocular camera is input into the neural network model for detection, and when the neural network model is trained, a right image is used, when the neural network model is detected, the right image collected by the binocular camera is input into the neural network model for detection.
S103, if the target illegal object determined according to the first illegal object and the second illegal object is identified to be in a preset area, controlling an alarm device to give an alarm.
Identifying a moving object and the size of the moving object in an image based on each frame of image acquired by a binocular camera, determining a first illegal object, identifying a second illegal object in the image based on a trained neural network model, wherein the first illegal object and the second illegal object may be the same or different, and determining the intersection of the first illegal object and the second illegal object as a target illegal object in order to ensure the detection accuracy; meanwhile, in order to ensure that the condition of missing detection does not occur any more, the union of the first illegal object and the second illegal object can be determined as the target illegal object.
After the target illegal object is determined, the target illegal object is tracked, a specific tracking process belongs to the prior art, and the process is not repeated in the embodiment of the invention. And for any target illegal object, if the target illegal object is determined to enter the preset area, controlling the alarm equipment to alarm.
The preset area is a preset area for alarm detection, and the article detection method provided by the embodiment of the invention is generally used for detecting illegal articles such as escalators and the like, so that the preset area can be an area before entering the escalators.
When the electronic equipment controls the alarm equipment to alarm, the electronic equipment can control the alarm equipment to alarm, and can also control other alarm equipment to alarm. When the electronic equipment controls the alarm, the alarm can be given to the staff through sound, light and the like. When the electronic equipment controls other equipment to alarm, for example, the electronic equipment sends an alarm signal to a terminal of a worker to control the terminal of the worker to alarm and inform the worker that a user has danger so that the worker can control the escalator to stop running, thereby avoiding the occurrence of dangerous accidents. Or the electronic equipment can also send alarm information to the control console, the control console is used as alarm equipment to give an alarm, the control console can control the start and stop of the escalator, and when the control console receives the alarm information and gives an alarm, the escalator can be controlled to stop running.
According to the embodiment of the invention, the size of the moving object in the image acquired by the binocular camera is judged, and the object is detected by the neural network model, so that the target illegal object is determined, the illegal object can be effectively detected, the phenomena of missing detection and false detection in the detection process are avoided, and the accuracy of illegal object detection is further improved.
Example 2:
in order to accurately determine the size of the article, on the basis of the above embodiment, in an embodiment of the present invention, the determining the size of each moving article includes:
and determining the size of the moving object in the image according to the physical distance between each pixel point in the image acquired by the corrected binocular camera and the height from the ground.
In the embodiment of the invention, in order to accurately identify the size of the moving object, a binocular camera is adopted, and the three-dimensional information of the object can be acquired based on the image acquired by the binocular camera. In the embodiment of the invention, a binocular camera is generally installed on the top of the escalator entrance. In order to accurately acquire the three-dimensional information of the object, the installed binocular camera needs to be calibrated in advance. When calibrating the binocular camera, firstly, acquiring the internal parameters of the left image and the right image of the binocular camera, the camera focal length and other information, measuring the installation height and angle of the binocular camera, and completing calibration of the binocular camera by adopting a calibration plate. The specific calibration process of the binocular camera belongs to the prior art, and is not described herein again.
And after the calibration of the binocular camera is finished, correcting the binocular camera. The specific correction process comprises the following steps: the method comprises the steps of obtaining a left image and a right image collected by a binocular camera, obtaining parallax information of the left image and the right image of the binocular camera by adopting a binocular stereo matching algorithm (SGBM), calculating the actual distance between every two adjacent pixel points and the actual height of each pixel point from the ground in the image collected by the binocular camera according to calibration of the binocular camera and the parallax information, and obtaining the distance and the height of each pixel point from the binocular camera in the image through calculation. And each pixel point in the left image acquired by the calibrated and corrected binocular camera corresponds to each pixel point in the right image.
Because the distance between each pixel point in the image acquired by the binocular camera and the binocular camera is determined, and the actual height represented by each pixel point is determined, when a moving object exists in the image, the size of the moving object can be determined according to the area where the moving object exists in the image, and the size of the moving object refers to the actual physical size of the moving object.
For convenience of judgment, in this embodiment of the present invention, the determining each first offending item according to the determined size of each moving item further includes:
and for each moving object, if the volume of the moving object is larger than a preset volume threshold value, determining that the moving object is a first illegal object.
Because some of the escalators positioned in places such as shopping malls and supermarkets have larger volume and larger inertia of articles with larger weight in the operation process, the escalators are easy to generate danger in the operation process. Therefore, in the embodiment of the invention, in order to accurately identify the large articles appearing in places such as shopping malls, supermarkets and the like and avoid dangers appearing on the escalators of the large articles, the moving articles appearing in the places such as the shopping malls, the supermarkets and the like can be judged, and whether the moving articles are the large articles or not can be judged.
In the embodiment of the invention, in order to effectively judge whether the moving object is a large object, a volume threshold value is preset, after the three-dimensional information of the moving object is obtained, the volume of the moving object can be obtained based on the three-dimensional information of the moving object, whether the volume of the moving object is larger than the preset volume threshold value is judged, and if the volume of the moving object is larger than the preset volume threshold value, the moving object is determined to be the large object and belongs to the first illegal object.
When the volume of the moving object is determined based on the three-dimensional information of the moving object, if the moving object is a regular-shaped object, such as a regular shape like a cuboid, a cube, a sphere, etc., the volume of the moving object is determined directly based on the three-dimensional information of the moving object; if the moving object is irregular in shape, when the volume of the moving object is determined, the volume of the moving object can be determined based on the size of the minimum circumscribed cube of the moving object. The specific process for determining the minimum bounding cube of the article belongs to the prior art, and is not described in detail in the embodiment of the present invention.
To improve the accuracy of detection, in an embodiment of the present invention, the determining each first offending item according to the determined size of each moving item further includes:
and for each moving article, if the height of the moving article is within any preset height range of the illegal articles, determining that the moving article is the first illegal article.
Because most illegal articles which are possibly dangerous and appear on the escalator are baby carriages, shopping carts and the like, and the heights of the illegal articles are in different height ranges, the heights of the illegal articles can be determined according to the illegal articles which often appear in a scene, and therefore the corresponding height ranges of the illegal articles are set. When the height of the moving object is obtained, whether the height of the moving object is within a preset height range of any illegal object is judged, and if the height of the moving object is within the preset height range of any illegal object, the moving object is determined to be a first illegal object.
When detecting whether the moving object is the first illegal object, in order to improve efficiency, the determination may be performed in any manner in the above embodiments, but in order to increase the accuracy of the determination and avoid accidents, the determination may be performed based on each of the above embodiments, and as long as the moving object is determined to be the first illegal object based on any embodiment, the moving object is determined to be the first illegal object. For example, a certain moving object a existing in the image is determined, and according to the acquired volume of the moving object a, the moving object a is determined to be not the first illegal object, but the height of the moving object a is within a preset certain height range of the illegal object, and then the moving object a is determined to be the first illegal object.
In addition, in order to avoid detection errors of overlapping areas during illegal article detection, a gaussian background modeling can be adopted for a left image of each frame of image acquired by a binocular camera to acquire a motion area of an article, after each motion area is acquired, the motion areas are firstly distinguished based on different heights, the motion areas are divided into different height ranges, actual three-dimensional information corresponding to pixel points of the motion areas in the same height range is clustered by adopting a kmeans clustering method and preset different clustering centers, so that the motion area information of each motion article is acquired, a corresponding motion article is determined in the image acquired by the binocular camera based on the motion area information of each motion article, the volume and/or the height of the motion article is determined, and according to the determined volume and/or the height of the motion article, and judging whether the moving object is the first illegal object.
Example 3:
in order to accurately detect the second illegal object existing in the image, on the basis of the above embodiments, in the embodiment of the present invention, the neural network model is trained by the following method:
acquiring any sample image in a sample set, first position information of each illegal item contained in the sample image and first identification information of the illegal item contained in the first position information;
inputting the sample image into an original neural network model, and acquiring second position information of each illegal item contained in the sample image and second identification information of the illegal item contained in the second position information;
and training the original neural network model according to the first position information, the second position information, the first identification information and the corresponding second identification information.
In order to train the neural network model, in an embodiment of the present invention, a sample set for training is stored, where the sample images in the sample set include images of illegal items such as different strollers, shopping carts, and large luggage, for example, the sample images in the sample set include strollers with different colors, different materials, and different sizes, shopping carts with different types, and large luggage with various types, such as red strollers, blue strollers, green strollers, wooden strollers, plastic strollers, composite material strollers, large-sized strollers, small-sized strollers, and shopping carts with various types, such as shopping carts commonly used in supermarkets and shopping carts carried by the elderly to buy dishes.
In order to train the neural network model conveniently, the sample set further stores, for each sample image, first location information and first identification information of the illegal item included in the sample image, and the specific first identification information may be used to identify a type of the illegal item included in the sample image, for example, 01 is a stroller, 02 is a shopping cart, and the like. The sample image may further include location information and identification information of a non-illegal article, for example, location information of a pedestrian and identification information identifying that the location information corresponds to the pedestrian.
In the embodiment of the present invention, the neural network model includes a plurality of convolutional layers and pooling layers, and the structure of the neural network model is not particularly limited in the embodiment of the present invention.
In the embodiment of the invention, after any sample image in a sample set and first position information and first identification information of an illegal article contained in the sample image are acquired, the sample image, the first position information and the first identification information are input into an original neural network model, and the original neural network model outputs second position information of the illegal article contained in the sample image and second identification information of the illegal article corresponding to the second position information.
After the original neural network model determines second position information of the illegal object contained in the sample image and second identification information of the illegal object of the second position information, the original neural network model is trained according to the first position information and the first identification information in the sample image, the second position information output by the original neural network model and the corresponding second identification information.
And training the neural network model in the above manner, and obtaining the trained neural network model when the preset conditions are met. The preset condition may be that the number of sample images in which the second position information and the second identification information obtained after the sample images in the sample set are trained through the original neural network model are consistent with the first position information and the first identification information is greater than a set number; or training the original neural network model until the iteration number reaches the set maximum iteration number, and the like. Specifically, the embodiment of the present invention is not limited to this.
Example 4:
the following describes an article detection process provided by an embodiment of the present invention in detail with reference to a specific embodiment.
Fig. 2 is a schematic diagram of a detailed implementation process of the article detection provided by the embodiment of the present invention, where the process includes:
in the embodiment of the invention, the binocular camera is adopted for image acquisition, and in order to determine the size of an article from the image acquired by the binocular camera, the binocular camera is calibrated and corrected firstly.
After calibrating and correcting the binocular camera, carrying out motion detection on the object in the image according to each frame of image collected by the binocular camera, acquiring a left image and a right image in each frame of image collected by the binocular camera after detecting the moving object, and acquiring a left-right parallax image corresponding to each frame of image. And determining the three-dimensional information of the moving object in the image according to the physical distance between each pixel point in the image acquired by the calibrated and corrected binocular camera and the height from the ground.
And obtaining the volume and the height of the moving object according to the three-dimensional information of the moving object, and comparing the volume and the height of the moving object with a corresponding volume threshold and a preset height range of any illegal object respectively, so as to determine whether the moving object is a first illegal object.
On the other hand, through a trained neural network model, for example, a CNN model, CNN detection is performed based on the CNN model, a left image or a right image in each frame of image acquired by a binocular camera is detected, and whether a second illegal article exists in the left image or the right image is judged.
And determining a final target illegal article according to the first illegal article and the second illegal article, tracking the target illegal article, and when the target illegal article enters a preset area, giving an alarm by the control console and enabling the control console to control the escalator to stop running.
Example 5:
fig. 3 is a schematic structural diagram of an article detection apparatus according to an embodiment of the present invention, where the apparatus includes:
the determining module 301 is configured to determine moving objects in the images according to each frame of image acquired by the binocular camera, and determine the size of each moving object; determining each first offending item according to the determined size of each moving item;
the identification module 302 is configured to obtain a second illegal item existing in any input frame of image through the trained neural network model;
and the control module 303 is configured to control an alarm device to alarm if it is identified that the target illegal item determined according to the first illegal item and the second illegal item is in a preset area.
In a possible implementation manner, the determining module 301 is specifically configured to determine the size of the image moving object according to the physical distance between each pixel point in the corrected image captured by the binocular camera and the binocular camera, and the height from the ground.
In a possible embodiment, the determining module 301 is specifically configured to, for each moving object, determine that the moving object is a first illegal object if the volume of the moving object is greater than a preset volume threshold.
In a possible embodiment, the determining module 301 is specifically configured to, for each moving object, determine that the moving object is the first illegal object if the height of the moving object is within any preset height range of the illegal object.
The device further comprises:
the training module is used for acquiring any sample image in a sample set, first position information of each illegal item contained in the sample image and first identification information of the illegal item contained in the first position information; inputting the sample image into an original neural network model, and acquiring second position information of each illegal item contained in the sample image and second identification information of the illegal item contained in the second position information; and training the original neural network model according to the first position information, the second position information, the first identification information and the corresponding second identification information.
According to the embodiment of the invention, the size of the moving object in the image acquired by the binocular camera is judged, and the object is detected by the neural network model, so that the target illegal object is determined, the illegal object can be effectively detected, the phenomena of missing detection and false detection in the detection process are avoided, and the accuracy of illegal object detection is further improved.
Example 6:
on the basis of the foregoing embodiments, an embodiment of the present invention further provides an electronic device, as shown in fig. 4, including: the system comprises a processor 401, a communication interface 402, a memory 403 and a communication bus 404, wherein the processor 401, the communication interface 402 and the memory 403 are communicated with each other through the communication bus 404.
The memory 403 has stored therein a computer program which, when executed by the processor 401, causes the processor 401 to perform the steps of:
determining moving objects in the images according to each frame of image acquired by the binocular camera, and determining the size of each moving object; determining each first offending item according to the determined size of each moving item;
acquiring a second illegal article existing in any input frame of image through the trained neural network model;
and if the target illegal object determined according to the first illegal object and the second illegal object is identified to be in a preset area, controlling an alarm device to give an alarm.
Further, the processor 401 is further configured to determine the size of the image moving object according to the physical distance between each pixel point in the image acquired by the corrected binocular camera and the binocular camera, and the height from the ground.
Further, the processor 401 is further configured to, for each moving object, determine that the moving object is a first illegal object if the volume of the moving object is greater than a preset volume threshold.
Further, the processor 401 is further configured to, for each moving object, determine that the moving object is the first illegal object if the height of the moving object is within any preset height range of the illegal object.
Further, the processor 401 is further configured to obtain any sample image in the sample set, and first location information of each illegal item included in the sample image and first identification information of the illegal item included in the first location information; inputting the sample image into an original neural network model, and acquiring second position information of each illegal item contained in the sample image and second identification information of the illegal item contained in the second position information; and training the original neural network model according to the first position information, the second position information, the first identification information and the corresponding second identification information.
The communication bus mentioned in the above server may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface 402 is used for communication between the above-described electronic apparatus and other apparatuses.
The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Alternatively, the memory may be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a central processing unit, a Network Processor (NP), and the like; but may also be a Digital instruction processor (DSP), an application specific integrated circuit, a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like.
Example 7:
on the basis of the foregoing embodiments, an embodiment of the present invention further provides a computer-readable storage medium, in which a computer program executable by an electronic device is stored, and when the program is run on the electronic device, the electronic device is caused to execute the following steps:
the memory having stored therein a computer program that, when executed by the processor, causes the processor to perform the steps of:
determining moving objects in the images according to each frame of image acquired by the binocular camera, and determining the size of each moving object; determining each first offending item according to the determined size of each moving item;
acquiring a second illegal article existing in any input frame of image through the trained neural network model;
and if the target illegal object determined according to the first illegal object and the second illegal object is identified to be in a preset area, controlling an alarm device to give an alarm.
Further, the determining the size of each moving object comprises: and determining the size of the image moving object according to the physical distance between each pixel point in the image acquired by the corrected binocular camera and the height from the ground.
Further, the determining each first offending item based on the determined size of each moving item includes: and for each moving object, if the volume of the moving object is larger than a preset volume threshold value, determining that the moving object is a first illegal object.
Further, the determining each first offending item based on the determined size of each moving item includes: and for each moving article, if the height of the moving article is within any preset height range of the illegal articles, determining that the moving article is the first illegal article.
Further, the neural network model is trained by: acquiring any sample image in a sample set, first position information of each illegal item contained in the sample image and first identification information of the illegal item contained in the first position information; inputting the sample image into an original neural network model, and acquiring second position information of each illegal item contained in the sample image and second identification information of the illegal item contained in the second position information; and training the original neural network model according to the first position information, the second position information, the first identification information and the corresponding second identification information.
According to the embodiment of the invention, the size of the moving object in the image acquired by the binocular camera is judged, and the object is detected by the neural network model, so that the target illegal object is determined, the illegal object can be effectively detected, the phenomena of missing detection and false detection in the detection process are avoided, and the accuracy of illegal object detection is further improved.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
For the system/apparatus embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference may be made to some descriptions of the method embodiments for relevant points.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.
Claims (10)
1. An item detection method, comprising:
determining moving objects in the images according to each frame of image acquired by the binocular camera, and determining the size of each moving object; determining each first offending item according to the determined size of each moving item;
acquiring a second illegal article existing in any input frame of image through the trained neural network model;
and if the target illegal object determined according to the first illegal object and the second illegal object is identified to be in a preset area, controlling an alarm device to give an alarm.
2. The method of claim 1, wherein said determining the size of each moving object comprises:
and determining the size of the image moving object according to the physical distance between each pixel point in the image acquired by the corrected binocular camera and the height from the ground.
3. The method of claim 1, wherein determining each first offending item based on the determined size of each moving item comprises:
and for each moving object, if the volume of the moving object is larger than a preset volume threshold value, determining that the moving object is a first illegal object.
4. The method of claim 1 or 3, wherein determining each first offending item based on the determined size of each moving item comprises:
and for each moving article, if the height of the moving article is within any preset height range of the illegal articles, determining that the moving article is the first illegal article.
5. The method of claim 1, wherein the neural network model is trained by:
acquiring any sample image in a sample set, first position information of each illegal item contained in the sample image and first identification information of the illegal item contained in the first position information;
inputting the sample image into an original neural network model, and acquiring second position information of each illegal item contained in the sample image and second identification information of the illegal item contained in the second position information;
and training the original neural network model according to the first position information, the second position information, the first identification information and the corresponding second identification information.
6. An article detection device, the device comprising:
the determining module is used for determining moving objects in the images according to each frame of image acquired by the binocular camera and determining the size of each moving object; determining each first offending item according to the determined size of each moving item;
the recognition module is used for acquiring a second illegal article existing in any input frame of image through the trained neural network model;
and the control module is used for controlling an alarm device to give an alarm if the target illegal object determined according to the first illegal object and the second illegal object is identified to be in a preset area.
7. The device according to claim 6, wherein the determining module is specifically configured to determine the size of the image moving object according to the physical distance between each pixel point in the image collected by the corrected binocular camera and the binocular camera, and the height from the ground.
8. The apparatus of claim 6, further comprising:
the training module is used for acquiring any sample image in the sample set, first position information of each illegal item contained in the sample image and first identification information of the illegal item contained in the first position information; inputting the sample image into an original neural network model, and acquiring second position information of each illegal item contained in the sample image and second identification information of the illegal item contained in the second position information; and training an original neural network model according to the first position information, the second position information, the first identification information and the corresponding second identification information.
9. An electronic device, characterized in that the electronic device comprises at least a processor and a memory, the processor being adapted to perform the steps of the item detection method of any of claims 1-5 when executing a computer program stored in the memory.
10. A computer-readable storage medium, characterized in that it stores a computer program which, when being executed by a processor, carries out the steps of the item detection method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010641007.7A CN111914659A (en) | 2020-07-06 | 2020-07-06 | Article detection method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010641007.7A CN111914659A (en) | 2020-07-06 | 2020-07-06 | Article detection method, device, equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111914659A true CN111914659A (en) | 2020-11-10 |
Family
ID=73227438
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010641007.7A Pending CN111914659A (en) | 2020-07-06 | 2020-07-06 | Article detection method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111914659A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112733985A (en) * | 2021-01-27 | 2021-04-30 | 昆山善思光电科技有限公司 | Method for automatically identifying large and small material trays |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9996890B1 (en) * | 2017-07-14 | 2018-06-12 | Synapse Technology Corporation | Detection of items |
| CN109753977A (en) * | 2017-11-01 | 2019-05-14 | 菜鸟智能物流控股有限公司 | Method, terminal, server and system for measuring article size information |
| CN109884721A (en) * | 2018-12-10 | 2019-06-14 | 深圳极视角科技有限公司 | Safety check prohibited items detection method, device and electronic equipment based on artificial intelligence |
| CN109975885A (en) * | 2019-03-28 | 2019-07-05 | 上海中通吉网络技术有限公司 | A kind of express delivery prohibited items recognition methods and safe examination system |
| CN110059618A (en) * | 2019-04-17 | 2019-07-26 | 北京易达图灵科技有限公司 | A kind of recognition methods of prohibited items and device |
| CN110738096A (en) * | 2019-08-23 | 2020-01-31 | 银河水滴科技（北京）有限公司 | intelligent security inspection method, equipment and computer readable storage medium |
| CN110956225A (en) * | 2020-02-25 | 2020-04-03 | 浙江啄云智能科技有限公司 | Contraband detection method and system, computing device and storage medium |
| CN111160220A (en) * | 2019-12-26 | 2020-05-15 | 厦门市美亚柏科信息股份有限公司 | Deep learning-based parcel detection method and device and storage medium |
-
2020
- 2020-07-06 CN CN202010641007.7A patent/CN111914659A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9996890B1 (en) * | 2017-07-14 | 2018-06-12 | Synapse Technology Corporation | Detection of items |
| US20190019318A1 (en) * | 2017-07-14 | 2019-01-17 | Synapse Technology Corporation | Generating Graphical Representations of Scanned Objects |
| CN109753977A (en) * | 2017-11-01 | 2019-05-14 | 菜鸟智能物流控股有限公司 | Method, terminal, server and system for measuring article size information |
| CN109884721A (en) * | 2018-12-10 | 2019-06-14 | 深圳极视角科技有限公司 | Safety check prohibited items detection method, device and electronic equipment based on artificial intelligence |
| CN109975885A (en) * | 2019-03-28 | 2019-07-05 | 上海中通吉网络技术有限公司 | A kind of express delivery prohibited items recognition methods and safe examination system |
| CN110059618A (en) * | 2019-04-17 | 2019-07-26 | 北京易达图灵科技有限公司 | A kind of recognition methods of prohibited items and device |
| CN110738096A (en) * | 2019-08-23 | 2020-01-31 | 银河水滴科技（北京）有限公司 | intelligent security inspection method, equipment and computer readable storage medium |
| CN111160220A (en) * | 2019-12-26 | 2020-05-15 | 厦门市美亚柏科信息股份有限公司 | Deep learning-based parcel detection method and device and storage medium |
| CN110956225A (en) * | 2020-02-25 | 2020-04-03 | 浙江啄云智能科技有限公司 | Contraband detection method and system, computing device and storage medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112733985A (en) * | 2021-01-27 | 2021-04-30 | 昆山善思光电科技有限公司 | Method for automatically identifying large and small material trays |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10282847B2 (en) | Monitoring system of a passenger conveyor and monitoring method thereof | |
| JP7025912B2 (en) | In-vehicle environment recognition device | |
| JP4456086B2 (en) | Vehicle periphery monitoring device | |
| CN110980197B (en) | Material detection method and device and electronic equipment | |
| JP2007334751A (en) | Vehicle periphery monitoring device | |
| CN108363953B (en) | Pedestrian detection method and binocular monitoring equipment | |
| JP2005352974A (en) | Vehicle circumference monitoring device | |
| US20150161447A1 (en) | Vision based pedestrian and cyclist detection method | |
| CN107516423B (en) | Video-based vehicle driving direction detection method | |
| CN110126734A (en) | A kind of collision early warning system being directed to Vehicular visual blind area, device and method | |
| CN108021849B (en) | Pedestrian early warning method and device | |
| CN113792586A (en) | Vehicle accident detection method and device and electronic equipment | |
| Kim et al. | System and method for detecting potholes based on video data | |
| CN114842560B (en) | Computer vision-based construction site personnel dangerous behavior identification method | |
| JP2015064752A (en) | Vehicle monitoring device and vehicle monitoring method | |
| KR101236234B1 (en) | Detection system of road line using both laser sensor and camera | |
| CN111914659A (en) | Article detection method, device, equipment and medium | |
| JP2014056295A (en) | Vehicle periphery monitoring equipment | |
| JP4813304B2 (en) | Vehicle periphery monitoring device | |
| CN110147748A (en) | A kind of mobile robot obstacle recognition method based on road-edge detection | |
| CN110634120B (en) | Vehicle damage judgment method and device | |
| CN103839035A (en) | Person number statistical method and person number statistical system | |
| CN111914704B (en) | Tricycle manned identification method and device, electronic equipment and storage medium | |
| CN111582171B (en) | Pedestrian red light running monitoring method, device and system and readable storage medium | |
| Nazir et al. | Vehicle detection on embedded single board computers |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |