CN113378602A - Method and device for detecting illegal stacking of articles, server and storage medium - Google Patents
Method and device for detecting illegal stacking of articles, server and storage medium Download PDFInfo
- Publication number
- CN113378602A CN113378602A CN202010159807.5A CN202010159807A CN113378602A CN 113378602 A CN113378602 A CN 113378602A CN 202010159807 A CN202010159807 A CN 202010159807A CN 113378602 A CN113378602 A CN 113378602A
- Authority
- CN
- China
- Prior art keywords
- image
- illegal
- stacking
- detected
- monitoring video
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 49
- 238000012544 monitoring process Methods 0.000 claims abstract description 118
- 238000001514 detection method Methods 0.000 claims abstract description 86
- 238000012545 processing Methods 0.000 claims abstract description 32
- 238000010606 normalization Methods 0.000 claims abstract description 28
- 238000012549 training Methods 0.000 claims description 51
- 238000002372 labelling Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 3
- 230000009466 transformation Effects 0.000 description 16
- 230000006870 function Effects 0.000 description 12
- 230000008569 process Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 7
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 4
- 238000000844 transformation Methods 0.000 description 4
- 238000013519 translation Methods 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 1
- 239000000969 carrier Substances 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000007599 discharging Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000009432 framing Methods 0.000 description 1
- GVVPGTZRZFNKDS-JXMROGBWSA-N geranyl diphosphate Chemical compound CC(C)=CCC\C(C)=C\CO[P@](O)(=O)OP(O)(O)=O GVVPGTZRZFNKDS-JXMROGBWSA-N 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000013526 transfer learning Methods 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
- 230000007306 turnover Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
Landscapes
- Business, Economics & Management (AREA)
- Engineering & Computer Science (AREA)
- Economics (AREA)
- Quality & Reliability (AREA)
- Entrepreneurship & Innovation (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Development Economics (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Alarm Systems (AREA)
Abstract
The application discloses a method, a device, a server and a storage medium for detecting illegal goods stacking, wherein the method for detecting illegal goods stacking comprises the following steps: acquiring a monitoring video image of article stacking; carrying out normalization processing on the monitoring video image to obtain an image to be detected; detecting an image to be detected, and determining whether illegal stacking exists in the image to be detected; and if the illegal code placement exists in the image to be detected, marking illegal code placement information in the monitoring video image corresponding to the image to be detected. According to the method for detecting the illegal article stacking, the images to be detected are obtained by carrying out normalization processing on the monitoring video images, then whether the illegal stacking exists or not is quickly determined from the images to be detected, and when the illegal stacking exists in the images to be detected, the illegal stacking information is marked in the corresponding monitoring video images, manual detection is not needed, and the labor cost for detecting whether the articles are illegally stacked is reduced.
Description
Technical Field
The application relates to the technical field of image recognition, in particular to a method, a device, a server and a storage medium for detecting illegal article stacking.
Background
In recent years, with the development of logistics industry, in order to pursue timeliness, the problem that article carriers are not regularly stacked in the process of stacking articles may occur, for example: the article that bulk article are random is piled up, is piled up together the fretwork condition appears and so on, and these article of irregularity are piled up the mode and are caused article easily and are crushed, fall into the scheduling problem, cause huge loss for the commodity circulation company.
In order to avoid the above problems, in the prior art, a supervisor generally supervises an article stacking site to prevent the problem of illegal stacking of articles, and the manual supervision method not only wastes human resources, but also easily causes the problem of incomplete supervision.
Disclosure of Invention
The embodiment of the application provides a method, a device, a server and a storage medium for detecting illegal goods stacking, and aims to improve a detection mode of the illegal goods stacking, so that the detection mode of the illegal goods stacking is more efficient, and the labor cost for detecting whether the illegal goods stacking is carried out is reduced.
The embodiment of the application provides a method for detecting illegal goods stacking, which comprises the following steps:
acquiring a monitoring video image of article stacking;
carrying out normalization processing on the monitoring video image to obtain an image to be detected;
detecting the image to be detected, and determining whether illegal stacking exists in the image to be detected;
and if the illegal code placement exists in the image to be detected, marking illegal code placement information in the monitoring video image corresponding to the image to be detected.
In some embodiments of the present application, the detecting the image to be detected, and determining whether there is illegal stacking in the image to be detected includes:
acquiring a preset violation code image;
calculating the similarity between the image to be detected and the violation stacking image;
and if the similarity reaches a preset threshold value, determining that illegal code placement exists in the image to be detected.
In some embodiments of the present application, the detecting the image to be detected, and determining that there is an illegal stacking in the image to be detected includes:
inputting the image to be detected into a detection model obtained through training, detecting, and determining whether the illegal code placement exists in the image to be detected, wherein the detection model is obtained by training an initial model according to a sample monitoring video image with the illegal code placement and corresponding illegal code placement information.
In some embodiments of the present application, before the image to be detected is input into a detection model obtained through training for detection, and whether there is illegal code placement in the image to be detected is determined, the method further includes:
acquiring a sample monitoring video image with illegal stacking;
marking the illegal stacking information in the sample monitoring video image to obtain a marked image;
and training an initial model according to the sample monitoring video image and the corresponding annotation image to obtain the detection model.
In some embodiments of the present application, the labeling the illegal stacking information in the sample monitoring video image to obtain a labeled image includes:
and marking the illegal code-placement type and the illegal code-placement area in the sample monitoring video image to obtain the marked image.
In some embodiments of the present application, the training an initial model according to the sample monitoring video image and the corresponding annotation image to obtain the detection model includes:
carrying out normalization processing on the sample monitoring video image to obtain a sample image;
and training an initial model according to the sample image and the corresponding marked image to obtain the detection model.
In some embodiments of the present application, if there is an illegal stacking in the image to be detected, after the illegal stacking type and the illegal stacking area are marked in the surveillance video image corresponding to the image to be detected, the method further includes:
and displaying the monitoring video image marked with illegal stacking information.
The embodiment of the present application further provides a device for detecting illegal stacking of articles, including:
the first acquisition module is used for acquiring monitoring video images of article stacking;
the first processing module is used for carrying out normalization processing on the monitoring video image to obtain an image to be detected;
the detection module is used for detecting the image to be detected and determining whether illegal stacking exists in the image to be detected;
and the first marking module is used for marking illegal code placement information in the monitoring video image corresponding to the image to be detected if the illegal code placement exists in the image to be detected.
In some embodiments of the present application, the detection module comprises:
the second acquisition module is used for acquiring a preset violation code image;
the calculation module is used for calculating the similarity between the image to be detected and the illegal code-placement image;
and the judging module is used for determining whether illegal stacking exists in the image to be detected if the similarity reaches a preset threshold value.
In some embodiments of the application, the detection module is configured to detect a detection model obtained by training input of the image to be detected, and determine whether an illegal stacking exists in the image to be detected, where the detection model is obtained by training an initial model according to a sample monitoring video image with the illegal stacking and corresponding illegal stacking information.
In some embodiments of the present application, the device for detecting illegal article stacking further includes:
the third acquisition module is used for acquiring the sample monitoring video images with illegal stacking;
the second labeling module is used for labeling the illegal code placement information in the sample monitoring video image to obtain a labeled image;
and the training module is used for training the initial model according to the sample monitoring video image and the corresponding annotation image to obtain the detection model.
In some embodiments of the application, the second labeling module is configured to label the violation code type and the violation code area in the sample monitoring video image, so as to obtain the labeled image.
In some embodiments of the present application, the training module comprises:
the second processing module is used for carrying out normalization processing on the sample monitoring video image to obtain a sample image;
and the sub-training module is used for training the initial model according to the sample image and the corresponding marked image to obtain the detection model.
In some embodiments of the present application, the device for detecting illegal article stacking further includes:
and the display module is used for displaying the monitoring video image marked with illegal stacking information.
An embodiment of the present application further provides a server, where the server includes:
one or more processors;
a memory; and
one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the item violation misplacement detection method as described above.
An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, where the computer program is loaded by a processor to execute the steps in the method for detecting illegal stacking of articles as described above.
The method for detecting illegal article stacking comprises the steps of obtaining a monitoring video image of the illegal article stacking, carrying out normalization processing on the monitoring video image to obtain an image to be detected, detecting the image to be detected, rapidly and accurately determining whether the illegal article stacking exists in the image to be detected or not, and automatically marking illegal article stacking information in the monitoring video image corresponding to the image to be detected when the illegal article stacking exists in the image to be detected, wherein manual detection is not needed, and labor cost for detecting whether the illegal article stacking consumes is reduced. Meanwhile, the stacking process of the articles can be detected in real time, and the detection efficiency is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a scene schematic diagram of an article illegal stacking detection system according to an embodiment of the present application;
fig. 2 is a schematic flow chart of an embodiment flow of an article illegal stacking detection method provided in an embodiment of the present application;
FIG. 3 is a schematic flow chart diagram illustrating one embodiment of training a detection model provided by embodiments of the present application;
fig. 4 is a schematic structural diagram of an embodiment of an article illegal stacking detection device provided in an embodiment of the present application;
fig. 5 is a schematic structural diagram of an embodiment of a server provided in an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
In the description of the present application, it is to be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", and the like indicate orientations or positional relationships based on those shown in the drawings, and are used merely for convenience of description and for simplicity of description, and do not indicate or imply that the referenced device or element must have a particular orientation, be constructed in a particular orientation, and be operated, and thus should not be considered as limiting the present application. Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", may explicitly or implicitly include one or more of the described features. In the description of the present application, "a plurality" means two or more unless specifically limited otherwise.
In this application, the word "exemplary" is used to mean "serving as an example, instance, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. The following description is presented to enable any person skilled in the art to make and use the application. In the following description, details are set forth for the purpose of explanation. It will be apparent to one of ordinary skill in the art that the present application may be practiced without these specific details. In other instances, well-known structures and processes are not set forth in detail in order to avoid obscuring the description of the present application with unnecessary detail. Thus, the present application is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.
The embodiment of the application provides a method and a device for detecting illegal stacking of articles, a server and a storage medium.
The following are detailed below.
Referring to fig. 1, fig. 1 is a schematic view of a scene of an illegal goods placement detection system according to an embodiment of the present disclosure, where the illegal goods placement detection system may include a server 200 and a camera 210, and the camera 210 is connected to the server 200 through a network, so that the camera 210 and the server 200 perform data interaction. The camera device 210 is arranged on an article stacking occasion and used for collecting monitoring video images of article stacking in the article stacking occasion and transmitting the monitoring video images to the server 200, and an article illegal stacking detection device is integrated in the server 200.
In the embodiment of the application, the server 200 is mainly used for acquiring monitoring video images of article stacking; carrying out normalization processing on the monitoring video image to obtain an image to be detected; detecting the image to be detected, and determining whether illegal stacking exists in the image to be detected; and if the illegal code placement exists in the image to be detected, marking illegal code placement information in the monitoring video image corresponding to the image to be detected.
In this embodiment, the server 200 may be an independent server, or may be a server network or a server cluster composed of servers, for example, the server 200 described in this embodiment includes, but is not limited to, a computer, a network host, a single network server, a plurality of network server sets, or a cloud server composed of a plurality of servers. Among them, the Cloud server is constituted by a large number of computers or web servers based on Cloud Computing (Cloud Computing). In the embodiment of the present application, the server and the User terminal may implement communication through any communication manner, including but not limited to mobile communication based on a third generation Partnership Project (3 GPP), Long Term Evolution (LTE), Worldwide Interoperability for Microwave Access (WiMAX), or computer network communication based on a TCP/IP Protocol Suite (TCP/IP), User Datagram Protocol (UDP) Protocol, and the like.
Those skilled in the art will understand that the application environment shown in fig. 1 is only one application scenario related to the present embodiment, and does not constitute a limitation on the application scenario of the present embodiment, and that other application environments may further include more cameras 210 than that shown in fig. 1, or a server network connection relationship, for example, only 1 server 200 and 1 camera 210 are shown in fig. 1, and it can be understood that the article illegal stacking detection system may further include one or more other cameras 210 connected to the server network, which is not limited herein.
In some embodiments, the system for detecting illegal goods stacking may further include a display device, and the display device is connected to the server 200 through a network, so that the display device and the server 200 perform data interaction. After the detection result of the article stacking in the monitoring video image is obtained in the server 200, the detection result can be displayed in the display device.
In addition, as shown in fig. 1, the system for detecting illegal goods stacking may further include a memory for storing monitoring video image data collected by the camera device.
It should be noted that the scene schematic diagram of the illegal goods stacking detection system shown in fig. 1 is only an example, and the illegal goods stacking detection system and the scene described in the embodiment of the present application are for more clearly illustrating the technical solution of the embodiment of the present application, and do not limit the technical solution provided in the embodiment of the present application.
First, an embodiment of the present application provides a method for detecting illegal stacking of articles, where the method for detecting illegal stacking of articles includes: acquiring a monitoring video image of article stacking; carrying out normalization processing on the monitoring video image to obtain an image to be detected; detecting the image to be detected, and determining whether illegal stacking exists in the image to be detected; and if the illegal code placement exists in the image to be detected, marking illegal code placement information in the monitoring video image corresponding to the image to be detected.
As shown in fig. 2, which is a schematic flowchart of an embodiment of an illegal article stacking detection method in the embodiment of the present application, an execution main body of the illegal article stacking detection method may be the illegal article stacking detection device provided in the embodiment of the present application, or a storage medium, a terminal, a server, and the like that are integrated with the illegal article stacking detection method device. The method for detecting illegal article stacking comprises steps 110 to 140, and is described in detail as follows:
110. and acquiring a monitoring video image of the article stacking.
In some embodiments, a camera may be disposed in an article stacking occasion, and the camera may be used to monitor the stacked articles in the article stacking occasion to obtain a surveillance video of the article stacking, and then extract a surveillance video image from the surveillance video. The monitoring video image may be a preset number of frames of images in the monitoring video, and the multiple frames of images may be continuous or one frame of image extracted every preset time interval.
In other embodiments, the monitoring video of the illegal article stacking can also be stored in a memory in advance, and the monitoring video image of the article is obtained from the memory.
120. And carrying out normalization processing on the monitoring video image to obtain an image to be detected.
In some embodiments, the surveillance video image is normalized by converting the surveillance video image into a corresponding unique standard form image, i.e., an image to be detected, through a series of transformations, where the standard form image has invariant characteristics to affine transformations such as translation, rotation, scaling, and the like. The principle of normalization processing on the monitoring video image is as follows: firstly, the parameters of the transformation function are determined by utilizing the moment which has invariance to affine transformation in the monitoring video image, and then the monitoring video image is transformed into a standard form of to-be-detected image (the image is not related to affine transformation) by utilizing the transformation function determined by the parameters. In general, the moment-based image normalization process includes 4 steps: namely coordinate centering, x-sharpening normalization, scaling normalization and rotation normalization.
130. And detecting the image to be detected, and determining whether illegal stacking exists in the image to be detected.
By detecting the normalized image to be detected, whether the illegal stacking condition exists in the image to be detected can be detected more quickly and accurately. The detection model which is obtained through training in advance and used for detecting whether articles in the image are placed in an illegal mode is used for detecting the image to be detected, so that whether the image to be detected is placed in the illegal mode is determined. Or, the similarity comparison can be performed between the image to be detected and a preset illegal code image, and whether illegal code exists in the image to be detected or not can be determined according to the similarity.
In some embodiments, the detecting the image to be detected and determining whether the illegal code placement exists in the image to be detected includes: inputting the image to be detected into a detection model obtained through training, detecting, and determining whether the illegal code placement exists in the image to be detected, wherein the detection model is obtained by training an initial model according to a sample monitoring video image with the illegal code placement and corresponding illegal code placement information.
In some embodiments, the detection model is a model obtained by training an initial model in advance according to a sample monitoring video image with illegal code and corresponding illegal code information, and the initial model may be a CNN (Convolutional Neural Network) model, or may also be a deep Neural Network model, and the like. Therefore, whether the illegal stacking condition exists in the image to be detected can be determined more quickly and accurately through the detection model.
Fig. 3 is a schematic flow chart of an embodiment of training a detection model in the embodiment of the present application. In some embodiments, as shown in fig. 3, before the image to be detected is input into a detection model obtained through training for detection, and it is determined whether there is an illegal stacking in the image to be detected, the method for detecting illegal stacking of articles further includes steps 310 to 330:
310. and acquiring a sample monitoring video image with illegal stacking.
In some embodiments, a camera may be disposed in an article stacking occasion, and the stacked article in the article stacking occasion is monitored by the camera to obtain a historical monitoring video image of the article stacking, and then the historical monitoring video image is used as a sample monitoring video image. The sample monitoring video image comprises stacked articles, and the goods in the sample monitoring video image are stacked illegally.
The illegal stacking of the articles can be divided into various types according to the stacking mode of the articles, and specifically comprises the random accumulation of a large number of articles; in the stacked articles, the volume of the article on the upper layer is obviously larger than that of the article on the lower layer; the stacked objects are hollowed out, and the like.
And acquiring a plurality of sample monitoring video images for each violation type of article stacking. Specifically, 1000 sample monitoring video images of each illegal stacking type can be selected manually.
320. And marking the illegal code information in the sample monitoring video image to obtain a marked image.
In some embodiments, the violation code-placement information in the sample monitoring video image is determined manually, and the violation code-placement information is labeled in the corresponding labeled image, so as to obtain the labeled image labeled with the violation code-placement information.
The illegal code information in the sample monitoring video image comprises the illegal code type, the illegal code area and the like in the sample monitoring video image. The illegal stacking area refers to an area where illegal stacking exists in the sample monitoring video image.
In some embodiments, the labeling the illegal stacking information in the sample monitoring video image to obtain a labeled image includes: and marking the illegal code-placement type and the illegal code-placement area in the sample monitoring video image to obtain the marked image.
In some embodiments, the violation code-placement type and the violation code-placement area in each sample surveillance video image are determined manually, then the violation code-placement area in each sample surveillance video image is framed and marked, and the violation code-placement type in the sample surveillance video image is marked.
330. And training an initial model according to the sample monitoring video image and the corresponding annotation image to obtain the detection model.
In some embodiments, manual framing and marking are carried out on the violation stacking type and the violation stacking area on each sample monitoring video image to obtain a marked image, then, a corresponding xml file is generated for the sample overhead video image marked with the violation stacking type and the violation stacking area, a VOC data set is generated for the xml file, and finally, the initial model is trained through the VOC data set, so that the training of the initial model is more convenient and efficient.
Wherein training the initial model through the VOC data set may specifically comprise the steps of:
1) and building a dependent environment of the model training model.
2) And constructing an initial model and defining a loss function of the initial model.
3) Modifying the configuration of the network training parameters, and adjusting the pre-training network information, the training step length and the training speed.
4) And importing the pre-training weight of the initial model.
4) And starting a session, importing a VOC data set for iterative tuning, and outputting a group of optimal weights to obtain a detection model for detecting whether the articles are illegally stacked.
In some embodiments, 80% of the sample surveillance video images of each illegal stacking type are used as a training set for training the initial model, and then the remaining 20% of the sample surveillance video images of each illegal stacking type are used as a verification set for verifying the trained initial model, so as to improve the detection accuracy of the detection model obtained after the initial model is trained.
In the process of training the initial model through the VOC data set, the training data in the VOC data set is subjected to data enhancement operations such as turnover transformation, random cutting, color dithering, translation transformation, scale transformation, contrast transformation, noise disturbance and the like at a certain probability, and then is input into the initial model for transfer learning, so that the detection model with high accuracy is obtained.
In some embodiments, the training of the initial model according to the sample monitoring video image and the corresponding annotation image to obtain the detection model may specifically include the following steps:
1) and carrying out normalization processing on the sample monitoring video image to obtain a sample image.
In some embodiments, the sample surveillance video image is normalized by converting the sample surveillance video image into a corresponding unique standard form image, i.e., a sample image, through a series of transformations, where the standard form image has invariant characteristics to affine transformations such as translation, rotation, scaling, and the like. The principle of normalization processing on the sample monitoring video image is as follows: the parameters of the transformation function are determined by using the moment which has invariance to affine transformation in the sample monitoring video image, and then the transformation function determined by the parameters is used for transforming the sample monitoring video image into a sample image in a standard form (the image is not related to affine transformation).
2) And training an initial model according to the sample image and the corresponding marked image to obtain the detection model.
By training the initial model by using the sample image obtained after the normalization processing of the sample monitoring video image, the detection precision of the detection model can be further improved.
In other embodiments, the process of detecting the image to be detected and determining whether the illegal stacking exists in the image to be detected may include the following steps:
1) and acquiring a preset violation code image.
In some embodiments, the illegal stacking image is an image in which an article is illegally stacked, and the illegal stacking image is also correspondingly divided into a plurality of images according to different article illegal stacking types. Specific examples thereof include: the type of the articles in the illegal stacking image can be randomly piled up in a large batch of articles; in the stacked articles, the volume of the article on the upper layer is obviously larger than that of the article on the lower layer; the stacked objects are hollowed out, and the like. The illegal code type in each illegal code image is different.
And the illegal code-placement type and the illegal code-placement area can be marked in the illegal code-placement image.
2) And calculating the similarity between the image to be detected and the violation code image.
In some embodiments, after the monitored video image is normalized to obtain the image to be detected, the similarity between the image to be detected and the illegal code image can be calculated by an image similarity calculation method. The similarity is used for representing the similarity between the image to be detected and the illegal stacking image, and the greater the similarity is, the more likely the stacking type of the article in the image to be detected is the same as the illegal stacking type in the illegal stacking image.
The method comprises the steps of obtaining a plurality of violation stacking images, calculating the similarity between an image to be detected and each violation stacking image, and taking the highest value as the final similarity.
The image similarity calculation method may be various methods of calculating the similarity between images, and for example, may be a SIFT (Scale Invariant Feature Transform) matching Algorithm, a histogram-based matching Algorithm, a Perceptual Hash Algorithm (Perceptual Hash Algorithm), or the like.
Taking a histogram matching algorithm as an example, histograms of an image to be detected and an illegal code image can be respectively obtained, then normalization processing is respectively carried out on the histograms of the image to be detected and the illegal code image to obtain two images to be compared, finally, a babbitt coefficient algorithm is used for calculating the two images to be compared to obtain the similarity of the image to be detected and the illegal code image, the value of the similarity is between [0 and 1], 0 represents extremely different, and 1 represents extremely similar (same).
It should be noted that the above methods for calculating the similarity between images are well-known technologies that are widely studied and applied at present, and are not described herein again.
3) And if the similarity reaches a preset threshold value, determining that illegal code placement exists in the image to be detected.
In some embodiments, a preset threshold of similarity may be preset, and if the similarity between the image to be detected and the illegal stacking image reaches the preset threshold, that is, the similarity between the image to be detected and the illegal stacking image is greater than or equal to the preset threshold, it indicates that the illegal stacking exists in the image to be detected, and the type of the illegal stacking of the article in the image to be detected is the same as the type of the illegal stacking in the illegal stacking image. On the contrary, if the similarity between the image to be detected and the illegal stacking image does not reach the preset threshold, the condition that illegal stacking does not exist in the image to be detected is indicated.
140. And if the illegal code placement exists in the image to be detected, marking illegal code placement information in the monitoring video image corresponding to the image to be detected.
When the illegal stacking condition exists in the image to be detected, the illegal stacking information is marked in the monitoring video image corresponding to the image to be detected, so that a person supervising the article stacking site can quickly and accurately know whether the illegal stacking exists in the article stacking site and the type and the area of the illegal stacking of the article.
The method for detecting illegal article stacking comprises the steps of obtaining a monitoring video image of the illegal article stacking, carrying out normalization processing on the monitoring video image to obtain an image to be detected, detecting the image to be detected, rapidly and accurately determining whether the illegal article stacking exists in the image to be detected or not, and automatically marking illegal article stacking information in the monitoring video image corresponding to the image to be detected when the illegal article stacking exists in the image to be detected, wherein manual detection is not needed, and labor cost for detecting whether the illegal article stacking consumes is reduced. Meanwhile, the stacking process of the articles can be detected in real time, and the detection efficiency is improved. In addition, the hardware structure needed by the method for detecting the illegal goods stacking is simple, and the cost is low.
In some embodiments, after marking an illegal stacking type and an illegal stacking area in a surveillance video image corresponding to the image to be detected if the illegal stacking exists in the image to be detected, the method further includes:
and displaying the monitoring video image marked with illegal stacking information.
In some embodiments, the violation placement information may include a violation placement type and a violation placement region. The system for detecting illegal goods stacking can further comprise a display device connected with the server 200, when the condition that illegal goods stacking exists in the image to be detected is detected, and after the illegal goods stacking type and the illegal goods stacking area are marked in the monitoring video image corresponding to the image to be detected, the monitoring video image marked with the illegal goods stacking type and the illegal goods stacking area can be displayed in the display device, so that the personnel for stacking goods can be prompted to re-stack the illegally stacked goods.
The display device may be a large display screen, a monitoring platform, or the like located in the article stacking site, or may be a terminal device such as a mobile phone or a tablet of a person in charge of the article stacking site, which is not limited herein.
Fig. 4 is a schematic structural diagram of an embodiment of an article illegal stacking detection device in the embodiment of the present application. In order to better implement the method for detecting illegal article stacking in the embodiment of the present application, based on the method for detecting illegal article stacking, as shown in fig. 4, an apparatus 400 for detecting illegal article stacking is further provided in the embodiment of the present application, where the apparatus 400 for detecting illegal article stacking includes:
a first obtaining module 410, configured to obtain a monitoring video image of an article stacking;
the first processing module 420 is configured to perform normalization processing on the monitoring video image to obtain an image to be detected;
the detection module 430 is configured to detect the image to be detected, and determine whether illegal stacking exists in the image to be detected;
the first labeling module 440 is configured to, if there is illegal stacking in the image to be detected, label illegal stacking information in the monitoring video image corresponding to the image to be detected.
The detection device for illegal article stacking is characterized in that the monitoring video images of the illegal article stacking are acquired, the monitoring video images are subjected to normalization processing to obtain images to be detected, the images to be detected are detected, whether illegal stacking exists in the images to be detected is rapidly and accurately determined, and when illegal stacking exists in the images to be detected, illegal stacking information is automatically marked in the monitoring video images corresponding to the images to be detected, manual detection is not needed, and the labor cost for detecting whether articles are illegally stacked is reduced. Meanwhile, the stacking process of the articles can be detected in real time, and the detection efficiency is improved. In addition, the hardware structure needed by the device for detecting illegal article stacking is simple, and the cost is low.
In some embodiments, the detection module 430 includes:
the second acquisition module is used for acquiring a preset violation code image;
the calculation module is used for calculating the similarity between the image to be detected and the illegal code-placement image;
and the judging module is used for determining that illegal stacking exists in the image to be detected if the similarity reaches a preset threshold value.
In some embodiments, the detection module 430 is configured to input the image to be detected into a detection model obtained through training, and detect to determine whether an illegal code placement exists in the image to be detected, where the detection model is obtained by training an initial model according to a sample monitoring video image with the illegal code placement and corresponding illegal code placement information.
In some embodiments, the article illegal stacking detection device 400 further includes:
the third acquisition module is used for acquiring the sample monitoring video images with illegal stacking;
the second labeling module is used for labeling the illegal code placement information in the sample monitoring video image to obtain a labeled image;
and the training module is used for training the initial model according to the sample monitoring video image and the corresponding annotation image to obtain the detection model.
In some embodiments, the second labeling module is configured to label the violation code-placement type and the violation code-placement area in the sample monitoring video image, so as to obtain the labeled image.
In some embodiments, the training module comprises:
the second processing module is used for carrying out normalization processing on the sample monitoring video image to obtain a sample image;
and the sub-training module is used for training the initial model according to the sample image and the corresponding marked image to obtain the detection model.
In some embodiments, the article illegal stacking detection device 400 further includes:
and the display module is used for displaying the monitoring video image marked with illegal stacking information.
The embodiment of the present application further provides a server, which integrates any one of the device for detecting illegal stacking of articles provided by the embodiment of the present application, and the server includes:
one or more processors;
a memory; and
one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to perform the steps of the method for detecting an illegal placing of an item described in any of the above embodiments of the method for detecting an illegal placing of an item.
As shown in fig. 5, it shows a schematic structural diagram of a server according to an embodiment of the present application, specifically:
the server may include components such as a processor 501 of one or more processing cores, memory 502 of one or more computer-readable storage media, a power supply 503, and an input unit 504. Those skilled in the art will appreciate that the server architecture shown in FIG. 5 is not meant to be limiting, and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components. Wherein:
the processor 501 is a control center of the server, connects various parts of the entire server by various interfaces and lines, and performs various functions of the server and processes data by running or executing software programs and/or modules stored in the memory 502 and calling data stored in the memory 502, thereby performing overall monitoring of the server. Optionally, processor 501 may include one or more processing cores; preferably, the processor 501 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 501.
The memory 502 may be used to store software programs and modules, and the processor 501 executes various functional applications and data processing by operating the software programs and modules stored in the memory 502. The memory 502 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data created according to the use of the server, and the like. Further, the memory 502 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, the memory 502 may also include a memory controller to provide the processor 501 with access to the memory 502.
The server further comprises a power supply 503 for supplying power to each component, and preferably, the power supply 503 may be logically connected to the processor 501 through a power management system, so that functions of managing charging, discharging, power consumption, and the like are realized through the power management system. The power supply 503 may also include any component of one or more dc or ac power sources, recharging systems, power failure detection circuitry, power converters or inverters, power status indicators, and the like.
The server may also include an input unit 504, and the input unit 504 may be used to receive input numeric or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.
Although not shown, the server may further include a display unit and the like, which will not be described in detail herein. Specifically, in this embodiment, the processor 501 in the server loads the executable file corresponding to the process of one or more application programs into the memory 502 according to the following instructions, and the processor 501 runs the application program stored in the memory 502, thereby implementing various functions as follows:
acquiring a monitoring video image of article stacking;
carrying out normalization processing on the monitoring video image to obtain an image to be detected;
and inputting the image to be detected into a detection model obtained through training to obtain a detection result of article stacking in the monitoring video image.
It will be understood by those skilled in the art that all or part of the steps of the methods of the above embodiments may be performed by instructions or by associated hardware controlled by the instructions, which may be stored in a computer readable storage medium and loaded and executed by a processor.
To this end, an embodiment of the present application provides a storage medium, which may include: read Only Memory (ROM), Random Access Memory (RAM), magnetic or optical disks, and the like. The storage medium stores a plurality of instructions, and the instructions can be loaded by the processor to execute the steps of any item illegal stacking detection method provided by the embodiment of the application. For example, the instructions may perform the steps of:
acquiring a monitoring video image of article stacking;
carrying out normalization processing on the monitoring video image to obtain an image to be detected;
and inputting the image to be detected into a detection model obtained through training to obtain a detection result of article stacking in the monitoring video image.
In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
The method, the device, the server and the storage medium for detecting illegal article stacking provided by the embodiment of the application are introduced in detail, a specific example is applied in the text to explain the principle and the implementation of the application, and the description of the embodiment is only used for helping to understand the technical scheme and the core idea of the application; those of ordinary skill in the art will understand that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications or substitutions do not depart from the spirit and scope of the present disclosure as defined by the appended claims.
Claims (10)
1. An article illegal stacking detection method is characterized by comprising the following steps:
acquiring a monitoring video image of article stacking;
carrying out normalization processing on the monitoring video image to obtain an image to be detected;
detecting the image to be detected, and determining whether illegal stacking exists in the image to be detected;
and if the illegal code placement exists in the image to be detected, marking illegal code placement information in the monitoring video image corresponding to the image to be detected.
2. The method for detecting the illegal article stacking according to claim 1, wherein the step of detecting the image to be detected and determining whether the illegal article stacking exists in the image to be detected comprises the following steps:
acquiring a preset violation code image;
calculating the similarity between the image to be detected and the violation stacking image;
and if the similarity reaches a preset threshold value, determining that illegal code placement exists in the image to be detected.
3. The method for detecting the illegal article stacking according to claim 1, wherein the step of detecting the image to be detected and determining that the illegal stacking exists in the image to be detected comprises the following steps:
inputting the image to be detected into a detection model obtained through training, detecting, and determining whether the illegal code placement exists in the image to be detected, wherein the detection model is obtained by training an initial model according to a sample monitoring video image with the illegal code placement and corresponding illegal code placement information.
4. The method for detecting illegal article stacking according to claim 3, wherein before the image to be detected is input into a detection model obtained through training for detection and whether illegal stacking exists in the image to be detected is determined, the method further comprises:
acquiring a sample monitoring video image with illegal stacking;
marking the illegal stacking information in the sample monitoring video image to obtain a marked image;
and training an initial model according to the sample monitoring video image and the corresponding annotation image to obtain the detection model.
5. The method for detecting illegal article stacking according to claim 4, wherein the step of labeling the illegal stacking information in the sample monitoring video image to obtain a labeled image comprises the steps of:
and marking the illegal code-placement type and the illegal code-placement area in the sample monitoring video image to obtain the marked image.
6. The method for detecting illegal stacking of articles according to claim 4, wherein the step of training an initial model according to the sample monitoring video image and the corresponding annotation image to obtain the detection model comprises the steps of:
carrying out normalization processing on the sample monitoring video image to obtain a sample image;
and training an initial model according to the sample image and the corresponding marked image to obtain the detection model.
7. The method for detecting illegal article stacking according to claim 1, wherein after the illegal stacking type and the illegal stacking area are marked in the surveillance video image corresponding to the image to be detected if the illegal stacking exists in the image to be detected, the method further comprises:
and displaying the monitoring video image marked with illegal stacking information.
8. The utility model provides an article violation of rules and regulations puts up detection device, its characterized in that, article violation of rules and regulations puts up detection device includes:
the acquisition module is used for acquiring monitoring video images of the stacked articles;
the processing module is used for carrying out normalization processing on the monitoring video image to obtain an image to be detected;
the detection module is used for detecting the image to be detected and determining whether illegal stacking exists in the image to be detected;
and the marking module is used for marking illegal code placement information in the monitoring video image corresponding to the image to be detected if the illegal code placement exists in the image to be detected.
9. A server, characterized in that the server comprises:
one or more processors;
a memory; and
one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the item violation placement detection method of any of claims 1-7.
10. A computer-readable storage medium, having stored thereon a computer program which is loaded by a processor to perform the steps of the method for detecting an illegal palletization of objects according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010159807.5A CN113378602A (en) | 2020-03-10 | 2020-03-10 | Method and device for detecting illegal stacking of articles, server and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010159807.5A CN113378602A (en) | 2020-03-10 | 2020-03-10 | Method and device for detecting illegal stacking of articles, server and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113378602A true CN113378602A (en) | 2021-09-10 |
Family
ID=77568766
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010159807.5A Pending CN113378602A (en) | 2020-03-10 | 2020-03-10 | Method and device for detecting illegal stacking of articles, server and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113378602A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170242148A1 (en) * | 2016-02-22 | 2017-08-24 | Rapiscan Systems, Inc. | Systems and Methods for Detecting Threats and Contraband in Cargo |
| CN109064467A (en) * | 2018-08-20 | 2018-12-21 | 贵州宜行智通科技有限公司 | Analysis method, device and the electronic equipment of community security defence |
| CN109147341A (en) * | 2018-09-14 | 2019-01-04 | 杭州数梦工场科技有限公司 | Violation vehicle detection method and device |
| CN109993047A (en) * | 2017-12-28 | 2019-07-09 | 杭州海康威视系统技术有限公司 | City huddles violation recognition methods, device and the electronic equipment of material |
| CN110348327A (en) * | 2019-06-24 | 2019-10-18 | 腾讯科技（深圳）有限公司 | Realize the method and device that Articles detecting is left in monitoring scene |
| CN110717448A (en) * | 2019-10-09 | 2020-01-21 | 杭州华慧物联科技有限公司 | Dining room kitchen intelligent management system |
| WO2020042489A1 (en) * | 2018-08-30 | 2020-03-05 | 平安科技（深圳）有限公司 | Authentication method and apparatus for illegal parking case, and computer device |
-
2020
- 2020-03-10 CN CN202010159807.5A patent/CN113378602A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170242148A1 (en) * | 2016-02-22 | 2017-08-24 | Rapiscan Systems, Inc. | Systems and Methods for Detecting Threats and Contraband in Cargo |
| CN109993047A (en) * | 2017-12-28 | 2019-07-09 | 杭州海康威视系统技术有限公司 | City huddles violation recognition methods, device and the electronic equipment of material |
| CN109064467A (en) * | 2018-08-20 | 2018-12-21 | 贵州宜行智通科技有限公司 | Analysis method, device and the electronic equipment of community security defence |
| WO2020042489A1 (en) * | 2018-08-30 | 2020-03-05 | 平安科技（深圳）有限公司 | Authentication method and apparatus for illegal parking case, and computer device |
| CN109147341A (en) * | 2018-09-14 | 2019-01-04 | 杭州数梦工场科技有限公司 | Violation vehicle detection method and device |
| CN110348327A (en) * | 2019-06-24 | 2019-10-18 | 腾讯科技（深圳）有限公司 | Realize the method and device that Articles detecting is left in monitoring scene |
| CN110717448A (en) * | 2019-10-09 | 2020-01-21 | 杭州华慧物联科技有限公司 | Dining room kitchen intelligent management system |
Non-Patent Citations (1)
| Title |
|---|
| 陈文玲: "《民航货物运输》", 31 January 2016, 中国民航出版社, pages: 202 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20190325197A1 (en) | Methods and apparatuses for searching for target person, devices, and media | |
| CN110049121B (en) | Data center inspection system based on augmented reality technology | |
| US11715290B2 (en) | Machine learning based models for object recognition | |
| CN112464803A (en) | Image comparison method and device | |
| CN111767915A (en) | License plate detection method, device, equipment and storage medium | |
| CN112861895B (en) | Abnormal article detection method and device | |
| CN115457466A (en) | Inspection video-based hidden danger detection method and system and electronic equipment | |
| CN114157829A (en) | Model training optimization method and device, computer equipment and storage medium | |
| CN112756265A (en) | Article sorting abnormity detection method, device industrial control equipment and storage medium | |
| US10282672B1 (en) | Visual content analysis system with semantic framework | |
| CN115471439A (en) | Method and device for identifying defects of display panel, electronic equipment and storage medium | |
| WO2021138893A1 (en) | Vehicle license plate recognition method and apparatus, electronic device, and storage medium | |
| CN113378602A (en) | Method and device for detecting illegal stacking of articles, server and storage medium | |
| CN108334602B (en) | Data annotation method and device, electronic equipment and computer storage medium | |
| CN108596068B (en) | Method and device for recognizing actions | |
| CN113591839B (en) | Feature extraction model construction method, target detection method and device | |
| CN115620315A (en) | Handwritten text detection method, device, server and storage medium | |
| US20220156311A1 (en) | Image retrieval method and image retrieval system | |
| CN112989957A (en) | Safety monitoring method and system suitable for equipment cabinet | |
| Deniziak et al. | Iot-based smart monitoring system using automatic shape identification | |
| Pietrini et al. | Embedded Vision System for Real-Time Shelves Rows Detection for Planogram Compliance Check | |
| CN110348422B (en) | Image processing method, image processing device, computer-readable storage medium and electronic equipment | |
| Zhao | Convolutional Neural Networks for Abnormal Behavior Detection in a Special Place | |
| CN114742989A (en) | Garbage type identification method and device, computer equipment and storage medium | |
| CN113449559A (en) | Table identification method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |