CN114663834A - Express storage site monitoring method - Google Patents
Express storage site monitoring method Download PDFInfo
- Publication number
- CN114663834A CN114663834A CN202210282392.XA CN202210282392A CN114663834A CN 114663834 A CN114663834 A CN 114663834A CN 202210282392 A CN202210282392 A CN 202210282392A CN 114663834 A CN114663834 A CN 114663834A
- Authority
- CN
- China
- Prior art keywords
- image
- express
- layer
- hidden layer
- neural network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 62
- 238000012544 monitoring process Methods 0.000 title claims abstract description 34
- 230000002159 abnormal effect Effects 0.000 claims abstract description 9
- 230000006870 function Effects 0.000 claims description 34
- 238000013528 artificial neural network Methods 0.000 claims description 32
- 238000003062 neural network model Methods 0.000 claims description 28
- 238000012549 training Methods 0.000 claims description 27
- 239000013598 vector Substances 0.000 claims description 9
- 239000011159 matrix material Substances 0.000 claims description 7
- 230000009286 beneficial effect Effects 0.000 claims description 3
- 238000001514 detection method Methods 0.000 description 14
- 238000011161 development Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 238000004590 computer program Methods 0.000 description 3
- 238000012886 linear function Methods 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 230000000977 initiatory effect Effects 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000001737 promoting effect Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 206010011409 Cross infection Diseases 0.000 description 1
- 241000764238 Isis Species 0.000 description 1
- 206010029803 Nosocomial infection Diseases 0.000 description 1
- 230000005856 abnormality Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000002716 delivery method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000005284 excitation Effects 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000011002 quantification Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
Abstract
The invention provides a method for monitoring an express storage site, which comprises the following steps: s1: the method comprises the steps that a camera acquires first video data of an acquisition area; s2: from the first viewTruncating a first set of images F from the frequency data1Analyzing each image, detecting whether an express delivery person exists, if so, entering S3, otherwise, returning to S1; s3: and sending express delivery notice information to the terminal. The method can autonomously realize monitoring and notification of express delivery and abnormal pickup events, assist the receiving client to remotely verify the express state, and solve the problem of delivery failure caused by the fact that the client cannot check and accept when not on site.
Description
Technical Field
The invention relates to the field of application of intelligent home furnishing and artificial intelligent equipment, in particular to the field of express delivery storage site monitoring methods.
Background
In modern society, along with the development of economy and technology, express delivery gradually becomes an indispensable important industry in people's daily life. Modern people rely more and more on the express delivery industry whether at home or at work. On the other hand, the development of the express delivery industry also plays a role in promoting economy and technology. From the economic perspective, the development of the express industry enables economic activities to be more intensive and efficiency to be higher, and economic development is promoted; from the technical perspective, the express industry has higher requirements on work efficiency, convenience and safety, and has wide requirements on automation and intelligent application, and the application requirements in turn drive the invention and generation of new technologies, thereby promoting the technical progress.
Express delivery is an important link in the express industry. Express delivery is the final link of express logistics, determines the acceptance degree of express service of a recipient client to a greater extent, and is a key link in which client disputes are easy to occur. One of the main contradictions is the conflict between the dispatch time and the client receiving time. Due to the particularity of the industry, express delivery is sensitive to time, and if the delivery time of a courier conflicts with the receiving time of a client, a series of problems such as timeliness of goods, delivery cost, goods loss and the like are easily caused. Therefore, for express enterprises or receiving customers, express delivery has a space to be improved urgently in the aspect.
Especially, in the epidemic situation period, in order to prevent cross infection, need to guarantee the distance between express delivery person and the customer, avoid closely direct contact, the intellectuality of school gate, resident downstairs carries out the judgement that express delivery arrived and is the problem that awaits a urgent need to be solved.
The express cabinet is a more effective scheme for solving the problems, and the storage of goods can be realized at lower cost by setting the encryption storage cabinet capable of automatically storing goods, so that the difficulty that customers can not receive goods in time is solved. There are still several problems with the express cabinet approach. Firstly, many express cabinets push receiving notifications in the modes of short messages, WeChat and the like, and in the modern society of information explosion, the information is easily filtered out as junk information, so that a customer cannot receive the information; even if the customer receives the message, the customer can forget to take the express delivery due to long time intervals and other reasons, so that the express delivery cabinet resources are occupied, and the loss of the goods is easily caused. Secondly, the express cabinet needs to occupy certain resources such as public space resources and electric power, so that the property management needs to be planned in advance, and most existing houses and office environments may not have the conditions, so that the modification and management cost is high. Thirdly, based on the above cost, the express cabinet needs a certain cost for setting and operating, and in the current situation, at the initial stage of deployment, the cost is usually borne by the express enterprise, but the part of cost, especially the operation and maintenance cost, is a long-term and continuous expenditure, so the express enterprise also has a need to transfer the part of cost for business needs, which invisibly increases the overall cost of express delivery.
Disclosure of Invention
In order to solve the problems, the express storage site monitoring method is provided, the express delivery of a courier is autonomously monitored by applying artificial intelligence and machine vision innovation technologies, when an event that the courier delivers the express is monitored, a preset receiving client is informed, the courier is prompted to place the express into a preset position, and a site video is called according to a client instruction to manually verify the express condition; and when the situation that people take the articles from the preset position is monitored, sending an abnormal notice to the receiver client. The method can autonomously realize monitoring and notification of express delivery and abnormal pickup events, assist the receiving client to remotely verify the express state, and solve the problem of delivery failure caused by the fact that the client cannot check and accept when not on site. The method can be realized only by one network camera, the material cost is low, the operation and maintenance cost does not exist, and the problems that the express cabinet occupies public resources and the operation and maintenance cost is high are solved.
The invention provides an express storage site monitoring method, which comprises the following steps:
s1: the method comprises the steps that a camera acquires first video data of an acquisition area;
s2: capturing a first set of images F from first video data1Analyzing each image, acquiring a sub-image of each image, detecting whether an express delivery person exists or not, if so, entering S3, and otherwise, returning to S1;
s3: sending express delivery notice information to a terminal;
using a neural network model N1In the sub-graph it is detected whether an express courier is present,
the neural network model N1The input layer is a sub-graph SI(u ', v'), the output layer of the system is a group of two-dimensional vectors, and the two dimensions of the vectors respectively represent whether an express delivery person exists or not and whether a general pedestrian exists or not in the input graph;
neural network model N1The hidden layer of (2) is defined as follows:
defining a neural network model N1First layer hidden layer:
in the formula ,
represents the weight of a convolution window centered at (u ', v') in the input layer, p, q represent integer coordinates of relative positions in the convolution window, the window size of the convolution is 9 x 9, and the values of p, q are in the range of-4 to 4. SI (u′+p，v′+q)Representing the pixel value of the input layer subgraph at the coordinates (u '+ p, v' + q);
representing a node with coordinates (x, y) in the hidden layer of the first layer, the node being based on the window parameter
Define, connect with 9 x 9 nodes of the input layer. b0Is a linear offset. σ (x) is a non-linear function:
exexpressing an exponential function to enable the neural network to classify the nonlinear data samples, wherein alpha is an empirical parameter;
defining a neural network N1Second layer hidden layer:
in the formula ,
the node with the coordinate (x, y) of the second hidden layer is connected with the node with 4x 4-16 of the first hidden layer, and max represents the node in the first hidden layer
The maximum value of 16 nodes at corresponding positions is defined by p and q in the x and y directions, namely the value ranges of p and q are 0,1, 2 and 3.
Representing a node with coordinates (4x + p, 4y + q) in the first hidden layer. b is a mixture of1Is a linear offset. σ (x) is defined by equation (2);
defining a neural network N1Third hidden layer of (2):
in the formula ,
represents a node with coordinates (x, y) in the hidden layer of the third layer,
representing a node in the second layer hidden layer at coordinates (x + p, y + q),
and representing the weight of a convolution window, wherein the size of the convolution window is 7 x 7, the value range of the corresponding p and q is-3 to 3, and p and q represent integer coordinates of relative positions in the convolution window.
According to
The weights indicated are connected to 7 x 7 nodes in the second hidden layer. b2Is a linear offset. σ (x) is a nonlinear function as defined in equation (2);
defining a neural network N1The fourth hidden layer of (2) is:
in the formula ,
the node with coordinates (x, y) of the fourth hidden layer is connected with the 2x 2-4 nodes of the third hidden layer, and max represents the node in the third hidden layer
The maximum value of 4 nodes corresponding to the positions is defined by p and q in the x and y directions, namely the value ranges of p and q are 0,1, 2 and 3.
And (3) representing a node with the coordinate of (2x + p, 2y + q) in the hidden layer of the third layer. b3Is a linear offset. σ (x) is defined by equation (2);
defining a neural network N1Fifth hidden layer of (2):
in the formula ,
the union represents a node in the fifth hidden layer, so that the fifth layer isIs composed of two-dimensional matrix nodes, wherein (X, y) represents coordinates in X and y directions, subscripts of X represent the number of one of the two matrixes,
representing a node in the fourth hidden layer at coordinates (x + p, y + q),
respectively represent and
and (3) corresponding convolution window weight, wherein the sizes of convolution windows are 7 x 7, the value ranges of p and q are-3 to 3, and p and q represent integer coordinates of relative positions in the convolution windows. In the knowledge that,
push button
The weights indicated are connected to 7 x 7 nodes in the fourth hidden layer,
push button
The weights indicated are connected to 7 x 7 nodes in the fourth hidden layer. b4Is a linear offset. σ (x) is a nonlinear function as defined in equation (2);
by setting up a neural network N1The five hidden layers combine the express deliverer target to be detected with a common pedestrian target, and common characteristics of the express deliverer target and the common pedestrian target are extracted; extracting the difference characteristics of an express delivery person target and a common pedestrian target by using the fifth hidden layer, and further distinguishing target characteristics;
definition of N1The output layer of (2):
in the formula ,
representing the nodes of the fifth hidden layer. Omega ═ omega1，ω2]TRepresenting two nodes of the output layer.
Is shown and
the corresponding connection weight, c, d, is in the same range as x, y, i.e. each
And a is
Correspond to each
And a is
And (7) correspondingly. b5Is a linear offset. σ (x) is defined by the formula (2);
output layer node omega ═ omega of neural network1，ω2]TThe value range is [0,1]]The probabilities of whether an express delivery person exists or not and whether a general pedestrian exists or not in the input map are respectively represented. When ω isiWhen i ═ 1, 2 tends to 0, it means that the target is not present, and when ω is larger than miWhen i ═ {1, 2} tends to 1, it indicates that an object is present. To further clarify the determination result, the thresholds are set as follows:
the weight parameters and bias parameters of each layer of the neural network in the formulas (1) to (7) need to be learned through training samplesAnd (4) obtaining. Preparing a plurality of groups of training samples in advance, wherein the training samples are divided into three classes, the first class is an image of an express delivery person who delivers a package, and the corresponding output is [1,0 ]]The second type is an image of a general pedestrian, corresponding to an output value of [0,1]]The third type is an image containing neither express deliverer nor general pedestrian, and the corresponding output value is 0,0]. Each group of training samples comprises images and output values thereof and is used for training a neural network model N1. Calculating an output result given an input value of the training sample according to the definitions of (1) to (7), and comparing the output result with a labeled value of the training sample to obtain a comparison value, wherein the comparison value is defined as a cost function:
wherein ,
denotes the labeled output value of the sample, omega ═ omega1，ω2]TRepresenting a model according to a neural network N1And calculating an output estimated value after the input image is calculated.
Representing a vector
Is a norm of. The parameter theta is a control parameter, and is beneficial to improving the noise robustness of the model;
the extreme value of the cost function (8) is solved by adopting a back propagation method to realize the neural network model N1Training of (2) determining neural network model N1The parameters of the formulae (1) to (7).
Optionally, the method further includes:
s4, the camera acquires second video data of the acquisition area;
s5, cutting a second group of images F from the second video data2Analyzing each image, obtaining subgraph of each image, detecting graphIf the express parcel exists in the sub-area, the step S6 is carried out, and if the express parcel does not exist in the sub-area, the step S4 is carried out;
and S6, sending express delivery completion notification information to the terminal.
Optionally, the method further includes:
s7, the camera acquires third video data of the acquisition area;
s8, cutting a third group of images F from the third video data3Analyzing each image, detecting whether express packages exist in a designated area in the image, if so, entering S9, otherwise, returning to S7;
and S9, sending express package abnormal notification information to the terminal.
Optionally, in S2, a plurality of sub-images in each image are obtained, and a neural network model N is adopted according to the specific characteristics of the dispatcher1And detecting whether the express deliverer exists from the subgraph.
Optionally, for each complete image I, the method for acquiring the subgraph includes:
ss.a, size of given subgraph;
SS.B, giving a step value;
SS.C, taking the initial pixel of the complete image I as a reference pixel, taking the reference pixel as a capture starting point, and capturing a sub-image according to the size given in the step SS.A;
ss.d, shifting the reference pixel of step ss.c by a step value given by ss.b in two independent directions of the image;
and SS.E, repeating the steps of SS.C and SS.D by taking a new reference pixel as a starting point until a new sub-image cannot be intercepted.
Optionally, in S5, the designated sub-region is a preset rectangular region [ (u)1，v1)，(u2，v2)，(u3，v3)，(u4，v4)]For marking the location of the courier deposit (u)1，v1)，(u2，v2)，(u3，v3)，(u4，v4) For the seating of four vertices of a rectangular region in an imageAnd (4) marking.
Optionally, the method for detecting whether an express package exists in the designated sub-area in the image includes:
when the express package is not delivered to the designated position, one frame of image is intercepted from the video shot by the camera, and the subgraph of the image in a preset area is recorded as a reference image R (u ', v') without the express package;
given another image I (u, v) to be determined in a sub-image S in the set areaI(u ', v'), taking the difference:
D(u′，v′)＝|SI(u′，v′)-R(u′，v′)|
representing the absolute value of the difference between the pixels of the two sub-images at the corresponding positions;
introducing a neural network model N2The input layer is a difference map D (u ', v'), and the output layer is a scalar psi, which indicates that there is an express parcel in the input image when psi is 1, and indicates that there is no express parcel in the input image when psi is 0.
Optionally, the capturing is performed by capturing a plurality of images from the video data at equal intervals within a certain time period.
Optionally, if the number of the images of the express delivery person exceeds a first preset value and at least one of the images does not detect a general pedestrian, it is determined that the express delivery person exists.
Optionally, if the number of the images of the express package is detected to exceed the second preset value and the image shooting sequence is continuous, the package is determined to exist.
The invention has the advantages that:
1. the invention innovatively provides that three events, such as (1) the arrival of an express delivery person, (2) the express delivery is placed at a designated position, and (3) the express is taken away from the designated position, are automatically identified, and the event processing analysis is carried out, so that the delivery and abnormal conditions of the express are rapidly and accurately monitored and judged.
2.A plurality of special neural networks are designed to accurately detect three events such as (1) express delivery person visiting, (2) express delivery placing at an appointed position, and (3) express delivery taking away from the appointed position, so that a network structure, an excitation function and a cost function are optimized, the detection accuracy is improved, and the detection time is considered.
3. The invention innovatively provides an express deliverer and general pedestrian detection method in a single-frame image based on a neural network, wherein an express deliverer target to be detected and a general pedestrian target are combined, and common characteristics of the express deliverer target and the general pedestrian target are extracted; particularly, the difference characteristics of the express delivery person target and the common pedestrian target are extracted by utilizing the hidden layer, so that the target characteristics can be further distinguished, and the target detection performance is improved.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
FIG. 1 is a diagram illustrating the components and relationships in an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
The embodiment of the invention provides a method for monitoring an express storage site, which comprises the following steps:
s1: the method comprises the steps that a camera acquires first video data of an acquisition area;
s2: capturing a first set of images F from first video data1Analyzing each image, detecting whether an express delivery person exists, if so, entering S3, otherwise, returning to S1;
s3: and sending express delivery notice information to the terminal.
Based on this, the above method further comprises: s4, the camera acquires second video data of the acquisition area;
s5, cutting a second group of images F from the second video data2Analyzing each image, detecting whether express packages exist in a designated area in the image, if so, entering S6, otherwise, returning to S4;
and S6, sending express delivery completion notification information to the terminal.
Further, the above method further comprises:
s7, the camera acquires third video data of the acquisition area;
s8, cutting a third group of images F from the third video data3Analyzing each image, detecting whether express packages exist in a designated area in the image, if so, entering S9, otherwise, returning to S7;
and S9, sending express package abnormal notification information to the terminal.
In the step 1-3, the method for acquiring the images and videos of the express storage site utilizes a wireless network camera assembled on the express storage site to acquire continuous videos for different intelligent methods. The client installs service software on a mobile phone or a personal computer, and the service software is communicated with the camera through a network to obtain the collected video and images and control the start and stop of the camera. Please refer to fig. 1.
The express delivery storage site refers to a place where a customer makes an appointment and needs to monitor to place the express delivery, such as a home door, an office door and the like, and generally belongs to a private place of the customer. In order to use the present invention and apparatus, the customer should have the right to take pictures and video of the scene.
The wireless network camera is a general camera device with wireless network access, communication function and video image recording function, and the camera can provide the function that other software systems can access the data collected by the camera in real time in the forms of secondary development and the like.
The service software is installed on general computing equipment such as a personal mobile phone or a personal computer of a client and provides functions of dispatch monitoring and notification, the software acquires videos and images of a camera through a communication network, automatically analyzes the acquired video and image data, further acquires an analysis result and notifies the user of the analysis result in preset content and format.
The images and videos are collected by a camera, and a two-dimensional and three-dimensional digital matrix is obtained by preprocessing methods such as sampling and quantification. Wherein the images are a mapped subset of the video in the time dimension. Several frames of images can be intercepted from the video, or a group of videos can be synthesized by several frames of images.
The customer mentioned here refers to a natural person or organization at home or at work, and is generally a recipient of express delivery goods, and the express delivery cannot be timely collected due to various reasons, so that the method and the device described here have use requirements. The method described herein is not limited theoretically or technically to the identity or role of the client.
In step 2, the express deliverer and the general pedestrian are detected in the single frame image, and the express deliverer and the general pedestrian are automatically detected from the single frame image and can be distinguished; directly perceived, express delivery person holds or utilizes the carrier to load the express delivery parcel, and general pedestrian does not possess above-mentioned characteristic. The detection method comprises the steps of inputting an image, and outputting an image coordinate set for marking the area of an express delivery person in the image, or outputting an image coordinate set for marking the area of a general pedestrian in the image; the image coordinate set is a vertex coordinate set of a rectangle surrounded by four points.
And intercepting a frame of image from the video shot by the camera, and detecting whether the express deliverer or the common pedestrian exists in the step. The image is represented in digital form by a two-dimensional matrix:
I(u，v)
wherein, I represents an image, I (u, v) represents an element in a two-dimensional matrix of the image, or a pixel of the image, and u, v represent coordinates of the pixel in the image I
A subset of the image matrix I is called a subgraph of I:
SI(u′，v′)
wherein ,SITo representSubfigure of I, SI(u ', v') represents a pixel in the sub-picture, and u ', v' represents the coordinates of the pixel in the sub-picture. At this time, I is referred to as the original image.
From an image I, a plurality of subgraphs can be obtained, which comprises the following steps:
ss.a, size of given subgraph;
SS.B, giving a step value;
SS.C, taking the initial pixel of the complete image I as a reference pixel, taking the reference pixel as an interception starting point, and intercepting a sub-image according to the size given by the step SS.A;
ss.d, shifting the reference pixel of step ss.c by a step value given by ss.b in two independent directions of the image;
and SS.E, repeating the steps of SS.C and SS.D by taking the new reference pixel as a starting point until a new sub-image cannot be intercepted.
Through the steps SS.A-SS.E, a plurality of sub-images can be intercepted, all positions of the complete image are covered, and if the target of an express delivery person or a common pedestrian exists in the original image I, the target can be detected from the sub-images through an intelligent algorithm.
If the size of the subgraph is not suitable, detecting whether the target exists or not by resetting the size of the subgraph and repeating the steps SS.A-SS.E.
If all possible options are tried, and the target cannot be detected from any sub-image, the corresponding target does not exist in the original image.
As a preferred configuration, the sub-picture sizes are set herein to 1/16, 1/12, 1/8, 1/6 of the original image size, and the step values are set to 4 pixels in the original image.
Using a neural network model N1And detecting an express delivery person or a general pedestrian target from the subgraph.
According to a general definition, a neural network model consists of an input layer, a hidden layer and an output layer; the hidden layer is composed of a plurality of layers, wherein the first hidden layer is a logic operation result of the input layer, the subsequent hidden layer is a logic operation result of the previous hidden layer, and the output layer is a logic operation result of the last hidden layer; each layer of the model consists of a plurality of scalars, which are also called nodes, and the number of layers of the hidden layer refers to the number of nodes passing through the shortest path from the layer to the input layer; the logical operation relationship between the nodes is defined by connection, and no connection exists between the nodes with the same layer number.
Neural network model N as described herein1The input layer is a sub-graph SI(u ', v'), the output layer is a group of two-dimensional vectors, and the two dimensions of the vectors respectively represent whether an express delivery person exists or not and whether a general pedestrian exists or not in the input graph.
Neural network model N1The hidden layer of (2) is defined as follows.
Defining a neural network model N1First layer hidden layer:
in the formula ,
represents the weight of a convolution window centered at (u ', v') in the input layer, p, q represent integer coordinates of relative positions in the convolution window, the window size of the convolution is 9 x 9, and the values of p, q are in the range of-4 to 4. SI (u′+p，v′+q)Representing the pixel value of the input layer subgraph at the coordinates (u '+ p, v' + q);
representing a node with coordinates (x, y) in the hidden layer of the first layer, the node being based on the window parameter
Define, connect to 9 x 9 nodes of the input layer. b0Is a linear offset. σ (x) is a non-linear function:
exan exponential function is expressed to enable the neural network to classify the nonlinear data samples, and α is an empirical parameter, preferably α ═ 10. The non-linear function helps to improve the classification effect of the model compared to the classical classification function.
Defining a neural network N1Second layer hidden layer:
in the formula ,
the node with the coordinate (x, y) of the second hidden layer is connected with the node with 4x 4-16 of the first hidden layer, and max represents the node in the first hidden layer
The maximum value of 16 nodes at corresponding positions is defined by p and q in the x and y directions, namely the value ranges of p and q are 0,1, 2 and 3.
Representing a node with coordinates (4x + p, 4y + q) in the first hidden layer. b1Is a linear offset. σ (x) is defined by equation (2).
Defining a neural network N1Third hidden layer of (2):
in the formula ,
represents a node with coordinates (x, y) in the hidden layer of the third layer,
representing a node at coordinates (x + p, y + q) in the second hidden layer,
and representing the weight of a convolution window, wherein the size of the convolution window is 7 x 7, the value range of the corresponding p and q is-3 to 3, and p and q represent integer coordinates of relative positions in the convolution window.
According to
The weights indicated are connected to 7 x 7 nodes in the second hidden layer. b2Is a linear offset. σ (x) is a nonlinear function as defined in equation (2).
Defining a neural network N1The fourth hidden layer is:
in the formula ,
the node with the coordinate (x, y) of the fourth hidden layer is connected with 2x 2-4 nodes of the third hidden layer, and max represents the node in the third hidden layer
The maximum value of 4 nodes corresponding to the positions is defined by p and q in the x and y directions, namely the value ranges of p and q are 0,1, 2 and 3.
And (3) representing a node with the coordinate of (2x + p, 2y + q) in the hidden layer of the third layer. b3Is a linear offset. σ (x) is defined by equation (2).
Defining a neural network N1Fifth hidden layer of (2):
in the formula ,
the associations represent nodes in the fifth hidden layer, and thus the fifth layer is composed of two-dimensional matrix node associations, (X, y) representing coordinates in the X, y directions, the subscript of X representing the number of one of the two matrices,
represents a node at coordinates (x + p, y + q) in the fourth hidden layer,
respectively represent and
and corresponding convolution window weights, the sizes of convolution windows are 7 × 7, the value ranges of corresponding p and q are-3 to 3, and p and q represent integer coordinates of relative positions in the convolution windows. In a clear view of the above, it is known that,
push button
The weights indicated are connected to 7 x 7 nodes in the fourth hidden layer,
push button
The weights are shown connected to 7 x 7 nodes in the fourth hidden layer. b4Is a linear offset. σ (x) is a nonlinear function as defined in equation (2).
By setting up a neural network N1The five hidden layers combine the express delivery member target to be detected with a common pedestrian target, and common characteristics of the express delivery member target and the common pedestrian target are extracted; in particular, the fifth layer hidden layer is used for extracting express deliverersThe difference characteristic of the target and the common pedestrian target is beneficial to further distinguishing the target characteristics and improving the performance of target detection.
Definition of N1The output layer of (2):
in the formula ,
representing the nodes of the fifth hidden layer. Omega ═ omega1，ω2]TRepresenting two nodes of the output layer.
Is shown and
the corresponding connection weight, c, d, is in the same range as x, y, i.e. each
And a is
Correspond to each
And a is
And (7) corresponding. b5Is a linear offset. σ (x) is defined by equation (2).
Output layer node omega ═ omega of neural network1，ω2]TThe value range is [0,1]]The probabilities of whether an express delivery person exists or not and whether a general pedestrian exists or not in the input map are respectively represented. When omegaiWhen i ═ {1, 2} tends to 0, it indicates that the target is absent, and when ω is equal toiWhen i ═ {1, 2} tends to 1, it indicates that an object is present. To further clarify the determination result, letThe thresholding is as follows:
the weight parameters and bias parameters of each layer of the neural network in the formulas (1) to (7) need to be obtained through training sample learning. Preparing a plurality of groups of training samples in advance, wherein the training samples are divided into three classes, the first class is an image of an express delivery person who delivers a package, and the corresponding output is [1,0 ]]The second type is an image of a general pedestrian, and the corresponding output value is [0,1]]The third type is an image containing neither express deliverer nor general pedestrian, and the corresponding output value is 0,0]. Each group of training samples comprises images and output values thereof and is used for training a neural network model N1. Calculating the output result given the input value of the training sample according to the definitions of (1) to (7), and comparing the output result with the label value of the training sample to obtain a comparison value, wherein the comparison value is defined as a cost function:
wherein ,
denotes the labeled output value of the sample, omega ═ omega1，ω2]TRepresenting a model according to a neural network N1And calculating the input image and then outputting an estimated value.
Representing a vector
A norm of. The parameter theta is a control parameter, and contributes to improving the noise robustness of the model. Preferably, θ is 0.045.
The extreme value of the cost function (8) is solved by adopting a backward propagation method to realize the neural network model N1Training of (2) determining neural network model N1The parameters of the formulae (1) to (7).
Through the model, an express delivery person target and a common pedestrian target can be detected and distinguished, and more accurate discrimination of events according to target types in subsequent steps is facilitated.
And 4-6, a method for detecting express packages at the designated position is used, and whether the express packages exist in the designated sub-area or not is automatically detected from one frame of image. The method inputs an image including a predetermined rectangular region (u)1，v1)，(u2，v2)，(u3，v3)，(u4，v4)]For marking the location of the courier deposit (u)1，v1)，(u2，v2)，(u3，v3)，(u4，v4) Coordinates of four vertexes of the rectangular area in the image; and outputting a scalar value psi, wherein when psi is 1, the express parcel is detected, or psi is 0, the express parcel is not detected in the area.
When the express package is not delivered to the designated position, one frame of image is intercepted from the video shot by the camera, and the sub-image of the image in the preset area is recorded as the reference image R (u ', v') without containing the express package.
Given another image I (u, v) to be determined in a sub-image S in the set areaI(u ', v'), taking the difference:
D(u′，v′)＝|SI(u′，v′)-R(u′，v′)|…(9)
representing the absolute value of the difference between the pixels of the two sub-images at the corresponding locations.
Introducing a neural network model N2The input layer is a difference map D (u ', v'), and the output layer is a scalar psi, which indicates that there is an express parcel in the input image when psi is 1, and indicates that there is no express parcel in the input image when psi is 0. Through regard as neural network input with above-mentioned difference, can effectively get rid of other express delivery parcel interferences. For example, a package may already be present at the location before the package arrives, and may cause an identification error. This is also the inventionOne of the inventions of (1).
Neural network model N2The hidden layer of (2) is defined as follows.
Defining a neural network model N2First layer hidden layer:
in the formula ,
represents the weight of a convolution window centered at (u ', v') in the input layer, p, q represent integer coordinates of relative positions in the convolution window, the window size of the convolution is 9 x 9, and the values of p, q are in the range of-4 to 4. D(u′+p，v′+q)Representing a pixel value of a difference subgraph of the input layer at coordinates (u '+ p, v' + q);
representing a node with coordinates (x, y) in the hidden layer of the first layer, the node being based on the window parameter
Define, connect with 9 x 9 nodes of the input layer. e.g. of the type0Is a linear offset. Sigma (x) and model N1Wherein said definitions (2) are the same.
Defining a neural network N2Second layer hidden layer:
in the formula ,
the node with the coordinate (x, y) of the second hidden layer is connected with the node with 4x 4-16 of the first hidden layer, and max represents the node in the first hidden layer
The maximum value of 16 nodes at corresponding positions is defined by p and q in the x and y directions, namely the value ranges of p and q are 0,1, 2 and 3. Y is1 (4x+p，4y+q)Representing a node with coordinates (4x + p, 4y + q) in the first hidden layer. e.g. of the type1Is a linear offset. σ (x) is defined by equation (2).
Definition of N2The output layer of (2):
in the formula ,
representing a node of a second layer of hidden layers. Psi denotes the nodes of the output layer.
Is shown and
the corresponding connection weight, c, d, is in the same range as x, y, i.e. each
And a is
And (7) corresponding. e.g. of the type2Is a linear offset. σ (x) is defined by equation (2).
The value range of the output layer node psi of the neural network is [0,1], which indicates whether the original image corresponding to the input difference image has a parcel or not. When ψ tends to 0, it indicates that the parcel is not present, and when ψ tends to 1, it indicates that the parcel is present. To further clarify the determination result, the thresholds were set as follows:
step (b) and step (b)Similarly in step 2, training sample images with and without parcels are prepared in advance and labeled to have an output value of 1 or 0, respectively, for model N2Training is performed to obtain the values of the parameters in equations (10) - (12). The cost function is defined as follows:
wherein ,
representing the output values of the labels of the training samples,. psi.2And calculating the input difference image and then outputting an estimated value. The parameter theta is a control parameter, and contributes to improving the noise robustness of the model. Preferably, θ is 0.065.
Through the model, whether packages exist in the preset area can be detected and used as a basis for autonomous judgment of events in subsequent steps.
And 7-9, a monitoring and informing method for express delivery and abnormal receiving, which means that after a client starts service software, the service software automatically runs in a background of client equipment, receives video data transmitted by a camera, performs autonomous detection according to a related method, further judges the occurrence of a monitoring event according to a detection result, and informs the client of the occurrence of the event.
The events described herein to notify the client include the following three categories: (1) the express delivery method comprises the following steps of (1) getting on a door by an express delivery person, (2) placing the express at a designated position, and (3) taking away the express from the designated position. Further, it is agreed that, in the above three types of events, the notification of the event (2) occurs only after the notification of the event (1) occurs, and the notification of the event (3) occurs only after the notification of the event (1) occurs.
In order to realize the autonomous detection and notification of certain events, a plurality of frames of images are acquired within a certain time by adopting a related method, the images are autonomously detected, whether the events occur is further judged, and the notification is initiated when the events occur. The related methods and steps are described in detail in steps 2-6 herein.
The service software is operated on general terminal equipment such as a mobile phone, a personal computer and the like specified by a client, and the specific implementation of the terminal equipment does not influence the implementation of the method. And after the service software is installed, entering an operating state according to a client instruction, and operating in a resident service mode in the equipment background. When the service is running, video data transmitted by the camera is automatically received, a relevant event is detected, and a notification is initiated to the client when the event is detected to occur.
The video data receiving function, the event detecting function and the notification function of the service can be respectively set and operated on different terminal devices according to requirements so as to better realize the function of the service. For example, the video data receiving function and the event detection function are arranged on a personal computer, so that the high computing performance of the personal computer is better utilized, and the data processing and detection efficiency is improved; meanwhile, the notification initiating function is arranged on the mobile phone terminal so as to notify the user in time, and the mobile phone is only used for initiating event notification, thereby greatly reducing the mobile communication flow, reducing the additional cost and improving the communication efficiency.
According to the express delivery monitoring and notifying method, after a client installs a camera, a rectangular area is preset in service software and used for marking an express delivery and monitoring area.
The express delivery monitoring and notifying method is realized by judging whether the following events occur or not: the event (1) is that a courier is on the door, and the event (2) is that a courier is put at a designated position. The express delivery event is a sequential combination of the event (1) and the event (2).
The event (1) occurrence condition judgment step is as follows:
s1.A in a certain time period T1Inner, equal interval intercepting F from video1And (3) detecting express deliverers and general pedestrians by adopting the method in the step 2 for each image, and recording the detection result.
S1.B if all F in step S1.A1The express deliverers are detected in all the images, namely omega in step 21Is more than 0.5, and at least one image has no general pedestrian detected,i.e. omega in step 22And (5) judging that the event (1) occurs if the value is less than or equal to 0.5.
S1.C otherwise, repeating the steps S1.A-S1.B until the event (1) occurs.
The service software in the step 1 sends a notice sent by the event (1) to the client to inform the client that an express delivery person is on the door after the event (1) occurs according to the method in the steps S1.A, S1.B and S1. C; and begins transmitting live video to the customer. The client can check the field video through the service software and observe the express delivery condition.
The above parameter T1、F1It has been empirically obtained that T is preferred herein through a number of experiments15 seconds, F1＝11.
When the event (1) occurs, the monitoring step of the event (2) is entered.
The step of judging the occurrence condition of the event (2) is as follows:
s2.A event (1) has occurred.
S2.B in a certain time period T2Inner, equal interval intercepting F from video2And (3) detecting the express packages by adopting the method in the step 3 for each image, and recording the detection result.
S2.C if the step S2.B exceeds H2Sheet image (H)2＜F2) Wrap detected, i.e. > 0.5 in step 3, and this H2If the image capturing order is continuous, it is determined that the event (2) has occurred.
And S2.D, otherwise, repeating the steps S2.B and S2. C.
The service software in the step 1 sends a notice of express delivery completion to the client according to the method in the steps S2.A, S2.B, S2.C and S2.D after the event (1) and the event (2) are both generated.
The above parameter T2、F2、H2It has been empirically obtained that T is preferred herein by a number of experiments210 seconds, F2＝6，H2＝4.
After the events (1) and (2) occur, the service software stops recording the field video to the client; and enters the monitoring step of event (3).
The step of judging the occurrence condition of the event (3) is as follows:
and S3.A events (1) and (2) occur.
S3.B in a certain time period T3Inner, equal interval intercepting F from video3And (3) detecting the express packages by adopting the method in the step 3 for each image, and recording the detection result.
S3.C if the step S3.B exceeds M3Sheet image (H)2＜F2) No package is detected, i.e. ψ ≦ 0.5 in step 3, and this M3If the image capturing order is continuous, it is determined that the event (3) has occurred.
And S3.D, otherwise, repeating the steps S3.B and S3. C.
The service software in the step 1 sends a notice of express package monitoring abnormity to the client after the event (3) occurs according to the method in the steps S3.A, S3.B, S3.C and S3.D, and sends the on-site video within a certain time (such as 60 seconds) before the event (3) occurs to the client.
After the client starts the service software, the service software automatically runs in the background of the client equipment, receives video data transmitted by the camera, automatically detects according to the step and the preorder step, automatically judges monitoring events such as express delivery, express abnormity and the like, informs the client of the occurred events, and starts a site video viewing function. By the method, the remote monitoring of the express delivery by the client is realized, and the property safety of the client is protected.
Table 1 shows test results of the method described herein, which include two types of indicators, that is, express delivery monitoring recognition rate and express delivery abnormality monitoring recognition rate. The express delivery monitoring and identifying rate is the probability that a courier can autonomously and correctly judge and initiate delivery event notification after delivering an express; the express abnormity monitoring and identifying rate is the probability that when someone takes the express from the site, the express can be automatically and correctly judged and abnormal event notification can be initiated. The response time refers to the time difference between the express delivery time and the time when the express delivery time is taken to the time when the method and the system autonomously report the occurrence of the event. The test result shows that the method has high recognition rate on two types of events and short response period, and realizes the autonomous and intelligent remote express delivery monitoring function.
TABLE 1
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. The procedures or functions according to the present application are generated in whole or in part when the computer program instructions are loaded and executed on a computer. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored on a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (e.g., coaxial cable, fiber optic, digital subscriber line) or wirelessly (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk), among others.
It is to be understood that the present invention includes, in addition to the above, conventional structures and conventional methods, which are well known and will not be described in detail. It is not intended that such structures and methods be present in the present invention.
It will be appreciated by those skilled in the art that while a number of exemplary embodiments of the invention have been shown and described in detail herein, many other variations or modifications can be made in accordance with the principles of the invention, which are directly identified or derived from the disclosure of the invention, without departing from the spirit and scope of the invention. Accordingly, the scope of the invention should be understood and interpreted to cover all such other variations or modifications.
Claims (9)
1. An express delivery storage site monitoring method is characterized by comprising the following steps:
b1: the method comprises the steps that a camera acquires first video data of an acquisition area;
s2: intercepting a first group of images F1 from the first video data, analyzing each image, acquiring a plurality of sub-images of each image, detecting whether an express delivery person exists, if so, entering S3, otherwise, returning to S1;
s3: sending express delivery notice information to a terminal;
using a neural network model N1In the sub-graph it is detected whether an express courier is present,
the neural network model N1The input layer is a sub-graph SI(u ', v'), the output layer of the system is a group of two-dimensional vectors, and the two dimensions of the vectors respectively represent whether an express delivery person exists or not and whether a general pedestrian exists or not in the input graph;
neural network model N1The hidden layer of (2) is defined as follows:
defining a neural network model N1First layer hidden layer:
in the formula ,
represents the weight of a convolution window centered at (u ', v') in the input layer, p, q represent integer coordinates of relative positions in the convolution window, the window size of the convolution is 9 x 9, and the values of p, q are in the range of-4 to 4. SI (u′+p，v′+q)Represents the pixel value of the input layer subgraph at the coordinates (u '+ p, v' + q);
representing a node with coordinates (x, y) in the hidden layer of the first layer, the node being based on the window parameter
Define, connect with 9 x 9 nodes of the input layer. b0Is a linear offset. σ (x) is a nonlinear function:
exexpressing an exponential function to enable the neural network to classify the nonlinear data samples, wherein alpha is an empirical parameter;
defining a neural network N1Second layer hidden layer:
in the formula ,
the node with the coordinate (x, y) of the second hidden layer is connected with the node with 4x 4-16 of the first hidden layer, and max represents the node in the first hidden layer
The maximum value of 16 nodes at corresponding positions is defined by p and q in the x and y directions, namely the value ranges of p and q are 0,1, 2 and 3.
Representing a node with coordinates (4x + p, 4y + q) in the first hidden layer. b1Is linearAn offset amount. σ (x) is defined by equation (2);
defining a neural network N1Third hidden layer of (2):
in the formula ,
represents a node with coordinates (x, y) in the hidden layer of the third layer,
representing a node at coordinates (x + p, y + q) in the second hidden layer,
and representing the weight of a convolution window, wherein the size of the convolution window is 7 x 7, the value range of the corresponding p and q is-3 to 3, and p and q represent integer coordinates of relative positions in the convolution window.
According to
The weights indicated are connected to 7 x 7 nodes in the second hidden layer. b2Is a linear offset. σ (x) is a nonlinear function as defined in equation (2);
defining a neural network N1The fourth hidden layer of (2) is:
in the formula ,
denotes the fourth hidden layer coordinate as (x, y)) And max represents the node in the third hidden layer and the node in the third hidden layer
The maximum value of 4 nodes corresponding to the positions is defined by p and q in the x and y directions, namely the value ranges of p and q are 0,1, 2 and 3.
And (3) representing a node with the coordinate of (2x + p, 2y + q) in the hidden layer of the third layer. b3Is a linear offset. σ (x) is defined by the formula (2);
defining a neural network N1Fifth hidden layer of (2):
in the formula ,
the associations represent nodes in the fifth hidden layer, and the fifth layer is thus composed of two-dimensional matrix node associations, (X, y) representing coordinates in the X, y directions, the subscript of X representing the number of one of the two matrices,
representing a node in the fourth hidden layer at coordinates (x + p, y + q),
respectively represent and
and (3) corresponding convolution window weight, wherein the sizes of convolution windows are 7 x 7, the value ranges of p and q are-3 to 3, and p and q represent integer coordinates of relative positions in the convolution windows. In a clear view of the above, it is known that,
push button
The weights indicated are connected to 7 x 7 nodes in the fourth hidden layer,
push button
The weights indicated are connected to 7 x 7 nodes in the fourth hidden layer. b4Is a linear offset. σ (x) is a nonlinear function as defined in equation (2);
by setting up a neural network N1The five hidden layers combine the express delivery member target to be detected with a common pedestrian target, and common characteristics of the express delivery member target and the common pedestrian target are extracted; extracting the difference characteristics of an express delivery person target and a common pedestrian target by using the fifth hidden layer, and further distinguishing target characteristics;
definition of N1The output layer of (2):
in the formula ,
representing the nodes of the fifth hidden layer. Omega ═ omega1，ω2]TRepresenting two nodes of the output layer.
Is shown and
the corresponding connection weight, c, d, is in the same range as x, y, i.e. each
And a is
Correspond to each
And a is
And (7) corresponding. b5Is a linear offset. σ (x) is defined by equation (2);
output layer node omega ═ omega of neural network1，ω2]TThe value range is [0,1]]The probabilities of whether an express delivery person exists or not and whether a general pedestrian exists or not in the input map are respectively represented. When ω isiWhen i ═ {1, 2} tends to 0, it indicates that the target is absent, and when ω is equal toiWhen i ═ {1, 2} tends to 1, it indicates that an object is present. To further clarify the determination result, the thresholds were set as follows:
the weight parameters and bias parameters of each layer of the neural network in the formulas (1) to (7) need to be obtained through training sample learning. Preparing a plurality of groups of training samples in advance, wherein the training samples are divided into three types, the first type is an image of an express delivery dispatcher who dispatches a package, and the corresponding output is [1,0 ]]The second type is an image of a general pedestrian, corresponding to an output value of [0,1]]The third type is an image containing neither express deliverer nor general pedestrian, and the corresponding output value is 0,0]. Each group of training samples comprises images and output values thereof and is used for training a neural network model N1. Calculating the output result given the input value of the training sample according to the definitions of (1) to (7), and comparing the output result with the label value of the training sample to obtain a comparison value, wherein the comparison value is defined as a cost function:
wherein ,
denotes the labeled output value of the sample, omega ═ omega1，ω2]TRepresenting a model according to a neural network N1And calculating an output estimated value after the input image is calculated.
Representing a vector
A norm of. The parameter theta is a control parameter and is beneficial to improving the noise robustness of the model;
the extreme value of the cost function (8) is solved by adopting a backward propagation method to realize the neural network model N1Training of (2) determining neural network model N1The parameters of the formulae (1) to (7).
2. The monitoring method of claim 1, wherein the method further comprises: s4, the camera acquires second video data of the acquisition area;
b5, intercepting a second set of images F from the second video data2Analyzing each image, detecting whether express packages exist in a designated area in the image, if so, entering S6, otherwise, returning to S4;
and S6, sending express delivery completion notification information to the terminal.
3. The monitoring method of claim 2, wherein the method further comprises:
s7, the camera acquires third video data of the acquisition area;
s8, cutting a third group of images F from the third video data3Analyzing each image, detectingWhether express packages exist in the designated sub-area or not is judged in the image, if yes, the step S9 is carried out, and if not, the step S7 is carried out;
and S9, sending express package abnormal notification information to the terminal.
4. The monitoring method of claim 1, wherein for each complete image I, the sub-graph acquisition method comprises:
ss.a, size of given subgraph;
SS.B, giving a step value;
SS.C, taking the initial pixel of the complete image I as a reference pixel, taking the reference pixel as an interception starting point, and intercepting a sub-image according to the size given by the step SS.A;
ss.d, shifting the reference pixel of step ss.c by a step value given by ss.b in two independent directions of the image;
and SS.E, repeating the steps of SS.C and SS.D by taking a new reference pixel as a starting point until a new sub-image cannot be intercepted.
5. The monitoring method according to claim 2 or 3, wherein in the step S5, the designated sub-region is a predetermined rectangular region [ (u) is1，v1)，(u2，v2)，(u3，v3)，(u4，v4)]For marking the location of the courier deposit (u)1，v1)，(u2，v2)，(u3，v3)，(u4，v4) Is the coordinate of four vertexes of the rectangular area in the image.
6. A method of monitoring as claimed in claim 2 or 3, wherein the method of detecting the presence of an express parcel in a designated sub-area of the image comprises:
when the express package is not delivered to the designated position, capturing a frame of image from a video shot by a camera, and recording a sub-image of the frame of image in a preset area as a reference image R (u ', v') without the express package;
given another image I (u, v) to be determined in a sub-image S in the set areaI(u ', v'), taking the difference:
D(u′，v′)＝|SI(u′，v′)-R(u′，v′)|
representing the absolute value of the difference between the pixels of the corresponding positions of the two sub-images;
introducing a neural network model N2The input layer is a difference map D (u ', v'), and the output layer is a scalar psi, which indicates that there is an express parcel in the input image when psi is 1, and indicates that there is no express parcel in the input image when psi is 0.
7. A method of monitoring as claimed in any one of claims 1 to 3, wherein the capturing is performed by capturing a plurality of images from the video data at regular intervals over a period of time.
8. The monitoring method according to claim 1, wherein the presence of the express courier is determined when the number of images of the express courier is detected to exceed a first preset value and at least one of the images does not detect a general pedestrian.
9. A monitoring method according to claim 2 or 3, characterised in that the presence of a parcel is determined if the number of images of the express parcel is detected to exceed a second predetermined value and the sequence of images taken is continuous.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210282392.XA CN114663834B (en) | 2022-03-22 | 2022-03-22 | On-site monitoring method for express storage |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210282392.XA CN114663834B (en) | 2022-03-22 | 2022-03-22 | On-site monitoring method for express storage |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114663834A true CN114663834A (en) | 2022-06-24 |
| CN114663834B CN114663834B (en) | 2023-04-28 |
Family
ID=82031729
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210282392.XA Active CN114663834B (en) | 2022-03-22 | 2022-03-22 | On-site monitoring method for express storage |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114663834B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114863734A (en) * | 2022-07-06 | 2022-08-05 | 新乡学院 | Evaluation method of organic polymer material synthesis experiment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107832799A (en) * | 2017-11-20 | 2018-03-23 | 北京奇虎科技有限公司 | Object identifying method and device, computing device based on camera scene |
| CN111757069A (en) * | 2020-07-10 | 2020-10-09 | 广州博冠智能科技有限公司 | Monitoring anti-theft method and device based on intelligent doorbell |
| CN113962363A (en) * | 2021-10-19 | 2022-01-21 | 国网河南省电力公司新野县供电公司 | Low-voltage distribution network power quality monitoring index data compression method based on big data learning |
-
2022
- 2022-03-22 CN CN202210282392.XA patent/CN114663834B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107832799A (en) * | 2017-11-20 | 2018-03-23 | 北京奇虎科技有限公司 | Object identifying method and device, computing device based on camera scene |
| CN111757069A (en) * | 2020-07-10 | 2020-10-09 | 广州博冠智能科技有限公司 | Monitoring anti-theft method and device based on intelligent doorbell |
| CN113962363A (en) * | 2021-10-19 | 2022-01-21 | 国网河南省电力公司新野县供电公司 | Low-voltage distribution network power quality monitoring index data compression method based on big data learning |
Non-Patent Citations (2)
| Title |
|---|
| QIONG HONG，ET AL: "《The Recognition Method of Express Logistics Restricted Goods Based on Deep Convolution Neural Networks》", 《2020 5TH IEEE INTERNATIONAL CONFERENCE ON BIG DATA ANALYTICS》 * |
| 李娇娇: "《基于大数据分析的快递安全监控系统的研究》", 《中国优秀博硕士学位论文全文数据库（硕士） 信息科技辑》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114863734A (en) * | 2022-07-06 | 2022-08-05 | 新乡学院 | Evaluation method of organic polymer material synthesis experiment |
| CN114863734B (en) * | 2022-07-06 | 2022-09-30 | 新乡学院 | Evaluation method for organic polymer material synthesis experiment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114663834B (en) | 2023-04-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10755259B2 (en) | Intelligent video verification of point of sale (POS) transactions | |
| US8903219B2 (en) | Auditing video analytics through essence generation | |
| US10701321B2 (en) | System and method for distributed video analysis | |
| US20130336531A1 (en) | Sequential event detection from video | |
| JP2014529778A (en) | System and method for improving field work by detecting anomalies | |
| KR20070053358A (en) | Target property maps for surveillance systems | |
| CN101208710A (en) | Target detection and tracking from overhead video streams | |
| JP2014531066A (en) | System and method for on-site anomaly recording and notification | |
| CN109743532B (en) | Doorbell control method, electronic equipment, doorbell system and storage medium | |
| US20160092740A1 (en) | Image auditing method and system | |
| CN114663834A (en) | Express storage site monitoring method | |
| US20060066719A1 (en) | Method for finding paths in video | |
| Yedla et al. | Real-time scene change detection with object detection for automated stock verification | |
| US11875657B2 (en) | Proactive loss prevention system | |
| US11216666B2 (en) | Understanding normality of an environment using semantic information from images | |
| CN109120896B (en) | Security video monitoring guard system | |
| CN103379312A (en) | Image monitoring system and method | |
| US20230245512A1 (en) | System and method for monitoring pickup and drop-off of items | |
| CN112560694B (en) | Data analysis method and device, electronic equipment and storage medium | |
| US20240161592A1 (en) | Proactive loss prevention system | |
| WO2024018711A1 (en) | Plant management system and control program | |
| US20230237558A1 (en) | Object recognition systems and methods | |
| CN116307656A (en) | Flow supervision method, device and system for freight security check service | |
| Kaur et al. | Framework for FOGIoT based Smart Video Surveillance System (SVSS) | |
| CN115147775A (en) | Video image processing and identifying system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |