CN111353436A

CN111353436A - Super store operation analysis method and device based on image deep learning algorithm

Info

Publication number: CN111353436A
Application number: CN202010130149.7A
Authority: CN
Inventors: 李国煌; 陈积银; 张龙
Original assignee: Ropt Technology Group Co ltd
Current assignee: Ropt Technology Group Co ltd
Priority date: 2020-02-28
Filing date: 2020-02-28
Publication date: 2020-06-30

Abstract

The invention discloses a super store operation analysis method and a device based on an image depth learning algorithm, which are characterized in that video data shot by a camera provided with at least one preset position is obtained, wherein the preset position comprises a position of a single angle monitored by the camera above a storefront; extracting images in the video data, detecting the images through an image depth learning algorithm, identifying the operation behaviors of the super stores and giving violation alarms; and carrying out merging statistics on the violation alarm conditions detected by the at least one preset bit to obtain the storefront super-portal operation information. The video data obtained by different preset positions are automatically detected through the image deep learning algorithm, the city super store operation behavior is obtained through analysis, and as the super store operation is a continuous behavior, the illegal behaviors of the same type of the same equipment can be continuously early warned through the uninterrupted operation and analysis of the image deep learning algorithm, so that law enforcement personnel can timely handle the illegal behaviors, and the urban management efficiency is improved.

Description

Super store operation analysis method and device based on image deep learning algorithm

Technical Field

The invention relates to the field of video image analysis, in particular to a method and a device for super store operation analysis based on an image deep learning algorithm.

Background

The super store operation refers to the operation of an operator by occupying a public place outside or near the storefront of the operation place, and is a common non-standard operation phenomenon in city management. In the process of managing the chaos, manual management modes such as patrol, investigation, supervision and the like are generally adopted, so that a lot of manpower and material resources are consumed, and the chaos of the super store operation can occur when a city manager is absent, so that the difficulty in managing the super store operation is high, the effect is poor, and the cost is very high.

At present, with the wide distribution of monitoring probes in cities, the super store operation phenomenon is also in a monitoring range, but the monitoring data volume is huge, and a large amount of time and labor cost are needed for manual data processing. With the rapid development of artificial intelligence technology, many new technologies and applications are brought, such as deep learning algorithms, target detection algorithms and the like, and new ideas and methods are provided for the management and treatment of the super stores.

In the prior art, a background modeling is performed by a dynamic object detection method to detect the store-exceeding store operation phenomenon, but the dynamic change of store-exceeding operation is weak, and the difficulty of data processing is obviously increased by adopting the dynamic object detection method. The video view angle is fixed, namely when the position of the storefront shot by each frame is unchanged, determining the out-of-store object by using a dynamic object detection method, and then classifying the out-of-store object by using an out-of-store object classification model; when the video visual angle is not fixed, the object detection model is adopted to determine the object classification and the object position in the frame image, and then the out-of-store object is determined according to the object position. Obviously, the method for detecting the super store operation is very complex and has large workload.

Disclosure of Invention

The problems of high treatment difficulty, complex detection means, high workload and the like of the super store operation phenomenon are solved. An object of the embodiments of the present application is to provide a method and an apparatus for conducting an operation analysis of a supermarket based on an image deep learning algorithm, so as to solve the technical problems mentioned in the background section above.

In a first aspect, an embodiment of the present application provides a method for conducting business analysis of a supermarket based on an image deep learning algorithm, including the following steps:

s1: acquiring video data captured by a camera provided with at least one preset bit, wherein the preset bit comprises a position of a single angle monitored by the camera above a storefront;

s2: extracting images in the video data, detecting the images through an image deep learning algorithm, identifying the operation behaviors of the super stores and giving violation alarms;

s3: and carrying out merging statistics on the violation alarm conditions obtained by detecting at least one preset position to obtain the operation information of the super store.

The super store operation behavior is identified on at least one preset position, and real-time monitoring and alarming are carried out, so that the super store operation behavior is effectively supervised.

In some embodiments, step S1 includes the steps of:

s11: setting at least one preset position for a camera above a storefront;

s12: and enabling the camera to perform polling tracking monitoring on at least one preset position, and acquiring video data shot by the camera on the at least one preset position in real time.

Polling tracking is carried out by setting at least one preset position, so that the obtained video data are more comprehensive, and the super store operation behavior identification is more accurate.

In some embodiments, step S2 includes the steps of:

s21: training images in the video data through a fast RCNN network based on VGG16 to obtain the positions and the types of objects in the images;

s22: screening out objects and categories outside the shop according to the positions of the objects;

s23: and when the object outside the store belongs to the store operation object, judging that the operation behavior of the super store is excessive and carrying out violation alarm.

The image deep learning algorithm is obtained by training the fast RCNN network of the VGG16, so that the image processing is more accurate and simpler, and objects at the doorway of the storefront can be effectively identified.

In some embodiments, the fast RCNN network based on VGG16 includes Conv Layers, RPN networks, roiploling, and fully-connected Layers, through which feature maps of images are extracted for input to the RPN networks and fully-connected Layers. The fast RCNN network based on VGG16 belongs to one of CNN network target detection, and has the advantages of simple overall framework, high calculation speed and good processing effect.

In some embodiments, Conv Layers comprise 13 Conv Layers, 13 relu Layers, and 4 pooling Layers, wherein in Conv Layers kernel _ size is 3, pad is 1, stride is 1, and in pooling Layers kernel _ size is 2, stride is 2. And processing the image through the conv layer, the relu layer and the pooling layer to obtain a feature map so as to be used for the subsequent RPN layer and the full connection layer.

In some embodiments, the RPN network receives the feature map output by the Conv Layers, performs a convolution 3 × 3 first to generate a stack of anchor boxes, and then performs two full convolutions, where one full convolution determines whether the anchors belong to the object by reshape and softmax, and the other full convolution corrects the anchor boxes by bounding box regression to form a proposal. The RPN network is used as a second classification, wherein one is used for judging whether anchors belong to the foreground or the background, namely judging whether the anchors belong to the object or not, and the other forms more accurate proposal.

In some embodiments, the fixed size propusal feature map is obtained by two quantization processes of Roi Pooling, using the feature maps obtained from the propusal and Conv Layers parts generated by the RPN network. The subsequent entry of the fixed-size proposal feature map can utilize full-connectivity operations for object recognition and localization.

In some embodiments, the full link layer performs full link operation on the propofol feature map formed by Roi posing in a fixed size, performs bounding box regression operation on the propofol again using L1 Loss to obtain the position of the object, and performs object classification on the propofol using Softmax to obtain the object class in the image. After the positions of the objects and the corresponding object types are obtained, the operation behavior of the super store in the storefront can be judged and early warned.

In some embodiments, step S3 includes the steps of:

s31: judging whether the violation alarms are the same camera or the same type of alarms, and if so, accumulating the alarm times of the same type of camera;

s32: calculating the time difference between the violation alarm of the same type and the first violation alarm of the camera to obtain the alarm duration;

s33: and merging the place showing the violation alarm, the first violation alarm time, the type and the alarm duration.

The video images shot by the same camera on different preset positions are analyzed to identify the operation behaviors of the super stores, so that the information such as the number of violations of the operation behaviors of the super stores, the time and the like can be obtained through statistical analysis, and subsequent law enforcement officers can manage the city conveniently.

In a second aspect, the present application further provides a supermarket operation analysis apparatus based on an image deep learning algorithm, including:

a data acquisition module configured to acquire video data captured by a camera that sets at least one preset bit, wherein the preset bit includes a position of a single angle monitored by the camera above the storefront;

the image detection module is configured to extract images in the video data, detect the images through an image deep learning algorithm, identify the operation behaviors of the super stores and give violation of regulations;

and the information summarizing module is configured to carry out merging statistics on the violation alarm conditions detected by the at least one preset position to obtain the operation information of the super store.

In a third aspect, the present application also proposes a computer storage medium having a computer program stored thereon, which when executed by a computer performs the steps mentioned in the first aspect.

The application provides a super store operation analysis method and device based on an image deep learning algorithm, which comprises the steps of obtaining video data shot by a camera provided with at least one preset position, wherein the preset position comprises a position of a single angle monitored by the camera above a storefront; extracting images in the video data, detecting the images through an image deep learning algorithm, identifying the operation behaviors of the super stores and giving violation alarms; and carrying out merging statistics on the violation alarm conditions detected by the at least one preset bit to obtain the storefront super-portal operation information. Carry out automated inspection through the video data that the deep learning algorithm of image obtained on different presetting bits, the analysis obtains city super store operation behavior, because super store operation is a continuation behavior, through the incessant operational analysis of the deep learning algorithm of image, to same equipment, the illegal action of same type sends the early warning continuously, lets city law enforcement person handle at the very first time, promotes urbanization managerial efficiency.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is an exemplary device architecture diagram in which one embodiment of the present application may be applied;

FIG. 2 is a schematic flow chart of a supermarket operation analysis method based on an image deep learning algorithm according to an embodiment of the present invention;

FIG. 3 is a flowchart illustrating step S1 of the method for analyzing the operation of the supermarket based on the image deep learning algorithm according to the embodiment of the present invention;

FIG. 4 is a flowchart illustrating step S2 of the method for analyzing the operation of the supermarket based on the image deep learning algorithm according to the embodiment of the present invention;

FIG. 5 is a flowchart illustrating step S3 of the method for analyzing the operation of the supermarket based on the image deep learning algorithm according to the embodiment of the present invention;

FIG. 6 is a schematic diagram of a supermarket operation analysis device based on an image deep learning algorithm according to an embodiment of the invention;

fig. 7 is a schematic structural diagram of a computer device suitable for implementing an electronic apparatus according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention clearer, the present invention will be described in further detail with reference to the accompanying drawings, and it is apparent that the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Fig. 1 shows an exemplary device architecture 100 of a supermarket operation analysis method based on an image deep learning algorithm or a supermarket operation analysis device based on an image deep learning algorithm, to which an embodiment of the present application can be applied.

As shown in fig. 1, the apparatus architecture 100 may include

terminal devices

101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the

terminal devices

101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.

The user may use the

terminal devices

101, 102, 103 to interact with the server 105 via the network 104 to receive or send messages or the like. Various applications, such as data processing type applications, file processing type applications, etc., may be installed on the

terminal apparatuses

101, 102, 103.

The

terminal apparatuses

101, 102, and 103 may be hardware or software. When the

terminal devices

101, 102, 103 are hardware, they may be various electronic devices including, but not limited to, smart phones, tablet computers, laptop portable computers, desktop computers, and the like. When the

terminal apparatuses

101, 102, 103 are software, they can be installed in the electronic apparatuses listed above. It may be implemented as multiple pieces of software or software modules (e.g., software or software modules used to provide distributed services) or as a single piece of software or software module. And is not particularly limited herein.

The server 105 may be a server that provides various services, such as a background data processing server that processes files or data uploaded by the

terminal devices

101, 102, 103. The background data processing server can process the acquired file or data to generate a processing result.

The portal business analysis method based on the image deep learning algorithm provided in the embodiment of the present application may be executed by the server 105 or the

terminal devices

101, 102, and 103, and accordingly, the portal business analysis device based on the image deep learning algorithm may be provided in the server 105 or the

terminal devices

101, 102, and 103.

It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation. In the case where the processed data does not need to be acquired from a remote location, the above device architecture may not include a network, but only a server or a terminal device.

Fig. 2 shows a supermarket operation analysis method based on an image deep learning algorithm, which is disclosed by an embodiment of the application and comprises the following steps:

The invention arranges the high-altitude camera at the front end, and erects the high-altitude camera at the urban height-control point so as to observe the urban violation behaviors from high altitude. The camera is divided into a gun camera and a ball camera, the gun camera is used for recording a large picture integrally, and the ball camera can track and monitor a local detail part. Firstly, setting a selected city violation monitoring area as a preset position, namely setting the preset position to include the position of a single angle monitored by a camera above a storefront, recording and monitoring information on each preset position through a ball machine, polling, tracking and monitoring from each preset position at regular time, and uploading a video recorded by the preset position to a background. The super store operation behavior is identified and detected on at least one preset position through an image deep learning algorithm, and real-time monitoring and alarming are carried out, so that the super store operation behavior is effectively supervised and early warned.

In a specific embodiment, as shown in fig. 3, step S1 includes the following steps:

s11: setting at least one preset position for a camera above a storefront;

Real-time monitoring and polling tracking are carried out by setting at least one preset position on the ball machine, real-time videos of each preset position are obtained, and therefore the real-time videos are analyzed through an image deep learning algorithm to obtain the super store operation behavior, so that the obtained video data are more comprehensive, and the super store operation behavior is identified more accurately. If under the control of rifle bolt, only a preset position, under the control of ball machine, can obtain a plurality of different preset positions, the video data's that different preset positions obtained angle is different, consequently can obtain more comprehensive and detailed real-time data and carry out the analysis to super store business behavior, can contrast the data in lasting a period moreover and judge super store business behavior.

In a specific embodiment, as shown in fig. 4, step S2 includes the following steps:

The image deep learning algorithm is obtained by fast RCNN network training of VGG16, the position and the type of an object are analyzed and identified through the image deep learning algorithm according to video data obtained by shooting a ball machine at least one preset position, the object and the type outside a shop are judged according to the position of the object, and finally the super-shop operation behavior of the shop is identified and detected. Therefore, the method for detecting the excessive store operation behavior is more accurate and simple, the object at the storefront doorway can be effectively identified, the excessive store operation behavior is judged, and the illegal store operation behavior can be monitored in real time.

In a specific embodiment, the fast RCNN network based on VGG16 includes Conv Layers, RPN networks, Roi Pooling and fully connected Layers, and feature maps of images are extracted by Conv Layers, and the feature maps are used for input of the RPN networks and the fully connected Layers. The fast RCNN network based on the VGG16 belongs to one of CNN network target detection, is obtained through fast RCNN network training of the VGG16, and is simple in overall framework, high in calculation speed and good in processing effect.

In a specific embodiment, Conv Layers comprise 13 Conv Layers, 13 relu Layers and 4 pooling Layers, wherein in Conv Layers kernel _ size is 3, pad is 1, stride is 1, and in pooling Layers kernel _ size is 2, stride is 2. And processing the image through the conv layer, the relu layer and the pooling layer to obtain a feature map so as to be used for the subsequent RPN layer and the full connection layer.

In a specific embodiment, an RPN network (Region pro laboratory network) receives a feature map output by Conv Layers, performs a convolution of 3 × 3 to generate a stack of anchor boxes, and then performs two full convolutions (kernel _ size ═ 1, pad ═ 0, and stride ═ 1), where one full convolution determines whether anchors belong to an object through reshape and softmax, and the other full convolution corrects the anchor boxes through bounding box regression to form a propassal. One of the full convolution is used for judging whether anchors belong to the foreground or the background, namely judging whether objects exist or not, and the other full convolution forms more accurate proposal. More precisely here with respect to the further boxregression of the following fully connected layer.

In a specific embodiment, a proposal feature map with a fixed size is obtained through two quantization processes of Roi Pooling by using feature maps obtained from proposal and Conv Layers generated by an RPN network. The subsequent entry of the fixed-size proposal feature map can utilize full-connectivity operations for object recognition and localization.

In a specific embodiment, the full link layer performs full link operation on a proposal feature map formed by Roi posing with a fixed size, performs bounding box regression operation on the proposal again by using L1 Loss to obtain the position of the object, and performs object classification on the proposal by using Softmax to obtain the object class in the image. The part belongs to classification operation, the position of the object is the accurate position of the object, and the judgment and early warning of the store-crossing operation behavior of the storefront can be carried out after the position of the object and the corresponding object type are obtained. If the object outside the store belongs to the object class in the store operation range and the action exists in the continuous time, the behavior of the store is the super store operation behavior, and if the object outside the store does not belong to the object class in the store operation range or the action does not exist in the continuous time, the behavior of the store is not the super store operation behavior. The super store operation is a continuous behavior, the image deep learning algorithm is subjected to uninterrupted operation and analysis, and the illegal behaviors of the same type can continuously give out early warning to the same equipment.

In a specific embodiment, as shown in fig. 5, step S3 includes the following steps:

And judging whether the violation behavior alarm comes from the same equipment and belongs to the same type of alarm, if so, accumulating the alarm times of the same type on the equipment on the current day, and calculating the time difference between the current alarm and the first alarm, so that the position and time of the violation behavior alarm of a certain type can be obtained, and when the first alarm is, the alarm lasts for a certain time and other information. Therefore, the information such as the number of violations of the super store operation behavior, the time and the like can be obtained through statistical analysis, and subsequent law enforcement officers can manage the city conveniently.

Corresponding to the above-mentioned super store operation analysis method based on the image deep learning algorithm, an embodiment of the present application further provides a super store operation analysis apparatus based on the image deep learning algorithm, as shown in fig. 6, including:

a data acquisition module 1 configured to acquire video data captured by a camera provided with at least one preset bit including a position of a single angle monitored by the camera above a storefront;

the image detection module 2 is configured to extract images in the video data, detect the images through an image deep learning algorithm, identify the operation behaviors of the super stores and give violation of regulations;

and the information summarizing module 3 is configured to carry out merging statistics on the violation alarm conditions detected by the at least one preset position to obtain the super store operation information.

In a specific embodiment, the data acquisition module 1 includes:

the preset position setting module is configured to set at least one preset position for a camera above the storefront;

and the video shooting module is configured to enable the camera to conduct polling tracking monitoring on the at least one preset position, and obtain video data shot by the camera on the at least one preset position in real time.

In a particular embodiment, the image detection module 2 comprises:

the deep learning module is configured to train images in the video data through a fast RCNN network based on VGG16, and obtain the positions and the types of objects in the images;

the object screening module is configured to screen out objects and categories outside the shop according to the positions of the objects;

and the violation alarm module is configured to judge that the business behavior of the super store is excessive and carry out violation alarm when the object outside the store belongs to the store business object.

In a specific embodiment, the information summarizing module 3 includes:

the alarm frequency accumulation module is configured to judge whether the violation alarm is the same camera or the same type of alarm, and if so, accumulate the alarm frequency of the same type of camera;

the duration calculation module is configured to calculate the time difference between the violation alarm and the first violation alarm of the same type of the camera to obtain the alarm duration;

and the alarm display module is configured to combine the place displaying the violation alarm, the first violation alarm time, the type and the alarm duration.

The application provides a super store operation analysis method and device based on an image deep learning algorithm, which comprises the steps of obtaining video data shot by a camera provided with at least one preset position, wherein the preset position comprises a position of a single angle monitored by the camera above a storefront; extracting images in the video data, detecting the images through an image deep learning algorithm, identifying the operation behaviors of the super stores and giving violation alarms; and carrying out merging statistics on the violation alarm conditions detected by the at least one preset bit to obtain the storefront super-portal operation information. Carry out automated inspection through the video data that the deep learning algorithm of image obtained on different presetting bits, the analysis obtains city super store operation action, because super store operation is a continuation action, consequently through the incessant operational analysis of the deep learning algorithm of image, can be to same equipment, the illegal action of same type continuously sends the early warning, lets city law enforcement person handle at the very first time, promotes urbanization managerial efficiency.

Referring now to fig. 7, a schematic diagram of a computer device 700 suitable for use in implementing an electronic device (e.g., the server or terminal device shown in fig. 1) according to an embodiment of the present application is shown. The electronic device shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.

As shown in fig. 7, the computer apparatus 700 includes a Central Processing Unit (CPU)701 and a Graphics Processing Unit (GPU)702, which can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)703 or a program loaded from a storage section 709 into a Random Access Memory (RAM) 704. In the RAM704, various programs and data necessary for the operation of the apparatus 700 are also stored. The CPU 701, GPU702, ROM 703, and RAM704 are connected to each other via a bus 705. An input/output (I/O) interface 706 is also connected to bus 705.

The following components are connected to the I/O interface 706: an input section 707 including a keyboard, a mouse, and the like; an output section 708 including a display such as a Liquid Crystal Display (LCD) and a speaker; a storage section 709 including a hard disk and the like; and a communication section 710 including a network interface card such as a LAN card, a modem, or the like. The communication section 710 performs communication processing via a network such as the internet. The driver 711 may also be connected to the I/O interface 706 as needed. A removable medium 712 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 711 as necessary, so that a computer program read out therefrom is mounted into the storage section 709 as necessary.

In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such embodiments, the computer program may be downloaded and installed from a network via the communication section 710, and/or installed from the removable media 712. The computer program performs the above-described functions defined in the method of the present application when executed by a Central Processing Unit (CPU)701 and a Graphics Processing Unit (GPU) 702.

It should be noted that the computer readable medium described herein can be a computer readable signal medium or a computer readable medium or any combination of the two. The computer readable medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor device, apparatus, or any combination of the foregoing. More specific examples of the computer readable medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution apparatus, device, or apparatus. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution apparatus, device, or apparatus. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations for aspects of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based devices that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The modules described in the embodiments of the present application may be implemented by software or hardware. The modules described may also be provided in a processor.

As another aspect, the present application also provides a computer-readable medium, which may be contained in the electronic device described in the above embodiments; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: acquiring video data captured by a camera provided with at least one preset bit, wherein the preset bit comprises a position of a single angle monitored by the camera above a storefront; extracting images in the video data, detecting the images through an image deep learning algorithm, identifying the operation behaviors of the super stores and giving violation alarms; and carrying out merging statistics on the violation alarm conditions detected by the at least one preset position to obtain the super store operation information.

The above description is only a preferred embodiment of the application and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention herein disclosed is not limited to the particular combination of features described above, but also encompasses other arrangements formed by any combination of the above features or their equivalents without departing from the spirit of the invention. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.

Claims

1. A super store operation analysis method based on an image deep learning algorithm is characterized by comprising the following steps:

s1: acquiring video data captured by a camera having at least one preset bit set therein, wherein the preset bit comprises a position of a single angle monitored by the camera above a storefront;

s3: and carrying out merging statistics on the violation alarm conditions detected by at least one preset position to obtain the super store operation information.

2. The image deep learning algorithm-based supermarket operation analysis method according to claim 1, wherein the step S1 comprises the steps of:

s11: by setting at least one preset bit for the camera above the storefront;

s12: and enabling the camera to perform polling tracking monitoring on at least one preset position, and acquiring video data shot by the camera on at least one preset position in real time.

3. The image deep learning algorithm-based supermarket operation analysis method according to claim 1, wherein the step S2 comprises the steps of:

s21: training the images in the video data through a fast RCNN network based on VGG16 to obtain the positions and the types of objects in the images;

s23: and when the object outside the shop belongs to the shop operation object, judging that the shop operation behavior is beyond the shop operation behavior and carrying out violation alarm.

4. The ultra store business analysis method based on image deep learning algorithm as claimed in claim 3, wherein the fast RCNN network based on VGG16 comprises Conv Layers, RPN networks, Roi Pooling and fully connected Layers, and the feature map of the image is extracted through the Conv Layers, and the feature map is used for the input of the RPN networks and the fully connected Layers.

5. The image deep learning algorithm-based supermarket operation analysis method according to claim 4, wherein the Conv Layers comprise 13 Conv Layers, 13 relu Layers and 4 pooling Layers, wherein kernel _ size ═ 3, pad ═ 1 and stride ═ 1 in the Conv Layers, and kernel _ size ═ 2 and stride ═ 2 in the pooling Layers.

6. The image deep learning algorithm-based supermarket operation analysis method according to claim 4, wherein the RPN network receives the feature map output by the Conv Layers, firstly performs one convolution by 3 × 3 to generate a stack of anchors, and then performs two full convolutions, wherein one full convolution judges whether the anchors belong to the object through reshape and softmax, and the other full convolution corrects the anchors through bounding box regression to form a proxy.

7. The image deep learning algorithm-based supermarket operation analysis method according to claim 4, wherein a prosal feature map with a fixed size is obtained through a twice quantization process of RoiPooling by using a feature map obtained by a prosal generated by the RPN and the Conv Layers.

8. The image deep learning algorithm-based supermarket operation analysis method according to claim 4, wherein the full link layer performs full link operation on the pro sale feature map formed by the Roi Pobing in a fixed size, performs bounding box regression operation on the pro sale again using L1 Loss to obtain the position of the object, and performs object classification on the pro sale using Softmax to obtain the object class in the image.

9. The image deep learning algorithm-based supermarket operation analysis method according to claim 1, wherein the step S3 comprises the steps of:

s31: judging whether the violation alarms are alarms of the same camera or the same type, and if so, accumulating the alarm times of the same type of the camera;

s32: calculating the time difference between the violation alarm of the same type of the camera and the first violation alarm to obtain the alarm duration;

s33: and merging the place for displaying the violation alarm, the first violation alarm time, the type and the alarm duration.

10. An image deep learning algorithm-based supermarket operation analysis device is characterized by comprising:

a data acquisition module configured to acquire video data captured by a camera that sets at least one preset bit, wherein the preset bit comprises a position of a single angle monitored by the camera above a storefront;

and the information summarizing module is configured to carry out merging statistics on the violation alarm conditions detected by the at least one preset position to obtain the super store operation information.

11. A computer storage medium having a computer program stored thereon, wherein the computer program, when executed by a computer, implements the steps of the method of any of claims 1 to 9.