CN111666916A - Kitchen violation identification method based on self-learning technology - Google Patents
Kitchen violation identification method based on self-learning technology Download PDFInfo
- Publication number
- CN111666916A CN111666916A CN202010558604.3A CN202010558604A CN111666916A CN 111666916 A CN111666916 A CN 111666916A CN 202010558604 A CN202010558604 A CN 202010558604A CN 111666916 A CN111666916 A CN 111666916A
- Authority
- CN
- China
- Prior art keywords
- frame
- pictures
- kitchen
- violation
- picture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Multimedia (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a self-learning technology-based kitchen violation identification method, and particularly relates to the field of image processing, wherein the method comprises the following steps: the method comprises the following steps that firstly, a plurality of cameras are arranged in a kitchen to acquire monitoring video data in the kitchen; classifying the video data according to the scene type of the frame pictures, collecting and centralizing the frame pictures of different classifications to form frame aggregation, and grouping highly repeated frames in different frame aggregation; and step three, extracting a single frame picture in the frame aggregation, and recording gamma frames. According to the invention, the frame pictures in the monitoring video data are classified, the frame pictures of corresponding categories are independently extracted, the classified frame pictures of the illegal behaviors are utilized to form the scanning sliding window, the illegal behaviors in the monitoring video are identified, redundant interference information does not exist in the whole identification process, the normal identification of the illegal behaviors is not interfered, the false alarm rate is low, and the illegal behaviors can be accurately identified.
Description
Technical Field
The invention relates to the field of image processing, in particular to a kitchen violation identification method based on a self-learning technology.
Background
Computer vision is a technology which is mature day by day in recent years, and an intelligent video monitoring technology based on the computer vision technology is widely applied to scenes such as restaurants, companies, gymnasiums, construction sites, railway stations and the like.
The general method for intelligent catering video monitoring based on computer vision technology is that firstly, a refined target in a current video frame is detected by adopting a target detection method, then actions and clothes of operators in the current frame are judged based on semantic information of a plurality of kitchens, so that a decision is made to judge whether the operators in the current frame all meet the kitchen standard, pictures with violation of the operators in the video frame are found and pushed to a catering manager, the human capital for finding out the violation of the kitchen operation in the catering industry is saved, and a reliable solution is provided for effective management and supervision of various shops in the catering industry.
However, the current algorithm has certain limitations, and firstly, in a kitchen scene in the catering industry, a visual background is disordered and a large amount of redundant interference information exists, so that certain interference is caused to a target detection algorithm, and the performance improvement of the target detection algorithm is limited. Secondly, a corresponding rule is artificially formulated based on the semantic information of the detected target combined with the kitchen through a target detection algorithm to find out the illegal item in the kitchen patrol standard. However, many inspection items in the kitchen cannot be formulated by definite rules, so that the intelligent algorithm can only find some illegal suspected items, the false alarm rate is high, and the improvement of the working efficiency is limited.
Disclosure of Invention
In order to overcome the defects in the prior art, the embodiment of the invention provides a kitchen violation behavior recognition method based on a self-learning technology, redundant interference information does not exist in the whole recognition process, the violation behavior is not interfered with in normal recognition, the false alarm rate is low, and the violation behavior can be accurately recognized.
In order to achieve the purpose, the invention provides the following technical scheme: a kitchen violation identification method based on self-learning technology comprises the following steps:
the method comprises the following steps that firstly, a plurality of cameras are arranged in a kitchen to acquire monitoring video data in the kitchen;
classifying the video data according to the scene type of the frame pictures, collecting and centralizing the frame pictures of different classifications to form frame aggregation, and grouping highly repeated frames in different frame aggregation;
step three, extracting a single frame picture in the frame aggregation, and recording gamma frames;
identifying frame group pictures in the frame aggregation, recording the completeness of a scene picture as alpha and the definition of the picture as beta, and independently extracting the pictures with the highest alpha value and the highest beta value and respectively recording a frames and b frames;
step five, summarizing the gamma frame, the a frame and the b frame to form an identification frame picture;
step six, inputting an illegal behavior video, manually extracting frame pictures with identifying illegal behaviors in the video by using video editing software, and grouping the frame pictures of different categories respectively according to the categories of the illegal behaviors to form different illegal behavior frame picture training sets;
step seven, sending different violation frame picture training sets into a Convolutional Neural Network (CNN), wherein the CNN analyzes and processes the frame picture, inputs pixel images with corresponding sizes according to the size of the whole training set, and spreads the frame picture training set on the pixel images to form a scanning sliding window;
step eight, extracting the frame-by-frame identification frame pictures in the step five, moving each frame picture according to pixel points, realizing full coverage on a scanning sliding window, and individually marking the overlapped frame pictures as violation behaviors;
and step nine, constructing a deep learning module, marking different violation frame pictures, extracting the frame pictures with the similarity higher than 90% with the overlapped frame pictures in the step eight, marking the pictures as auxiliary frame pictures by using the learning behaviors, and forming a next scanning sliding window together with the marked violation frame pictures.
In a preferred embodiment, in the second step, the classification of the scene categories is identified by human, the scene basis in the scene categories is kitchen, and the basis of the category classification is kitchen behavior based.
In a preferred embodiment, the frame group picture is specifically: the method comprises the following steps of continuously obtaining multiple high-repetition single-type frame pictures, wherein the gamma frame pictures specifically comprise: only one frame exists in the frame picture corresponding to the type of the kitchen behavior.
In a preferred embodiment, the values of both α and β are greater than 99%.
In a preferred embodiment, in the step eight, the overlapped frame picture threshold is 80% and is dynamically adjustable.
The invention has the technical effects and advantages that:
1. the frame pictures in the monitoring video data are classified, the frame pictures of corresponding categories are independently extracted, the classified frame pictures of the illegal behaviors are utilized to form a scanning sliding window, the illegal behaviors in the monitoring video are identified, redundant interference information does not exist in the whole identification process, the normal identification of the illegal behaviors is not interfered, the false alarm rate is low, and the illegal behaviors can be accurately identified;
2. based on a deep learning model, similar violation frame images are captured to construct a scanning sliding window for identifying the next violation behavior, so that the method has a self-updating effect and avoids system redundancy and errors.
Detailed Description
The present invention will be described in further detail with reference to specific embodiments. The embodiments of the present invention have been presented for purposes of illustration and description, and are not intended to be exhaustive or limited to the invention in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiment was chosen and described in order to best explain the principles of the invention and the practical application, and to enable others of ordinary skill in the art to understand the invention for various embodiments with various modifications as are suited to the particular use contemplated.
Example 1
A kitchen violation identification method based on self-learning technology comprises the following steps:
the method comprises the steps that a plurality of cameras are arranged in a kitchen to obtain monitoring video data in the kitchen, different cameras are arranged according to different kitchen scenes, the monitoring video data can be compressed before being transmitted, and the memory occupation is reduced;
classifying video data according to scene categories of frame pictures, collecting and centralizing the frame pictures of different classifications to form frame aggregation, and grouping highly repeated frames in different frame aggregation into a group, further, wherein:
the classification of the scene categories is carried out through manual identification, the scene base in the scene categories is a kitchen, the classification of the categories is based on the kitchen behavior categories, namely, a user can firstly obtain the kitchen behavior categories in the video data according to the video data, the kitchen behavior categories comprise various behaviors such as impurity removing, vegetable washing, cooking, vegetable cutting and the like, then the different behavior categories are classified according to the kitchen as the scene base, specifically, the frame pictures related to the impurity removing behavior can have highly repeated contents due to the fact that the behaviors are impurity removing, the partial frame pictures are integrated into groups to form an impurity removing frame group, and the steps are repeated in sequence to form a vegetable washing frame group, a vegetable cooking frame group, a vegetable cutting frame group and the like;
the frame groups are all continuous multiple high-repetition single-category frame pictures;
extracting a single frame picture in the frame aggregation, and recording a gamma frame, wherein the gamma frame picture specifically comprises the following steps: only one frame exists in the frame picture corresponding to the kitchen behavior category, so that the complexity is low;
identifying frame group pictures in the frame aggregation, recording the integrity of scene pictures as alpha and the definition of the pictures as beta, independently extracting the pictures with the highest alpha value and beta value, and respectively recording a frame and a frame b, wherein the values of alpha and beta are both more than 99 percent, namely the difference between the integrity and the definition of the frame pictures and an original picture is small;
on the basis, because different frame groups consist of a plurality of frames, the content is highly repeated, and the frame tearing and the frame blurring are easy to occur in the frame, the frames in the frame groups are identified and extracted, and two frames with the highest integrity and definition are independently extracted and respectively counted as a frame a and a frame b;
summarizing the gamma frame, the a frame and the b frame to form an identification frame picture, wherein the identification frame picture is a total set of all kitchen behaviors occurring in the monitoring video data;
inputting an illegal behavior video, manually extracting frame pictures with identifying illegal behaviors in the video by using video clipping software, and respectively grouping the frame pictures of different categories according to the categories of the illegal behaviors to form different illegal behavior frame picture training sets;
sending different violation frame picture training sets into a Convolutional Neural Network (CNN), wherein the CNN analyzes and processes the frame picture, inputting a pixel image with a corresponding size according to the size of the whole training set, and spreading the frame picture training set on the pixel image to form a scanning sliding window, namely the content of the scanning sliding window is the whole violation picture formed by kitchen violations;
extracting identification frame pictures frame by frame, moving each frame picture according to pixel points, sequentially passing through the whole scanning sliding window, realizing full coverage on the scanning sliding window, independently marking the overlapped frame pictures, wherein the critical value of the overlapped frame pictures is 80%, and when the critical value exceeds 80%, recording that the behavior in the frame picture is illegal, wherein redundant interference information does not exist in the whole identification process, the normal identification of the illegal is not interfered, the false alarm rate is low, and the illegal can be accurately identified;
example 2
On the basis of the embodiment, a deep learning module is constructed, different violation frame pictures are marked, the frame pictures with the critical value higher than 90% of the overlapped frame pictures are extracted, the learning behavior is utilized to mark the pictures as auxiliary frame pictures, and the auxiliary frame pictures and the marked violation frame pictures form a next scanning sliding window together;
namely, the scanning sliding window of each illegal behavior contrast is updated and supplemented, so that the false alarm rate is lower and lower.
It is to be understood that the described embodiments are merely a few embodiments of the invention, and not all embodiments. All other embodiments, which can be derived by one of ordinary skill in the art and related arts based on the embodiments of the present invention without any creative effort, shall fall within the protection scope of the present invention. Structures, devices, and methods of operation not specifically described or illustrated herein are generally practiced in the art without specific recitation or limitation.
Claims (5)
1. A kitchen violation identification method based on self-learning technology is characterized by comprising the following steps:
the method comprises the following steps that firstly, a plurality of cameras are arranged in a kitchen to acquire monitoring video data in the kitchen;
classifying the video data according to the scene type of the frame pictures, collecting and centralizing the frame pictures of different classifications to form frame aggregation, and grouping highly repeated frames in different frame aggregation;
step three, extracting a single frame picture in the frame aggregation, and recording gamma frames;
identifying frame group pictures in the frame aggregation, recording the completeness of a scene picture as alpha and the definition of the picture as beta, and independently extracting the pictures with the highest alpha value and the highest beta value and respectively recording a frames and b frames;
step five, summarizing the gamma frame, the a frame and the b frame to form an identification frame picture;
step six, inputting an illegal behavior video, manually extracting frame pictures with identifying illegal behaviors in the video by using video editing software, and grouping the frame pictures of different categories respectively according to the categories of the illegal behaviors to form different illegal behavior frame picture training sets;
step seven, sending different violation frame picture training sets into a Convolutional Neural Network (CNN), wherein the CNN analyzes and processes the frame picture, inputs pixel images with corresponding sizes according to the size of the whole training set, and spreads the frame picture training set on the pixel images to form a scanning sliding window;
step eight, extracting the frame-by-frame identification frame pictures in the step five, moving each frame picture according to pixel points, realizing full coverage on a scanning sliding window, and individually marking the overlapped frame pictures as violation behaviors;
and step nine, constructing a deep learning module, marking different violation frame pictures, extracting the frame pictures with the similarity higher than 90% with the overlapped frame pictures in the step eight, marking the pictures as auxiliary frame pictures by using the learning behaviors, and forming a next scanning sliding window together with the marked violation frame pictures.
2. The self-learning technology-based kitchen violation identification method as recited in claim 1, wherein: in the second step, the classification of the scene categories is manually identified, the scene basis in the scene categories is kitchen, and the basis of the category classification is kitchen behavior-based.
3. The self-learning technology-based kitchen violation identification method according to claim 2, wherein the frame group pictures specifically comprise: the method comprises the following steps of continuously obtaining multiple high-repetition single-type frame pictures, wherein the gamma frame pictures specifically comprise: only one frame exists in the frame picture corresponding to the type of the kitchen behavior.
4. The self-learning technology-based kitchen violation identification method as recited in claim 1, wherein: both the values of alpha and beta are greater than 99%.
5. The self-learning technology-based kitchen violation identification method as recited in claim 1, wherein: in the eighth step, the threshold value of the overlapped frames is 80% and is dynamically adjustable.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010558604.3A CN111666916B (en) | 2020-06-18 | 2020-06-18 | Kitchen violation identification method based on self-learning technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010558604.3A CN111666916B (en) | 2020-06-18 | 2020-06-18 | Kitchen violation identification method based on self-learning technology |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111666916A true CN111666916A (en) | 2020-09-15 |
| CN111666916B CN111666916B (en) | 2023-03-28 |
Family
ID=72388403
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010558604.3A Active CN111666916B (en) | 2020-06-18 | 2020-06-18 | Kitchen violation identification method based on self-learning technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111666916B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112949619A (en) * | 2021-05-17 | 2021-06-11 | 北京梦天门科技股份有限公司 | Guest room sanitation monitoring method, electronic equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080193010A1 (en) * | 2007-02-08 | 2008-08-14 | John Eric Eaton | Behavioral recognition system |
| US20150222562A1 (en) * | 2014-01-31 | 2015-08-06 | Aruba Networks, Inc. | System and method for efficient frame aggregation |
| CN110135400A (en) * | 2019-05-30 | 2019-08-16 | 杭州视在科技有限公司 | A kind of food and drink scene violation recognition methods based on metric learning |
-
2020
- 2020-06-18 CN CN202010558604.3A patent/CN111666916B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080193010A1 (en) * | 2007-02-08 | 2008-08-14 | John Eric Eaton | Behavioral recognition system |
| US20150222562A1 (en) * | 2014-01-31 | 2015-08-06 | Aruba Networks, Inc. | System and method for efficient frame aggregation |
| CN110135400A (en) * | 2019-05-30 | 2019-08-16 | 杭州视在科技有限公司 | A kind of food and drink scene violation recognition methods based on metric learning |
Non-Patent Citations (2)
| Title |
|---|
| 张文沛: "油气生产过程安全环保违规行为智能识别技术应用", 《中国安全生产》 * |
| 杨茜等: "多粒度运动分割的时频分析算法", 《计算机辅助设计与图形学学报》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112949619A (en) * | 2021-05-17 | 2021-06-11 | 北京梦天门科技股份有限公司 | Guest room sanitation monitoring method, electronic equipment and storage medium |
| CN112949619B (en) * | 2021-05-17 | 2022-02-11 | 北京梦天门科技股份有限公司 | Guest room sanitation monitoring method, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111666916B (en) | 2023-03-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107222780B (en) | Method for comprehensive state perception and real-time content supervision of live broadcast platform | |
| CN111507283B (en) | Student behavior identification method and system based on classroom scene | |
| Zhang et al. | Deep convolutional neural networks for forest fire detection | |
| US9251425B2 (en) | Object retrieval in video data using complementary detectors | |
| CN101711393A (en) | System and method based on the fire detection of video | |
| CN108509893A (en) | Video display methods of marking, storage medium and intelligent terminal based on micro- Expression Recognition | |
| CN108257347B (en) | Flame image sequence classification method and device by using convolutional neural network | |
| CN111222478A (en) | Construction site safety protection detection method and system | |
| CN110880175A (en) | Welding spot defect detection method, system and equipment | |
| CN110096945B (en) | Indoor monitoring video key frame real-time extraction method based on machine learning | |
| CN110619308A (en) | Aisle sundry detection method, device, system and equipment | |
| CN114648714A (en) | YOLO-based workshop normative behavior monitoring method | |
| CN110958467A (en) | Video quality prediction method and device and electronic equipment | |
| CN110348329B (en) | Pedestrian detection method based on video sequence interframe information | |
| CN111666916B (en) | Kitchen violation identification method based on self-learning technology | |
| CN114885119A (en) | Intelligent monitoring alarm system and method based on computer vision | |
| CN112749747A (en) | Garbage classification quality evaluation method and system | |
| CN105095837A (en) | Station logo recognition method and system | |
| CN108765454A (en) | A kind of smog detection method, device and device end based on video | |
| CN111445442A (en) | Crowd counting method and device based on neural network, server and storage medium | |
| Li et al. | Crowd density estimation: An improved approach | |
| CN114913438A (en) | Yolov5 garden abnormal target identification method based on anchor frame optimal clustering | |
| CN112270361B (en) | Face data processing method, system, storage medium and equipment | |
| CN115049875A (en) | Detection method for wearing insulating gloves in transformer substation based on deep learning | |
| CN114677638A (en) | Detection method based on deep learning and abnormal clustering of clustered people |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |