CN111860307A - Intelligent kitchen violation judgment method based on video behavior recognition - Google Patents

Intelligent kitchen violation judgment method based on video behavior recognition Download PDF

Info

Publication number
CN111860307A
CN111860307A CN202010695617.5A CN202010695617A CN111860307A CN 111860307 A CN111860307 A CN 111860307A CN 202010695617 A CN202010695617 A CN 202010695617A CN 111860307 A CN111860307 A CN 111860307A
Authority
CN
China
Prior art keywords
kitchen
behavior
video
scene
specifications
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010695617.5A
Other languages
Chinese (zh)
Inventor
周凯凯
高善恒
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou Enterprise Intelligence Information Technology Co ltd
Original Assignee
Suzhou Enterprise Intelligence Information Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou Enterprise Intelligence Information Technology Co ltd filed Critical Suzhou Enterprise Intelligence Information Technology Co ltd
Priority to CN202010695617.5A priority Critical patent/CN111860307A/en
Publication of CN111860307A publication Critical patent/CN111860307A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/50Context or environment of the image
    • G06V20/52Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/21Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/40Scenes; Scene-specific elements in video content
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Data Mining & Analysis (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Human Computer Interaction (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a kitchen violation intelligent judgment method based on video behavior recognition. The method relates to the fields of informatization, video analysis and image recognition, in particular to a method for detecting object behavior specifications based on a kitchen operation scene environment. The behavior specification intelligent judgment and detection method only aims at relevant state scenes of kitchen operators and kitchen equipment, under the scene, the operators need to operate according to wearing states and behavior specifications specified by industry and specific kitchen, and various articles under the scene, such as operating tables, garbage cans, cooking benches and other equipment, need to show specific states under certain conditions. The method comprises the steps of carrying out intelligent training on people and objects in a scene according to two layers of industry specifications and enterprise specifications in a kitchen scene, combining a face recognition technology and a video behavior analysis technology, depending on correct and wrong behavior sample videos in the scene, intelligently judging whether the current states and behaviors of the people and the objects meet the specifications or not, and combining the states and behaviors which do not meet the specifications with illegal pictures for outputting.

Description

Intelligent kitchen violation judgment method based on video behavior recognition
Technical Field
The invention relates to the technical field of computers, in particular to an algorithm based on image recognition.
Background
In the past, the kitchen regulations of dining halls such as restaurants, public institutions, schools and the like have been the focus of attention of society and enterprises. The behavior specification of the kitchen also goes through three stages, namely an unscheduled patrol stage of a manager, a crowd supervision stage of a transparent glass isolation kitchen and a video monitoring covering stage, and in the three stages, the core concept of the method is 'manpower supervision', the method has greater randomness and contingency, and natural kitchen workers can have greater leave-luck psychology.
Aiming at the current situation, the behavior monitoring of the kitchen basically realizes informatization and networking, is limited to network transmission limitation, and can realize real-time monitoring and retrospective based on a monitoring camera, a network storage device and a display device in a local area network under most conditions.
At present, the method for integrating the standard monitoring of kitchen behaviors is based on video pictures in the video monitoring system, hires human to watch videos, depends on human judgment to find out abnormal conditions in the videos, and issues the abnormal conditions through IM communication tools. The whole process has the following disadvantages:
1. The information amount is too large, and the manpower can be relaxed
2. The standard judges the experience of the dependent person, and the standard is difficult to unify
3. Lack of smooth information issuing mode and no subsequent optimization capability
Disclosure of Invention
Aiming at the defects, the invention provides an intelligent, fair, efficient and extensible behavior specification intelligent monitoring method for a kitchen operation place.
The technical scheme for solving the technical problems of the invention is as follows: an intelligent kitchen violation judging method based on video behavior recognition. The method comprises continuous training of standard behaviors of the kitchen, and related samples need to be continuously collected and marked in the training process, so that a better effect is achieved. In this process, the understanding of the catering industry chef regulations, chef conventions, and idiosyncratic behaviors are included, and in the implementation process, the materials need to be continuously accumulated, covering both correct examples and incorrect examples. The materials cover video materials and picture materials, and for the video materials, the video materials need to be imported into a GPU server, the materials are cut into pictures according to proper rules, and subsequent processing is carried out based on the pictures.
The picture processing described in the present invention includes behavior tagging and training of pictures. In this process, specific behaviors, which performances or states of the specific behaviors are in compliance and which performances and states are in violation, need to be set in advance. For example, in the kitchen, the chef enters, whether the behavior of wearing the mask is in the standard or not is judged, the next behavior is that the mask covers the mouth and the nose, and the behavior that the mask covers the mouth and the nose is in the standard. After the pictures are taken out, the pictures need to be marked, mask samples which are worn or not worn are marked, whether the mask samples are worn according to the standard or not is marked, and the samples are input into a GPU server to perform behavior recognition training.
The method comprises the steps of collecting materials of a specific kitchen scene, slicing and marking the materials, training the behavior specification of the specific kitchen scene, and introducing, processing, identifying and early warning of a specific kitchen monitoring camera. Under the special specification of a specific scene, the behavior logics have no universality in the traditional sense and belong to customized kitchen scene recognition. On the basis that the materials identified by the scenes are not universal, different kitchen objects cannot be shared, and generally, the materials also cannot be shared, the method needs to research the behavior characteristics of each kitchen object, the materials with specific behavior characteristics are provided by depending on the specific kitchen objects, and the materials are mainly video materials. And importing the video materials into a GPU cluster, cutting the materials into pictures according to proper rules, and processing based on the pictures.
The process of studying the regulation of the behavior of the kitchen is essentially the process of studying the management of the kitchen. The management of the kitchen essentially needs to follow some general rules, the behavior and scene recognition of the general rules construct the first layer judgment logic of the intelligent behavior specification judgment model, and the judgment logic is not only a pre-judgment standard, but also the setting of the industry bottom line.
Compared with the general regulation of the kitchen, the general regulation has different industry standards and specifications aiming at different industries. Therefore, the general rules of different types of kitchens, such as student canteens, enterprise and public institution canteens and chain restaurants, are common and have a great deal of difference. The regulations often need supervisors to jointly provide corresponding material requirements or provide normative materials, a second-layer judgment logic of intelligent behavior normative judgment is constructed based on the materials, and the judgment logic is often the supervision core of a manager and is a behavior rule and a norm which must be observed for a kitchen.
Often, a particular kitchen management entity has its own more stringent or specific management behavior guidelines than industry standards and specifications. These requirements are themselves specific to a particular behavioral specification and do not have industry commonality. For example, some kitchens require that "three whites" be worn, i.e., white caps, white clothes, and white masks. Some kitchen needs to wear clothes, hats and masks printed with logo of enterprises and special colors and standards. Often in many kitchens, managers pay more attention to the behavior specification under the condition that the requirements of the first and second layers are met. The material sources can only be specific materials of a specific kitchen, and a third layer judgment logic of intelligent behavior specification judgment is constructed based on the materials, is a core of attention of managers, and is a behavior rule and a specification which must be observed for enterprise culture and kitchen management.
All the objects analyzed and judged by the model are divided into a dynamic object ' person ' of the behavior main body and a static object ' of the behavior object.
The dynamic object "person" is classified into "behavior pattern judgment" and "attribute state judgment". The behavior pattern determines whether or not an inappropriate event, such as smoking in a kitchen, leaving from a kitchen on fire, or the like, has been made at an inappropriate time. The attribute state is to judge whether the attribute of the person is in accordance with the specification requirement, such as wearing according to the specification, and the like.
The static object "has only" attribute state judgment ". I.e. whether a particular static object exhibits a particular state at a particular time.
The behavior rule judgment model is the most important core module of the method. In general, the method can carry out continuous training or migration training based on a continuously constructed general behavior rule model to achieve better effect. The general behavior judgment model and the specific kitchen scene model are trained and fused separately, and through the attempt, the efficiency can be improved remarkably, and the cost can be reduced.
The model deployment mode is a general, portable and cross-platform deployment mode. The subsequent behavior specification judgment model can only be deployed on a GPU server or a server cluster in principle, and after the subsequent behavior specification judgment model is deployed, video streams can be processed according to rules.
According to the behavior judgment method, the data source is the monitoring video of the kitchen. The video can be single-path or multi-path, and the capability of processing the video is mainly limited by the calculation power limit of the GPU server. In the GPU server, the model cuts the video, judges whether the object behavior is in accordance with the standard or not, and outputs abnormal information which is not in accordance with the standard.
Drawings
The invention is further described below with reference to the accompanying drawings.
Fig. 1 is a schematic block diagram of a kitchen violation intelligent judgment method based on video behavior recognition.
Detailed Description
The invention is further described with reference to the following figures and specific examples.
The invention relates to a kitchen specification intelligent judgment method based on video behavior recognition, which is based on a basic behavior specification complied with by a kitchen bottom layer, an industry behavior specification required by a supervision unit and a specific kitchen behavior specification defined by each kitchen management organization. In the whole process, research objects are 'people' and 'objects' in a kitchen, the objects in the scene are positioned based on an object detection algorithm, and the behavior specifications of the objects are analyzed one by one according to a model generated by three types of behavior specification materials. In the process, all processes of material collection, material marking, model training, model deployment, video access, model judgment and result output are covered.
Example 1:
a specific school back was determined by the technician as an example of implementation. Generally, the kitchen itself needs to follow the basic behavior specification of the kitchen, the management rule of the school canteen kitchen and the relevant regulation of the school canteen management.
And arranging the basic behavior specifications based on the basic behavior specifications of the kitchen. For example, a person working in a kitchen should wear a mask, the person working in the kitchen should not have four pests, namely mice, cockroaches, flies and mosquitoes, and a technician needs to collect the related resources of the four pests, namely pictures and video data, worn by the mask. After these data are acquired, the video is first processed and cut into pictures according to a predetermined rule.
On the basis of school kitchen regulations specified by a supervision unit, technicians are required to collect and comb the regulations and extract the part in which behavior identification can be carried out, for example, kitchen workers cannot wear jewelry and watches, and the time for which a kitchen mosquito killer lamp and a disinfection lamp are required to be fully opened every day is long. These requirements are translated into product requirements and perhaps related video material and a small amount of auxiliary picture material with the support of regulatory and back-office. Since these requirements generally limit the industry and even some segment of the industry, the picture data is difficult to reach the amount and scale required by training, so the material is mainly video. After the data are acquired, the video is processed and cut into pictures.
Based on the self-defined specifications of school logistics management departments, technical staff are required to directly communicate with the school logistics departments, the collection and the combing are carried out based on communication contents, and parts which can be used for behavior identification are extracted, for example, kitchen workers need to be distributed according to regions and cannot operate across regions, the workers need to wear working clothes uniformly equipped in schools and cannot wear irregular clothes, the workers need to return kitchenware before going off duty, the kitchenware cannot be visited sporadically, and the like, and the requirements are converted into products. The material intelligent school logistics part is provided, and the logistics department is also required to set a plurality of states to prepare the material, such as the returning of kitchen ware and the scattered visit state, and the material is mainly a video. After the data are acquired, the video is processed and cut into pictures.
The method comprises the steps that technicians mark all types of pictures, the marked pictures enter a training GPU cluster to be trained based on a deep learning algorithm, a verification model is generated according to the trained result, and the model is deployed to a GPU server cluster.
And then, the intelligent judgment of kitchen violation can be carried out in real time only by accessing the video stream of the real-time monitoring camera.
With the above embodiments, all kitchen managers can easily implement the present invention. Any technical engineer may implement the process in our way.

Claims (4)

1. An intelligent judgment method for character and article behavior specifications and state specifications based on a kitchen operation scene comprises the steps of carrying out slicing, marking and scene behavior training on various research objects including people and objects, correct and wrong behavior videos in a kitchen, generating a behavior recognition model according to a training result, guiding the model into a GPU server cluster, accessing a conventional monitoring video in the kitchen into the GPU server cluster, slicing the video, carrying out rule judgment according to rules, and outputting a conclusion. In the whole process, manual intervention judgment is not needed, all intelligent realization is realized, and meanwhile, the industrial standard characteristics are covered, and the special standard requirements of enterprises are not lacked.
2. The method for introducing the video covering the correct and wrong specification set in the kitchen into the training server to finally generate the model for judging the behavior of the kitchen specification according to claim 1, wherein the video itself needs to contain the correct and wrong demonstration of the operation of the kitchen, and the video is contained in various light, time and environmental characteristic sample segments to ensure the effect of the generation of the final model.
S1, collecting standard training videos is divided into three categories, wherein the first category is general standard scene training, such as whether a mask is worn in the operation process, the second category is special behavior standard, such as whether enterprise work clothes and hats are worn correctly, and the third category is specific state standard, such as whether a workbench is cleaned in non-working time.
And S2, entering the training standard video into a GPU server for slicing to generate a plurality of pictures, and marking the standard area in the pictures based on a picture marking tool.
And S3, training based on the marked picture, and finally generating a model for intelligent judgment.
3. The model is deployed to a GPU server cluster according to claim 1, and multiple video streams can be processed simultaneously, and the server cluster is expanded as required, so that commercial feasibility is guaranteed.
4. Accessing a subsequent surveillance video stream to a cluster of servers according to claim 1, the cluster satisfying the ability to automate processing of the video stream and output a corresponding conclusion.
CN202010695617.5A 2020-07-17 2020-07-17 Intelligent kitchen violation judgment method based on video behavior recognition Pending CN111860307A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010695617.5A CN111860307A (en) 2020-07-17 2020-07-17 Intelligent kitchen violation judgment method based on video behavior recognition

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010695617.5A CN111860307A (en) 2020-07-17 2020-07-17 Intelligent kitchen violation judgment method based on video behavior recognition

Publications (1)

Publication Number Publication Date
CN111860307A true CN111860307A (en) 2020-10-30

Family

ID=73000676

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010695617.5A Pending CN111860307A (en) 2020-07-17 2020-07-17 Intelligent kitchen violation judgment method based on video behavior recognition

Country Status (1)

Country Link
CN (1) CN111860307A (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112560759A (en) * 2020-12-24 2021-03-26 中再云图技术有限公司 Bright kitchen range standard detection and identification method based on artificial intelligence, storage device and server
CN112907258A (en) * 2021-01-26 2021-06-04 云南易见纹语科技有限公司 Product production process visual tracing method and system, electronic equipment and storage medium
CN112926507A (en) * 2021-03-24 2021-06-08 安徽超视野智能科技有限公司 Kitchen wearable monitoring device and method based on image recognition
CN113705413A (en) * 2021-08-23 2021-11-26 深圳市康索特软件有限公司 Kitchen monitoring method and device and storage medium
CN114863371A (en) * 2022-07-11 2022-08-05 济南大学 Food rough machining violation detection method based on deep learning
CN116994208A (en) * 2023-08-22 2023-11-03 中广核服务集团有限公司 Intelligent monitoring system based on artificial intelligence and used for kitchen behavior standardization

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110081082A1 (en) * 2009-10-07 2011-04-07 Wei Jiang Video concept classification using audio-visual atoms
CN105554591A (en) * 2015-12-02 2016-05-04 蓝海大数据科技有限公司 Video analysis method and device
US20180007429A1 (en) * 2015-01-26 2018-01-04 Hangzhou Hikvision Digital Technology Co., Ltd. Intelligent processing method and system for video data
US20190392266A1 (en) * 2018-06-20 2019-12-26 Agora Lab, Inc. Video Tagging For Video Communications
CN110717448A (en) * 2019-10-09 2020-01-21 杭州华慧物联科技有限公司 Dining room kitchen intelligent management system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110081082A1 (en) * 2009-10-07 2011-04-07 Wei Jiang Video concept classification using audio-visual atoms
US20180007429A1 (en) * 2015-01-26 2018-01-04 Hangzhou Hikvision Digital Technology Co., Ltd. Intelligent processing method and system for video data
CN105554591A (en) * 2015-12-02 2016-05-04 蓝海大数据科技有限公司 Video analysis method and device
US20190392266A1 (en) * 2018-06-20 2019-12-26 Agora Lab, Inc. Video Tagging For Video Communications
CN110717448A (en) * 2019-10-09 2020-01-21 杭州华慧物联科技有限公司 Dining room kitchen intelligent management system

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112560759A (en) * 2020-12-24 2021-03-26 中再云图技术有限公司 Bright kitchen range standard detection and identification method based on artificial intelligence, storage device and server
CN112560759B (en) * 2020-12-24 2022-04-22 中再云图技术有限公司 Bright kitchen range standard detection and identification method based on artificial intelligence, storage device and server
CN112907258A (en) * 2021-01-26 2021-06-04 云南易见纹语科技有限公司 Product production process visual tracing method and system, electronic equipment and storage medium
CN112926507A (en) * 2021-03-24 2021-06-08 安徽超视野智能科技有限公司 Kitchen wearable monitoring device and method based on image recognition
CN113705413A (en) * 2021-08-23 2021-11-26 深圳市康索特软件有限公司 Kitchen monitoring method and device and storage medium
CN114863371A (en) * 2022-07-11 2022-08-05 济南大学 Food rough machining violation detection method based on deep learning
CN116994208A (en) * 2023-08-22 2023-11-03 中广核服务集团有限公司 Intelligent monitoring system based on artificial intelligence and used for kitchen behavior standardization
CN116994208B (en) * 2023-08-22 2024-04-09 中广核服务集团有限公司 Intelligent monitoring system based on artificial intelligence and used for kitchen behavior standardization

Similar Documents

Publication Publication Date Title
CN111860307A (en) Intelligent kitchen violation judgment method based on video behavior recognition
Souitaris Firm–specific competencies determining technological innovation: A survey in Greece
Greasley et al. Modelling people’s behaviour using discrete-event simulation: a review
CN105740339A (en) Civil administration big data fusion and management system
CN106779592A (en) A kind of improving production management banner system and application process
Rejeb et al. Enablers of augmented reality in the food supply chain: a systematic literature review
CN111340652A (en) Meal diet safety dynamic supervision method based on big data analysis
CN114925873A (en) Chemical industry gathering area oriented actual combat emergency management system
CN111767329A (en) Key field food safety supervision informatization system
CN104700227A (en) Station affair management system platform of power supply station of county level power supply enterprise
CN110287856A (en) A kind of security personnel's behavior analysis system, method and device
Calkins et al. Taxonomy for surveying the use and value of geographical information!
CN117521969B (en) Intelligent park operation index calculation system based on digital twinning
CN111210377A (en) Network meal ordering supervision system and method based on cloud computing
Kim et al. Residents' perception of local brownfields in rail corridor area in the City of Roanoke: the effect of people's preconception and health concerns factors
CN111445168A (en) Quality safety third-party supervision system and method
Zhong et al. A heterogeneous data analytics framework for RFID-enabled factories
CN112929404A (en) Campus building automation thing networking system
Cortés-Polo et al. A novel methodology based on orthogonal projections for a mobile network data set analysis
Altarawneh et al. Business Intelligence and Information System Management: A Conceptual View
CN115439935A (en) Food processing safety management method and system based on machine vision
Zhuk et al. Informational and analytical supply in the management system
Emami et al. Decision framing and critical success factors of new product development
CN118097198B (en) Automatic dressing compliance management and control system and method based on artificial intelligence
Agrawal et al. Big Data: A Strategic Tool for Enlightening Business Decision Making

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination