CN114037937A - Real-time refrigerator food material identification method based on multi-target tracking - Google Patents
Real-time refrigerator food material identification method based on multi-target tracking Download PDFInfo
- Publication number
- CN114037937A CN114037937A CN202111320020.3A CN202111320020A CN114037937A CN 114037937 A CN114037937 A CN 114037937A CN 202111320020 A CN202111320020 A CN 202111320020A CN 114037937 A CN114037937 A CN 114037937A
- Authority
- CN
- China
- Prior art keywords
- food material
- video
- real
- target tracking
- extracting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Probability & Statistics with Applications (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention combines deep learning and computer vision algorithm, and particularly discloses a real-time refrigerator food material identification method with multi-target tracking, which comprises the following steps: s1, acquiring food material information in the refrigerator through a camera; s2, uploading the video information of the obtained food materials to a local server; s3, preprocessing the uploaded video, and performing equal-interval framing; s4, extracting the characteristics of the video information by using a convolutional neural network, and detecting and extracting the matching characteristics of the video information by using a multi-target tracking method; s5, comprehensively analyzing the characteristics by using a deep learning method, and judging the type and the motion track of the food material; and s6, judging the state of the food material, and feeding back the result in real time. According to the method, the food material video information is analyzed by using the convolutional neural network, the relation characteristics among each frame of the video are fully mined by using the multi-target tracking method, and the state of the food material to be detected is detected in real time.
Description
Technical Field
The invention combines deep learning and a computer vision algorithm, and particularly discloses a real-time refrigerator food material identification method based on multi-target tracking.
Background
The intelligent home system utilizes advanced computer technology, network communication technology, intelligent cloud control, comprehensive wiring technology and medical electronic technology to integrate individual requirements according to the principle of human engineering, organically combines various subsystems related to home life such as security protection, light control, curtain control, gas valve control, information household appliances, scene linkage, floor heating, health care, epidemic prevention, security protection and the like, and realizes the brand-new home life experience of people-oriented through networked comprehensive intelligent control and management. The intelligent refrigerator is a part of intelligent home, and in the development of the intelligent refrigerator, the identification of refrigerator food materials is an inevitable problem.
In the current implementation method, one method is to punch an electronic tag on the food material put into the refrigerator, and scan the electronic tag after the food material is taken out, so as to determine the information of the food material such as putting, taking and storing time. However, this method may make the user experience worse, and may make the function of identifying food material cumbersome; in addition, another method is refrigerator food identification based on images, and the method processes the images by using a deep learning method to judge the types of food materials in the images so as to achieve the identification purpose, but the method has great defects at present, one is that the influence of a background is not eliminated, and in addition, the motion track of the food materials cannot be known, so that the method cannot judge whether the food materials are put in or taken out.
The existing methods can be divided into a traditional method and a deep learning-based method, the traditional method causes the process of food material identification to become complicated, the use experience of a user is influenced to a great extent, and the required intelligent level cannot be achieved. The method based on deep learning is still imperfect at present, and a great improvement space exists for the influence of the background, the position of a camera, a feature extraction method and the like.
Therefore, it is very necessary to construct a more convenient method for identifying refrigerator food materials by combining deep learning and computer vision.
Disclosure of Invention
The invention aims to provide a multi-target tracking amount real-time refrigerator food material identification method, which adopts the following scheme:
a real-time refrigerator food material identification method based on multi-target tracking comprises the following steps:
s1, acquiring food material information in the refrigerator through a camera;
s2, uploading the video information of the obtained food materials to a local server;
s3, preprocessing the uploaded video, and performing equal-interval framing;
s4, extracting the characteristics of the video information by using a convolutional neural network, and detecting and extracting the matching characteristics of the video information by using a target tracking method;
s5, comprehensively analyzing the characteristics by using a deep learning method, and judging the type and the motion track of the food material;
and s6, judging the state of the food material, and feeding back the result in real time.
Further, in the step s1, the motion characteristic information of the food material is captured in real time and all-around manner by adjusting the camera angle and performing multi-camera fusion.
Further, in the step s2, the information obtained in the step s1 needs to be uploaded to the local server at regular intervals, and the state of the food material needs to be analyzed.
Further, in the step s3, 10 video frame pictures are extracted every second according to the algorithm response time by the video framing technique.
Further, in the above step s4, feature extraction for detection and matching fusion is completed.
Further, the data processing comprises the following specific steps:
s41, detecting each frame of picture food material and extracting corresponding type and position characteristics through a detection network on the basis of the step s 3;
s42, calculating the IOU difference value and the feature difference value on the basis of the step s41 by extracting the detection features of the next frame image;
s43, completing detection box matching between the connected frames on the basis of the step s42 through Hungarian algorithm, and recording the types of the detection boxes;
further, in the step s5, the analysis processing is performed according to the feature extraction result in the step s4, and the analysis processing is sent to the track distinguishing and optimizing module to identify the motion track of the food material.
Further, in the step s6, the state of the food material is fed back in real time according to the analysis result in the step s 5.
The invention has the following advantages:
according to the method, the food material state is judged by using a video method creatively in the detection work of the refrigerator food materials through a deep neural network and a computer vision method, the Hungarian matching algorithm is added to match the detection frame by using the IOU difference value and the characteristic difference value, the matching efficiency is enhanced, the algorithm processing time is greatly shortened by using a video processing method of target detection and matching, the food material state is more efficiently and quickly identified, the characteristics of a time domain are considered compared with an image method, and the running speed is high with small parameter number.
Drawings
Fig. 1 is a flow chart of a real-time refrigerator food material identification method based on multi-target tracking in the invention.
Detailed description of the invention
The invention is described in further detail below with reference to the following figures and detailed description:
referring to fig. 1, a real-time refrigerator food material identification method based on multi-target tracking includes the following steps:
s1, acquiring food material information in the refrigerator through a camera;
in order to fully acquire the characteristic information of the food in the refrigerator, the method uses a plurality of cameras for shooting and fuses images, so that the information of the food can be fully captured by the cameras.
s2, uploading the video information of the obtained food material to a local server
The food material state is obtained in real time, and the change of the food material needs to be detected in real time, so that the method needs to finish uploading data every short time so as to detect the food material.
s3, preprocessing the video, framing at equal intervals, and converting into a standard data form;
since a section of finished video is uploaded to the server, and if the complete video is detected, the complexity of detection is greatly increased, for this reason, the video is framed by a video framing technique, specifically 10 frames per second are extracted and converted into a standard data form.
s4, extracting the characteristics of the video information by using a convolutional neural network;
in order to fully extract all visual features of food material motion, the method considers the influence of detection and matching on food material identification, and constructs a corresponding feature extraction architecture.
s41, detecting each frame of picture food material and extracting corresponding type and position characteristics through a detection network on the basis of the step s 3;
s42, calculating the IOU difference value and the feature difference value on the basis of the step s41 by extracting the detection features of the next frame image;
s43, completing detection box matching between the connected frames on the basis of the step s42 through Hungarian algorithm, and recording the types of the detection boxes;
through the above operations, the feature extraction work for the current data is completed.
s5, comprehensively analyzing the characteristics based on the deep neural network model, and judging the food material types and the motion tracks;
and processing and analyzing the characteristics according to the characteristics obtained by the previous data processing, distinguishing the food material types by adopting a Softmax classifier, and recording the motion trail of the food material.
And s6, finishing the food material state judgment work and feeding back the result in real time.
It should be understood, however, that the description above is only for the purpose of illustrating preferred embodiments of the present invention, and it is intended by those skilled in the art that the present invention not be limited to the embodiments described above, but that all equivalent and obvious modifications can be made within the spirit and scope of the present invention as defined by the appended claims.
Claims (7)
1. A real-time refrigerator food material identification method based on multi-target tracking is characterized by comprising the following steps:
s1, acquiring food material information in the refrigerator through a camera;
s2, uploading the video information of the obtained food materials to a local server;
s3, preprocessing the uploaded video, and performing equal-interval framing;
s4, extracting the characteristics of the video information by using a convolutional neural network, and detecting and extracting the matching characteristics of the video information by using a target tracking method;
s5, comprehensively analyzing the characteristics by using a deep learning method, and judging the type and the motion track of the food material;
and s6, judging the state of the food material, and feeding back the result in real time.
2. The method as claimed in claim 1, wherein in the step s1, video characteristic information is obtained during the motion of the food material.
3. The method as claimed in claim 1, wherein in the step s2, the data obtained in the step s1 is uploaded to a local server for analysis.
4. The method as claimed in claim 1, wherein in step s3, the preprocessing of data is performed, and the video framing technique is used to extract 10 frames per second and extract consecutive video frames.
5. The method as claimed in claim 1, wherein in the step s4, the specific processing procedure of feature extraction is as follows:
s41, detecting each frame of picture food material and extracting corresponding type and position characteristics through a detection network on the basis of the step s 3;
s42, calculating the IOU difference value and the feature difference value on the basis of the step s41 by extracting the detection features of the next frame image;
s43, completing detection box matching between the connected frames on the basis of the step s42 through Hungarian algorithm, and recording the types of the detection boxes;
6. the method as claimed in claim 1, wherein in the step s5, the method is used for distinguishing the type of the target food material and judging the track by sending the characteristics extracted in the step s4 to a deep learning network.
7. The method as claimed in claim 1, wherein in the step s6, the real-time feedback of the food material status is performed according to the analysis result of s5, so as to reduce the misjudgment and property loss caused by time lag.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111320020.3A CN114037937A (en) | 2021-11-09 | 2021-11-09 | Real-time refrigerator food material identification method based on multi-target tracking |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111320020.3A CN114037937A (en) | 2021-11-09 | 2021-11-09 | Real-time refrigerator food material identification method based on multi-target tracking |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114037937A true CN114037937A (en) | 2022-02-11 |
Family
ID=80143596
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111320020.3A Pending CN114037937A (en) | 2021-11-09 | 2021-11-09 | Real-time refrigerator food material identification method based on multi-target tracking |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114037937A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117746304A (en) * | 2024-02-21 | 2024-03-22 | 浪潮软件科技有限公司 | Refrigerator food material identification and positioning method and system based on computer vision |
-
2021
- 2021-11-09 CN CN202111320020.3A patent/CN114037937A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117746304A (en) * | 2024-02-21 | 2024-03-22 | 浪潮软件科技有限公司 | Refrigerator food material identification and positioning method and system based on computer vision |
| CN117746304B (en) * | 2024-02-21 | 2024-05-14 | 浪潮软件科技有限公司 | Refrigerator food material identification and positioning method and system based on computer vision |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110543867B (en) | Crowd density estimation system and method under condition of multiple cameras | |
| CN109145759B (en) | Vehicle attribute identification method, device, server and storage medium | |
| CN108846365B (en) | Detection method and device for fighting behavior in video, storage medium and processor | |
| Lao et al. | Automatic video-based human motion analyzer for consumer surveillance system | |
| US9462160B2 (en) | Color correction device, method, and program | |
| CN111626201B (en) | Commodity detection method, commodity detection device and readable storage medium | |
| CN113139521B (en) | Pedestrian boundary crossing monitoring method for electric power monitoring | |
| CN102521578B (en) | Method for detecting and identifying intrusion | |
| TW201737134A (en) | System and method for training object classifier by machine learning | |
| CN109727275B (en) | Object detection method, device, system and computer readable storage medium | |
| EP2549759B1 (en) | Method and system for facilitating color balance synchronization between a plurality of video cameras as well as method and system for obtaining object tracking between two or more video cameras | |
| CN107654406B (en) | Fan air supply control device, fan air supply control method and device | |
| EP4035070B1 (en) | Method and server for facilitating improved training of a supervised machine learning process | |
| CN110096945B (en) | Indoor monitoring video key frame real-time extraction method based on machine learning | |
| WO2022213540A1 (en) | Object detecting, attribute identifying and tracking method and system | |
| CN111815528A (en) | Bad weather image classification enhancement method based on convolution model and feature fusion | |
| Shirsat et al. | Proposed system for criminal detection and recognition on CCTV data using cloud and machine learning | |
| CN114037937A (en) | Real-time refrigerator food material identification method based on multi-target tracking | |
| US20240096094A1 (en) | Multi-view visual data damage detection | |
| KR20200123324A (en) | A method for pig segmentation using connected component analysis and yolo algorithm | |
| CN112883836A (en) | Video detection method for deformation of underground coal mine roadway | |
| CN116886874A (en) | Ecological garden security monitoring and early warning data acquisition method and system | |
| CN114782860A (en) | Violent behavior detection system and method in monitoring video | |
| CN113052059A (en) | Real-time action recognition method based on space-time feature fusion | |
| KR101809031B1 (en) | Composition fire detection processing apparatus and video processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication |