CN115330376A - Diet health analysis system based on edge cloud integration - Google Patents

Diet health analysis system based on edge cloud integration Download PDF

Info

Publication number
CN115330376A
CN115330376A CN202210800770.9A CN202210800770A CN115330376A CN 115330376 A CN115330376 A CN 115330376A CN 202210800770 A CN202210800770 A CN 202210800770A CN 115330376 A CN115330376 A CN 115330376A
Authority
CN
China
Prior art keywords
dishes
cloud
image
information
algorithm
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210800770.9A
Other languages
Chinese (zh)
Inventor
陈泓宇
刘滔
储继盛
张博韬
吴艳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Harbin Institute of Technology
Original Assignee
Harbin Institute of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Harbin Institute of Technology filed Critical Harbin Institute of Technology
Priority to CN202210800770.9A priority Critical patent/CN115330376A/en
Publication of CN115330376A publication Critical patent/CN115330376A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q20/00—Payment architectures, schemes or protocols
    • G06Q20/08—Payment architectures
    • G06Q20/20—Point-of-sale [POS] network systems
    • G06Q20/208—Input by product or record sensing, e.g. weighing or scanner processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q20/00—Payment architectures, schemes or protocols
    • G06Q20/08—Payment architectures
    • G06Q20/20—Point-of-sale [POS] network systems
    • G06Q20/202—Interconnection or interaction of plural electronic cash registers [ECR] or to host computer, e.g. network details, transfer of information from host to ECR or from ECR to ECR
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q20/00—Payment architectures, schemes or protocols
    • G06Q20/08—Payment architectures
    • G06Q20/20—Point-of-sale [POS] network systems
    • G06Q20/204—Point-of-sale [POS] network systems comprising interface for record bearing medium or carrier for electronic funds transfer or payment credit
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q20/00—Payment architectures, schemes or protocols
    • G06Q20/30—Payment architectures, schemes or protocols characterised by the use of specific devices or networks
    • G06Q20/34—Payment architectures, schemes or protocols characterised by the use of specific devices or networks using cards, e.g. integrated circuit [IC] cards or magnetic cards
    • G06Q20/352—Contactless payments by cards
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/06—Buying, selling or leasing transactions
    • G06Q30/0601—Electronic shopping [e-shopping]
    • G06Q30/0631—Item recommendations
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/42—Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/48—Extraction of image or video features by mapping characteristic values of the pattern into a parameter space, e.g. Hough transformation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/56—Extraction of image or video features relating to colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/94—Hardware or software architectures specially adapted for image or video understanding
    • G06V10/95—Hardware or software architectures specially adapted for image or video understanding structured as a network, e.g. client-server architectures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/68—Food, e.g. fruit or vegetables
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
    • G07G1/00—Cash registers
    • G07G1/0009—Details of the software in the checkout register, electronic cash register [ECR] or point of sale terminal [POS]
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
    • G07G1/00—Cash registers
    • G07G1/0018—Constructional details, e.g. of drawer, printing means, input means
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
    • G07G1/00—Cash registers
    • G07G1/0036—Checkout procedures
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07G—REGISTERING THE RECEIPT OF CASH, VALUABLES, OR TOKENS
    • G07G1/00—Cash registers
    • G07G1/12—Cash registers electronically operated
    • G07G1/14—Systems including one or more distant stations co-operating with a central processing unit
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
    • G16H20/60—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to nutrition control, e.g. diets

Abstract

The invention relates to a diet health analysis system based on edge cloud integration. The invention relates to the technical field of computer vision and artificial intelligence, and the invention utilizes machine vision to automatically and rapidly identify and price-calculate the color, shape, size and other characteristics of dinner plates, drinks and other articles, effectively improves the calculation efficiency, saves manpower and brings more convenient experience to customers. Meanwhile, the invention utilizes the front-end machine to visually acquire information and continuously update the cloud database for the cloud server to use. And carrying a deep learning algorithm on the cloud, further accurately identifying the food varieties, acquiring the diet data of the customers, analyzing the nutritional information, and constructing a data sharing and analyzing system with cloud coordination.

Description

Diet health analysis system based on edge cloud integration
Technical Field
The invention relates to the technical field of computer vision and artificial intelligence, in particular to a food health analysis system based on edge cloud integration.
Background
The optional restaurant is a restaurant which is selected by customers to have dinner suitable for the tastes of the customers, various dishes are placed in dinner plates with different colors, shapes or sizes for the customers to select, and the customers are charged according to the characteristics of the dinner plates during settlement. At present, the self-service restaurant mainly takes manual settlement as a main part, the automatic settlement system is not developed completely, and the problems of high application cost, limitation of settlement types to dishes or bowls of food, low settlement efficiency and the like exist. In addition, because the types of food in the self-service restaurant are few, the selection of the food by the customer is difficult, and the reasonable decision is not easy to be made according to the self diet health condition, so that the historical dining data of the customer cannot be fully utilized. Therefore, in order to improve the service quality of the free dining room and improve the dining experience of diners in the free dining room, an efficient and practical free dining room settlement and catering recommendation service system is urgently needed, the processing efficiency of automatic settlement of dishes is improved, and dish recommendation is performed according to historical diet data of diners and aiming at diet health states.
In view of some of the problems of the conventional cafeterias, in recent years, many service systems applied to the cafeterias have appeared on the market. However, these systems often have defects, and the defects of the service system of the self-service dining room in the current market are summarized as follows:
[1] in the market, a disk-loaded RFID radio frequency chip is mostly adopted, and the settlement function is completed by identifying the radio frequency chip at the edge end, so that the chip is easy to damage and the maintenance cost is high;
[2] most of mainstream free restaurant service systems in the market adopt a single camera to complete the task of image acquisition, and have certain limitations, such as only taking pictures and identifying from right above the food, neglecting side features, and being difficult to effectively identify canned beverages with similar top features;
[3] at present, most of automatic settlement systems for the selected restaurants deploy a deep learning method at an embedded edge end to realize the identification of dishes and further complete settlement. The processing mode has high requirement on the calculation performance of the edge processing chip, high cost, low recognition speed and long settlement time, is not suitable for the condition of large-scale passenger flow volume real-time settlement, and is not beneficial to popularization. The existing market lacks a self-service restaurant service system with edge-cloud cooperation, so that efficient settlement processing is realized while intelligent analysis and calculation of a large amount of dish data are met;
[4] the current free dining room lacks a nutrition analysis system for dining data of customers, and can not recommend dishes and provide dining suggestions according to the needs of users, so that a great deal of diet data of customers is wasted.
Based on the problems, the invention provides a healthy diet analysis system based on edge-cloud integration, which utilizes the advantages of high settlement efficiency and rich cloud computing resources of edge terminals to directly identify dinner plates, can perform edge-cloud cooperation for providing strong processing capability for a deep learning algorithm to improve the current situation of a self-service restaurant, improves the experience quality of diners and the interactivity of diners and restaurants, and promotes the restaurants to develop in an intelligent and humanized direction. In the invention, the edge cloud integration refers to the cooperative work of combining the advantages of rapidness and convenience of edge terminals (settlement stations and clients) and the characteristics of a cloud terminal (server) capable of storing a large amount of data and carrying a deep learning algorithm. The diet health analysis system is used for processing and analyzing the dining data to obtain nutrition information of the user and giving suggestions such as dish recommendation and nutritional dining based on the nutrition information.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention utilizes machine vision to automatically and quickly identify and price-calculate the color, shape, size and other characteristics of dinner plates, drinks and other articles at a settlement table, thereby effectively improving the settlement efficiency, saving the labor and bringing more convenient experience to customers. Meanwhile, the invention utilizes the front-end machine to visually acquire information and continuously update the cloud database for the cloud server to use. And carrying a deep learning algorithm at the cloud end, further accurately identifying the food varieties, acquiring the food data of the customers, analyzing the nutrition information, and constructing a data sharing and analyzing system with cloud coordination.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
The invention provides a diet health analysis system based on edge cloud integration, and the invention provides the following technical scheme:
a cloud-based integrated dietary health analysis system, the system comprising:
the food product image acquisition system comprises an image information acquisition end, an edge end control pressure sensor and three camera modules, wherein the image information acquisition end acquires food product image information from three angles by using a camera, and acquires food product images from three angles;
the main controller intelligently identifies pricing, the price of dishes is determined by the shape and the color of the dinner plate, the price is calculated by identifying the color and the shape of the dinner plate through the main controller, and the main controller controls the RFID card reader module to complete fee deduction;
the server comprises a cloud data storage and processor, an edge end transmits acquired food image data to a cloud end, the cloud end runs a trained ResNet152 neural network model to identify received dish images, stores the data, analyzes the nutritional structure and taste of a user in a pertinence mode by using stored historical data, records the preference of the user and recommends dishes;
and the side cloud interaction end and the cloud end are data centers, receive images of the edge section settlement tables, and transmit the analysis data to the user side WeChat applet after the analysis is completed.
Preferably, the main controller adopts an STM32 single chip microcomputer, and the STM32 single chip microcomputer receives data of the pressure sensor and triggers the functions of dinner plate identification and pricing settlement; when a dinner plate is placed in the device, the pressure sensor is triggered, a trigger signal is transmitted to the STM32 single chip microcomputer through serial port communication, the STM32 single chip microcomputer sends a trigger message to the three camera modules through the serial port communication, and the camera modules are triggered to take a picture; when a dinner plate is placed next time, the sensor is triggered again to control the camera module to work.
Preferably, the STM32 single chip microcomputer is in serial port communication with the camera modules, the dinner plates are respectively identified by the three camera modules at three angles, three prices are calculated and are transmitted to the STM32 single chip microcomputer, the STM32 receives the prices, the final commodity price is obtained according to the pricing strategy, and then the final price is displayed on the screen.
Preferably, the STM32 singlechip carries out serial communication with the card reader, and when the customer punched the card the operation, the card reader passed through serial communication and gives the STM32 singlechip with IC-card information transfer, and the STM32 singlechip reads information and returns the price that should deduct according to the price that the camera module obtained, carries out the operation of deducting, and the result of deducting shows on the screen.
Preferably, by adopting a UART serial port communication IC card reader of M3650A-HA model, when the card reader swipes a card, a card number command packet is automatically returned to the STM32 single chip microcomputer and is displayed on a display screen of the STM32 single chip microcomputer after processing, then the STM32 single chip microcomputer sends a money deduction command packet to the card reader according to price information, the card reader changes the balance in the card, and the balance information is returned to the STM32 single chip microcomputer and is displayed on the display screen after processing.
Preferably, when settlement is carried out at the edge end, capturing a circle of the dinner plate, identifying the circle by using Hough transform, wherein the basic idea of detecting the circle by Hough transform is to map edge points in an image space into a parameter space, carrying out accumulation statistics on accumulated values corresponding to all coordinate point elements obtained in the parameter space, and judging the radius of the circle and the position of the circle center according to the accumulated values;
for the circular edge part, when the edge range is from 0.7 times to 1 time of radius away from the circle center, color acquisition is carried out, and color judgment is carried out according to the mode of an LAB value;
the identification of beverages is completed by adding the cameras at the side and the rear, the side view camera module and the rear view camera module respectively utilize a template matching algorithm to meter non-dinner plate-shaped containers, such as bowls and beverage articles, the orthogonal two camera modules avoid the problem of mutual shielding in beverage placement to the maximum extent, when one camera module cannot observe shielded beverages, the other camera module realizes observation, the identification of the articles in a three-dimensional space is completed, accurate and non-missing metering of various articles is realized, and then the two cameras upload the identified data to an STM32 main control board for data processing;
the recognition algorithm with NCC algorithm template matching as the main part and feature point recognition as the auxiliary part finishes the recognition and the pricing of the intercepted image, and the specific recognition process is as follows: firstly, comparing a gray-scale image with a template by using an NCC (non-uniform correlation) algorithm to obtain a gray-scale image of the image, and completing the identification process of the beverage by using a mode of traversing the template; when the NCC multi-template matching algorithm identification is insufficient, a feature point detection algorithm is started for identification, and the method specifically comprises the following steps:
the method comprises the steps of shooting an image, constructing an n multiplied by n neighborhood as a matching window for any pixel point (px, py) in an original image, and constructing an n multiplied by n domain as the matching window at a target pixel position (px + d, py) in the same way;
and (3) judging the correlation between the two neighborhoods by adopting a normalized cross correlation algorithm:
Figure BDA0003737508010000051
wherein, wp is a matching window, I1 (x, y) is a pixel value of the original image, I1 (px, py) is an average value of pixels in the original window, I2 (x + d, y) is a pixel value of the original image after the corresponding point position on the target image shifts by d in the x direction, and I2 (px + d, py) is an average value of pixels in the matching window of the target image;
the NCC calculation result is between [ -1,1], when NCC = -1, it means that the two matching windows are completely uncorrelated, and conversely, when NCC =1, it means that the correlation degree of the two matching windows is very high;
defining a region-of-interest rectangular tuple roi (x, y, w, h), step size step and similarity threshold, performing correlation calculation on the region-of-interest by using a fixed step value, comparing the correlation calculation with the threshold, completing template matching, and successfully identifying the article;
when the multi-template matching is insufficient, completing feature point matching by calculating the Euclidean distance between the image and the set feature point;
setting up a beverage template with high selection rate in a front setting, setting a beverage with low selection rate in a rear setting, and setting the beverage templates with low selection rate in a rear setting, wherein the traversing sequences of the two cameras are different, so that the speed is increased by means of the total number of the beverages and the jumping-out circulation, and the beverage templates can still meet the actual speed requirement in the identification of a large number of types of beverages;
the data that two camera module discernments obtained can carry out the processing of contrasting with the top camera module when handling data, can reset when appearing discerning the object total number and mismatch and discern again, and the machine will remind customer to adjust the dinner plate and discern again when discerning many times still can't solve the problem.
A diet health recommendation method based on edge cloud integration comprises the following steps:
a nutrition analysis algorithm is needed for nutrition analysis of the food, the nutrition taken by the user is judged according to the recent dining condition of the user through the establishment of the nutrition analysis algorithm, and the food with less nutrition taken is recommended;
preprocessing the nutrient information of the dishes, obtaining the nutrient content information characteristics of each dish of similar dishes in the selected restaurant in advance, wherein the nutrient content information characteristics comprise five contents of protein, fat, carbohydrate, sodium and cholesterol, and carrying out standardization and regularization operation;
and (4) clustering dishes in the self-service restaurant according to the characteristic groups based on a K-means algorithm to be used as the basis for analyzing the dishes eaten by the subsequent users.
Preferably, the information of the dishes is preprocessed, firstly, min-Max standardization is adopted for the dishes, and the nutrition information of the dishes is mapped in a [0,1] interval and is represented by the following formula:
Figure BDA0003737508010000061
wherein, a max,j Is the maximum value in the sample over the jth feature; a is a min,j Is the minimum value in the sample at the j-th feature;
the nutritional ingredients are regularized and expressed as a percentage by the following formula:
Figure BDA0003737508010000062
adopting a K-means algorithm to cluster the dishes, and adopting an Euclidean distance to be represented by the following formula:
Figure BDA0003737508010000071
the method of single connection clustering calculates the distance of dishes to the center of each cluster, uses the error square sum SSE as a target function, and takes the minimum value:
Figure BDA0003737508010000072
wherein K represents the Kth cluster, C i Denotes the ith center, dist denotes the euclidean distance;
for the k center C k Solving, the SSE is derived to make the derivative 0 and C is calculated k The calculation process is as follows:
Figure BDA0003737508010000073
Figure BDA0003737508010000074
analyzing and classifying recently eaten dishes of the user, and comparing the recently eaten dishes with the daily required nutrition standard of the human body proposed by the international union of nutrition science, recommending the corresponding high-nutrition-content dishes with less nutrition intake;
the specific implementation process is as follows:
judging the cluster distribution of dishes eaten by the user for N times, matching the eating conditions of the user for N times with the original clusters, and determining that fewer clusters are involved in corresponding characteristic cluster in the dishes eaten by the user;
adding the diet information of the dishes eaten for N times, comparing the sum with the daily required nutrition standard of the human body proposed by the international union of nutrition science, if the sum exceeds the standard, reminding, and if the sum does not exceed the standard, recording the cluster family corresponding to the nutrition characteristics with the intake amount greatly different from the standard intake;
and randomly recommending 5-6 dishes in the cluster group according to the obtained 2-3 characteristic cluster groups.
A computer-readable storage medium having stored thereon a computer program for execution by a processor for implementing, for example, a cloud-based integrated dietary health recommendation method.
A computer device comprising a memory and a processor, the memory having stored therein a computer program, the processor executing a dietary health recommendation method based on edge cloud integration when the processor runs the computer program stored by the memory.
The invention has the following beneficial effects:
the invention uses machine vision instead of radio frequency chip to settle accounts, does not need to frequently replace the loss tableware such as dinner plates and the like, and the modularized vision system is easier to maintain, thereby reducing the application cost;
the automatic settlement system is optimized aiming at the problem that the single angle detection angle is single in the traditional machine vision detection, the edge end image acquisition system is provided with three camera devices, the three camera devices are used for identifying from the upper direction, the side direction and the rear direction, the information of the articles to be settled is obtained from multiple angles, other articles such as drinks and the like can be identified besides the meal, the limitation of the traditional automatic settlement system is broken, and the automatic settlement function is optimized;
the invention utilizes machine vision to automatically and rapidly identify and price-calculate the color, shape, size and other characteristics of dinner plates, drinks and other articles at the settlement table, thereby effectively improving the settlement efficiency, saving manpower and bringing more convenient experience to customers. Meanwhile, the invention utilizes the front-end machine to visually acquire information and continuously update the cloud database for the cloud server to use. Carrying a deep learning algorithm on the cloud, further accurately identifying food varieties, acquiring food data of customers, analyzing nutritional information, and constructing a data sharing and analyzing system with cloud coordination;
in order to meet the health and diet requirements of customers, the invention designs a health management auxiliary small program, and dish recommendation is carried out according to cloud diet data and nutrition information to give a healthy diet suggestion.
The intelligent catering service system based on the edge cloud integrated structure is designed mainly aiming at application scenes of dining room catering settlement, healthy diet recommendation and intelligent distribution, and the application of the Internet of things in the intelligent campus is realized.
At the edge end, the STM32 single chip microcomputer is used for realizing measurement and control of each unit and communication with the cloud, the OpenMV platform is used for realizing multi-angle cooperative identification, the dinner plate is subjected to template matching and color and shape identification, and the RFID card reading technology and the pressure sensor module are matched, so that the rapid calculation function of the edge end is realized.
At the cloud, through the neural network that PyTorch built and the training set of up to 11 ten thousand pictures, realized the accurate discernment of dish. By utilizing the obtained dining data, an effective solution is provided for catering selection and diet health guarantee of guests.
On the whole, through communication and the cooperation of high in the clouds and edge, reached quick settlement and the high-efficient utilization of user's data of having dinner. The cloud end and the edge end have respective advantages, the cloud end provides high-efficiency computing power based on a large amount of centralized computing resources, the edge end solves the problems of large aggregate flow, prolonged communication time and the like, and better support is provided for realizing low-delay settlement work. The cost of the restaurant is reduced, and the income and information utilization rate are improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a general block diagram;
FIG. 2 is a process flow of an edge-side checkout station;
FIG. 3 is a flow chart of NCC template matching;
fig. 4 is a comet-eye meal-recognition small program interface.
Detailed Description
The technical solutions of the present invention will be described clearly and completely with reference to the accompanying drawings, and it should be understood that the described embodiments are only some embodiments, but not all embodiments, of the present invention. All other embodiments, which can be obtained by a person skilled in the art based on the embodiments of the present invention without any creative effort, belong to the protection scope of the present invention.
In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicate orientations or positional relationships based on those shown in the drawings, and are only for convenience of description and simplicity of description, but do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed in a particular orientation, and be operated, and thus, should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
In the description of the present invention, it should be noted that unless otherwise explicitly stated or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
In addition, the technical features related to the different embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
The present invention will be described in detail with reference to specific examples.
The first embodiment is as follows:
as shown in fig. 1 to 4, the specific optimization technical solution adopted by the present invention to solve the above technical problem is: the invention relates to a diet health analysis system based on edge cloud integration.
A cloud-based integrated dietary health analysis system, the system comprising:
the food product image acquisition system comprises an image information acquisition end, an edge end control pressure sensor and three camera modules, wherein the image information acquisition end acquires food product image information from three angles by using a camera, and acquires food product images from three angles;
the main controller intelligently identifies pricing, the price of dishes is determined by the shape and the color of the dinner plate, the price is calculated by identifying the color and the shape of the dinner plate through the main controller, and the main controller controls the RFID card reader module to complete fee deduction;
the server comprises a cloud data storage and processor, an edge end transmits acquired food image data to a cloud end, the cloud end runs a trained ResNet152 neural network model to identify received dish images, stores the data, analyzes the nutritional structure and taste of a user in a pertinence mode by using stored historical data, records the preference of the user and recommends dishes;
the cloud shoulder of the invention is used for providing personalized catering recommendation service for guests by using historical diet data, so that specific food needs to be accurately identified. The input of the recognition algorithm is a food image, and the output is a predicted food name after the food image is processed by a neural network.
According to the invention, the precise identification of the food variety is completed by using a deep learning algorithm carried by a cloud. By adopting a deep residual learning ResNet152 neural network, the network model adds residual operation on the basis of a convolutional neural network. By adopting a residual error learning mode, the prediction accuracy of the trained model cannot be gradually disappeared along with the increase of the number of network layers, which means that the prediction accuracy of the model is increased along with the increase of the number of network layers, but the accuracy is not reduced.
After dish identification is achieved through a deep learning algorithm, a cloud runs a nutrition recommendation algorithm and a dish recommendation algorithm to generate a diet suggestion for a specific user.
And the side cloud interaction end and the cloud end are data centers, receive images of the edge section settlement tables, and transmit the analysis data to the user side WeChat applet after the analysis is completed.
The second embodiment is as follows:
the difference between the second embodiment and the first embodiment is only that:
the main controller adopts an STM32 single chip microcomputer, and the STM32 single chip microcomputer receives data of the pressure sensor and triggers the functions of dinner plate identification and pricing settlement; when a dinner plate is placed in the device, the pressure sensor is triggered, a trigger signal is transmitted to the STM32 single chip microcomputer through serial port communication, the STM32 single chip microcomputer sends trigger information to the three camera modules through serial port communication, and the camera modules are triggered to take photos; when the dinner plate is placed next time, the sensor is triggered again to control the camera module to work.
The third concrete embodiment:
the difference between the third embodiment and the second embodiment is only that:
the STM32 singlechip carries out serial communication with the camera module, and three camera module is after three angle is discerned the dinner plate respectively, calculates three price and to STM32 singlechip transmission price, and STM32 receives the price and obtains final commodity price according to the pricing strategy, then shows final price on the screen.
And the pressure sensor is used for sensing that the dinner plate is placed, so that the camera is turned on. By the method, the direct use of camera sensing is avoided, and all cameras can be turned off when not used, so that the camera load can be effectively reduced, and the electric energy consumption is saved.
And three cameras are used for identification, are respectively positioned on the upper part, the side part and the rear part of the settlement table and are respectively marked as a camera 1,2,3, and the dishes are photographed from three angles.
The top camera (camera 1) adopts color and shape recognition to complete the pricing of dishes, and solves the problem of container shielding which can occur. The specific implementation method comprises the following steps: firstly, shoot, discernment dish utensil shape and size, discern the colour at dinner plate edge afterwards, effectively solved the problem that food sheltered from the dinner plate colour.
The circle of the dinner plate needs to be captured first and recognized using the hough transform. The basic idea of detecting the circle by Hough transform is to map edge points in an image space into a parameter space, then accumulate and count accumulated values corresponding to all coordinate point elements obtained in the parameter space, and judge the radius of the circle and the position of the circle center according to the accumulated values.
Secondly, color collection is carried out on the edge part of the circle, such as the edge range from 0.7 times of radius to 1 time of radius away from the circle center, and color judgment is carried out according to the mode of the LAB value.
In order to avoid the limitation of a single visual angle, the identification of the beverage is completed by adding the cameras at the side and the back (the cameras 2 and the cameras 3), and meanwhile, the identification accuracy is greatly enhanced by introducing multiple cameras.
Wherein, side look camera module (camera 2) and rear view (camera 3) head module utilize the template matching algorithm to meter to non-dinner plate shape container respectively, if the bowl, beverage and other article are discerned, two camera modules of quadrature have avoided the drink to put the mutual problem of sheltering from that exists in to the at utmost, another camera module can realize observing when one of them camera module can't observe the drink that is sheltered from, accomplished the discernment to article in the three-dimensional space, the realization is accurate to multiple article, the pricing of no omission, later two cameras are uploaded the data of discerning among the STM32 main control panel and are carried out data processing.
The fourth concrete embodiment:
the difference between the fourth embodiment and the third embodiment is only that:
the STM32 singlechip carries out serial communication with the card reader, and when the customer was punched the card the operation, the card reader passed through serial communication and gives the STM32 singlechip with IC-card information transfer, and the STM32 singlechip reads information and returns the price that should deduct according to the price that camera module obtained, carries out the operation of deducting, and the result of deducting shows on the screen.
The fifth concrete embodiment:
the difference between the fifth embodiment and the fourth embodiment is only that:
when the card reader is used for swiping a card, a card number command packet is automatically returned to the STM32 single chip microcomputer and is displayed on a display screen of the STM32 single chip microcomputer after processing, then the STM32 single chip microcomputer sends a money deduction command packet to the card reader according to price information, the balance in the card is changed by the card reader, and balance information is returned to the STM32 single chip microcomputer and is displayed on the display screen after processing. The sixth concrete embodiment:
the difference between the sixth embodiment and the fifth embodiment is only that:
capturing the circle of the dinner plate when settlement is carried out at the edge end, identifying the circle by Hough transform, mapping the edge points in an image space into a parameter space, carrying out accumulation statistics on accumulated values corresponding to all coordinate point elements obtained in the parameter space, and judging the radius of the circle and the position of the circle center according to the accumulated values;
for the circular edge part, when the edge range is from 0.7 times to 1 time of radius away from the circle center, color acquisition is carried out, and color judgment is carried out according to the mode of an LAB value;
the identification of beverages is completed by adding the cameras at the side and the rear, the side view camera module and the rear view camera module respectively utilize a template matching algorithm to meter non-dinner plate-shaped containers, such as bowls and beverage articles, the orthogonal two camera modules avoid the problem of mutual shielding in beverage placement to the maximum extent, when one camera module cannot observe shielded beverages, the other camera module realizes observation, the identification of the articles in a three-dimensional space is completed, accurate and non-missing metering of various articles is realized, and then the two cameras upload the identified data to an STM32 main control board for data processing;
the recognition algorithm which takes NCC algorithm template matching as the main part and takes feature point recognition as the auxiliary part completes recognition and pricing of the intercepted image, and the specific recognition process is as follows: firstly, comparing a gray-scale image with a template by using an NCC (non-uniform correlation) algorithm to obtain a gray-scale image of the image, and completing the identification process of the beverage by using a mode of traversing the template; when the NCC multi-template matching algorithm identification is insufficient, a feature point detection algorithm is started for identification, and the method specifically comprises the following steps:
the method comprises the steps of shooting an image, constructing an n multiplied by n neighborhood as a matching window for any pixel point (px, py) in an original image, and constructing an n multiplied by n domain as the matching window at a target pixel position (px + d, py) in the same way;
and (3) judging the correlation between the two neighborhoods by adopting a normalized cross correlation algorithm:
Figure BDA0003737508010000151
wherein, wp is a matching window, I1 (x, y) is a pixel value of the original image, I1 (px, py) is an average value of pixels in the original window, I2 (x + d, y) is a pixel value of the original image after the corresponding point position on the target image shifts by d in the x direction, and I2 (px + d, py) is an average value of pixels in the matching window of the target image;
the NCC calculation result is between [ -1,1], when NCC = -1, it means that the two matching windows are completely uncorrelated, and conversely, when NCC =1, it means that the correlation degree of the two matching windows is very high;
defining an interested area rectangular element group roi (x, y, w, h), step size step and similarity threshold value threshold, performing correlation calculation on the interested area by using a fixed step value, comparing the step value with the threshold value, completing template matching, and successfully identifying the article;
when the multi-template matching is insufficient, completing feature point matching by calculating the Euclidean distance between the image and the set feature point;
setting up a beverage template with high selection rate in a front setting, setting a beverage with low selection rate in a rear setting, and setting the beverage templates with low selection rate in a rear setting, wherein the traversing sequences of the two cameras are different, so that the speed is increased by means of the total number of the beverages and the jumping-out circulation, and the beverage templates can still meet the actual speed requirement in the identification of a large number of types of beverages;
the data that two camera module discernments obtained can carry out the processing of contrasting with the top camera module when handling data, can reset when appearing discerning the object total number and mismatch and discern again, and the machine will remind customer to adjust the dinner plate and discern again when discerning many times still can't solve the problem.
The seventh specific embodiment:
the seventh embodiment of the present application differs from the sixth embodiment only in that:
the invention provides a diet health recommendation method based on edge cloud integration, which comprises the following steps:
a nutrition analysis algorithm is needed for nutrition analysis of the food, the nutrition taken by the user is judged according to the recent dining condition of the user through the establishment of the nutrition analysis algorithm, and the food with less nutrition taken is recommended;
preprocessing the nutrient information of the dishes, obtaining the nutrient content information characteristics of each dish of similar dishes in the selected restaurant in advance, wherein the nutrient content information characteristics comprise five contents of protein, fat, carbohydrate, sodium and cholesterol, and carrying out standardization and regularization operation;
and (4) clustering dishes in the self-service restaurant according to the characteristic groups based on a K-means algorithm to be used as the basis for analyzing the dishes eaten by the subsequent users.
The eighth embodiment:
the eighth embodiment of the present application differs from the seventh embodiment only in that:
preprocessing the information of the dishes, firstly standardizing the dishes by Min-Max, mapping the nutrition information of the dishes in a [0,1] interval, and representing the nutrition information by the following formula:
Figure BDA0003737508010000161
wherein, a max,j Is the maximum value in the sample over the jth feature; a is min,j Is the minimum value in the sample at the j-th feature;
the nutritional ingredients are regularized and expressed as a percentage by the following formula:
Figure BDA0003737508010000162
and clustering the dishes by adopting a K-means algorithm, wherein the Euclidean distance is represented by the following formula:
Figure BDA0003737508010000163
the method of single-connection clustering calculates the distance of dishes to the center of each cluster, uses the sum of squared errors SSE as a target function, and takes the minimum value:
Figure BDA0003737508010000164
wherein K represents the Kth cluster, C i Denotes the ith center, dist denotes the euclidean distance;
for the kth center C k Solving, the SSE is derived to make the derivative thereof be 0 and C is calculated k The calculation process is as follows:
Figure BDA0003737508010000171
Figure BDA0003737508010000172
analyzing and classifying recently eaten dishes of the user, and comparing the recently eaten dishes with the daily required nutrition standard of the human body proposed by the international union of nutrition science, recommending the corresponding high-nutrition-content dishes with less nutrition intake;
the specific implementation process is as follows:
judging cluster family distribution of dishes eaten by the user for N times, matching the eating condition of the user for N times with the original cluster, and determining that fewer cluster families are involved in corresponding characteristic cluster families in the dishes ingested by the user;
adding the diet information of the dishes eaten for N times, comparing the sum with the daily required nutrition standard of the human body proposed by the international union of nutrition science, if the sum exceeds the standard, reminding, and if the sum does not exceed the standard, recording the cluster family corresponding to the nutrition characteristics with the intake amount greatly different from the standard intake;
and randomly recommending 5-6 dishes in the cluster group according to the obtained 2-3 characteristic cluster groups.
The specific example is nine:
the difference between the ninth embodiment and the eighth embodiment is only that:
the invention provides a computer-readable storage medium, on which a computer program is stored, the program being executed by a processor for implementing a dietary health recommendation method based on edge-cloud integration.
The specific embodiment ten:
the difference between the tenth embodiment and the ninth embodiment is only that:
the invention provides computer equipment which comprises a memory and a processor, wherein a computer program is stored in the memory, and when the processor runs the computer program stored in the memory, the processor executes a diet health recommendation method based on edge cloud integration.
The concrete example eleven:
the eleventh embodiment of the present application differs from the tenth embodiment only in that:
the invention adopts a side-cloud integrated structure, as shown in figure 1, the side-cloud integrated structure comprises two edge ends, namely a self-service settlement platform and user interaction software, wherein the settlement platform is used for completing the catering settlement of a user, and the interaction software is used for completing the interaction with the user, namely the dish recommendation information based on taste and health is transmitted to the user. The cloud serves as a data center and is responsible for data integration and calculation, user diet behavior data are arranged, and recommended dish information is calculated according to a corresponding algorithm. In the examples, the dish of braised pork in brown sauce, tomato scrambled eggs, panned pork, rice, dittany, palace chicken Ding Jichong was used for verification.
The settlement table part of the workflow is shown in fig. 2, and when the user selects the dish, the dish is placed on the settlement table, and the pressure sensor on the settlement table can sense that the dish is placed. And then, a signal is sent to an STM32 single chip microcomputer, the STM32 receives a pressure sensor signal and then controls 3 OpenMV modules to identify dishes on the settlement table, and a traditional machine vision algorithm is used. And the camera above identifies the color and the shape of the dinner plate, counts the distribution of an LAB three-channel histogram in the dinner plate area, and judges the color according to the peak value area.
The two cameras at the side positions identify the drinks, an NCC template matching algorithm is used, a matching window is constructed by using a template, and an NCC value is calculated to realize audio identification, wherein the algorithm identification flow is shown in figure 3.
And after recognition, price calculation is completed, the STM32 controls the RFID card reader to swipe a card for payment, the task of the settlement table part is completed, and then the STM32 transmits the food pictures to the cloud.
The work flow of the cloud is shown in fig. 4, a ResNet152 neural network model trained by using a large number of actual Chinese dish images is carried by the cloud, and specific dishes in the dish pictures are identified based on the neural network model.
And after a specific dish is identified, calculating a recommended dish according to the historical eating behavior data of the user, and updating the eating data of the user. The calculation of recommended dishes relies on two algorithms, dish recommendation and nutrient structure mining.
The nutritional structure mining algorithm clusters the food in the restaurant by adopting a Kmeans algorithm, and then traverses and matches the recent diet condition of the user, and outputs the dishes with the least eating clusters.
The dish recommending algorithm firstly extracts the features of dishes, and then carries out similarity analysis according to the historical dining scoring information of all users, so as to recommend dishes with more agreeable taste to the users.
And after the cloud function is completed, pushing the information to the WeChat small program to interact with the user, and displaying the food records and dish recommendation information in the small program.
Recommendation algorithm based on user taste
The proposal of the diet proposal of the user needs to utilize a dish recommendation algorithm. The method mainly comprises the steps of analyzing the catering data of the user, extracting features, analyzing the similarity of the features, and finally finding out two people with similar hobbies to recommend each other.
Firstly, selecting seven-dimensional characteristics for extraction, wherein the characteristics are respectively vegetable series, degree of greasiness, meat and vegetable matching, food materials, seasoning, preparation process and price. After the user has used up the dishes, the dishes can be scored according to the seven aspects through the APP, and the data are transmitted to the background.
And then, carrying out similarity analysis after feature extraction, and adopting a common decision tree and a random forest algorithm.
Calculating a random forest:
[1] importing data, conditions and results thereof;
[2] setting a constant N as the number of screened samples;
[3] setting a constant a as the number of screening conditions;
[4] setting a constant X as the number of decision trees and creating X decision trees;
[5] after the creation is finished, the test sample is brought in, the result is subjected to majority or average value as the final test result, and the final test result is compared with the real result to judge whether the result is in accordance with the reality.
And finally, through simulation of a random forest algorithm, the scores of the user on other dishes can be predicted, dishes in the dish library are continuously adjusted and judged, the recommendation results are classified by applying cluster analysis, the classification number is applied to the LDA topic model, and finally 10 recommended dishes can be output.
Settlement, identification and recommendation integrated service for the optional restaurant based on edge cloud cooperation.
In the invention, a system is formed in a side-cloud mode, the edge settlement part mainly pursues real-time performance and has high requirement on processing speed, and the whole process of analysis after the edge settlement part is transmitted into a cloud end is too slow and needs networking, so that the system is not suitable for practical application scenes; the cloud deploys an intelligent deep learning method to perform large-scale dish type identification and nutrition analysis processing, so that the calculation efficiency is effectively improved, and meanwhile, the intelligent deep learning method has the intelligent service capability. By adopting the edge cloud cooperation system architecture, the edge end can be used for rapidly settling accounts, and the cloud end provides personalized services for guests according to the food images and the historical data.
And multi-angle image acquisition and real-time quick settlement based on machine vision.
The image acquisition device arranged at the edge end is different from a single-camera single-view angle mode in the traditional visual detection, and the three camera devices are adopted for carrying out characteristic acquisition on the put-in object from multiple angles so as to acquire the multi-angle information of the target object. Compared with the existing single-camera system of partial products, the system can provide more accurate and comprehensive visible light image information of catering targets for multi-source machine vision fusion, and simultaneously well meets the requirement of high-speed settlement at the edge end. The invention not only can identify the dinner plate with relatively rule through the characteristics of color, size and shape, but also can identify other common objects in the catering industry such as beverage, wine, paper towel, fruit and the like through template matching, thereby eliminating the limitation that the traditional self-service settlement system can only identify settlement fee by tableware. Specifically, the invention obtains color, size and shape information through the upper camera, and charges the dinner plate; meanwhile, the upper camera acquires the characteristics of the dinner plate such as color, shape, size and the like and provides the quantity information of the dinner plate and the drinking object; the side camera and the rear camera carry out charging on other types of objects through template matching; the information is then aggregated by the control hub.
The cloud data sharing and analyzing system is constructed by utilizing the visual collection of the front-end machine and continuously updating the cloud sample base through the NB _ IoT technology, and data analysis and utilization are realized for a restaurant at the cloud. The cloud deep learning algorithm can efficiently and accurately identify the type of the catering target, meanwhile, large-scale data analysis and calculation are carried out by utilizing nutrition analysis, dish recommendation algorithm and the like based on the dish data of the cloud, and personalized catering recommendation service is provided for the user. By applying the large-scale historical dining data, data waste is avoided.
The above description is only a preferred embodiment of the diet health analysis system based on edge cloud integration, and the protection scope of the diet health analysis system based on edge cloud integration is not limited to the above embodiments, and all technical solutions belonging to the idea belong to the protection scope of the present invention. It should be noted that modifications and variations which do not depart from the gist of the invention will be those skilled in the art to which the invention pertains and which are intended to be within the scope of the invention.

Claims (10)

1. A diet health analysis system based on edge cloud integration is characterized in that: the system comprises:
the system comprises an image information acquisition end, a pressure sensor and three camera modules, wherein the image information acquisition end acquires food image information from three angles by using a camera, and the edge end controls the pressure sensor and the three camera modules to acquire food images from three angles;
the main controller intelligently identifies pricing, the price of dishes is determined by the shape and the color of the dinner plate, the calculation of the price is completed by identifying the color and the shape of the dinner plate through the main controller, and the main controller controls the RFID card reader module to complete fee deduction;
the edge end transmits the acquired food image data to the cloud end, the cloud end runs a trained ResNet152 neural network model to identify the received food image, stores the data, analyzes the nutritional structure and taste of the user in a targeted manner by using the stored historical data, records the preference of the user and recommends the food;
and the side cloud interaction end and the cloud end are the data center, receive the image of the edge section settlement table, and transmit the analysis data to the user side WeChat small program after the analysis is completed.
2. The system of claim 1, wherein the system comprises: the main controller adopts an STM32 single chip microcomputer, and the STM32 single chip microcomputer receives data of the pressure sensor and triggers the functions of dinner plate identification and pricing settlement; when a dinner plate is placed in the device, the pressure sensor is triggered, a trigger signal is transmitted to the STM32 single chip microcomputer through serial port communication, the STM32 single chip microcomputer sends trigger information to the three camera modules through the serial port communication, and the camera modules are triggered to take photos; when a dinner plate is placed next time, the sensor is triggered again to control the camera module to work.
3. The system of claim 2, wherein the system comprises: the STM32 singlechip carries out serial communication with the camera module, and three camera module is after three angle is discerned the dinner plate respectively, calculates three price and to STM32 singlechip transmission price, and STM32 receives the price and obtains final commodity price according to the pricing strategy, then shows final price on the screen.
4. The system of claim 3, wherein the system comprises: the STM32 singlechip carries out serial communication with the card reader, and when the customer was punched the card the operation, the card reader passed through serial communication and gives the STM32 singlechip with IC-card information transfer, and the STM32 singlechip reads information and returns the price that should deduct according to the price that the camera module obtained, deducts the operation of paying, deducts the result and shows on the screen.
5. The system for analyzing the dietary health based on the integration of the cloud and the side cloud as claimed in claim 4, wherein: by adopting the UART serial port communication IC card reader of the M3650A-HA model, when the card reader swipes a card, a card number command packet is automatically returned to the STM32 single chip microcomputer and is displayed on a display screen of the STM32 single chip microcomputer after processing, then the STM32 single chip microcomputer sends a money deduction command packet to the card reader according to price information, the card reader changes the balance in the card, and the balance information is returned to the STM32 single chip microcomputer and is displayed on the display screen after processing.
6. The system of claim 5, wherein the system comprises: when settlement is carried out at an edge end, capturing the circle of the dinner plate, identifying the circle by Hough transform, mapping an edge point in an image space into a parameter space, carrying out accumulation statistics on accumulated values corresponding to all coordinate point elements obtained in the parameter space, and judging the radius of the circle and the position of the circle center according to the accumulated values;
for the circular edge part, when the edge range is from 0.7 times to 1 time of radius away from the circle center, color acquisition is carried out, and color judgment is carried out according to the mode of an LAB value;
the identification of beverages is completed by adding the cameras at the side and the rear, the side view camera module and the rear view camera module respectively utilize a template matching algorithm to meter non-dinner plate-shaped containers, such as bowls and beverage articles, the orthogonal two camera modules avoid the problem of mutual shielding in beverage placement to the maximum extent, when one camera module cannot observe shielded beverages, the other camera module realizes observation, the identification of articles in a three-dimensional space is completed, accurate and non-missing metering of various articles is realized, and then the two cameras upload the identified data to an STM32 main control board for data processing;
the recognition algorithm with NCC algorithm template matching as the main part and feature point recognition as the auxiliary part finishes the recognition and the pricing of the intercepted image, and the specific recognition process is as follows: firstly, comparing a gray-scale image with a template by using an NCC algorithm to obtain a gray-scale image of the image, and completing the identification process of the beverage by using a mode of traversing the template; when the NCC multi-template matching algorithm identification is insufficient, a feature point detection algorithm is started for identification, and the method specifically comprises the following steps:
the method comprises the steps of shooting an image, constructing an n multiplied by n neighborhood as a matching window for any pixel point (px, py) in an original image, and constructing an n multiplied by n neighborhood as a matching window at a target pixel position (px + d, py) in the same way;
and (3) judging the correlation between the two neighborhoods by adopting a normalized cross correlation algorithm:
Figure FDA0003737506000000031
wherein, wp is a matching window, I1 (x, y) is a pixel value of an original image, I1 (px, py) is an average value of pixels in the original window, I2 (x + d, y) is a pixel value of the original image after a corresponding point position on a target image shifts by d in the x direction, and I2 (px + d, py) is a pixel average value of the target image matching window;
the NCC calculation results are between [ -1,1], when NCC = -1, it means that the two matching windows are completely uncorrelated, and conversely, when NCC =1, it means that the degree of correlation of the two matching windows is very high;
defining a region-of-interest rectangular tuple roi (x, y, w, h), step size step and similarity threshold, performing correlation calculation on the region-of-interest by using a fixed step value, comparing the correlation calculation with the threshold, completing template matching, and successfully identifying the article;
when the multi-template matching is insufficient, completing feature point matching by calculating the Euclidean distance between the image and the set feature point;
setting up a beverage template with high selection rate in a front setting, setting a beverage with low selection rate in a rear setting, and setting the beverage templates with low selection rate in a rear setting, wherein the traversing sequences of the two cameras are different, so that the speed is increased by means of the total number of the beverages and the jumping-out circulation, and the beverage templates can still meet the actual speed requirement in the identification of a large number of types of beverages;
the data that two camera module discernments obtained can contrast with top camera module and handle when handling data, can reset and discern again when discernment object total number appears not matching, and the machine will remind customer to adjust the dinner plate and discern again when discernment still can't solve the problem many times.
7. A diet health recommendation method based on edge cloud integration is characterized by comprising the following steps: the method comprises the following steps:
a nutrition analysis algorithm is needed for nutrition analysis of the food, the nutrition taken by the user is judged according to the recent dining condition of the user through the establishment of the nutrition analysis algorithm, and the food with less nutrition taken is recommended;
preprocessing the nutrient information of the dishes, obtaining the nutrient content information characteristics of each dish of similar dishes in the selected restaurant in advance, wherein the nutrient content information characteristics comprise five contents of protein, fat, carbohydrate, sodium and cholesterol, and carrying out standardization and regularization operation;
and (4) clustering dishes in the self-service restaurant according to the characteristic groups based on a K-means algorithm to be used as the basis for analyzing the dishes eaten by the subsequent users.
8. The diet health recommendation method based on the edge cloud integration as claimed in claim 7, wherein:
preprocessing the information of the dishes, firstly standardizing the dishes by Min-Max, mapping the nutrition information of the dishes in a [0,1] interval, and expressing the nutrition information by the following formula:
Figure FDA0003737506000000041
wherein, a max,j Is the maximum value in the sample over the jth feature; a is min,j Is the minimum value in the sample at the j-th feature;
the nutritional ingredients are regularized and expressed as a percentage by the following formula:
Figure FDA0003737506000000042
and clustering the dishes by adopting a K-means algorithm, wherein the Euclidean distance is represented by the following formula:
Figure FDA0003737506000000043
the method of single-connection clustering calculates the distance of dishes to the center of each cluster, uses the sum of squared errors SSE as a target function, and takes the minimum value:
Figure FDA0003737506000000051
wherein K represents the Kth cluster, C i Denotes the ith center, dist denotes the euclidean distance;
for the k center C k Solving, the SSE is derived to make the derivative 0 and C is calculated k The calculation process is as follows：
Figure FDA0003737506000000052
Figure FDA0003737506000000053
Analyzing and classifying recently eaten dishes of the user, and comparing the recently eaten dishes with the daily required nutritional standard of the human body proposed by the international union of nutrition science, recommending the corresponding high-nutrient-content dishes with less nutrient intake;
the specific implementation process is as follows:
judging cluster family distribution of dishes eaten by the user for N times, matching the eating condition of the user for N times with the original cluster, and determining that fewer cluster families are involved in corresponding characteristic cluster families in the dishes ingested by the user;
adding the diet information of the dishes eaten for N times, comparing the sum with a human body daily required nutrition standard provided by the International Union of Nutrition science, if the sum exceeds the standard, reminding, and if the sum does not exceed the standard, recording a cluster group corresponding to the nutrition characteristic with the intake amount greatly different from the standard intake;
and randomly recommending 5-6 dishes in the cluster family according to the obtained 2-3 characteristic cluster families.
9. A computer-readable storage medium, on which a computer program is stored, the program being executable by a processor for implementing a cloud-based dietary health recommendation method according to claims 7-8.
10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor performs a diet health recommendation method based on edge cloud integration according to 7-8.
CN202210800770.9A 2022-07-08 2022-07-08 Diet health analysis system based on edge cloud integration Pending CN115330376A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210800770.9A CN115330376A (en) 2022-07-08 2022-07-08 Diet health analysis system based on edge cloud integration

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210800770.9A CN115330376A (en) 2022-07-08 2022-07-08 Diet health analysis system based on edge cloud integration

Publications (1)

Publication Number Publication Date
CN115330376A true CN115330376A (en) 2022-11-11

Family

ID=83917827

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210800770.9A Pending CN115330376A (en) 2022-07-08 2022-07-08 Diet health analysis system based on edge cloud integration

Country Status (1)

Country Link
CN (1) CN115330376A (en)

Similar Documents

Publication Publication Date Title
CN103208156B (en) A kind of food and drink Automated Clearing House system and method
WO2019228063A1 (en) Product inspection terminal, method and system, computer apparatus and readable medium
CN110059654A (en) A kind of vegetable Automatic-settlement and healthy diet management method based on fine granularity identification
CN107767590A (en) Automatic identification commercialization bar code electronic scale and Automatic identification method
CN111080493B (en) Dish information identification method and device and dish self-service settlement system
CN108875669B (en) Commodity identification technology based on visible light and infrared image fusion
CN108537166B (en) Method and device for determining shelf browsing amount and analyzing browsing amount
CN101477729A (en) Self-help meal sale system and information processing method of the system
CN112329866A (en) Intelligent healthy ordering system and method for restaurant
CN106022405A (en) Dinner plate automatic pricing system based on shape
CN103942569A (en) Chinese style dish recognition device based on computer vision
CN112037440A (en) Intelligent settlement system and settlement method based on machine learning
WO2018162297A1 (en) System, method and computer program
CN105976407B (en) A kind of service plate shape recognition process suitable under obstruction conditions
JP7376489B2 (en) Methods and systems for classifying foods
CN105741139A (en) Automatic charging method for self-service restaurant and implementation apparatus thereof
RU2724797C1 (en) Cash register system and method for identification of courses on tray
US11562338B2 (en) Automated point of sale systems and methods
CN105809587A (en) Intelligent cash receiving method and system
EP3964793A1 (en) Food measurement method, device, and program
CN115330376A (en) Diet health analysis system based on edge cloud integration
CN109858448A (en) Item identification method and equipment under a kind of public safety
CN115187972A (en) Dish identification method based on feature comparison
US20220222844A1 (en) Method, device, and program for measuring food
CN113450505B (en) Temperature control method and device of self-service vending machine

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination