CN110966835B - Method for detecting nutrient components of food materials in refrigerator and refrigerator - Google Patents
Method for detecting nutrient components of food materials in refrigerator and refrigerator Download PDFInfo
- Publication number
- CN110966835B CN110966835B CN201811163253.5A CN201811163253A CN110966835B CN 110966835 B CN110966835 B CN 110966835B CN 201811163253 A CN201811163253 A CN 201811163253A CN 110966835 B CN110966835 B CN 110966835B
- Authority
- CN
- China
- Prior art keywords
- refrigerator
- data
- food material
- hyperspectral
- imaging device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
- F25—REFRIGERATION OR COOLING; COMBINED HEATING AND REFRIGERATION SYSTEMS; HEAT PUMP SYSTEMS; MANUFACTURE OR STORAGE OF ICE; LIQUEFACTION SOLIDIFICATION OF GASES
- F25D—REFRIGERATORS; COLD ROOMS; ICE-BOXES; COOLING OR FREEZING APPARATUS NOT OTHERWISE PROVIDED FOR
- F25D23/00—General constructional features
- F25D23/12—Arrangements of compartments additional to cooling compartments; Combinations of refrigerators with other equipment, e.g. stove
-
- F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
- F25—REFRIGERATION OR COOLING; COMBINED HEATING AND REFRIGERATION SYSTEMS; HEAT PUMP SYSTEMS; MANUFACTURE OR STORAGE OF ICE; LIQUEFACTION SOLIDIFICATION OF GASES
- F25D—REFRIGERATORS; COLD ROOMS; ICE-BOXES; COOLING OR FREEZING APPARATUS NOT OTHERWISE PROVIDED FOR
- F25D29/00—Arrangement or mounting of control or safety devices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
- F25—REFRIGERATION OR COOLING; COMBINED HEATING AND REFRIGERATION SYSTEMS; HEAT PUMP SYSTEMS; MANUFACTURE OR STORAGE OF ICE; LIQUEFACTION SOLIDIFICATION OF GASES
- F25D—REFRIGERATORS; COLD ROOMS; ICE-BOXES; COOLING OR FREEZING APPARATUS NOT OTHERWISE PROVIDED FOR
- F25D2500/00—Problems to be solved
- F25D2500/06—Stock management
Abstract
The invention provides a method for detecting nutrient components of food materials in a refrigerator and the refrigerator. Wherein the storage compartment of the refrigerator is internally provided with a hyperspectral imaging device. The method for detecting the information of the food materials in the refrigerator comprises the following steps: acquiring hyperspectral data of the food material shot by a hyperspectral imaging device; respectively extracting image data and spectrum data from the hyperspectral data; detecting the type of food materials according to the extracted image data, acquiring a nutrient component detection model corresponding to the type of the food materials, and inputting the extracted spectral data into the nutrient component detection model to obtain the freshness and nutrient component information of the food materials in the refrigerator. The food material management is convenient for the user.
Description
Technical Field
The invention relates to the technical field of storage, in particular to a method for detecting nutrient components of food materials in a refrigerator and the refrigerator.
Background
With the progress of society and the improvement of living standard of people, consumers pay attention to the nutritional value and safety of food when purchasing the food, and also consider factors such as price, taste, appearance, freshness and the like, and the role of the refrigerator is gradually changed from pure storage and preservation to a food material management center and a family nutrition center, which also provides a new challenge for the refrigerator, and meanwhile, provides a chance for various intelligent identification technologies to be applied to the refrigerator. The method for storing food material types in the refrigerator is changed from opening the refrigerator door to actually checking the food material types into intelligent identification. The realization of the food type recognition function on a household refrigerator by using an automatic recognition technology has become a development trend of an intelligent refrigerator.
The automatic identification technology is a technology which applies a specific identification device, enables an identified article to approach the identification device, automatically obtains relevant information of the identified article, and provides the relevant information to a computer processing system to complete relevant subsequent processing. The technology for automatically identifying the refrigerator at present comprises radio frequency identification, image identification and the like, wherein radio frequency identification is realized by pasting a radio frequency identification code on food materials put in the refrigerator and identifying the food materials by using a radio frequency identification device arranged on the refrigerator. The image recognition technology is also applied to refrigerators, but the correct recognition rate is low, and since the technology mainly relies on the recognition of different colors, shapes and textures of food materials, the correct recognition of food materials with similar colors and shapes is difficult, and the detection of the nutritional ingredients of the food materials cannot be realized.
For the existing detection of the nutrient components of food materials, the existing method is to use different instruments such as a glucometer and a hardness meter to respectively detect each physicochemical index so as to obtain the total nutrient component information of food.
The fast detection means, namely the spectrum method, which is gradually developed provides a fast detection means, and the spectrum information is related to the chemical composition and the molecular structure of the substance to be detected, so that the nutrient component information of the food material can be accurately carried out. When the food materials to be detected are identified, firstly, the spectrum information of the food materials to be detected is obtained, then the characteristic spectrum information which can represent the food materials is extracted, a series of food materials and the respective characteristic spectrum information are modeled, and then the nutrient components of the food materials to be detected can be detected. However, the method generally performs single-point detection, and the user experience is not good; meanwhile, the area of the detected area is small, and the result is often inconsistent when a certain food material is subjected to multi-point detection, so that the accuracy of the technology needs to be further improved.
Disclosure of Invention
The invention aims to provide a method for automatically detecting the nutrient content of food materials.
It is a further object of the present invention to improve the accuracy of the detection of nutritional ingredients.
The invention firstly provides a method for detecting the nutritional ingredients of food materials in a refrigerator, which is suitable for a hyperspectral imaging device for shooting the food materials in a storage chamber of the refrigerator, and comprises the following steps: after an instruction for identifying the nutritional ingredients of the food materials is received, starting a hyperspectral imaging device, and acquiring hyperspectral data obtained by shooting; preprocessing the hyperspectral data, and respectively extracting image data and spectral data; identifying the type of the food material according to the image data; acquiring a corresponding nutritional component detection model according to the variety of food materials, wherein the nutritional component detection model is obtained by training according to the spectral data of the food materials with different qualities in advance; and carrying out classification analysis calculation on the spectral data by using a nutrient content detection model to obtain nutrient content information of each pixel point of the food material, and carrying out comprehensive calculation on the nutrient content information of each pixel point of the food material so as to determine the nutrient content of the food material.
Optionally, the step of identifying the type of food material according to the image data comprises: acquiring image data; identifying the food material type according to the image data to obtain first type information and a first identification rate; judging whether the first identification rate is higher than a preset first threshold value or not; if yes, determining the type of the food material.
Optionally, the step of classifying the spectral data captured by the hyperspectral imaging device by using the nutrient content detection model comprises: extracting spectral data required by the nutrient component detection model from hyperspectral data shot by the hyperspectral imaging device; inputting the spectral data required by the nutrient component detection model into the nutrient component detection model; and carrying out pattern recognition by the nutrient component detection model to obtain the nutrient components of the food material.
Optionally, the hyperspectral data includes a set number of ternary data sets, each ternary data set includes two image pixel elements and one spectral wavelength element of one pixel point, each pixel point has a plurality of sets of ternary data sets, the image data is obtained by analyzing and extracting data in the image pixel elements, and the spectral data is obtained by analyzing and extracting data in the spectral wavelength elements.
Optionally, the resolution of the spectral wavelength of each pixel in the hyperspectral data is less than or equal to 2nm.
Optionally, in the process of starting the hyperspectral imaging device, a light source system matched with the hyperspectral imaging device is also started simultaneously to provide light required by the hyperspectral imaging device for shooting, wherein the spectral range of the light source system is 400nm to 1100nm.
Optionally, the step of determining the nutritional ingredients of the food material further comprises: and outputting the nutritional ingredients of the food materials through a display screen of the refrigerator or a mobile terminal bound with the refrigerator.
According to another aspect of the invention, a refrigerator is also provided. The refrigerator includes: the refrigerator comprises a refrigerator body, a storage compartment and a storage box, wherein the refrigerator body is internally limited with the storage compartment; the hyperspectral imaging device is arranged in the storage room and is configured to shoot food materials in the storage room; the controller comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the controller is used for realizing the method for detecting the nutrient content of the food in the refrigerator.
Optionally, the refrigerator further comprises: and the light source system is matched with the hyperspectral imaging device, is arranged in the storage room and is configured to provide light required by shooting for the hyperspectral imaging device, and the spectral range of the light source system is 400nm to 1100nm.
Optionally, the refrigerator further comprises: and an information output interface configured to provide the food material category to a display screen of the refrigerator or a mobile terminal bound with the refrigerator for output to a user.
According to the method for detecting the nutritional ingredients of the food materials in the refrigerator and the refrigerator, the hyperspectral imaging device is arranged in the refrigerator, hyperspectral data of the food materials are obtained by shooting, the hyperspectral data are used for detecting the nutritional ingredients of the food materials, the detection accuracy is high, and the requirement for quickly and nondestructively obtaining the nutritional ingredients is met.
Furthermore, the method for detecting the nutritional ingredients of the food materials in the refrigerator and the refrigerator provided by the invention utilize the characteristics of the spectral information of the food materials and the nutritional ingredients thereof, adopt a pattern recognition technology, and detect the nutritional ingredients of the food materials by means of a nutritional ingredient detection model, so that the accuracy of detection is obviously improved, and the food materials can be conveniently managed by a user.
The above and other objects, advantages and features of the present invention will become more apparent to those skilled in the art from the following detailed description of specific embodiments thereof, taken in conjunction with the accompanying drawings.
Drawings
Some specific embodiments of the invention will be described in detail hereinafter, by way of illustration and not limitation, with reference to the accompanying drawings. The same reference numbers in the drawings identify the same or similar elements or components. Those skilled in the art will appreciate that the drawings are not necessarily drawn to scale. In the drawings:
fig. 1 is a schematic view of a refrigerator according to one embodiment of the present invention;
FIG. 2 is a functional schematic block diagram of a refrigerator according to one embodiment of the present invention;
fig. 3 is a schematic view of a refrigerator according to another embodiment of the present invention;
FIG. 4 is a schematic diagram of a method of identifying a category of food material in a refrigerator according to one embodiment of the present invention; and
fig. 5 is a schematic view of a method of detecting nutritional components of food materials in a refrigerator according to one embodiment of the present invention.
Detailed Description
Fig. 1 is a schematic diagram of a refrigerator 10 according to one embodiment of the present invention. The refrigerator 10 of the present embodiment may generally include: the device comprises a box body 110, a door body 120 and a hyperspectral imaging device 210.
The cabinet 110 defines at least one storage compartment 130, typically a plurality of compartments, such as a refrigerating compartment, a freezing compartment, a temperature-changing compartment, and the like, having an open front side therein. The number and functions of the storage compartments 130 can be configured according to the predetermined requirement, and in some embodiments, the storage temperature of the refrigerating compartment can be 2 to 9 ℃, or 4 to 7 ℃; the storage temperature of the freezing chamber can be-22 to-14 ℃, or can be-20 to 16 ℃. The freezing chamber is arranged below the refrigerating chamber, and the temperature-changing chamber is arranged between the freezing chamber and the refrigerating chamber. The temperature in the freezer compartment is typically in the range of-14 ℃ to-22 ℃. The temperature-changing chamber can be adjusted according to the requirements to store suitable food materials or be used as a fresh-keeping storage chamber.
And a door 120 provided at the front side of the cabinet 110 to open and close the storage compartment 130. For example, the door bodies 120 may be hingedly disposed at one side of the front portion of the cabinet 110, and the storage compartments 130 may be opened and closed by pivoting, and the number of the door bodies 120 may be matched with the number of the storage compartments 130, so that the storage compartments 130 may be individually opened one by one. For example, a refrigerating chamber door body, a freezing chamber door body and a temperature changing chamber door body can be respectively arranged for the refrigerating chamber, the freezing chamber and the temperature changing chamber. In some alternative embodiments, the door 120 may also be in the form of a vertical hinged door, a side-sliding door, a sliding door, or the like.
The storage chamber 130 is provided with cold energy by a refrigerating system so as to realize storage environments of refrigeration, freezing and temperature changing. The refrigeration system may be a refrigeration cycle system constituted by a compressor, a condenser, a throttle device, an evaporator, and the like. The evaporator is configured to provide cooling directly or indirectly into the storage compartment 130. For example, in a compression type direct cooling refrigerator, the evaporator can be arranged on the outer side or the inner side of the rear wall surface of the inner container of the refrigerator. In the compression type air-cooled refrigerator, the cabinet 110 further has an evaporator chamber therein, the evaporator chamber is communicated with the storage compartment 130 through an air path system, an evaporator is provided in the evaporator chamber, and a fan is provided at an outlet thereof to perform circulating refrigeration to the storage compartment 130. Since the box 110, the door 120, and the refrigeration system themselves are well known and easy to implement by those skilled in the art, the details of the box 110, the door 120, and the refrigeration system themselves are not described herein after in order to not obscure and obscure the invention of the present application.
The hyperspectral imaging device 210 is disposed in the storage compartment 130 of the refrigerator 10, and is configured to capture food materials inside the storage compartment 130 and output hyperspectral data.
The hyperspectral data can be a series of ternary data sets, each ternary data set comprises two image pixel elements and a spectral wavelength element of a pixel point, and each pixel point is provided with a plurality of sets of ternary data sets. Therefore, the hyperspectral data simultaneously obtain the continuous spectrum data of each pixel point and the continuous image data of each spectrum wave band. And the spectral data is obtained by analyzing and extracting data in the pixel elements of the image, and the spectral data is obtained by analyzing and extracting data in the wavelength elements of the spectrum. The hyperspectral image is an optical image with continuous wavelength, the spectral range can be set to be 200nm to 2500nm, the hyperspectral image has higher spectral resolution, and the resolution can reach 2-3 nm. The hyperspectral data may be represented by a three-dimensional data block, where two dimensions are image pixel information (x, y) and the third dimension is wavelength information (λ). The data cube obtained at n wavelengths by an image detector array with a resolution of x y pixels is a three-dimensional array of x y x λ.
In the present embodiment, it is preferable to use the spectral data in the spectral range of 400nm to 1100nm, since the detection of the information on the type of the food material 300 and the nutritional composition is facilitated by a large amount of research on the spectral data in the above spectral range. The resolution requirement of the spectral wavelength of each pixel point in the hyperspectral data shot by the hyperspectral imaging device 210 is less than or equal to 2nm, so that the requirement is met.
The nutrient component detection model can be obtained by training hyperspectral data of a large number of food materials with different qualities, and the training algorithm which can be adopted by the nutrient component detection model can comprise a BP neural network, a Support Vector Machine (SVM) and Adaboost. Various different nutrient component detection models can be trained in advance according to the types of food materials, for example, corresponding nutrient component detection models can be trained for various meats, various fruits and various vegetables.
When performing species detection, the following steps may be performed: extracting image data required by a species detection model from hyperspectral data shot by the hyperspectral imaging device 210, and identifying the species of the food material 300 according to the image data to obtain first species information and a first identification rate; judging whether the first identification rate is higher than a preset first threshold value or not; if so, the type of the food material 300 is determined. An alternative embodiment is: the first threshold is set to 90%, and if the calculated first recognition rate is higher than 90%, the kind of the food material 300 is determined.
When nutrient component detection is carried out, the following steps can be carried out: acquiring a nutrient component detection model corresponding to the type of the food material 300, extracting spectral data required by the nutrient component detection model from the hyperspectral data shot by the hyperspectral imaging device 210, inputting the spectral data into the nutrient component detection model, and performing mode detection by the nutrient component detection model respectively to obtain the nutrient components of the food material 300.
The nutrient content detection can also be realized by means of a cloud technology, for example, after the hyperspectral data shot by the hyperspectral imaging device 210 is acquired, the data processing device of the refrigerator 10 performs primary processing, the hyperspectral data after the primary processing is uploaded to the cloud, the mode detection step of the nutrient content detection model is completed by the cloud, and then the nutrient content of the food material 300 is provided for the refrigerator 10 or the mobile terminal bound with the refrigerator 10 so as to be provided for a user. The nutrient content detection model is stored in the cloud, reducing the data processing pressure of the refrigerator 10.
The nutritional ingredients can reflect the information of moisture, sugar degree, soluble solid, etc. of the food materials. The interaction with the user may be implemented by a human-machine interaction system of the refrigerator 10, for example, outputting information of the kind of the food material 300 and the nutritional ingredients on a display screen of the refrigerator 10. In another embodiment, a message including information of the kind of the food material 300 and the nutritional ingredients may be transmitted to a mobile terminal bound to the refrigerator 10, and a message fed back by the user through the mobile terminal may be received.
Fig. 2 is a functional schematic block diagram of a refrigerator 10 according to another embodiment of the present invention. The following components can be flexibly selected and added to the refrigerator 10 of this embodiment: controller 270, light source system 230, information output interface 250.
A controller 270, disposed in the refrigerator 10, configured to extract image data and spectrum data from the hyperspectral data captured by the hyperspectral imaging apparatus 210, detect a type of the food material 300 according to the extracted image data, and obtain a nutritional component detection model corresponding to the type of the food material 300, where the nutritional component detection model is obtained by training in advance according to spectrum data of food materials of different qualities; classifying, analyzing and calculating the spectral data by using a nutritional component detection model so as to determine the nutritional components of the food material 300; the controller 270 includes a memory 271 and a processor 272, a computer program is stored in the memory 271, and the processor 272 is configured to execute the computer program in the memory 271, and the computer program is used to implement the method for detecting the nutritional ingredients of the food in the refrigerator in this embodiment.
The light source system 230 is disposed in the storage compartment 130 and configured to provide light required for photographing to the hyperspectral imaging device 210, wherein the spectral range of the light source system 230 is set to be 400-1100 nm. The light source system 230 may be disposed at the rear portion of the top wall of the storage compartment 130 to provide a photographing light in an oblique downward direction. The light source system 230 may be activated simultaneously with the hyperspectral imaging apparatus 210 to provide light to the enclosed storage compartment 130.
The information output interface 250 may be configured to provide the kind of the food material 300 and the nutritional ingredient information to a display screen of the refrigerator or a mobile terminal bound to the refrigerator to be output to a user.
In order to ensure that the hyperspectral imaging device 210 can photograph the full view of the food material 300 placed in the storage compartment 130. The hyperspectral imaging device 210 preferably uses a wide-angle lens or a fisheye lens, and is disposed directly above the storage compartment 130.
Fig. 3 is a schematic view of a refrigerator 10 according to another embodiment of the present invention. In the refrigerator 10 of this example, aiming at the problem that the internal space of the refrigerator 10 is narrow and small, the hyperspectral imaging device 210 is difficult to shoot the overall appearance of the storage room 130, and by arranging the reflector 260, hyperspectral data reflecting the overall appearance of the storage room 130 is obtained by means of shooting a reflection image.
The reflector 260 and the hyperspectral imaging device 210 are oppositely arranged inside the storage compartment 130. The hyperspectral imaging apparatus 210 may be configured to photograph the mirror 260 to obtain hyperspectral data of an image reflected by the mirror 260. Since the space inside the refrigerator 10 is narrow and the storage compartment 130 is generally a flat layered structure for facilitating storage, in such a flat region where the space is narrow and narrow, the conventional hyperspectral imaging device 210 is difficult to photograph the entire appearance of the storage compartment 130, and therefore in this embodiment, the problem can be effectively solved by photographing the reflected image of the reflector 260. In some alternative embodiments, the reflector 260 may optionally be a convex mirror to reflect the entire storage compartment 130.
The mirror 260 is disposed on, for example, a top wall of the storage compartment, and the hyperspectral imaging apparatus 210 is disposed in, for example, a bottom wall of the storage compartment. The region where the hyperspectral imaging device 210 is can be set as a blank region, so that the user is prevented from placing the food material 300 to be detected above the hyperspectral imaging device 210 and shielding the lens.
No matter the hyperspectral imaging device 210 adopts an angle lens or a fisheye lens, or adopts the reflection mode of the reflector 260, the hyperspectral imaging device can obtain hyperspectral data reflecting the overall appearance of the storage chamber 130, so that the requirement of shooting the food material 300 is met.
When using the information detection function of the refrigerator 10, a specific example is: after a user places an apple in the storage compartment 130, the user issues a detection instruction through a button or a mobile terminal on the refrigerator 10. The hyperspectral imaging device 210 shoots the storage room 130 to obtain hyperspectral data including apples. From the hyperspectral data, image data and spectrum data of the food material 300 (apple) can be extracted, pattern recognition is carried out on the image data through a food material type recognition model, and first type information and a first recognition rate are obtained; judging whether the first identification rate is higher than a preset first threshold value or not; if so, the type of the food material 300 is determined. Acquiring nutrient component detection models corresponding to the types of the food materials 300, wherein the nutrient component detection models are obtained by training in advance according to spectral data of food materials with different qualities; and carrying out classification analysis calculation on the spectral data by using a nutritional component detection model so as to determine the nutritional components of the food materials.
In addition, the type detection function of the refrigerator 10 may be automatically started at regular time intervals to periodically detect the type of the food material and the nutritional component information in the storage compartment 130.
By using the detection result, a storage file of the types of the food materials 300 and the nutritional ingredient information in the refrigerator 10 can be further established, the types of the food materials 300 and the nutritional ingredient storage information are recorded, and a data base is provided for intelligent management of the food materials.
The present embodiment further provides a method for detecting the type of food material 300 in the refrigerator 10, which can be used in the refrigerator 10 of any of the above embodiments to detect the type information of the food material in the storage compartment 130 inside the refrigerator 10.
Fig. 4 is a schematic diagram of a method for detecting the type of food material in the refrigerator 10 according to an embodiment of the present invention. The method of detecting the kind of food material in the refrigerator 10 may generally include:
step S402, acquiring hyperspectral data captured by the hyperspectral imaging apparatus 210;
step S404, preprocessing the hyperspectral data and extracting image data;
step S406, identifying the type of the food material 300 according to the image data to obtain first type information and a first identification rate;
step S408, judging whether the first identification rate is higher than a preset first threshold value; if yes, go to step S412; if not, steps S410 to S412 are executed.
Step S410, outputting failure prompt information for identifying the type of the food material, and acquiring the type information of the food material input by a user;
in step S412, the type information of the food material is determined.
In step S402, the hyperspectral data may include a set number of ternary data sets, each ternary data set includes two image pixel elements and one spectral wavelength element of one pixel, each pixel has multiple sets of ternary data sets, and the resolution of the spectral wavelength of each pixel in the hyperspectral data is less than or equal to 2nm. The light source system 230 is also required to provide a light source when the hyperspectral imaging device 210 is started for shooting, and in order to ensure that the spectral range of the spectral data can be within the range of 400nm to 1100nm which meets the detection requirement of the food material types, the spectral range of the light source system 230 needs to be within the range of 400nm to 1100nm.
In step S408, an optional embodiment is: the first threshold is set to 90%, and if the calculated first recognition rate is higher than 90%, it is determined that the food material recognition is passed, and the type of the food material 300 is determined.
The embodiment also provides a method for detecting the nutritional components of the food material 300 in the refrigerator 10, which can be used in the refrigerator 10 of any embodiment to detect the nutritional component information of the food material 300 in the refrigerator 10.
Fig. 5 is a schematic view of a method of detecting nutritional components of food material 300 in the refrigerator 10 according to one embodiment of the present invention. The method of detecting the nutritional ingredients of the food material 300 in the refrigerator 10 may generally include:
step S502, acquiring hyperspectral data of the food material 300 photographed by the hyperspectral imaging apparatus 210;
step S504, respectively extracting image data and spectrum data from the hyperspectral data;
step S506, detecting the type of food material according to the image data;
step S508, acquiring a nutrient component detection model corresponding to the type of food materials;
in step S510, the spectral data are classified respectively by using the nutritional component models, so as to determine the nutritional components of the food material 300.
The nutrient component detection model can be obtained by training hyperspectral data of a large number of food materials with different qualities, and the training algorithm which can be adopted by the nutrient component detection model can comprise a BP neural network, a Support Vector Machine (SVM) and Adaboost. A plurality of different nutrient component detection models can be trained in advance according to the types of food materials, for example, corresponding nutrient component detection models can be trained for various meats, various fruits and various vegetables.
One specific implementation of step S510 may include: extracting spectral information required by the nutrient component detection model from hyperspectral data shot by the hyperspectral imaging device 210; inputting the spectral information required by the nutrient component detection model into the nutrient component detection model; pattern recognition is performed by the nutritional ingredient model to obtain the nutritional ingredients of the food material 300.
The hyperspectral data can comprise a set number of ternary data sets, each ternary data set comprises two image pixel elements and a spectral wavelength element of one pixel point, each pixel point is provided with a plurality of sets of ternary data sets, the spectral data are obtained by analyzing and extracting data in the image pixel elements, and the spectral data are obtained by analyzing and extracting data in the spectral wavelength elements. The resolution of the spectral wavelength of each pixel point in the hyperspectral data is less than or equal to 2nm. In order to ensure that the spectral range of the spectral data can be in the range of 400nm to 1100nm that satisfies the detection requirement, the spectral range of the light source system 230 needs to be in the range of 400nm to 1100nm.
The method for detecting information of food materials in a refrigerator in the embodiment extracts image data and spectrum data from hyperspectral data, detects the type of the food material 300 in the refrigerator 10 by using the image data, selects a corresponding nutritional component model according to the type of the food material 300, inputs the spectrum data into the nutritional component model, and detects the nutritional component information of the food material 300. The hyperspectral imaging technology combines the spectrum detection technology with the image recognition technology, and can simultaneously acquire the spatial image data of food materials and the spectrum data of each point. The variety information of the food materials can be rapidly and nondestructively acquired by utilizing the image data; and then, comprehensive analysis is carried out by combining with the spectral data, the nutrient component information of a plurality of pixel points on the image can be obtained at the same time, comprehensive calculation is carried out on the nutrient component information of a plurality of points, and finally the high-accuracy nutrient component information of the food material is obtained.
Thus, it should be appreciated by those skilled in the art that while a number of exemplary embodiments of the invention have been illustrated and described in detail herein, many other variations or modifications consistent with the principles of the invention may be directly determined or derived from the disclosure of the present invention without departing from the spirit and scope of the invention. Accordingly, the scope of the invention should be understood and interpreted to cover all such other variations or modifications.
Claims (9)
1. A method for detecting nutritional ingredients of food materials in a refrigerator, wherein a hyperspectral imaging device for shooting food materials in a storage chamber of the refrigerator is arranged in the storage chamber, and the method comprises the following steps:
after an instruction for identifying the nutritional ingredients of the food materials is received, starting the hyperspectral imaging device to acquire hyperspectral data obtained by shooting;
preprocessing the hyperspectral data, and respectively extracting image data and spectral data;
identifying the type of the food material according to the image data;
acquiring a corresponding nutritional component detection model according to the variety of the food material, wherein the nutritional component detection model is obtained by training according to the spectral data of the food materials with different qualities in advance; and
classifying, analyzing and calculating the spectral data by using the nutritional component detection model to obtain nutritional component information of each pixel point of the food material, and comprehensively calculating the nutritional component information of each pixel point to determine the nutritional components of the food material;
wherein the step of classifying the spectral data captured by the hyperspectral imaging apparatus using the nutrient detection model comprises:
extracting spectral data required by the nutrient component detection model from hyperspectral data shot by the hyperspectral imaging device;
inputting the spectral data required by the nutrient component detection model into the nutrient component detection model;
and carrying out pattern recognition by the nutritional component detection model to obtain the nutritional components of the food material.
2. The method of claim 1, wherein identifying the type of the food material from the image data comprises:
acquiring the image data;
identifying the type of the food material according to the image data to obtain first type information and a first identification rate;
judging whether the first identification rate is higher than a preset first threshold value or not;
and if so, determining the type of the food material.
3. The method of claim 1, wherein,
the high spectrum data comprises a set number of ternary data sets, each ternary data set comprises two image pixel elements and a spectrum wavelength element of a pixel point, each pixel point is provided with a plurality of groups of the ternary data sets, and
the image data is obtained by analyzing and extracting data in the image pixel elements, and the spectrum data is obtained by analyzing and extracting data in the spectrum wavelength elements.
4. The method of claim 3, wherein,
and the resolution ratio of the spectral wavelength of each pixel point in the hyperspectral data is less than or equal to 2nm.
5. The method of claim 1, wherein,
in the process of starting the hyperspectral imaging device, a light source system matched with the hyperspectral imaging device is started simultaneously to provide light required by the hyperspectral imaging device for shooting, wherein the spectral range of the light source system is 400nm to 1100nm.
6. The method of claim 1, wherein the step of determining the nutritional composition of the food material further comprises:
and outputting the nutritional ingredients of the food materials through a display screen of the refrigerator or a mobile terminal bound with the refrigerator.
7. A refrigerator, comprising:
the refrigerator comprises a refrigerator body, a storage compartment and a storage box, wherein the refrigerator body is internally limited with the storage compartment;
the hyperspectral imaging device is arranged in the storage room and is configured to shoot food materials in the storage room;
a controller comprising a memory and a processor, the memory having stored therein a computer program, and the computer program, when executed by the processor, for implementing the method of any of claims 1-6.
8. The refrigerator of claim 7, further comprising:
and the light source system is matched with the hyperspectral imaging device, is arranged in the storage room and is configured to provide light required by shooting for the hyperspectral imaging device, wherein the spectral range of the light source system is 400nm to 1100nm.
9. The refrigerator of claim 7, further comprising:
an information output interface configured to provide the food material category to a display screen of the refrigerator or a mobile terminal bound with the refrigerator for output to a user.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811163253.5A CN110966835B (en) | 2018-09-30 | 2018-09-30 | Method for detecting nutrient components of food materials in refrigerator and refrigerator |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811163253.5A CN110966835B (en) | 2018-09-30 | 2018-09-30 | Method for detecting nutrient components of food materials in refrigerator and refrigerator |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110966835A CN110966835A (en) | 2020-04-07 |
| CN110966835B true CN110966835B (en) | 2023-01-24 |
Family
ID=70029425
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811163253.5A Active CN110966835B (en) | 2018-09-30 | 2018-09-30 | Method for detecting nutrient components of food materials in refrigerator and refrigerator |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110966835B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111638190A (en) * | 2020-05-09 | 2020-09-08 | 珠海格力电器股份有限公司 | Plant growth monitoring method and plant growth cabinet |
| CN113409915A (en) * | 2021-05-10 | 2021-09-17 | 王蕴秋 | Safety recommendation system capable of preventing food from deteriorating and working method thereof |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104406916A (en) * | 2014-11-04 | 2015-03-11 | 百度在线网络技术（北京）有限公司 | Method and apparatus for detecting food |
| CN105115899A (en) * | 2015-08-28 | 2015-12-02 | 青岛海尔股份有限公司 | Refrigerator and detection method for food nutrition and/or heat of refrigerator |
| US20160150213A1 (en) * | 2014-11-21 | 2016-05-26 | Christopher M. MUTTI | Imaging system for object recognition and assessment |
| CN105928889A (en) * | 2016-04-29 | 2016-09-07 | 合肥华凌股份有限公司 | Refrigerator and food nutrient real-time detection method for refrigerator |
| US20160260352A1 (en) * | 2015-03-02 | 2016-09-08 | Fitly Inc. | Apparatus and method for identifying food nutritional values |
| WO2016186301A1 (en) * | 2015-05-15 | 2016-11-24 | 피도연 | Food condition measuring device and food condition measuring system using same |
| CN106871568A (en) * | 2016-12-30 | 2017-06-20 | 青岛海尔智能技术研发有限公司 | The category identification method and refrigerator of refrigerator inside food |
| US20180172510A1 (en) * | 2016-12-08 | 2018-06-21 | Verifood, Ltd. | Spectrometry system applications |
| WO2018165605A1 (en) * | 2017-03-09 | 2018-09-13 | Northwestern University | Hyperspectral imaging sensor |
-
2018
- 2018-09-30 CN CN201811163253.5A patent/CN110966835B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104406916A (en) * | 2014-11-04 | 2015-03-11 | 百度在线网络技术（北京）有限公司 | Method and apparatus for detecting food |
| US20160150213A1 (en) * | 2014-11-21 | 2016-05-26 | Christopher M. MUTTI | Imaging system for object recognition and assessment |
| US20160260352A1 (en) * | 2015-03-02 | 2016-09-08 | Fitly Inc. | Apparatus and method for identifying food nutritional values |
| WO2016186301A1 (en) * | 2015-05-15 | 2016-11-24 | 피도연 | Food condition measuring device and food condition measuring system using same |
| CN105115899A (en) * | 2015-08-28 | 2015-12-02 | 青岛海尔股份有限公司 | Refrigerator and detection method for food nutrition and/or heat of refrigerator |
| CN105928889A (en) * | 2016-04-29 | 2016-09-07 | 合肥华凌股份有限公司 | Refrigerator and food nutrient real-time detection method for refrigerator |
| US20180172510A1 (en) * | 2016-12-08 | 2018-06-21 | Verifood, Ltd. | Spectrometry system applications |
| CN106871568A (en) * | 2016-12-30 | 2017-06-20 | 青岛海尔智能技术研发有限公司 | The category identification method and refrigerator of refrigerator inside food |
| WO2018165605A1 (en) * | 2017-03-09 | 2018-09-13 | Northwestern University | Hyperspectral imaging sensor |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110966835A (en) | 2020-04-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110966833B (en) | Method for detecting food material information in refrigerator and refrigerator | |
| CN108663331A (en) | Detect the method and refrigerator of food freshness in refrigerator | |
| CN106871568A (en) | The category identification method and refrigerator of refrigerator inside food | |
| CN108662842A (en) | The detecting system and refrigerator of food in refrigerator | |
| CN107873101B (en) | Imaging system for object recognition and evaluation | |
| CN110966834A (en) | Method for identifying food material types in refrigerator and refrigerator | |
| Cubero et al. | Advances in machine vision applications for automatic inspection and quality evaluation of fruits and vegetables | |
| CN106871569A (en) | The category identification method and refrigerator of refrigerator inside food | |
| Goel et al. | HyperCam: hyperspectral imaging for ubiquitous computing applications | |
| Wang et al. | Application of hyperspectral imaging to discriminate the variety of maize seeds | |
| Barbin et al. | NIR hyperspectral imaging as non-destructive evaluation tool for the recognition of fresh and frozen–thawed porcine longissimus dorsi muscles | |
| CN108662843A (en) | The detecting system and refrigerator of food in refrigerator | |
| CN110966835B (en) | Method for detecting nutrient components of food materials in refrigerator and refrigerator | |
| CN110411957A (en) | The lossless method for quick predicting and device of fruit shelf life and freshness | |
| Valous et al. | Supervised neural network classification of pre-sliced cooked pork ham images using quaternionic singular values | |
| CN110017659A (en) | Domestic refrigerator equipment | |
| CN115131783B (en) | Autonomous sensing method of user diet nutrition ingredient information based on machine vision | |
| Jiang et al. | Identification of tomato maturity based on multinomial logistic regression with kernel clustering by integrating color moments and physicochemical indices | |
| CN109869966A (en) | Refrigerator | |
| CN110366731B (en) | Systems, methods, and storage media for image capture for instruction meal | |
| CN112906780A (en) | Fruit and vegetable image classification system and method | |
| Intaravanne et al. | Ripeness level indication of bananas with visible and fluorescent spectral images | |
| CN108663333A (en) | High light spectrum image-forming component and refrigerator for detecting food in refrigerator | |
| CN209640977U (en) | Intelligent commodity shelf system | |
| Qian et al. | A study on phenotypic micro-variation of stored melon based on weight loss rate |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20221221 Address after: 266101 Haier Industrial Park, 1 Haier Road, Laoshan District, Shandong, Qingdao Applicant after: QINGDAO HAIER SMART TECHNOLOGY R&D Co.,Ltd. Applicant after: Haier Zhijia Co.,Ltd. Address before: 266101 Haier Road, Laoshan District, Qingdao, Qingdao, Shandong Province, No. 1 Applicant before: QINGDAO HAIER SMART TECHNOLOGY R&D Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |