WO2020125456A1 - 一种食物监控方法、冰箱及具有存储功能的装置 - Google Patents

一种食物监控方法、冰箱及具有存储功能的装置 Download PDF

Info

Publication number
WO2020125456A1
WO2020125456A1 PCT/CN2019/123804 CN2019123804W WO2020125456A1 WO 2020125456 A1 WO2020125456 A1 WO 2020125456A1 CN 2019123804 W CN2019123804 W CN 2019123804W WO 2020125456 A1 WO2020125456 A1 WO 2020125456A1
Authority
WO
WIPO (PCT)
Prior art keywords
food
maturity
refrigerator
date
image
Prior art date
Application number
PCT/CN2019/123804
Other languages
English (en)
French (fr)
Inventor
周均扬
Original Assignee
广东美的白色家电技术创新中心有限公司
美的集团股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 广东美的白色家电技术创新中心有限公司, 美的集团股份有限公司 filed Critical 广东美的白色家电技术创新中心有限公司
Publication of WO2020125456A1 publication Critical patent/WO2020125456A1/zh

Links

Images

Classifications

    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F25—REFRIGERATION OR COOLING; COMBINED HEATING AND REFRIGERATION SYSTEMS; HEAT PUMP SYSTEMS; MANUFACTURE OR STORAGE OF ICE; LIQUEFACTION SOLIDIFICATION OF GASES
    • F25D—REFRIGERATORS; COLD ROOMS; ICE-BOXES; COOLING OR FREEZING APPARATUS NOT OTHERWISE PROVIDED FOR
    • F25D29/00—Arrangement or mounting of control or safety devices
    • F25D29/005—Mounting of control devices

Definitions

  • This application relates to the field of smart home technology, in particular to a food monitoring method, a refrigerator, and a device with a storage function.
  • the refrigerator is an indispensable home appliance for modern family cooking and eating.
  • the existing smart refrigerator can only achieve a simple food digital recording function, especially for the shelf life of food. Only the user can manually enter the placement time and expiration time to expire Prompt at time. This method requires the user to manually input the placement time and expiration time of each food, and the operation is cumbersome and not smart enough.
  • the technical problem mainly solved by this application is to provide a food monitoring method, a refrigerator and a device with a storage function, which can simplify user operations and realize intelligent monitoring of food.
  • a technical solution adopted by the present application is to provide a food monitoring method including: acquiring an image of food placed in a refrigerator; identifying the image to obtain the maturity of the food; acquiring the food The edible boundary date corresponding to the maturity; within the edible boundary date or within a preset time range before the edible boundary date, a corresponding prompt message is issued to remind the user to process the food in time.
  • a refrigerator including: an image acquisition device and a processor connected to each other; the image acquisition device is used to acquire images of food placed in the refrigerator; the processor To execute instructions to implement the food monitoring method as described above.
  • another technical solution adopted by the present application is to provide a device with a storage function that stores program instructions internally, and the program instructions are executed to implement the food monitoring method as described above.
  • the beneficial effect of the present application is that it is different from the situation in the prior art.
  • the refrigerator by acquiring an image of the food placed in the refrigerator, after recognizing the image, the maturity of the food is obtained, and then according to the food The maturity degree obtains the corresponding consumption boundary date, and within the preset time range before the consumption boundary date, a corresponding prompt message is issued to remind the user to process the food in time, so that the user only needs to put the food
  • the refrigerator can automatically judge the maturity of the food and the corresponding consumption boundary date according to the acquired food image, and make corresponding prompts, promptly prompt the user to process the ingredients, without requiring the user to manually set the consumption time, etc. It simplifies user operation, realizes the intelligent monitoring of food by the refrigerator, and improves the intelligent level of the refrigerator.
  • FIG. 1 is a schematic flowchart of a first embodiment of a food monitoring method of this application
  • FIG. 2 is a schematic flowchart of step S11 in FIG. 1;
  • FIG. 3 is a schematic diagram of a refrigerator suitable for the food monitoring method of the present application.
  • FIG. 4 is a schematic flowchart of a second embodiment of the food monitoring method of the present application.
  • FIG. 5 is a schematic flowchart of a third embodiment of the food monitoring method of the present application.
  • Figure 6 is a schematic diagram of a maturity curve of a certain food A
  • FIG. 7 is a schematic flowchart of a fourth embodiment of the food monitoring method of the present application.
  • FIG. 8 is a schematic flowchart of a fifth embodiment of the food monitoring method of the present application.
  • FIG. 9 is a schematic flowchart of a sixth embodiment of the food monitoring method of the present application.
  • FIG. 10 is a schematic flowchart of a seventh embodiment of the food monitoring method of the present application.
  • FIG. 11 is a schematic flowchart of an eighth embodiment of the food monitoring method of the present application.
  • FIG. 12 is a schematic structural view of an embodiment of the refrigerator of the present application.
  • FIG. 13 is a schematic structural diagram of an embodiment of a device with a storage function according to the present application.
  • the first embodiment of the food monitoring method of the present application includes:
  • an image of food placed in the refrigerator may be captured by an image acquisition device installed in the refrigerator.
  • an image acquisition device installed in the refrigerator.
  • a camera is installed on the top of the refrigerator, or a camera is installed on the top and inside of the refrigerator at the same time, and the installed camera can be used to obtain an image of the food when the user puts in the food.
  • step S11 includes:
  • S111 Receive a trigger signal to open the refrigerator door.
  • the refrigerator door is provided with a sensing device, such as a distance sensor, etc.
  • a sensing device such as a distance sensor, etc.
  • the sensing device When the refrigerator door is opened, the sensing device generates a trigger signal, and the refrigerator receives the trigger signal to know that the refrigerator door is opened.
  • S112 In response to the trigger signal, trigger the camera in the refrigerator to take an image of food in the refrigerator.
  • the camera in the refrigerator (for example, the camera C on the top of the refrigerator shown in FIG. 3) is triggered, and the camera can take an image of the food in the refrigerator, thus You can get an image of the food put in the refrigerator.
  • the camera can take multiple shots during the opening time of the refrigerator door, take multiple images, or take a video directly.
  • the maturity of the food is a judgment index corresponding to the preset freshness of the food.
  • the maturity of food can be divided into different levels, so that the approximate freshness of the food can be determined according to different levels.
  • a maturity of 1 represents the freshest food.
  • the maturity of fresh picking is 1, and 0 represents that the food has spoiled.
  • the maturity can be divided into three levels: excellent, medium and poor.
  • 0.7-1 is a good grade, which means that the food is at the best time to eat
  • 0.3-0.7 is a medium grade, which means that the food is within the expiry time
  • 0-0.3 is a poor grade, which means that the food has expired.
  • the relationship between the maturity of different foods and the storage time can be pre-stored in the refrigerator, for example, the maturity of apples after one day is 0.9, the maturity after two days is 0.8, and so on, so as to judge the freshness of the food from its current maturity degree.
  • an image recognition algorithm may be used to recognize the acquired image, and to recognize the characteristics of the category, quantity, and maturity of the food in the image.
  • the characteristics corresponding to the food can be determined according to the characteristics of the food in the image, such as the shape, color, texture, and fullness.
  • the neural network model trained in deep learning mode can also be used to identify features such as food maturity.
  • the consumption boundary date is at least one predetermined boundary date that can be consumed before the food expires.
  • there are two boundary dates corresponding to the goodness grade of the food that is, the boundary dates corresponding to the maturity of 1 and 0.7 respectively
  • the consumption boundary date corresponding to the excellent grade is the boundary date corresponding to 0.7, which is the best consumption date
  • the edible boundary date corresponding to the medium or poor maturity level of the food is the boundary date corresponding to the maturity level of 0.3, that is, the inedible date.
  • the specific value of the edible boundary date may be the storage time from the freshest maturity. If the edible date falls to 0.7 for 2 days, then 2 days is the best edible date.
  • the edible boundary date may also be the placement date from the time corresponding to the current maturity of the food. For example, if it is placed for 3 days from the current maturity of 0.5 to 0.3, then 3 days is the inedible date.
  • the maturity level to which the maturity belongs can be obtained according to the maturity of the food, and then the corresponding edible boundary date can be obtained from the maturity level .
  • the maturity level of the food is 0.5
  • the maturity level of the food is medium
  • the edible boundary date corresponding to the medium level is the inedible date (such as 5 days).
  • the preset time range is a time range preset according to the date when the food is put in the refrigerator, and the preset time range is not greater than the time difference between the date when the food is put in the refrigerator and the edible boundary date.
  • the value of the preset time range may be fixed, for example, one or two days before the consumption boundary date, or may be non-fixed, the preset time range of each food is different, for example, according to the maturity curve of the food.
  • the calculated optimal prompt time range corresponding to a certain food can prompt the user to process the corresponding food in time for different foods.
  • Food is usually fresh when it is put in the refrigerator. After the maturity of the food is obtained from the image of the food, it can be sent out according to the state of the food within the preset date range of the food's consumption boundary date or before the consumption boundary date. Prompt information corresponding to the consumption boundary date to remind the user to process the food in time, so that the user can be promptly reminded to eat before the food is still in a good state or is about to expire.
  • the prompt information can be displayed by a display device provided on the refrigerator, or by voice, or by pushing information to other connected terminal devices (such as the user's mobile phone).
  • the prompt information may include information such as the type, placement location and quantity of food, and the optimal consumption period of the food, so that the user can know which food needs to be processed.
  • the prompt message may be periodically prompted, for example, at 9 o'clock every day, or a food report may be given every week, or it may be aperiodic.
  • the edible boundary date corresponding to the maturity is 5 days, that is, the placement time from the freshest maturity is 5 days, and the current The maturity of the food corresponds to 2 days, which is 2 days away from the freshest maturity. It can be calculated that the current maturity will expire after 3 days, that is, a reminder message needs to be issued within 3 days.
  • the information may include the type of the food and the best consumption period of the food (such as the current date reminder within 3 days) to remind the user to process the food in time.
  • the user's mobile phone when the user's mobile phone enters the preset range, it automatically connects and pushes the prompt information of the food to the user's mobile phone to remind the user to process the food in time.
  • the refrigerator may also periodically remind during the optimal consumption time, and each reminder may simultaneously update the optimal consumption period (for example, the reminder will be updated within 2 days after being placed for one day).
  • the refrigerator can also be provided with a database to store food data, including information such as the type, quantity, placement time, placement location, and maturity of the food, so as to update the food maturity and other information in time to obtain the corresponding consumption Boundary date.
  • the image of the food put in the refrigerator is obtained, the image is recognized, the maturity of the food is obtained, and then the corresponding edible boundary date is obtained according to the maturity of the food, and the edible boundary date or the Before the consumption boundary date, a corresponding prompt message is issued to remind the user to process the food in time, so that the user only needs to put the food in the refrigerator, and the refrigerator can automatically judge the maturity of the food and the corresponding consumption according to the acquired food image Boundary date, and make corresponding prompts, prompting the user to process the ingredients in time, without requiring the user to manually set the consumption time, etc., simplifying the user operation, realizing the intelligent monitoring of the food by the refrigerator, and improving the intelligence of the refrigerator.
  • this embodiment uses images for recognition, and can remind according to the preservation time and preservation conditions of different ingredients. The reminder is more accurate, to avoid that the accuracy of the prompt is affected by the user setting information being wrong.
  • prompting the user to eat when the food is fresh is better, which may enable the user to eat when the food maintains a higher nutritional value.
  • step S13 includes:
  • a maturity range may be set as the maturity range corresponding to the optimal consumption time of the food.
  • the maturity range corresponding to the optimal consumption time may also be different.
  • the database of the refrigerator stores the maturity range corresponding to the optimal consumption time of different foods. After obtaining the maturity of a certain food (such as 0.8), the most suitable food can be obtained from the database The maturity range corresponding to the best consumption time (such as 0.7-1), that is, whether the current maturity of the food is within the maturity range corresponding to the optimal consumption time can be determined.
  • step S132 If the maturity of the food is within the maturity range corresponding to the optimal consumption time, the following step S132 is executed, otherwise step S133 is executed.
  • the optimal consumption time of food has two boundary dates, one of which is the boundary date corresponding to the freshest degree, and the other is the boundary date that is close to expired, and the optimal consumption boundary date is the boundary date that is close to expired.
  • the closest to the expired boundary date in the best time to obtain the food can be corresponding to the aging to maturity of the food at the best time Out of the maturity range, that is, the food is not in the best time to be served, promptly reminded.
  • step S132 it also includes:
  • the first preset time range is a time range preset based on the date when the food is placed in the refrigerator and the best consumption date of the food, and the time range is not greater than the date when the food is placed in the refrigerator and the optimal consumption boundary date Time difference.
  • the value of the first preset time range may be fixed or non-fixed.
  • the best eating reminder message is issued, which can promptly prompt the user to eat when the food remains in good condition.
  • the best eating boundary date can be converted to the first time from the current date, and then when the best eating prompt message is issued, the user is prompted to the first date from the current date Eat within the time (such as 5 days) to eat when the nutritional value is high, to maximize the effectiveness of food.
  • the inedible boundary date of the food is the upper limit of the food's expired maturity range or the upper limit of the maturity level of the food maturity level (such as 0.3 in 0-0.3), or the lower limit of the maturity level of the food maturity level (such as 0.7 in 0.3-0.7), the corresponding time.
  • the specific prompting method may be determined according to the range of the maturity of the food. Specifically, as shown in FIG. 4, after step S133, it further includes:
  • S134 Determine whether the maturity of the food is outside the range of maturity corresponding to the best consumption date and the inedible boundary date.
  • each food also has its corresponding expiration time (such as the shelf life on the food packaging, etc.), that is, the inedible boundary date, beyond the inedible boundary date, the food has expired and deteriorated and can not be eaten. Therefore, in this embodiment, according to the maturity of the food, an edible time (or soon to expire) between a food inedible boundary date and an optimal edible date is set. Within the range of maturity corresponding to the edible time, the food It is edible, but it should be consumed as soon as possible.
  • the database of the refrigerator also stores the maturity range (such as 0.3-0.7) corresponding to the expiration time (or about to expire time) of the food. After the maturity of the food is obtained, the food can be judged Whether the maturity of is within this range, if it is, then in response to the maturity of the food being within the maturity range corresponding to the optimal consumption date and the inedible boundary date, step S142 is executed.
  • the maturity range such as 0.3-0.7
  • the food can be judged Whether the maturity of is within this range, if it is, then in response to the maturity of the food being within the maturity range corresponding to the optimal consumption date and the inedible boundary date.
  • the second preset time range is a time range preset based on the date when the food is placed in the refrigerator and the date when the food is inedible, and the time range is not greater than the time difference between the date when the food is placed in the refrigerator and the inedible boundary date .
  • the value of the second preset time range may be fixed or non-fixed.
  • the second preset time range may be the same as the first preset time range, or may be different. For example, to avoid wasting food, the second time range can be set larger to prompt the user to process food in time.
  • S143 Issue an expiry reminder message to remind the user that the food has expired.
  • the inedible date of the food is obtained, and it is determined that the food is within the edible time, the inedible boundary date can be converted to the second time from the current date, and then issued soon
  • the user is prompted to eat within the second time (for example, 4 days) from the current date, so as to process the food before it expires to avoid wasting food.
  • an expiration reminder message is directly sent to remind the user that the food has expired.
  • the maturity of the food can be used to determine whether the food is within the optimal consumption time, and the user is promptly reminded to eat when the food is within the optimal consumption time to maximize the effectiveness of the food, while the food is not at the optimal Good time to eat, but within the edible time, promptly remind the user to process, so that the food can be processed before the expiration of the food, to avoid wasting food; at the same time, the food can also be expired when the food has expired, to remind the user that the food is expired and processed in time To avoid contaminating the refrigerator or other food.
  • the refrigerator may also save the maturity curve of the food to automatically estimate the food boundary date, so as to promptly notify the user to process.
  • step S132 or S133 includes:
  • the maturity curve of food is the relationship between the freshness of food and time. Different types of food have different maturity curves. For example, the maturity of vegetables and fruits changes faster than that of meat products. Therefore, vegetables 3. The maturity curve of fruit is steeper than that of meat products.
  • S62 Obtain the optimal consumption boundary date corresponding to the optimal consumption time from the food maturity curve; or, obtain the inedible boundary date corresponding to the food expiration time from the food maturity curve.
  • the maturity curves of different types of food are pre-stored in the refrigerator, and when the type of food is acquired, the maturity curve can be obtained. Since the maturity range corresponding to the optimal consumption time of the food or the maturity range corresponding to the food expiration time is set in advance, and the maturity of the food describes the correspondence between the maturity of the food and the placement time, After the maturity range corresponding to the optimal consumption time, or the maturity range corresponding to the food expiration time, the best consumption boundary date corresponding to the optimal consumption time of the food, or the food expiration time can be obtained directly from the maturity curve of the food Corresponding inedible boundary date.
  • a step of issuing corresponding prompt information before the edible boundary date or before the edible boundary date can be performed to remind the user to process the food in time.
  • the optimal consumption time of Food A corresponds to a maturity range of 0.7-1
  • the date of its optimal consumption boundary is the time corresponding to maturity 0.7 (as shown in FIG. 6 1.5 days).
  • the current maturity of Food A is 0.8
  • the obtained best eating boundary date of 1.5 days it can be converted that the time from Food A to the best eating boundary date is 0.5 days, then within 0.5 days Can send out the best food reminder message.
  • the refrigerator may also have a networking function, and the maturity curve corresponding to the food can be found from the cloud or a network server.
  • the maturity curve of the food is also related to the storage environment of the food, such as temperature and humidity. Therefore, for different temperature and humidity environments, the maturity curve of the same type of food is also different.
  • the environmental conditions at different locations in the refrigerator are different, and the position of the food in the refrigerator can also be combined to obtain the maturity curve corresponding to the food.
  • suggestions for eating food may be given, as shown in FIG. 5, after step S62, further including:
  • the third preset time range is a time range preset according to the date when the food is put into the refrigerator and the date when the food is inedible, and the time range is not greater than the date when the food is put into the refrigerator and the inedible boundary date Time difference.
  • the value of the third preset time range may be fixed or non-fixed.
  • the fourth preset time range is a time range preset according to the date when the food is put in the refrigerator and the best consumption date of the food, and the time range is not greater than the date when the food is put in the refrigerator and the best consumption boundary date Time difference.
  • the value of the fourth preset time range may be fixed or non-fixed.
  • the third preset time range may be the same as the fourth preset time range, or may be different.
  • the food optimization suggestions such as the best time to eat food, the best way to eat food, the inedible date, or the time from expiration, etc., for individuals with different maturities of the same food
  • the refrigerator can not only estimate the food boundary date according to the maturity curve, and give corresponding prompt information so that the user can process the food in time, but also can give food optimization before the food has expired. It is recommended to give users more intelligent and optimized eating reminders to further improve the intelligence of the refrigerator.
  • the camera when a camera is provided inside the refrigerator, the camera may be used to periodically capture images of food placed in the refrigerator to monitor the status of the food in real time.
  • the fourth embodiment of the food monitoring method of the present application is based on the first embodiment of the food monitoring method of the present application, and after further defining step S12, it may further include:
  • the camera on the top of the refrigerator or the top of the refrigerator may be used to periodically capture images of the food in the refrigerator, so that Food in the refrigerator is monitored in real time.
  • each time the image of the food in the refrigerator is acquired it can be compared with the image of the food in the refrigerator acquired last time.
  • the information of the food in each image can be identified, such as the number and type of food , Placement, maturity, etc. Comparing the recognition results of the images obtained at different times, you can get the changes of food in the refrigerator, such as the change of food maturity, so as to monitor the status of the food in the refrigerator in real time, so that you can notify the user to process in time, and you can also Collect the maturity change of the food to obtain the maturity curve of the food, so as to estimate the date of the food consumption boundary, so that the user can be promptly reminded to process the food on or before the food boundary date.
  • a deep learning neural network model may be used to process food images to identify the maturity of the food.
  • step S12 includes:
  • S121 Input the image into the image recognition model to obtain the recognition result output by the image recognition model.
  • the image recognition model is stored in the refrigerator in advance, and it is a neural network model trained in advance by the method of deep learning, which can intelligently recognize the information of the food in the image according to the input image and output the recognition result.
  • the recognition result includes information such as the type, quantity and maturity of the food in the image.
  • the training samples used by the image recognition model include a variety of foods put in and take out images under different lights (including the lights in the refrigerator and the ambient background light), etc. Therefore, the image recognition model can be applied to many different environments Identification.
  • the image is input into an image recognition model, and the image recognition model can automatically recognize the information of the food in the image and output the recognition result, from which the maturity of the food in the image can be obtained .
  • multiple different storage spaces can be provided in the refrigerator, and the environment in each storage space is different (such as different temperature and humidity), and the applicable food types are also different.
  • the image recognition model can identify the type of food. In order to instruct the user to put the food in the designated location. Specifically, as shown in FIG. 8, before step S122, it further includes:
  • the image recognition model is a neural network model trained with limited sample data, there may be cases where the recognition is unsuccessful, for example, it is usually impossible to recognize the food contained in the opaque bag.
  • the recognition fails, it will output a signal that cannot be recognized or reported an error, and when it is successfully recognized, it will output a recognition result, which is different from the signal that cannot be recognized or reported an error. Therefore, through the output result of the image recognition model, it can be judged whether the recognition is successful.
  • step S124 is executed.
  • the image recognition model can use the input image to recognize the type of food in the image and its corresponding maturity, and the type of food can be obtained from the recognition result output from it.
  • Different storage spaces are set in the refrigerator, each storage space may have a different storage environment, and the types of foods suitable for storage are different, so after identifying the type of food, a storage prompt (such as a light or voice prompt) can be issued to guide the user The recommended place to put food in the refrigerator.
  • the fresh-keeping area of the refrigerator is provided with multiple compartments.
  • the storage environment of each compartment is different.
  • the compartment L1 is suitable for fruits and the compartment L2 is suitable for vegetables.
  • Each compartment is provided with a controllable light X is usually off. When the recognized food type is suitable for being placed in a certain compartment, the controllable light X of the corresponding compartment is lit, so that the user can be directed to put the food into the shiny compartment in.
  • step S125 is performed.
  • S125 A voice interaction message is issued, prompting the user to confirm the type of food to be put in or taken out, so as to prompt the user to put the food in the recommended place or update the data of the food stored in the refrigerator.
  • the refrigerator cannot know the type of food that the user puts in or takes out.
  • the refrigerator can use a voice interaction device, such as a speaker, to input voice interaction information to the user to prompt the user to enter the type of food , Or select several possible types for users to choose. Then, it receives the food type input or confirmed by the user.
  • a voice interaction device such as a speaker
  • the user puts the food in the refrigerator he can continue to prompt the user to put the food in the recommended storage location, and can also update the data of the food stored in the refrigerator at the same time.
  • you can update the data (such as type, quantity, etc.) of the food stored in the refrigerator.
  • the user can input or confirm the food type through voice or touch screen input, or terminal text input interaction, etc.
  • step S122 it includes:
  • the environmental information may include temperature, humidity, remaining space and other information.
  • the refrigerator can pre-store information on the change trend of the maturity of different food types in different locations in the refrigerator.
  • the type of food, environmental information and current food maturity are obtained After the degree, you can find the corresponding maturity change trend information, that is, the maturity curve of the food.
  • the food maturity curve is the corresponding relationship between the maturity of food and the storage time of food, and the maturity curve of different foods may be different.
  • step S13 may be executed to obtain the edible boundary date corresponding to the maturity of the food, so as to issue corresponding prompt information before the edible boundary date or the edible boundary date of the food to remind the user to process the food.
  • the refrigerator can also give food optimization suggestions or purchase suggestions based on the data of the food in the refrigerator.
  • the refrigerator may also issue a prompt to guide the user to place the food in the designated location.
  • the method further includes:
  • the refrigerator may continue to obtain an image of the food in the refrigerator, and the image includes the suggested placement location.
  • the refrigerator can identify whether the corresponding food is placed in the recommended placement position in the image through an image recognition algorithm or a pre-trained image recognition model, and if the corresponding food is not placed in the recommended placement position, perform the following step S131 Otherwise, continue to execute S122.
  • a voice prompt is issued to prompt the user to place the food in the recommended placement location.
  • the refrigerator may issue a voice prompt to prompt the user to place the food in the recommended placement location, so as to maximize the food preservation time.
  • the refrigerator may also recognize whether the user puts food into the refrigerator or takes food out of the refrigerator, so as to give different suggestions.
  • the sixth embodiment of the food monitoring method of the present application is based on the first embodiment of the food monitoring method of the present application, and further includes:
  • an image acquisition device is installed in the refrigerator, for example, a camera is provided at the top of FIG. 3, and when the refrigerator door is opened, the camera is triggered to take multiple shots, so that the food can be taken into the refrigerator or taken out of the refrigerator
  • Process image of food the process image includes multiple images.
  • the camera can also directly shoot video, and then can obtain images of the process of putting food into the refrigerator or taking food out of the refrigerator from the acquired video.
  • S32 Process the process image to recognize the food picking action in the process image.
  • multiple consecutive images at adjacent times in the process image may be compared, for example, processing two images taken at a previous time and taken at a time after the next time to identify the food
  • the position changes, or the position of the hand that takes the food changes it can be recognized whether the food picking action is put in or taken out.
  • the hand is holding food in the image at the initial stage, and after a period of time it is recognized that the hand is not holding food when the hand leaves the viewable area of the refrigerator camera, you can determine the picking action from the captured image frame Yes; otherwise, it is recognized that in the initial stage of the image, the hand entered the camera's visible area without taking food, and after a period of time the image left the camera's visible area in the image, the hand is holding the food, you can determine the picking action For removal.
  • images of food taken from the refrigerator can be pre-collected, including two types of images of food being put in and taken out.
  • a neural network model based on deep learning is trained to identify food taking Whether the action is in or out. After training the neural network model, after obtaining the process image of putting food into the refrigerator or taking food out of the refrigerator, input the process image directly to the neural network model, the neural network model can be identified, and then directly output Whether the recognition result of the picking action is to put in or take out.
  • step S34 is executed.
  • S34 Update the data of the food stored in the refrigerator to give recommendations for food optimization or purchase recommendations for the food in the refrigerator.
  • the information of the food in the single frame image of the process image may also be recognized, including the type, quantity, and maturity of the food.
  • the neural network may be the same image recognition model as step S121, and the image recognition network may not only recognize the food taking action, but also the food information.
  • the data of the food stored in the refrigerator can be updated to give food optimization suggestions or purchase suggestions for the food in the refrigerator, so as to intelligently manage the food in the refrigerator.
  • the type of food in the process image can be identified, and then when a food picking action is recognized as being put in, a storage prompt can be given. Specifically, as shown in FIG. 9, after step S33, if the food picking action is recognized as being put in, the following step S35 is executed.
  • the food information (at least including the type of food) can also be obtained. Since the types of foods that can be stored in different storage locations in the refrigerator may be different, placing the foods in a suitable storage location can maximize the preservation time of the food. Therefore, according to the type of the food, the refrigerator may issue a storage reminder (for example, a light or a voice prompt) to guide the recommended location of the food in the refrigerator, so as to maximize the freshness of the food.
  • a storage reminder for example, a light or a voice prompt
  • step S33 after it is determined in step S33 that the picking action is put in, the type and quantity of food in the refrigerator also change accordingly. At this time, the data of the food stored in the refrigerator may also be updated.
  • step S12 includes:
  • step S42 is executed, otherwise step S43 is executed.
  • the pre-expired maturity range is a pre-set maturity range of the food that will soon expire.
  • the value can be different.
  • the refrigerator can pre-store the pre-expiry maturity range corresponding to different types of food, or the freezer can search the pre-expiry maturity range of the same food from the cloud or the Internet.
  • S43 Recognize the maturity of a single individual or the same maturity of different individuals as the maturity of the same food.
  • the maturity of different individuals When the maturity of different individuals is the same or there is only one individual, only the maturity of the identified food is required as the maturity of the food, and the maturity of the identified individual or the same maturity of different individuals can be regarded as the same kind
  • For the maturity of the food proceed to step S13, and determine whether the food is expired or about to expire according to the maturity of the food.
  • the maturity closest to the pre-expired maturity range can be regarded as the maturity of the same food. In order to avoid that an individual will expire due to the inappropriate maturity of similar foods without reminding.
  • using the maturity closest to the pre-expiry range to represent the maturity of similar foods can also reduce the maturity data of foods that need to be stored in the refrigerator and save storage resources.
  • the gas in the refrigerator can be sensed to assist in identifying changes in the maturity of the food.
  • the eighth embodiment of the food monitoring method of the present application is based on the first embodiment of the food monitoring method of the present application, and after further defining step S12, further includes:
  • the number of the odor sensor may be multiple, which are arranged in different compartments in the refrigerator to recognize the odor of food in different compartments.
  • the odor sensor can sense the odor type, concentration and other information in the refrigerator and generate data.
  • the refrigerator can obtain the data of the odor sensor in different compartments, and according to the data, information such as the composition and concentration of the odor emitted by the food placed in the different compartments of the refrigerator can be obtained, and then the different compartments can be estimated.
  • the approximate maturity of the food in the layer is combined with the recognition result of the food image to comprehensively determine the maturity of the food.
  • the judgment of food maturity is combined with the data of the odor sensor, which can supplement the result of image recognition and improve the accuracy of judgment of food maturity.
  • the refrigerator 10 includes: an image acquisition device 101 and a processor 102 connected to each other.
  • the image acquisition device 101 may be a camera, the number of which may be one or more, it may be provided on the top of the refrigerator 10, or the top and the interior of the refrigerator 10 may be provided at the same time.
  • the image acquisition device 10 is used to acquire images of food placed in the refrigerator 10.
  • the processor 102 is used to control the operation of the refrigerator 10, for example, to control the image acquisition device 101 to perform shooting.
  • the processor 102 may also be called a CPU (Central Processing Unit, central processing unit).
  • the processor 102 may be an integrated circuit chip with signal processing capabilities.
  • the processor 102 may also be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components .
  • the general-purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
  • the processor 102 is specifically configured to execute instructions to implement the method provided by any one of the first to eighth embodiments of the food monitoring method of the present application or a non-conflicting combination thereof.
  • the refrigerator 10 further includes an odor sensor 103 connected to the processor 102.
  • the odor sensor 103 is used to sense the odor in the refrigerator 10 and generate data.
  • the processor 102 is used to acquire the data of the odor sensor 103, and combine the data and the recognition result of the food image to obtain the maturity of the food.
  • the refrigerator 10 further includes a voice device 104 connected to the processor 101.
  • the voice device 104 may include a microphone and a speaker for voice interaction with the user.
  • the refrigerator 10 further includes a memory 105 connected to the processor 102.
  • the memory 105 is used to store instructions and quantities required by the processor 102 to execute, for example, information about food placed in the refrigerator.
  • the memory 105 may also be connected to the image acquisition device 101, the odor sensor 103, and the voice device 104 to save the acquired data.
  • the refrigerator 10 may further include other components such as a display (not shown), which is not specifically limited herein.
  • the processor of the refrigerator uses the image acquisition device to acquire an image of the food placed in the refrigerator, and then recognizes the image to obtain the maturity of the food, and then determines whether it is expired or is about to expire according to the maturity of the food.
  • an expiration reminder message is sent to remind the user that the food is expired
  • a reminder message is issued to remind the user that the food is about to expire, so that the user only needs to put the food into the refrigerator
  • the refrigerator can automatically judge whether the food expires or is about to expire based on the acquired food image, and prompt the user without the user manually setting the expiration time, etc., simplifying the user operation, realizing the intelligent monitoring of the food by the refrigerator, and improving the intelligence of the refrigerator ⁇ Degree.
  • a program instruction 210 is stored inside the device 20 with a storage function, and the program instruction 210 is executed to implement the first to eighth food monitoring methods of the present application The method provided by any embodiment or its non-conflicting combination.
  • the device 20 with a storage function may be a portable storage medium such as a U disk, an optical disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), or a magnetic disk.
  • the medium storing the program instructions may also be a mobile terminal, a home appliance controller, or an independent component that can be integrated in the refrigerator, such as a control chip, or it may be a server that stores the program instructions, and the server may store the stored program instructions Send to other devices to run, or you can run the stored program instructions by yourself.
  • the device 20 with a storage function may also be a memory as shown in FIG. 11.
  • the disclosed method and device may be implemented in other ways.
  • the device implementation described above is only schematic.
  • the division of modules or units is only a division of logical functions.
  • there may be other divisions for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical, or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above integrated unit can be implemented in the form of hardware or software function unit.
  • the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.
  • the technical solution of the present application may be essentially or part of the contribution to the existing technology or all or part of the technical solution may be embodied in the form of a software product, and the computer software product is stored in a storage medium , Including several instructions to enable a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of the embodiments of the present application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program code .

Landscapes

  • Engineering & Computer Science (AREA)
  • Chemical & Material Sciences (AREA)
  • Combustion & Propulsion (AREA)
  • Physics & Mathematics (AREA)
  • Mechanical Engineering (AREA)
  • Thermal Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Cold Air Circulating Systems And Constructional Details In Refrigerators (AREA)

Abstract

一种食物监控方法，包括：获取放入冰箱中的食物的图像；对该图像进行识别，以得到食物的成熟度；获取该食物的成熟度对应的食用边界日期；在该食用边界日期或者该食用边界日期之前的预设时间范围内，发出相应的提示信息，以提醒用户及时处理该食物，实现冰箱对食物的智能监控，提高冰箱的智能化程度。

Description

一种食物监控方法、冰箱及具有存储功能的装置
本申请要求于2018年12月18日提交的申请号为2018115490158，发明名称为“一种食物监控方法、冰箱及具有存储功能的装置”的中国专利申请的优先权，其通过引用方式全部并入本申请。
【技术领域】
本申请涉及智能家居技术领域，特别是涉及一种食物监控方法、冰箱及具有存储功能的装置。
【背景技术】
随着技术的发展与人民生活水平的提高，越来越多的智能家电开始走进了千家万户，智能化技术对日常生活的方方面面都在进行影响。冰箱作为现代家庭烹饪饮食必不可缺的家电，现有的智能冰箱，仅仅能实现简单的食物数字化记录功能，尤其是对食物的保质时间，仅能通过用户手动输入放置时间和过期时间，到过期时间时进行提示。这种方式需要用户手动输入每种食物的放置时间和过期时间，操作繁琐，不够智能。
【发明内容】
本申请主要解决的技术问题是提供一种食物监控方法、冰箱及具有存储功能的装置，能够简化用户操作，实现智能监控食物。
为解决上述技术问题，本申请采用的一个技术方案是：提供一种食物监控方法，包括：获取放入冰箱中的食物的图像；对该图像进行识别，以得到食物的成熟度；获取食物的成熟度对应的食用边界日期；在该食用边界日期或者该食用边界日期之前的预设时间范围内，发出相应的提示信息，以提醒用户及时处理该食物。
为解决上述技术问题，本申请采用的另一个技术方案是：提供一种冰箱，包括：相互连接的图像获取装置和处理器；图像获取装置用于获取放入冰箱中的食物的图像；处理器用于执行指令以实现如上所述的食物监控方法。
为解决上述技术问题，本申请采用的又一个技术方案是：提供一种具有存储功能的装置，内部存储有程序指令，该程序指令被执行以实现如上所述的食物监控方法。
本申请的有益效果是：区别于现有技术的情况，本申请的实施例中，通过获取放入冰箱中的食物的图像，对该图像进行识别后，得到食物的成熟度，然后根据食物的成熟度获取对应的食用边界日期，并在该食用边界日期或者该食用边界日期之前的预设时间范围内，发出相应的提示信息，以提醒用户及时处理该食物，从而使得用户只需要将食物放入冰箱中，冰 箱就能够自动根据获取的食物图像判断食物的成熟度及其对应的食用边界日期，并做出相应的提示，以及时提示用户处理食材，而不需要用户手动设置食用时间等，简化了用户操作，实现冰箱对食物的智能监控，提高冰箱的智能化程度。
【附图说明】
图1是本申请食物监控方法第一实施例的流程示意图；
图2是图1中步骤S11的具体流程示意图；
图3是本申请食物监控方法一种适用的冰箱的示意图；
图4是本申请食物监控方法第二实施例的流程示意图；
图5是本申请食物监控方法第三实施例的流程示意图；
图6是某种食物A一条成熟度曲线的示意图；
图7是本申请食物监控方法第四实施例的流程示意图；
图8是本申请食物监控方法第五实施例的流程示意图；
图9是本申请食物监控方法第六实施例的流程示意图；
图10是本申请食物监控方法第七实施例的流程示意图；
图11是本申请食物监控方法第八实施例的流程示意图；
图12是本申请冰箱一实施例的结构示意图；
图13是本申请具有存储功能的装置一实施例的结构示意图。
【具体实施方式】
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅是本申请的一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
如图1所示，本申请食物监控方法第一实施例包括：
S11：获取放入冰箱中的食物的图像。
具体地，可以通过安装在冰箱中的图像获取装置拍摄放入冰箱中的食物的图像。例如，在冰箱顶部安装一个摄像头，或者在冰箱顶部和内部同时安装摄像头，利用安装的摄像头即可在用户放入食物时获取放入的食物的图像。
可选地，摄像头可以在冰箱被打开时触发拍摄，具体如图2所示，步骤S11包括：
S111：接收冰箱门开启的触发信号。
具体地，冰箱门安装有感测装置，例如距离传感器等，当冰箱门被打开时，感测装置会产生触发信号，冰箱接收到该触发信号，即可以得知冰箱门被打开。
S112：响应该触发信号，触发冰箱中的摄像头，拍摄食物放入冰箱过程中的图像。
当冰箱接收到冰箱门开启的触发信号时，响应该触发信号，触发冰箱中的摄像头(例如图3所示冰箱顶部的摄像头C)，该摄像头即可以拍摄食物放入冰箱过程中的图像，从而可以得到放入冰箱中食物的图像。其中，摄像头可以在冰箱门开启时间内进行多次拍摄，拍摄多张图像，或者直接拍摄一段视频。
当冰箱中存在多个摄像头时，也可以利用多个摄像头从多个不同角度拍摄放入冰箱或从冰箱取出食物的图像，以便结合多个图像获取食物的完整图像，提高后续对食物的识别精度。
S12：对该图像进行识别，以得到食物的成熟度。
其中，食物的成熟度是预先设定的食物新鲜程度对应的判断指标。食物的成熟度可以划分为不同等级，从而可以根据不同等级确定食物的大致新鲜程度。例如，成熟度为1代表食物最新鲜，如新鲜采摘时的成熟度为1，0代表食物已经腐败变质，成熟度可以划分为优良、中等和差三个等级。例如，0.7-1为优良等级，代表食物处于最佳食用时间，0.3-0.7为中等等级，代表食物处于即将过期时间内，0-0.3为差等级，代表食物已经过期。
冰箱中可以预先存储不同食物的成熟度与放置时间的关系，例如苹果放置一天后的成熟度为0.9，放置两天后的成熟度为0.8，以此类推，以便从食物当前的成熟度判断其新鲜程度。
具体地，在一个应用例中，可以利用图像识别算法，对获取的图像进行识别，识别图像中食物的类别、数量及其成熟度等特征。其中可以根据图像中食物的形状、颜色、纹理、饱满程度等特征，判断该食物对应的特征。当然，也可以采用深度学习方式训练的神经网络模型识别食物的成熟度等特征。
S13：获取食物的成熟度对应的食用边界日期。
其中，该食用边界日期是预先设定的该食物过期之前可食用的至少一个边界日期。例如，该食物的成熟度优良等级对应的边界日期有两个，即成熟度为1和0.7分别对应的边界日期，则优良等级对应的食用边界日期为0.7对应的边界日期，即最佳食用日期；而该食物的成熟度中等等级或差等级对应的食用边界日期为成熟度为0.3时对应的边界日期，即不可食用日期。该食用边界日期具体取值可以是距离最新鲜的成熟度的放置时间，如放置2天下降到0.7，则2天为该最佳食用日期。当然，该食用边界日期也可以是距离食物当前成熟度对应的时间的放置日期，如放置3天从当前成熟度0.5下降到0.3，则3天为该不可食用日期。
具体地，在一个应用例中，通过食物图像识别得到食物的成熟度后，根据该食物的成熟度即可以获取该成熟度所属成熟度等级，然后从该成熟度等级可以获取对应的食用边界日期。例如，识别得到食物的成熟度为0.5时，可以得知该食物的成熟度等级为中等，继而可以得到中等等级对应的食用边界日期为不可食用日期(如5天)。
S14：在该食用边界日期或者该食用边界日期之前的预设时间范围内，发出相应的提示信息，以提醒用户及时处理该食物。
其中，该预设时间范围是根据该食物放入冰箱的日期预先设置的一个时间范围，该预设时间范围不大于该食物放入冰箱的日期和该食用边界日期之间的时间差。该预设时间范围的取值可以是固定的，例如该食用边界日期之前的一天或两天，也可以是非固定的，每个食物的该预设时间范围不同，例如根据该食物的成熟度曲线计算得到的对应某个食物的最佳提示 时间范围，以针对不同食物均能及时提醒用户处理对应的食物。
食物在放入冰箱时通常较为新鲜，通过食物的图像获取到食物的成熟度后，可以根据食物的状态，在食物的食用边界日期或者该食用边界日期之前的预设时间范围内，发出与该食用边界日期对应的提示信息，以提醒用户及时处理该食物，从而可以在食物还处于较好状态或者即将过期之前及时提醒用户进行食用。
其中，该提示信息可以通过冰箱上设置的显示装置进行显示，也可以通过语音方式进行提示，还可以通过向连接的其他终端设备(如用户的手机)推送信息的方式进行提示。该提示信息可以包括食物的类型、放置位置和数量，以及食物的最佳食用期限等信息，以便用户可以得知具体需要对哪个食物进行处理。发出提示信息可以是周期性地提示，例如每天9点进行提示，也可以每周给出一个食物报表，还可以是非周期性地提示。
具体地，在一个应用例中，识别某个食物的成熟度为0.5后，可以得到该成熟度对应的食用边界日期为5天，即距离最新鲜的成熟度的放置时间为5天，而当前食物的成熟度对应的时间为2天，即距离最新鲜的成熟度的放置时间为2天，可以计算得到当前成熟度再放置3天就会过期，即在3天内需要发出提示信息，该提示信息内可以包括该食物的类型以及该食物的最佳食用期限(如当前日期提醒则为3天内)，以提醒用户及时处理该食物。例如，在用户手机进入预设范围内时自动连接，并向用户手机推送食物的提示信息，提醒用户及时处理该食物。该冰箱也可以在该最佳食用时间内周期性地进行提醒，每次提醒可以同步更新该最佳食用期限(如放置一天后提醒则更新为2天内)。
此外，冰箱内部还可以设置有数据库存储食物的数据，包括食物的类型、数量、放置时间、放置位置、放置时的成熟度等信息，以便及时更新食物的成熟度等信息，以获取对应的食用边界日期。
本实施例中，通过获取放入冰箱中的食物的图像，对该图像进行识别后，得到食物的成熟度，然后根据食物的成熟度获取对应的食用边界日期，并在该食用边界日期或者该食用边界日期之前，发出相应的提示信息，以提醒用户及时处理该食物，从而使得用户只需要将食物放入冰箱中，冰箱就能够自动根据获取的食物图像判断食物的成熟度及其对应的食用边界日期，并做出相应的提示，以及时提示用户处理食材，而不需要用户手动设置食用时间等，简化了用户操作，实现冰箱对食物的智能监控，提高冰箱的智能化程度。同时，本实施例利用图像进行识别，可以根据不同食材的保鲜时间、保鲜条件进行提醒，提示较为准确，避免由于用户设定信息有误影响提示的准确性。
在其他实施例中，在食物处于新鲜程度较佳时提示用户食用，可以使得用户在食物保持较高营养价值时食用。
具体如图4所示，本申请食物监控方法第二实施例，是在本申请食物监控方法第一实施例的基础上，进一步限定步骤S13包括：
S131：判断该食物的成熟度是否在最佳食用时间所对应的成熟度范围内。
每种食物都有其最佳食用时间，超过该最佳食用时间，食物的营养价值或者水分等会流失较多，口感可能不加，若能够在食物处于最佳食用时间内食用，则有利于发挥食物的最大 营养功效。因此，本实施例中，针对食物的成熟度，可以设置一个成熟度范围作为食物的最佳食用时间对应的成熟度范围。当然，针对不同食物，该最佳食用时间对应的成熟度范围也可以不同。
具体地，在一个应用例中，冰箱的数据库中存储有不同食物的最佳食用时间对应的成熟度范围，获取某个食物的成熟度(如0.8)后，可以在数据库中获取该食物的最佳食用时间对应的成熟度范围(如0.7-1)，即可以判断该食物当前成熟度是否处于该最佳食用时间对应的成熟度范围内。
若该食物的成熟度在最佳食用时间所对应的成熟度范围内，则执行如下步骤S132，否则执行步骤S133。
S132：获取该食物的最佳食用边界日期。
食物的最佳食用时间具有两个边界日期，其中一个为最新鲜程度对应的边界日期，另一个为与接近过期的边界日期，该最佳食用边界日期为该接近过期的边界日期。
在食物处于最佳食用时间内，即该食物的新鲜度还处于优良状态，获取该食物的最佳食用时间中最接近过期的边界日期，可以在食物老化到成熟度在最佳食用时间对应的成熟度范围之外，即食物不是处于最佳食用时间之前，及时进行提醒。
进一步地，步骤S132之后，还包括：
S141：在最佳食用边界日期或者最佳食用边界日期之前的第一预设时间范围内，发出最佳食用提示信息，以提醒用户在该食物保持在优良状态时进行食用。
该第一预设时间范围是根据该食物放入冰箱的日期和该食物最佳食用日期预设的时间范围，其时间范围不大于该食物放入冰箱的日期和该最佳食用边界日期之间的时间差。该第一预设时间范围的取值可以是固定的，也可以是非固定的。
在食物的成熟度还处于最佳食用时间所对应的成熟度范围内，即食物还处于最佳食用时间内时，获取得到该食物的最佳食用边界日期后，在该最佳边界日期或之前的第一预设时间范围内(如最佳边界日期之前的一天内)，发出最佳食用提示信息，可以及时提示用户在该食物保持在优良状态时进行食用。例如，在获取得到最佳食用日期后，还可以将该最佳食用边界日期换算为距离当前日期的第一时间，然后在发出最佳食用提示信息时，提示用户在距离当前日期的该第一时间(如5天)内进行食用，以在食用营养价值较高时进行食用，最大程度发挥食物的功效。
S133：获取该食物的不能食用边界日期。
食物的不能食用边界日期是食物过期成熟度范围的上限或者食物成熟度等级为差等级的上限成熟度(如0-0.3中的0.3)，又或者食物成熟度等级为中等的下限成熟度(如0.3-0.7中的0.7)，所对应的时间。
可选地，若食物不是处于最佳食用时间，该食物可能已经过期不可食用，或者未过期还可食用，则可以根据食物的成熟度范围确定具体的提示方式。具体如图4所示，步骤S133之后，还包括：
S134：判断该食物的成熟度是否在最佳食用日期和不能食用边界日期所对应的成熟度范 围之外。
其中，每种食物还有其对应的过期时间(如食物包装上的保质期等)，即不可食用边界日期，超过该不可食用边界日期，则食物已经过期变质，不能食用。因此，本实施例中，根据食物的成熟度，设置一个食物不可食用边界日期和最佳食用日期之间的可食用时间(或即将过期时间)，该可食用时间对应的成熟度范围内，食物可以食用，但应尽快食用。
具体地，在一个应用例中，冰箱的数据库中还存储有食物的过期时间(或即将过期时间)所对应的成熟度范围(如0.3-0.7)，获取食物的成熟度后，即可以判断食物的成熟度是否处于该范围内，若处于，则响应于食物的成熟度在最佳食用日期和不能食用边界日期所对应的成熟度范围之内，执行步骤S142。
S142：在该不能食用边界日期或者该不能食用边界日期之前的第二预设时间范围内，发出即将过期提示信息，以提醒用户在该食物即将过期之前进行处理。
该第二预设时间范围是根据该食物放入冰箱的日期和该食物不能食用日期预设的时间范围，其时间范围不大于该食物放入冰箱的日期和该不能食用边界日期之间的时间差。该第二预设时间范围的取值可以是固定的，也可以是非固定的。其中，该第二预设时间范围可以与该第一预设时间范围相同，也可以不同。例如，为了避免浪费食物，可以将第二时间范围设置更大，以便更早提示用户及时处理食物。
当该食物的成熟度处于最佳食用日期和不能食用边界日期所对应的成熟度范围外，则响应于食物的成熟度在最佳食用日期和不能食用边界日期所对应的成熟度范围之外，执行步骤S143。
S143：发出过期提示信息，以提醒用户该食物过期。
具体地，在上述应用例中，获取得到该食物的不能食用日期，且判定该食物处于可食用时间内，可以将该不能食用边界日期换算为距离当前日期的第二时间，然后在发出即将过期提示信息时，提示用户在距离当前日期的该第二时间(如4天)内进行食用，以在食物过期前进行处理，避免浪费食物。而在食物在该可食用时间之外时，直接发出过期提示信息，以提醒用户该食物过期。
本实施例中，通过食物的成熟度可以判断食物是否处于最佳食用时间内，并在食物处于最佳食用时间内及时提醒用户进行食用，以最大程度发挥食物的功效，而在食物不处于最佳食用时间，但处于可食用时间内，及时提醒用户进行处理，从而可以在食物过期前进行处理，避免浪费食材；同时在食物已经过期时还可以进行过期提示，以提醒用户食物过期及时进行处理，避免污染冰箱或污染其他食物。
在其他实施例中，针对不同的食物，冰箱还可以保存有食物的成熟度曲线，以自动预估食物的食用边界日期，以便及时通知用户进行处理。
具体如图5所示，本申请食物监控方法第三实施例是在本申请食物监控方法第二实施例的基础上，进一步限定步骤S132或S133包括：
S61：获取该食物的成熟度曲线。
其中，食物的成熟度曲线是食物的新鲜度与时间的关系曲线，不同类型的食物的成熟度 曲线不同，例如蔬菜、水果类食物的成熟度变化比肉制品的成熟度变化快，因此，蔬菜、水果的成熟度曲线比肉制品的成熟度曲线陡。
S62：从食物的成熟度曲线中获取最佳食用时间对应的最佳食用边界日期；或者，从食物的成熟度曲线中获取食物过期时间对应的不能食用边界日期。
具体地，在一个应用例中，冰箱中预先存储有不同类型食物的成熟度曲线，当获取食物的类型后，即可以得到其成熟度曲线。由于预先设置有食物的最佳食用时间对应的成熟度范围，或者食物过期时间对应的成熟度范围，且食物的成熟度描述了食物的成熟度与放置时间的对应关系，因此，获取食物的最佳食用时间对应的成熟度范围，或者食物过期时间对应的成熟度范围后，可以直接从该食物的成熟度曲线中获取该食物的最佳食用时间对应的最佳食用边界日期，或者食物过期时间对应的不能食用边界日期。获取该食用边界日期后，即可以执行在食用边界日期或者食用边界日期之前，发出相应的提示信息，以提醒用户及时处理该食物的步骤。例如图6所示，食物A的成熟度曲线中，食物A的最佳食用时间对应的成熟度范围为0.7-1，则其最佳食用边界日期是成熟度0.7对应的时间(如图6中的1.5天)。当前食物A的成熟度为0.8，则根据该当前成熟度0.8和获取的最佳食用边界日期1.5天，可以换算得到该食物A距离该最佳食用边界日期的时间为0.5天，则在0.5天内可以发出最佳食用提示信息。在其他应用例中，冰箱也可以具有联网功能，其可以从云端或网络服务器中查到到该食物对应的成熟度曲线。
此外，该食物的成熟度曲线还与食物的保存环境相关，例如温度、湿度等，因此，针对不同的温度、湿度环境，同种类型的食物其成熟度曲线也不同。冰箱中不同位置的环境条件不同，还可以结合冰箱中食物的放置位置，获取食物对应的成熟度曲线。
可选地，在该食物过期之前，可以给出食物的进食建议，如图5所示，步骤S62之后，还包括：
S63：在不能食用边界日期之前的第三预设时间范围内或最佳食用边界日期之前的第四预设时间范围内，输出食物的进食优化建议。
其中，该第三预设时间范围是根据该食物放入冰箱的日期和该食物不能食用日期预设的一个时间范围，其时间范围不大于该食物放入冰箱的日期和该不能食用边界日期之间的时间差。该第三预设时间范围的取值可以是固定的，也可以是非固定的。该第四预设时间范围是根据该食物放入冰箱的日期和该食物最佳食用日期预设的一个时间范围，其时间范围不大于该食物放入冰箱的日期和该最佳食用边界日期之间的时间差。该第四预设时间范围的取值可以是固定的，也可以是非固定的。其中，该第三预设时间范围可以与该第四预设时间范围相同，也可以不同。
具体地，在食物还没过期，即食物还可以食用的情况下，例如食物还处于最佳食用时间内，或者处于可食用时间内，可以在最佳食用边界日期之前的两天内，或者在不可食用边界日期之前的三天内，给用户输出食物的进食优化建议，例如食物最佳食用时间、食物较佳食用方式、不可食用日期、或距离过期的时间等，针对同种食物具有不同成熟度个体时，可以给出分批次食用建议，以保持较佳新鲜度，或者给出优先食用新鲜食物或不新鲜食物的建议 以便用户灵活选择等。因此，本实施例中，冰箱不仅可以根据成熟度曲线预估食物的食用边界日期，给出对应的提示信息，以便用户及时处理食物，还可以在食物还未过期前，给出食物的进食优化建议，以给用户更智能更优化的进食提醒，进一步提高冰箱的智能化程度。
在其他实施例中，当冰箱内部设置有摄像头时，可以利用摄像头周期性地拍摄放入冰箱内的食物的图像，以实时监控食物的状态。
具体如图7所示，本申请食物监控方法第四实施例是在本申请食物监控方法第一实施例的基础上，进一步限定步骤S12之后，还可以包括：
S21：周期性拍摄冰箱中食物的图像。
具体地，当冰箱内部设置有摄像头，或者冰箱顶部的摄像头可以在冰箱门关闭时拍摄到冰箱内部的食物的图像时，可以利用冰箱内部或顶部的摄像头周期性拍摄冰箱中食物的图像，以便对冰箱中的食物进行实时监控。
S22：对不同时间获取的图像进行比较处理，以对冰箱中食物的变化进行跟踪记录。
具体地，每次获取冰箱中食物的图像后，即可以跟前一次获取的冰箱中食物的图像进行比较，通过对图像的处理，可以识别出每张图像中食物的信息，例如食物的数量、类型、放置位置、成熟度等。对不同时间获取的图像的识别结果进行比较，可以得到冰箱中食物的变化情况，例如食物成熟度的变化情况等，以便实时监控冰箱中的食物的状态，从而可以及时通知用户进行处理，还可以采集食物的成熟度变化，得到食物的成熟度曲线，以便进行食物食用边界日期的估计，从而可以在食用边界日期或之前及时提醒用户处理食物。此外，还可以针对冰箱中食物的数量、类型和成熟度变化等，将冰箱中当前存在的食物和用户的饮食计划或冰箱默认的策略(如进食顺序为优先进食即将过期的食物等)进行匹配，给用户提供进食优化建议或采购建议等，从而实现对冰箱中食物的实时监控和智能管理，进一步提高冰箱的智能化程度。
本申请的食物监控方法中，可以采用深度学习的神经网络模型对食物图像进行处理，以识别出食物的成熟度。
具体如图8所示，本申请食物监控方法第五实施例是在本申请食物监控方法第一实施例的基础上，进一步限定步骤S12包括：
S121：将图像输入图像识别模型中，以获取图像识别模型输出的识别结果。
其中，该图像识别模型预先存储在冰箱中，是预先利用深度学习的方法训练好的神经网络模型，其可以根据输入的图像，智能识别出图像中食物的信息并输出识别结果。该识别结果包括图像中食物的类型、数量和成熟度等信息。其中，该图像识别模型采用的训练样本包括多种食物在不同灯光(包括冰箱内灯光及环境背景光)环境下的放入和取出图像等，因此，该图像识别模型可以适用于多种不同环境下的识别。
S122：获取该识别结果中食物的成熟度。
具体地，获取食物图像后，将该图像输入图像识别模型中，该图像识别模型即可以自动识别出图像中食物的信息并输出识别结果，从该识别结果中可以获取该图像中食物的成熟度。
可选地，冰箱中可以设置有多个不同的存储空间，每个存储空间中的环境不同(如温度、 湿度不同)，其适用的食物类型也不同，该图像识别模型可以识别食物的类型，以便指示用户将食物放入指定位置。具体如图8所示，步骤S122之前，还包括：
S123：判断图像识别模型是否成功识别。
具体地，由于图像识别模型是利用有限样本数据训练得到的神经网络模型，其可能存在识别不成功的情况，例如针对被不透明袋子装着的食物时通常无法识别。当识别失败时，其会输出无法识别或报错的信号，而成功识别时，其会输出识别结果，区别于无法识别或报错的信号。因此，通过图像识别模型的输出结果，即可以判断是否成功识别。
若成功识别，则执行如下步骤S124。
S124：根据识别结果中食物的类型，发出储存提示，以指引出食物在冰箱中的建议放置位置。
具体地，该图像识别模型可以利用输入的图像，识别出图像中的食物类型和其对应的成熟度，从其输出的识别结果中即可以得到食物的类型。冰箱中设置有不同的存储空间，每个存储空间其保存环境可能不同，适用于保存的食物类型不同，因此在识别出食物的类型后，可以发出储存提示(例如灯光或语音提示)，指引用户将食物放入冰箱中的建议放置位置。
例如图3中所示，冰箱保鲜区域设置有多个隔层，每个隔层的保存环境不同，隔层L1适合放置水果，隔层L2适合放置蔬菜，每个隔层设置有一个可控灯X，通常为不亮状态，当识别出的食物类型适合放入某个的隔层时，该对应隔层的可控灯X发亮，从而可以指引用户将该食物放入发亮的隔层中。
可选地，若图像识别模型识别失败，则执行如下步骤S125。
S125：发出语音交互信息，提示用户确认放入或取出的食物类型，以提示用户将食物放入建议放置位置或更新冰箱中保存的食物的数据。
具体地，当图像识别模型识别失败时，冰箱无法得知用户放入或取出的食物的类型，此时，冰箱可以利用语音交互设备，例如扬声器向用户输入语音交互信息，提示用户输入食物的类型，或者选择几种可能类型供用户选择。然后，接收用户输入或确认的食物类型，当用户是将食物放入冰箱时，可以继续提示用户将食物放入建议放置位置，也可以同时更新冰箱中保存的食物的数据，当用户是从冰箱中取出食物时，可以更新冰箱中保存的食物的数据(如类型、数量等)。其中，用户输入或确认食物类型可以通过语音或触屏输入，又或者终端文字输入交互等。
进一步地，步骤S122之后，包括：
S126：获取该建议放置位置对应的环境信息。
其中，该环境信息可以包括温度、湿度、剩余空间等信息。
S127：结合该环境信息和食物的类型，获取食物的成熟度曲线。
具体地，不同环境条件会影响不同食物成熟度的变化速度，冰箱可以预先保存有不同食物类型在冰箱中不同位置的成熟度变化趋势信息，当获取到食物的类型、环境信息和当前食物的成熟度后，可以查找到对应的成熟度变化趋势信息，即该食物的成熟度曲线。
S128：结合该成熟度曲线，继续执行获取食物的成熟度对应的食用边界日期的步骤。
其中，食物成熟度曲线是食物的成熟度与食物的放置时间之间的对应关系，不同食物的成熟度曲线可能不同。利用该食物的成熟度曲线，可以执行步骤S13，获取食物的成熟度对应的食用边界日期，以便在该食物的食用边界日期或者食用边界日期之前，发出相应的提示信息，以提醒用户及时处理该食物。
在该食物的可食用时间内，冰箱还可以根据冰箱内食物的数据，给出食物的进食优化建议或采购建议等。
可选地，在用户未将食物放入指定位置时，冰箱还可以发出提示以引导用户将食物放到指定位置。具体如图8所示，步骤S124之后，还包括：
S129：获取冰箱中该建议放置位置的图像。
其中，当用户将食物放入冰箱后，冰箱可以继续获取冰箱中食物的图像，该图像中包括该建议放置位置。
S130：识别该建议放置位置中是否放置该食物。
具体地，冰箱可以通过图像识别算法或者预先训练好的图像识别模型，识别该图像中建议放置位置中是否放置有对应的食物，若该建议放置位置中未放置对应的食物，则执行如下步骤S131，否则继续执行S122。
S131：发出语音提示，以提示用户将该食物放置到该建议放置位置。
当识别出该建议放置位置未放置对应的食物时，冰箱可以发出语音提示，提示用户将食物放置到该建议放置位置，以便最大限度延长食物的保鲜时间。
在其他实施例中，冰箱还可以识别用户是将食物放入冰箱还是将食物取出冰箱，以便给出不同的建议。
具体如图9所示，本申请食物监控方法第六实施例是在本申请食物监控方法第一实施例的基础上，进一步包括：
S31：获取食物放入冰箱或者从冰箱中取出食物的过程图像。
具体地，冰箱中安装有图像获取装置，例如图3中顶部设置有一摄像头，在冰箱门被打开时，该摄像头被触发，进行多次拍摄，从而可以拍摄得到食物放入冰箱或者从冰箱中取出食物的过程图像，该过程图像包括多张图像。该摄像头也可以直接进行视频拍摄，进而可以从获取的视频中得到食物放入冰箱或者从冰箱中取出食物的过程图像。
S32：对过程图像进行处理，以识别过程图像中的食物拿取动作。
具体地，在一个应用例中，可以对该过程图像中相邻时刻的连续多张图像进行比较，例如对前一时刻拍摄和相邻后一时刻拍摄的两张图像进行处理，识别出食物的位置变化，或者拿取食物的手部位置变化，即可以识别出食物拿取动作是放入还是取出。例如，在初始阶段的图像中识别到手拿着食物，之后一段时间的图像中识别到手离开冰箱摄像头可视区域时手没有拿着食物，则可以从拍摄的这段图像帧中判定该拿取动作是放入；反之，识别到初始阶段的图像中手进入摄像头可视区域是没有拿食物，而一段时间后的图像中手离开摄像头可视区域时手拿着食物，则可以判定该拿取动作为取出。
在另一个应用例中，可以预先采集从冰箱中拿取食物的图像，包括放入和取出食物两种 类型的图像，作为样本，训练一基于深度学习的神经网络模型，以识别食物的拿取动作是放入还是取出。训练好该神经网络模型后，在获取到食物放入冰箱或者从冰箱中取出食物的过程图像后，将该过程图像直接输入到该神经网络模型，该神经网络模型即可以进行识别，之后直接输出拿取动作的识别结果是放入还是取出。
S33：判断该过程图像中的食物拿取动作是否为取出。
若识别出食物拿取动作为取出，则执行步骤S34。
S34：更新冰箱中保存的食物的数据，以给出冰箱中食物的进食优化建议或采购建议。
具体地，在对该过程图像进行处理时，还可以识别过程图像的单帧图像中的食物的信息，包括食物的类型、数量、成熟度等，例如当采用神经网络进行识别时，该神经网络模型可以是与步骤S121同一个的图像识别模型，该图像识别网络不仅可以识别食物的拿取动作，还可以识别食物的信息。进而在识别出用户的拿取动作为取出的情况下，可以更新冰箱中保存的食物的数据，以给出冰箱中食物的进食优化建议或采购建议等，以对冰箱中的食物进行智能管理。
可选地，对该过程图像进行处理时，可以识别过程图像中的食物的类型，进而在识别出食物拿取动作为放入时，可以给出储存提示。具体如图9所示，步骤S33之后，识别出食物拿取动作为放入的情况下，执行如下步骤S35。
S35：根据食物的类型，发出储存提示，以指引食物在冰箱中的建议放置位置。
具体地，对该过程图像进行处理后，识别出食物的拿取动作为放入的同时，还可以得到食物的信息(至少包括食物的类型)。由于冰箱中不同储存位置适合储存的食物类型可能不同，将食物放置到适合的储存位置可以最大程度的延长食物的保鲜时间。因此，根据该食物的类型，冰箱可以发出储存提示(例如灯光或语音提示)，以指引食物在冰箱中的建议放置位置，以便最大程度延长食物的新鲜度。
当然，在其他实施例中，步骤S33判定拿取动作为放入后，冰箱中食物的种类、数量也随之变化，此时也可以更新冰箱中保存的食物的数据。
当放入冰箱中的食物为多个同类食物时，为了减少存储量，可以选择一个最接近过期的成熟度作为代表进行监控。
具体如图10所示，本申请食物监控方法第七实施例是在本申请食物监控方法第一实施例的基础上，进一步限定步骤S12包括：
S41：判断对该图像进行识别是否识别出同类食物的不同个体的成熟度。
具体地，用户放入冰箱中的食物可能为多个，例如多个苹果，每个苹果均有其自身的成熟度。此时，若对获取的食物图像进行识别后，识别出同类食物的不同个体的成熟度，例如多个苹果的成熟度，则执行如下步骤S42，否则执行步骤S43。
S42：获取该同类食物的预过期成熟度范围，以不同个体的成熟度中成熟度最接近该预过期成熟度范围的成熟度作为同类食物的成熟度。
其中，该预过期成熟度范围为预先设定的食物即将过期的成熟度范围。针对不同食物类型，其取值可以不同。冰箱可以预先存储有不同类型食物对应的预过期成熟度范围，或者冰 箱可以从云端或网络搜索该同类食物的预过期成熟度范围。
S43：将识别到的单一个体的成熟度或不同个体的相同成熟度作为该同类食物的成熟度。
当不同个体的成熟度相同或者只有一个个体时，只需要将识别出的成熟度作为该类食物的成熟度，即可将识别到的单一个体的成熟度或不同个体的相同成熟度作为该同类食物的成熟度，继续执行步骤S13，根据食物的成熟度判断是否过期或即将过期。而当多个不同个体的成熟度不同时，必然存在一个最接近预过期成熟度范围的成熟度，此时，可以将该最接近预过期成熟度范围的成熟度作为该同类食物的成熟度，以避免由于同类食物的成熟度不合适导致某个个体过期而不提醒。此外，以最接近预过期范围的成熟度代表同类食物的成熟度，还可以减少冰箱需要存储的食物的成熟度数据，节省存储资源。
针对某些食物，其会产生某些气体，在即将过期时产生的气体浓度或成份等会变化，因此可以感测冰箱中的气体，以辅助识别食物的成熟度变化。
具体如图11所示，本申请食物监控方法第八实施例是在本申请食物监控方法第一实施例的基础上，进一步限定步骤S12之后，还包括：
S51：获取冰箱中气味传感器的数据。
其中，该气味传感器的数量可以为多个，设置在冰箱中的不同隔层中，以识别不同隔层中食物的气味。当然，该气味传感器也可以只有一个，感测整个冰箱中的气味。该气味传感器可以感测冰箱中的气味类型、浓度等信息，并生成数据。
S52：结合该气味传感器的数据及对食物图像进行识别的识别结果，得到食物的成熟度。
具体地，在一个应用例中，冰箱可以获取不同隔层中的气味传感器的数据，根据该数据可以得到冰箱中不同隔层放着的食物散发的气味的成份和浓度等信息，然后估计不同隔层的食物的大致成熟度，并与对食物图像的识别结果进行结合，以综合确定食物的成熟度。
本实施例中，结合气味传感器的数据进行食物成熟度的判断，可以对图像识别的结果进行补充，提高食物成熟度判断的准确性。
本申请食物监控方法上述实施例均可以与其他不冲突的实施例相结合。
如图12所示，本申请冰箱一实施例中，冰箱10包括：相互连接的图像获取装置101和处理器102。
其中，图像获取装置101可以是摄像头，其数量可以是一个或多个，其可以设置于冰箱10的顶部，也可以同时设置冰箱10顶部和内部。该图像获取装置10用于获取放入冰箱10中的食物的图像。
处理器102用于控制冰箱10的操作，例如控制图像获取装置101进行拍摄。该处理器102还可以称为CPU(Central Processing Unit，中央处理单元)。处理器102可能是一种集成电路芯片，具有信号的处理能力。处理器102还可以是通用处理器、数字信号处理器(DSP)、专用集成电路(ASIC)、现成可编程门阵列(FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
该处理器102具体用于执行指令以实现如本申请食物监控方法第一至第八任一实施例或 其不冲突的组合所提供的方法。
可选地，该冰箱10还包括与处理器102连接的气味传感器103。该气味传感器103用于感测冰箱10中的气味，并生成数据。
处理器102用于获取该气味传感器103的数据，并结合该数据及对食物图像进行识别的识别结果，得到食物的成熟度。
可选地，该冰箱10还包括与处理器101连接的语音设备104。该语音设备104可以包括麦克风和扬声器，用于与用户进行语音交互。
可选地，该冰箱10还包括与处理器102连接的存储器105。该存储器105用于存储处理器102执行所需的指令和数量，例如存储冰箱中放置的食物的信息。其中，该存储器105也可以与图像获取装置101、气味传感器103和语音设备104连接，以保存其获取的数据。
当然，在其他实施例中，该冰箱10还可以包括显示器(图未示)等其他部件，此处不做具体限定。
本实施例中，冰箱的处理器利用图像获取装置获取放入冰箱中的食物的图像后，对该图像进行识别，得到食物的成熟度，然后判断根据食物的成熟度判断是否过期或即将，在食物过期的情况下，发出过期提示信息，以提醒用户该食物过期，在食物即将过期的情况下，发出即将过期提示信息，以提醒用户该食物即将过期，从而使得用户只需要将食物放入冰箱中，冰箱就能够自动根据获取的食物图像判断食物是否过期或即将过期，并提示用户，而不需要用户手动设置过期时间等，简化了用户操作，实现冰箱对食物的智能监控，提高冰箱的智能化程度。
如图13所示，本申请具有存储功能的装置一实施例中，具有存储功能的装置20内部存储有程序指令210，该程序指令210被执行以实现如本申请食物监控方法第一至第八任一实施例或其不冲突的组合所提供的方法。
其中，具有存储功能的装置20可以是便携式存储介质如U盘、光盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)或者磁碟等可以存储程序指令的介质，也可以是移动终端、家电控制器或可集成于冰箱中的独立部件，例如控制芯片等，或者也可以为存储有该程序指令的服务器，该服务器可将存储的程序指令发送给其他设备运行，或者也可以自运行该存储的程序指令。
在一实施例中，具有存储功能的装置20还可以为如图11所示的存储器。
在本申请所提供的几个实施例中，应该理解到，所揭露的方法和装置，可以通过其它的方式实现。例如，以上所描述的装置实施方式仅仅是示意性的，例如，模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。 可以根据实际的需要选择其中的部分或者全部单元来实现本实施方式方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)或处理器(processor)执行本申请各个实施方式方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述仅为本申请的实施方式，并非因此限制本申请的专利范围，凡是利用本申请说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本申请的专利保护范围内。

Claims (20)

  1. 一种食物监控方法，其特征在于，包括：
    获取放入冰箱中的食物的图像；
    对所述图像进行识别，以得到所述食物的成熟度；
    获取所述食物的成熟度对应的食用边界日期；
    在所述食用边界日期或者所述食用边界日期之前的预设时间范围内，发出相应的提示信息，以提醒用户及时处理所述食物。
  2. 根据权利要求1所述的方法，其特征在于，所述获取所述食物的成熟度对应的食用边界日期包括：
    判断所述食物的成熟度是否在最佳食用时间所对应的成熟度范围内；
    响应于所述食物的成熟度在所述最佳食用时间所对应的成熟度范围内，获取所述食物的最佳食用边界日期；
    响应于所述食物的成熟度不在所述最佳食用时间所对应的成熟度范围内，获取所述食物的不能食用边界日期。
  3. 根据权利要求2所述的方法，其特征在于，所述在所述食用边界日期或者所述食用边界日期之前的预设时间范围内，发出相应的提示信息，以提醒用户及时处理所述食物包括：
    在所述最佳食用边界日期或者所述最佳食用边界日期之前的第一预设时间范围内，发出最佳食用提示信息，以提醒用户在所述食物保持在优良状态时进行食用；或者，
    在所述不能食用边界日期或者所述不能食用边界日期之前的第二预设时间范围内，发出即将过期提示信息，以提醒用户在所述食物即将过期之前进行处理。
  4. 根据权利要求3所述的方法，其特征在于，所述获取所述食物的不能食用边界日期之后，包括：
    判断所述食物的成熟度是否在所述最佳食用日期和所述不能食用边界日期所对应的成熟度范围之外；
    响应于所述食物的成熟度在所述最佳食用日期和所述不能食用边界日期所对应的成熟度范围之外，发出过期提示信息，以提醒用户所述食物过期；
    响应于所述食物的成熟度在所述最佳食用日期和所述不能食用边界日期所对应的成熟度范围之内，继续执行所述在所述不能食用边界日期或者所述不能食用边界日期之前的预设时间范围内，发出即将过期提示信息，以提醒用户在所述食物即将过期之前进行处理的步骤。
  5. 根据权利要求2所述的方法，其特征在于，
    所述获取所述食物的最佳食用边界日期包括：
    获取所述食物的成熟度曲线；
    从所述食物的成熟度曲线中获取所述最佳食用时间对应的所述最佳食用边界日期；
    所述获取所述食物的不能食用边界日期包括：
    获取所述食物的成熟度曲线；
    从所述食物的成熟度曲线中获取所述食物过期时间对应的所述不能食用边界日期。
  6. 根据权利要求2所述的方法，其特征在于，所述在所述食用边界日期或者所述食用边界日期之前的预设时间范围内，发出相应的提示信息，以提醒用户及时处理所述食物包括：
    在所述不能食用边界日期之前的第三预设时间范围内或在所述最佳食用边界日期之前的第四预设时间范围内，输出所述食物的进食优化建议。
  7. 根据权利要求1所述的方法，其特征在于，所述获取放入冰箱中的食物的图像包括：
    接收冰箱门开启的触发信号；
    响应所述触发信号，触发冰箱中的摄像头，拍摄食物放入冰箱过程中的图像。
  8. 根据权利要求1所述的方法，其特征在于，所述对所述图像进行识别，以得到所述食物的成熟度之后，包括：
    周期性拍摄冰箱中食物的图像；
    对不同时间获取的所述图像进行比较处理，以对冰箱中食物的变化进行跟踪记录。
  9. 根据权利要求1所述的方法，其特征在于，所述对所述图像进行识别，以得到所述食物的成熟度包括：
    将所述图像输入图像识别模型中，以获取所述图像识别模型输出的识别结果；
    获取所述识别结果中所述食物的成熟度；
    其中，所述图像识别模型是预先存储在所述冰箱中的基于深度学习的神经网络模型。
  10. 根据权利要求9所述的方法，其特征在于，
    所述获取所述识别结果中所述食物的成熟度之前，包括：
    判断所述图像识别模型是否成功识别；
    响应于所述图像识别模型成功识别，根据所述识别结果中所述食物的类型，发出储存提示，以指引所述食物在冰箱中的建议放置位置；
    所述获取所述识别结果中所述食物的成熟度之后，包括：
    获取所述建议放置位置对应的环境信息；
    结合所述环境信息和所述食物的类型，获取所述食物的成熟度曲线；
    结合所述成熟度曲线，继续执行所述获取所述食物的成熟度对应的食用边界日期的步骤。
  11. 根据权利要求10所述的方法，其特征在于，所述根据所述识别结果中所述食物的类 型，发出储存提示，以指引所述食物在冰箱中的建议放置位置之后，包括：
    获取冰箱中所述建议放置位置的图像；
    识别所述建议放置位置中是否放置所述食物；
    响应于所述建议放置位置中未放置所述食物，发出语音提示，以提示用户将所述食物放置到所述建议放置位置。
  12. 根据权利要求10所述的方法，其特征在于，所述判断所述图像识别模型是否成功识别之后，还包括：
    响应于所述图像识别模型识别失败，发出语音交互信息，提示用户确认放入或取出的食物类型，以提示用户将食物放入所述建议放置位置或更新冰箱中保存的食物的数据。
  13. 根据权利要求1所述的方法，其特征在于，进一步包括：
    获取食物放入冰箱或者从冰箱中取出食物的过程图像；
    对所述过程图像进行处理，以识别所述过程图像中的食物拿取动作；
    判断所述过程图像中的食物拿取动作是否为取出；
    响应于所述过程图像中的食物拿取动作为取出，更新冰箱中保存的食物的数据，以给出冰箱中食物的进食优化建议或采购建议。
  14. 根据权利要求1所述的方法，其特征在于，所述对所述图像进行识别，以得到所述食物的成熟度包括：
    判断所述对所述图像进行识别是否识别出同类食物的不同个体的成熟度；
    响应于识别出同类食物的不同个体的成熟度，获取所述同类食物的预过期成熟度范围；
    以所述不同个体的成熟度中成熟度最接近所述预过期成熟度范围的成熟度作为所述同类食物的成熟度。
  15. 根据权利要求1所述的方法，其特征在于，所述对所述图像进行识别，以得到所述食物的成熟度之后，还包括：
    获取冰箱中气味传感器的数据；
    结合所述气味传感器的数据及所述对所述图像进行识别的识别结果，得到所述食物的成熟度。
  16. 一种冰箱，其特征在于，包括：相互连接的图像获取装置和处理器；
    所述图像获取装置用于获取放入冰箱中的食物的图像；
    所述处理器用于执行指令以实现如权利要求1-15任一项所述的方法。
  17. 根据权利要求16所述的冰箱，其特征在于，还包括：与所述处理器连接的存储器，用于存储所述处理器执行所需的指令和数据。
  18. 根据权利要求16所述的冰箱，其特征在于，还包括：与所述处理器连接的气味传感器，用于感测所述冰箱中的气味，并生成数据；
    所述处理器用于获取所述数据，并结合所述数据及所述对所述图像进行识别的识别结果，得到所述食物的成熟度。
  19. 根据权利要求16所述的冰箱，其特征在于，还包括：与所述处理器连接的语音设备，用于与用户进行语音交互。
  20. 一种具有存储功能的装置，内部存储有程序指令，其特征在于，所述程序指令被执行以实现如权利要求1-15任一项所述的方法。
PCT/CN2019/123804 2018-12-18 2019-12-06 一种食物监控方法、冰箱及具有存储功能的装置 WO2020125456A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811549015.8A CN109631486A (zh) 2018-12-18 2018-12-18 一种食物监控方法、冰箱及具有存储功能的装置
CN201811549015.8 2018-12-18

Publications (1)

Publication Number Publication Date
WO2020125456A1 true WO2020125456A1 (zh) 2020-06-25

Family

ID=66075104

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/123804 WO2020125456A1 (zh) 2018-12-18 2019-12-06 一种食物监控方法、冰箱及具有存储功能的装置

Country Status (2)

Country Link
CN (1) CN109631486A (zh)
WO (1) WO2020125456A1 (zh)

Families Citing this family (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109631486A (zh) * 2018-12-18 2019-04-16 广东美的白色家电技术创新中心有限公司 一种食物监控方法、冰箱及具有存储功能的装置
CN110457004A (zh) * 2019-07-29 2019-11-15 上海秒针网络科技有限公司 提示信息的播放方法及装置
CN112394149B (zh) * 2019-08-13 2023-12-22 青岛海尔智能技术研发有限公司 食材成熟度检测提示的方法及装置、厨电设备
US11335010B2 (en) * 2019-08-15 2022-05-17 Haier Us Appliance Solutions, Inc. Methods for viewing and tracking stored items
CN110598591A (zh) * 2019-08-28 2019-12-20 卓尔智联(武汉)研究院有限公司 一种信息处理方法及装置、设备、存储介质
CN110659676A (zh) * 2019-09-09 2020-01-07 卓尔智联(武汉)研究院有限公司 一种信息处理方法、装置和存储介质
CN110942286A (zh) * 2019-11-27 2020-03-31 上海秒针网络科技有限公司 食品管理方法和装置、存储介质
CN110956217A (zh) 2019-12-06 2020-04-03 广东美的白色家电技术创新中心有限公司 食物成熟度识别方法、装置以及计算机存储介质
CN111258858A (zh) * 2020-01-07 2020-06-09 青岛海尔电冰箱有限公司 冰箱及其控制方法
CN113218889A (zh) * 2020-01-21 2021-08-06 青岛海尔电冰箱有限公司 水果的检测方法、冰箱及存储介质
CN113154784A (zh) * 2020-01-22 2021-07-23 青岛海尔电冰箱有限公司 食物提醒方法、冰箱、计算机可读存储介质
CN113154786B (zh) * 2020-01-22 2022-10-04 青岛海尔电冰箱有限公司 食物提醒方法、冰箱、计算机可读存储介质
CN111277671A (zh) * 2020-03-18 2020-06-12 天津商业大学 一种物联网智能家用冰箱
CN111486656B (zh) * 2020-04-08 2022-04-08 海信（山东）冰箱有限公司 一种冷藏设备及其控制方法
CN113633184B (zh) * 2020-04-27 2022-10-11 佛山市顺德区美的电热电器制造有限公司 烹饪器具、检测方法、检测装置和计算机可读存储介质
CN112393521B (zh) * 2020-12-04 2022-12-20 合肥荣事达电子电器集团有限公司 一种冰箱远程食物售卖和收购控制方法及冰箱
CN112432414A (zh) * 2020-12-11 2021-03-02 恩亿科(北京)数据科技有限公司 一种基于图像识别的冰箱鲜度管理方法与系统
US12063127B2 (en) 2021-02-15 2024-08-13 International Business Machines Corporation Smart refrigerator for smart consumption of food
CN112842062B (zh) * 2021-03-01 2022-05-17 广东美的厨房电器制造有限公司 烹饪设备的控制方法、装置、烹饪设备和可读存储介质
CN113915939B (zh) * 2021-03-25 2023-04-18 海信冰箱有限公司 冰箱及食材管理方法
CN114877611B (zh) * 2021-03-31 2023-09-29 青岛海尔电冰箱有限公司 提高图像识别准确率的方法、设备及冰箱
CN113701432A (zh) * 2021-08-11 2021-11-26 安徽康佳同创电器有限公司 一种具有催熟水果功能的多功能冰箱及其控制方法
US20230298351A1 (en) * 2022-02-14 2023-09-21 Springhouse Technologies Inc. Live inventory and internal environmental sensing method and system for an object storage structure
US20230316755A1 (en) * 2022-04-05 2023-10-05 Haier Us Appliance Solutions, Inc. Refrigerator appliance and methods for tracking stored items in a freezer chamber

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105444511A (zh) * 2014-08-29 2016-03-30 酷派软件技术（深圳）有限公司 一种冰箱的智能温控方法、装置及冰箱
CN106568294A (zh) * 2016-10-14 2017-04-19 青岛海尔特种电冰箱有限公司 冰箱的控制方法及冰箱
CN106996676A (zh) * 2016-01-25 2017-08-01 希姆通信息技术（上海）有限公司 冰箱管理系统
CN107024433A (zh) * 2016-02-02 2017-08-08 苏州三星电子有限公司 一种检测冰箱内食物新鲜度的方法及系统
CN107076675A (zh) * 2014-07-21 2017-08-18 三星电子株式会社 气体传感器、包括其的冰箱和为此的控制方法
CN107782059A (zh) * 2016-08-24 2018-03-09 南京乐朋电子科技有限公司 一种冰箱内食品保鲜提醒系统
CN108131879A (zh) * 2017-11-21 2018-06-08 珠海格力电器股份有限公司 冰箱及其清洁方法、装置和系统
CN108224894A (zh) * 2018-01-08 2018-06-29 合肥美的智能科技有限公司 基于深度学习的食材新鲜度识别方法、装置、冰箱和介质
CN108662842A (zh) * 2017-03-27 2018-10-16 青岛海尔智能技术研发有限公司 冰箱内食物的检测系统与冰箱
CN108759294A (zh) * 2018-07-14 2018-11-06 王长贵 一种冰箱智能识别和管理系统
CN109631486A (zh) * 2018-12-18 2019-04-16 广东美的白色家电技术创新中心有限公司 一种食物监控方法、冰箱及具有存储功能的装置

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN202281453U (zh) * 2011-09-16 2012-06-20 海尔集团公司 一种设有新鲜度检测装置的冰箱
KR20160000931A (ko) * 2014-06-25 2016-01-06 (주)한국크레딕라이프 냉장고에 보관되는 식품을 사물인터넷(IoT)을 통해 관리하는 시스템 및 이에 관한 냉장고

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107076675A (zh) * 2014-07-21 2017-08-18 三星电子株式会社 气体传感器、包括其的冰箱和为此的控制方法
CN105444511A (zh) * 2014-08-29 2016-03-30 酷派软件技术（深圳）有限公司 一种冰箱的智能温控方法、装置及冰箱
CN106996676A (zh) * 2016-01-25 2017-08-01 希姆通信息技术（上海）有限公司 冰箱管理系统
CN107024433A (zh) * 2016-02-02 2017-08-08 苏州三星电子有限公司 一种检测冰箱内食物新鲜度的方法及系统
CN107782059A (zh) * 2016-08-24 2018-03-09 南京乐朋电子科技有限公司 一种冰箱内食品保鲜提醒系统
CN106568294A (zh) * 2016-10-14 2017-04-19 青岛海尔特种电冰箱有限公司 冰箱的控制方法及冰箱
CN108662842A (zh) * 2017-03-27 2018-10-16 青岛海尔智能技术研发有限公司 冰箱内食物的检测系统与冰箱
CN108131879A (zh) * 2017-11-21 2018-06-08 珠海格力电器股份有限公司 冰箱及其清洁方法、装置和系统
CN108224894A (zh) * 2018-01-08 2018-06-29 合肥美的智能科技有限公司 基于深度学习的食材新鲜度识别方法、装置、冰箱和介质
CN108759294A (zh) * 2018-07-14 2018-11-06 王长贵 一种冰箱智能识别和管理系统
CN109631486A (zh) * 2018-12-18 2019-04-16 广东美的白色家电技术创新中心有限公司 一种食物监控方法、冰箱及具有存储功能的装置

Also Published As

Publication number Publication date
CN109631486A (zh) 2019-04-16

Similar Documents

Publication Publication Date Title
WO2020125456A1 (zh) 一种食物监控方法、冰箱及具有存储功能的装置
CN108852023B (zh) 一种可实现智能互联的蒸烤设备
CN107796172B (zh) 食物过期预警方法、装置、存储介质及食物储存设备
RU2650561C1 (ru) Способ и устройство для генерации информации
CN107423421A (zh) 菜谱推荐方法、装置和冰箱
CN106369910B (zh) 冰箱及冰箱控制方法
CN103884152B (zh) 提供健康指数的冰箱和系统
CN106773859B (zh) 一种智能烹饪控制方法
CN113551478B (zh) 一种食品剩余保质期的更新方法、装置及冰箱
CN112289407A (zh) 基于健康管理的餐饮管理的方法、系统、装置和存储介质
JP2021165604A (ja) 保管容器、冷蔵庫及び熟成度推定装置
CN111277671A (zh) 一种物联网智能家用冰箱
CN109780813A (zh) 一种冰箱的控制方法、装置、存储介质及冰箱
CN112069403A (zh) 菜谱推荐方法、装置、计算机设备和存储介质
CN108937554B (zh) 一种利用终端进行饮食提醒的蒸烤设备及方法
CN110020165A (zh) 一种饮食推荐方法和家庭机器人
CN113742579A (zh) 冰箱的食物推荐方法、食物推荐装置、存储介质及冰箱
CN114119552A (zh) 基于计算机视觉的密闭空间冷冻冷藏环境控制系统及方法
CN107798572A (zh) 用于智能冰箱的购物触发方法
WO2021147247A1 (zh) 冰箱菜品存储温度管理方法、冰箱及存储介质
CN117935506B (zh) 一种冰箱食材保鲜的提醒方法及系统
CN114486886A (zh) 食物检测方法、装置、存储介质及电子设备
CN108139317A (zh) 存储结构的对象识别
CN112100499A (zh) 一种基于数据模型的共享平台食材分配方法、系统及可读存储介质
CN110598591A (zh) 一种信息处理方法及装置、设备、存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19899537

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 03/11/2021)

122 Ep: pct application non-entry in european phase

Ref document number: 19899537

Country of ref document: EP

Kind code of ref document: A1