WO2022041992A1 - Refrigerator, display device, server, recommendation method, and message reminding method - Google Patents

Abstract

The present application discloses a refrigerator, a display device, a server, a recommendation method, and a message reminding method. The refrigerator comprises: a refrigerator body in which a storage chamber is provided, at least one food material being stored in the storage chamber; a refrigerator door disposed at an opening of the storage chamber; a display screen disposed on the refrigerator door and used for displaying a user interface; and a controller connected to the display screen and configured to: receive at least one recipe issued by a server according to existing food materials in the storage chamber, each recipe comprising at least one of the existing food materials; determine, according to current time and prestored time-related information of the existing food materials, the freshness of the existing food materials in the recipe; determine the recommendation priority of the recipe according to the freshness of the existing food materials in the recipe; and display, by taking the descending order of recommendation priorities of all recipes as the display order of the recipes, some or all of the recipes in the user interface.

Description

Refrigerator, display device, server, recommended method and message reminder method

This application claims the priority of the Chinese patent application filed with the China Patent Office on August 24, 2020, the application number is 202010855107.X, and the application name is "a display device, server, refrigerator and recommended method", and in 2021 The priority of the Chinese patent application filed with the Chinese Patent Office on May 25 with application number 202110606093.2 and the application name "Refrigerator and Information Recommendation Method", and filed with the Chinese Patent Office on May 28, 2021 with application number 202110617466.6, The priority of the Chinese patent application entitled "Refrigerator and Message Reminder Method" is claimed, the entire contents of which are incorporated herein by reference.

technical field

The present application relates to the technical field of image display, and in particular, to a refrigerator, a display device, a server, a recommendation method, and a message reminder method.

Background technique

With the popularization of AIOT (Artificial Intelligence Internet of Things), traditional home appliances also have the function of smart devices. Recipe recommendation based on the ingredients in the refrigerator is more and more popular among users.

In the implementation of some recommended recipes, the smart refrigerator usually first recognizes all the ingredients it stores, then generates various popular recipes containing the identified ingredients, and finally displays the popular recipes on the display screen of the refrigerator, or recommends the recipes in other related recipes. on an associated display device, such as a smart TV.

SUMMARY OF THE INVENTION

This application provides some refrigerators, display devices, servers, recommendation methods and message reminder methods.

In a first aspect, the application provides a refrigerator, comprising:

The box body is provided with a storage room inside, and at least one food material is stored in the storage room;

a box door, located at the opening of the storage room;

a display screen, arranged on the box door, for displaying a user interface;

A controller connected to the display screen is configured to:

receiving at least one recipe issued by the server according to the existing ingredients in the storage room, each recipe including at least one of the existing ingredients;

determining the freshness of each existing ingredient in the recipe according to the current time and the pre-stored time-related information of the existing ingredient;

Determine the recommendation priority of the recipe according to the freshness of each existing ingredient in the recipe;

A part or all of the recipes are displayed in the user interface by taking the descending order of the recommended priority of each recipe as the display order of each recipe.

In a second aspect, the present application further provides a method for pushing information, which is applied to a refrigerator, where the refrigerator includes a display screen, and the method includes:

receiving at least one recipe issued by the server according to the existing ingredients in the storage room, each recipe including at least one of the existing ingredients;

determining the freshness of each existing ingredient in the recipe according to the current time and the pre-stored time-related information of the existing ingredient;

Determine the recommendation priority of the recipe according to the freshness of each existing ingredient in the recipe;

A part or all of the recipes are displayed in the user interface by taking the descending order of the recommended priority of each recipe as the display order of each recipe.

In a third aspect, the application provides a refrigerator, comprising:

The box body has a storage room inside for storing ingredients;

a box door, located at the opening of the storage room;

a display, arranged on the box door, for displaying a user interface;

A communicator for communicating with the associated end device;

A controller connected to the display and the communicator is configured to:

Determining a target ingredient, the target ingredient at least includes an ingredient whose freshness satisfies a preset reminder condition determined according to time-related information corresponding to the existing ingredient in the storage room;

generating a reminder message according to the target ingredient, the reminder message including the ingredient name of the target ingredient;

Pushing the reminder message to at least one of the terminal devices to display the reminder message on at least one of the terminal devices.

In a fourth aspect, the present application further provides a message reminder method, which is applied to a refrigerator, and the method includes:

Determining target ingredients, the target ingredients at least include ingredients whose freshness satisfies the preset reminder condition determined according to the time-related information corresponding to the existing ingredients in the refrigerator;

generating a reminder message according to the target ingredient, the reminder message including the ingredient name of the target ingredient;

Pushing the reminder message to at least one associated terminal device to display the reminder message on at least one of the terminal devices.

In a fifth aspect, a display device is provided, comprising: a display screen configured to display a user interface; and a third controller configured to: in response to a user's operation on a recommended recipe application, store ingredients in a refrigerator according to the user's operation. The main ingredient determines the recommended recipe data that can be provided by the server, and the ingredients stored in the user's refrigerator include main ingredients and auxiliary ingredients; control the user interface to display a recommended recipe operation interface including controls according to the recommended recipe data, and the controls are used for After receiving the user's confirmation operation, jump to the recommended recipe details interface.

In a sixth aspect, a server is provided, comprising: a media repository for storing recipe data corresponding to ingredients; a second controller configured to: receive a recommendation request sent from a display device, and store in the media repository according to the The recommended recipe data that can be provided by querying the main ingredients of the ingredients stored in the user's refrigerator, the ingredients stored in the user's refrigerator are identified by the server as main ingredients or auxiliary ingredients; and the recommended recipe data is sent to the display device.

In a seventh aspect, a method for recommending recipes is provided, the method comprising: after a recipe recommendation application is started, sending a first recommendation request to the server, where the first recommendation request is used to make the server feed back the storage according to the user's refrigerator storage Recommended recipe data determined by the main ingredient in the ingredients; the user interface displays a recommended recipe operation interface including controls according to the recommended recipe data, and the controls are used to jump to the recommended recipe details interface after receiving the user's confirmation operation .

In an eighth aspect, a method for recommending recipes is provided, the method comprising: receiving a recommendation request sent from a display device, and querying available recommended recipe data according to main ingredients in the ingredients stored in the user's refrigerator, and the user's refrigerator stores ingredients Recognized by the server as a main ingredient or an auxiliary ingredient; and sending the recommended recipe data to the display device.

In a ninth aspect, a refrigerator is provided, comprising: a camera configured to capture an internal image of the refrigerator; a display screen configured to display a user interface; and a controller configured to: The ingredients stored in the user's refrigerator obtained from the image inside the refrigerator are sent to the server, and the ingredients stored in the user's refrigerator are identified by the server as main ingredients or auxiliary ingredients; in response to the user's operation on the recommended recipe application, according to the user's refrigerator storage ingredients determine the recommended recipe data that the server can provide; control the user interface to display a recommended recipe operation interface including controls according to the recommended recipe data, the controls are used to jump to the recommended recipe after receiving the user's confirmation operation Recipe details interface.

Description of drawings

In order to illustrate the embodiments of the present application more clearly, the accompanying drawings required in the embodiments will be briefly introduced below. Obviously, for those of ordinary skill in the art, without creative efforts Additional drawings can be obtained from these drawings.

1a is a perspective view of a refrigerator according to an exemplary embodiment of the present application;

Fig. 1b is a schematic diagram of the appearance of a refrigerator according to an exemplary embodiment of the present application;

FIG. 2 is a schematic diagram of the hardware configuration of a refrigerator according to an exemplary embodiment of the present application;

Fig. 3 is an ingredient management interface shown in the present application according to an exemplary embodiment;

FIG. 4 is another food material management interface shown in the present application according to an exemplary embodiment;

FIG. 5 is another food material management interface shown in this application according to an exemplary embodiment;

FIG. 6 is another food material management interface shown in this application according to an exemplary embodiment;

FIG. 7 is a recipe display interface shown in the present application according to an exemplary embodiment;

FIG. 8 is another food material management interface shown in the present application according to an exemplary embodiment;

FIG. 9 is a recipe detail interface shown in the present application according to an exemplary embodiment;

FIG. 10 is another food material management interface shown in this application according to an exemplary embodiment;

FIG. 11 is another recipe display interface shown in this application according to an exemplary embodiment;

FIG. 12 is a flowchart of a method for pushing information according to an exemplary embodiment of the present application;

Fig. 13 is a flowchart showing the steps of configuring a refrigerator controller according to an exemplary embodiment of the present application;

FIG. 14 is an application scenario shown in the present application according to an exemplary embodiment;

FIG. 15 is a user interface of a display device shown in the present application according to an exemplary embodiment;

FIG. 16 is a user interface of a refrigerator display according to an exemplary embodiment of the present application;

FIG. 17A shows a UI schematic diagram of a TV application according to an embodiment of the present application;

FIG. 17B shows a UI schematic diagram of displaying device recommended recipe data according to an embodiment of the present application;

FIG. 17C shows a UI schematic diagram of a display device recommending recipe data according to another embodiment of the present application;

17D shows a schematic diagram of an operation interface of recommended recipe data according to an embodiment of the present application;

FIG. 18 shows a schematic flowchart of a method for recommending recipes according to an embodiment of the present application;

FIG. 19 shows a schematic flowchart of a method for recommending recipes according to another embodiment of the present application;

FIG. 20A shows a schematic side view of the appearance of a refrigerator according to an embodiment of the present application;

FIG. 20B shows a schematic front view of a refrigerator according to an embodiment of the present application.

detailed description

In order to make the purpose and implementation of the present application clearer, the exemplary embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only the Some embodiments are claimed, but not all embodiments.

It should be noted that the brief description of the terms in the present application is only for the convenience of understanding the embodiments described next, rather than intending to limit the embodiments of the present application. Unless otherwise specified, these terms are to be understood according to their ordinary and ordinary meanings.

The terms "first", "second", "third", etc. in the description and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and are not necessarily meant to limit specific Sequential or sequential, unless otherwise noted. It is to be understood that the terms so used are interchangeable under appropriate circumstances.

The terms "comprising" and "having", and any variations thereof, are intended to cover but not exclusively include, for example, a product or device that incorporates a series of components is not necessarily limited to all components explicitly listed, but may include no explicit other components listed or inherent to these products or devices.

The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware or/and software code capable of performing the functions associated with that element.

Fig. 1a is a perspective view of a refrigerator according to an exemplary embodiment of the present application. The shape of the refrigerator provided by the embodiment of the present invention is approximately a rectangular parallelepiped. The appearance of the refrigerator is defined by the storage compartment 101 providing a limited storage space and a plurality of box doors 200 provided at the opening of the storage compartment 101 . The storage compartment 101 is a box body 100 with an opening. In the example shown in FIG. 1a, the storage compartment 101 includes a freezing compartment 101A located below and a refrigerating compartment 101B located above, and the freezing compartment 101A and the refrigerating compartment 101B respectively have one or more an independent storage space.

In some embodiments, freezer compartment 101A may be selectively covered by freezer drawer door 200A. The refrigerator compartment 101B is partitioned into left and right sides. The refrigerating compartment 101B is selectively opened or closed by a refrigerating compartment door 200B pivotally mounted on the refrigerating compartment 101B.

Fig. 1b is a schematic view of the appearance of a refrigerator according to an exemplary embodiment of the present application. As shown in Fig. 1b, a display screen is provided on the refrigerator compartment door 200B, and the display screen is embedded in the door body. A tag reading area is provided at a position below the display screen on the refrigerator compartment door 200B, and the tag reading area has a built-in antenna for reading RFID tags, so as to identify RFID tags at a short distance.

In some embodiments, the plane on which the screen of the display screen is located is flush with the outer surface of the box door.

As shown in FIG. 1a and FIG. 1b, the refrigerator provided by the embodiment of the present application is provided with at least one storage compartment inside the box, such as at least one of a freezing compartment, a refrigerating compartment, a changing room, and/or a fresh-keeping room. Each storage compartment can have multiple independent storage spaces, such as drawer-style compartments.

2 is a schematic diagram of the hardware configuration of a refrigerator according to an exemplary embodiment of the present application. As an example, the refrigerator 200 may include a display screen 210, a controller 220, a detector 240, a memory 250, a communicator 260, and several user input interfaces. The display screen 210, the antenna 230, the detector 240, and the memory 250 are coupled with the controller 220 through a communication interface. The display 210 is used for receiving the image signal output by the controller 220 , performing components for displaying video content and images and a menu manipulation interface, and displaying a user manipulation UI interface for controlling the refrigerator 200 .

The controller 220 may include one or more processing units, such as a system on a chip (SoC), a central processing unit (CPU), a microcontroller (MCU), a memory controller, and the like. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.

In some embodiments, the refrigerator further includes a plurality of antennas 230 for detecting RFID tags, and the controller 220 communicates with the antennas 230 through a serial port.

In some embodiments, the controller 220 includes an RFID module, the RFID module, the antenna 230 and the RFID tag constitute an RFID reading and writing system, the antenna 230 is used to transmit radio frequency signals between the RFID module and the RFID tag, and the RFID module completes the pairing process through the antenna 230. Read and write operations of RFID tags.

In some embodiments, the RFID module communicates with the antenna through a serial port.

The memory 250 may include one or more storage units, for example, may include volatile memory (volatile memory), such as: dynamic random access memory (dynamic random access memory, DRAM), static random access memory (static random access memory) , SRAM), etc.; can also include non-volatile memory (non-volatile memory, NVM), such as: read-only memory (read-only memory, ROM), flash memory (flash memory), etc. Wherein, different storage units may be independent devices, or may be integrated or packaged in one or more processors or communication interfaces, and become part of the processors or communication interfaces.

Program instructions and applications, such as voice applications, are stored in the memory 250 . The controller 220 can call program instructions in the memory 250 or run an application program, so that the refrigerator executes a related method, such as the method for scanning food labels provided in the embodiments of the present application.

The detector 240, including at least a sound collector, such as a microphone, may be used to receive the user's voice. Exemplarily, the voice signal includes a control instruction of the user to control the refrigerator 200 . The detector 240 may also include a box door switch state sensor, such as a magnetic sensitive switch, a mechanical switch, for detecting the switch signal of each box door, and when the switch state of the box door is recorded according to the detected switch signal, the switch is sent. Status information to the controller 220 . Exemplarily, when the detector detects an opening signal or a closing signal of any box door, it sends the latest switch state information to the controller 220 . The detector may further include an image collector, such as a camera, and the camera is used to record the putting-in/out-out time and storage location of the food ingredients.

The user input interface includes at least one of other input interfaces such as a microphone, a touch panel, a sensor, and a button. For example, the user can implement the user command input function through actions such as voice, touch, gesture, pressing, etc. The input interface converts the received analog signal into a digital signal, and converts the digital signal into a corresponding command signal, and sends it to the controller 220.

The communicator 260 is used to communicate with the server or the terminal device in the local area network.

In some embodiments, the controller 220 may include a voice recognition module, and the voice recognition module further includes a voice parsing unit and a voice instruction database, so that the refrigerator can independently complete the voice recognition of the voice data input by the user, and the recognized voice The process of matching content with voice commands in a voice command database.

The present application does not limit the number of antennas 230 . For example, the antenna 230 may include an antenna disposed in each storage chamber and an antenna disposed in a tag reading area, wherein the antenna disposed in each storage chamber is mainly used for scanning RFID tags in each storage chamber to read the RFID tags The antenna set inside the tag reading area is mainly used to scan the RFID tag on the tag reading area to read the tag information of the RFID tag.

In some embodiments, the power of the antennas disposed in different storage compartments may be different. In addition, the power of the antenna of each storage room can ensure that the antenna can recognize the ingredients included in its own compartment, so as to avoid that the false recognition rate will increase when the power is too high, and the ingredients in its own compartment will not be recognized when the power is too small. In a specific implementation, the power of the antenna can be determined in advance according to the space size of each storage room, and the power of each antenna can be set in the refrigerator. The larger the space, the higher the power.

In some embodiments, the antenna disposed under the tag reading area is referred to as the first antenna, and the antenna disposed in the storage compartment is referred to as the second antenna.

In order to realize the intelligent management of ingredients, RFID tags can be used to identify the ingredients in the storage room. By establishing the corresponding relationship between the RFID tag and the ingredient information, the virtual association between the RFID tag and the ingredient information is realized. By modifying the corresponding relationship between the RFID tag and the food information, the same RFID tag is associated with different food information successively, and the flexibility of using the RFID tag is improved. The ingredients with the RFID tags are stored in the refrigerator, the RFID tags in the storage room are detected by the second antenna to obtain the tag information, and the ingredients identified by the tags are determined according to the association relationship between the tag information and the ingredient information.

In some embodiments, the label information includes a label code, and a specific code segment in the label code is a label identifier. Establishing the corresponding relationship between the identification information of the RFID tag and the food material information can realize the virtual association between the RFID tag and the food material. Furthermore, after the label identification of the RFID tag in the storage room is obtained, the food material corresponding to the RFID tag can be determined according to the label identification and the pre-established correspondence.

In some embodiments, the refrigerator controller implements an ingredient management function by running an ingredient management application, and the application interfaces provided by the ingredient management application include an ingredient management interface, a tag editing interface, an ingredient library interface, an ingredient adding interface, and the like.

In some embodiments, in response to the refrigerator terminal being powered on or an instruction input by the user to instruct to start the ingredient management application, the controller starts the ingredient management application. After the ingredient management application is started, the user can input an instruction to trigger the display of the label editing interface in the ingredient management application, and in the display label editing interface, enter the ingredient information associated with the specified label, and the ingredient management application receives the user input and the specified label. After the ingredient information associated with the tag, establish an association relationship between the ingredient information and the tag identifier of the specified tag, for example, save the tag identifier and ingredient information in a local ingredient information collection.

In some embodiments, the ingredient information includes one or more of ingredient ID, ingredient name, storage location identifier, ingredient quantity, ingredient shelf life, start time, and the like. Wherein, the storage room in which the ingredients are stored can be determined according to the storage location identifier. For example, if the storage location identifier included in the ingredient information "apples" is "refrigerating room", it can be determined that the apples are stored in the refrigerating room. The freshness of the ingredients can be determined according to the current time, the shelf life of the ingredients and the starting time. The starting time can be the real production date of the ingredients input by the user, such as the production date of bread on May 1, 2021, or the refrigerator received After the user enters the ingredient information, the ingredient insertion time saved according to the information reception time is stored.

In some implementations, a freshness rating can be used as a quantitative indication of the freshness of an ingredient. Exemplarily, the freshness level includes, but is not limited to: "expired", "nearly expired", "fresh", etc. For example, it may also include: "severely expired", "just expired", "relatively fresh", " very fresh" etc.

In some implementations, the expiration time of the ingredient is first calculated according to the shelf life and the starting time of the ingredient, expiration time = starting time + shelf life; then the remaining time until the ingredient expires is calculated according to the current time and the expiration time, remaining time = expiration time - Current time; then determine the freshness level of the ingredients according to the ratio of the remaining time to the shelf life. Specifically, the freshness level corresponding to a certain ratio may be determined according to the corresponding relationship between the preset ratio and the freshness level.

Exemplarily, the corresponding relationship between the preset ratio and the freshness level is shown in the following table:

Remaining time/shelf life	freshness level
<0	expired
0～0.3	Expiring soon
>0.3	Fresh

Another example, the corresponding relationship between the preset ratio and the freshness level is shown in the following table:

Remaining time/shelf life	freshness level
<-0.3	seriously overdue
-0.3～0	expired
0～0.3	generally fresh
0.3～0.7	relatively fresh
>0.7	very fresh

In some embodiments, the ingredient information corresponding to the existing ingredients in the storage room is stored in the local ingredient information set. In addition, the local ingredient information collection can also store ingredient information corresponding to historical ingredients. Among them, the existing ingredients refer to the ingredients currently stored in the refrigerator, and the historical ingredients refer to the ingredients that were stored in the refrigerator in the past and are currently removed from the refrigerator. In specific implementation, the existing ingredients and historical ingredients can be distinguished by the storage status mark corresponding to the ingredient information in the local ingredient information set, for example, the ingredient information of the historical ingredient is marked as the removed state, and the ingredient information of the existing ingredient is marked as the put in state .

In some embodiments, after the ingredient management application is started, the information of the ingredients whose storage state is the put-in state is read from the local ingredient information set, so as to obtain ingredient information of the existing ingredients in the storage room. Then, the ingredient management interface is displayed according to the ingredient information of the existing ingredients. The ingredient management interface includes ingredient icons drawn according to the ingredient information corresponding to the existing ingredients in the storage room, so that the existing ingredients in the refrigerator are displayed in the form of icons, pictures and texts. . In addition, various functional controls, such as RFID tag editing controls, ingredient adding controls, recipe recommendation controls, etc., may also be displayed in the ingredient management interface for user operations to trigger the controller 220 to implement corresponding functions.

In some embodiments, the ingredient management interface may include multiple ingredient display areas, and one ingredient display area may correspond to one storage room. The ingredient management application determines the storage room where the existing ingredient is located according to the storage location identifier in the ingredient information of the existing ingredient, and displays the ingredient icon of the existing ingredient in the ingredient display area corresponding to the storage room where it is located, so as to display the storage room in the refrigerator. The food section is displayed. For example, the ingredient icons of the ingredients in the refrigerating chamber are displayed in the ingredient display area corresponding to the refrigerating chamber.

In other embodiments, the user may input an ingredient adding instruction by operating the ingredient adding control in the ingredient management interface. The ingredient management application displays an ingredient library interface in response to an ingredient addition instruction input by the user, and the ingredient library interface displays a large number of selectable ingredient icons, each ingredient icon corresponding to an ingredient information. The user can select the ingredient to be added by inputting the selection operation of an ingredient icon. On the one hand, the selected ingredient icon can be added to the ingredient management interface, and on the other hand, the ingredient information corresponding to the selected ingredient icon can be added to the local. A collection of ingredient information and mark it as put.

The above process of adding ingredients by inputting an ingredient adding instruction is at least applicable to the following scenarios: after the user stores real ingredients (such as apples) in the refrigerator, the user operates the control for adding ingredients in the ingredient management interface to trigger the display of ingredients in the ingredient library interface. In the library interface, select the ingredient icon (eg apple icon) corresponding to the real ingredient (eg apple), add the selected ingredient icon to the ingredient management interface, and at the same time add the ingredient information associated with the ingredient icon to the local ingredient information collection. In this way, there is no need to attach RFID tags to real ingredients, and the corresponding ingredient information can also be added to the local ingredient information collection, and the corresponding ingredient icons can also be displayed in the ingredient management interface, so as to realize the intelligent management of ingredients in the refrigerator. .

In some implementations, the ingredient library interface includes a tab switch bar and a content area, the tab switch bar includes multiple tab controls, and when different tab controls are selected, the content area will display ingredient icons in different ranges. For example, when the different tab controls are selected, ingredient icons for different categories of ingredients are displayed in the content area. In this way, the user can first select the label control of the corresponding category according to the category of the ingredient to be added to trigger the display of the icon of the ingredient of the corresponding category in the content area, and then select the ingredient icon to be added from the ingredient icons displayed in the content area. ingredients icon.

In some embodiments, the ingredient management application may also draw a corresponding ingredient icon containing an explicit freshness identifier according to the freshness identifier corresponding to the existing ingredient. As the name implies, the dominant freshness indicator is a display representation of the freshness of the ingredients, and the freshness of the ingredients is the freshness of the ingredients. It should be noted that this application does not limit the style of the freshness explicit mark, which can be a text mark, a color mark that directly indicates the freshness level, or a graphic mark that represents the relationship between the remaining days and the shelf life, such as progress. Bars can also be various combinations of text, colors, and graphics.

In some embodiments, for a smart refrigerator with a camera function, the ingredients in the refrigerator can be recorded by taking pictures of the ingredients in the refrigerator, and the corresponding ingredients in the collected images can be parsed and identified by the controller, or sent to the The server does the parsing and identification.

For example, when the refrigerator door is closed, the camera is not enabled to detect images; when the refrigerator door is opened, the camera is triggered to turn on and take pictures of the inside of the refrigerator. The controller sends the data collected by the camera to the server, and the server analyzes and recognizes the captured pictures, and feeds back the specific ingredients in the refrigerator. The controller is also used to record when the ingredients are first put in, and the shelf life of the stored ingredients.

In some embodiments, since both the RFID module and the camera are used to record food data, one can be applied to the refrigerator.

FIG. 3 is a user interface shown in the present application according to an exemplary embodiment, which is specifically an exemplary ingredient management interface. Referring to FIG. 3 , the ingredient management interface includes an RFID tag editing control 301, a recommended recipe control 302, an ingredient adding control 306, and also includes an ingredient display area corresponding to each storage room, such as an ingredient display area 303 corresponding to a refrigerator room, and a corresponding change room. The food material display area 304 and the food material display area 305 corresponding to the cold greenhouse. Each ingredient display area displays an ingredient icon corresponding to each ingredient in the storage room, such as ingredient 1, ingredient 2, and ingredient 3 in ingredient display area 303, ingredient 4 and ingredient 5 in ingredient display area 304, and ingredient display area. Ingredient 6 and ingredient 7 in 305. Each ingredient icon includes an ingredient image and ingredient name.

In some embodiments, the ingredient management application determines the freshness of the existing ingredients at preset time intervals according to the current time, the shelf life and the starting time of the existing ingredients. After determining the freshness of the existing ingredients, use the corresponding freshness identifier to mark the corresponding ingredient information in the local ingredient information set, or update the previously marked freshness identifier. Exemplarily, the shelf life of "apple" in the local ingredient information set is 30 days, and the starting time is May 1. On May 20, the freshness indicator corresponding to the ingredient information corresponding to "apple" is "fresh". The controller recalculated the freshness level of "apple" at 0:00 on May 21, and obtained the latest freshness level as "expiring soon", and updated the freshness identifier corresponding to "apple" in the local ingredient information collection as "Expiring soon".

Based on this, in some embodiments, the ingredient icon may include, in addition to the image of the ingredient and the name of the ingredient, an explicit identifier of the freshness of the ingredient. As the name implies, the dominant freshness indicator is a display representation of the freshness of the ingredients, and the freshness of the ingredients is the freshness of the ingredients. During specific implementation, the ingredient management application draws a corresponding ingredient icon containing an explicit freshness identifier according to the freshness identifier corresponding to the existing ingredient during the process of displaying the ingredient management interface. It should be noted that this application does not limit the style of the freshness explicit mark, which can be a text mark, a color mark that directly indicates the freshness level, or a graphic mark that represents the relationship between the remaining days and the shelf life, such as progress. Bars can also be various combinations of text, colors, and graphics.

FIG. 4 is another food material management interface shown in this application according to an exemplary embodiment. Different from the ingredient management interface shown in FIG. 3 , in FIG. 4 , each ingredient icon also includes an explicit freshness indicator. Among them, the freshness of Ingredient 1 and Ingredient 12 is explicitly marked as the word "Expired" displayed on the picture of the ingredient, indicating that the freshness level of Ingredient 1 and Ingredient 12 is "Expired"; the freshness of Ingredient 4 and Ingredient 6 The dominant mark is a text mark and a progress bar with the content "2 days remaining" displayed below the image of the ingredients. The progress of the progress bar represents the remaining time or the ratio of the remaining time to the shelf life, indicating that the remaining two days of the ingredients 4 and 6 expire. That is, it is about to expire; the freshness of the remaining ingredients is explicitly marked as a progress bar, and the progress of the progress bar represents the remaining time or the ratio of the remaining time to the shelf life.

In some embodiments, when the ingredient management application displays the ingredient management interface according to the ingredient information of the existing ingredients in the local ingredient information set, according to the freshness identifiers and storage location identifiers corresponding to the existing ingredients, the existing ingredients in each storage room are sorted. The icon of the ingredients is displayed in the corresponding ingredient display area, wherein the ingredient icons of the existing ingredients in each storage room are arranged in the order of decreasing freshness in the corresponding ingredient display area.

In some embodiments, the controller 220 detects the switch signal of each of the cabinet doors separately through the detector 240, records the switch state of the cabinet door according to the detected switch signal, and analyzes the user's operation of the cabinet door according to the switch state information behavior. When it is detected that the last opened door is closed, the RFID tag in the storage room is detected to obtain the identification information of the RFID tag; according to the identification information of the RFID tag in the storage room, the storage status of the ingredient information in the local ingredient information collection is updated . Specifically, the identification information of the RFID tag in the storage room is compared with the identification information in the local food material information set to determine the latest removed label and the latest inserted label; the storage state corresponding to the latest removed label is marked as the removed state, and the latest removed label is marked as the removed state. The storage state corresponding to the put-in label is marked as the put-in state. In this way, the storage state of the ingredient information in the local ingredient information set can be updated in time according to the existing ingredients in the refrigerator.

In other embodiments, the user may input an ingredient adding instruction by operating the ingredient adding control in the ingredient management interface. The ingredient management application displays an ingredient library interface in response to an ingredient addition instruction input by the user, and the ingredient library interface displays a large number of optional ingredient icons, each ingredient icon corresponding to an ingredient information. The user can select the ingredient to be added by inputting the selection operation of an ingredient icon. On the one hand, the selected ingredient icon can be added to the ingredient management interface, and on the other hand, the ingredient information corresponding to the selected ingredient icon can be added to the local. A collection of ingredient information and mark it as put.

The above process of adding ingredients by inputting an ingredient adding instruction is at least applicable to the following scenarios: after the user stores real ingredients (such as apples) in the refrigerator, the user operates the control for adding ingredients in the ingredient management interface to trigger the display of ingredients in the ingredient library interface. In the library interface, select the ingredient icon (eg apple icon) corresponding to the real ingredient (eg apple), add the selected ingredient icon to the ingredient management interface, and at the same time add the ingredient information associated with the ingredient icon to the local ingredient information collection. In this way, there is no need to attach RFID tags to real ingredients, and the corresponding ingredient information can also be added to the local ingredient information collection, and the corresponding ingredient icons can also be displayed in the ingredient management interface, so as to realize the intelligent management of ingredients in the refrigerator. .

In some implementations, the ingredient library interface includes a tab switch bar and a content area, the tab switch bar includes multiple tab controls, and when different tab controls are selected, the content area will display ingredient icons in different ranges. For example, when the different tab controls are selected, ingredient icons for different categories of ingredients are displayed in the content area. In this way, the user can first select the label control of the corresponding category according to the category of the ingredient to be added to trigger the display of the icon of the ingredient of the corresponding category in the content area, and then select the ingredient icon to be added from the ingredient icons displayed in the content area. ingredients icon.

In some embodiments, the ingredient management application displays the ingredient management interface while displaying the ingredient management interface according to the existing The ingredients display recommended recipe information to the user.

In some implementations, the ingredient management application reads the ingredient information whose storage state is in the put-in state from the local ingredient information set, that is, reads out ingredient information of the existing ingredients in the refrigerator; sends a recipe recommendation request to the server, and the recipe recommendation request Include at least the ingredient name of the existing ingredient. After receiving the recipe recommendation request sent by the ingredient management application, the server matches the recipe according to the ingredient name of the existing ingredient, obtains at least one recipe including one or more existing ingredients, and returns the matched recipe information to the ingredient management application. After receiving the recipe information returned by the server, the ingredient management application displays the recipe returned by the server.

It is worth noting that the recipe recommendation request may also include the first recommended quantity and/or the second recommended quantity, where the first recommended quantity is used to limit the quantity of recipes containing the same ingredient, and the second recommended quantity is used to limit the delivery by the server. The total number of recipes. After receiving the recipe recommendation request sent by the ingredient management application, the server matches the recipe to be recommended according to the ingredient name of the existing ingredient. The first recommended quantity, and/or the principle that the total number of recipes is not greater than the second recommended quantity, select a certain number of recipes from the recipes to be recommended and return them to the ingredient management application. Since the same ingredient can correspond to multiple different recipes, the application can limit the total number and/or diversity of recipes returned by the server through the first recommended quantity and/or the second recommended quantity, so as to avoid unnecessary data volume caused by excessive data volume. Waste of resources while improving user experience.

In some embodiments, the recipe recommendation request may further include user information. The user information may include user characteristic tags and user preference tags. User characteristic tags such as ordinary users, ordinary member users, diamond member users, etc. User's taste preferences, such as "no spicy food", "light health care", etc.

After receiving the recipe recommendation request, the server may determine the recipe library corresponding to the user characteristic tag according to the user characteristic tag therein, and then match the ingredient name of the existing ingredient with the corresponding recipe library. In this way, when the recipe libraries corresponding to different user characteristic characteristic tags are different, the selectable ranges of recommended recipes for users with different characteristics such as ordinary users, ordinary member users, and diamond member users are different. The user preference tag provides the server with a reference dimension for recommending recipes for users. While recommending recipes to users based on existing ingredients, it can also recommend recipes that meet their taste preferences. For example, after matching a large number of recipes to be recommended according to the ingredient names of the existing ingredients, and according to the taste preference represented by the user preference tag, a recipe that meets the user's taste preference is further screened from the recipes to be recommended.

In some embodiments, the recipe information returned by the server includes, but is not limited to, the recipe name, cover image, the ingredient name and amount of the ingredients involved, the recipe introduction and the playback address of the recipe video, and the purchase link of the ingredient.

In some embodiments, the ingredient management interface includes a plurality of ingredient display areas corresponding to respective storage compartments, and further includes a recipe display area. After receiving the M recipe information returned by the server, the ingredient management application displays N recipe controls in the recipe display area according to the N recipe information with the highest recommendation priority in the M recipe information, and one recipe control corresponds to one recipe information. Load the recipe name and/or recipe cover in the recipe control, N≤M. Wherein, N is less than or equal to the preset number of display positions for recipe controls in the recipe display area, and the preset number of display positions depends on the size of the recipe display area, the size of the recipe controls, and the distance between the recipe controls.

FIG. 5 is an ingredient management interface shown in some exemplary embodiments of the present application. On the basis of the ingredient management interface shown in FIG. 3, the ingredient management interface further includes a recipe display area 510. The recipe display area 510 displays recipe controls 511-515 generated according to the recipe AE returned by the server. The corresponding recipe name and cover image are displayed. The user can click any recipe control to view the detailed content of the recipe, and the user can click the page turning button ">" to view more recipes.

In other embodiments, after receiving the M recipe information returned by the server, the ingredient management application displays a pop-up window on the ingredient management interface, and displays N recipe controls in the pop-up window according to N recipe information in the M recipe information , one recipe control corresponds to one recipe information, the recipe name and/or recipe cover are loaded in the recipe control, N≤M. Wherein, N is less than or equal to the preset number of display positions for recipe controls in the recipe display area, and the preset number of display positions depends on the size of the recipe display area, the size of the recipe controls, and the distance between the recipe controls.

FIG. 6 is an ingredient management interface shown in some exemplary embodiments of the present application. On the basis of the ingredient management interface shown in FIG. 3 and FIG. 4 , a pop-up window 610 is displayed in the lower part of the ingredient management interface. The pop-up window 610 displays a close button and recipe controls 611-615 generated according to the recipe AE returned by the server. , the corresponding recipe name and cover image are displayed in the recipe control. The user can click any recipe control to view the detailed content of the recipe, and can click the close button to close the pop-up window.

In some embodiments, the user may input a recipe recommendation instruction by operating the recommended recipe control in the ingredient management interface. In response to the input recipe recommendation instruction, the ingredient management application reads the ingredient information whose storage state is the put-in state from the local ingredient information set, that is, reads out the ingredient information of the existing ingredients in the refrigerator; and sends the information to the server that includes at least the existing ingredients. The recipe recommendation request for the name of the ingredient. After receiving the recipe recommendation request sent by the ingredient management application, the server matches the recipe according to the ingredient name of the existing ingredient, obtains at least one recipe including one or more existing ingredients, and returns the matched recipe information to the ingredient management application. After receiving the recipe information returned by the server, the ingredient management application enters the recipe display interface from the ingredient management interface, and displays the recipe returned by the server in the recipe display interface. As mentioned above, these embodiments are different from the previous embodiments in that the ingredient management application provides an independent recipe display interface, and the recipe returned by the server is displayed in the recipe display interface.

FIG. 7 is a recipe display interface shown in some exemplary embodiments of the present application, which may specifically be an interface displayed after the user clicks on the recommended recipe interface when the ingredient management interface shown in FIG. 3 is displayed. As shown in FIG. 7 , the recipe display interface displays a return button and recipe controls 701-710 generated according to recipes A-J returned by the server, and the recipe controls display the corresponding recipe name and cover image. The user can click any recipe control to view the detailed content of the recipe, and can click the back button to return to the ingredient management interface.

In some embodiments, after receiving the recipe information returned by the server, the ingredient management application determines the freshness of each existing ingredient in each recipe according to the current time and the pre-stored time-related information of the existing ingredient; The freshness of each existing ingredient in each recipe is based on the principle that the lower the overall freshness of all the existing ingredients in the recipe, the higher the recommendation priority corresponding to the recipe, the recommendation priority of each recipe is determined. Finally, the display order of each recipe is in descending order of the recommended priority of each recipe, and some or all of the recipes are displayed in the ingredient management interface. The time-related information of the ingredients includes the shelf life and start time of the ingredients, and the start time is the production time or the insertion time of the ingredients.

In some implementations, the ingredient management application searches the local ingredient information set for the freshness identifier corresponding to each ingredient according to the ingredient name of the existing ingredient in the recipe information, so as to determine the freshness level of each existing ingredient according to the freshness identifier. Alternatively, the ingredient management application searches the local ingredient information collection for the shelf life and starting time corresponding to each existing ingredient according to the ingredient name of the existing ingredient in the recipe information; and then according to the current time and the shelf life and starting time corresponding to each existing ingredient time to determine the freshness level of each existing ingredient. Next, the ingredient management application determines the weight value corresponding to each existing ingredient according to the freshness level of each existing ingredient in each recipe, wherein the weight values corresponding to different freshness levels are different; and then calculates the corresponding weight value of all existing ingredients in each recipe. The sum of the weight values is used to reflect the overall freshness of all the existing ingredients in the recipe; finally, the recipes are sorted based on the corresponding addition results of each recipe to determine the recommended priority of each recipe.

In one example, the recipe information returned by the server includes recipe A-recipe E, and the existing ingredients included in recipe A-recipe E respectively are shown in the following table:

menu	existing ingredients
A	ingredient 1, ingredient 2
B	Ingredients 3
C	ingredient 1, ingredient 3
D	ingredients 4
E	ingredients 3, ingredients 5

According to the ingredient names of ingredients 1-5 included in recipe A-recipe E, look up the freshness labels of ingredients 1-5 from the local information collection, and determine the freshness levels of ingredients 1-5 as shown in the following table:

Ingredients	freshness level
Ingredients 1	Fresh
ingredients 2	Expiring soon
Ingredients 3	expired
ingredients 4	Expiring soon

ingredients 5

Fresh

The preset correspondence between the freshness level and the weight value is shown in the following table, where the lower the freshness level, the greater the corresponding weight value.

freshness level	Weights
Fresh	1
Expiring soon	5
expired	15

The weight values corresponding to the ingredients in recipes A-E are summed up, and the results are as follows:

menu	Summation result
A	1+5=6
B	15
C	1+15=16
D	5
E	15+1=16

In this example, the lower the freshness level, the higher the corresponding weight value, and the summation result corresponding to the recipe can reflect the overall freshness of the existing ingredients in the recipe, so the overall freshness based on the existing ingredients in the recipe is lower. , the corresponding recommendation priority is higher, and the recipes AE are arranged according to the descending order of the summation results corresponding to the recipes AE, and the recommended priority order of the recipes AE can be determined as follows:

menu	Recommended priority
C	1
E	2
B	3
A	4
D	5

As can be seen from the above example, the priority of the recommended recipes is determined according to the freshness of the existing ingredients, and the recipes AE are displayed in the order of the recommended priorities shown in the table above, which can remind users to eat ingredients with low freshness in time , and give priority to displaying corresponding recipes for users to avoid spoilage and waste of ingredients and improve user experience.

FIG. 8 is an ingredient management interface shown in some exemplary embodiments of the present application. Different from the ingredient management interface shown in FIG. 6 , in the ingredient management interface, the display order of recipes A-E in the pop-up window 810 is recipe C, recipe E, recipe B, recipe A, recipe D.

In some embodiments, the ingredient management application displays a recipe detail interface in response to a user's selection operation (eg, a click operation) on the displayed recipe control. Among them, the detailed content of the corresponding recipe is displayed in the recipe details interface. FIG. 9 is a recipe details interface shown in some exemplary embodiments of the present application. As shown in FIG. 9 , the interface displays a cover image 901 of a recipe, a recipe name 902 , an ingredient list area 903 and a recipe introduction area 904 , among them, the ingredients and amounts involved in the recipe are listed in the ingredient list area, and the method steps are displayed in the method introduction area.

It is worth noting that, in other embodiments, the ingredient management application, in response to the user's selection operation on the displayed recipe control, jumps to the recipe video playback page according to the recipe video playback address in the recipe information, and further, the user can click the recipe in the recipe. Watch how-to videos on the video playback page. In some embodiments, the pop-up window on the ingredient management interface or the independent recipe display interface further includes an expired ingredient reminder area corresponding to each storage room, and the expired ingredient reminder area displays the expired ingredients in the corresponding storage room. ingredients icon. Specifically, while displaying the recipe control according to the recipe information returned by the server, the ingredient management application determines each existing ingredient in each storage room according to the storage location identifier, time-related information or freshness identifier of the existing ingredient in the local ingredient information set. The freshness of the ingredients is displayed, and the icon of the ingredients corresponding to the "expired" existing ingredients in each storage room is displayed in the corresponding expired ingredient reminder area.

FIG. 10 is an ingredient management interface shown in some exemplary embodiments of the present application. As shown in FIG. 10 , on the basis of the ingredient management interface shown in FIG. 8 , the pop-up window further includes a plurality of expired ingredient reminder areas , which are respectively an expired food reminder area 101 corresponding to the refrigerating room, an expired food reminder area 102 corresponding to the changing room, and an expired food reminder area 103 corresponding to the cooling room. Each expired ingredient reminder area displays the expired existing ingredients in the corresponding storage room. For example, the expired ingredient reminder area 101 displays the expired existing ingredients 11-13 in the refrigerator compartment, and the expired ingredient reminder area 102 displays Expired existing ingredients 14-15 in the freezing chamber, expired ingredients 16-17 in the changing chamber are displayed in the expired ingredient reminder area 103.

FIG. 11 is a recipe display interface shown in some exemplary embodiments of the present application. As shown in FIG. 11 , on the basis of the recipe display interface shown in FIG. 9 , a plurality of expired ingredient reminder areas are further included below the recipe control. , which are respectively an expired food reminder area 111 corresponding to the refrigerating room, an expired food reminder area 112 corresponding to the freezing room, and an expired food reminder area 113 corresponding to the changing room. Each expired ingredient reminder area displays the expired existing ingredients in the corresponding storage room. For example, the expired ingredient reminder area 111 displays the expired existing ingredients 11-13 in the refrigerator compartment, and the expired ingredient reminder area 112 displays Expired existing ingredients 14-15 in the freezing chamber, expired ingredients 16-17 in the changing chamber are displayed in the expired ingredient reminder area 113.

In some embodiments, the recipe includes a main ingredient and an auxiliary ingredient, the main ingredient is generally an ingredient in a larger amount, and the auxiliary ingredient generally refers to an ingredient or a condiment in a smaller amount. After obtaining the recommended priority ranking of each recipe, the ingredient management application determines the top N recipes to be displayed according to the recommended priority of each recipe, where N≤M, where M is the total number of recipes delivered by the server. Then, the frequency n2 of each main ingredient in the first N recipes in the N recipes is counted. If the frequency n2 of a main ingredient in the N recipes is greater than n1, it is determined that the main ingredient is a high-frequency main ingredient. For high-frequency main ingredients, randomly delete (n2-n1) recipes containing the high-frequency main ingredients, and make the (N+1)th to [N+(n2-n1)]th recipes to be supplemented forward until it is to be displayed The frequency of each main ingredient in the top N recipes is not greater than n1.

In one example, the total number of recipes delivered by the server is M=10, N=5, the frequency of each main ingredient in these 5 recipes is 2, and the main ingredients of the first 5 recipes to be displayed are shown in the following table:

menu	main ingredient
1	ingredient 1, ingredient 2
2	Ingredients 1
3	ingredient 1, ingredient 3
4	ingredient 2, ingredient 4
5	ingredients 2

As shown in the above table, the frequency of occurrence of each main ingredient in the 5 recipes is counted, and the frequency corresponding to each main ingredient is obtained as shown in the following table:

Ingredients	frequency
1	3
2	3
3	1
4	1

It can be determined that the frequency of ingredient 1 and ingredient 2 is greater than 2, so ingredient 1 and ingredient 2 are high-frequency main ingredients. For Ingredient 1 and Ingredient 2, (3-2) recipes containing Ingredient 1 and/or Ingredient 2 need to be deleted randomly, for example, Recipe 1 is deleted, or Recipe 2 and Recipe 5 are deleted. If Recipe 1 is deleted, the recipe with the sixth recommended priority will be supplemented upward, and the frequency and subsequent process of each main ingredient in the latest top 5 recipes will be re-determined. If Recipe 2 and Recipe 5 will be deleted, the recommended Recipes with priorities 6-7 are supplemented upwards, and the frequency of each main ingredient in the latest top 5 recipes and the subsequent process are re-determined until the frequency of each main ingredient in the top 5 recipes to be displayed is no longer. greater than 2, so as to avoid that most of the top N recipes displayed to the user are recipes using the same ingredients, and improve the user experience.

It can be seen from the above embodiments that the present application provides a refrigerator, including a display screen and a controller, the controller is configured to: receive at least one recipe issued by the server according to the existing ingredients in the storage room, and each recipe includes at least one recipe. Existing ingredients; determine the freshness of each existing ingredient in the recipe according to the current time and the pre-stored time-related information of the existing ingredient; according to the freshness of each existing ingredient in the recipe, based on the overall The lower the freshness is, the higher the recommendation priority corresponding to the recipe is, the recommendation priority of each recipe is determined; the descending order of the recommended priority of each recipe is used as the display order of each recipe, and some or all recipes are displayed in the user interface. . Based on the refrigerator provided by this application, the priority of the recommended recipes can be determined according to the freshness of the existing ingredients, the user can be reminded to eat the ingredients with lower freshness in time, and the corresponding recipes can be displayed for the user first, so as to avoid the spoilage and waste of the ingredients, Improve user experience.

Based on the refrigerator provided by the above embodiment, the embodiment of the present application further provides a method for pushing information, specifically a method for pushing recipe information for a user on the display screen of the refrigerator, as shown in FIG. 12 , the method may include:

S101: Receive at least one recipe issued by the server according to the existing ingredients in the storage room, where each recipe includes at least one of the existing ingredients.

In some embodiments, before the receiving the at least one recipe delivered by the server according to the existing ingredients in the storage room, the method further includes: reading the existing ingredients in the storage room from the local ingredient information set, the existing ingredients In the local ingredient information, the storage state is an ingredient in the state of being put in; send a recipe recommendation request to the server, the recipe recommendation request at least includes the ingredient name of the existing ingredient, and the recipe recommendation request is used to make the server according to the The ingredient name of the existing ingredient matches a recipe that includes at least one of the existing ingredient.

S102: Determine the freshness of each existing ingredient in the recipe according to the current time and the pre-stored time-related information of the existing ingredient.

In some embodiments, the existing food material has a freshness identifier used to represent freshness; the time-related information includes a shelf life and a start time, and the start time is the time when the food material is put into the storage room or the time of the food material. production time; the method further includes: at preset time intervals, determining the freshness of the existing ingredients according to the current time, the shelf life and starting time of the existing ingredients; according to the freshness of the existing ingredients The freshness indicator of the existing food item is updated. Determining the freshness of each existing ingredient in the recipe according to the current time and the pre-stored time-related information of the existing ingredient includes: according to the ingredient name of each existing ingredient in the recipe, from local ingredients. The freshness identifier corresponding to each existing ingredient is searched in the information set, so as to determine the freshness of each existing ingredient according to the freshness identifier.

In some embodiments, the time-related information includes a shelf life and a start time, and the start time is the time when the food was put into the storage room or the production time of the food, and the time according to the current time and the pre-stored existing Time-related information of the ingredients, and determining the freshness of each existing ingredient in the recipe, including: according to the ingredient name of each existing ingredient in the recipe, searching for the shelf life and start date corresponding to each existing ingredient from the local ingredient information collection start time; determine the expiration time of the existing food according to the shelf life and start time corresponding to the existing food; calculate the difference between the expiration time of the existing food and the current time, according to the difference and the The ratio of the shelf life of the existing ingredients determines the freshness level of the existing ingredients.

S103: Determine a recommendation priority of the recipe according to the freshness of each existing ingredient in the recipe.

In some embodiments, the weight value corresponding to each existing ingredient is determined according to the freshness level of each existing ingredient in the recipe, and the weight values corresponding to different freshness levels are different; The sum of the weight values is used to sort each recipe based on the sum corresponding to each recipe, so as to determine the recommendation priority of each recipe.

S104: Display some or all of the recipes in the user interface by using the descending order of the recommended priorities of the recipes as the display order of the recipes.

In some embodiments, N recipe controls are displayed in the recipe display area, and the recipe names and recipe covers of the N recipes with the highest recommended priority among the M recipes returned by the server are sequentially loaded on the N recipes In the control, N≤M, and N is less than or equal to the preset number of display positions for the recipe controls in the recipe display area;

In some embodiments, the method of the present application further includes: in response to the user's selection operation on the displayed target recipe control, displaying the detailed content of the corresponding recipe.

In some embodiments, the existing ingredients have storage location identifiers used to represent storage compartments; the user interface further includes a plurality of reminder areas for specific ingredients corresponding to each storage compartment, and the method of the present application further includes: in the recipe While displaying the recipe controls in the display area, according to the storage location identification and freshness corresponding to the existing ingredients, the ingredients icons of the existing ingredients in each storage room whose freshness meets the preset reminder conditions are displayed in the corresponding specific ingredients. In the food reminder area.

In some embodiments, the method of the present application further includes: displaying on the user interface a plurality of food material display areas corresponding to each storage room, and according to the freshness identification and storage location identification corresponding to the existing food ingredients, each storage room is displayed. The ingredient icons of the existing ingredients in the room are displayed in the corresponding ingredient display area, wherein the ingredient icons of the existing ingredients in each storage room are arranged in the order of decreasing freshness in the corresponding ingredient display area.

In some embodiments, the user can select an ingredient icon in the ingredient library interface through a single-click or long-press operation. In response to the user's selection of the ingredient icon in the ingredient library interface, the ingredient management application directly adds ingredient information corresponding to the ingredient icon selected by the user to the local ingredient information set, and marks it as a storage state. In some embodiments, the refrigerator controller periodically executes the following steps shown in FIG. 13 by running an ingredient management application, that is, every preset time interval, executes the following steps shown in FIG. 13 , the preset time interval The interval is the preset reminder period. As shown in Figure 13:

S201: Determine a target ingredient, where the target ingredient at least includes an ingredient whose freshness satisfies a preset reminder condition determined according to time-related information corresponding to the existing ingredient in the storage room.

In some embodiments, the preset reminder condition is "expired", that is, the target ingredient is an expired ingredient. In other embodiments, the preset reminder conditions are "expired" and "expiring soon", that is, the target ingredients include expired ingredients and ingredients that are about to expire.

In some embodiments, the target ingredient further includes an ingredient corresponding to the ingredient name included in the unread historical reminder message. For the convenience of distinction, the target ingredient determined according to the time-related information corresponding to the existing ingredients in the storage room may be referred to as the first expired ingredient, and the expired ingredient determined according to the unread historical reminder messages may be referred to as the second expired ingredient.

In some embodiments, the first expired food item is the newly generated expired food item in the storage room, for example, the expired food item produced in the last reminder period. In this way, it is possible to avoid repeated reminders for the same expired ingredient, which may cause confusion to the user. Wherein, the expired ingredients in the existing ingredients in the storage room include at least the first expired ingredients.

In some embodiments, the ingredient information in the local ingredient information set has a reminder state identifier, and the reminder state identifier is divided into a first identifier and a second identifier. The first identifier is a default reminder state identifier for each ingredient information. After the ingredient management application generates a reminder message according to an expired ingredient, it modifies the reminder status identifier of the ingredient information of the expired ingredient in the local ingredient information set from the first identifier to the second identifier, that is, the first identifier marked. The ingredients are the ingredients that have not been reminded, and the ingredients marked by the second identification are the ingredients that have been reminded. Based on this, as an implementation manner of S201, first determine the expired ingredients in the existing ingredients according to the time-related information or the freshness identifier of the existing ingredients in the storage room; and then determine the expired ingredients whose reminder status is identified as the first identifier as The first expired food. That is to say, the first expired ingredient determined in S201 refers to an expired ingredient whose reminder status is identified as the first identifier among all expired ingredients, that is, an expired ingredient that has not been reminded.

In addition, since the ingredient management application periodically executes S201-S203 to remind the user to consume the expired ingredients in the refrigerator in time, the expired ingredients whose expiration time is longer than the reminder period should be reminded ingredients, and the expiration time is less than or equal to the expiration of the reminder period The ingredients should be the ingredients that have not been reminded. Based on this, as another implementation of S201, first determine the expired ingredients in the existing ingredients and the expiration time of each expired ingredient according to the time-related information or the freshness identifier of the existing ingredients in the storage room; Or the expired ingredient equal to the reminder period is determined as the first expired ingredient. Wherein, expiration time = current time - expiration time.

In the above embodiment, the ingredient management application first reads the ingredient information of the existing ingredient from the local ingredient information set, for example, reads the ingredient information whose access state is the put state. Then, the expired ingredients in the existing ingredients are determined according to the time-related information in the ingredient information of the existing ingredients, and the expired ingredients are the ingredients whose expiration time is earlier than the current time. In other implementations, the ingredient management application first reads ingredient information of existing ingredients from the local ingredient information set. Then, the expired ingredients in the existing ingredients are determined according to the freshness identifier in the ingredient information of the existing ingredients, and the expired ingredients are the ingredients whose freshness identifier is "expired". The method for calculating the expiration time and the method for marking the freshness identifier may refer to the foregoing embodiments, and details are not described here.

In some embodiments, the second expired ingredient is the ingredient indicated by the unread reminder message before the preset time cut-off point, and the ingredient indicated by the unread reminder message is the ingredient name contained in the unread reminder message or the ingredient corresponding to the ingredient ID. The preset time cut-off point refers to a time point earlier than the current time point by a preset time interval. For example, if the current time point is 8:00 on May 24, 2021, and the preset time interval is 3 days, then the preset time cutoff point is 8:00 on May 21, 2021. It is worth noting that, in the implementation situation where the preset time interval is 0, the second expired ingredients include expired ingredients indicated by all unread reminder messages in the notification application. Based on this, as an implementation manner of S201, the unread historical reminder messages before the preset time cut-off point in the notification application are obtained, and the preset time cut-off point refers to a time point earlier than the preset time interval of the current time point; according to The name of the ingredient contained in the historical reminder message determines the target ingredient indicated by the historical reminder message.

It should be noted that the notification application may be a system application installed on the refrigerator independent of the food management application, or may be a functional module integrated in the food management application. The notification application is used to receive notification messages generated in the system, reminder messages or notification messages sent by other applications, and display the received notification messages or reminder messages in the user interface. For example, a reminder message sent by an ingredient management application is received, and the reminder message is displayed in the user interface.

In some embodiments, all expired ingredients in the storage compartment are identified as target ingredients.

S202: Generate a reminder message according to the target ingredient, where the reminder message includes an ingredient name of the target ingredient.

In some embodiments, a reminder message is generated according to the first expired ingredient and/or the second expired ingredient, and the generated reminder message includes the ingredient name of the first expired ingredient and/or the second expired ingredient.

It can be seen that, according to the embodiment in which the reminder message is generated for the first expired ingredient, it can be ensured that only one reminder is performed for each expired ingredient, and the repeated reminder strategy can be prevented from causing trouble to the user. After the reminder message is generated according to the first expired ingredient, the reminder state identifier corresponding to the first expired ingredient is modified from the first identifier to the second identifier.

In addition, due to the limitations of refrigerator usage scenarios, users usually do not view every reminder message in time, resulting in historical reminder messages accumulated in the notification application and ignored by the user, thus failing to achieve the purpose of reminding the user. According to the embodiment of generating the reminder message for the second expired ingredient, the problem caused by the unread historical reminder message on the refrigerator end for a long time can be solved, and the user is reminded to consume the expired ingredient in time while avoiding trouble to the user.

In S202, a corresponding reminder message can be generated according to each target ingredient, that is, a target ingredient corresponds to a separate reminder message, and the reminder message corresponding to each target ingredient at least includes the name of the ingredient corresponding to the target ingredient. It is also possible to generate a unique reminder message according to all the target ingredients, and the reminder message includes the ingredient names of all the target ingredients. It should be understood that the reminder message may also correspond to the expiration time, expiration time, and/or the ID of the target ingredient.

S203. Push the reminder message to at least one associated terminal device, so as to display the reminder message on the at least one associated terminal device.

The terminal devices associated with the refrigerator include terminal devices located in the same local area network or the same group network as the refrigerator, such as display devices, portable mobile terminal devices, and PC devices in a home. The terminal device associated with the refrigerator may also be a terminal device associated with the refrigerator based on a service account.

Exemplarily, the account logged in by the ingredient management application on the refrigerator is called the main business account. Then, the terminal device associated with the refrigerator may be a terminal device bound with the main business account, such as a mobile phone, a TV, a tablet computer and other terminal devices bound with the main business account. It may also be a terminal device corresponding to a business account associated with the main business account, such as a mobile phone, a computer, and the like corresponding to the WeChat account associated with the main business account. It can also be other terminal devices that log in to the main business account. For example, when a user installs an ingredient management application on the TV and uses the main business account to log in the ingredient management application on the TV, the TV is called the terminal device associated with the refrigerator.

In some implementations, S203 specifically includes: the ingredient management application sends a message push request to the server, where the message push request includes the main business account and the reminder message generated in S202. After receiving the message push request, the server searches for the terminal device associated with the refrigerator according to the main service account, and sends a reminder message to the terminal device associated with the refrigerator.

In some implementations, S203 specifically includes: sending a device discovery message to the terminal device in the local area network, and determining whether there is an active device in the local area network according to whether a response message sent by the terminal device is received.

It should be understood that an active device includes at least a terminal device in a powered-on state, and may also include a terminal device in an incomplete standby state, such as a terminal device in an off-screen state while some data processing functions and communication functions are in a working state.

FIG. 14 is an application scenario exemplarily shown in this application. As shown in FIG. 14 , the scenario includes LAN I and refrigerators and terminal devices A-E in LAN I, where terminal device A, terminal device B, and terminal device C are active devices in LAN I. Based on the local area network I, the ingredient management application on the refrigerator can send device discovery messages through the communicator, such as UDP (User Datagram Protocol, User Datagram Protocol) broadcast messages and/or multicast messages. The active devices A-C can receive the device discovery message sent by the refrigerator, and send a response message, such as a UDP unicast message, to the ingredient management application on the refrigerator according to the IP address of the refrigerator in the device discovery message. When the ingredient management application sequentially receives the response messages sent by the active devices A-C within the preset waiting time period (5S), it can be determined that the active devices A-C exist in the local area network.

In some implementation manners, the device discovery message includes a UDP broadcast message carrying a device discovery identifier, and the UDP broadcast message includes the local IP address, that is, the IP address of the refrigerator terminal. After receiving the UDP broadcast message sent by the refrigerator, the active device in the local area network confirms that the message is a device discovery message according to the device discovery identifier in the UDP broadcast message, and parses the refrigerator from the UDP broadcast message in response to confirming that the message is a device discovery message. The IP address of the UDP broadcast message is sent to the refrigerator according to the parsed IP address, and the response message includes the IP address of the active device.

In other implementation manners, the device discovery message includes a multicast message sent to the terminal device in the target network, and the multicast message carries the device discovery identifier and the local IP address. The target network refers to the network where the refrigerator terminal is located. After receiving the multicast message sent by the refrigerator, the active device in the target network confirms that the message is a device discovery message according to the device discovery identifier in the multicast message, and parses the message from the multicast message in response to confirming that the message is a device discovery message The IP address of the refrigerator is obtained, and a response message to the multicast message is sent to the refrigerator according to the resolved IP address, and the response message includes the IP address of the active device.

In other implementations, the device discovery message includes both the above-mentioned UDP broadcast message and multicast message, that is, the ingredient management application simultaneously sends the UDP broadcast message to the terminal device in the local area network and the multicast message to the terminal device in the target network. After receiving the UDP broadcast message or the multicast message, the active device in the local area network or the active device in the target group network sends a response message to the ingredient management application according to the IP address of the refrigerator in the UDP broadcast message or the multicast message.

In some embodiments, the terminal devices in the target network are all or part of the terminal devices in the local area network.

When there are active devices in the local area network, push a reminder message to at least one active device. When there is no active device in the local area network, the reminder message is displayed on the display through a notification application.

In some embodiments, the IP addresses of the respective active devices are parsed from the response messages sent by the respective active devices, and then according to the IP addresses of the respective active devices, the reminder messages generated in S202 are sent to the respective active devices in a UDP unicast manner. Each active device, so that each active device displays the reminder message on its screen after receiving the reminder message sent by the refrigerator.

It can be understood that the terminal devices in the local area network include display devices such as smart TVs or smart large screens. After receiving the reminder message sent by the refrigerator end, the terminal device displays the received reminder message in the user interface according to the preset display logic and method. It should be noted that the present application does not limit the logic and manner in which the terminal device displays the reminder message.

For example, after receiving the reminder message sent by the refrigerator, the display device may display the reminder message in the current user interface in the form of a modeless dialog box, so as to remind the user to consume the expired ingredients in time without affecting the user's viewing or operation of the current User Interface. FIG. 15 is a user interface of a display device exemplarily shown in the application. As shown in FIG. 15 , the user interface includes a video playback screen 701 and a pop-up window prompt 702 displayed at the upper right corner of the upper layer of the video playback screen 70. The pop-up window prompt 702 displays the text content of the reminder message, specifically "watermelon and yogurt have expired, please eat them in time".

In some embodiments, each terminal device in the local area network or target network corresponds to a priority, and different terminal devices correspond to different priorities. In S203, the ingredient management application determines the preset priority of each active device according to the device information of each active device, such as the IP address, and determines the active device with the highest priority as the target device. Then, according to the IP address of the target device, the reminder message generated in S202 is sent to the target device, so as to display the reminder message in the user interface of the target device, thereby realizing the purpose of reminding the user to consume expired ingredients in time.

With reference to the application scenario shown in FIG. 14 , it is assumed that the priorities of terminal devices A-E in LAN I are terminal device A, terminal device B, terminal device C, terminal device D, and terminal device E in descending order. After determining that there is an active device AC in the local area network according to the received response message, the food management application determines that the active device A is the target device according to the priority corresponding to the active device AC, and then sends a reminder message to the active device A according to the IP address of the active device A. .

In some embodiments, the notification application is further configured to receive a preset operation on the displayed reminder message input by the user, such as a viewing operation, a clearing operation, and the like. When the notification application receives the aforementioned preset operation, it will mark the reminder message as read or clear. After clearing the reminder message, delete the reminder message operated by the user. Otherwise (ie, in the case where the aforementioned preset operation is not received), the state of the reminder message is kept as the unread state or the reminder message is retained.

It should be noted that this application does not limit the logic and manner in which the notification application displays the reminder message. For example, the notification application can immediately display the reminder message on the current user interface when receiving the reminder message, or display the received reminder in the notification bar or notification message interface when the user opens the notification bar or notification message interface Alert message. Fig. 16 is a schematic diagram of a refrigerator display user interface exemplarily shown in this application. As shown in Fig. 16 , a notification message box 802 and a "clear" button are displayed on the upper layer of the ingredient management interface 801, and a reminder is displayed in the notification message box 802 The text content of the message is specifically "watermelon and yogurt have expired, please eat in time". The user can click the "Clear" button to cancel the display of the notification message box, and instruct the notification application to clear the reminder message. The user can also click on the notification message box to view the message details.

Based on the refrigerator provided by the embodiment of the present application, the target ingredient can be determined according to the newly generated expired ingredient in the storage room and the ingredient indicated by the unread historical reminder message. Reminding the expired ingredients indicated by the unread historical reminder message and/or the newly generated expired ingredients in the refrigerator can avoid repeating multiple reminders for the same expired ingredient and affecting the user experience. Then, a reminder message is generated according to the target ingredients, and active devices in the local area network are actively discovered at the same time. When an active device is found in the local area network, a reminder message is pushed to at least one active device, and other active devices are used to remind users to consume expired ingredients in time, not limited to reminding on the refrigerator side, and no need to rely on cloud platforms and external networks, flexibility Powerful and resource-saving. When there is no active device in the local area network, push the reminder message to the notification application.

According to the refrigerator provided in the above embodiment, the embodiment of the present application further provides a message reminding method, the execution subject of the method includes but is not limited to the controller of the above refrigerator.

In some embodiments, the recommended recipe can be completed by the interaction between the refrigerator and the server, and one or more other terminal devices (including but not limited to: smart TVs, set-top boxes, refrigerators, microwave ovens, ovens, and other household appliances including display screens) can also be into the interaction. No matter how several terminal devices interact with the server, they are finally displayed on their corresponding displays, so as to be more intuitively provided to the user, and to improve the correlation between the ingredients in the refrigerator and the user.

In some embodiments the server has a second controller, the smart TV has a third controller, and the smart TV third display. It should be noted that the content displayed on any terminal device can be presented on other terminal devices through a local area network, a wide area network, a wired connection, or the like.

FIG. 17A shows a UI schematic diagram of a TV application according to an embodiment of the present application.

The figure shows the application UI interface of the TV displayed on the display screen. For example, the application UI interface includes 4 applications installed on the TV, namely news headlines, theater on-demand, recommended recipes, and K songs. Different applications, or other function buttons can be selected by moving the focus on the display using a controller such as a remote control.

In some embodiments, the TV display screen is configured to display other interactive controls while displaying the application UI interface, and the interactive controls may include, for example, TV homepage controls, search controls, message button controls, mailbox controls, browsing Controls, Favorites Controls, Signal Bar Controls, etc.

In order to improve the convenience and visualization of the TV UI, in some embodiments, the third controller included in the display device in the embodiments of the present application controls the TV UI in response to the operation of the interactive control. For example, when a user clicks a search control through a controller such as a remote control, the search UI can be displayed on top of other UIs, that is, the UI of the application component that controls the mapping of the interactive control can be enlarged, or run and displayed in full screen.

In some embodiments, the interactive controls are also operable via sensors, which may be, but are not limited to, acoustic input sensors, such as microphones, that can detect voice commands that include an indication of the desired interactive control. For example, the user may identify a desired interactive control, such as a recommended recipe control, using "recommended recipe" or any other suitable identification, and may also describe the desired action to be performed in relation to the desired interactive control. The third controller may recognize the voice command and submit data characterizing the interaction to the UI or its processing component or engine.

In some embodiments, the user can control the focus of the display screen through the remote control, select the recommended recipe application, so that its icon is highlighted on the user interface of the TV display screen; then by clicking on the highlighted icon, the user can achieve Open the icon-mapped application.

It should be noted that the icons and texts of the UI interface in the embodiments of the present application are only used as examples to illustrate the technical solution of recommended recipes. The icons and texts of the UI interface in the accompanying drawings can also be implemented as other contents, which are not described in the accompanying drawings of the present application. specific restrictions.

FIG. 17B shows a UI schematic diagram of a display device recommending recipe data according to an embodiment of the present application.

The display device provided by the present application includes a display screen, and the display screen is used to display a user interface, and the user interface can provide interactive functions for the user. different functions. In some embodiments, the display screen of the display device may be implemented as the third display 280 of the drawings.

The display device provided by the present application further includes a third controller, the third controller controls the user interface to display a recommended recipe operation interface including controls according to the determined recommended recipe data, and the controls are used for after receiving the user's confirmation operation to jump to the recommended recipe details interface.

The following will describe the technical solution for implementing the recommended recipe by the display device provided by the present application.

When the user starts the recommended recipe application, the third controller of the display device, in response to the user's operation on the recommended recipe application, determines the recommended recipe data that the server can provide according to the main ingredients in the ingredients stored in the user's refrigerator, and the user's refrigerator stores the recommended recipe data. The ingredients include main ingredients and auxiliary ingredients.

In some embodiments, the server provided by the present application includes a second controller, and the second controller identifies the user's refrigerator storage ingredients sent from the refrigerator as main ingredients and auxiliary ingredients.

Wherein, the auxiliary ingredients may be implemented as green onions, and/or ginger, and/or garlic, and the main ingredients are the ingredients stored in the user's refrigerator other than the auxiliary ingredients. For example, when the smart refrigerator recognizes that the stored ingredients include: fish, beef, tomato, green pepper, onion, ginger, and garlic, the server provided in this application determines the onion, ginger, and garlic as auxiliary ingredients; , tomatoes, and green peppers are determined as the main ingredients.

The smart refrigerator can automatically identify the ingredients stored in the user's refrigerator, and upload the identified ingredients to the server provided in this application. When accessing the server, the recommended recipe application in the display device provided by the present application can read the stored ingredients in the user's refrigerator that have been saved by the server. It should be noted that the main ingredients and auxiliary ingredients can be preset according to actual conditions, and the specific contents thereof are not limited in this application.

In some embodiments, the foodstuffs stored in the user's refrigerator include the names of the foodstuffs, the length of time for which each foodstuffs are stored, and the like. In practical applications, the information of the ingredients in the refrigerator can be obtained through the refrigerator. For example, the server provided in this application can send a query request to the user's refrigerator in a wired or wireless manner. After receiving the query request sent by the server, the refrigerator can scan the ingredients on the The information of the ingredients stored in the refrigerator is obtained by means of the RFID tag of the refrigerator, or the image acquisition module takes pictures, and the information of the ingredients is fed back to the server in a wired or wireless manner.

In some embodiments, the refrigerator can obtain the information of the stored food in real time through the following methods. First, the controller of the refrigerator acquires an image of the storage room, and processes the image of the storage room to obtain the grid position; then, the refrigerator acquires the user's hand trajectory, and selects the corresponding fitting trajectory based on the movement trend of the hand trajectory , the fitting trajectory is the hand trajectory at the time of item access; according to the fitting trajectory and grid position, the item access position is calculated and obtained, so that the storage or retrieval of ingredients in the refrigerator can be monitored in real time, and the user's refrigerator storage can be obtained. Detailed information of the ingredients, such as the location of the ingredients in the refrigerator, the storage time, the remaining days of the shelf life, etc.; finally, the refrigerator uploads the ingredients stored in the user's refrigerator to the server provided in this application.

In some embodiments, the recommended recipe data includes attribute information of the recipe, wherein the attribute information of the recipe includes main ingredients required by the recipe, dish taste, cooking method, suitable crowd, suitable scene, cuisine, dish, efficacy, and Properties elements such as required kitchen tools. In practical applications, the recipes and the attribute information of the recipes are associated and stored in the media resource library of the server provided by this application, so that they can be accessed and used.

In some embodiments, the server provided by this application includes a media repository, where the media repository is used to store recipe data corresponding to ingredients. For example, the media repository of the server has a built-in dictionary of correspondence between ingredients and recipes, and the dictionary contains the names of all known ingredients and the correspondence between ingredients and all recipes containing the ingredients. After obtaining the ingredient information of all the ingredients in the refrigerator, the recipe corresponding to the ingredient name is obtained from the dictionary according to the name of the ingredient in the ingredient information. The recipe corresponding to each ingredient name may be one or several.

For example, assuming that the ingredients in the refrigerator include onions and cucumbers, the corresponding recipes for onions and cucumbers can be obtained through the above dictionary lookup. Further, after the recipes corresponding to the onions and the cucumbers are obtained, the recommended recipe data is obtained by obtaining the union between the recipes corresponding to the onions and the recipes corresponding to the cucumbers.

In some embodiments, the third controller, in response to the user's operation on the recommended recipe application, determining the recommended recipe data that can be provided by the server according to the main ingredients in the ingredients stored in the user's refrigerator includes, and the third controller responds to the The startup confirmation operation of the recommended recipe application sends a first recommendation request to the server, and the first recommendation request is used to make the server feed back the determined recommended recipe data, and the recommended recipe data is determined by the server according to the identified recipe data. The main ingredient in the ingredients stored in the user's refrigerator is determined; then, the third controller receives the recommended recipe data sent by the server.

When the display device starts the recommended recipe application, the third controller will send a first recommendation request to the server to obtain recommended recipe data. That is, the server determines the recommended recipe data that can be provided to the display device according to the main ingredients in the ingredients stored in the user's refrigerator uploaded by the user's refrigerator. It should be noted that in the process of generating the recommended recipe data, the second controller of the server stores the user's refrigerator storage The auxiliary ingredients in the ingredients are filtered out, that is, the onion, ginger, and garlic in the above-mentioned embodiment are filtered out, so that the server can generate recommended recipe data based on the main ingredients that are more in line with the user's needs, and can avoid the server from generating the onion, ginger, and garlic based on the auxiliary ingredients. Recommended recipes, but the main ingredients required in the recommended recipes are not stored in the user's refrigerator.

Continuing to refer to FIG. 17B , after determining the recommended recipe data that the server can provide, the third controller of the display device controls the user interface to display a recommended recipe operation interface including controls according to the recommended recipe data, the controls are used for receiving user After confirming the operation, jump to the recommended recipe details interface.

The second controller of the server receives the recommendation request sent from the display device, and queries the available recommended recipe data in the media repository according to the main ingredients of the ingredients stored in the user's refrigerator, and the ingredients stored in the user's refrigerator are stored by the server Identify the main ingredient or auxiliary ingredient; and send the recommended recipe data to the display device.

For example, the second controller of the server receives the first recommendation request sent from the display device, and the first recommendation request indicates that the display device is turned on and online, that is, it can be considered that as long as the display device starts the recommended recipe application, the server will receive The first recommendation request sent by the display device is displayed; then the second controller determines the available recommended recipe data in the media repository according to the received main ingredient query in the ingredients stored in the user's refrigerator, and sends the recommended recipe data to the display screen.

In some embodiments, the user's smart refrigerator reports the information of the ingredients stored in the user's refrigerator to the server in the cloud, and the second controller of the server or the recommended recipe business system will regularly query every day to determine the main ingredient information of the current refrigerator, and according to the The main ingredient information is queried to determine the recommended recipe data, and the recommended recipe data is saved. When the recommended recipe application of the display device requests the recommended recipe data, the saved recommended recipe data is sent to the display device to improve the response speed.

In some embodiments, the third controller controlling the user interface to display the recommended recipe operation interface including controls according to the recommended recipe data includes, the third controller includes the first recommended recipe data and the second recommended recipe data when the received recommended recipe data includes the first recommended recipe data and the second recommended recipe. When the recipe data is used, a mapping relationship is constructed according to the preset matching characteristics of different users bound by the display device and the recipes in the recommended recipe data, a user list is generated based on the mapping relationship, and a focus position in the user list is generated according to the The user at the location controls the second recommendation window to display the recommended recipe data mapped by the user, wherein the user list and the second recommendation window are simultaneously displayed in the user interface.

As shown in FIG. 17B , when the recommended recipe data generated by the server includes the first recommended recipe data and the second recommended recipe data, that is, when it includes multiple recommended recipe data, the third controller can use the display device to bind different users' The preset matching features and the recipes in the recommended recipe data build a mapping relationship, and a user list is generated based on the mapping relationship, that is, the multiple recipes produced are classified according to different user preset matching features, and the recommended recipes are more Targeted to improve the fit of user needs.

For example, the user list in the figure includes 3 users: father, mother, Dabao Dabaoda, among which,

The preset matching features of the user's father are: taste: spicy, fresh; others: reduce blood lipids;

The preset matching features of the user's mother are: taste: sour; others: early pregnancy;

The preset matching features of user Dabao Dabaoda are: taste: sweet; others: protect eyesight.

The third controller controls the left side of the user interface to display the user list, and controls the right side of the user interface to generate a second recommendation window; the user's father is the focus position in the user list, and the third controller controls the second recommendation window The window displays the recommended recipe data mapped by the user's father, including pickled fish, Mapo tofu, and couple's lung slices.

That is, the third controller of the display device uses the recipes in the recommended recipe data determined by the server that are spicy, fresh, and have the function of lowering blood lipids as the mapped recommended recipe data of the user's father. By operating the user list, the recommended recipe data of the corresponding user can be displayed in the second recommendation window.

In some embodiments, the third controller controlling the user interface to display the recommended recipe operation interface including controls according to the recommended recipe data includes: when the received recommended recipe data only includes the first recommended recipe data, the third controller: The first recommendation window is controlled to display the first recommended recipe data.

In some embodiments, a mapping relationship is constructed according to the preset matching features of different users bound to the display device and the recipes in the recommended recipe data, wherein the preset matching features can also be preset from the following perspectives.

For example, the crowd is roughly divided into infants, children, adults, the elderly, pregnant women, college entrance examination people, etc.; the scenes are divided into breakfast, lunch, dinner and supper, etc.; the recipes are divided into Hunan cuisine, Sichuan cuisine, Shandong cuisine, etc. according to geographical areas; recipes Varieties are divided into meat dishes, vegetarian dishes, soups, desserts, snacks, steamed dishes, boiled dishes, stir-fried dishes, etc.; efficacy is divided into blood, calcium, iron, blood pressure, blood lipid lowering, etc.; taste preferences can include sour, spicy, bitter, sweet , salty, sour, fresh, etc.; cooking methods can be divided into steaming, stir-frying, frying, baking, etc.

FIG. 17D shows a schematic diagram of a recommended recipe data operation interface according to an embodiment of the present application.

In some embodiments, the third controller of the display device controls the user interface to display, according to the recommended recipe data, a recommended recipe operation interface including controls for jumping to the recommended recipe details interface after receiving the user's confirmation operation , wherein the recommended recipe operation interface includes a purchase control, and the purchase control is used to jump to the purchase operation interface of the ingredients corresponding to the recommended recipe after receiving the user's confirmation operation.

In the recommended recipe operation interface containing donuts, after the user clicks the donut poster control, the third controller controls the donut poster control to jump to the operation interface of the recipe, including the name, picture, and introduction of the recipe , materials, and a production method, wherein the production method may include graphic tutorials or video tutorials; the operation interface further includes a purchase control, and the user can jump to the recommendation by operating the purchase control. The recipe corresponds to the purchase operation interface of the ingredients, and users can purchase various materials needed to make donuts online.

In some embodiments, the recommended recipe operation interface displayed on the user interface includes a plurality of recipes, the recipes include a poster control, and the poster control is displayed as a picture corresponding to the recipe; and the server is the poster control when sending the recommended recipe data A URL or ID is preset, so that after the user clicks the poster control, the user interface is loaded as the recommended recipe details interface. In some embodiments, the server includes the recommended recipe details of the recipe when sending the recommended recipe data, and after the user clicks the poster control, the user interface is loaded into the recommended recipe details interface according to the recommended recipe details. FIG. 17C shows a UI schematic diagram of a display device recommending recipe data according to another embodiment of the present application.

In some embodiments, the third controller of the display device, in response to the user's operation on the recommended recipe application, determining the recommended recipe data that the server can provide according to the main ingredients in the ingredients stored in the user's refrigerator includes, the third controller responding Constructing a first set of ingredients based on the user's selection of ingredients stored in the user's refrigerator; sending a second recommendation request to the server, where the second recommendation request includes the first set of ingredients, and the second recommendation request is used to make The server determines recommended recipe data that can be provided by the server according to the main ingredient in the first ingredient set; and receives the recommended recipe data sent from the server.

The third controller displays the user's refrigerator storage ingredients received by the server from the refrigerator on the user interface. Based on the user's operation on the focus in the user interface, the third controller selects and confirms that at least one ingredient constitutes the first ingredient set, and reports to the server. sending a second recommendation request containing the first set of ingredients;

The second controller of the server receives a second recommendation request including the first ingredient set sent from the display device, and queries the media library included in the server according to the main ingredient in the first ingredient set in the second recommendation request Determining the available recommended recipe data and sending the recommended recipe data to the display device.

The third controller of the display device receives recommended recipe data fed back from the server, where the recommended recipe data is generated based on the main ingredient in the first ingredient set selected by the user. Thereby, the scope of the recommended recipe data can be narrowed, so that the recommended recipe data is more suitable for the user's needs.

In some embodiments, the third controller controls the first set of ingredients selected by the user to be less than 3 kinds of ingredients, and the server matches the main ingredients in the first set of ingredients, constructs recommended recipe data for the recipe with the highest matching degree, and Give feedback.

In some embodiments, constructing the first set of ingredients by the third controller of the display device in response to the user's selection of ingredients stored in the user's refrigerator includes, the third controller displaying on the user interface a set of ingredients generated according to the ingredients stored in the user's refrigerator A side dish list, where the ingredients included in the side dish list are arranged in ascending order according to the remaining days of the shelf life; in response to a user's selection operation on the ingredients in the side dish list, a first ingredient set is constructed.

In the smart side dish user interface shown in FIG. 17C , the side dish list generated by the third controller includes all the ingredients stored in the user's refrigerator, and the figure shows the name, picture, and remaining days of the shelf life of each ingredient. For example, the list of side dishes in the figure includes apples, broccoli, grapes, carrots, cabbage, ham, strawberries, watermelon, milk, bread, soy milk, fruit juice, red wine, shrimp, cherries, cakes, etc.

The third controller arranges the ingredients according to the remaining days of the shelf life of the ingredients in the side dish list from small to large, and the smaller the remaining days of the shelf life, the higher the order, so as to prompt the user to select and use the ingredients that are close to the shelf life to avoid waste. , to improve the fit of the recommended recipes to user needs and practicability.

In some embodiments, the third controller generates a progress bar for the ingredients according to the remaining days in the shelf life of the ingredients, and displays the progress bar of the ingredients whose remaining days in the shelf life is less than or equal to a preset threshold as a high brightness or a preset color to remind the user to quickly Use ingredients.

Based on the introduction of the technical solution for realizing the recommended recipe data by the display device and the description of the drawings, the present application also provides a method for recommending recipes on the display device side.

FIG. 18 shows a schematic flowchart of a method for recommending recipes according to an embodiment of the present application.

In step 301, after the recommended recipe application is started, a first recommendation request is sent to the server, and the first recommendation request is used to make the server feed back recommended recipe data determined according to the main ingredients in the ingredients stored in the user's refrigerator.

In step 302, a recommended recipe operation interface including controls is displayed on the user interface according to the recommended recipe data, and the controls are used to jump to the recommended recipe details interface after receiving the user's confirmation operation.

The specific operation of implementing the recommended recipe by the display device has been described in detail in the introduction of the technical solution for implementing the recommended recipe on the TV side above, and will not be repeated here.

In some embodiments, after the recommended recipe application is started, the method further includes: sending a second recommendation request to the server, where the second recommendation request includes a first set of ingredients selected and constructed by the user for the ingredients stored in the refrigerator of the user, The second recommendation request is used to enable the server to determine, according to the main ingredient in the first ingredient set, recommended recipe data that the server can provide; and to receive the recommended recipe data sent from the server. The specific operation of implementing the recommended recipe by the display device has been described in detail in the introduction of the technical solution for implementing the recommended recipe on the TV side above, and will not be repeated here.

Based on the introduction of the technical solution for realizing the recommended recipe data by the display device and the server and the description of the drawings, the present application also provides a method for recommending recipes on the server side.

FIG. 19 shows a schematic flowchart of a method for recommending recipes according to another embodiment of the present application.

In step 401, a recommendation request sent from a display device is received, and the recommended recipe data that can be provided is queried according to the main ingredients in the user's refrigerator storage ingredients, and the user's refrigerator storage ingredients are identified by the server as main ingredients or auxiliary ingredients ;

In step 402, the recommended recipe data is sent to the display device.

The specific operation of implementing the recommended recipe by the server has been described in detail in the introduction of the technical solution for implementing the recommended recipe on the TV side above, and will not be repeated here.

In some embodiments, receiving a recommendation request sent from a display device, and querying the available recommended recipe data according to the main ingredients in the ingredients stored in the refrigerator of the user includes: receiving a second recommendation including a first set of ingredients sent from the display device request, determine the recommended recipe data that can be provided according to the main ingredient in the first ingredient set in the second recommendation request; and send the recommended recipe data to the display device. The specific operation of implementing the recommended recipe by the server has been described in detail in the introduction of the technical solution for implementing the recommended recipe on the TV side above, and will not be repeated here.

FIG. 20A shows a schematic side view of the appearance of a refrigerator according to an embodiment of the present application.

As shown in the figure, FIG. 20A also shows a refrigerator provided by the present application as an example, including a camera (not shown in the figure), a display screen and a controller (not shown in the figure).

The refrigerator in the figure is a 4-door refrigerator, and its camera is arranged inside the refrigerator and is configured to capture the internal images of the refrigerator, for example, to obtain the images of the refrigerator compartment and the freezer compartment inside the refrigerator, and send them to the controller for identifying the ingredients As well as generating recommended recipe data.

The display screen, which is configured to display the user interface, is usually provided on the outside of the door panel of the refrigerator compartment.

In some embodiments, the controller is configured to send the user's refrigerator storage ingredients obtained according to the image of the interior of the refrigerator captured by the camera to the server, where the user's refrigerator storage ingredients are identified by the server as main ingredients or auxiliary ingredients; responding In response to the user's operation of the recommended recipe application, the recommended recipe data that can be provided by the server is determined according to the main ingredients in the ingredients stored in the user's refrigerator; the user interface is controlled to display a recommended recipe operation interface including controls according to the recommended recipe data, The control is used to jump to the recommended recipe details interface after receiving the user's confirmation operation. The position diagram of the recommended recipe operation interface is shown in FIG. 20B . The specific operation of the controller, camera, and display screen in the refrigerator to implement recommended recipes is similar to the implementation of recommended recipes on the TV side, which has been described in detail in the introduction of the technical solutions for the implementation of recommended recipes on the TV and server sides above, and will not be repeated here. .

In a specific implementation, the present application also provides some non-volatile computer storage media, wherein the computer storage medium can store a program, and when the program is executed, it can include the various embodiments of the screen saver display method and the screen saver jump method provided by the present application some or all of the steps in . The storage medium can be a magnetic disk, an optical disk, a read-only memory (English: read-only memory, abbreviated as: ROM) or a random access memory (English: random access memory, abbreviated as: RAM) and the like.

Those skilled in the art can clearly understand that the technology in the embodiments of the present application can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solutions in the embodiments of the present application can be embodied in the form of software products in essence or in the parts that make contributions to related technologies, and the computer software products can be stored in storage media, such as ROM/RAM, A magnetic disk, an optical disk, etc., includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present application.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The technical solutions described in the foregoing embodiments can still be modified, or some or all of the technical features thereof can be equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the embodiments of the present application. Scope.

For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussions are not intended to be exhaustive or to limit implementations to the specific forms disclosed above. Numerous modifications and variations are possible in light of the above teachings. The above embodiments are chosen and described to better explain the principles and practical applications, so as to enable those skilled in the art to better utilize the described embodiments and various modified embodiments suitable for specific use considerations.

Claims (21)

1. A refrigerator, characterized in that, comprising:

   The box body is provided with a storage room inside, and at least one food material is stored in the storage room;

   a box door, located at the opening of the storage room;

   a display screen, arranged on the box door, for displaying the user interface;

   A controller connected to the display screen is configured to:

   receiving at least one recipe issued by the server according to the existing ingredients in the storage room, each recipe including at least one of the existing ingredients;

   Determine the freshness of each existing ingredient in the recipe according to the current time and the pre-stored time-related information of the existing ingredient;

   Determine the recommendation priority of the recipe according to the freshness of each existing ingredient in the recipe;

   A part or all of the recipes are displayed in the user interface in the descending order of the recommended priority of each recipe as the display order of each recipe.
2. The refrigerator according to claim 1, wherein before the receiving the at least one recipe issued by the server according to the existing ingredients in the storage room, the method further comprises:

   Read the existing ingredients in the storage room from the local ingredient information collection, where the existing ingredients are the ingredients in the local ingredient information whose storage state is the put-in state;

   Sending a recipe recommendation request to the server, where the recipe recommendation request includes at least an ingredient name of the existing ingredient, and the recipe recommendation request is used to make the server match at least one of the existing ingredients according to the ingredient name of the existing ingredient food recipes.
3. The refrigerator according to claim 2, wherein the existing food material has a freshness mark used to represent freshness; the time-related information includes a shelf life and a start time, and the start time is when the food material is put in The time in the storage room or the production time of the ingredients;

   The controller is also configured to:

   At every preset time interval, determine the freshness of the existing ingredients according to the current time, the shelf life and the starting time of the existing ingredients;

   Update the freshness identifier of the existing food according to the freshness of the existing food;

   The determining the freshness of each existing ingredient in the recipe according to the current time and the pre-stored time-related information of the existing ingredient, including:

   According to the ingredient name of each existing ingredient in the recipe, the freshness identifier corresponding to each existing ingredient is searched from the local ingredient information set, so as to determine the freshness of each existing ingredient according to the freshness identifier.
4. The refrigerator according to claim 1, wherein the time-related information includes a shelf life and a start time, and the start time is the time when the ingredients are put into the storage room or the production time of the ingredients. The pre-stored time-related information of the existing ingredients determines the freshness of each existing ingredient in the recipe, including:

   According to the ingredient name of each existing ingredient in the recipe, search the shelf life and start time corresponding to each existing ingredient from the local ingredient information collection;

   The freshness of each existing ingredient is determined according to the current time, the shelf life and starting time corresponding to each existing ingredient.
5. The refrigerator according to claim 3 or 4, wherein the determining the freshness of each existing ingredient according to the current time, the shelf life and the starting time corresponding to each existing ingredient, including:

   Determine the expiration time of the existing ingredients according to the shelf life and starting time corresponding to the existing ingredients;

   Calculate the difference between the expiration time of the existing food and the current time, and determine the freshness level of the existing food according to the ratio of the difference to the shelf life of the existing food.
6. The refrigerator according to claim 5, wherein the determining the recommendation priority of the recipe according to the freshness of each existing ingredient in the recipe comprises:

   The weight value corresponding to each existing ingredient is determined according to the freshness grade of each existing ingredient in the recipe, and the weight value corresponding to different freshness grades is different;

   The sum of the weight values corresponding to the existing ingredients in the recipe is added, and the recipes are sorted based on the sum corresponding to each recipe, so as to determine the recommendation priority of each recipe.
7. The refrigerator according to claim 6, wherein the user interface includes a recipe display area, and the displaying part or all of the recipes in the user interface includes:

   Display N recipe controls in the recipe display area, and load the recipe names and recipe covers of the N recipes with the highest recommended priority among the M recipes returned by the server into the N recipe controls, N≤M , N is less than or equal to the preset number of display positions for recipe controls in the recipe display area;

   The controller is further configured to:

   In response to the user's selection operation on the displayed target recipe control, the detailed content of the corresponding recipe is displayed.
8. The refrigerator according to claim 1, wherein the existing food material has a storage location identifier used to represent a storage room; the user interface further comprises a plurality of reminder areas for specific food materials corresponding to each storage room, wherein the The controller is further configured to:

   While displaying the recipe controls in the recipe display area, according to the storage location identification and freshness corresponding to the existing ingredients, the ingredients icons of the existing ingredients in each storage room whose freshness meets the preset reminder conditions are displayed. In the corresponding specific ingredient reminder area.
9. The refrigerator according to claim 1, wherein the existing food material has a storage position identification used to represent a storage compartment; the controller is further configured to:

   A plurality of ingredient display areas corresponding to each storage room are displayed on the user interface, and according to the freshness identification and storage location identification corresponding to the existing ingredients, the ingredient icons of the existing ingredients in each storage room are displayed in the corresponding In the ingredient display area, the ingredient icons of the existing ingredients in each storage room are arranged in the order of decreasing freshness in the corresponding ingredient display area.
10. A method for pushing information, characterized in that it is applied to a refrigerator, wherein the refrigerator includes a display screen, and the method includes:

    receiving at least one recipe issued by the server according to the existing ingredients in the storage room, each recipe including at least one of the existing ingredients;

    determining the freshness of each existing ingredient in the recipe according to the current time and the pre-stored time-related information of the existing ingredient;

    Determine the recommendation priority of the recipe according to the freshness of each existing ingredient in the recipe;

    A part or all of the recipes are displayed in the user interface by taking the descending order of the recommended priority of each recipe as the display order of each recipe.
11. A refrigerator, characterized in that, comprising:

    The box body has a storage room inside for storing ingredients;

    a box door, located at the opening of the storage room;

    a display, arranged on the box door, for displaying a user interface;

    A communicator for communicating with the associated end device;

    A controller connected to the display and the communicator is configured to:

    Determining a target ingredient, the target ingredient at least includes an ingredient whose freshness satisfies a preset reminder condition determined according to time-related information corresponding to the existing ingredient in the storage room;

    generating a reminder message according to the target ingredient, the reminder message including the ingredient name of the target ingredient;

    Pushing the reminder message to at least one of the terminal devices to display the reminder message on at least one of the terminal devices.
12. The refrigerator according to claim 11, wherein determining the target ingredients according to the time-related information corresponding to the existing ingredients in the storage room comprises:

    Obtain the ingredient information of the existing ingredient from the local ingredient information collection, the ingredient information includes time-related information, and the ingredient information has a reminder status identifier;

    determining expired ingredients in the existing ingredients according to the time-related information corresponding to the existing ingredients;

    An expired ingredient whose expiration duration is less than or equal to the preset reminder period or whose reminder status is identified as the first identifier is determined as the target ingredient.
13. The refrigerator according to claim 11, wherein the target ingredient further comprises an ingredient corresponding to an ingredient name contained in an unread historical reminder message.
14. The refrigerator according to claim 13, wherein the determining the target ingredient comprises:

    Acquire unread historical reminder messages before a preset time point in the notification application, where the notification application is used to display the generated reminder message on the display, where the preset time point refers to a predetermined time point earlier than the current time point. Set the time point of the time interval;

    The ingredient corresponding to the ingredient name included in the historical reminder message is determined as the target ingredient.
15. The refrigerator according to claim 11, wherein the pushing the reminder message to at least one of the terminal devices to display the reminder message on the at least one terminal device comprises:

    sending a device discovery message to a terminal device in the local area network, where the device discovery message carries the IP address of the refrigerator;

    If at least one response message from the terminal device to the device discovery message is received within the preset waiting time period, the reminder message is pushed to the terminal according to the IP address of the terminal device in the response message. equipment;

    If a response message from the terminal device to the device discovery message is not received within a preset waiting time period, the reminder message is displayed on the display through a notification application.
16. The refrigerator according to claim 15, wherein the sending a device discovery message to the terminal device in the local area network comprises:

    Send a UDP broadcast message to the terminal device in the local area network and/or send a multicast message to the terminal device in the target network, the target network refers to the network corresponding to the refrigerator, the UDP broadcast message and the group All broadcast messages carry the IP address of the refrigerator.
17. The refrigerator according to claim 15, wherein, if at least one response message from the terminal device to the device discovery message is received within a preset waiting time period, according to the response message in the terminal device The IP address of the device to push the reminder message to the terminal device, including:

    determining the preset priority of each of the terminal devices, and determining the terminal device with the highest priority as the target device;

    The reminder message is pushed to the target device according to the IP address of the target device.
18. The refrigerator of claim 12, wherein the controller is further configured to:

    After the expired ingredients whose reminder status is identified as the first identification are determined as the target ingredients, the reminder status identification of the expired ingredients whose reminder status is identified as the first identification is modified to the second identification.
19. The refrigerator according to claim 11, wherein the time-related information includes a shelf life and a start time, and the expired ingredients in the existing ingredients are determined according to the time-related information corresponding to the existing ingredients, including:

    Determine the expiration time of the existing ingredients according to the shelf life and starting time corresponding to the existing ingredients;

    Existing ingredients whose expiration time is earlier than the current time are determined as expired ingredients.
20. The refrigerator of claim 14, wherein the controller is further configured to:

    receiving an input preset operation on the reminder message through the notification application;

    In response to the preset operation, the reminder message is marked as a read state or the reminder message is cleared.
21. A message reminder method, characterized in that, applied to a refrigerator, the method comprising:

    Determining target ingredients, the target ingredients at least include ingredients whose freshness satisfies the preset reminder condition determined according to the time-related information corresponding to the existing ingredients in the refrigerator;

    generating a reminder message according to the target ingredient, the reminder message including the ingredient name of the target ingredient;

    Pushing the reminder message to at least one associated terminal device to display the reminder message on the at least one associated terminal device.

Images