WO2022048221A1

WO2022048221A1 - Video progress synchronization method, display device, and electronic device

Info

Publication number: WO2022048221A1
Application number: PCT/CN2021/098708
Authority: WO
Inventors: 王光强
Original assignee: 聚好看科技股份有限公司
Priority date: 2020-09-07
Filing date: 2021-06-07
Publication date: 2022-03-10
Also published as: CN116195258A

Abstract

The present application discloses a video progress synchronization method, a display device, and an electronic device. The display device provided in the present application comprises: a display; a user interface; a communication component used to connect the display device to a server; and a first controller respectively connected to the display, the communication component and the user interface, and configured to perform the following: receiving a first operation of a user, and playing video according to the received first operation; and sending a playback record of the video to the server when the user leaves, such that the server, upon receiving an acquisition request sent, in response to a trigger operation of the user, by an electronic device corresponding to the display device, sends the playback record to the electronic device, so as to enable the electronic device to continue the playback of the video according to the playback record. In this way, the user can enjoy the synchronized video on the electronic device having the same playback progress as the video on the display device, thereby improving user experience.

Description

A kind of audio and video progress synchronization method, display device and electronic device

This application is required to be submitted to the China Patent Office on September 7, 2020, with the application number of 202010930923.2 and the invention titled "An audio and video progress synchronization method, display device and refrigeration equipment" Chinese patent application, on September 7, 2020 Submitted to the China Patent Office with the application number of 202010929852.4 and the invention titled "A method for synchronizing audio and video progress, display equipment and refrigeration equipment", and submitted to the China Patent Office on September 30, 2020, with the application number of 202011057568.9, The priority of the Chinese patent application entitled "Information Recommendation Method, Food Material Storage Device and Server", the entire contents of which are incorporated herein by reference.

technical field

The present application relates to the technical field of audio and video data processing, and in particular, to an audio and video progress synchronization method, a display device and an electronic device.

Background technique

With the popularization of AIOT (Artificial Intelligence & Internet of Things), smart home appliances are becoming more and more popular, and users can watch audio and video on different smart home appliances. However, when users watch the same audio and video on different smart home appliances, they find that the audio and video playback progress is not synchronized on different smart home appliances, which affects the user experience. In some embodiments, the smart home appliances include display devices and electronic devices. Exemplarily, the electronic devices may be devices such as refrigerators, coffee tables, and fitting mirrors, and may also be devices such as computers, mobile phones, and tablet computers.

SUMMARY OF THE INVENTION

In a first aspect, a display device provided by the present application includes: a display; a user interface; a communicator for connecting the display device to a server; a first controller respectively connected to the display, the communicator and the user interface for executing :

Receive the user's first operation, and play the audio and video according to the received first operation; when the user leaves, send the audio and video playback record to the server, and the server receives the audio and video corresponding to the display device. When the electronic device responds to the acquisition request sent by the user's trigger operation, it sends the playback record to the electronic device, so that the electronic device continues to play the audio and video according to the playback record, wherein the playback record Including audio and video identification and playback progress.

In a second aspect, a display device provided by the present application includes: a display; a user interface; a communicator for connecting the display device to a server; a first controller respectively connected to the display, the communicator and the user interface for executing :

Receive the trigger operation of the user; respond to the acquisition request sent to the server with the trigger operation, so that the server sends the audio and video playback record sent by the electronic device to the server when the user leaves to the display device, wherein all the The playback record includes an audio and video identification and a playback progress; the audio and video are continued to be played at the playback progress according to the audio and video identification.

In a third aspect, the present application provides an electronic device, comprising: a display screen for displaying an interactive interface; a second controller for performing: receiving a trigger operation; and acquiring a display corresponding to the electronic device in response to the trigger operation A playback record of audio and video on the device; acquire audio and video according to the audio and video identification of the playback record, and play the audio and video from the acquired playback progress, wherein the playback record includes the audio and video identification and the playback progress.

In a fourth aspect, the present application provides a method for synchronizing audio and video progress, which is applied to a display device, including:

Receive the user's first operation, and play the audio and video according to the received first operation; send the audio and video playback record to the server, and the server responds to the user after receiving the electronic device corresponding to the display device When the acquisition request sent by the trigger operation is sent, the playback record is issued to the electronic device, so that the electronic device continues to play the audio and video according to the playback record, wherein the playback record includes the audio and video identification and Playback progress.

In a fifth aspect, the application provides a method for synchronizing audio and video progress, which is applied to electronic equipment, including:

Receive a trigger operation; in response to the trigger operation, acquire the playback record corresponding to the user account; acquire the audio and video according to the audio and video identification of the playback record, and play the audio and video from the acquired playback progress, wherein the The playback record includes the audio and video identification and the playback progress.

Description of drawings

FIG. 1 is a schematic diagram of a communication scenario between a display device and an electronic device in some embodiments of the present application;

FIG. 2 is a schematic diagram of another communication scenario between a display device and an electronic device in some embodiments of the present application;

3 is a schematic diagram of a user interface of a display device in some embodiments of the present application;

4 is a schematic diagram of another user interface of a display device in some embodiments of the present application;

5 is a schematic diagram of a user interface of a health screen in some embodiments of the present application;

FIG. 6 is another schematic diagram of a user interface of a display device in some embodiments of the present application;

7 is a signaling diagram of a method for synchronizing audio and video progress in some embodiments of the present application;

8 is a method for synchronizing audio and video progress in some embodiments of the present application;

FIG. 9 is a schematic structural diagram of a refrigeration device in some embodiments of the present application;

FIG. 10 is a flowchart of still another information recommendation method in some embodiments of the present application;

11 is a display interface diagram of a display screen of a refrigeration device in some embodiments of the present application;

FIG. 12 is a display interface diagram of a display screen of another refrigerating device in some embodiments of the present application;

FIG. 13 is a display interface diagram of a display screen of still another refrigerating apparatus in some embodiments of the present application;

FIG. 14 is a display interface diagram of a display screen of another refrigerating apparatus in some embodiments of the present application.

detailed description

In order to make the objectives, implementations and advantages of the present application clearer, the exemplary embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the exemplary embodiments of the present application. Obviously, the exemplary embodiments described It is only a part of the embodiments of the present application, but not all of the embodiments.

Based on the exemplary embodiments described in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the appended claims of this application. Furthermore, although the disclosures in this application have been presented in terms of illustrative example or instances, it should be understood that various aspects of this disclosure may also constitute a complete embodiment in isolation.

The terms "first", "second", "third", etc. in the description and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and are not necessarily meant to limit specific Order or precedence unless otherwise indicated. It is to be understood that the terms so used are interchangeable under appropriate circumstances, eg, can be implemented in accordance with an order other than those presented in the illustration or description of some embodiments of the present application.

The term "remote control," as used in this application, may refer to a component of a display device that can wirelessly control the display device, usually over a short distance. Generally, infrared and/or radio frequency (RF) signals and/or Bluetooth are used to connect with the display device, and functional modules such as WiFi, wireless USB, Bluetooth, and motion sensors may also be included. For example, a hand-held touch remote control replaces most of the physical built-in hard keys in a general remote control device with a user interface in a touch screen.

The term "gesture" used in this application refers to a user's behavior that is used by a user to express an expected thought, action, purpose/or result through an action such as a change of hand shape or hand movement.

FIG. 1 exemplarily shows a schematic diagram of an operation scenario between a display device and a control apparatus according to an embodiment. As shown in FIG. 1 , a user may operate the display apparatus 200 through the mobile terminal 300 and the control apparatus 100 .

In some embodiments, the control device 100 may be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication or Bluetooth protocol communication, and other short-range communication methods, etc., and controls the display device 200 by wireless or other wired methods. The user can control the display device 200 by inputting user instructions through keys on the remote control, voice input, control panel input, and the like. For example, the user can control the display device 200 by inputting corresponding control commands through the volume up/down keys, channel control keys, up/down/left/right movement keys, voice input keys, menu keys, power-on/off keys, etc. on the remote control. function.

In some embodiments, mobile terminals, tablet computers, computers, notebook computers, and other electronic devices may also be used to control the display device 200 . For example, the display device 200 is controlled using an application running on the electronic device. The application can be configured to provide users with various controls in an intuitive user interface (UI) on the screen associated with the electronic device.

Taking the electronic device as a terminal as an example, in some embodiments, the mobile terminal 300 can install a software application with the display device 200 to realize connection and communication through a network communication protocol, so as to realize the purpose of one-to-one control operation and data communication. For example, a control command protocol can be established between the mobile terminal 300 and the display device 200, the remote control keyboard can be synchronized to the mobile terminal 300, and the function of controlling the display device 200 can be realized by controlling the user interface on the mobile terminal 300. The audio and video content displayed on the mobile terminal 300 may also be transmitted to the display device 200 to implement a synchronous display function. The mobile terminal 300 may also be in communication connection with the server.

As also shown in FIG. 1 , the display device 200 also performs data communication with the server 400 through various communication methods. The display device 200 may be allowed to communicate via local area network (LAN), wireless local area network (WLAN), and other networks. The server 400 may provide various contents and interactions to the display device 200 . For example, the display device 200 interacts by sending and receiving information, and electronic program guide (EPG), receiving software program updates, or accessing a remotely stored digital media library. The server 400 may be a cluster or multiple clusters, and may include one or more types of servers. Other network service contents such as audio and video on demand and advertising services are provided through the server 400 .

The display device 200 may be a liquid crystal display, an OLED display, or a projection display device. The specific display device type, size and resolution are not limited. Those skilled in the art can understand that the display device 200 can make some changes in performance and configuration as required.

The display device 200 may additionally provide a smart IPTV function that provides computer-supported functions, including but not limited to, IPTV, smart TV, Internet Protocol TV (IPTV), and the like, in addition to the broadcast receiving TV function.

The electronic device may also be a refrigeration device with audio and video output functions.

FIG. 2 is a schematic diagram of another scenario provided by some embodiments of the present application. The server 400 and a plurality of refrigerating apparatuses 10 connected in communication are included, and the refrigerating apparatuses 10 may be refrigerators. In some embodiments, target accounts may be logged into the plurality of refrigerating apparatuses 10 . Some embodiments of the present application take the example that the plurality of refrigerating apparatuses 10 include two refrigerating apparatuses, such as a first refrigerating apparatus and a second refrigerating apparatus. In some embodiments, the information recommendation system may include one server 400 or multiple servers 400. FIG. 2 takes the information recommendation system including only one server 400 as an example, and the number of refrigeration devices with the target account registered in the information recommendation system may also be There are three, four or even more, which are not limited in some embodiments of the present application. The server may also be connected to refrigeration devices logged in with other accounts, and the number of refrigeration devices logged in with other accounts may also be any number, which is not limited in some embodiments of the present application.

Smart home appliances are becoming more and more popular, and users can watch audio and video on different smart home appliances. However, when users watch the same audio and video on different smart home appliances, they find that the audio and video playback progress is not synchronized on different smart home appliances, which affects the user experience. In some embodiments, the smart home appliance includes a display device and an electronic device. Electronic devices can be refrigerated devices, coffee tables, fitting mirrors, and other devices, as well as computers, mobile phones, tablet computers, and other devices.

In some embodiments, audiovisual refers to audio and/or video. Audio-video synchronization refers to audio synchronization and/or video synchronization.

The methods of some embodiments of the present application are developed for the synchronization of audio and video between the display device and the electronic device, and the display device receives the user's first operation and plays the audio and video. Exemplarily, the first operation may be completed by the user operating the control device to move the selector to select the control corresponding to the audio and video to be watched on the interface of the display device, and click the confirm button to complete.

In some embodiments of the present application, in order for the user to synchronously watch the audio and video played on the display device on the electronic device, the display device plays the audio and video according to the first operation, and sends the playback record of the audio and video to the server. The server stores the playback record with the user account as the dimension. In some embodiments, the playback record includes audio and video identification and playback progress. In the subsequent steps, the electronic device may obtain the playback record through the server, and then continue to play the audio and video according to the playback progress in the playback record.

In some embodiments of the present application, the audio and video identifiers are in one-to-one correspondence with audio and video. Exemplarily, the playback progress is the playback progress of the audio and video when the playback record is sent, which may be 11min28s, 5min12s, or the like. In some embodiments of the present application, the playback records of audio and video are stored with the user account as the dimension. Exemplarily, the user account logged on the display device is Xiaoming, and the playback records of the user account Xiaoming stored on the server may include ZHZ:11min28s, GTX :5min12s, another example, the user account logged on the display device is Xiaohong, and the playback record of the user account Xiaohong stored in the server includes MYZ:1h12min. In some embodiments, the audio-visual identification may be represented by numbers, characters, and/or images.

In some embodiments of the present application, the display device may send audio and video playback records to the server in various scenarios. In some embodiments, an input pause operation is received, and a playback record of the audio and video is sent to the server. In some embodiments, the display device further includes a human body detector, and the human body detector determines whether the user displays a preset viewing range of the device, and then determines whether to send audio and video playback records to the server. In some embodiments, the playback record of the audio and video is sent to the server according to a preset period. In other embodiments, it is determined whether to send the playback record to the server by determining whether other devices logged into the same user account are playing audio and video. The method for synchronizing the audio and video progress involved in sending audio and video playback records under the above different scenarios is described below by specific description.

In some embodiments, before sending the audio and video playback record to the server, the method further includes: receiving an input pause operation. Exemplarily, after responding to the pause operation, the interface is as shown in FIG. 3 . In some embodiments, after receiving an input pause operation, a user input play operation is received, that is, after pausing the audio and video, continue to watch the audio and video. Since there is a time difference between pausing the audio and video playback and continuing to watch the audio and video playback, if the audio and video is played by other electronic devices logged in to the same user account within the time period corresponding to the time difference, the playback progress will be changed. The audio and video content is repeated, so, in some embodiments of the present application, if the playback progress recorded by the server is updated, the audio and video are played according to the updated playback progress sent by the received server. Exemplary, as shown in Figure 3 The playback progress of the audio and video is 11min5s. When the audio and video are played by other electronic devices within the time period corresponding to the time difference, and the display device replays the audio and video, as shown in Figure 4, the playback starts from the updated playback progress of 28min5s.

If the playback progress recorded on the server is not updated, it means that the user does not play the audio and video through other electronic devices, so the audio and video are continued to be played directly according to the playback progress after receiving the pause operation.

In some embodiments, when the playback progress recorded by the server is updated, the audio and video are played according to the received updated playback progress; when the playback progress recorded by the server is not updated, according to the received The step of playing the audio and video according to the playback progress after the operation includes: in response to the playback operation, sending an acquisition request for a playback record to the server; receiving a playback record fed back by the server in response to the acquisition request for the playback record; When the playback progress of the feedback is different from the playback progress during the pause, the audio and video are played according to the playback progress of the feedback; when the playback progress of the feedback is the same as the playback progress during the pause, from the The playback progress during the pause continues to play the audio and video.

Exemplarily, when the playback progress after the pause operation is 11min28s and the feedback playback progress is 12min5s, the audio and video will continue to be played according to the feedback playback progress of 12min5s. When the playback progress after the pause operation is 11min28s, and the feedback playback progress is 11min28s, the audio and video will continue to be played according to the playback progress after the pause for 11min28s.

In some embodiments, the display device further includes a human body detector; before sending the audio and video playback record to the server, the display device further includes: acquiring a detection signal input by the detector; when the detection signal indicates that the user has left a preset viewing range When the detection signal indicates that the user has not left the preset viewing range, the playback record of the audio and video is not sent to the server.

In some embodiments, when the user leaves the preset viewing range, it means that the user is no longer in the preset viewing range to watch the audio and video through the display device. At this time, it is possible that the user needs to watch the audio and video through other devices. The playback record is sent to the server. If the user watches the audio and video in the preset viewing range, the playback record of the audio and video is not sent to the server.

In some embodiments, in order to save resources, when the detection signal indicates that the user leaves the preset viewing range, when the audio and video playback record is sent to the server, the audio and video playback is also suspended. Since the number of users who use the display device to watch audio and video may be a single person or a number of people, in some embodiments, when multiple people watch audio and video, the detection signal indicates that a user leaves the preset viewing range, and the audio and video are paused at this time. Video playback. In some embodiments, when multiple people watch the audio and video, the detection signal indicates that a user leaves the preset viewing range, and the playback of the audio and video is not paused at this time, but the playback record of the audio and video is uploaded to the server, so that the user who leaves can Use other devices to watch audio and video simultaneously. In some embodiments, when multiple people watch the audio and video, and the detection signal indicates that all users have left the preset viewing range, the playback of the audio and video is suspended. In some embodiments, when the number of users watching the audio and video is one, when the user leaves the preset viewing range, the playback of the audio and video is paused.

In some embodiments, after the playback of the audio and video is paused, the detection signal indicates that the user who has left re-enters the preset viewing range, that is, the user re-enters the preset viewing range after leaving the preset viewing range. The display device continues to play the audio and video, and before playing the audio and video, first determine whether there is an update of the playback progress recorded by the server;

When the playback progress recorded by the server is updated, play the audio and video according to the received updated playback progress; when the playback progress recorded by the server is not updated, play the audio and video according to the playback progress after receiving the pause operation Continue to play the audio and video.

In some embodiments, in order to avoid affecting the normal playback of other devices that are playing the audio and video, the step of sending the audio and video playback record to the server includes: sending a playback status confirmation request to the server to determine the user account. Whether other devices under the user account are playing the audio and video; when other devices under the user account are not playing the audio and video, send the audio and video playback records to the server;

When other devices under the user account are playing the audio and video, the playback record of the audio and video is not sent to the server. In some embodiments, the server includes a service server and an Internet of Things server; the playback record of the audio and video is stored in the service server, and whether other devices under the user account are playing the audio and video is confirmed by the Internet of Things server. For example, after the device is powered on, a long link is maintained with the IoT server, and the playback status of the device is reported to the IoT server in real time.

Taking the electronic device as a refrigerating device as an example, in some embodiments of the present application, the refrigerating device includes a box body for placing ingredients, a door body arranged on the box body, and a display screen arranged on the door body, The display screen is used to display an interactive interface. When the user wants to watch the same audio and video on the display device through the refrigerated device, at this time, the user can log in to the same user account as the display device on the refrigerated device, receive a trigger operation, and send an acquisition request to obtain the playback record in response to the trigger operation to server. The server receives the acquisition request, sends the playback record to the refrigeration equipment, and the refrigeration equipment acquires the audio and video according to the audio and video identification in the playback record, and continues to play the audio and video according to the playback progress in the playback record. In some embodiments, if the playback record is not acquired, the display screen is controlled to display a specified interface. In some embodiments, the designated interface may be a health screen, and the health screen mainly displays functions related to health. Exemplarily, as shown in FIG. 5 , the health screen displays health management, ice control, and ingredient management. and recipes etc. Some embodiments of the present application do not limit the content of the specified interface.

In some embodiments, the refrigerating device receives the operation of quitting playing audio and video; in response to quitting the operation of playing audio and video, it controls the display screen to display a specified interface, and sends the audio and video playback record to the server.

In some embodiments, the triggering operation includes an operation of starting a display screen, or an operation of switching the display screen from a first mode to a second mode, wherein the first mode is an ingredient management mode, and the second mode It is a mode different from the food management mode; or, an operation of opening the door. In some embodiments, the triggering operation further includes re-powering on the refrigeration device or re-logging in to the user account. The operation of activating the display screen may include operations such as opening the door, people approaching the refrigerating device, and clicking the display screen.

In some embodiments, after the refrigerating device continues to play the audio and video according to the playback progress in the playback record, an operation of exiting the audio and video playback is received. In response to exiting the audio and video playback operation, the display screen is controlled to display the specified interface, and the audio and video playback record is sent to the server. In some embodiments of the present application, in order to ensure that both the display device and the refrigeration device can synchronize the playback progress of other devices when playing audio and video, in addition to the display device needing to upload the audio and video playback records, the refrigeration device also needs to upload the audio and video playback records. to the server.

In some embodiments, the step of acquiring the playback record corresponding to the user account in response to the trigger operation includes: sending a playback record acquisition request to the server according to the user account of the refrigerating device; and receiving the playback record fed back by the server.

In some embodiments, the step of playing the audio and video from the obtained playback progress includes:

Obtain the playback record corresponding to the user account from the server, and query the playback status of the display device. In some embodiments, the server includes a business server and an Internet of Things server, the audio and video playback records are obtained from the business server, the playback status of the inquiring mind device is determined by the Internet of Things server, and a long link is maintained with the Internet of Things after the display device is turned on. The server displays the playback status of the device and reports it to the IoT server in real time.

In response to the state of the display device being the playback pause state, a synchronization prompt message is displayed on the upper layer of the user interface of the control display device to prompt the user to watch the audio and video played on the display device synchronously. When the state of the display device is in the paused state, the user does not need to watch audio and video on the display device, and a synchronization prompt message is displayed on the display screen at this time. The synchronization prompt message is exemplarily shown in FIG. 6 . The synchronization prompt message in 6 shows the text "Do you want to continue watching, it is detected that you are watching "QYN" on the TV side, is it synchronized on the refrigerator side?" In addition, the synchronization prompt message includes a viewing control and a cancellation control. The user interface of the display can also be seen when the synchronization prompt message is displayed.

In response to selecting the viewing control in the synchronous prompt message, control the upper layer of the user interface not to display the synchronous prompt message, play the corresponding audio and video according to the playback progress, and update the playback record in the server in real time . Exemplarily, after selecting the viewing control in the synchronous prompt message, the user can move the focus to the viewing control through the user control device, and press the confirmation key on the control device to complete the operation. In some embodiments, the playback record of the refrigeration device is also uploaded to the server in real time.

In response to selecting the cancel control in the synchronization prompt message, the display screen is controlled to stop displaying the synchronization prompt message and only display the designated interface.

In response to the state of the display device being the normal play state, the display screen is controlled to display the specified interface. In some embodiments, when the display device plays audio and video normally, the user may still want to watch the audio and video on the display device, so there is no need to control the display screen to play the audio and video synchronously, and a specified interface can be displayed.

In view of the above description, in some embodiments, when the number of users watching audio and video is one, after watching the audio and video on the display device, the user enters the kitchen and uses the refrigeration device to watch the audio and video, and then returns to the living room to continue to display the audio and video. The device watches the audio and video. In this scenario, a method for synchronizing the audio and video progress is shown in FIG. 7 . The method includes: S101 . The display device receives a first operation of the user. S102. Play audio and video according to the first operation. S103: Send the audio and video playback records to the server. S104. The server stores the playback record. When the user leaves the living room and enters the kitchen, after the user performs the trigger operation, S105 , the refrigerating device receives the trigger operation. S106, in response to the triggering operation, acquire the play record corresponding to the user account on the refrigeration device. S107: Receive the playback record sent by the server. S108: Acquire audio and video according to the audio and video identifiers in the playback record, and play the audio and video from the acquired playback progress. S109: Send the audio and video playback records to the server. S110. The server stores the playback record. When the user leaves the kitchen, he enters the living room again to watch audio and video through the display device. The user performs a playback operation, and S111, the display device receives the playback operation input by the user. S112. In response to the playing operation and the playing record is updated, play the audio and video according to the updated playing progress. S113. Send the audio and video playback records to the server. S114. Store the playback record. With the methods in some embodiments of the present application, when a user watches audio and video content through different devices, repeated viewing of audio and video content can be avoided, thereby improving user experience.

In some embodiments, as shown in FIG. 8 , the present application provides a method for synchronizing audio and video progress, which is applied to a display device, including:

Receive the first operation of the user, and play audio and video according to the received first operation;

Send the playback record of the audio and video to the server, and when the server receives an acquisition request sent by the refrigeration device corresponding to the display device in response to the user's trigger operation, the server sends the playback record to the refrigeration device equipment, so that the refrigerating equipment continues to play the audio and video according to the playing record, wherein the playing record includes the audio and video identification and the playing progress.

In some embodiments, a method for synchronizing audio and video progress, applied to refrigeration equipment, includes:

receive trigger action;

In response to the trigger operation, obtain the playback record corresponding to the user account;

Acquire the audio and video according to the audio and video identifier of the playback record, and play the audio and video from the acquired playback progress, wherein the playback record includes the audio and video identifier and the playback progress.

In some embodiments, the electronic device may be multiple terminals logged in with the same account, so it is impossible to directly determine which terminal the user will watch audio and video on. Trigger the operation to determine on which terminal the user may continue to watch the audio and video content.

In some embodiments, the electronic device that receives the user's operation may not have the ability to play audio and video. At this time, the terminal that does not have the ability to play the audio and video may report/broadcast the event that has obtained the user's operation, so that the users in the same room can When a terminal with audio and video playback capability receives an instruction of broadcasting/or a corresponding reporting time, it enters the step of prompting the user whether to continue watching.

In some embodiments, the display device can determine whether there are other users watching through the acquisition element when uploading the audio and video playback progress. If there are other users, the display device does not pause, and pauses if there are no other users.

In some embodiments, when other users are watching, the display device periodically uploads the audio and video playback progress until receiving information that the television device starts to continue playing. In this way, the viewing progress of the display device and the electronic device can be kept consistent.

In some embodiments, the display device and the electronic device can be interchanged.

In some embodiments, when the television device is a refrigerating device, the refrigerating device may be as shown in FIG. 9 , and the refrigerating device 10 may include: a cabinet 101 , a door 102 , a display screen 103 , a communication interface and a controller (not shown in FIG. 9 ) Shows). The casing 101 includes a storage room with an opening K; the door 102 is movably connected to the casing 101 for blocking the opening K; the display screen 103 is located on the surface of the door 102 facing away from the opening K for displaying a graphical user interface (Graphical User Interface, GUI). In some embodiments, the display screen 103 is a touch display screen, and the user can perform operations on the touch display screen to cause the controller to receive corresponding instructions. A communication interface is a component used to communicate with external devices or external servers according to various communication protocol types. For example, the communication interface may be a Wifi chip, a Bluetooth communication protocol chip, a wired Ethernet communication protocol chip and other network communication protocol chips or a near field communication protocol chip. The controller controls the operation of the refrigerating apparatus 10 and responds to the user's operation through various software control programs in the refrigerating apparatus. In some embodiments, application software may also be installed in the refrigerating device, icons of each application software may be displayed on the display screen 103, the user may click the icon of the application software, and then the controller may start the application software according to the operation, And control the display screen 103 to display the interface corresponding to the application software. The communication interface and the controller may be located at any position inside or outside the casing 101 and the door 102, and some embodiments of the present application do not limit the setting positions of the communication interface and the controller. It is also possible to control other devices that can communicate.

In some embodiments, the refrigerating apparatus may further include a plurality of structures, such as an image collector (eg, a video camera or a camera), a door switch detector, a speaker, a microphone, and a memory, and the plurality of structures may be communicatively connected to the controller. Wherein, at least one of the speaker and the display screen can be used as an information output component to output information for interacting with the user; at least one of the microphone and the display screen can be used as an information input component to receive the information that the user interacts with the refrigerating device . When the information output component includes a speaker, the information output by the information output component is the voice information played by the speaker; when the information output component includes a display screen, the information output by the information output component is the information displayed on the display screen (such as text or image information). When the information input component includes a microphone, the information input by the information input component is the voice information collected by the microphone; when the information input component includes a display screen, the information input by the information input component is the information that the user operates on the display screen to trigger the display screen to display. (such as text or image information).

In some embodiments, the image collector can be used to collect external environment scenes to adaptively change display parameters; it can also be used to collect user attributes or interactive gestures with the user to realize the function of interacting with the user. In some embodiments, the refrigeration device may include a first image capturer 104a and a second image capturer 104b. The first image collector 104a can be used to collect images of the food in the storage room, for example, the first image collector 104a is located on the top of the cabinet 101; the second image collector 104b can be used to collect images of the environment outside the storage room Or capture the user's interactive gesture, etc., the second image capturer 104b is located outside the storage room, such as above the display screen 103 . In some embodiments, the first image capturer 104a may capture images in the storage compartment when the door of the refrigerating apparatus is in an open state, and stop capturing images when the door of the refrigerating apparatus is in a closed state. In some embodiments, the first image capturer 104a may also be located in the storage room, and at this time, the first image capturer 104a may capture images of the food items in the storage room when the door of the refrigerating apparatus is in a closed state. In some embodiments, a light source may also be provided in the storage compartment for illuminating the storage compartment when the door of the refrigerating apparatus is in a closed state, so that the first image capturer 104a can capture images of the food in the storage compartment. The first image collector 104a may also be located at other positions, such as the top of the door 102, which is not limited in some embodiments of the present application. In some embodiments, the "top" of a structure described in some embodiments of the present application is the end of the structure away from the ground when the refrigeration equipment is placed on the ground for normal use.

In some embodiments, the memory may be located in the cabinet or the door, or may be separate from the cabinet and the door. The memory can be used to store the image collected by the image collector, the recognition result obtained by the controller recognizing the food image, and other information that needs to be stored. In some embodiments, a door switch detector may be used to detect whether the door is in an open state or a closed state. In some embodiments, a door switch detector may be provided in a position in the housing capable of making contact with the door, determining that the door is in an open state when the door switch detector is not in contact with the door, and determining that the door is in an open state when the door switch detector contacts the door Disabled. The controller can control the display screen to display the recognition result of the food image or other information, and at this time, the display screen is used as an information output component. The display screen can also be a touch display screen, and the user can interact with the refrigerating device through the touch display screen. For example, the user can touch the touch display screen to trigger the controller to generate corresponding instructions, and The generated instruction is executed, and the display screen is used as an information input component at this time.

In some embodiments, the speaker and microphone may be located on the cabinet or on the door. In some embodiments, the speaker and microphone may be co-located with the image grabber. For example, a speaker, a microphone and an image collector can form a sensing module. The sensing module is arranged on the top of the casing. The sensing module can be connected to a driving component. The driving component can be used to drive the sensing module from the casing. The top extends or retracts. The controller can control the speaker to send out voice information to prompt the user to perform corresponding operations. The controller can control the microphone to collect sound information in the environment where the refrigerating device is located, so as to generate corresponding instructions.

In some embodiments, the controller may control the driving part to move the sensing module according to the state (open state or closed state) of the door or a voice command issued by the user. In some embodiments, when the controller determines that the door is in an open state, the controller may control the driving component to push the sensing module toward the door, and control the driving component to stop the pushing action when the sensing module is pushed out to a predetermined position. Then trigger the image collector in the perception module to work. When the controller determines that the door is in a closed state, it can control the image collector to stop working, and control the driving part to retract the sensing module away from the door; when the sensing module is retracted to its original position, it can control the driving part to stop the retracting action .

A refrigerating device is generally used to store food, and in some embodiments, the embodiments of the present application further provide a method for recommending information according to the food in the refrigerating device.

An information recommendation method provided by some embodiments of the present application, the method can be used for any refrigerating apparatus 10 among the plurality of refrigerating apparatuses shown in FIG. 2 , or the refrigerating apparatus 10 shown in FIG. 9 , the method may include:

Step 401: After receiving the dish recommendation instruction, send a dish recommendation request to the server, where the dish recommendation request carries the identifier of the refrigerating device and the target account registered on the refrigerating device.

Step 402: Receive recommendation information of a dish sent by the server, where the recommended information of the dish is determined by the server based on the dish recommendation request, according to the ingredients stored in the refrigeration device and the reference dish corresponding to the target account.

Step 403 , displaying recommended information of the dish.

To sum up, in the information recommendation methods provided by some embodiments of the present application, the dishes recommended by the refrigeration device to the user are determined according to the ingredients stored in the refrigeration device and the reference dishes corresponding to the target account logged in the refrigeration device. If the reference dishes corresponding to the target accounts are different, the dishes recommended by the refrigerating device may also be different, thus improving the flexibility of the refrigerating device in recommending dishes.

Another information recommendation method provided by some embodiments of the present application, the method can be used in the server 400 shown in FIG. 2 , and the method can include:

Step 501: Receive a dish recommendation request sent by a refrigerating device, where the dish recommendation request carries the identifier of the refrigerating device and the target account logged in the refrigerating device.

Step 502: Determine the ingredients stored in the refrigerating device and the reference dish corresponding to the target account according to the dish recommendation request.

Step 503 , according to the ingredients stored in the refrigerating device and the reference dishes corresponding to the target account, determine the recommended information of the dishes.

Step 504: Send recommendation information of dishes to the refrigerating device.

To sum up, in the information recommendation method provided by some embodiments of the present application, the server may determine the recommended information of the dish according to the ingredients stored in the refrigerating device and the reference dish corresponding to the target account logged in the refrigerating device, and then send the information to the refrigerating device. The device sends the recommended information of the dish, so that the refrigerating device can display the recommended information of the dish. If the reference dishes corresponding to the target accounts are different, the recommended information of the determined dishes is also different, and then the dishes recommended by the refrigeration device to the user according to the recommended information are also different, thus improving the flexibility of the refrigeration device to recommend dishes.

FIG. 10 is a flowchart of still another information recommendation method provided by some embodiments of the present application, and the method can be used in the information recommendation system shown in FIG. 2 . The following embodiments of the present application take the information recommendation system including a server, a first refrigerating device and a second refrigerating device, and both the first refrigerating device and the second refrigerating device have target accounts logged in as an example, to recommend information provided by some embodiments of the present application. method is explained. As shown in Figure 10, the information recommendation method may include:

Step 601 , the first refrigeration device identifies the ingredients stored therein.

In some embodiments, the refrigerating apparatus may perform item identification when the stored item changes, so as to determine the item currently stored in the refrigerating apparatus. In some embodiments, the refrigerator may identify the food item stored therein when the door changes from an open state to a closed state. Alternatively, when the user deposits and takes out ingredients, the refrigeration device may identify the accessed ingredients in real time, and then determine the ingredients currently stored in the refrigeration device in combination with the ingredients originally stored in the refrigeration device.

In the first food identification method, the refrigerating device may use image recognition to identify the food.

In some embodiments, the controller of the refrigerating apparatus may control the light source in the storage room to turn on, and control the image collector in the storage room to capture, when it is determined by the door switch detector that the door of the refrigerating apparatus changes from the open state to the closed state Image of ingredients in pantry. The controller can input the image collected by the image collector into the food material recognition model, and then obtain the recognition result, and determine the type of food material stored in the refrigeration device according to the recognition result. Alternatively, the controller of the refrigerating device may control the image collector to collect an image of the opening of the storage compartment when it is determined by the door switch detector that the door of the refrigerating device has changed from a closed state to an open state, and then identify the image to determine User's access action, and determine the type of food that the user deposits or takes out. When it is determined that the user stores the ingredients into the storage room, the original stored ingredients and the newly stored ingredients are jointly determined as the ingredients currently stored in the storage room. When it is determined that the user takes out an ingredient from the storage compartment, an ingredient other than the ingredient to be taken out among the originally stored ingredients is determined as an ingredient currently stored in the storage compartment. In some embodiments, in addition to determining the type of the food, other information of the food, such as the volume of the food and the storage location of the food in the storage room, may also be determined by means of image recognition.

In the second method of food identification, the refrigeration equipment may use a radio frequency identification (Radio Frequency Identification, RFID) method to identify the food.

In some embodiments, please continue to refer to FIG. 9 , the refrigerating device may further include an electronic label box 105, which may be used to place electronic labels (not shown in the figure), each electronic label may correspond to a food item . In some embodiments, FIG. 9 takes the label box 105 installed on the surface of the door 102 close to the opening K as an example for illustration. In some embodiments, the label box can also be installed on the surface of the door 102 away from the opening K. Alternatively, the label box may also be located at any other position on the cabinet 101 or the door 102, which is not limited in some embodiments of the present application.

When a user places a certain food material in the storage room of the refrigerating device, the user can take the corresponding electronic label from the electronic label box and attach the electronic label to the food material. The controller of the refrigerating device can determine the type of the food material stored in the storage room according to the food material corresponding to the missing electronic tag in the label box. When the user takes out the food from the storage room, the electronic label on the food can be removed and put into the label box. The controller of the refrigerating device may determine the type of the food material to be taken out from the storage room according to the food material corresponding to the newly added electronic label in the label box. Further, the controller may determine the ingredients currently stored in the storage chamber in combination with the ingredients originally stored in the storage chamber.

In some embodiments, the refrigeration apparatus may also include an antenna, which may be wired to the controller. The controller can transmit radio frequency signals through the antenna, and the transmitted radio frequency signals can cover a certain range around the antenna. When the electronic tag is in this range, it can receive the radio frequency signal, and send the information stored in the electronic tag to the antenna by virtue of the energy obtained under the excitation of the radio frequency signal, such as the information is the identification of the electronic tag; further, The controller can receive the information sent by the electronic tag through the antenna to determine the ingredients corresponding to the electronic tag. In some embodiments, the antenna may be located in a label box, such as at the bottom of the label box, and the label box is made of a material capable of shielding radio frequency signals, for example, the material of the label box may include metal. At this time, the radio frequency signal emitted by the antenna can only cover the space in the label box, and then only the electronic label in the label box can receive the radio frequency signal sent by the antenna, and send the information stored in the electronic label to the antenna; the controller can Only the electronic labels in the label box are detected. In some embodiments, the controller can also adjust the transmit power of the radio frequency signal, so that the radio frequency signal only covers the space in the label box, and the label box can be made of any material.

In some embodiments, the refrigerating device may further include a reader, which may be connected to the antenna, and the reader is configured to transmit radio frequency signals through the antenna, and receive information sent by the electronic tag through the antenna. The reader can communicate with the controller, and after receiving the information sent by the electronic tag, send the information to the controller.

In another optional method for determining the ingredients, the user can perform operations on the display screen to input the current operation as the operation of depositing ingredients or taking out ingredients each time the user stores ingredients into the storage room or takes out ingredients. operation, and you can also input the type of ingredients to be deposited and the type of ingredients to be taken out. The controller may determine the ingredients currently stored in the storage room according to the operation type input by the user and the type of the ingredients. In some embodiments, the user may also input other information such as the storage location of the ingredients and the volume and shelf life of the ingredients.

Step 602: The first refrigeration device sends the information of the ingredients stored therein to the server.

The information of the ingredient may include the type of the ingredient. In some embodiments, the information of the food material may further include the shelf life, volume, and storage location of the food material, etc., which are not limited in some embodiments of the present application. In some embodiments, the refrigerating device may also send the identifier of the refrigerating device to the server when sending the information of the food item stored therein, so that the server can store the information of the food item from the device dimension.

Step 603: The server stores the information of the ingredients stored in the first refrigeration device.

The server may store the information of the ingredients sent by the first refrigerating device in a storage space corresponding to the first refrigerating device, and the storage space may be used to exclusively store the information of the ingredients stored in the first refrigerating device.

In some embodiments, some embodiments of the present application take the example of a refrigerating device to identify an ingredient, thereby determining an ingredient stored in the refrigerating device, and sending the information of the ingredient to a server. In some embodiments, the refrigerating device may also directly send the captured image of the food item to the server, and the server will identify the food item based on the image, determine the food item stored in the refrigerating apparatus, and then directly store the information of the food item stored in the refrigerating apparatus. In some embodiments, the server may also send the information of the identified food item to the refrigerating device, so that the refrigerating device can display and store the information of the food item.

In some embodiments, FIG. 11 is a display interface diagram of a display screen of a refrigeration device provided by some embodiments of the present application, and FIG. 12 is a display interface diagram of a display screen of another refrigeration device provided by some embodiments of the present application. 13 is a display interface diagram of another display screen of a refrigerating device provided by some embodiments of the present application. In some embodiments, after closing the door of the refrigerating apparatus, the user may perform an operation on the display screen to trigger the display screen to display the ingredients stored in the currently stored storage compartment. For example, an ingredient management button may be displayed on the display screen, and the user may click the button to trigger the display screen to display the ingredient management interface shown in FIG. 11 . In some embodiments, the storage compartment of the refrigerating apparatus includes a plurality of compartments including a refrigeration compartment, a changing room, and a freezing compartment, and the controller may determine the foodstuffs stored in each compartment, and control the display screen to display different items in the foodstuff management interface. The area displays information about the ingredients stored in each bin. As shown in FIG. 11 , the ingredient management interface includes: a first area D1 corresponding to the refrigerating chamber, a second area D2 corresponding to the changing chamber, and a third area D3 corresponding to the freezing chamber. A representative picture P1 of each ingredient is stored, and the name of the ingredient is displayed below the representative picture P1 (eg, tomatoes in FIG. 11 ).

In some embodiments, as shown in FIG. 11 , multiple target controls may also be displayed in the ingredient management interface. For example, the multiple target controls may include a recipe recommendation control J1, an ingredient adding control J2, and a photographing control J3. The user can trigger the controller to execute the corresponding command by clicking the corresponding control. For example, when the user clicks the control J2 for adding ingredients, the controller can be triggered to control the display screen to display a virtual keyboard, the user can input the type of ingredients through the virtual keyboard, and then the controller can store the type of ingredients and control the display screen to display the type of ingredients. The representative picture and name are displayed in the corresponding area in the ingredient management interface. For example, when the user clicks the photographing control J3, the controller can control the image collector to collect images of the ingredients in the storage room, identify the collected images, and then control the display screen to display the ingredient identification interface shown in FIG. 12 .

In some embodiments, image grabbers may be provided in each of the storage compartments. As shown in FIG. 12 , the ingredient identification interface may include images of ingredients in each warehouse collected by the image acquisition component. For example, image 1 is an image of an ingredient in a refrigerating chamber, image 2 is an image of an ingredient in a temperature-changing chamber, and image 3 An image of the ingredients in the freezer. The ingredient identification interface may also include the identification results of ingredients in each warehouse. For example, the identification results of each ingredient can be displayed in the form of a representative picture P2, or in the form of the name of the ingredient, or in the form of a representative image combined with the name of the ingredient. form display. In some embodiments, the identification result of each ingredient also has a corresponding deletion control J4, for example, the deletion control J4 can be displayed by an icon representing a small circle with a horizontal bar in the middle displayed in the upper right corner of the picture P2. The user can click the delete control J4 to delete the corresponding recognition result. In some embodiments, the ingredient identification interface may further include a re-identification control J5 and a one-key input control J6. The user can click the re-identification control J5 to trigger the controller to re-identify the ingredients stored in the storage room. The user can click the one-key input control J6 to trigger the controller to store and display the identified information of the ingredients in each warehouse. For example, the controller can control the display screen to display the input ingredients display interface shown in FIG. 13 . The input ingredient display interface may include information about newly added ingredients in each warehouse, such as a representative picture and ingredient name of each ingredient. In this way, when the user triggers the display screen again to display the ingredient management interface, the ingredient management interface may include the information of the newly entered ingredient, that is, the information of the ingredient entered in the ingredient display interface.

Step 604 , the second refrigeration device identifies the ingredients stored therein. In some embodiments, reference may be made to step 601 in step 604, which is not repeated in some embodiments of the present application.

Step 605: The second refrigerating device sends the information of the ingredients stored therein to the server. In some embodiments, reference may be made to step 602 in step 605, which is not repeated in some embodiments of the present application.

Step 606, the server stores the information of the ingredients stored in the second refrigeration device. In some embodiments, reference may be made to step 603 in step 606, which is not repeated in some embodiments of the present application.

Step 607: When displaying the dish, the first refrigerating device receives a marking instruction for the displayed first dish.

FIG. 14 is a display interface diagram of another display screen provided by some embodiments of the present application. In some embodiments, recipe software may be installed in the refrigerating device. When the recipe software is running, the display screen may display a variety of dishes, such as the interface shown in FIG. 14 , and the user may also mark the displayed dishes. . For example, when the user performs a specific operation (such as a long press) on the displayed dish, the display screen will be triggered to display various operation controls, and the user can click the corresponding operation control to trigger the controller to perform corresponding operations. As shown in FIG. 14 , the various operation controls may include: a favorite control, a favorite control, a forward control, a save control, and a viewing control, etc. In some embodiments, other operation controls may also be set, and some embodiments of the present application do not Do limit. In some embodiments, the operations of the user clicking on these operation controls may be marked operations, or the operations of the user clicking some specific operation controls may be marked operations, for example, the specific operation controls may include: favorite controls and favorite controls. When the refrigerating apparatus detects a marking operation for a certain dish (eg, the first dish), the refrigerating apparatus may determine that a marking instruction for the first dish is received.

In some embodiments, the above-mentioned embodiments of the present application take the startup of the dish software to trigger the display screen to display the dish as an example. In some embodiments, at any time when the display screen of the refrigerating device displays the dish, the user can mark the dish. operate. For example, when the user searches for a certain dish in the webpage displayed on the display screen, the user may also perform a marking operation on the dish, or directly determine the search operation for the dish as the marking operation on the dish.

Step 608: The first refrigerating device sends a marking request of the first dish to the server based on the marking instruction, where the marking request carries the target account number.

When receiving the marking instruction of the first dish, the refrigeration device can send a marking request of the first dish carrying the target account to the server, so that the server can store the information of the first dish from the account dimension.

Step 609: The server determines the first dish as the reference dish corresponding to the target account according to the marking request of the first dish.

In some embodiments of the present application, the reference dishes corresponding to the target account include dishes marked by the user of the target account, that is, the dishes that the user of the target account likes, and the reference dishes corresponding to the target account can be used to reflect the personal characteristics of the user of the target account like. In some embodiments, when the server receives a marking request for the first dish, and the marking request carries the target account, the server can store the information of the first dish in a storage space corresponding to the target account, and the storage space can be used for special Stores the information of the reference dish corresponding to the target account. In some embodiments, the target account may correspond to a variety of reference dishes, and the server may determine the dish as a newly added reference dish after each receiving a tagging request for a dish, and for other dishes corresponding to the target account Reference dishes are not changed.

Step 610: When displaying the dish, the second refrigerating device receives a marking instruction for the displayed second dish.

In some embodiments, reference may be made to step 607 in step 610, which is not repeated in some embodiments of the present application.

Step 611: The second refrigerating device sends a marking request of the second dish to the server based on the marking instruction, where the marking request carries the target account number.

In some embodiments, reference may be made to step 608 in step 611, which is not repeated in some embodiments of the present application.

Step 612: The server updates the reference dish corresponding to the target account according to the marking request of the second dish, where the reference dish corresponding to the target account includes the first dish and the second dish.

In some embodiments, step 612 may refer to step 609, and the server may also determine the second dish as the reference dish corresponding to the target account. Since the reference dish corresponding to the target account also includes the first dish, the reference dish corresponding to the target account at this time It may include the first dish and the second dish, which is equivalent to the server updating the reference dish corresponding to the target account. The first dish and the second dish described in some embodiments of the present application are both target dishes, and the target dish refers to the marked dish in the dishes displayed by the refrigerating device.

Some embodiments of the present application take step 612 performed after step 609 as an example. After step 612, the reference dishes corresponding to the target account determined by the server include the first dish and the second dish. In some embodiments, if before step 609, that is, before the server receives the mark request of the first dish, if the mark request of other dishes is also received, and the mark request also carries the target account, then in step 609 After that, all the reference dishes corresponding to the target account may include the other dishes and the first dish. After step 611, the reference dishes corresponding to the target account include the first dish, the second dish and the other dishes.

Step 613: After receiving the dish recommendation instruction, the first refrigerating device sends a dish recommendation request to the server, and the dish recommendation request carries the identifier of the first refrigerating device and the target account number.

In some embodiments, recipe software may be installed in the refrigerating device. When the recipe software is running, the display screen may display recommended dishes, and when a startup operation for the recipe software is detected, it may be confirmed that a dish recommendation instruction is received. For example, a recipe application icon may be displayed on the display screen of the refrigerating device, and when the user clicks the recipe application icon, a startup operation for the recipe is performed, and then the refrigerating device can receive a dish recommendation instruction. In some embodiments, when the user clicks the recipe recommendation control J1 in the ingredient management interface shown in FIG. 11 , the refrigeration device may also receive a recipe recommendation instruction. There may also be other ways of triggering the refrigerating device to receive the dish recommendation instruction, which is not limited in some embodiments of the present application.

After the refrigerating device receives the dish recommendation instruction, it can send a dish recommendation request to the server, and the dish recommendation request can carry the identifier of the refrigerating device and the target account registered in the refrigerating device, so that the server can use the refrigerating device and the target account number according to the refrigerating device and the target account. Determine the recommended dishes.

Step 614: The server determines the ingredients stored in the first refrigeration device and the reference dish corresponding to the target account according to the dish recommendation request.

In some embodiments, after receiving the dish recommendation request, the server may obtain food information in the storage space corresponding to the first refrigerating device according to the identifier of the first refrigerating device carried in the dish recommendation request to determine the first refrigerating device food stored in. The reference dish corresponding to the target account may also be obtained in the storage space corresponding to the target account according to the target account carried in the dish recommendation request.

Step 615: The server determines at least one dish that can be prepared using the ingredients stored in the first refrigeration device.

In some embodiments, the server may determine the at least one dish based on a recipe library, and the recipe library may include preparation methods of multiple dishes and ingredients required for preparing each dish. If the server determines that the ingredients stored in the first refrigerating device include all the ingredients required for making a certain dish among the multiple dishes, the server may determine that the dish is a dish that can be made by using the ingredients stored in the first refrigerating device . In some embodiments, the at least one dish may include all dishes among the plurality of dishes in the recipe library that can be prepared by using the ingredients stored in the first refrigeration device. In some embodiments, the recipe library may be stored in the server, or may be obtained by the server on the Internet or other locations.

Step 616: The server determines the recommendation priority of each dish according to the similarity between each dish in the at least one dish and the reference dish corresponding to the target account.

After determining at least one dish that can be made using the ingredients stored in the first refrigerating device, the server may determine the recommendation priority of each dish in the at least one dish according to the reference dish corresponding to the target account. Since the reference dish corresponding to the target account reflects the user's preference, the recommendation priority of the dish is determined according to the reference dish, and then the dish is recommended to the user according to the recommendation priority, which can ensure that the recommended dish is more in line with the user's preference and improve the dish recommendation. of accuracy.

In some embodiments, the recommendation priority of each dish is positively related to the similarity between each dish and the reference dish, for example, the recommendation priority of each dish is positively related to the each dish and all reference dishes and the similarity of . For example, the recommendation priority of each dish may be equal to the sum of the similarity between each dish and all reference dishes. In some embodiments, the similarity of any two dishes may include the similarity of taste and the overlap of ingredients used in production, and the like.

For the convenience of distinction, each dish in the at least one dish is hereinafter referred to as a recommended dish. In some embodiments, the server may compare the taste of each recommended dish in the at least one recommended dish with the taste of each reference dish to determine that the taste of each recommended dish is similar to the taste of each reference dish The server may also compare the ingredients used in each recommended dish in the at least one recommended dish with the ingredients used in each reference dish to determine the degree of overlap between the ingredients used in each recommended dish and the ingredients used in each reference dish . Furthermore, the server may determine the sum of the similarity of the tastes of a recommended dish and a reference dish and the coincidence degree of the ingredients used as the similarity of the recommended dish and the reference dish.

In some embodiments, the similarity of taste and the degree of coincidence of the ingredients used may respectively have corresponding weights, and the product of the similarity of taste and the corresponding weight may be added to the product of the degree of overlap of the ingredients used and the corresponding weight, as the similarity of the two dishes. For example, the weight of the similarity of the taste is lower than the weight of the similarity of the ingredients used, or the weight of the similarity of the taste is higher than the weight of the similarity of the ingredients used. In some embodiments, different reference dishes may also have corresponding weights. After it is determined that the similarity of each reference dish is determined for one recommended dish, it may be determined that the product of the similarity of each reference dish and its weight is added together, Get the recommendation priority of the final recommended dishes.

Step 617: Determine recommended information of the dish, where the recommended information of the dish includes: the at least one dish and the recommendation priority of the at least one dish.

After determining the at least one recommended dish and the recommendation priority of each recommended dish, the server can determine the dish recommendation information corresponding to the first refrigeration device. The recommended information of the dish includes: the identifier of the at least one recommended dish and the recommendation priority of each recommended dish in the at least one recommended dish. For example, the identifier of each recommended dish may include a representative picture and name of the recommended dish. .

Step 618: Send the determined recommendation information of the dish to the first refrigerating device.

Step 619: The first refrigerating device displays the at least one dish according to the recommendation priority of the at least one dish in the received dish recommendation information.

Since the recommended dish with a higher recommendation priority has a higher similarity with the reference dish preferred by the user and is more in line with the user's preference, the information of the recommended dish can be displayed in a more conspicuous position on the display screen, so that the user can quickly Pay attention to the recommended dishes to avoid frustration when users do not find a satisfactory dish after viewing multiple dishes, and improve the user experience. In some embodiments, since the user usually watches the contents on the display screen in the order from left to right and from top to bottom, the refrigeration device may recommend the items with higher priority among the recommended information of the received dishes. The recommended dishes are displayed in the upper and left positions of the display screen, so that the user can notice the recommended dishes more quickly, and then quickly find the dishes that meet their expectations.

In some embodiments, the display position of the recommended dish with a higher recommendation priority may also be set by the user. If a user is accustomed to viewing the content displayed on the display screen from the middle position of the display screen first, the user can set the middle area of the display screen as an area for displaying recommended dishes with a higher recommendation priority. Furthermore, the refrigerating apparatus may display the recommended dishes with higher recommendation priority in the middle area of the display screen according to the received recommended information of the dishes.

In some embodiments, the server sends recipes of each dish that may also be included in the dish recommendation information to the refrigerating device, and the user can operate on each dish displayed on the display screen to trigger the display screen to display the dish. recipe. In some embodiments, the recommended information of the dish may also include other dishes in addition to the at least one recommended dish above. For example, the other dishes may include dishes that are most popular with users, dishes that are the most convenient to prepare, and dishes with relatively low fat content. There are few dishes, etc., the ingredients required for the preparation of these dishes may be independent of the ingredients stored in the refrigerating device.

In some embodiments, when the refrigerating device displays the recommended dish according to the received recommended dish, the user also performs a marking operation on the displayed recommended dish, and then the refrigerating device can determine that the recommended dish is a target dish, and determines to receive the recommended dish. After receiving the marking instruction of the recommended dish, a marking request of the recommended dish can be sent to the server. For details, please refer to the introduction of step 607 to step 609, and some embodiments of the present application will not be described again.

In some embodiments, the server may determine the recommendation information of the dish according to the preference information corresponding to the ingredients stored in the refrigeration device and the target account. The above embodiments of the present application are described by taking the preference information reflected by the reference dishes marked by the user's history, and then determining the recommendation priority of the dishes according to the similarity between the dishes and the reference dishes. In some embodiments, the preference information can also be reflected by other information input by the user, for example, the user can directly input his own taste and food preference, such as a sweet taste, a preference for tomatoes, etc.; and then the server can according to the other information information to prioritize recommendations for dishes. Alternatively, the server may also infer the user's taste and food preference according to the reference dishes marked by the user's history, and then determine the recommended priority of dishes according to the taste and food preference.

In some embodiments of the present application, the at least one dish in the recommended information of dishes received by the refrigerating device may include: all or part of the dishes that can be made using ingredients stored in the refrigerating device, which are not limited in some embodiments of the present application . In some embodiments, the user can input information such as unacceptable tastes and ingredients through the display screen, and the refrigerating device can send the information and the target account to the server, and when the server receives the dish recommendation request carrying the target account, The dishes in the determined recommendation information may not include dishes made with the ingredients and dishes with the taste.

In some embodiments, the above embodiments are explained by taking as an example that only one target account is logged into the refrigerating device. In some embodiments, multiple target accounts may be logged on the refrigeration device, for example, a first target account and a second target account may be logged in the first refrigeration device. The server may determine the recommended information of the dish and the account carried in the tag request according to the currently selected target account among the multiple target accounts logged in the refrigerating device. If the currently selected first target account is the first target account, the server determines the recommendation information of the dish according to the reference dish corresponding to the first target account. If the currently selected second target account is the second target account, the server determines the recommendation information of the dish according to the reference dish corresponding to the second target account. When a certain dish is marked on the page corresponding to the first target account, the generated mark request carries the first target account. When a certain dish is marked on the page corresponding to the second target account, the generated mark request carries the second target account. In some embodiments, please continue to refer to FIG. 14 , three target accounts can be logged into the refrigerating device, and the three target accounts are marked by "Dad", "Mom" and "Dabao" respectively. The user can select his own target account among the three target accounts, and the target account selected by the user is regarded as the target account currently used by the refrigeration device. When the user selects different target accounts, the recommended dishes displayed on the display are also different.

In the related art, the information of the ingredients stored in the refrigeration device is only marked by the account dimension. When multiple refrigerating devices log in to the same account, when recommending dishes, each refrigerating device will recommend dishes based on all the ingredients stored in the multiple refrigerating devices. In this way, the ingredients for preparing a recommended dish seen by a user from a refrigerating device may not be stored in the refrigerator, and the accuracy of the dish recommendation is low.

However, in some embodiments of the present application, the server may store the preference information of the user corresponding to the target account and the information of the ingredients stored in the refrigerating device, that is, the server stores the user-related information from the account dimension, and stores the information in the device from the device dimension. Food related information. The target account can be logged in on multiple refrigerating devices, and the server can overall determine the reference dishes corresponding to the target account according to the marking operations performed by the user on the dishes on the multiple refrigerating devices. In this way, it can be ensured that the server collects more comprehensive preference information of the user, and then can more accurately recommend dishes that meet the user's preference, thereby improving the accuracy of dish recommendation. And when determining the recommended information of the dish, the server may only determine the dishes that can be made from the ingredients stored in the refrigeration device that sends the dish recommendation request, and then the dishes recommended by the refrigeration device to the user are all available through the ingredients stored in the refrigeration device. prepared directly. It avoids recommending dishes based on the ingredients stored in each refrigerating device logged in with the target account, resulting in the situation that the ingredients of the dishes required for some dishes are not stored in the refrigerating device currently operated by the user, which improves the effectiveness of the recommended information for dishes. sex and practicality.

In some embodiments of the present application, the steps performed by the refrigerating apparatus may be performed by a controller therein, and the steps performed by the server may be performed by a processing unit therein; for the display steps performed by the refrigerating apparatus, the controller of the refrigerating apparatus may control the display to display For the step of interacting with the server performed by the refrigerating equipment, it can be realized by the controller in the refrigerating equipment through the communication interface; As for the step of interacting with the refrigerating equipment performed by the server, it can be realized by the processing unit in the server through the network interface unit .

In some embodiments, the first refrigerating device and the second refrigerating device in some embodiments of the present application are only any two refrigerating devices connected to the server and logged in with the same target account, and also for other refrigerating devices logged in with the target account. The above steps can be performed. In some embodiments of the present application, the information recommendation method is described by taking as an example that both the first refrigerating device and the second refrigerating device receive the dish marking instruction. In some embodiments, only one refrigerating device may receive the dish marking instruction. Some embodiments of the present application are not limited. In addition, some embodiments of the present application only take the first refrigerating device receiving a dish recommendation instruction as an example to describe the information recommendation method. In some embodiments, the second refrigerating device can also perform the above step 613 if it receives a dish recommendation instruction. Go to step 619.

To sum up, in the information recommendation methods provided by some embodiments of the present application, the dishes recommended by the refrigeration device to the user are determined according to the ingredients stored in the refrigeration device and the reference dishes corresponding to the target account logged in the refrigeration device. If the reference dishes corresponding to the target accounts are different, the dishes recommended by the refrigerating device may also be different, thus improving the flexibility of the dish recommendation of the refrigerating device.

For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussions are not intended to be exhaustive or to limit implementations to the specific forms disclosed above. Numerous modifications and variations are possible in light of the above teachings. The above embodiments are chosen and described to better explain the principles and practical applications, so as to enable those skilled in the art to better utilize the described embodiments and various modified embodiments suitable for specific use considerations.

Claims

A display device comprising:

monitor;

user interface;

Communicator, used to display the connection between the device and the server;

a first controller connected to the display, the communicator and the user interface, respectively, for performing:

Receive the first operation of the user, and play audio and video according to the received first operation;

When the user leaves, the playback record of the audio and video is sent to the server, and the server records the playback when receiving the acquisition request sent by the electronic device corresponding to the display device in response to the user's triggering operation. Send it to the electronic device, so that the electronic device continues to play the audio and video according to the playback record, wherein the playback record includes the audio and video identifier and the playback progress.
The display device according to claim 1, wherein the when the user leaves is when a pause operation is received as an input.
The display device according to claim 2, after receiving the input pause operation, the first controller is further configured to:

Receive the playback operation input by the user;

When the playback record recorded by the server is updated, play the audio and video according to the received updated playback record;

When the playback record recorded by the server is not updated, the audio and video are continued to be played according to the playback record after receiving the pause operation.
The display device according to claim 3, wherein when the playback record recorded by the server is updated, the audio and video are played according to the received updated playback record; the playback record recorded on the server side is not updated , the steps of playing the audio and video according to the playback record after receiving the pause operation include:

In response to the playback operation, sending an acquisition request for the playback record to the server;

receiving the playback record fed back by the server in response to the request for obtaining the playback record;

When the playback record of the feedback is different from the playback record during the pause, play the audio and video according to the playback record of the feedback;

When the playback record of the feedback is the same as the playback record at the time of pause, the audio and video are continued to be played from the playback record at the time of pause.
The display device of claim 1, further comprising a human body detector;

Before sending the audio and video playback records to the server, the first controller is further configured to:

Obtain the detection signal input by the detector;

When the detection signal indicates that the user has left the preset viewing range, it is determined that the user has left, and the playback record of the audio and video is sent to the server;

When the detection signal indicates that the user has not left the preset viewing range, the playback record of the audio and video is not sent to the server.
The display device according to claim 5, when the detection signal indicates that the user leaves a preset viewing range, the first controller is further configured to: pause the playback of the audio and video.
The display device according to claim 6, after pausing the playback of the audio and video, the first controller is further configured to:

When the detection signal indicates that the user enters the preset viewing range, determine whether there is an update in the playback record recorded by the server;

When the playback record recorded by the server is updated, play the audio and video according to the received updated playback record;

When the playback record recorded by the server is not updated, the audio and video are continued to be played according to the playback record after receiving the pause operation.
The display device of claim 1, the method further comprising:

Receive user trigger actions;

In response to the acquisition request sent to the server by the trigger operation, the server sends to the display device the playback record of the audio and video sent by the electronic device to the server when the user leaves, wherein the playback record includes audio and video identification and playback progress;

Continue to play the audio and video at the playback progress according to the audio and video identifier.
An electronic device comprising:

Display screen, used to display the interactive interface;

a second controller, configured to perform: receiving a trigger operation;

In response to the triggering operation, acquiring a playback record of audio and video on a display device corresponding to the electronic device;

Acquire the audio and video according to the audio and video identifier of the playback record, and play the audio and video from the acquired playback progress, wherein the playback record includes the audio and video identifier and the playback progress.
The electronic device according to claim 9, further comprising a door body, the display screen is arranged on the door body, and the triggering operation is:

initiating the operation of said display screen; or,

an operation to switch the display screen from a first mode to a second mode, wherein the first mode is an ingredient management mode and the second mode is a different mode from the ingredient management mode; or,

The operation of opening the door body.
The electronic device according to claim 9, wherein in response to the trigger operation, the step of acquiring the playback record corresponding to the user account comprises:

Send a playback record acquisition request to the server according to the account information of the electronic device;

Receive the playback record fed back by the server.
The electronic device according to claim 9, wherein the step of playing the audio and video from the obtained playback record comprises:

Obtain the playback record corresponding to the user account from the server, and query the playback status of the display device;

In response to the state of the display device being a paused playback state, the upper layer of the user interface of the control display screen displays a synchronization prompt message to prompt the user to synchronously watch the audio and video played on the display device;

In response to selecting the viewing control in the synchronous prompt message, the control is not to display the synchronous prompt message on the upper layer of the user interface of the display screen, the corresponding audio and video are played according to the play record, and the play record in the server is updated in real time. .
A method for synchronizing audio and video progress, comprising:

Receive the first operation of the user, and play audio and video according to the received first operation;

Send the playback record of the audio and video to the server, and when the server receives an acquisition request sent by the electronic device corresponding to the display device in response to the user's triggering operation, the server sends the playback record to the electronic device. device, so that the electronic device continues to play the audio and video according to the playing record, wherein the playing record includes the audio and video identification and the playing progress.
A method for synchronizing audio and video progress, comprising:

receive trigger action;

In response to the trigger operation, obtain the playback record corresponding to the user account;

Acquire the audio and video according to the audio and video identifier of the playback record, and play the audio and video from the acquired playback progress, wherein the playback record includes the audio and video identifier and the playback progress.