CN115150643A

CN115150643A - Application awakening method, display device, mobile terminal and server

Info

Publication number: CN115150643A
Application number: CN202110583580.1A
Authority: CN
Inventors: 王光强; 邱若强
Original assignee: Juhaokan Technology Co Ltd
Current assignee: Juhaokan Technology Co Ltd
Priority date: 2021-03-30
Filing date: 2021-05-27
Publication date: 2022-10-04
Also published as: WO2022206392A1; CN117501700A; CN115150644B; CN115150644A

Abstract

The embodiment of the application provides an application awakening method of a display device, the display device, a mobile terminal and a server, wherein the method comprises the following steps: and if the display equipment is in a starting-up state, the display equipment receives a first awakening instruction from the server, wherein the first awakening instruction is an instruction generated according to the first connection request. And the display equipment processor controls the preset application to be started according to the first awakening instruction. And if the display equipment is in the starting-up state, the display equipment receives a second awakening instruction from the server, wherein the second awakening instruction is an instruction generated according to the second connection request. And after the processor of the display equipment controls the display equipment to enter the starting state from the standby state according to the second awakening instruction, the processor of the display equipment controls the preset application to be started according to the second awakening instruction. The method provided by the application can realize remote awakening of the application installed on the display device, so that user experience is improved.

Description

Application awakening method, display device, mobile terminal and server

The present application claims priority of chinese patent application entitled "display device wake-up method, display device, mobile terminal, and server" filed by chinese patent office on 30/03/2021 with application number 202110342826.6, which is incorporated herein by reference in its entirety.

Technical Field

The application relates to the technical field of display equipment, in particular to an application awakening method, display equipment, a mobile terminal and a server.

Background

In the prior art, a television is generally provided with a key and a remote control receiving assembly, the television controls the television to be started by receiving key operation of a user or receiving a starting key value of a remote controller through the remote control receiving assembly, and the television generally directly enters a preset system program after being started.

Disclosure of Invention

In a first aspect, the present embodiment provides a mobile terminal, including: a communication interface for communication between the processor and a server; a processor to:

responding to the received operation, displaying an online list of display equipment, wherein a display equipment control in the online list is marked as one of a starting state, a standby state or a shutdown state, the starting state represents a state that a main processor of the corresponding display equipment, which is provided with an operating system, is in a starting state, the standby state represents a state that the main processor of the corresponding display equipment is in a non-starting state but a sub processor is in a starting state, and the main processor of the corresponding display equipment is in a non-starting state and the sub processor is in a non-starting state;

receiving an operation on a first display device control in the online list;

when the first display device control is marked to be in a starting state or a standby state, sending a connection request to the server, wherein the connection request comprises a first display device identifier of first display device corresponding to the first display device control, the connection request is used for starting a preset application in the first display device, and a call connection is established through the preset application;

displaying a prompt indicating that the first display device is not connectable when the first display device control is marked as an off-key state.

In a second aspect, the present embodiments provide a server comprising a memory and a processor, the processor configured to:

receiving a connection request sent by a mobile terminal, wherein the connection request comprises a first display device identifier of a first display device corresponding to the first display device control;

responding to the connection request, establishing a virtual session room, and feeding back a message representing to-be-connected to the mobile terminal;

and sending a wake-up instruction to the first display device, wherein the wake-up instruction includes a room identifier corresponding to the virtual conversation room, and the wake-up instruction is used for enabling the first display device to start a preset application and enabling the preset application in the first display device to establish conversation connection according to the room identifier.

In a third aspect, the present embodiment provides a display device including:

a display;

a communicator to: receiving a wake-up instruction, sending the wake-up instruction to a main processor when the main processor of the display device is in a starting state, and sending the wake-up instruction to a sub-processor when the main processor of the display device is in a non-starting state and the sub-processor is in a starting state, wherein the wake-up instruction comprises an instruction for starting a preset application;

the primary processor configured to: after the awakening instruction is received, starting a preset application corresponding to the instruction for starting the preset application;

the sub-processor configured to: and after receiving the awakening instruction, controlling the main processor to start, and sending the instruction for starting the preset application to the main processor.

In a fourth aspect, this embodiment provides an application wake-up method, where the method includes:

when the first display device is determined to be in a power-on state, sending a first awakening instruction to the first display device, wherein the first awakening instruction comprises an instruction for starting a preset application, and the instruction for starting the preset application is used for enabling a main processor in the first display device to start the preset application;

and when the first display equipment is determined to be in a standby state, sending a second awakening instruction to the first display equipment, wherein the second awakening instruction comprises a starting instruction and a preset application starting instruction, the starting instruction is used for enabling a sub-processor in the first display equipment to control a main processor in the first display equipment to start, and the preset application starting instruction is used for enabling the main processor in the first display equipment to start a preset application.

The application awakening method, the display device, the mobile terminal and the server can realize remote awakening of the application installed on the display device, and therefore user experience is improved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

FIG. 1 illustrates a usage scenario of a display device according to some embodiments;

fig. 2 illustrates a block diagram of a hardware configuration of a control apparatus according to some embodiments;

FIG. 3A illustrates a block diagram of a hardware configuration of a display device according to some embodiments;

FIG. 3B illustrates a block diagram of a hardware configuration of a display device according to some embodiments

FIG. 4 illustrates a software configuration diagram in a display device according to some embodiments;

FIG. 5 illustrates an icon control interface display of an application in a display device in accordance with some embodiments;

FIG. 6 illustrates a remote wake-up system diagram of a display device according to some embodiments;

FIG. 7 illustrates a schematic diagram of a user interface in a mobile terminal according to some embodiments;

FIG. 8 illustrates a schematic user interface in a mobile terminal according to some embodiments;

FIG. 9 illustrates a schematic user interface in a mobile terminal according to some embodiments;

FIG. 10 illustrates an application wakeup method signaling diagram for a display device according to some embodiments;

FIG. 11 illustrates a signaling diagram of an application wakeup method for a further display device, in accordance with some embodiments.

Detailed Description

To make the purpose and embodiments of the present application clearer, the following will clearly and completely describe the exemplary embodiments of the present application with reference to the attached drawings in the exemplary embodiments of the present application, and it is obvious that the described exemplary embodiments are only a part of the embodiments of the present application, and not all the embodiments.

The terms "first," "second," "third," and the like in the description and claims of this application and in the above-described drawings are used for distinguishing between similar or analogous objects or entities and not necessarily for describing a particular sequential or chronological order, unless otherwise indicated. It is to be understood that the terms so used are interchangeable under appropriate circumstances.

Fig. 1 is a schematic diagram of a usage scenario of a display device according to an embodiment. As shown in fig. 1, the display apparatus 200 is also in data communication with a server 400, and a user can operate the display apparatus 200 through the smart device 300 or the control device 100.

In some embodiments, the control apparatus 100 may be a remote controller, and the communication between the remote controller and the display device includes at least one of an infrared protocol communication or a bluetooth protocol communication, and other short-distance communication methods, and controls the display device 200 in a wireless or wired manner. The user may control the display apparatus 200 by inputting a user instruction through at least one of a key on a remote controller, a voice input, a control panel input, and the like.

In some embodiments, the smart device 300 may include any of a mobile terminal 300A, a tablet, a computer, a laptop, an AR/VR device, and the like.

In some embodiments, the smart device 300 may also be used to control the display device 200. For example, the display device 200 is controlled using an application program running on the smart device.

In some embodiments, the smart device 300 and the display device may also be used for communication of data.

In some embodiments, the display device 200 may also be controlled in a manner other than the control apparatus 100 and the smart device 300, for example, the voice instruction control of the user may be directly received by a module configured inside the display device 200 to obtain a voice instruction, or may be received by a voice control apparatus provided outside the display device 200.

In some embodiments, the display device 200 is also in data communication with a server 400. The display device 200 may be allowed to be communicatively connected through a Local Area Network (LAN), a Wireless Local Area Network (WLAN), and other networks. The server 400 may provide various contents and interactions to the display apparatus 200. The server 400 may be a cluster or a plurality of clusters, and may include one or more types of servers.

In some embodiments, software steps executed by one step execution agent may be migrated on demand to another step execution agent in data communication therewith for execution. Illustratively, software steps performed by the server may be migrated to be performed on a display device in data communication therewith, and vice versa, as desired.

Fig. 2 exemplarily shows a block diagram of a configuration of the control apparatus 100 according to an exemplary embodiment. As shown in fig. 2, the control device 100 includes a controller 110, a communication interface 130, a user input/output interface 140, a memory, and a power supply. The control apparatus 100 may receive an input operation instruction from a user and convert the operation instruction into an instruction recognizable and responsive by the display device 200, serving as an interaction intermediary between the user and the display device 200.

In some embodiments, the communication interface 130 is used for external communication, and includes at least one of a WIFI chip, a bluetooth module, NFC, or an alternative module.

In some embodiments, the user input/output interface 140 includes at least one of a microphone, a touchpad, a sensor, a key, or an alternative module.

Fig. 3A illustrates a hardware configuration block diagram of the display apparatus 200 according to an exemplary embodiment.

In some embodiments, the display apparatus 200 includes at least one of a tuner demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, a user interface.

In some embodiments the controller comprises a central processor, a video processor, an audio processor, a graphics processor, a RAM, a ROM, a first interface to an nth interface for input/output.

In some embodiments, the display 260 includes a display screen component for displaying pictures, and a driving component for driving image display, a component for receiving image signals from the controller output, displaying video content, image content, and menu manipulation interface, and a user manipulation UI interface, etc.

In some embodiments, the display 260 may be at least one of a liquid crystal display, an OLED display, and a projection display, and may also be a projection device and a projection screen.

In some embodiments, the tuner demodulator 210 receives broadcast television signals via wired or wireless reception, and demodulates audio/video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

In some embodiments, communicator 220 is a component for communicating with external devices or servers according to various communication protocol types. For example: the communicator may include at least one of a Wifi module, a bluetooth module, a wired ethernet module, and other network communication protocol chips or near field communication protocol chips, and an infrared receiver. The display apparatus 200 may establish transmission and reception of control signals and data signals with the control device 100 or the server 400 through the communicator 220.

In some embodiments, the detector 230 is used to collect signals of the external environment or interaction with the outside. For example, detector 230 includes a light receiver, a sensor for collecting ambient light intensity; alternatively, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, attributes of the user, or user interaction gestures, or the detector 230 includes a sound collector, such as a microphone, which is used to receive external sounds.

In some embodiments, the external device interface 240 may include, but is not limited to, the following: high Definition Multimedia Interface (HDMI), analog or data high definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, and the like. Or may be a composite input/output interface formed by the plurality of interfaces.

In some embodiments, the controller 250 and the modem 210 may be located in different separate devices, that is, the modem 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

In some embodiments, the controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in memory. The controller 250 controls the overall operation of the display apparatus 200. For example: in response to receiving a user command for selecting a UI object to be displayed on the display 260, the controller 250 may perform an operation related to the object selected by the user command.

In some embodiments, the object may be any one of selectable objects, such as a hyperlink, an icon, or other actionable control. The operations related to the selected object are: displaying an operation connected to a hyperlink page, document, image, or the like, or performing an operation of a program corresponding to the icon.

In some embodiments the controller comprises at least one of a Central Processing Unit (CPU), a video processor, an audio processor, a Graphics Processing Unit (GPU), a RAM Random Access Memory (RAM), a ROM (Read-Only Memory), a first to nth interface for input/output, a communication Bus (Bus), and the like.

A CPU processor. For executing operating system and application program instructions stored in the memory, and executing various application programs, data and contents according to various interactive instructions receiving external input, so as to finally display and play various audio-video contents. The CPU processor may include a plurality of processors.

In some embodiments, the CPU processor may include a main processor and a sub-processor. The display device may be configured to be in a power-on state and a standby state according to a user's operation. When the display device is configured in the power-on state, the audio-video system of the display device works normally to output sound or graphics. At this time, the main processor is in a normal operating state, and the sub-processors may or may not be in an operating state. When the display device is configured in the standby state, the audio-video system of the display device does not operate, and the output of sound or graphics is not performed. At this time, the sub-processors are in an operating state, and the main processor is not in an operating state. And the main processor is a processor which can receive, decode and output the audio and video code stream sent by the server. The sub-processor is a processor which can receive a simple instruction sent by the server and wake up the main processor. The subprocessors do not have the audio and video processing function of the main processor.

In some embodiments, the main processor and the sub-processors may be packaged in one chip or may be packaged separately. The embodiment shown in fig. 3B is an embodiment in which the main processor and the sub-processors are packaged separately. In a separately packaged embodiment, the sub-processors are connected on one side to the communicator and on one side to the main processor. When the display device is configured in a standby state, the sub-processor acquires an instruction from the server through the communicator, and wakes up the main processor according to the instruction.

In some embodiments, the communicator may be connected to both the main processor and the sub-processors.

In some embodiments, a graphics processor for generating various graphics objects, such as: at least one of an icon, an operation menu, and a user input instruction display figure. The graphic processor comprises an arithmetic unit, which performs operation by receiving various interactive instructions input by a user and displays various objects according to display attributes; the system also comprises a renderer for rendering various objects obtained based on the arithmetic unit, wherein the rendered objects are used for being displayed on a display.

In some embodiments, the video processor is configured to receive an external video signal, and perform at least one of video processing such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, and image synthesis according to a standard codec protocol of the input signal, so as to obtain a signal displayed or played on the direct display device 200.

In some embodiments, the video processor includes at least one of a demultiplexing module, a video decoding module, an image composition module, a frame rate conversion module, a display formatting module, and the like. The demultiplexing module is used for demultiplexing the input audio and video data stream. And the video decoding module is used for processing the video signal after demultiplexing, including decoding, scaling and the like. And the image synthesis module is used for carrying out superposition mixing processing on the GUI signal input by the user or generated by the user and the video image after the zooming processing by the graphic generator so as to generate an image signal for display. And the frame rate conversion module is used for converting the frame rate of the input video. And the display formatting module is used for converting the received video output signal after the frame rate conversion, and changing the signal to be in accordance with the signal of the display format, such as an output RGB data signal.

In some embodiments, the audio processor is configured to receive an external audio signal, decompress and decode the received audio signal according to a standard codec protocol of the input signal, and perform at least one of noise reduction, digital-to-analog conversion, and amplification processing to obtain a sound signal that can be played in the speaker.

In some embodiments, a user may enter user commands on a Graphical User Interface (GUI) displayed on display 260, and the user input interface receives the user input commands through the Graphical User Interface (GUI). Alternatively, the user may input the user command by inputting a specific sound or gesture, and the user input interface receives the user input command by recognizing the sound or gesture through the sensor.

In some embodiments, a "user interface" is a media interface for interaction and information exchange between an application or operating system and a user that enables conversion between an internal form of information and a form that is acceptable to the user. A commonly used presentation form of the User Interface is a Graphical User Interface (GUI), which refers to a User Interface related to computer operations and displayed in a graphical manner. It may be an interface element such as an icon, a window, a control, etc. displayed in the display screen of the electronic device, where the control may include at least one of an icon, a button, a menu, a tab, a text box, a dialog box, a status bar, a navigation bar, a Widget, etc. visual interface elements.

In some embodiments, user interface 280 is an interface that may be used to receive control inputs (e.g., physical buttons on the body of the display device, or the like).

In some embodiments, the system of the display device may include a Kernel (Kernel), a command parser (shell), a file system, and an application. The kernel, shell, and file system together form the basic operating system structure that allows users to manage files, run programs, and use the system. After power-on, the kernel is started, kernel space is activated, hardware is abstracted, hardware parameters are initialized, and virtual memory, a scheduler, signals and interprocess communication (IPC) are operated and maintained. And after the kernel is started, loading the Shell and the user application program. The application program is compiled into machine code after being started, and a process is formed.

Referring to fig. 4, in some embodiments, the system is divided into four layers, which are an Application (Applications) layer (abbreviated as "Application layer"), an Application Framework (Application Framework) layer (abbreviated as "Framework layer"), an Android runtime (Android runtime) and system library layer (abbreviated as "system runtime library layer"), and a kernel layer from top to bottom.

In some embodiments, at least one application program runs in the application program layer, and the application programs may be windows (Window) programs carried by an operating system, system setting programs, clock programs or the like; or an application developed by a third party developer. In particular implementations, the application packages in the application layer are not limited to the above examples.

The framework layer provides an Application Programming Interface (API) and a programming framework for the application program of the application layer. The application framework layer includes a number of predefined functions. The application framework layer acts as a processing center that decides to let the applications in the application layer act. The application program can access the resources in the system and obtain the services of the system in execution through the API interface.

As shown in fig. 4, in the embodiment of the present application, the application framework layer includes a manager (Managers), a Content Provider (Content Provider), and the like, where the manager includes at least one of the following modules: an Activity Manager (Activity Manager) is used for interacting with all activities running in the system; a Location Manager (Location Manager) for providing access to the system Location service to the system service or application; a Package Manager (Package Manager) for retrieving various information about an application Package currently installed on the device; a Notification Manager (Notification Manager) for controlling display and clearing of Notification messages; the Window Manager (Window Manager) is used for managing icons on the user interface windows, toolbars, wallpaper, and desktop components.

In some embodiments, the activity manager is used to manage the lifecycle of the various applications as well as general navigational fallback functions, such as controlling exit, opening, fallback, etc. of the applications. The window manager is used for managing all window programs, such as obtaining the size of a display screen, judging whether a status bar exists, locking the screen, intercepting the screen, controlling the change of the display window (for example, reducing the display window, displaying a shake, displaying a distortion deformation, and the like), and the like.

In some embodiments, the system runtime layer provides support for the upper layer, i.e., the framework layer, and when the framework layer is used, the android operating system runs the C/C + + library included in the system runtime layer to implement the functions to be implemented by the framework layer.

In some embodiments, the kernel layer is a layer between hardware and software. As shown in fig. 4, the core layer includes at least one of the following drivers: audio drive, display driver, bluetooth drive, camera drive, WIFI drive, USB drive, HDMI drive, sensor drive (like fingerprint sensor, temperature sensor, pressure sensor etc.) and power drive etc..

In some embodiments, the display device may directly enter an interface of a preset video-on-demand program after being started, and the interface of the video-on-demand program may include at least a navigation bar 510 and a content display area located below the navigation bar 510, as shown in fig. 5, where content displayed in the content display area may change according to a change of a selected control in the navigation bar. The programs in the application program layer can be integrated in the video-on-demand program and displayed through one control of the navigation bar, and can also be further displayed after the application control in the navigation bar is selected.

In some embodiments, the display device may directly enter a display interface of a signal source selected last time after being started, or a signal source selection interface, where the signal source may be a preset video-on-demand program, or may be at least one of an HDMI interface and a live tv interface, and after a user selects a different signal source, the display may display content obtained from the different signal source.

The remote awakening is a process of enabling the intelligent television to be awakened to enter a starting state from the state to be awakened by sending a specific data packet to the intelligent television in the state to be awakened (a dormant state or a standby state) through matching of the network card with other software and hardware.

However, the remote wake-up currently only can realize remote wake-up of the smart tv in the standby state, the application on the smart television cannot be remotely awakened, so that the use experience of the user is poor.

In order to solve the above problem, as shown in fig. 6, the present application provides a remote wake-up system including at least one display device 200, a server 400, and at least one mobile terminal 300A (only one mobile terminal is shown in the figure).

Based on the remote awakening system, the specific process of remote awakening is as follows:

if a user turns on a certain display device in the system (which may be turned on in a normal mode through a remote controller), a main chip (main processor) of the display device is in an activated state. The low power consumption chip (sub-processor) is in an inactivated state. In this state, the main chip of the display device continuously reports the state information of the display device to the server. Specifically, the heartbeat data is uploaded, and the server determines that the display device is in a starting state according to the heartbeat data uploaded by the main chip. In some embodiments, the main chip is an implementation form of a main processor, and the low-power chip is an implementation form of a sub-processor. In some embodiments, the main processor and the sub-processors may be packaged in the same chip.

If the user turns off a certain display device in the system (which may be by turning off the display device via a remote control so that the display device is in a standby state). The main chip of the display device is in a closed and non-started state, and before entering the non-started state, the main chip triggers the low-power chip to enter the started state. In this state, the low-power chip continuously uploads heartbeat data to the server, and the server determines that the display device is in a standby state according to the heartbeat data uploaded by the low-power chip. And the server generates an online list of the display equipment according to the equipment state information reported by the at least one display equipment.

The low-power chip is a chip which still works after the main chip is closed when the display device is in standby. The computing power and power consumption of the chip are strongly correlated, and a low power chip may be used only to implement wake-up related functions. By arranging the low-power chip, the state information of the display equipment can be continuously reported to the server on the premise of saving energy. The main Chip may be a System-on-a-Chip (SOC) Chip, and the low power consumption Chip may be a Digital Signal Processing (DSP) Chip or an ARM core-M4 (embedded processor developed by ARM corporation, uk). In this embodiment of the present application, the low power consumption chip may be integrated on the main chip, or may be separated from the main chip, and this embodiment of the present application is not limited.

In some embodiments, the main processor has a functional unit therein that communicates with the communicator, so that the sub-processors can be shut down when the main processor is started. The communicator transmits a signal to the main processor when the main processor is started and transmits a signal to the sub-processors when the sub-processors are started. When the main processor and the sub processor work, different heartbeat data are sent to the server, so that the server confirms whether the display equipment is in a starting state or a standby state. In some embodiments, the different heartbeat data may be different in characteristics of heartbeat data, or may carry different characterization identifiers.

In some embodiments, the main processor does not include a functional unit for communicating with the communicator, and it is necessary to maintain the sub-processors on when the processor is started. The communication of the main processor must pass through the sub-processor to be transmitted from the communicator, and the signal is received through the sub-processor.

In some embodiments, the sub-processors may send different heartbeat data depending on the on-off state of the main processor.

In some embodiments, the sub-processors autonomously send the first heartbeat data to the server when the main processor is not operating, and send the heartbeat data of the main processor to the server as the second heartbeat data when the main processor is operating without sending the first heartbeat data.

In some embodiments, a preset application, such as a mini-home application (audio-video communication application), a convergence application (audio-video audiovisual application), a remote control application, or the like, is installed on the mobile terminal. Taking an audio and video communication application as an example, after a user opens a preset application on a mobile terminal, a page data request (namely, a state query request) is sent to a server, and the server feeds back a display device online list to the mobile terminal according to the page data request.

In some embodiments, the preset application is an application executed in an operating system, for example, an operating program of a television itself is a system program, the system program is executed on a machine as long as a user starts the television, the preset application is a program of an application layer, if only the television is started and the preset application is not started, no thread is used to execute the preset application afterwards, the preset application is started on the premise that the system program has a condition of starting other applications in the system program, and for a chip, a starting process of the chip is a process of loading the system program.

In some embodiments, by opening the preset application, the user may view the display device online list. The online list includes at least one control of the display device. The display device online list includes display devices (display devices for establishing an association relationship) which are authorized to be accessed by a current account (an account of a current mobile terminal logging in a preset application). And presenting corresponding state identifications in the online list of the display equipment according to the states of different display equipment.

Illustratively, the display device a continuously reports the state information of the display device a to the server for the main chip, and the reported data may carry a tag of the main chip. Through the label, the server can judge that the display device A is in a power-on state. The display device B continuously reports the state information of the display device a to the server for the low power consumption chip, and the reported data can carry a tag of the low power consumption chip. Through the tag, the server can determine that the display apparatus B is in the standby state. The display device C is in a power-off state (completely powered-off state), so the server cannot receive the state information reported by the display device C, and the server can judge that the display device C is in the power-off state.

And the server generates an online list of the display equipment according to the state information reported by each display equipment and feeds the online list of the display equipment back to the mobile terminal. And simultaneously displaying the online list of the display equipment on a preset application page of the mobile terminal. A schematic view of a user interface of a mobile terminal as shown in fig. 7. And if the display device A is in an online state (a power-on state), displaying an online state identifier. And if the display device B is in the standby state, displaying a standby state identifier. And if the display equipment C is in an off-line state (shutdown state), displaying a shutdown state identifier or displaying an off-line state identifier.

The online list of display devices may include all display devices that have been authorized to be accessible to the current account, i.e., display devices in the above three states. The display device presence list may also contain only display devices in the presence state and the standby state. The display device online list may also contain only display devices that are in a standby state. Therefore, on one hand, page resources are saved, and on the other hand, a user can conveniently select display equipment capable of being wakened remotely.

In some embodiments, the status information reported by the display device to the server is in the form of heartbeat data packets. The display device periodically sends heartbeat data packets to the server, and the server judges whether the display device is in an online state or a standby state. If the heartbeat data packet sent by the display equipment is not received by the server when the preset time threshold value is exceeded, for example, the preset time threshold value exceeds one minute, the display equipment is judged to be in an off-line state. And the server feeds back the updated online list of the display equipment to the mobile terminal, and the mobile terminal changes the state identifier of the display equipment into an offline state identifier on a user interface. This may be the case when the user suddenly unplugs the power cord of the display device, causing the display device to go from a standby state to an offline state.

After the user looks up the online list of the display equipment on a preset application interface of the mobile terminal, the user can select the display equipment needing to be connected. The mobile terminal generates a connection request according to the selection of the user and sends the connection request to the server. The generated connection request carries the device information of the selected display device.

In some embodiments, if the display device selected by the user on the preset application interface of the mobile terminal is in a power-on state, the first connection request is sent to the server. The first connection request carries device information of the display device. And after receiving the first connection request, the server searches the corresponding display equipment according to the equipment information. The server further generates a first awakening instruction according to the equipment state of the display equipment and sends the first awakening instruction to the corresponding display equipment. Because the display device is in the power-on state, the first wake-up instruction is only used for starting the preset application. And finally, the processor of the display device controls the preset application to be started according to the first awakening instruction.

In some embodiments, the communicator of the display device receives and sends the first wake-up instruction to the main chip. And the main chip responds to the mark of the preset application in the first awakening instruction and starts the preset application.

In some embodiments, when the display device is in the power-on state, the first wake-up instruction may be a preset application start instruction.

In some embodiments, if the display device selected by the user on the preset application interface of the mobile terminal is in a standby state, the second connection request is transmitted to the server. The second connection request carries device information of the display device. And after receiving the second connection request, the server searches the corresponding display equipment according to the equipment information. The server further generates a second awakening instruction according to the equipment state of the display equipment and sends the second awakening instruction to the corresponding display equipment. Since the display device is in the standby state, the second wake-up instruction is not only used for starting the preset application, but also used for the controller display device to enter the power-on state from the standby state. The processor of the display device controls the display device to enter a starting state from a standby state according to the second awakening instruction, and then controls the preset application to be started according to the second control awakening instruction.

In some embodiments, the first connection request and the second connection request may be the same, illustratively including device ID information of the display device, but not device status information of the display device. Since the server stores the device state information of each display device, after the server receives the connection request, the server can search the device state information corresponding to the device ID information in the stored device state information according to the device ID information. For example, the first connection request carries device ID information of the display device a. After receiving the first connection request, the server finds that the equipment state information of the display equipment A is in a starting state according to the equipment ID information of the display equipment A. The server can generate a corresponding first awakening instruction for the starting-up state according to the equipment state information. In some embodiments, the device state information may include one of a power-on state identifier and a standby state identifier.

In some embodiments, since the mobile terminal may obtain the device state information of each display device from the server, the first connection request and the second connection request in the embodiments of the present application may also be different, which exemplarily include the display device information and/or the device state information. After the server acquires the connection request, the server does not need to search corresponding equipment state information according to the equipment ID information. For example, the second connection request carries the device state information of the display device B as a standby state, and the server may generate a corresponding second wake-up instruction for the standby state according to the device state information.

In some embodiments, when the display device is in a standby state, the second wake-up instruction may include a power-on instruction and a first wake-up instruction (preset application start instruction). The processor controls the display equipment to enter a power-on state from a standby state according to the power-on instruction, and then controls the preset application to be started according to the preset application instruction.

In some embodiments, if the display device selected by the user on the preset application interface of the mobile terminal is in a power-on state, the main chip of the display device is in a power-on state, and the low-power chip is in a power-off state. The server directly sends a first awakening instruction to the main chip, wherein the first awakening instruction is a preset application starting instruction. And the main chip controls the preset application to be started according to the preset application starting instruction.

In some embodiments, when the preset application is an audio/video communication application, the connection request (the first connection request and/or the second connection request) further includes a session request identifier, and the session request identifier is used for establishing a session connection between the display device ID and the mobile terminal ID by the server.

In some embodiments, the state of the display device in the list presented on the mobile terminal may be marked as one of an online state (first state), a standby state (second state), and a power-off state (third state). The first processor of the display device is started in an on-line state, the first processor is not started in a standby state, and the first processor and the second processor are not started in a shutdown state.

In some embodiments, the mobile terminal receives an operation instruction input by a user, determines a state of the display device, sends a connection request not including the state of the display device to the server when the display device is in an on or standby state, and displays a notification message of non-connection when the display device is in an off state.

In some embodiments, after receiving the connection request, the server queries the state of the display device according to the display device ID in the connection request when the connection request does not carry the display device state identifier, and the state of the display device is stored in a relationship between the display device ID and the display state established by the server according to the heartbeat connection of the display device.

In some embodiments, after receiving the connection request, the server determines the state of the display device according to the display device state identifier carried in the connection request.

In some embodiments, the server generates a wake-up instruction according to the ID of the display device and the state of the display device, generates a wake-up instruction including a power-on instruction and a preset application start instruction when the state of the display device is a standby state, and generates a wake-up instruction including a preset application start instruction and not including a power-on instruction when the state of the display device is a power-on state. And transmitting the generated wake-up command to a display device.

In some embodiments, when the state of the display device is the power-off state, the server feeds back a message that the display device cannot respond to the mobile terminal.

In some embodiments, when the preset application is an audio-video application, the server determines whether to create a virtual conversation room according to the conversation request identifier and the state of the display device.

In some embodiments, when the display device is in a power-on or standby state, a virtual room is created for a session between the display device and the mobile terminal according to the display device identifier and the mobile terminal identifier, and a waiting instruction is fed back to the mobile terminal, so that the mobile terminal responds to and waits for a session waiting interface to be displayed.

In some embodiments, when the state of the display device is a power-off state, the server does not create a virtual room according to the display device identifier and the mobile terminal identifier session, and feeds back a prompt message that the display device cannot be accessed to the mobile terminal. In some embodiments, the non-access-free alert message may be different from or the same as the non-response message.

In some embodiments, after the server creates the session virtual room, an instruction to access the session is sent to the display device to cause the display device to join the audio-video session.

In some embodiments, the instruction of accessing the session is issued to the mobile terminal as a part of the wake-up instruction, the mobile terminal receives and caches the instruction of accessing the session, and after the preset audio/video communication application is started, the preset audio/video communication application responds to the instruction of accessing the session to join the session, or prompt information is displayed.

In some embodiments, the instruction of accessing the session is not issued to the mobile terminal together with the wake-up instruction, and after the display device starts the preset audio/video communication application, the instruction of accessing the session is sent to the server, wherein the mark indicates that the application is started and completed, and the instruction of accessing the session is received by the server in response to the mark which indicates that the application is started and completed.

The session procedure and the closing procedure after the session connection is subsequently established can refer to the patents previously applied by the applicant.

In some embodiments, the server may not set a virtual room, only schedule an access port with the mobile terminal and the display device, the display device uploads audio and video data to a designated port, the server correspondingly stores the audio and video data uploaded by the display device and a unique identifier of the display device, and the mobile terminal polls a designated window according to the unique identifier of the display device and acquires uploaded impression data.

In some embodiments, the server sets a virtual session room and sends a wake-up instruction to the display device, where the wake-up instruction further includes a room identifier of the virtual session room. And the display equipment is added into the virtual conversation room according to the room identifier after starting the preset application so as to establish the audio and video conversation.

In some embodiments, if the display device selected by the user on the preset application interface of the mobile terminal is in the standby state, the main chip of the display device is in the non-activated state, and the low-power chip is in the activated state. The server sends a second awakening instruction to the display device, and a communicator of the display device receives and sends the second awakening instruction to the low-power chip, wherein the second awakening instruction comprises a starting instruction and a preset application starting instruction. The low-power chip firstly controls the main chip to enter a starting state from an un-starting state according to the starting instruction and simultaneously caches a preset application starting instruction. And after the main chip enters the starting state from the non-starting state, the state result is fed back to the low-power chip. And after the low-power chip receives that the state result of the main chip is the starting state, the preset application starting instruction in the second awakening instruction is sent to the main chip. And finally, the main chip controls the preset application to be started according to the preset application starting instruction. It should be noted that, after the process of sending the preset application start instruction to the main chip is completed, the low power consumption chip may maintain the start state. In order to reduce resource occupation, the main chip can send a closing instruction to the low-power chip after receiving a preset application opening instruction, so that the low-power chip enters an un-started state from a started state according to the closing instruction.

In some embodiments, the low power chip may send a level signal to a designated interface/pin of the main chip in response to the power-on command to wake up the main chip.

In some embodiments, the low power consumption chip may further cache a session request instruction, and may send the preset application start instruction and the cache session request instruction to the main chip together after the main chip is started, where the main chip starts the preset application by responding to the preset application start instruction, and after the preset application is started, the preset application is made to respond to the session request instruction. In some embodiments, after receiving the message that the preset application is started and completed, which is fed back by the master chip, the session request instruction may be sent to the master chip, so that the master chip controls the preset application to respond to the session request instruction.

In some embodiments, the preset application opening instruction further carries peer invitation information corresponding to the mobile terminal. And after the preset application is opened, receiving the invitation information of the opposite terminal, and analyzing the invitation information of the opposite terminal, thereby realizing the connection establishment between the preset application of the mobile terminal and the preset application of the display equipment.

In some embodiments, after the preset application is started, a response is made to a session request instruction, and a prompt message is displayed on a display of the display device, where the prompt message is used to enable the display device to prompt a user whether to accept a session request, and after an operation instruction that the user does not receive is received, the display device feeds back a rejected identifier to the server, so that the server feeds back a message that the display device is not added to the audio/video session to the mobile terminal, and after the operation instruction that the user receives is received, the display device feeds back an agreed identifier to the server, so as to establish session connection.

In some embodiments, the preset application response of the display device automatically responds with the session request instruction to automatically establish the session connection.

In some embodiments, when the server receives a connection request containing a session request identifier, session authentication is performed according to the unique identifier of the display device and the unique identifier of the mobile terminal, and when the unique identifier of the mobile terminal is authorized by the unique identifier of the display device in the relationship between the server and the storage, a virtual session room is established and subsequent processes are executed. If not, the virtual room is not established, and information is fed back to the mobile terminal to remind that the mobile terminal is not authorized by the display equipment.

In some embodiments, after the display device likes to start the preset application, a secondary authentication may be performed on whether the session can be automatically established for the unique identifier of the display device and the unique identifier of the mobile terminal.

In some embodiments, a router is provided between the display device and the server. The device information includes an IP address of the display device, a router port number used by the display device, and a MAC address of the display device. And the server sends a wakeup data Packet (which can be a Magic Packet unicast Packet) to the corresponding router according to the IP address and the router port number of the display device contained in the connection request. And after receiving the awakening data packet, the router identifies the awakening data packet as a directional broadcast packet. The router broadcasts this broadcast packet to all ports, and the broadcast packet contains the MAC address of the display device. And after the display equipment corresponding to the MAC address receives the broadcast packet, the preset application of the display equipment is awakened. And after the display equipment of other MAC addresses connected with the router receives the broadcast packet, directly abandoning the data packet.

Illustratively, the preset application is a housekeeper APP, the display equipment needing to be connected is selected from the housekeeper APP of the mobile terminal, and after the selected display equipment receives an opening instruction of the housekeeper APP, the housekeeper APP can be directly opened and the connection with the mobile terminal is established through the housekeeper APP. The household environment can be monitored by using a camera of the display equipment.

The user selects the display device B in the user interface of the mobile terminal shown in fig. 7, and in response to a control instruction for the user to instruct the selection of the display device B, the user interface shown in fig. 8 is entered from the user interface shown in fig. 7. The user interface shown in fig. 8 includes a monitoring screen and setting controls of the housekeeping APP. At this time, the mobile terminal and the display device B are not yet connected, and a "go home and see" control is displayed in the monitoring screen shown in fig. 8 for confirming that the connection with the display device B is established.

When the user selects the 'go home to see' control, a connection request is sent to the server in response to a control instruction of the user indicating selection of the 'go home to see' control. The connection request carries device information of the display device B. And the server generates a connection instruction according to the equipment information of the display equipment B. Since the display apparatus B is in the standby state, the server first transmits a connection instruction to the low power chip of the display apparatus B. And the low-power chip analyzes the starting instruction and the preset application starting instruction from the connecting instruction. The low-power chip firstly sends a starting instruction to the main chip and triggers the main chip to enter a starting state from an un-starting state.

After the main chip is started, the starting result is fed back to the low-power-consumption chip, and the low-power-consumption chip continues to send a preset application starting instruction to the main chip. At this time, the preset application starting instruction is an instruction for starting the housekeeping APP. And after receiving the instruction of starting the housekeeping APP, the main chip executes the service logic of starting the housekeeping APP. The preset application opening instruction also comprises opposite end invitation information sent by the mobile terminal. And the main chip sends the service instruction to the housekeeper APP, and the housekeeper APP analyzes the invitation information of the opposite terminal from the service instruction. The opposite-end invitation information may include device information of the mobile terminal. And according to the invitation information of the opposite terminal, establishing communication connection between the mobile terminal and the display equipment B through the housekeeper APP.

After the mobile terminal is in communication connection with the display device B, the display device B sends the picture shot by the camera in real time to the mobile terminal. As shown in fig. 9, the mobile terminal may view a picture taken by the camera of the display device B in real time, so as to implement a remote monitoring function.

In some embodiments, the server includes a service system and an AIOT (artificial intelligence internet of things) system, and the low power consumption chip and the main chip report the status information of the display device to the AIOT system. And the AIOT system sends the state information to the service system, and the service system packages the state information and generates an online list of the display equipment. The mobile terminal directly obtains the relevant information of the online list of the display equipment from the service system. In some embodiments, when the mobile terminal sends the same connection request when the display device is in a standby or on state, the server generates and sends a preset application starting instruction according to the unique identifier of the display device in the received connection request, wherein the preset application starting instruction is used for controlling the main processor to start a preset application, the communicator of the display device receives the preset application starting instruction and sends the preset application starting instruction to the main processor when the main processor is started, the main processor starts the preset application in response to the preset application starting instruction, the communicator sends the preset application starting instruction to the main processor when the main processor is not started, the sub processor generates a starting instruction according to the preset application starting instruction and caches the preset application starting instruction, and the sub processor sends the starting instruction and the preset application starting instruction to the main processor simultaneously or sequentially so as to start the main processor and enable the started main processor to start the preset application.

In some embodiments, the instruction for starting the preset application may be a call request including information such as a unique identifier of the mobile terminal, virtual room information, and a session request instruction.

In some embodiments, the host processor booting refers to the host processor performing loading of an operating system (e.g., any of an android operating system, an apple operating system, a hong kong operating system, etc.). The working state of the main processor refers to the state of the operating system running in the main chip. Starting a default application refers to starting an application in the operating system of the host processor.

Based on the foregoing embodiments, an embodiment of the present application provides an application wake-up method for a display device, as shown in a signaling diagram of the application wake-up method for the display device shown in fig. 10, where the method includes the following steps:

step one, a user opens a preset application user interface on the mobile terminal and selects display equipment in the preset application user interface. The mobile terminal receives a control instruction for selecting the display device, which is input by a user.

And step two, the mobile terminal responds to the control instruction, and generates a first connection request when the display equipment is in a power-on state, wherein the equipment state of the display equipment pointed by the first connection request is the power-on state. The mobile terminal sends the first connection request to the server, and the server generates a first awakening instruction according to the first connection request and sends the first awakening instruction to the corresponding display equipment. And the processor of the display equipment controls the preset application to be started according to the first awakening instruction.

And fourthly, generating a second connection request when the display equipment is in the standby state, wherein the equipment state of the display equipment pointed by the second connection request is in the standby state. And the mobile terminal sends the second connection request to the server, and the server generates a second awakening instruction according to the second connection request and sends the second awakening instruction to the corresponding display equipment. And after the processor of the display equipment controls the display equipment to enter the starting state from the standby state according to the second awakening instruction, the processor of the display equipment controls the preset application to be started according to the second awakening instruction.

In the embodiment of the method, the online list of the display device is displayed on the preset application user interface of the mobile terminal. The display device online list is a list generated by the server according to the state information reported by at least one display device. After a user opens a preset application user interface on the mobile terminal, the mobile terminal sends a page data request related to a preset application to the server, and the server feeds back an online list of display equipment to the mobile terminal according to the page data request.

An embodiment of the present application provides another application wake-up method for a display device, as shown in a signaling diagram of the application wake-up method for a display device in fig. 11, where the method includes the following steps:

step one, when the display device is in a starting state, the main chip is in a starting state, the low-power chip is in a starting state, and the main chip continuously reports the device state information (uploads heartbeat data) to the server. When the display device enters the standby state from the power-on state, the main chip enters the non-starting state from the starting state. Before the main chip enters the non-starting state from the starting state, the low-power consumption chip is triggered to enter the starting state from the non-starting state. The low-power chip continuously reports the equipment state information to the server. And the server generates an online list of the display equipment according to the equipment state information reported by the at least one display equipment.

And step two, the user opens a preset application on the mobile terminal, such as a housekeeping APP. The mobile terminal transmits a page data request to the server in response to an operation of the user. And the server feeds back the online list of the display equipment to the mobile terminal according to the page data request. After the mobile terminal receives the display device online list, the display device online list is presented on the user interface (step one and step two are not shown in fig. 11).

And step three, the mobile terminal receives a control instruction for selecting the display equipment, which is input by a user. And the mobile terminal responds to the control instruction, and if the selected display equipment is in the power-on state, a first connection request is generated, wherein the equipment state of the display equipment pointed by the first connection request is in the power-on state. The mobile terminal sends the first connection request to the server, and the server generates a first awakening instruction according to the first connection request and sends the first awakening instruction to the corresponding display equipment. And the server sends the first awakening instruction to a main chip of the selected display equipment to be awakened. The main chip controls the preset application to be started according to the first awakening instruction (namely the preset application starting instruction).

And step four, if the selected display equipment is in a standby state, the server generates a second awakening instruction according to the connection request, wherein the second awakening instruction comprises a starting instruction and a first awakening instruction. And the server sends the starting instruction and the first awakening instruction to a low-power chip of the selected display device to be awakened.

The low-power chip triggers the main chip to start according to the starting instruction. And after the main chip is started, feeding back a starting result to the low-power-consumption chip. And after receiving the starting result, the low-power chip continues to send a first awakening instruction to the main chip. The first wake-up instruction is associated with a housekeeping APP. The main chip executes logic for opening the housekeeping APP according to the first awakening instruction, and meanwhile, opposite end invitation information carried by the first awakening instruction is sent to the housekeeping APP. And after the housekeeping APP is opened, establishing communication connection with the mobile terminal according to the invitation information of the opposite terminal.

After the mobile terminal is in communication connection with the display device, the display device sends the pictures shot by the camera in real time to the mobile terminal. As shown in fig. 9, the mobile terminal can be used to view the real-time picture taken by the camera of the display device B, thereby implementing the remote monitoring function.

In some embodiments, the mobile terminal may be replaced by a display device, which may also be replaced by a mobile terminal.

The same or similar contents in the embodiments of the present application may be referred to each other, and the related embodiments are not described in detail.

The foregoing description, for purposes of explanation, has been presented in conjunction with specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit the embodiments to the precise forms disclosed above. Many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles and the practical application, to thereby enable others skilled in the art to best utilize the embodiments and various embodiments with various modifications as are suited to the particular use contemplated.

Claims

1. A mobile terminal, comprising:

a communication interface for communication between the processor and a server;

a processor to:

receiving an operation on a first display device control in the online list;

2. The mobile terminal of claim 1, wherein sending a connection request when the first display device control is marked as on-state or standby-state comprises:

when the first display equipment control is marked to be in a starting state, sending a first connection request to a server, wherein the connection request comprises a first display equipment identifier and a starting state identifier;

and when the first display device control is marked to be in a standby state, sending a second connection request to the server, wherein the connection request comprises a standby state identifier besides the first display device identifier.

3. A server, comprising a memory and a processor, the processor to:

4. The server according to claim 3, wherein the sending the wake-up instruction to the first display device comprises:

when the first display device is determined to be in a power-on state, sending a first wake-up instruction to the first display device, wherein the first wake-up instruction comprises an instruction for starting a preset application and a room identifier corresponding to the virtual conversation room, the instruction for starting the preset application is used for enabling a main processor in the first display device to start the preset application, and the preset application is used for establishing conversation connection according to the room identifier after being started;

when the first display device is determined to be in a standby state, sending a second wake-up instruction to the first display device, wherein the second wake-up instruction comprises a start-up instruction, an instruction for starting a preset application and a room identifier corresponding to the virtual conversation room, the start-up instruction is used for enabling a sub-processor in the first display device to control a main processor in the first display device to start, the instruction for starting the preset application is used for enabling the main processor in the first display device to start the preset application, and the preset application is used for establishing conversation connection according to the room identifier after the start-up.

5. The server according to claim 4,

and the processor determines the state of the display equipment according to a preset relation and the first display equipment identifier in the connection request, wherein the preset relation is a mapping relation between the display equipment identifier and the state of the display equipment determined according to the heartbeat connection between the display equipment and the server.

6. The server according to claim 4, wherein the connection request further comprises a power-on status identifier or a standby status identifier in addition to the first display device identifier;

when the connection request contains a starting-up state identifier, the processor determines that the display equipment is in a starting-up state;

and the processor determines that the display equipment is in a standby state when the connection request contains a standby state identifier.

7. A display device, comprising:

a display;

a communicator to: receiving a wake-up instruction, sending the wake-up instruction to a main processor when the main processor of the display device is in a starting state, and sending the wake-up instruction to a sub-processor when the main processor of the display device is in an un-starting state and the sub-processor is in a starting state, wherein the wake-up instruction comprises an instruction for starting a preset application;

8. The display device of claim 7, wherein the controlling the main processor to activate comprises:

responding to the received awakening instruction, and generating a starting instruction;

and controlling the main processor to start according to the starting instruction.

9. The display device according to claim 7, wherein the wake-up instruction comprises a first wake-up instruction and a second wake-up instruction, wherein the first wake-up instruction comprises the instruction to start the preset application and does not comprise a power-on instruction, and the second wake-up instruction comprises the instruction to start the preset application and the power-on instruction;

sending the wake-up instruction to the main processor comprises: sending the first wake-up instruction to the main processor;

sending the wake-up instruction to the sub-processor comprises: sending the second wake-up instruction to the sub-processor;

said controlling said main processor to boot comprises: and controlling the main processor to start according to the starting instruction in the second awakening instruction.

10. The display device according to claim 7, wherein the display device is a liquid crystal display device

The wake-up instruction further comprises: a room identification corresponding to the virtual conversation room;

the main processor is further configured to: and responding to the room identifier of the virtual conversation room received from the communicator or the room identifier of the virtual conversation room received from the sub-processor, and controlling the preset application to establish conversation connection according to the room identifier of the virtual conversation room after the preset application is started.

11. An application wakeup method, comprising: the method comprises the following steps: