CN111726664A

CN111726664A - Method for controlling function operation of display device through mobile terminal and display device

Info

Publication number: CN111726664A
Application number: CN202010530676.7A
Authority: CN
Inventors: 崔会会; 田鹏升; 高峰凯
Original assignee: Hisense Visual Technology Co Ltd
Current assignee: Hisense Visual Technology Co Ltd
Priority date: 2020-06-11
Filing date: 2020-06-11
Publication date: 2020-09-29

Abstract

The application discloses a method for controlling function operation of display equipment through a mobile terminal, which is used for realizing control of function operation of the display equipment through clicking or touching and sliding function operation items displayed by the mobile terminal by a user, simplifying user operation steps and further improving user experience. The method comprises the following steps: responding to an input instruction sent by a user through a mobile terminal, and executing an operation corresponding to the input instruction; the mobile terminal displays a user interface, the user interface comprises at least one function operation item, and the input instruction is input by clicking or touching the function operation item by a user.

Description

Method for controlling function operation of display device through mobile terminal and display device

Technical Field

The present application relates to the field of display technologies, and in particular, to a method for controlling functional operations of a display device through a mobile terminal and a display device.

Background

With more and more countries and regions exported by the intelligent television, the number of users of the intelligent television is continuously increased, and the operation requirements of the users on television adjustment are stronger and stronger. The basic functions of the smart television include a series of functions such as television channel playing, EPG (electronic program guide), program recording reservation (PVR recording reservation), program playing reservation, recorded program playing (PVR playing), time-shift playing, etc. However, these functions are operated on the television through a remote controller, the focus is moved one by one, the operation steps are troublesome, and the user experience is poor. For example: when the user has a relatively large adjustment range, for example, from currently being in channel 1, to adjust to channel 20, the user needs to press the down key of the remote controller to move the focus one time, and the adjustment is slow. The appointment adding interface shown in fig. 2 has many parameters to be selected, such as the selection of time. After the user selects Start Time, as shown in the reserved Time selection interface shown in fig. 3, the user needs to press the direction key of the remote controller and move the focus to select the input number, which is also quite complicated in operation and poor in user experience.

Disclosure of Invention

The embodiment of the application provides a method for controlling function operation of a display device through a mobile terminal and the display device, which are used for realizing control of the function operation of the display device through clicking or touching and sliding a function operation item displayed by the mobile terminal by a user, simplifying the operation steps of the user and further improving the user experience.

In a first aspect, there is provided a display device comprising:

a display;

a communicator for communicating with a mobile terminal;

a controller for performing:

responding to an input instruction sent by a user through a mobile terminal, and executing an operation corresponding to the input instruction;

the mobile terminal displays a user interface, the user interface comprises at least one function operation item, and the input instruction is input by clicking or touching the function operation item by a user.

In some embodiments, the functional operations include channel switching, program recording scheduling, electronic program guide acquisition, recorded program playing, program playing scheduling, time-shift starting, recorded program playing progress bar synchronization, and time-shift playing progress bar synchronization.

In some embodiments, the controller is further configured to perform:

and sending the operation completion result to the mobile terminal.

In some embodiments, the communicator and the mobile terminal are connected through an MQTT protocol to communicate in a manner of sending a theme.

In a second aspect, there is provided a display device comprising:

a display;

a communicator for communicating with a mobile terminal;

a controller for performing:

responding to an input instruction sent by a user through a mobile terminal, and switching to a channel corresponding to the item;

the mobile terminal displays a user interface, the user interface comprises a channel list, the channel list comprises at least one item, the item comprises channel information, and the input instruction is input by clicking the item by a user.

In some embodiments, the controller is further configured to perform:

and sending the channel switching result to the mobile terminal.

In some embodiments, the controller is further configured to perform:

and monitoring a database where the channel list is located, and sending the channel list information to the mobile terminal so that the mobile terminal displays the channel list.

In a third aspect, there is provided a display device comprising:

a display;

a communicator for communicating with a mobile terminal;

a controller for performing:

responding to an input instruction sent by a user through the mobile terminal, and executing an addition reservation operation;

the mobile terminal displays a user interface, the user interface comprises a reserved time modification item and a reserved addition item, and the input instruction is input by a user through touching and sliding the reserved time modification item and clicking the reserved addition item.

In some embodiments, the controller is further configured to perform:

and sending the reservation result to the mobile terminal.

In some embodiments, the controller is further configured to perform:

updating the reservation list;

and sending the updated reservation list to the mobile terminal.

In a fourth aspect, a method for controlling functional operations of a display device through a mobile terminal is provided, which includes:

In a fifth aspect, a method for controlling channel switching of a display device through a mobile terminal is provided, including:

In a sixth aspect, a method for controlling a display device to add a reservation through a mobile terminal is provided, which includes:

A seventh aspect provides a mobile terminal, where the mobile terminal displays a user interface, where the user interface includes at least one function operation item, and sends an input instruction input by a user through clicking or sliding the function operation item by touch to a display device, so that the display device responds to the input instruction and executes an operation corresponding to the input instruction.

In an eighth aspect, there is provided a system for controlling functional operations of a display device through a mobile terminal, comprising:

the mobile terminal is used for displaying a user interface, the user interface comprises at least one functional operation item, and an input instruction input by clicking or touching and sliding the functional operation item by a user is sent to the display equipment;

and the display equipment is connected with the mobile terminal and used for responding to an input instruction and executing the operation corresponding to the input instruction.

In the above embodiment, the user performs the functional operation operations such as channel switching, electronic program guide acquisition, program recording reservation, program playing reservation and the like by clicking or touching and sliding on the mobile terminal. For example, in channel switching operation, a user only needs to click a channel to be played on a channel list at a mobile phone end, and can switch to the channel to be played in one step, unlike a remote controller which needs to move a focus one by one. The appointment time can be changed only by sliding the calendar with fingers to select the time when the program is reserved for recording and playing. The user operation steps are simplified, great convenience is brought to the user, and the user experience is improved.

Drawings

Fig. 1A is a schematic diagram illustrating an operation scenario between the display device 200 and the control apparatus 100;

fig. 1B is a block diagram schematically illustrating a configuration of the control apparatus 100 in fig. 1A;

fig. 1C is a block diagram schematically illustrating a configuration of the display device 200 in fig. 1A;

a block diagram of the architectural configuration of the operating system in the memory of the display device 200 is illustrated in fig. 1D.

Fig. 2 is a schematic diagram illustrating a reservation addition interface provided by the display apparatus 200;

fig. 3 is a schematic view schematically showing a reserved time selection interface provided by the display device 200;

fig. 4 is a diagram illustrating a first GUI provided by the mobile terminal 100B;

fig. 5 is a diagram illustrating a second GUI provided by the mobile terminal 100B;

fig. 6 is a diagram illustrating a third GUI provided by the mobile terminal 100B;

fig. 7 is a diagram illustrating a fourth GUI provided by the mobile terminal 100B;

FIGS. 8A-8B are diagrams that illustrate a fifth GUI provided by the mobile terminal 100B;

fig. 9 is a diagram illustrating a sixth GUI provided by the mobile terminal 100B;

fig. 10 is a diagram illustrating a seventh GUI provided by the mobile terminal 100B;

fig. 11 is a schematic view exemplarily showing an eighth GUI provided by the mobile terminal 100B;

fig. 12 is a diagram illustrating a ninth GUI provided by the mobile terminal 100B;

fig. 13 is a diagram illustrating a tenth GUI provided by the mobile terminal 100B;

FIG. 14 is a flow chart illustrating a method for controlling functional operation of a display device by a terminal;

fig. 15 is a flowchart illustrating a method of controlling channel switching and adding reservation of a display device by a terminal;

fig. 16 is a flowchart illustrating authentication between a display device and a terminal.

Detailed Description

To make the objects, technical solutions and advantages of the exemplary embodiments of the present application clearer, the technical solutions in the exemplary embodiments of the present application will be clearly and completely described below with reference to the drawings in the exemplary embodiments of the present application, and it is obvious that the described exemplary embodiments are only a part of the embodiments of the present application, but not all the embodiments.

All other embodiments, which can be derived by a person skilled in the art from the exemplary embodiments shown in the present application without inventive effort, shall fall within the scope of protection of the present application. Moreover, while the disclosure herein has been presented in terms of exemplary one or more examples, it is to be understood that each aspect of the disclosure can be utilized independently and separately from other aspects of the disclosure to provide a complete disclosure.

The terms "comprises" and "comprising," and any variations thereof, as used herein, are intended to cover a non-exclusive inclusion, such that a product or device that comprises a list of elements is not necessarily limited to those elements explicitly listed, but may include other elements not expressly listed or inherent to such product or device.

The term "module," as used herein, refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and/or software code that is capable of performing the functionality associated with that element.

The term "gesture" as used in this application refers to a user's behavior through a change in hand shape or an action such as hand motion to convey a desired idea, action, purpose, or result.

Fig. 1A is a schematic diagram illustrating an operation scenario between the display device 200 and the control apparatus 100. As shown in fig. 1A, the control apparatus 100 and the display device 200 may communicate with each other in a wireless manner.

Among them, the control apparatus 100 is configured to control the display device 200, which may receive an operation instruction input by a user and convert the operation instruction into an instruction recognizable and responsive by the display device 200, serving as an intermediary for interaction between the user and the display device 200. Such as: the user operates the channel up/down key on the control device 100, and the display device 200 responds to the channel up/down operation.

The control device 100 may be a remote controller 100A, which includes infrared protocol communication or bluetooth protocol communication, and other short-distance communication methods, etc. to control the display apparatus 200 in a wireless or other wired manner. For example, the user may input a user instruction through a key on a remote controller, a voice input, a control panel input, or the like to control the display apparatus 200. Such as: the user can input a corresponding control command through a volume up/down key, a channel control key, up/down/left/right movement key direction keys, a voice input key, a menu key, a power on/off key, etc. on the control device, thereby implementing a function of controlling the display apparatus 200.

The control device 100 may also be an intelligent device, such as a mobile terminal 100B, a tablet computer, a notebook computer, and the like. For example, the display device 200 is controlled using an application (e.g., a remote connection application described below) running on the mobile terminal 100B. The application program may provide various controls to a user through an intuitive User Interface (UI) on a screen associated with the mobile terminal 100B through configuration.

For example, the mobile terminal 100B may install a software application with the display device 200 to implement connection communication via a network communication protocol (e.g., MQTT protocol described below), so as to implement the purpose of one-to-one control operation and data communication. Such as: the mobile terminal 100B may be caused to establish a control instruction protocol with the display device 200 to implement the function of physical keys as arranged in a remote controller by operating various function keys or virtual buttons of a user interface provided on the mobile terminal. And the audio and video content displayed on the mobile terminal can be transmitted to the display equipment, so that the synchronous display function is realized.

The display apparatus 200 may be implemented as a television, and may provide an intelligent network television function of a broadcast receiving television function as well as a computer support function. Examples of the display device include a digital television, a web television, a smart television, an Internet Protocol Television (IPTV), and the like.

The display device 200 may be a liquid crystal display, an organic light emitting display, a projection display device. The specific display device type, size, resolution, etc. are not limited.

The display apparatus 200 also performs data communication with the server 300 through various communication means. Here, the display apparatus 200 may be allowed to be communicatively connected through a Local Area Network (LAN), a Wireless Local Area Network (WLAN), and other networks. The server 300 may provide various contents and interactions to the display apparatus 200. By way of example, the display device 200 may send and receive information such as: receiving Electronic Program Guide (EPG) data, receiving software program updates, or accessing a remotely stored digital media library. The servers 300 may be a group or groups of servers, and may be one or more types of servers. Other web service contents such as a video on demand and an advertisement service are provided through the server 300.

Fig. 1B is a block diagram illustrating the configuration of the control device 100. As shown in fig. 1B, the control device 100 includes a controller 110, a memory 120, a communicator 130, a user input interface 140, an output interface 150, and a power supply 160.

The controller 110 includes a Random Access Memory (RAM)111, a Read Only Memory (ROM)112, a processor 113, a communication interface, and a communication bus. The controller 110 is used to control the operation of the control device 100, as well as the internal components of the communication cooperation, external and internal data processing functions.

Illustratively, when an interaction of a user operating a function key or a virtual button of a user interface provided on the mobile terminal is detected, the controller 110 may control to generate a signal corresponding to the detected interaction and transmit the signal to the display device 200.

And a memory 120 for storing various operation programs, data and applications for driving and controlling the control apparatus 100 under the control of the controller 110. The memory 120 may store various control signal commands input by a user.

The communicator 130 enables communication of control signals and data signals with the display apparatus 200 under the control of the controller 110. Such as: the control apparatus 100 transmits a control signal (e.g., a touch signal or a button signal) to the display device 200 via the communicator 130, and the control apparatus 100 may receive the signal transmitted by the display device 200 via the communicator 130. The communicator 130 may include an infrared signal interface 131 and a radio frequency signal interface 132. For example: when the infrared signal interface is used, the user input instruction needs to be converted into an infrared control signal according to an infrared control protocol, and the infrared control signal is sent to the display device 200 through the infrared sending module. The following steps are repeated: when the rf signal interface is used, a user input command needs to be converted into a digital signal, and then the digital signal is modulated according to the rf control signal modulation protocol and then transmitted to the display device 200 through the rf transmitting terminal.

The user input interface 140 may include at least one of a microphone 141, a touch pad 142, a sensor 143, a key 144, and the like, so that a user can input a user instruction regarding controlling the display apparatus 200 to the control apparatus 100 through voice, touch, gesture, press, and the like.

The output interface 150 outputs a user instruction received by the user input interface 140 to the display apparatus 200, or outputs an image or voice signal received by the display apparatus 200. Here, the output interface 150 may include an LED interface 151, a vibration interface 152 generating vibration, a sound output interface 153 outputting sound, a display 154 outputting an image, and the like. For example, the remote controller 100A may receive an output signal such as audio, video, or data from the output interface 150, and display the output signal in the form of an image on the display 154, in the form of audio on the sound output interface 153, or in the form of vibration on the vibration interface 152.

And a power supply 160 for providing operation power support for each element of the control device 100 under the control of the controller 110. In the form of a battery and associated control circuitry.

A hardware configuration block diagram of the display device 200 is exemplarily illustrated in fig. 1C. As shown in fig. 1C, the display apparatus 200 may further include a tuner demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a memory 260, a user interface 265, a video processor 270, a display 275, an audio processor 280, an audio output interface 285, and a power supply 290.

The tuner demodulator 210 receives the broadcast television signal in a wired or wireless manner, may perform modulation and demodulation processing such as amplification, mixing, and resonance, and is configured to demodulate, from a plurality of wireless or wired broadcast television signals, an audio/video signal carried in a frequency of a television channel selected by a user, and additional information (e.g., EPG data).

The tuner demodulator 210 is responsive to the user selected frequency of the television channel and the television signal carried by the frequency, as selected by the user and controlled by the controller 250.

The tuner demodulator 210 can receive a television signal in various ways according to the broadcasting system of the television signal, such as: terrestrial broadcasting, cable broadcasting, satellite broadcasting, internet broadcasting, or the like; and according to different modulation types, a digital modulation mode or an analog modulation mode can be adopted; and can demodulate the analog signal and the digital signal according to the different kinds of the received television signals.

In other exemplary embodiments, the tuning demodulator 210 may also be in an external device, such as an external set-top box. In this way, the set-top box outputs a television signal after modulation and demodulation, and inputs the television signal into the display apparatus 200 through the external device interface 240.

The communicator 220 is a component for communicating with an external device or an external server according to various communication protocol types. For example, the display apparatus 200 may transmit content data to an external apparatus connected via the communicator 220, or browse and download content data from an external apparatus connected via the communicator 220. The communicator 220 may include a network communication protocol module or a near field communication protocol module, such as a WIFI module 221, a bluetooth communication protocol module 222, and a wired ethernet communication protocol module 223, so that the communicator 220 may receive a control signal of the control device 100 according to the control of the controller 250 and implement the control signal as a WIFI signal, a bluetooth signal, a radio frequency signal, and the like.

The detector 230 is a component of the display apparatus 200 for collecting signals of an external environment or interaction with the outside. The detector 230 may include an image collector 231, such as a camera, a video camera, etc., which may be used to collect external environment scenes to adaptively change the display parameters of the display device 200; and the function of acquiring the attribute of the user or interacting gestures with the user so as to realize the interaction between the display equipment and the user. A light receiver 232 may also be included to collect ambient light intensity to adapt to changes in display parameters of the display device 200, etc.

In some other exemplary embodiments, the detector 230 may further include a temperature sensor, such as by sensing an ambient temperature, and the display device 200 may adaptively adjust a display color temperature of the image. For example, when the temperature is higher, the display apparatus 200 may be adjusted to display a color temperature of an image that is cooler; when the temperature is lower, the display device 200 may be adjusted to display a warmer color temperature of the image.

In some other exemplary embodiments, the detector 230, which may further include a sound collector, such as a microphone, may be configured to receive a sound of a user, such as a voice signal of a control instruction of the user to control the display device 200; alternatively, ambient sounds may be collected that identify the type of ambient scene, enabling the display device 200 to adapt to ambient noise.

The external device interface 240 is a component for providing the controller 250 to control data transmission between the display apparatus 200 and an external apparatus. The external device interface 240 may be connected to an external apparatus such as a set-top box, a game device, a notebook computer, etc. in a wired/wireless manner, and may receive data such as a video signal (e.g., moving image), an audio signal (e.g., music), additional information (e.g., EPG), etc. of the external apparatus.

The external device interface 240 may include: a High Definition Multimedia Interface (HDMI) terminal 241, a Composite Video Blanking Sync (CVBS) terminal 242, an analog or digital Component terminal 243, a Universal Serial Bus (USB) terminal 244, a Component terminal (not shown), a red, green, blue (RGB) terminal (not shown), and the like.

The controller 250 controls the operation of the display device 200 and responds to the operation of the user by running various software control programs (such as an operating system and various application programs) stored on the memory 260.

As shown in fig. 1C, the controller 250 includes a Random Access Memory (RAM)251, a Read Only Memory (ROM)252, a graphics processor 253, a CPU processor 254, a communication interface 255, and a communication bus 256. The RAM251, the ROM252, the graphic processor 253, and the CPU processor 254 are connected to each other through a communication bus 256 through a communication interface 255.

The ROM252 stores various system boot instructions. When the display apparatus 200 starts power-on upon receiving the power-on signal, the CPU processor 254 executes a system boot instruction in the ROM252, copies the operating system stored in the memory 260 to the RAM251, and starts running the boot operating system. After the start of the operating system is completed, the CPU processor 254 copies the various application programs in the memory 260 to the RAM251 and then starts running and starting the various application programs.

A graphic processor 253 for generating screen images of various graphic objects such as icons, images, and operation menus. The graphic processor 253 may include an operator for performing an operation by receiving various interactive instructions input by a user, and further displaying various objects according to display attributes; and a renderer for generating various objects based on the operator and displaying the rendered result on the display 275.

A CPU processor 254 for executing operating system and application program instructions stored in memory 260. And according to the received user input instruction, processing of various application programs, data and contents is executed so as to finally display and play various audio-video contents.

In some example embodiments, the CPU processor 254 may comprise a plurality of processors. The plurality of processors may include one main processor and a plurality of or one sub-processor. A main processor for performing some initialization operations of the display apparatus 200 in the display apparatus preload mode and/or operations of displaying a screen in the normal mode. A plurality of or one sub-processor for performing an operation in a state of a standby mode or the like of the display apparatus.

The communication interface 255 may include a first interface to an nth interface. These interfaces may be network interfaces that are connected to external devices via a network.

The controller 250 may control the overall operation of the display apparatus 200. For example: in response to receiving a user input command for selecting a GUI object displayed on the display 275, the controller 250 may perform an operation related to the object selected by the user input command.

Where the object may be any one of the selectable objects, such as a hyperlink or an icon. The operation related to the selected object is, for example, an operation of displaying a link to a hyperlink page, document, image, or the like, or an operation of executing a program corresponding to an icon. The user input command for selecting the GUI object may be a command input through various input means (e.g., a mouse, a keyboard, a touch panel, etc.) connected to the display apparatus 200 or a voice command corresponding to a user uttering voice.

A memory 260 for storing various types of data, software programs, or applications for driving and controlling the operation of the display device 200. The memory 260 may include volatile and/or nonvolatile memory. And the term "memory" includes the memory 260, the RAM251 and the ROM252 of the controller 250, or a memory card in the display device 200.

In some embodiments, the memory 260 is specifically used for storing an operating program for driving the controller 250 of the display device 200; storing various application programs built in the display apparatus 200 and downloaded by a user from an external apparatus; data such as visual effect images for configuring various GUIs provided by the display 275, various objects related to the GUIs, and selectors for selecting GUI objects are stored.

In some embodiments, the memory 260 is specifically configured to store drivers and related data for the tuner demodulator 210, the communicator 220, the detector 230, the external device interface 240, the video processor 270, the display 275, the audio processor 280, and the like, external data (e.g., audio-visual data) received from the external device interface, or user data (e.g., key information, voice information, touch information, and the like) received from the user interface.

In some embodiments, memory 260 specifically stores software and/or programs representing an Operating System (OS), which may include, for example: a kernel, middleware, an Application Programming Interface (API), and/or an application program. Illustratively, the kernel may control or manage system resources, as well as functions implemented by other programs (e.g., the middleware, APIs, or applications); at the same time, the kernel may provide an interface to allow middleware, APIs, or applications to access the controller to enable control or management of system resources.

A block diagram of the architectural configuration of the operating system in the memory of the display device 200 is illustrated in fig. 1D. The operating system architecture comprises an application layer, a middleware layer and a kernel layer from top to bottom.

The application layer, the application programs built in the system and the non-system-level application programs belong to the application layer and are responsible for direct interaction with users. The application layer may include a plurality of applications such as NETFLIX applications, setup applications, media center applications, and the like. These applications may be implemented as Web applications that execute based on a WebKit engine, and in particular may be developed and executed based on HTML, Cascading Style Sheets (CSS), and JavaScript.

Here, HTML, which is called HyperText Markup Language (HyperText Markup Language), is a standard Markup Language for creating web pages, and describes the web pages by Markup tags, where the HTML tags are used to describe characters, graphics, animation, sound, tables, links, etc., and a browser reads an HTML document, interprets the content of the tags in the document, and displays the content in the form of web pages.

CSS, known as Cascading Style Sheets (Cascading Style Sheets), is a computer language used to represent the Style of HTML documents, and may be used to define Style structures, such as fonts, colors, locations, etc. The CSS style can be directly stored in the HTML webpage or a separate style file, so that the style in the webpage can be controlled.

JavaScript, a language applied to Web page programming, can be inserted into an HTML page and interpreted and executed by a browser. The interaction logic of the Web application is realized by JavaScript. The JavaScript can package a JavaScript extension interface through a browser, realize the communication with the kernel layer,

the middleware layer may provide some standardized interfaces to support the operation of various environments and systems. For example, the middleware layer may be implemented as multimedia and hypermedia information coding experts group (MHEG) middleware related to data broadcasting, DLNA middleware which is middleware related to communication with an external device, middleware which provides a browser environment in which each application program in the display device operates, and the like.

The kernel layer provides core system services, such as: file management, memory management, process management, network management, system security authority management and the like. The kernel layer may be implemented as a kernel based on various operating systems, for example, a kernel based on the Linux operating system.

The kernel layer also provides communication between system software and hardware, and provides device driver services for various hardware, such as: provide display driver for the display, provide camera driver for the camera, provide button driver for the remote controller, provide wiFi driver for the WIFI module, provide audio driver for audio output interface, provide power management drive for Power Management (PM) module etc..

A user interface 265 receives various user interactions. Specifically, it is used to transmit an input signal of a user to the controller 250 or transmit an output signal from the controller 250 to the user. For example, the remote controller 100A may transmit an input signal, such as a power switch signal, a channel selection signal, a volume adjustment signal, etc., input by the user to the user interface 265, and then the input signal is transferred to the controller 250 through the user interface 265; alternatively, the remote controller 100A may receive an output signal such as audio, video, or data output from the user interface 265 via the controller 250, and display the received output signal or output the received output signal in audio or vibration form.

In some embodiments, a user may enter user commands on a Graphical User Interface (GUI) displayed on the display 275, and the user interface 265 receives the user input commands through the GUI. Specifically, the user interface 265 may receive user input commands for controlling the position of a selector in the GUI to select different objects or items.

Alternatively, the user may input a user command by inputting a specific sound or gesture, and the user interface 265 receives the user input command by recognizing the sound or gesture through the sensor.

The video processor 270 is configured to receive an external video signal, and perform video data processing such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, and image synthesis according to a standard codec protocol of the input signal, so as to obtain a video signal that is directly displayed or played on the display 275.

Illustratively, the video processor 270 includes a demultiplexing module, a video decoding module, an image synthesizing module, a frame rate conversion module, a display formatting module, and the like.

The demultiplexing module is configured to demultiplex an input audio/video data stream, where, for example, an input MPEG-2 stream (based on a compression standard of a digital storage media moving image and voice), the demultiplexing module demultiplexes the input audio/video data stream into a video signal and an audio signal.

And the video decoding module is used for processing the video signal after demultiplexing, including decoding, scaling and the like.

And the image synthesis module is used for carrying out superposition mixing processing on the GUI signal input by the user or generated by the user and the video image after the zooming processing by the graphic generator so as to generate an image signal for display.

The frame rate conversion module is configured to convert a frame rate of an input video, for example, convert a frame rate of an input 60Hz video into a frame rate of 120Hz or 240Hz, where a common format is implemented by using, for example, an interpolation frame method.

And a display formatting module for converting the signal output by the frame rate conversion module into a signal conforming to a display format of a display, such as converting the format of the signal output by the frame rate conversion module to output an RGB data signal.

And a display 275 for receiving the image signal from the output of the video processor 270 and displaying video, images and menu manipulation interfaces. For example, the display may display video from a broadcast signal received by the tuner demodulator 210, may display video input from the communicator 220 or the external device interface 240, and may display an image stored in the memory 260. The display 275, while displaying a user manipulation interface UI generated in the display apparatus 200 and used to control the display apparatus 200.

And, the display 275 may include a display screen assembly for presenting a picture and a driving assembly for driving the display of an image. Alternatively, a projection device and projection screen may be included, provided display 275 is a projection display.

The audio processor 280 is configured to receive an external audio signal, decompress and decode the received audio signal according to a standard codec protocol of the input signal, and perform audio data processing such as noise reduction, digital-to-analog conversion, and amplification processing to obtain an audio signal that can be played by the speaker 286.

Illustratively, audio processor 280 may support various audio formats. Such as MPEG-2, MPEG-4, Advanced Audio Coding (AAC), high efficiency AAC (HE-AAC), and the like.

Audio output interface 285 receives audio signals from the output of audio processor 280. For example, the audio output interface may output audio in a broadcast signal received via the tuner demodulator 210, may output audio input via the communicator 220 or the external device interface 240, and may output audio stored in the memory 260. The audio output interface 285 may include a speaker 286, or an external audio output terminal 287, such as an earphone output terminal, that outputs to a generating device of an external device.

In other exemplary embodiments, video processor 270 may comprise one or more chips. Audio processor 280 may also comprise one or more chips.

And, in other exemplary embodiments, the video processor 270 and the audio processor 280 may be separate chips or may be integrated with the controller 250 in one or more chips.

And a power supply 290 for supplying power supply support to the display apparatus 200 from the power input from the external power source under the control of the controller 250. The power supply 290 may be a built-in power supply circuit installed inside the display apparatus 200 or may be a power supply installed outside the display apparatus 200.

A schematic diagram of a first GUI provided by the mobile terminal 100B is exemplarily shown in fig. 4.

As shown in fig. 4, mobile terminal 100B may provide a GUI400 to its display, the GUI400 including multiple presentation areas providing different image content, each presentation area including one or more different items arranged therein. For example, items 411 to 414 are disposed in the first display area 41, items 421 to 422 are disposed in the second display area 42, and items 431 to 419 are disposed in the third display area 43. The user may select an item by clicking on it.

It should be noted that items refer to visual objects displayed in respective presentation areas of a GUI in a mobile terminal or a display device to represent corresponding contents such as icons, thumbnails, video clips, links, etc., and the items may provide a user with various conventional program contents received through data broadcasting, and various application and service contents set by a content manufacturer.

In some embodiments, the item may represent an interface or a set of interfaces to which the display device 200 is connected with an external device, or may represent a name of an external device connected to the display device, or the like. Such as: a signal source input interface set, or an HDMI interface, a USB interface, a PC terminal interface, etc.

The presentation forms of items are often diverse. For example, the items may include text content and/or images for displaying thumbnails related to the text content, or video clips related to the text. As another example, the item may be text and/or an icon of an application.

A schematic diagram of a second GUI provided by the mobile terminal 100B is illustrated in fig. 5.

As shown in fig. 5, the GUI may be implemented as a home page of the mobile terminal. The first showing area 51 is a Channel showing area, the Channel showing area comprises items 511-514 for providing commonly used channels for users, and the items 511-514 are Channel 1 (Channel 1), Channel 2 (Channel 2), Channel 3 (Channel 3) and All (Channel list) respectively; the second display area 52 is an application display area, the application display area comprises items 521-524 for providing common application programs for users, and the items 521-524 are App 1 (application 1), App 2 (application 2), App 3 (application 3) and All (application list) respectively; the third display area 53 is an Input end display area, the Input end display area includes items 531-534 providing commonly used Input ends for users, and the items 531-534 are Input 1 (Input end 1), Input 2 (Input end 2), Input 3 (Input end 3) and All (Input end list), respectively; the fourth presentation area 54 includes items 541-542 that provide the user with common use, items 541-542 being TV Content and personal center, respectively.

In addition, the current system language of the mobile terminal is selected to be english, which is the system language set by the mobile terminal after the first startup or the system language set based on user input after the mobile terminal is started, so the GUI content provided by the mobile terminal is also english.

In fig. 5, the user clicks All (channel list), and the mobile terminal provides a third GUI displaying a channel list of television programs in response to an instruction input by the user, as shown in fig. 6. In fig. 6, the first display area 61 is a channel system display area, the channel system display area includes items 611 to 612, the items 611 to 612 are two lists, namely, antenna and cable, respectively, and may further include an item 613, and the item 613 is a Fav List; the favorite channel list marks favorite channels as favorite channels for users, and all the favorite channels are gathered into a favorite channel list. The second display area 62 is a Channel display area, the Channel display area includes items 621 to 628, and the items 621 to 628 are Channel 1 (Channel 1), Channel 2 (Channel 2), Channel 3 (Channel 3), Channel 4 (Channel 4), Channel 5 (Channel 5), Channel6 (Channel 6), Channel 7 (Channel 7), and Channel 8 (Channel 8), respectively; the user can select a desired channel system by touching and sliding the channel system display area, and the channel display area displays a channel list under the channel system selected by the user. The user can watch more channels by touching the sliding channel presentation area.

In fig. 6, the user may select a channel desired to be viewed by clicking on an item of the channel presentation area. For example, the user has clicked on item 621, i.e., channel 1. The mobile terminal provides a fourth GUI as shown in fig. 7 in response to the instruction input by the user. The first display area 41 comprises items 411-414, wherein the items 411-414 are Basic (Basic interface), Live TV (application program), Media (Media) and Text (keyboard interface), respectively; the second display area 42 is a current Channel and Program display area, the second display area 42 includes items 421 to 422, the items 421 to 422 are Channel 1 (a Channel) and Program103:25-03:50 (a Program and Program broadcasting time), respectively; the third presentation area 43 includes items 431 to 439, and items 431 to 439 are CH (channel switching), EPG (electronic program guide), PVR (program recording reservation), t.shift (time shift start), Subtitle (Subtitle), TXT (teletext), Language (Language), Last View (Last) and volume adjustment, respectively. For example: the user can click the EPG to watch the electronic program guide corresponding to the channel, where the electronic program guide is shown in fig. 8A; a user can click the PVR to enable the display equipment to enter a program recording reservation interface; the user can click T.shift to enable the display device to start time shifting; the user can click to enable the display device to display the language modification interface; the user can change the volume of the display device by touching the sliding volume adjustment bar.

In FIG. 6, the items 621-628 all include an item 63, and the item 63 is more program information. After the user clicks item 63 of items 621, the mobile terminal responds to the instruction input by the user, and as shown in fig. 8A, the mobile terminal provides a fifth GUI, i.e., an interface of the electronic program guide corresponding to channel 1. In FIG. 8A, the first display area 81 includes items 811-819, where the items 811-819 correspond to 9 pieces of program information respectively. The items 811-819 include item 83, and item 83 is a further option. When the user clicks the item 83, the mobile terminal responds to the instruction input by the user, as shown in fig. 8B, the second display area 82 is displayed on the first display area, the second display area 82 includes items 821 to 822, and the items 821 to 822 are a PVR (program recording reservation) and a Reminder (program playing reservation), respectively. After the user clicks the register, the mobile terminal responds to the instruction input by the user, as shown in fig. 9, a program playing reservation interface is displayed, and the user can further change the program playing reservation Time by touching the sliding Start Time (Start Time) and also modify the reservation repetition period. After the user clicks the PVR, the mobile terminal displays a Program Recording reservation interface in response to an instruction input by the user as shown in fig. 10, and the user may further change the Start and End times of the Program Recording reservation by touching and sliding Start Time (Start Time) and End Time (End Time), and may also modify Repeat Mode (reservation kind), as shown in fig. 11, may select Time (Time) and Program, and may also modify Recording Repeat (reservation repetition period), as shown in fig. 12, may select Once and Program, day to day, Saturday to day, and Weekly (week). After all the reservation information is selected, the Save is clicked, so that a reservation adding instruction can be sent to the display device, and the display device executes reservation adding operation.

In fig. 5, the user clicks on item 542, i.e., the personal center, and there is a user saved PVR recording file and time shift file in the personal center interface. After the user selects a time shift file, as shown in fig. 13, the user may implement synchronization of the time shift progress bar by touching and sliding the time shift file play progress bar.

A flow chart of a method for controlling display device functional operation by a mobile terminal is illustrated in fig. 14.

Referring to fig. 14, the method includes the following steps:

In some embodiments, the mobile terminal client (i.e. the remote connection application running in the mobile terminal) and the display device client (i.e. the remote connection application running in the display device) are connected by MQTT protocol, and communicate by sending topic to each other. The display device can be an android television, and the mobile terminal can be a mobile phone, a computer and the like. The remote connection application program running in the display device needs to complete control over LiveTv (application for controlling a series of operations such as playing of the display device and the like by the display device side), and the remote connection application program and the LiveTv are communicated in an AIDL mode.

The remote connection application program running in the mobile terminal is an application installed on the mobile terminal, can be regarded as an intelligent remote controller and can control Live Tv. The remote connection application running in the display device is an application installed on the display device to assist the remote connection application running in the mobile terminal in controlling the display device and interacting with other applications on the display device, etc. Live Tv is an application on a television, and some functions of a traditional television which need to be completed on an intelligent television include a series of functions of playing channels, channel lists, PVRs, EPGs, t.shifts and the like.

MQTT is a lightweight message publish/subscribe protocol, which is a solution for implementing a message push server based on a mobile terminal client.

The MQTT protocol is realized by communication between a client and a server, and in the communication process, the MQTT protocol has three identities: publisher (Publish), Broker (Broker) (server), subscriber (Subscribe). The message publisher and the message subscriber are clients, the message broker is a server, and the message publisher can be a subscriber at the same time.

The transmitted messages of the MQTT are divided into: theme (Topic) and load (payload) two parts:

(1) topic, which can be understood as the type of message, after a subscriber subscribes (Subscribe), the subscriber will receive the message content (payload) of the Topic;

(2) payload, which may be understood as the content of a message, refers to content that a subscriber specifically wants to use.

In the scheme, the mobile terminal belongs to MQTT Client, and the display equipment end comprises an MQTT service part and an MQTTClient part. The MQTT client is connected to the MQTT server end by using a fixed user name and a fixed password. The mobile terminal MQTT client and the display equipment MQTT client are both publishers and subscribers of messages, and the messages are firstly sent to an MQTT server end and then sent to the other MQTT client in the communication between the mobile terminal MQTT client and the display equipment MQTT client.

AIDL: android Interface Definition Language, namely Android Interface Definition Language; the method and the device are used for performing cross-process communication between a certain Service and a plurality of application program components, so that a plurality of application programs can share the same Service.

The remote connection application program running in the display device serves as a service end in the AIDL, AIDL files, data encapsulation classes and the like are provided, and when data are transmitted, the data classes are converted into Json character strings to be transmitted. The remote connection application program running in the display device provides a Service for binding a client terminal (LivetV). All the data are packaged in a jar package, and the Livetv only needs to load jar into a project and then realize a function in the Livetv Client.

Step S141: receiving an input instruction sent by a user through a mobile terminal, wherein the mobile terminal displays a user interface, the user interface comprises at least one function operation item, and the input instruction is input by clicking or touching the function operation item by the user;

Step S142: responding to an input instruction, and executing an operation corresponding to the input instruction.

In some embodiments, when the display device is powered on, the remote connection application running in the display device is started after receiving the power-on broadcast. And then monitoring a database tv.db where the channel list is located, and sending the channel list information to the mobile terminal. After a user operates the mobile terminal, for example, a Channel switching operation is performed, a remote connection application program running in the mobile terminal transmits information such as a Channel ID (Channel ID) and a Tuner Mode (Channel format) to a remote connection application program running in the display device in a topic form through an MQTT protocol, and the remote connection application program running in the display device receives the information and transmits the information to LiveTv in an aid model to complete Channel switching. Similarly, a series of operations related to LiveTv, such as EPG acquisition, PVR scheduled recording, program play scheduling, PVR playing, time shift starting, PVR playing progress bar synchronization, and time shift playing progress bar synchronization, are all completed in this way.

The remote connection application program running in the mobile terminal and the remote connection application program running in the display device send topic through the MQTT to complete communication, and the structure of the topic comprises action and message. The two determine the operation to be executed according to the action, for example, the action is channel change, and then finish the operation of switching to which station according to the channel information in the message (json).

Step S143: and sending the operation completion result to the mobile terminal.

Fig. 15 is a flowchart illustrating a method for controlling channel switching and adding a reservation of a display device by a mobile terminal.

In some embodiments, when the remote connection application running in the display device communicates with the remote connection application running in the mobile terminal through the topic, the topic is divided into two types, one type is one-to-one type, that is, one display device communicates with one mobile terminal, and another type is one-to-many type, that is, one display device communicates with a plurality of mobile terminals. When one-to-one, the topic contains the uuid of the connected mobile phone, and the MQTT service transmits the message to the appointed mobile terminal after receiving the message. When the mobile phone is one-to-many, the topic sent by the display equipment end does not contain uuid any more, but the topic is changed into character string broadCast, and after the MQTT service receives the topic containing broadCast, the topic is sent to all mobile phones connected with the display equipment.

Fig. 15 shows two examples of controlling the display device to play and other functions by the mobile terminal, one is to control the display device to switch channels, and the other is to control the display device to add a PVR recording reservation. In this embodiment, the display device is a television, and the mobile terminal is a mobile phone.

Referring to fig. 15, the method for switching channels includes the following steps:

1. the remote connection application program at the television end receives the topic sent by the remote connection application program at the mobile phone end, firstly judges the action in the topic, and judges what operation is to be performed by the mobile phone, such as switching channels.

2. The television-side remote connection application program extracts the Channel Mode and Channel ID field information according to the message contained in the topic, and calls a Channel For remote connection application program (int Tuner Mode, int Channel ID) function in the Livetv Client in an AIDL Mode.

And 3, Live Tv completes the table cutting action.

And 4, after the Live Tv receives the channel switching completion callback (bottom layer callback), sending the current channel information state to the television remote connection application program in an AIDL mode.

5. After receiving the state sent by the LiveTv, the remote connection application program at the television end broadcasts the state information to all mobile phones connected with the television through topic.

6. After receiving the request, the mobile phone jumps to a channel control interface, as shown in fig. 7.

Referring to fig. 15, the method for scheduling the addition of PVR recording includes the following steps:

1. the user performs a reservation operation through the PVR recording reservation interface of fig. 10.

2. After the remote connection application program at the television end receives topic, the action is judged, for example, the action is addbinding (reservation is added)

3. And the television remote connection application program sends the message in the topic to LivetV in an AIDL mode.

And 4, completing the addition reservation operation by the Livetv according to the message, and informing the reservation result to the television end to remotely connect the application program. And simultaneously sending a BookingListChange message to the television end to remotely connect the application program. The time or time zone of the television end is not necessarily the same as that of the mobile phone end, and the transmission time of the television end and the time or time zone of the mobile phone end is unified and is firstly converted into the time of 0 time zone.

5. After receiving the message of the reservation result sent by the Livetv in an AIDL mode, the television remote connection application program informs the specified mobile phone (here, the mobile phone performing the adding reservation operation) of the message by sending the topic of the specified uuid.

6. And the remote connection application program of the television end receives the BookingListChange message sent by the Livetv in an AIDL mode, and sends the message to the mobile terminal in a topic broadcasting mode.

7. After the mobile phone end receives the Topic of BookingListChange, the mobile phone end judges that the current interface is not in the interface of the reservation list

And 8, if the client is in the reservation list interface, actively sending topic of getTokingList (acquisition reservation list) to the remote connection application program of the television, and if the client is not in the reservation list interface, not doing any operation.

9. After the remote connection application program of the television end receives the topic of the getBlockingList sent by the mobile phone, the remote connection application program calls a getBlockingList interface in the Livetv Client in an AIDL mode to finish the acquisition of the reservation list.

10. The remote connection application program of the television end transmits the acquired BookingList to the mobile phone end in a topic form

11. And the mobile phone terminal updates the reservation list after receiving the reservation list.

In some embodiments, controlling the functional operation of the display device via the mobile terminal is premised on the mobile terminal establishing a connection with the display device. As shown in fig. 16, the display device is a television, and the mobile terminal is a mobile phone terminal. In the application, a television and a mobile phone are connected through a local area network (unified wifi network), the wifi MAC address of each device is fixed and unique, uuid (unique identification of one device) is generated through the wifi MAC address of each device, the mobile phone terminal actively initiates connection, a topic for verifying whether the verification is legal is sent out, the topic is provided with uuid information of the mobile phone terminal, the television terminal checks the mobile phone terminal after receiving the topic, whether the uuid of the mobile phone terminal is already in a connected list is judged, if the connection is allowed, if the uuid is not, a password box pops up, the mobile phone terminal inputs a password and then sends the password to the television terminal, and the television terminal determines to prevent the mobile phone terminal from being connected according to whether the password input by a user is consistent with the display screen.

In the above embodiment, after the display device is connected to the mobile terminal through the communicator (WIFI module), the information such as the channel list and the electronic program guide in the display device is sent to the mobile terminal, so that the user can complete the functional operation operations such as channel switching, electronic program guide acquisition, program recording reservation, program playing reservation and the like by clicking or touching and sliding on the mobile terminal. For example, in channel switching operation, a user only needs to click a channel to be played on a channel list at a mobile phone end, and can switch to the channel to be played in one step, unlike a remote controller which needs to move a focus one by one. The appointment time can be changed only by sliding the calendar with fingers to select the time when the program is reserved for recording and playing. The user operation steps are simplified, great convenience is brought to the user, and the user experience is improved.

The foregoing embodiments have been presented for the purposes of illustration and description, and are not intended to be exhaustive or limiting of the application. Individual elements or features of a particular embodiment are generally not limited to that particular embodiment, but may be used or interchanged in selected embodiments where applicable, even if not specifically shown or described. Also, many modifications may be made which are not to be regarded as departure from the scope of the claims appended hereto, and all such modifications are intended to be included within the scope of the claims appended hereto.

Claims

1. A display device, comprising:

a display;

a communicator for communicating with a mobile terminal;

a controller for performing:

2. The display device of claim 1, wherein the functional operation comprises a channel switch, a program recording schedule, an electronic program guide acquisition, a recorded program play, a program play schedule, a time shift launch, a recorded program play progress bar synchronization, and a time shift play progress bar synchronization.

3. The display device according to claim 1, wherein the controller is further configured to perform:

and sending the operation completion result to the mobile terminal.

4. The display device according to claim 1, wherein the communicator communicates with the mobile terminal by transmitting a theme by connecting with the mobile terminal through MQTT protocol.

5. A display device, comprising:

a display;

a communicator for communicating with a mobile terminal;

a controller for performing:

responding to an input instruction sent by a user through the mobile terminal, and switching to a channel corresponding to the project;

6. The display device according to claim 5, wherein the controller is further configured to perform:

and sending the channel switching result to the mobile terminal.

7. The display device according to claim 5, wherein the controller is further configured to perform:

8. A display device, comprising:

a display;

a communicator for communicating with a mobile terminal;

a controller for performing:

9. The display device according to claim 8, wherein the controller is further configured to perform:

and sending the reservation result to the mobile terminal.

10. The display device according to claim 9, wherein the controller is further configured to perform:

updating the reservation list;

and sending the updated reservation list to the mobile terminal.

11. A method for controlling functional operations of a display device through a mobile terminal, comprising:

12. A method for controlling channel switching of a display device through a mobile terminal is characterized by comprising the following steps:

13. A method for controlling a display device to add a reservation through a mobile terminal is characterized by comprising the following steps:

14. The mobile terminal is characterized in that a user interface is displayed, the user interface comprises at least one function operation item, and an input instruction input by a user through clicking or touch sliding the function operation item is sent to a display device, so that the display device responds to the input instruction and executes an operation corresponding to the input instruction.

15. A system for controlling functional operations of a display device through a mobile terminal, comprising: