WO2023065976A1

WO2023065976A1 - Terminal device, display device and display method thereof

Info

Publication number: WO2023065976A1
Application number: PCT/CN2022/121223
Authority: WO
Inventors: 王光强; 李金昆; 庞丽军; 卢可敬; 杨绍栋; 刘文静; 杨明
Original assignee: 聚好看科技股份有限公司
Priority date: 2021-10-19
Filing date: 2022-09-26
Publication date: 2023-04-27
Also published as: CN115993919A; CN118202324A

Abstract

The present application relates to a terminal device, a display device, and a display method. The terminal device comprises: a display unit; a communication unit configured to be communicatively connected to a display device; and a processor configured to: in response to an input application scenario control instruction, send the application scenario control instruction to the display device, so that the display device feeds back an application scenario type, wherein application scenario type is the type of an application scenario to which a user interface displayed in the display device relates; receive the application scenario type sent by the display device; and generate an application scenario control interface according to the application scenario type and control the display unit to display the application scenario control interface, wherein the application scenario control interface is used for controlling the user interface in the application scenario, and different application scenario types corresponding to different application scenario control interfaces.

Description

Terminal device, display device and display method thereof

Cross References to Related Applications

This application claims the priority of Chinese Patent Application No. 202111214613.1 filed on October 19, 2021 and No. 202210210654.1 filed on March 4, 2022, the entire contents of which are incorporated herein by reference.

technical field

The present application relates to the technical field of display devices, and in particular, to a display device and a terminal device.

Background technique

Display devices refer to terminal devices capable of outputting specific display images, such as smart TVs, mobile terminals, smart advertising screens, projectors, etc. With the rapid development of display devices, the functions of display devices will become more and more abundant, and the performance will become more and more powerful. It can realize two-way human-computer interaction function, integrate audio-visual, entertainment, data and other functions into one, to meet the needs of users. Diverse and individual needs.

With the wide application of display devices, people have higher and higher requirements for display devices. Users can not only use remote controls to control display devices, but also use terminal devices, such as mobile phones, to control display devices.

However, when the terminal device is used to control the display device, the functions that the terminal device can realize are generally the same as those that can be realized by the remote control. The terminal device only replaces the remote control, and the functions that can be realized are limited, resulting in user experience Sex is poor.

Contents of the invention

An embodiment of the present application provides a terminal device, including: a display unit; a communication unit configured to communicate with the display device; a processor configured to: respond to an input application scene control instruction, The control instruction is sent to the display device, so that the display device feeds back the application scene type, and the application scene type is the type of the application scene to which the user interface displayed on the display device belongs; receiving the application scene sent by the display device type; generate an application scenario control interface according to the application scenario type and control the display unit to display the application scenario control interface, the application scenario control interface is used to control the user interface in the application scenario, wherein different applications Scene types correspond to different application scene control interfaces.

The embodiment of the present application also provides a display device, including: a display configured to display a user interface; a communicator configured to communicate with a terminal device; a controller configured to: detect the display device and the terminal device Establish a communication connection, respond to the application scene control instruction sent by the terminal device, detect the application scene type, the application scene type is the type of the application scene to which the user interface belongs; send the application scene type to the terminal device, so that the terminal device generates and displays an application scenario control interface according to the application scenario type, and the service scenario control interface is used to control a user interface in the application scenario.

Description of drawings

FIG. 1 shows a usage scenario of a display device according to some embodiments;

Fig. 2 shows a hardware configuration block diagram of the control device 100 according to some embodiments;

FIG. 3 shows a block diagram of a hardware configuration of a display device 200 according to some embodiments;

FIG. 4 shows a software configuration diagram in a display device 200 according to some embodiments;

Figure 5 shows a schematic diagram of a user interface in some embodiments;

Fig. 6 shows a schematic diagram of displaying communication mode confirmation information in a display in some embodiments;

Fig. 7 shows a schematic diagram of setting an identity authentication mode in a display device in some embodiments;

Fig. 8 shows a schematic flowchart of communication interaction between a terminal device and a display device in some embodiments;

Fig. 9 shows a schematic diagram of a terminal interface of a terminal device in some embodiments;

Fig. 10 shows a schematic diagram of a display device displaying a "video interface" in some embodiments;

Figure 11 shows the "Movie Introduction" page corresponding to "Movie A" in some embodiments;

Figure 12 shows a schematic diagram of an application panel in some embodiments;

Figure 13a shows a schematic diagram of an initial control interface of a terminal device in some embodiments;

Figure 13b shows a schematic diagram of an initial control interface of a terminal device in some embodiments;

Figure 13c shows a schematic diagram of an initial control interface of a terminal device in some embodiments;

Fig. 14 shows a schematic diagram of an initial control interface of a terminal device in some embodiments;

Fig. 15 shows a flowchart of interaction between a display device and a terminal device in some embodiments;

Fig. 16a shows a schematic diagram of a media asset recommendation scenario in some embodiments;

Figure 16b shows a schematic diagram of a media asset search scenario in some embodiments;

Figure 17 shows an interaction diagram of components in a display device in some embodiments;

Fig. 18a shows a schematic diagram of a display device displaying a media asset recommendation page in some embodiments;

Fig. 18b shows a schematic diagram of a terminal device displaying a media asset recommendation page in some embodiments;

Fig. 19 shows a schematic diagram of a terminal device displaying an application scene control interface in some embodiments;

Fig. 20 shows a schematic diagram of a terminal device displaying an application scene control interface in some embodiments;

Fig. 21a shows a schematic diagram of a display showing a progress bar of a media resource in some embodiments;

Figure 21b shows a schematic diagram of displaying a pause sign on a display in some embodiments;

Figure 21c shows a schematic diagram of double-speed prompt information in some embodiments;

Fig. 22 shows a schematic diagram of a terminal device displaying an application scene control interface in some embodiments;

Fig. 23 shows a schematic diagram of a display device in a picture scene in some embodiments;

Fig. 24 shows a schematic diagram of a terminal device displaying an application scene control interface in some embodiments;

Fig. 25a shows a schematic diagram of displaying picture switching information on a display in some embodiments;

Fig. 25b shows a schematic diagram of zooming in on a display image displayed on a display in some embodiments;

Fig. 26 shows a schematic diagram of screen casting prompt information in some embodiments;

Fig. 27 shows a schematic diagram of a control interface of a terminal device in some embodiments;

Fig. 28 shows the situation where the terminal device displays the control interface in some embodiments;

Fig. 29 shows a schematic diagram of application scenarios in some embodiments;

Fig. 30 shows a schematic diagram of screens on a display in a multi-window mode in a feasible embodiment;

Fig. 31 shows a schematic diagram of a multi-window application scenario in some embodiments;

Fig. 32 shows an overall flow chart of a display device control method in some embodiments.

Detailed ways

In order to make the purpose and implementation of the application clearer, the following will clearly and completely describe the exemplary implementation of the application in conjunction with the accompanying drawings in the exemplary embodiment of the application. Obviously, the described exemplary embodiment is only the present application. Claim some of the examples, not all of them.

It should be noted that the brief description of the terms in this application is only for the convenience of understanding the implementations described below, and is not intended to limit the implementations of this application. These terms are to be understood according to their ordinary and usual meaning unless otherwise stated.

The display devices in the application scenarios of the embodiments of the present application include but are not limited to display devices with data sending and receiving and processing functions, image display functions and/or sound output functions, such as TVs, smart TVs, smart refrigerators, laser projection devices, monitors ), electronic bulletin board, electronic table, etc.

Fig. 1 is a schematic diagram of a usage scenario of a display device according to an embodiment. As shown in FIG. 1 , the display device 200 also performs data communication with the server 400 , and the user can operate the display device 200 through the smart device 300 or the control device 100 .

In some embodiments, the control device 100 may be a remote controller, and the communication between the remote controller and the display device includes at least one of infrared protocol communication, Bluetooth protocol communication, and other short-distance communication methods, and the display device is controlled wirelessly or wiredly. Device 200. The user can control the display device 200 by inputting user instructions through at least one of buttons on the remote control, voice input, and control panel input.

In some embodiments, the smart device 300 may include any one of a mobile terminal, a tablet computer, a computer, a notebook computer, an AR/VR device, and the like.

In some embodiments, the smart device 300 can also be used to control the display device 200 . For example, the display device 200 is controlled using a camera application running on the smart device.

In some embodiments, the smart device 300 and the display device may also be used for data communication.

In some embodiments, the display device 200 can also be controlled in a manner other than the control device 100 and the smart device 300. For example, the module for obtaining voice commands configured inside the display device 200 can directly receive the user's voice command control , the user's voice command control can also be received through the voice control device provided outside the display device 200 .

In some embodiments, the display device 200 also performs data communication with the server 400 . The display device 200 may be allowed to communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks. The server 400 may provide various contents and interactions to the display device 200 . Server 400 may be one or more types of servers. The number of display devices may also be one or more. For example, multiple display devices 100 may exist in a scene at the same time.

Fig. 2 shows a configuration block diagram of the control device 100 according to some embodiments. As shown in FIG. 2 , the control device 100 includes a controller 110 , a communication interface 130 , a user input/output interface 140 , a memory, and a power supply. The control device 100 can receive the user's input operation instruction, and convert the operation instruction into an instruction that the display device 200 can recognize and respond to, and play an intermediary role between the user and the display device 200 . The communication interface 130 is used for communicating with the outside, and includes at least one of a WIFI chip, a Bluetooth module, NFC or an alternative module. The user input/output interface 140 includes at least one of a microphone, a touch pad, a sensor, a key or an alternative module.

FIG. 3 shows a block diagram of a hardware configuration of a display device 200 according to some embodiments. As shown in FIG. 3 , the display device 200 includes a tuner and demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface 280. at least one of . The controller includes a central processing unit, a video processor, an audio processor, a graphics processor, RAM, ROM, the first interface to the nth interface for input/output. In some embodiments, the display 260 includes a display screen component for presenting images, and a drive component for driving image display, for receiving image signals output from the controller, and displaying video content, image content, and menu manipulation interface. Components and user manipulation of the UI interface, etc. The display 260 may be at least one of a liquid crystal display, an OLED display, a touch display, and a projection display, and may also be a projection device and a projection screen. The tuner demodulator 210 receives broadcast TV signals through wired or wireless reception, and demodulates audio and video signals, such as EPG data signals, from multiple wireless or cable broadcast TV signals. In some embodiments, the communicator 220 is a component for communicating with external devices or servers according to various communication protocol types. For example, the communicator may include at least one of a Wifi module, a Bluetooth module, a wired Ethernet module and other network communication protocol chips or near field communication protocol chips, and an infrared receiver. The display device 200 can establish transmission and reception of control signals and data signals with the control device 100 or the server 400 through the communicator 220 . For example, the detector 230 includes a light receiver, which is a sensor for collecting ambient light intensity; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environmental scenes, user attributes or user interaction gestures, or , the detector 230 includes a sound collector, such as a microphone, for receiving external sound.

In some embodiments, the external device interface 240 may include, but is not limited to, the following: high-definition multimedia interface (HDMI), analog or data high-definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), Any one or more interfaces such as RGB ports. It may also be a composite input/output interface formed by the above-mentioned multiple interfaces. The detector 230 is used to collect signals of the external environment or interaction with the outside. The controller 250 and the tuner-demodulator 210 may be located in different split devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

In some embodiments, the controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operations of the display device 200 . For example, in response to receiving a user command for selecting a UI object to be displayed on the display 260, the controller 250 may perform an operation related to the object selected by the user command.

In some embodiments, the controller includes a central processing unit (Central Processing Unit, CPU), a video processor, an audio processor, a graphics processing unit (Graphics Processing Unit, GPU), RAM Random Access Memory, RAM), ROM (Read- Only Memory, ROM), at least one of the first interface to the nth interface for input/output, a communication bus (Bus), and the like.

In some embodiments, the video processor is used to receive an external video signal and perform video decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, image synthesis, etc. according to the standard codec protocol of the input signal. At least one of the processing can obtain a signal that can be directly displayed or played on the display device 200 .

In some embodiments, the audio processor is used to receive an external audio signal, perform decompression and decoding according to the standard codec protocol of the input signal, and perform at least one of processing such as noise reduction, digital-to-analog conversion, and amplification processing , to obtain a sound signal that can be played on a loudspeaker.

The user can input a user command through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, the user may input a user command by inputting a specific sound or gesture, and the user input interface recognizes the sound or gesture through a sensor to receive the user input command.

FIG. 4 shows a schematic diagram of software configuration in the display device 200 according to some embodiments. As shown in FIG. 4 , the system is divided into four layers, which are the Applications (Applications) layer (abbreviated as "application layer") from top to bottom. , Application Framework (Application Framework) layer (referred to as "framework layer"), Android runtime (Android runtime) and system library layer (referred to as "system runtime layer"), and the kernel layer. The kernel layer contains at least one of the following drivers: audio driver, display driver, bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power supply drive etc.

With the wide application of display devices, people have higher and higher requirements for display devices. Users can not only use remote controls to control display devices, but also use terminal devices, such as mobile phones, to control display devices. However, when the terminal device is used to control the display device, the functions that the terminal device can realize are generally the same as those that can be realized by the remote control. The terminal device only replaces the remote control, and the functions that can be realized are limited, resulting in user experience Sex is poor.

The present application provides a display device and a terminal device. Display devices include displays, communicators and controllers. Wherein, the display is used to display the user interface. The user interface may contain specific target images, such as various media resources obtained from network signal sources, including videos, pictures and other content. The user interface may also be some UI interfaces of the display device.

The communicator is used for communicating with the terminal equipment, and can also communicate with the server.

In some embodiments, after the user controls the display device to be turned on, the controller may control the display to display the user interface. Figure 5 shows a schematic diagram of a user interface in some embodiments. The user interface includes a first navigation bar 500, a second navigation bar 510, a function bar 520 and a content display area 530, and the function bar 520 includes a plurality of functional controls such as "watch history", "my favorites" and "my applications", etc. . Wherein, the content displayed in the content display area 530 will change with the selected controls in the first navigation bar 500 and the second navigation bar 510 . When on the application panel page, the user can click on the "My Application" control to input a display instruction for the application panel page to trigger entry to the corresponding application panel. It should be noted that the user may also input a selection operation on the functional control in other ways to trigger entry into the application panel. For example, using a voice control function or a search function, etc., to control access to an application panel page.

In some embodiments, the user can use the terminal device to communicate with the display device, so as to realize information interaction between the terminal device and the display device. The terminal device can be a smart phone, a tablet computer, and the like.

A terminal device may include a display unit, a communication unit, and a processor. Wherein, the display unit is used for displaying some pictures. The communication unit is used for communicating with the display device. The terminal device may send a communication connection request to the display device, so that the terminal device and the display device perform a communication connection. After the terminal device and the display device are connected by communication, communication and interaction can be performed. For example, the terminal device can obtain some data from the display device, such as media asset data and current display data of the display device. The terminal device can also upload data such as media resources in the terminal device to the display device for playback. The user can also use the terminal device to send some control instructions to the display device, so as to control the display device to realize corresponding functions.

In some embodiments, the display device may be provided with a communication mode. When the communication mode is off, the display device will not receive the communication connection request sent by the terminal device. At this time, the display device does not allow the terminal device to communicate with each other, and the display device will not exchange information with any terminal device at this time. When the communication mode is turned on, the display device will receive a communication connection request sent by the terminal device. At this time, the display device may allow the terminal device to communicate with the display device, so as to realize information interaction. The user can input an instruction to start the communication mode to the display device, and when the controller receives the instruction to start the communication mode, it can control the display device to enter the communication mode.

In some embodiments, the user can send a communication mode command to the display device by operating a designated key on the remote controller. In the process of actual application, the corresponding relationship between the communication mode command and the key of the remote control is bound in advance. For example, a communication mode button is set on the remote controller. When the user touches the button, the remote controller sends a communication mode command to the controller, and the controller controls the display device to enter the communication mode. When the user touches the button again, the controller can control the display device to exit the communication mode.

In some embodiments, the correspondence between the communication mode command and multiple remote control keys may also be bound in advance, and when the user touches multiple keys bound to the communication mode command, the remote control sends out the communication mode command. In a feasible embodiment, the keys bound to the communication mode command are direction keys (left, down, left, down) in turn, that is, when the user touches the keys (left, down, left, down) continuously within a preset time In the case of , the remote controller sends the communication mode command to the controller. By adopting the above-mentioned binding method, it is possible to prevent the communication mode command from being issued due to a user's misoperation. The embodiment of the present application is only an example to provide the binding relationship between several communication mode commands and keys. In the process of actual application, the binding relationship between communication mode commands and keys can be set according to the user's habits. This is not too restrictive.

In some embodiments, the user can use a sound collector of the display device, such as a microphone, to send a communication mode instruction to the display device through voice input, so as to control the display device to enter the communication mode. An intelligent voice system may be installed in the display device, and the intelligent voice system may recognize the user's voice to extract the instruction content input by the user. The user can enter the preset wake-up word through the microphone to activate the intelligent voice system, so that the controller can respond to the user's input. And a communication mode command is input within a certain period of time, so that the display device enters the communication mode. For example, the user can input "so-and-so classmate" to activate the intelligent voice system. Then enter "Enter Communication Mode" to send communication mode instructions to the display device.

In some embodiments, the user may also send a communication mode instruction to the display device through a preset gesture. The display device can detect the behavior of the user through an image collector, such as a camera. When the user makes a preset gesture, it can be considered that the user sends a communication mode instruction to the display device. For example, it may be set as follows: when it is detected that the user draws the word V, it is determined that the user has input a communication mode instruction to the display device. The user can also send a communication mode instruction to the display device through a preset action. For example, it may be set that: when it is detected that the user lifts the left foot and the right hand simultaneously, it is determined that the user has input a communication mode instruction to the display device.

In some embodiments, when the user uses a smart device to control the display device, for example, a mobile phone, a communication mode instruction may also be sent to the display device. In the process of practical application, a control can be set in the mobile phone, and the control can be used to select whether to enter the communication mode, so as to send a communication mode command to the controller. At this time, the controller can control the display device to enter the communication mode.

In some embodiments, when the user uses the mobile phone to control the display device, a continuous clicking instruction can be issued to the mobile phone. The continuous click instruction refers to: within a preset period, the number of times the user clicks on the same area of the touch screen of the mobile phone exceeds a preset threshold. For example: when the user taps a certain area of the touch screen of the mobile phone 3 times within 1 second, it is regarded as a continuous tap instruction. After receiving the continuous click instruction, the mobile phone can send a communication mode instruction to the display device, so that the controller controls the display device to enter the communication mode.

In some embodiments, when the user uses the mobile phone to control the display device, it can also be set to: when it is detected that the user's touch pressure value on a certain area of the mobile phone touch screen exceeds the preset pressure threshold, the mobile phone can send a message to the display device. Communication mode command.

The communication mode option can also be set in the UI interface of the display device, and when the user clicks on this option, the display device can be controlled to enter or exit the communication mode.

In some embodiments, in order to prevent the user from triggering the communication mode by mistake, when the controller receives the communication mode command, it can control the display to display communication mode confirmation information, so that the user can make a second confirmation whether to control the display device to enter the communication mode. Fig. 6 shows a schematic diagram of displaying communication mode confirmation information on a display in some embodiments.

In some embodiments, when the display device is in the communication mode, an identity authentication mode may be further set in consideration of security. When the identity authentication mode is off, the display device will not perform identity authentication on the terminal device, and at this time the terminal device can directly communicate with the display device. That is, when the terminal device sends a communication connection request to the display device, the display device may directly perform a communication connection with the terminal device without verifying the communication connection request.

When the identity authentication mode is turned on, the display device will authenticate the terminal device. That is, when the display device receives the communication connection request sent by the terminal device, it will verify the communication connection request, and when the verification is passed, the communication connection request is allowed, so as to communicate with the terminal device.

Fig. 7 shows a schematic diagram of setting an identity authentication mode in a display device in some embodiments. Wherein, when the user chooses to turn on the communication mode, he may further choose to turn on the identity authentication mode or turn off the identity authentication mode.

In some embodiments, when the display device is in the communication mode, the terminal device can communicate and interact with the display device. The terminal device can send a communication connection request to the display device, and when the display device receives the communication connection request, it can establish a communication connection with the terminal device.

Fig. 8 shows a schematic flowchart of communication and interaction between a terminal device and a display device in some embodiments.

In some embodiments, the user can establish a communication connection between the terminal device and the display device by binding and associating the terminal device with the display device. The user can use the terminal device to directly search for all display devices in the LAN, and select a target display device to communicate with the terminal device. Fig. 9 shows a schematic diagram of a terminal interface of a terminal device in some embodiments. The "connect device" control can be installed in the terminal device, and the user can click the "connect device" control to instruct the terminal device to scan for connectable display devices in the local area network, and at the same time trigger to enter the corresponding device list page. As shown in FIG. 8, in step S810, a device list page may be displayed on the terminal device. When the device list page is displayed, in step S820, a display device to be connected can be selected on the page.

After detecting that the user selects a certain display device, in step S830, the terminal device may send a communication connection request to the display device. After receiving the communication connection request, the display device can bind and associate with the terminal device, that is, establish a communication connection.

In some embodiments, the user can also log in his own account in the terminal device. After the user uses the terminal device to bind and associate with a certain display device, the user's account is also bound and associated with the display device.

In some embodiments, after the terminal device sends a communication connection request to the display device, in step S840, the display device may first detect its own device type. Specifically, the device type of the display device may include: a TV, a refrigerator, an air conditioner, and the like. After the display device determines its own device type, it can send the device type to the terminal device. At the same time, the display device can establish a communication connection with the terminal device.

In step S850, the display device may send the device type to the terminal device. After receiving the device type sent by the display device, the terminal device may display a control interface for controlling the display device corresponding to the device type, so that the user can use the control interface to control the display device. For different device types, the terminal device can display different initial control interfaces.

In step S860, the terminal device may generate an initial control interface according to the device type. The initial control interface may be a UI interface for controlling the display device of the device type. Wherein, different device types correspond to different initial control interfaces, that is, the display devices of each device type have their own initial control interfaces. The initial control interface may include several functional controls for controlling the display device to implement some functions. Further, the terminal device may display the initial control interface on the display unit. The user can directly use the initial control interface to control the display device.

In some embodiments, the initial control interface can be used to control some basic functions of the display device. Display devices of different device types have different initial control interfaces and can control different functions. For example, for a TV, you can control the volume, homepage switching, etc.; for an air conditioner, you can control the temperature and air conditioning mode, etc.

In some embodiments, the display device can support multiple functions, such as media asset playback and screen projection, and the user can control the display device to implement the supported functions. For example, for the user interface shown in Figure 5, the system home page is provided with a "video" entry option, and the user can directly select the "video" entry option, for example, by controlling the focus cursor to move to the selected "video" through a control device or terminal equipment. The entry option, thereby triggering the display device to display the "video interface". Fig. 10 shows a schematic diagram of a display device displaying a "video interface" in some embodiments. The "film and television interface" can include multiple entry options for film and television categories, such as "Selected", "Movies", "TV Dramas" and "Variety Shows". The display device is triggered to display the corresponding entry interface. For example, clicking on the "Movie" entry option triggers the display device to display the "Movie" entry interface. The "Movie" entry interface includes multiple media resource options, such as "Movie A", "Movie B" and "Movie C". The user controls the focus cursor and clicks on any media asset option, which can trigger the display device to display the corresponding "movie introduction" page. For example, the user controls the focus cursor to click "Movie A", which triggers the display device to display the "Movie Introduction" page corresponding to "Movie A". Figure 11 shows the "Movie Introduction" page corresponding to "Movie A" in some embodiments, including the "Start Play" option and the "Back" option, and the user can further choose to play "Movie A" or return to the previous interface.

In some embodiments, for the user interface shown in FIG. 5 , the user can select the "My Application" control, for example, control the focus cursor to move to "My Application" through the terminal device, so as to trigger the display device to display the application panel. The user can check the installed application programs of the display device through the application panel, that is, the functions supported by the display device. Figure 12 shows a schematic diagram of an application panel in some embodiments. The application panel includes three controls: "media playback", "picture browsing" and "casting screen". Among them, the user can control the display device to play the media resource through the "media asset playback" control. Through the "picture browsing" control, the user can control the display device to display various pictures, such as pictures stored inside the display device or pictures searched from network sources. Users can control the display device to receive and display resources sent by other devices through the "cast screen" control, so as to realize the screen projection function. Other application programs, such as "video chat" and "game", may also be installed on the display device, and the application programs that may be installed on the display device are not specifically limited in this embodiment of the present application.

In some embodiments, for a display device, its user interface may display different content, that is, the user interface may be in different application scenarios. In the embodiment of the present application, the application scene where the display device is located refers to the application scene where the user interface is located.

The user can use the terminal device to control the display device, for example, to control the display device to play media resources and so on. When the display device is in different application scenarios, the control instructions for interaction between the display device and the terminal device are also different. It should be noted that the functions that the display device can implement in different application scenarios may be different, and the application scenarios of the display device may include media asset scenarios, picture scenarios, screen projection scenarios, and so on. When the user selects the "Media Play" control in the application panel, the display device is in the media scene, and the display device can play media resources. When the terminal device interacts with the display device, it can control the media resources, such as adjusting The progress of the media resource and the current volume of the display device, etc. When the user selects the "Picture View" control in the application panel, the display device is in the picture scene, and the display device can display pictures. When the terminal device interacts with the display device, it can switch or zoom the pictures. Therefore, in different application scenarios, the control instructions required by the display device are also different. When the display device is in different application scenarios, the terminal device may also include different control interfaces, thereby displaying different control instructions.

Fig. 13a shows a schematic diagram of an initial control interface of a terminal device in some embodiments, where the device type corresponding to the initial control interface is a TV. Specifically, the control interface includes 11 kinds of operation controls: power control 1001, search control 1002, keyboard switching control 1003, program source control 1004, volume control 1005, return control 1006, main interface control 1007, menu control 1008, signal source Control 1009 , Smart Assistant Control 1010 , and Favorites Control 1011 . Wherein, the power control 1001 is used to control the opening and closing of the display device. The search control 1002 is used for searching media resources and the like. The keyboard switching control 1003 is used to switch the displayed keyboard. The keyboard includes a numeric keypad and a directional keypad. After the user clicks this control, the terminal device can switch the numeric keypad to the directional keypad. Program source control 1004 is a numeric keyboard, used to adjust the program source currently played by the display device. The user can press a specific number key to switch to the corresponding TV program, or switch TV programs back and forth according to the sequence of the program source. The volume control 1005 is used to adjust the volume of the display device. The return control 1006 is used to return to the user's last operation. The main interface control 1007 is used to return to the main interface of the display device. The menu control 1008 is used to jump to the menu interface of the display device. The signal source control 1009 is used to display a list of signal sources of the display device, including signal sources such as HDMI, USB, ATV, etc., for the user to choose. The smart assistant control 1010 is used to start the smart assistant of the display device, and the smart assistant can implement functions such as image recognition. The favorite control 1011 is used to control the display device to display the user's favorite interface, and the favorite interface may include the user's favorite media resources and the like.

Fig. 13b shows a schematic diagram of an initial control interface of a terminal device in some embodiments, where the device type corresponding to the initial control interface is a TV. For the initial control interface shown in FIG. 10a, after the user touches the keyboard switching control 1003, the terminal device will switch to display the initial control interface shown in FIG. 10b. The initial control interface shown in FIG. 10 b also includes a focus control 1012 . The focus control 1012 is a direction keyboard, and the user can use this control to adjust and confirm the focus position in the display device. In the current interface, when the user clicks the keyboard switching control 1003, the terminal device can switch the direction keyboard to the number keyboard.

Fig. 13c shows a schematic diagram of an initial control interface of a terminal device in some embodiments, and the device type corresponding to the initial control interface is an air conditioner. Specifically, the control interface includes a display area 1031 and six types of operation controls: a temperature adjustment control 1032 , a power control 1033 , a cooling control 1034 , a heating control 1035 , a wind speed control 1036 and a timing control 1037 . Wherein, the display area 1031 is used to display some working conditions of the air conditioner, such as temperature, wind speed, timing and so on. The power control 1032 is used to control the opening and closing of the air conditioner. The cooling control 1034 is used to make the air conditioner perform cooling work. The heating control 1035 is used to make the air conditioner perform heating work. The wind speed control 1036 is used to adjust the wind speed when the air conditioner is working. The timing control 1037 is used to set the timing of the air conditioner.

When the initial control interface is displayed, the terminal device can replace the remote controller and send a control instruction to the display device, so that the user can control the display device through the terminal device. Specifically, the user can click or touch a certain control in the initial control interface to input a corresponding control command. Further, the terminal device may send the control instruction input by the user to the display device. After receiving the corresponding control instruction, the display device can analyze the control instruction, so as to determine the function corresponding to the control instruction and implement it.

In some embodiments, a control interface template library may be preset in the terminal device. The control interface template library can store several initial control interfaces, each initial control interface can correspond to a device type, and is used to control the display device of the corresponding device type. After receiving the device type sent by the display device, the terminal device can directly search for an initial control interface corresponding to the device type in the control interface template library. Further, the terminal device may display the initial control interface on the display unit, and the user may use the initial control interface to control the display device.

In some embodiments, the user may use the initial control interface to control some functions of the display device. These functions may be basic functions of the display device, and may meet some basic needs of the user when using the display device.

However, the display device may have multiple application scenarios, and the application scenarios may be different modes set in the control system or application according to resource types. According to different operating system forms, display devices can provide different types and numbers of application scenarios. For example, the application scenarios may include: media asset scenarios, picture scenarios, screen projection scenarios, and so on.

In different application scenarios, the display device may recommend different resources to the user. For example, in a media resource scenario, a display device may display various media resources. In a picture scenario, the display device can display pictures for users to browse. In the screen projection scenario, the display device can display media assets set in the screen projection device.

When the display device is in different application scenarios, the functions that the display device can implement may be different. For example, in the media resource scenario, the progress of the media resource can be adjusted, the current volume of the display device can be displayed, and so on. In the image scene, the image can be switched or zoomed. For each application scenario, the display device will have some specific functions. In order to enhance the user's experience and enable the user to use the terminal device to control these specific functions, the terminal device may further display controls for controlling these specific functions. However, since each application scenario corresponds to some specific functions, it is obviously unrealistic to display all of them in the initial control interface. At this point, the terminal device can be switched from the initial control interface to a control interface for controlling functions of the display device in a current application scenario.

In some embodiments, an application scene control control may be set in the initial control interface of the terminal device. Fig. 14 shows a schematic diagram of an initial control interface of a terminal device in some embodiments. Wherein, the initial control interface includes an application scene control control 1101 . The user can tap and touch the application scene control control, for example, the user can press and hold the control and perform an upward swipe operation. When the terminal device detects the user's touch operation on the application scene control control, it can be considered that the user has input an application scene control command, that is, the user wants to control and realize some specific functions of the current application scene of the display device.

In some embodiments, the scene control control is an indicator control, indicating that the scene interface can be called under the current initialization control interface.

In some embodiments, different initial control interfaces correspond to different controlled devices. After the terminal device switches the initial control interface, according to the identifier and preset relationship of the controlled device, it is determined that the device corresponding to the current initial control interface is in the scene interface to display the scene control control. If it does not exist, the scene control will not be displayed. . The preset relationship refers to the mapping relationship between the device identifier and whether the device corresponding to the identifier is configured with a scene interface.

In some embodiments, the initial control interface of the same controlled device may invoke different scene interfaces according to the scene of the interface currently displayed on the display device.

Further, the terminal device may display a control interface corresponding to the current application scene of the display device, so that the user can control the display device. Specifically, after detecting that the user has input the application scene control instruction, the terminal device may send the application scene control instruction to the display device, and may use the communication unit to send the application scene control instruction to the display device, so that the display device feeds back the current The type of application scenario to which the user interface belongs. In the embodiment of the present application, the application scenario type is used to refer to the type of the application scenario to which the user interface belongs. Further, the terminal device can display an application scene control interface according to the type of the application scene, so that the user can control the display device. In some instances, the scene control instruction may be an instruction generated after the operation of the scene control control is selected, or an instruction generated after the detector detects an input preset gesture.

Compared with sending the scene control instruction when the initial interface is displayed, and sending it after the user detects the input operation, the operating pressure of the terminal device can be reduced. When used to display the initial interface, the user can operate the TV through other control devices such as the remote control, and can also operate the display device through the initial interface of the terminal device. Therefore, when the initial interface starts to be displayed, the scene of the display device and when the user needs to query The scenarios may be inconsistent, and it will take more computing power to establish periodic detection. Therefore, sending after detecting the preset operation input by the user is beneficial to reduce the operating pressure of the device.

In some embodiments, the terminal device may pre-store all application scenarios corresponding to each device type. It should be noted that some types of display devices may have only one application scenario, and some types of display devices may have multiple application scenarios. Therefore, after the display device sends its own device type to the terminal device, the terminal device can determine whether the display device of this device type has multiple application scenarios. If this type of display device has only one application scene, the initial control interface displayed by the terminal device may not contain application scene control controls, and the display device does not have the function of switching application scenes, and the display can be completely controlled based on the current initial control interface equipment.

If this type of display device has multiple application scenarios, the initial control interface displayed by the terminal device may include an application scenario control control. Further, when the user touches the application scene control control, the terminal device may display the application scene control interface, so that the user can better control the display device.

Fig. 15 shows a flowchart of interaction between a display device and a terminal device in some embodiments.

In some embodiments, at step S1510, the terminal device may send an application scene control instruction to the display device. After the display device receives the application scene control instruction sent by the terminal device, in step S1520, it can first check whether it has established a communication connection with the terminal device, that is, whether it is bound with the terminal device, so as to ensure the smoothness of information interaction. safety and accuracy. This is because the terminal device can switch the initial interface to operate devices including the display device, but in the initial interface, switch different short-distance control modes according to the user. For example, when the initial interface of the display device is displayed, the terminal device is in the display device short In the distance control mode, when the initial interface of the air conditioner is displayed, it is displayed that the device is in the air conditioner short-distance control mode. In some embodiments the short range control modality may be a control mode via infrared or NFC communication. Therefore, after sending the application scene control command through the short-distance control mode corresponding to the display device, the display device needs to establish a second communication mode with the terminal device that can perform complex data transmission. The second communication mode is different from the short-distance control mode. . In some embodiments, the second communication mode may be a Bluetooth communication mode. In some embodiments, the short range control modality may be an infrared control mode. Different controlled devices correspond to different infrared encoding modes. The corresponding terminal equipment also needs to set two different signal sending and receiving channels.

When it is detected that the communication connection between itself and the terminal device is established, in step S1530, the display device may detect the current application scene, so as to determine the type of the current application scene. In the embodiment of the present application, the application scenario of the display device is the application scenario to which the user interface displayed on the display belongs. In step S1540, the display device may send the application scenario type to the terminal device. Next, in step S1550, the terminal device may generate and display a corresponding application scenario control interface according to the application scenario type of the display device.

In some embodiments, the connection between the terminal device and the display device can both adopt the second communication mode. At this time, the scene command can be sent through the second communication mode, and the subsequent display device may not be provided with the steps of detecting and establishing a communication connection. This improves the responsiveness of operations.

When the display device is in different application scenarios, the specific functions it supports are different, and the control commands for interaction between the display device and the terminal device are different, so the control interfaces of the application scenarios are also different. For example, when the display device is in a media asset scene, it can also be specifically divided into a media asset recommendation scenario, a media asset playback scenario, and a media asset search scenario.

The media asset recommendation scene may be a scenario where the display device displays a media asset recommendation page, and the media asset recommendation page may include multiple recommended media asset options, and the user may select a target media asset on the media asset recommendation page. Specifically, the user can control the focus cursor to move to a certain target media asset option through the control device or the terminal device. Further, the display device can play the target media assets. Fig. 16a shows a schematic diagram of a media asset recommendation scenario in some embodiments. Specifically, when the user controls the display device to start some media asset playback applications, such as "Jukankan" or "Iqiyi", the controller can control the display to display the media asset recommendation page corresponding to each APP.

In some embodiments, the display device may determine the current scene type of the display device through the attributes of the currently displayed interface or the identifier of the currently running application, as well as a preset mapping relationship. The preset relationship may be a mapping relationship between the attribute of the currently displayed interface or the identifier of the currently running application, and the scene type.

In some embodiments, the attributes of the currently displayed interface may be marked in advance, or may be determined in real time by means of image recognition.

The media asset playing scene may be a scene in which the display device plays the media asset. When the display device is in the media playing scene, the user can watch the media resource, and can also adjust the playback progress of the media resource, the current volume of the display device, and so on.

The media resource search scene may be a scene in which a display device displays a media resource search page. Fig. 16b shows a schematic diagram of a media asset search scenario in some embodiments. When the display device is in the media asset search scene, an input method keyboard can be displayed in the user interface, and the user can input the name of the media asset through the input method keyboard, thereby controlling the display device to implement the media asset search function.

In order to enable the user to better control the user interface under the current application scenario type, the application scenario control interface displayed on the terminal device may also include some relevant information displayed on the current user interface. In the embodiment of the present application, the application scene data is used to refer to the data corresponding to the user interface under the current application scene type of the display device. The terminal device can generate an application scenario control interface according to the application scenario type and the application scenario data, so that the user can better control the user interface.

In some embodiments, after receiving the application scenario control instruction sent by the terminal device, the display device may directly send the application scenario type and application scenario data to the terminal device, so that the terminal device generates an application scenario control interface.

Specifically, the display device may detect the type of the application scenario to which the user interface belongs, and acquire data corresponding to the application scenario. Furthermore, the display device can send these data to the terminal device, and the terminal device can display the application scene control interface according to these data, and the user can use the application scene control interface to control the user interface in the current application scene of the display device.

In some embodiments, when detecting the current application scene, the display device may first detect the application program corresponding to the application scene to which the user interface belongs, that is, the application program running on the display device corresponding to the user interface. After the application program is determined, the controller may send an application scene information acquisition instruction to the application program. After receiving the instruction, the application program can feed back application scenario information to the controller, including the application scenario type and application scenario data corresponding to the current application scenario.

Figure 17 shows an interaction diagram of components in a display device in some embodiments. For example, the display device is running the Jukanan APP, and media assets are displayed on the display. At this time, Jukanan APP can store the current application scenario information of the display device, including the application scenario type and application scenario data, and the application scenario data is some relevant data of the currently displayed media assets. In step S1710, the terminal device sends an application scene control instruction to the controller. After receiving the application scene control instruction sent by the terminal device, in step S1720, the controller may directly send an application scene information acquisition instruction to an application program such as Jukankan APP. In step 1730, Jukanan APP can determine the application scenario information, and in step S1740, can directly feed back the stored application scenario information to the display device (such as the controller of the display device), and then, in step S1750, the display device will apply The scene information is sent to the terminal device.

In some embodiments, after receiving the application scene control instruction sent by the terminal device, the display device may first detect the application scene type, and send the application scene type to the terminal device.

After confirming the application scenario type of the display device, the terminal device may send a request for acquiring application scenario data to the display device. Further, the display device can acquire application scene data, and send the application scene data to the terminal device. The terminal device can generate an application scenario control interface according to the application scenario type and the application scenario data.

It should be noted that, for a display device, no matter what the current application scenario type of the display device is, the corresponding application scenario data can be obtained by the display device itself. Therefore, after receiving the application scene control command sent by the terminal device, the display device can directly send the application scene type and application scene data to the terminal device; After the scene data is requested, the application scene data is sent to the terminal device. The above two technical solutions are not limited by the types of application scenarios.

In some embodiments, the operation server of the display device can also obtain the application scenario data of the display device. Therefore, the terminal device can also obtain the application scene data of the display device through the server.

However, not all application scenario data corresponding to all application scenario types can be obtained by the server. For example, when the display device is in the media asset playing scene and is playing the media asset, some specific playback data of the media asset, such as playback progress and other information, cannot be obtained by the server. For another example, when the display device is in a media asset search scene, and the user inputs some text content on the display device, the server cannot obtain the text content. Therefore, when the display device is in a media asset playback scenario or a media asset search scenario, the terminal device cannot obtain application scenario data through the server.

When the display device is in a media resource recommendation scene, a media resource recommendation page may be displayed on the display, for example, media resource resources corresponding to various media resource types may be displayed. It should be noted that, the media asset recommendation page of the display device is obtained from the media asset recommendation data acquired from the server. That is, all relevant data in the media asset recommendation page can be obtained from the server. The server may send the media resource recommendation data sent to the display device to the terminal device.

Therefore, after receiving the application scene control instruction sent by the terminal device, the display device can detect the application scene type and send the application scene type to the terminal device. The terminal device can detect the application scenario type. If it is detected that the application scenario type is a media asset recommendation scenario, a request for acquiring application scenario data may be sent to the server. The server may send the application scenario data corresponding to the media asset recommendation scenario, that is, the media asset recommendation data corresponding to the media asset recommendation page of the display device, to the terminal device. The terminal device can generate an application scenario control interface according to the application scenario type and the application scenario data.

In some embodiments, the terminal device may generate an application scenario control interface according to a preset control interface template library. Specifically, for each application scenario, a control interface template and its correspondence can be set in advance. That is, each application scenario will have its corresponding control interface template. All control interface templates and the corresponding relationship between each control interface template and application scenarios can be stored in the control interface template library. At this point, in the control interface template library, the corresponding target control interface template can be obtained according to the application scenario type. Further, the application scenario control interface can be generated according to the target control interface template and the application scenario data.

In some embodiments, when the display device is in the media asset recommendation scene, the display device may directly send the current application scene type and application scene data to the terminal device. The application scenario data may be media resource recommendation data, that is, information about media resources contained in the current media resource recommendation page. After receiving the application scenario type and application scenario data sent by the display device, the terminal device can determine that the current application scenario of the display device is a media asset recommendation scenario, and further, can generate an application scenario control interface according to the application scenario type and application scenario data.

In some embodiments, when the target control corresponds to the type of media assets, for example, when the target control is "fashion". The controller may acquire a media asset recommendation page corresponding to the media asset type "fashion". For example, for a display device, when the user clicks on the "Ancient Costume" control in the media asset recommendation scene shown in Figure 18a, the controller may control the display to display a media asset recommendation page corresponding to "Ancient Costume". Fig. 18a shows a schematic diagram of a display device displaying a media asset recommendation page in some embodiments. The media asset recommendation page may include several media asset resources corresponding to the "fashion" media asset type. The controller may send the media asset recommendation page corresponding to the "fashion" media asset type to the terminal device.

Further, the terminal device may display a control interface, and the control interface may be a media asset recommendation page corresponding to the "fashion" media asset type. Fig. 18b shows a schematic diagram of a terminal device displaying a media asset recommendation page in some embodiments. The user can select the media resource in the media resource recommendation page.

The control interface is also provided with a control interface return control 1501 for controlling the terminal device to switch the control interface to the initial control interface.

Fig. 19 shows a schematic diagram of a terminal device displaying an application scenario control interface in some embodiments, and the application scenario type corresponding to the application scenario control interface is a media asset recommendation scenario. At this time, the terminal device may display some media resource information on the media resource recommendation page of the display device. The user can manipulate the control interface, such as selecting a specific media resource. After the terminal device detects the user's operation, it can determine the target media asset selected by the user, and send the target media asset to the display device. Further, the display device can play the target media assets. The application scene control interface is also provided with an interface return control 1501 for switching the control interface displayed on the terminal device. When it is detected that the user clicks the interface to return to the control 1501, or the user touches the interface to return to the control 1501, for example, the user can press and hold the control and perform a downward operation, and the terminal device can switch the control interface to the initial control interface. At this time, The terminal device no longer displays the application scene control interface, but updates and displays the initial control interface, so that the user can input control instructions in the initial control interface.

In some embodiments, when detecting that the display device enters a media playing scene, the controller may send the application scene type and application scene data to the terminal device. After receiving the application scenario type and application scenario data, the terminal device can determine that the current application scenario of the display device is a media asset playback scenario, and display an application scenario control interface for controlling the media asset playback scenario. Wherein, the application scene data may include media resource information of the media resource currently played by the display device.

The media asset information may include: the name of the media asset, the total duration of the media asset, and the playback progress of the media asset. For example, the total duration of the media asset is 1 hour, 50 minutes, and 20 seconds, and the current playback progress is 1 hour, 20 minutes, and 20 seconds. Media asset information may include media asset anthology and media asset clarity. Among them, the collection of episodes can include the total number of episodes of the media resource and the nth episode of the TV series where the current media resource is, and the resolution of the media resource refers to all the resolutions supported by the media resource and the current resolution, for example, the media resource It can support Blu-ray, ultra-clear, high-definition and standard-definition four definition, the definition currently playing is ultra-clear. The media asset information may also include the media asset playback rate, that is, the playback rate supported by the media asset resource, the current playback rate, and the like. Specifically, the controller may obtain the media asset information from the application program corresponding to the user interface.

After the display device sends the application scenario type and application scenario data to the terminal device, the terminal device can generate the application scenario control interface, specifically, the application scenario control interface can be generated by using the preset control interface template library.

Fig. 20 shows a schematic diagram of a terminal device displaying an application scene control interface in some embodiments. The application scene control interface can control the media resources played by the display device. Specifically, the display area of the application scene control interface can be provided with an interface return control 1501, a mute control 1601, a media asset playback progress area 1610, a media asset collection area 1620, a media asset definition area 1630, a playback control area 1640, and a media asset playback area 1640. Play rate field 1650 and media asset name field 1660 .

Wherein, the interface return control 1501 is used to control the terminal device to switch the control interface to the initial control interface. When the user clicks the mute control 1601, the terminal device may send a mute instruction to the display device, thereby controlling the display device to perform mute processing. The media resource playing progress area 1610 may be a progress bar of the media resource, which is used to adjust the playing progress of the media resource. The user can check the total duration of the media resource and the current playback progress of the media asset through the media asset playback progress area 1610, and can also touch the media asset playback progress area 1610, for example, press and hold the playback progress control 1602 and move it horizontally. The playing progress corresponding to the moved playing progress control 1602 can be determined, and the playing progress can be sent to the display device, and the controller can adjust the media resource according to the playing progress. After adjusting the playing progress of the media resources, the controller can also control the display to display the current progress bar to prompt the user of the current playing progress, as shown in FIG. 17a. After a preset period, for example, after 5 seconds, the controller controls the display to no longer display the progress bar. At this time, only media resources are displayed on the display, which will not affect the viewing effect of the user, so as to improve the experience of the user.

The media asset selection area 1620 is used to switch the number of episodes of the media resource. The user can view the current episode of the media resource and several adjacent episodes. The user can click on other episodes to control the display device to play Media assets corresponding to the number of episodes. The media resource resolution area 1630 is used to switch the resolution of the media resource. The user can view all the resolutions supported by the media resource and the resolution currently playing. The user can click on other resolutions to control the display device Play the media resource with the corresponding definition. The media asset playback speed area 1650 is used to switch the playback speed of the media assets. The user can select one of 0.8x, 1.0x, 1.25x, 1.5x or 2.0x speed, and control the display device to play the media at this speed resource.

The playback control area 1640 can further control the display device, and control the playback speed and volume of the media resources. Specifically, the playback control area 1640 may include a first area, a second area and a third area. Wherein, the first area is used to adjust the media resources to before the preset T1 time, for example, when the user clicks on the first area, the terminal device can send a rewind instruction to the display device, and the rewind instruction is used to rewind the media resources Go back 15s. The second area is used to adjust the media resources to after the preset T2 time. For example, when the user clicks on the second area, the terminal device can send a fast-forward command to the display device, and the fast-forward command is used to fast-forward the media resources. 15s. The T1 time and the T2 time may be the same or different, which is not limited in this embodiment of the present application.

In the third area, the playback speed and volume of the media resources can be controlled. When the user clicks on the third area, the terminal device may send a playback pause instruction to the display device, and when receiving the playback pause instruction, the controller controls the display device to temporarily stop playing the media resource. The controller may also control the display to display a pause sign, as shown in FIG. 21b. When the user clicks on the third area again, the terminal device can send a resuming playback instruction to the display device, and when receiving the resuming playback instruction, the controller controls the display device to continue playing the media resource. When the user keeps touching the third area for a preset period, for example, when the user presses and holds the third area for 1 second, the terminal device can send a double-speed playback instruction to the display device, and the double-speed playback instruction is used to control the display device to play at the preset double speed Media resources, such as 2.0x speed. When the display device plays the media resource at the preset double speed, the controller can also control the display to display double speed prompt information, for example, "playing at 2.0x speed". Fig. 17c shows a schematic diagram of double speed prompt information in some embodiments. When the user no longer touches the third area, the terminal device sends a recovery command to the display device to control the display device to play media resources at a normal double speed, ie, 1.0 times speed. When it is detected that the user performs an up-swipe operation or a down-swipe operation in the third area, the terminal device may respectively send a volume increase instruction and a volume decrease instruction to the display device, so as to adjust the current playback volume of the display device.

The terminal device can control the third area to display control command prompt information, so as to prompt the user of the control commands supported by the current control interface. The media asset name area can display the media asset name of the media asset resource. The terminal device can adjust the display area in the target control interface template according to the application scenario data, so that the data in the display area is the application scenario data.

In some embodiments, when the display device is in a media asset search scene, the display device may send the application scene type and application scene data to the terminal device. After receiving the application scenario type and application scenario data, the terminal device can determine that the current application scenario of the display device is a media asset search scenario, and further, the terminal device can display an application scenario control interface for controlling the media asset search scenario.

In some embodiments, the application scenario data may include historical search records of the display device, search text entered by the user in the text search box, and the like. After receiving the application scenario type and the application scenario data, the terminal device can generate the application scenario control interface, specifically, the application scenario control interface can be generated using a preset control interface template library. Fig. 22 shows a schematic diagram of a terminal device displaying an application scenario control interface in some embodiments. An interface return control 1501 is set in the display area of the application scene control interface, and is used to control the terminal device to switch the control interface to the initial control interface. A search area 1810 , a history search area 1820 and an input method keyboard 1830 are also set in the display area. Users can input the content they want to search in the search area, and can directly click the input method keyboard to input characters, which improves the user experience.

The terminal device can adjust the display area in the target control interface template according to the application scenario data. Specifically, the search text entered by the user on the display device can be displayed in the search area, and the historical search data in the display device can be displayed in the historical search area.

In some embodiments, the display device can display pictures, and at this time the display device is in a picture scene. Fig. 23 shows a schematic diagram of a display device in a picture scene in some embodiments. The controller can control the display to display thumbnails of pictures, and when the user selects a thumbnail, the controller can control the display to display the target picture in full screen.

In some embodiments, when it is detected that the display device is in a picture scene, the controller may send the application scene type and application scene data to the terminal device. After receiving the application scene type and application scene data, the terminal device can determine that the current application scene of the display device is a picture scene, and further, the terminal device can display an application scene control interface for controlling the picture scene. The application scene data may include picture information of all pictures currently displayable by the display device.

In some embodiments, after receiving the application scenario type and the application scenario data, the terminal device can generate the application scenario control interface, specifically, the application scenario control interface can be generated using a preset control interface template library. The application scene control interface can control the pictures displayed by the display device, such as performing zooming and switching operations on the pictures, and so on.

Fig. 24 shows a schematic diagram of a terminal device displaying an application scenario control interface in some embodiments. The control interface includes a thumbnail area 2010 and a control area 2020 . There can be several thumbnails in the thumbnail area, including the target picture currently displayed on the display device and the pictures adjacent to the target picture. The user can slide left and right on the thumbnail area to select other pictures, the terminal device sends the picture selected by the user to the display device, and the controller controls the display to display the selected picture in full screen, thereby realizing the picture switching function. When switching the picture displayed by the display device, the controller can also control the display to display picture switching information. There may be several thumbnails in the picture switching information, including the target picture currently displayed on the display device and pictures adjacent to the target picture, as shown in FIG. 25a , to remind the user of the current picture situation.

The user can perform operations such as zooming in and out of the picture displayed on the display device through the control area. When the user sends a continuous clicking instruction to the terminal device, it is considered that the user has sent a picture enlargement instruction. The continuous click instruction refers to: within a preset period, the number of times the user clicks on the same area of the touch screen of the mobile phone exceeds a preset threshold. For example: when the user taps a certain area of the touch screen of the mobile phone 3 times within 1 second, it is regarded as a continuous tap command. After receiving the continuous click instruction, the terminal device may send an image enlargement instruction to the display device, so that the display device can enlarge and display the image. The controller can control the display to display the magnification of the picture, as shown in FIG. 25b. The picture zoom-in situation can be placed on the upper layer of the enlarged picture displayed on the monitor. The picture zoom-in situation includes the thumbnail of the target picture and the partial zoom-in frame in the thumbnail. The image in the partial zoom-in frame is the zoomed-in picture currently displayed on the monitor. .

When the display device zooms in on the picture, the user can touch the control area, for example, press and hold the control area and slide it, so as to control the partial zoom frame in the picture zoom-in situation. For example, when the user slides to the left in the control area, the terminal device sends a control command to the display device to control the partial zoom frame to move to the left relative to the thumbnail of the target picture. Further, the controller determines the content in the moved partial magnification frame, and controls the display to display the content in the moved partial magnification frame in full screen. When the display device zooms in on the picture, the user can click the control area for a preset number of times, for example, once, and the terminal device can send an instruction to cancel zooming in to the display device. After receiving the cancel zoom instruction, the controller cancels the zoom display of the target picture, and controls the display to display the target picture in full screen.

In some embodiments, when the user touches the control area with two fingers and moves toward the edge of the control area, it can be considered that the user has sent a picture zoom-in command, and the terminal device can send the picture zoom-in command to the display device, so that the display device can zoom in on the picture. to zoom in. When the user touches the control area with two fingers and moves toward the center of the control area, it can be considered that the user has sent a picture shrinking command, and the terminal device can send the picture shrinking command to the display device, so that the display device can zoom out and display the picture.

In some embodiments, the terminal device may determine whether the display device performs enlarged display processing on the target picture. If the display device zooms in on the target picture, when the user presses and slides the control area, the movement of the partial zoom-in frame can be controlled. If the display device does not enlarge and display the target picture, when the user presses and slides the control area, the terminal device sends a picture switching instruction to the display device. For example, when the user slides to the left in the control area, the terminal device may instruct the display device to display the previous picture, and when the user slides to the right in the control area, the terminal device may instruct the display device to display the next picture. The control command prompt information may be displayed in the control area, as shown in FIG. 24 , so as to prompt the user of the control commands supported by the current control interface.

In some embodiments, when the terminal device displays the application scene control interface, the user can use the application scene control interface to control the user interface in the display device. It should be noted that the display device may also be controlled by using a remote controller or directly touching the display device.

When the user uses other control methods to change the application scene of the display device, the application scene control interface displayed on the terminal device at this time obviously does not match the current application scene of the display device, resulting in the user being unable to use the application currently displayed on the terminal device. The scene control interface controls the display device.

At this point, the user can first touch the interface to return to the control, and control the terminal device to display the initial control interface. Then, by touching the application scene control control in the initial control interface, the terminal device regenerates and displays the application scene control interface, so as to update the application scene control interface displayed in the terminal device. At this time, the application scene control interface matches the application scene of the display device, and the display device can be controlled.

In some embodiments, if the application scene of the display device changes, if the user manually updates the application scene control interface, the user's experience will be poor. The display device can detect in real time the application scenario of the display device, that is, whether the application scenario to which the user interface belongs has changed. When it is detected that the application scene changes, the display device may reacquire the changed application scene information, that is, detect the changed application scene type and obtain the changed application scene data.

Further, the display device may actively send the changed application scenario type and the changed application scenario data to the terminal device. After receiving the above information, the terminal device can update and display the application scene control interface, so that the user can continue to use the terminal device to control the display device.

Specifically, after receiving the changed application scenario type and the changed application scenario data sent by the display device, the terminal device can use the preset control interface template library to obtain the control interface template corresponding to the changed application scenario type, And generate a new application scenario control interface according to the control interface template and the changed application scenario data. Further, the terminal device may cause the display unit to update and display the new application scene control interface. At this point, the user can use the application scene control interface to control the display device.

In some embodiments, when the display device is in a media asset playing scene, the terminal device may display an application scene control interface as shown in FIG. 20 . It should be noted that since the application scene control interface includes a media asset playback progress area, it is necessary to ensure that the media asset playback progress in this area is consistent with the playback progress in the display device. The display device can continuously feed back the playback progress to the terminal device, so that the playback progress of the two is unified. However, this processing method will lead to excessive information interaction between the display device and the terminal device, which seriously wastes resources.

Therefore, after the terminal device and the display device synchronize the playback progress once, the terminal device can count the time by itself. For example, after the terminal device generates the application scene control interface, it may send a playback progress acquisition request to the display device, so that the display device can feed back the current playback progress to achieve a synchronization. Afterwards, the display device and the terminal device only need to time their own playback progress separately, and do not need to synchronize in real time, thereby greatly reducing the waste of resources.

In some embodiments, when the display device is in the media asset playing scene, when the playing state of the media asset being played in the media asset playing scene changes, the playback progress between the display device and the terminal device will be inconsistent. Synchronize. For example, when the user uses the remote control to pause, replay, and fast-forward the media assets played on the display device, or the media assets are buffered due to network reasons, the playback progress on the display device is obviously no longer consistent with that of the terminal. The playback progress in the device is synchronized. At this time, the playback progress in the terminal device needs to be changed.

Specifically, the display device can detect the playing status of the media assets in the media assets playing scene in real time. When it is detected that the playing state of the media asset has changed, the playing progress of the media asset after the playing state has changed can be obtained, and the changed playing state can also be determined at the same time. Further, the display device may send the changed playing progress and the changed playing state to the terminal device together. After receiving the above information, the terminal device can adjust the application scene control interface according to the information, specifically, adjust the media asset playback progress area, so that the playback progress in the terminal device and the display device are synchronized.

For example, when the media asset in the display device is paused, the display device will send the target playback progress and the pause status to the terminal device, and the terminal device can keep the playback progress at the target playback progress, and pause synchronously, waiting for the display device to download. information sent.

When the media asset in the display device is fast forwarded or rewinded, the display device can send the target playback progress and the current playback rate to the terminal device, and the terminal device will count at the same playback rate at the target playback progress, so as to Display devices are kept in sync.

The control command prompt information may be displayed in the control area, as shown in FIG. 24 , so as to prompt the user of the control commands supported by the current control interface.

In some embodiments, when the user selects the "screen projection" control in the application panel, the display device can detect whether there is a terminal device connected to itself for screen projection, and if not, display a prompt message for screen projection. Fig. 26 shows a schematic diagram of screen casting prompt information in some embodiments. Users can use terminal devices and display devices to perform screen projection connections. For example, the terminal device may be installed with a "cast screen" control, and the user may click the control to instruct the terminal device to confirm all display devices that can be connected to the screen, and select a display device for screen projection connection. After the terminal device and the display device are connected for screen projection, the display device can realize screen projection, and the display device is in the screen projection scene at this time.

In some embodiments, when it is detected that the display device enters the screen projection scene, the controller may send application scene information to the terminal device, where the application scene information is used to indicate that the display device is currently in the screen projection scene. After receiving the application scene information, the terminal device may display a screen projection control interface corresponding to the screen projection scene.

In some embodiments, when the terminal device determines according to the application scene information that the display device is currently in a screen projection scene, the terminal device may directly display a preset control interface. The preset control interface may be preset by a display device designer. Fig. 27 shows a schematic diagram of a control interface of a terminal device in some embodiments. The control interface can control the screen projection of the display device. Specifically, the control interface may be provided with a control interface return control 1501 , a screen projection control 2401 , a mute control 2402 , a playback progress area 2410 , a playback control area 2420 , a playback speed area 2430 , and a sharpness area 2440 .

Wherein, the control interface return control 1501 is used to control the terminal device to switch the control interface to the initial control interface. When the user clicks the screen projection control 2401, the terminal device may send a cancel screen projection instruction to the display device, thereby controlling the display device to cancel the screen projection operation. The mute control 2402 may control the display device to perform mute processing. The play progress area 2410 may be a progress bar of the media resource, which is used to adjust the play progress of the media resource. The playback control area 2420 controls the playback speed and volume of the media resources, and can also control the media resources to perform pause processing and so on. The play speed area 2430 is used to switch the play speed of the media resource. The sharpness area 2440 is used to switch the sharpness of the media resources.

The terminal device may control the display of control instruction prompt information in the playback control area 2420 , as shown in FIG. 27 , so as to prompt the user of the control instructions supported by the current control interface.

In some embodiments, when the terminal device displays the initial control interface as shown in FIG. 14 , if the user clicks or touches the control interface switching control 1101, the terminal device may display the control interface, as shown in FIG. 28 . The control interface includes several control interfaces supported by the terminal device, including: a media asset recommendation control interface, a media asset playback control interface, a media asset search control interface, a picture control interface, and a picture control interface. The user can select one of the control interfaces, and the terminal device further switches the initial control interface to the target control interface selected by the user, and the user can use the target control interface to control the display device.

In some embodiments, the terminal device may be provided with application scene controls. After the user clicks the application scene control, the terminal device can display the application scene situation, which includes all the application scenarios supported by the display device and the current application scene. Fig. 29 shows a schematic diagram of application scenarios in some embodiments. The user can select one of the application scenarios, and further, the terminal device can send the target application scenario selected by the user to the display device. After receiving the target application scene, the controller can control the display device to enter the target application scene. At the same time, the terminal device can display the control interface corresponding to the target application scenario.

In some embodiments, the display device has a screen splitting function, which can divide the user interface displayed on the display into multiple windows, and each window can display different screen content, that is, the display device has a multi-window mode. In the multi-window mode, the display device can display multiple windows at the same time, so that the user can watch different picture contents on one display device.

In some embodiments, the user can send a multi-window mode instruction to the display device by operating the control device or the terminal device. In the process of actual application, the corresponding relationship between the multi-window mode command and the key of the remote control is bound in advance. For example, set a multi-window mode button on the remote control. When the user touches the button, the remote control sends a multi-window mode command to the controller. At this time, the controller controls the display device to enter the multi-window mode and controls the display to display multiple windows. window. When the user touches the button again, the controller can control the display device to exit the multi-window mode.

In some embodiments, two windows can be displayed in the multi-window mode, or more than two windows can be displayed. Fig. 30 shows a schematic diagram of the screen in the display in the multi-window mode in a feasible embodiment. After the display device enters the multi-window mode, the first window and the second window are simultaneously displayed in the display, wherein the first window and the second window The windows can display different picture content respectively, the first picture is displayed in the first window, and the second picture is displayed in the second window.

It should be noted that the contents of the screens displayed by the two windows are different, and the application scenarios of the two windows may be the same or different. For example, specific media resources may be displayed in the first window, and specific media resources may also be displayed in the second window. At this time, the application scenarios of the two windows are the same, and both are media playback scenarios. It may also be that specific media resources are displayed in the first window, and local pictures are displayed in the second window. At this time, the first window is in the media playing scene, the second window is in the picture scene, and the application scenes of the two windows are different. In this way, different needs of users can be met, and user experience is enhanced.

In some embodiments, when the terminal device displays the initial control interface, when it is detected that the user clicks or touches the control interface switching control 1101, the terminal device may first send an application scene acquisition instruction to the display device.

After receiving the application scene acquiring instruction, the controller may first detect whether the display device is in the multi-window mode. When the display device is in the multi-window mode, the controller can obtain the application scene information corresponding to each window, and can send the application scene information of all windows to the terminal device.

After receiving the application scene information of all windows sent by the display device, the terminal device can display the multi-window application scene situation. FIG. 31 shows a schematic diagram of the multi-window application scene situation in some embodiments. According to the multi-window application scenario, the user can know how many windows are currently displayed on the display device and the application scenario of each window. The user can select one of the windows. After determining the target window selected by the user, the terminal device may display a control interface corresponding to the application scene where the target window is located. The user can control the target window of the display device through the control interface.

The embodiment of the present application also provides a method for controlling a display device, which is applied to a display device. The method includes: responding to a control instruction sent by a terminal device, controlling the display device to implement a function corresponding to the control instruction.

The embodiment of the present application also provides a method for controlling a display device, which is applied to a terminal device, and the method includes:

A control interface is displayed, and the control interface is used to control the display device. In response to the user's operation on the control interface, the control instruction is sent to the display device, so that the display device realizes the function corresponding to the real-time control instruction.

The embodiment of the present application also provides a method for controlling a display device. As shown in FIG. 32 , in step S3210, the display device (for example, on a display) displays a user interface. In step S3220, the display device (for example, through the controller) sends the application scenario information of the user interface to the terminal device. In step S3230, the terminal device displays a control interface corresponding to the application scenario information according to the application scenario information. In response to the user's operation on the control interface, in step S3240, the terminal device sends a control instruction to the display device. Finally, in step S3250, in response to the control instruction sent by the terminal device, the display device implements the function corresponding to the control instruction.

For convenience of explanation, the above description has been made in conjunction with specific implementation manners. However, the above exemplary discussion is not intended to be exhaustive or to limit the implementations to the precise forms disclosed above. Many modifications and variations are possible in light of the above teachings. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various modified embodiments suitable for specific use considerations.

Claims

A terminal device comprising:

Display unit;

a communication unit configured to communicate with the display device;

Processor, configured as:

In response to the input application scene control instruction, sending the application scene control instruction to the display device, so that the display device feeds back the application scene type, and the application scene type belongs to the user interface displayed on the display device Types of application scenarios;

receiving the application scenario type sent by the display device;

Generate an application scene control interface according to the application scene type and control the display unit to display the application scene control interface, the application scene control interface is used to control the user interface in the application scene, wherein different application scene types Corresponding to different application scenarios control interface.
The terminal device according to claim 1, wherein the processor is further configured to:

Before sending the application scene control instruction to the display device,

sending a communication connection instruction to the display device, so that the display device feedbacks the device type; and

An initial control interface is generated according to the device type and the display unit is controlled to display the initial control interface, and the initial control interface is used to control a display device corresponding to the device type.
The terminal device according to claim 2, wherein the initial control interface includes an application scene control control; the processor is further configured to:

Before sending the application scene control instruction to the display device,

When a touch operation on the application scene control control is detected, it is determined that an application scene control instruction is input.
The terminal device according to claim 1, wherein the processor is further configured to:

Before generating the application scenario control interface according to the application scenario type,

receiving application scenario data sent by the display device, where the application scenario data is data corresponding to the user interface under the application scenario type;

The processor is further configured to:

An application scenario control interface is generated according to the application scenario type and the application scenario data.
The terminal device according to claim 1, wherein the application scenario type includes: a media asset recommendation scenario, a media asset search scenario, and a media asset playback scenario;

The processor is further configured to:

When it is detected that the application scenario type is a media asset recommendation scenario, send a request to obtain application scenario data to the display device or server, and the application scenario data is the data corresponding to the user interface under the application scenario type ;

receiving application scenario data sent by the display device or the server; and

Generate an application scenario control interface according to the application scenario type and the application scenario data; or

When it is detected that the application scenario type is a media asset search scenario or a media asset playback scenario, send a request for acquiring application scenario data to the display device;

receiving application scenario data sent by the display device; and

An application scenario control interface is generated according to the application scenario type and the application scenario data.
The terminal device according to claim 5, wherein the processor is further configured to:

The first control interface template corresponding to the application scenario type is obtained from the preset control interface template library, wherein the preset control interface template library stores a plurality of control interface templates and the corresponding Application scenario type;

An application scenario control interface is generated according to the target control interface template and the application scenario data.
The terminal device according to claim 6, wherein the processor is further configured to:

In response to the target application scenario type and the target application scenario data sent by the display device when the application scenario to which the user interface belongs changes, the target application scenario type is acquired from the preset control interface template library The corresponding second control interface template, the target application scenario type is the type of the changed application scenario, and the target application scenario data is the data corresponding to the changed application scenario;

generating a target application scenario control interface according to the second control interface template and the target application scenario data;

and controlling the display unit to update and display the target application scene control interface.
The terminal device according to claim 6, wherein the application scenario type is a media asset playback scenario; the application scenario data includes media asset name, media asset selection, media asset playback progress, media asset clarity, and media asset playback rate;

The processor is further configured to:

According to the application scenario data, the display area in the target control interface template is adjusted, so that the data in the display area is the application scenario data, and the display area includes: media asset name area, media asset selection area, media asset playback progress area, media asset clarity area, and media asset playback rate area;

In response to the target play progress and the changed play state sent by the display device when the play state of the media asset in the media asset play scene changes, the media asset play progress area is adjusted; the target play The progress is the playing progress after the playing state of the media asset changes.
The terminal device according to claim 6, wherein the application scenario type is a media asset search scenario; the application scenario data includes search text and historical search data in the text search box of the display device;

The processor is further configured to:

Adjust the display area in the target control interface template according to the application scenario data, so that the search text is displayed in the search area, and the historical search data is displayed in the historical search area, and the display area includes a search area and a search history area. History search area.
A display device comprising:

a display configured to display a user interface;

a communicator configured to communicate with a terminal device;

Controller, configured as:

Detecting that the display device and the terminal device establish a communication connection, and responding to the application scene control instruction sent by the terminal device, detecting an application scene type, where the application scene type is the type of the application scene to which the user interface belongs;

Sending the application scenario type to the terminal device, so that the terminal device generates and displays an application scenario control interface according to the application scenario type, and the service scenario control interface is used to control the user interface in the application scenario .