CN116647721A

CN116647721A - Display equipment and application pairing method

Info

Publication number: CN116647721A
Application number: CN202210137419.6A
Authority: CN
Inventors: 朱子鸣; 商潮
Original assignee: Vidaa Netherlands International Holdings BV
Current assignee: Vidaa Netherlands International Holdings BV
Priority date: 2022-02-15
Filing date: 2022-02-15
Publication date: 2023-08-25

Abstract

The application discloses a display device and an application pairing method. When the communication terminal plays media assets through the first application, receiving an application starting instruction sent by the communication terminal to the display equipment; the method comprises the steps that an application starting instruction is used for starting a second application of a display device end, and the first application is an application used for playing media in a communication terminal; the second application is used for playing the media resource sent by the first application through the screen throwing connection; judging whether a second application exists in the display equipment or not after receiving an application starting instruction; if the second application exists in the display device, starting the second application in response to an application starting instruction; and if the second application does not exist in the display device, downloading the second application. The technical scheme can solve the problem that the connection failure can only be prompted under the condition that the pairing between the communication terminal and the display equipment in the existing screen-throwing scene fails, and provides an application downloading method in time to play the media asset under the condition that the display equipment does not have the corresponding application playing media asset.

Description

Display equipment and application pairing method

Technical Field

The present application relates to the field of display devices, and in particular, to a display device and an application pairing method.

Background

The display device refers to a terminal device capable of outputting a specific display screen, and may be a terminal device such as a smart television, a communication terminal, a smart advertisement screen, and a projector. Taking intelligent electricity as an example, the intelligent television is based on the Internet application technology, has an open operating system and a chip, has an open application platform, can realize a bidirectional man-machine interaction function, and is a television product integrating multiple functions of video, entertainment, data and the like, and the intelligent television is used for meeting the diversified and personalized requirements of users.

The display device can also establish a screen-throwing connection relation with the communication terminal through the wireless local area network, namely, the display device can receive screen-throwing data shared by the communication terminal and display the received screen-throwing data in real time, namely, display a current picture of the communication terminal or a media picture played on the communication terminal. However, on the existing Linux system platform, when the communication terminal projects a screen to the display device through a certain media application, encryption information and a control instruction are sent to the display device, and decryption and execution can be performed only when the same media application exists in the display device, so that the screen projection between the communication terminal and the display device is a point-to-point screen projection mode; when the same media asset application does not exist in the display device, the communication terminal prompts the user of connection failure.

In the existing application pairing process, when the screen projection of the communication terminal to the display device fails, the connection failure is only prompted by the communication terminal, so that a user does not know where the problem exists and cannot solve the problem.

Disclosure of Invention

The application provides a display device and an application pairing method, which can solve the problem that connection failure can only be prompted under the condition that the pairing between a communication terminal and the display device fails in the existing screen-throwing scene, and provide an application downloading method in time to play media resources under the condition that the display device end does not have corresponding application play media resources.

In a first aspect, the present application shows a display device comprising: a display; a communicator configured to establish a screen-drop connection with a communication terminal; a controller configured to: when the communication terminal plays media assets through the first application, receiving an application starting instruction sent by the communication terminal to the display equipment; the method comprises the steps that an application starting instruction is used for starting a second application of a display device end, and the first application is an application used for playing media in a communication terminal; the second application is an application corresponding to the first application in the display equipment and is used for playing media resources sent by the first application through the screen projection connection; judging whether a second application exists in the display equipment or not after receiving an application starting instruction; if the second application exists in the display device, starting the second application in response to an application starting instruction; and if the second application does not exist in the display device, downloading the second application. By adopting the embodiment of the application, the problem that the connection failure can only be prompted under the condition that the pairing between the communication terminal and the display equipment in the existing screen-throwing scene fails can be solved, and a downloading scheme is provided for a user in time so as to play media resources, so that better interaction experience is provided for the user.

In some embodiments, the step of receiving an application start instruction sent by the communication terminal to the display device is further configured to: receiving and analyzing an application starting instruction sent by a communication terminal through a dial server; the dialing server is used for providing equipment discovery service; the device discovery service is a service for causing the communication terminal to discover the display device based on the SSDP protocol; after the dialing server analyzes the application starting instruction sent by the communication terminal, checking whether the second application is legal or not through the dialing server; and if the second application is legal, executing the step of judging whether the second application exists in the display device. By adopting the embodiment, the connection between the display device and the communication terminal can be established, and the second application is judged in the connection establishment process, so that the validity of the second application is ensured.

In some embodiments, when the second application is not present in the display device, the controller is further configured to: and the control dial server sends a first prompt to the communication terminal according to the SSDP protocol so that a display interface of the communication terminal displays the first prompt, and the first prompt is used for prompting a user that the second application is not installed in the display equipment. By adopting the embodiment, the user can be prompted to lack the second application corresponding to the communication terminal at the display equipment end, and the user is required to install the second application.

In some embodiments, the controller performs the step of downloading the second application, and is further configured to: determining whether a second application exists in an application store of the display device when the second application does not exist in the display device; if the second application exists in the application store of the display device, sending an application download address of the second application to the dial-up server; and when the dial-up server receives the application download address of the second application, responding to a download instruction of the user to download the second application. By adopting the embodiment, the user can be automatically prompted to download the second application, so that the problem that the user cannot know how to process when the screen projection fails is solved.

In some embodiments, when the dial-up server receives the application download address of the second application, the controller is further configured to: the display interface of the display device is controlled to display a second prompt, the second prompt being used to characterize that the second application is not installed on the display device and that the second application is downloadable from an application store of the display device. With the present embodiment, the user can be prompted that the second application should be installed from the application store of the display device.

In some embodiments, when the second application is not present in the display device, the controller is further configured to: controlling a display interface of the display device to display a third prompt, wherein the third prompt is used for recommending the associated application of the second application to the user; the third prompt comprises a link for downloading the related application; when the user clicks on a link to download the associated application, the associated application is downloaded. By adopting the embodiment, the user can be recommended to download the associated application when the second application does not exist in the display device.

In some embodiments, when the second application is not present in the display device, the controller is further configured to: controlling a display interface of the display device to display a fourth prompt, wherein the fourth prompt is used for providing a search function for a user to search media resources being played by the communication terminal; and when the user searches the media asset through the searching function, playing the media asset. By adopting the embodiment, the user can search the media assets from other ways to achieve the same viewing effect.

In some embodiments, when the second application is not present in the display device, the controller is further configured to: receiving an application starting address of a second application sent by the communication terminal through the dial server; the application starting addresses of the first application and the second application are the same; when the dialing server receives the application starting address, the dialing server is controlled to load the media asset through the browser according to the application starting address so as to play the media asset. By adopting the embodiment, the media assets can be played by applying the starting address.

In a second aspect, the present application also shows an application pairing method, the method comprising: when the communication terminal plays media assets through the first application, receiving an application starting instruction sent by the communication terminal to the display equipment; the method comprises the steps that an application starting instruction is used for starting a second application of a display device end, and the first application is an application used for playing media in a communication terminal; the second application is an application corresponding to the first application in the display equipment and is used for playing media resources sent by the first application through the screen projection connection; judging whether a second application exists in the display equipment or not after receiving an application starting instruction; if the second application exists in the display device, starting the second application in response to an application starting instruction; and if the second application does not exist in the display device, downloading the second application.

In some embodiments, receiving an application starting instruction sent by a communication terminal to a display device includes: receiving and analyzing an application starting instruction sent by a communication terminal through a dial server; the dialing server is used for providing equipment discovery service; the device discovery service is a service for causing the communication terminal to discover the display device based on the SSDP protocol; after the dialing server analyzes the application starting instruction sent by the communication terminal, checking whether the second application is legal or not through the dialing server; and if the second application is legal, executing the step of judging whether the second application exists in the display device.

According to the technical scheme, under the condition that the display equipment end does not have the corresponding application playing media assets, the application downloading method can be provided timely to play the media assets, in various embodiments, various media asset playing modes are provided, so that a user can finally watch the media assets to be watched through various ways, and better interactive experience can be provided for the user.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described below, and it will be obvious to those skilled in the art that other drawings can be obtained from these drawings without inventive effort.

FIG. 1 illustrates a schematic diagram of an operational scenario between a display device and a control apparatus according to some embodiments;

fig. 2 shows a block diagram of a configuration of a control device 100 according to some embodiments;

fig. 3 illustrates a hardware configuration block diagram of a display device 200 according to some embodiments;

FIG. 4 illustrates a software configuration block diagram of a display device 200 according to some embodiments;

fig. 5 illustrates a hardware architecture diagram of a communication terminal 300 according to some embodiments;

fig. 6 shows a software architecture block diagram of a communication terminal 300 according to some embodiments;

FIG. 7 illustrates a user opening a media playback application scenario, in accordance with some embodiments;

FIG. 8 illustrates a schematic diagram of a setup screen connect scenario, according to some embodiments;

FIG. 9 illustrates a user send drop connection request scenario diagram according to some embodiments;

FIG. 10 illustrates a media asset application drop scene schematic in accordance with some embodiments;

FIG. 11 illustrates a display device configuration flow diagram in accordance with some embodiments;

FIG. 12 illustrates a first prompt and second prompt display interface schematic in accordance with some embodiments;

FIG. 13 illustrates a first prompt and third prompt display interface schematic in accordance with some embodiments;

FIG. 14 illustrates a first prompt, a third prompt, and a fourth prompt display interface schematic in accordance with some embodiments.

Detailed Description

For the purposes of making the objects and embodiments of the present application more apparent, an exemplary embodiment of the present application will be described in detail below with reference to the accompanying drawings in which exemplary embodiments of the present application are illustrated, it being apparent that the exemplary embodiments described are only some, but not all, of the embodiments of the present application.

It should be noted that the brief description of the terminology in the present application is for the purpose of facilitating understanding of the embodiments described below only and is not intended to limit the embodiments of the present application. Unless otherwise indicated, these terms should be construed in their ordinary and customary meaning.

The terms first, second, third and the like in the description and in the claims and in the above-described figures are used for distinguishing between similar or similar objects or entities and not necessarily for describing a particular sequential or chronological order, unless otherwise indicated. It is to be understood that the terms so used are interchangeable under appropriate circumstances.

The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a product or apparatus that comprises a list of elements is not necessarily limited to all elements explicitly listed, but may include other elements not expressly listed or inherent to such product or apparatus.

The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware or/and software code that is capable of performing the function associated with that element.

Fig. 1 is a schematic diagram of an operation scenario between a display device and a control apparatus according to an embodiment. As shown in fig. 1, a user may operate the display device 200 through the smart device 300 or the control apparatus 100.

In some embodiments, the control apparatus 100 may be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication or bluetooth protocol communication, and other short-range communication modes, and the display device 200 is controlled by a wireless or wired mode. The user may control the display device 200 by inputting user instructions through keys on a remote control, voice input, control panel input, etc.

In some embodiments, a smart device 300 (e.g., mobile terminal, tablet, computer, notebook, etc.) may also be used to control the display device 200. For example, the display device 200 is controlled using an application running on a smart device.

In some embodiments, the display device 200 may also perform control in a manner other than the control apparatus 100 and the smart device 300, for example, the voice command control of the user may be directly received through a module configured inside the display device 200 device for acquiring voice commands, or the voice command control of the user may be received through a voice control device configured outside the display device 200 device.

In some embodiments, the display device 200 is also in data communication with a server 400. The display device 200 may be permitted to make communication connections via a Local Area Network (LAN), a Wireless Local Area Network (WLAN), and other networks. The server 400 may provide various contents and interactions to the display device 200. The server 400 may be a cluster, or may be multiple clusters, and may include one or more types of servers.

Fig. 2 exemplarily shows a block diagram of a configuration of the control apparatus 100 in accordance with an exemplary embodiment. As shown in fig. 2, the control device 100 includes a controller 110, a communication interface 130, a user input/output interface 140, a memory, and a power supply. The control apparatus 100 may receive an input operation instruction of a user and convert the operation instruction into an instruction recognizable and responsive to the display device 200, and function as an interaction between the user and the display device 200.

Fig. 3 shows a hardware configuration block diagram of the display device 200 in accordance with an exemplary embodiment.

In some embodiments, display apparatus 200 includes at least one of a modem 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, memory, a power supply, a user interface.

In some embodiments the controller includes a processor, a video processor, an audio processor, a graphics processor, RAM, ROM, a first interface for input/output to an nth interface.

In some embodiments, the display 260 includes a display screen component for presenting a picture, and a driving component for driving an image display, for receiving image signals from the controller output, for displaying video content, image content, and a menu manipulation interface, and for manipulating a UI interface by a user.

In some embodiments, the display 260 may be a liquid crystal display, an OLED display, a projection device, and a projection screen.

In some embodiments, communicator 220 is a component for communicating with external devices or servers according to various communication protocol types. For example: the communicator may include at least one of a Wifi module, a bluetooth module, a wired ethernet module, or other network communication protocol chip or a near field communication protocol chip, and an infrared receiver. The display device 200 may establish transmission and reception of control signals and data signals with the external control device 100 or the server 400 through the communicator 220.

In some embodiments, the user interface may be configured to receive control signals from the control device 100 (e.g., an infrared remote control, etc.).

In some embodiments, the detector 230 is used to collect signals of the external environment or interaction with the outside. For example, detector 230 includes a light receiver, a sensor for capturing the intensity of ambient light; alternatively, the detector 230 includes an image collector such as a camera, which may be used to collect external environmental scenes, user attributes, or user interaction gestures, or alternatively, the detector 230 includes a sound collector such as a microphone, or the like, which is used to receive external sounds.

In some embodiments, the external device interface 240 may include, but is not limited to, the following: high Definition Multimedia Interface (HDMI), analog or data high definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, or the like. The input/output interface may be a composite input/output interface formed by a plurality of interfaces.

In some embodiments, the modem 210 receives broadcast television signals via wired or wireless reception and demodulates audio-video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

In some embodiments, the controller 250 and the modem 210 may be located in separate devices, i.e., the modem 210 may also be located in an external device to the main device in which the controller 250 is located, such as an external set-top box or the like.

In some embodiments, the controller 250 controls the operation of the display device and responds to user operations through various software control programs stored on the memory. The controller 250 controls the overall operation of the display apparatus 200. For example: in response to receiving a user command to select a UI object to be displayed on the display 260, the controller 250 may perform an operation related to the object selected by the user command.

In some embodiments, the object may be any one of selectable objects, such as a hyperlink, an icon, or other operable control. The operations related to the selected object are: displaying an operation of connecting to a hyperlink page, a document, an image, or the like, or executing an operation of a program corresponding to the icon.

In some embodiments the controller includes at least one of a central processing unit (Central Processing Unit, CPU), video processor, audio processor, graphics processor (Graphics Processing Unit, GPU), RAM Random Access Memory, RAM), ROM (Read-Only Memory, ROM), first to nth interfaces for input/output, a communication Bus (Bus), and the like.

A CPU processor. For executing operating system and application program instructions stored in the memory, and executing various application programs, data and contents according to various interactive instructions received from the outside, so as to finally display and play various audio and video contents. The CPU processor may include a plurality of processors. Such as one main processor and one or more sub-processors.

In some embodiments, a graphics processor is used to generate various graphical objects, such as: icons, operation menus, user input instruction display graphics, and the like. The graphic processor comprises an arithmetic unit, which is used for receiving various interactive instructions input by a user to operate and displaying various objects according to display attributes; the device also comprises a renderer for rendering various objects obtained based on the arithmetic unit, wherein the rendered objects are used for being displayed on a display.

In some embodiments, the video processor is configured to receive an external video signal, perform video processing such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, image composition, etc., according to a standard codec protocol of an input signal, and may obtain a signal that is displayed or played on the directly displayable device 200.

In some embodiments, the video processor includes a demultiplexing module, a video decoding module, an image synthesis module, a frame rate conversion module, a display formatting module, and the like. The demultiplexing module is used for demultiplexing the input audio and video data stream. And the video decoding module is used for processing the demultiplexed video signal, including decoding, scaling and the like. And an image synthesis module, such as an image synthesizer, for performing superposition mixing processing on the graphic generator and the video image after the scaling processing according to the GUI signal input by the user or generated by the graphic generator, so as to generate an image signal for display. And the frame rate conversion module is used for converting the frame rate of the input video. And the display formatting module is used for converting the received frame rate into a video output signal and changing the video output signal to be in accordance with a display format, such as outputting RGB data signals.

In some embodiments, the audio processor is configured to receive an external audio signal, decompress and decode the audio signal according to a standard codec protocol of an input signal, and perform noise reduction, digital-to-analog conversion, and amplification processing to obtain a sound signal that can be played in a speaker.

In some embodiments, a user may input a user command through a Graphical User Interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the Graphical User Interface (GUI). Alternatively, the user may input the user command by inputting a specific sound or gesture, and the user input interface recognizes the sound or gesture through the sensor to receive the user input command.

In some embodiments, a "user interface" is a media interface for interaction and exchange of information between an application or operating system and a user that enables conversion between an internal form of information and a form acceptable to the user. A commonly used presentation form of the user interface is a graphical user interface (Graphic User Interface, GUI), which refers to a user interface related to computer operations that is displayed in a graphical manner. It may be an interface element such as an icon, a window, a control, etc. displayed in a display screen of the electronic device, where the control may include a visual interface element such as an icon, a button, a menu, a tab, a text box, a dialog box, a status bar, a navigation bar, a Widget, etc.

In some embodiments, a system of display devices may include a Kernel (Kernel), a command parser (shell), a file system, and an application program. The kernel, shell, and file system together form the basic operating system architecture that allows users to manage files, run programs, and use the system. After power-up, the kernel is started, the kernel space is activated, hardware is abstracted, hardware parameters are initialized, virtual memory, a scheduler, signal and inter-process communication (IPC) are operated and maintained. After the kernel is started, shell and user application programs are loaded again. The application program is compiled into machine code after being started to form a process.

As shown in fig. 4, a system of display devices may include a Kernel (Kernel), a command parser (shell), a file system, and an application program. The kernel, shell, and file system together form the basic operating system architecture that allows users to manage files, run programs, and use the system. After power-up, the kernel is started, the kernel space is activated, hardware is abstracted, hardware parameters are initialized, virtual memory, a scheduler, signal and inter-process communication (IPC) are operated and maintained. After the kernel is started, shell and user application programs are loaded again. The application program is compiled into machine code after being started to form a process.

As shown in fig. 4, the system of the display device is divided into three layers, an application layer, a middleware layer, and a hardware layer, from top to bottom.

The application layer mainly comprises common applications on the television, and an application framework (Application Framework), wherein the common applications are mainly applications developed based on Browser, such as: HTML5 APPs; native applications (Native APPs);

the application framework (Application Framework) is a complete program model with all the basic functions required by standard application software, such as: file access, data exchange, and the interface for the use of these functions (toolbar, status column, menu, dialog box).

Native applications (Native APPs) may support online or offline, message pushing, or local resource access.

The middleware layer includes middleware such as various television protocols, multimedia protocols, and system components. The middleware can use basic services (functions) provided by the system software to connect various parts of the application system or different applications on the network, so that the purposes of resource sharing and function sharing can be achieved.

The hardware layer mainly comprises a HAL interface, hardware and a driver, wherein the HAL interface is a unified interface for all the television chips to be docked, and specific logic is realized by each chip. The driving mainly comprises: audio drive, display drive, bluetooth drive, camera drive, WIFI drive, USB drive, HDMI drive, sensor drive (e.g., fingerprint sensor, temperature sensor, pressure sensor, etc.), and power supply drive, etc.

Fig. 5 shows a schematic configuration diagram of the communication terminal 300.

The embodiment will be specifically described below taking the communication terminal 300 as an example. It should be understood that the communication terminal 300 shown in fig. 5 is only one example, and that the communication terminal 300 may have more or fewer components than shown in fig. 5, may combine two or more components, or may have a different configuration of components. The various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application specific integrated circuits.

As shown in fig. 5, the communication terminal 300 includes: radio Frequency (RF) circuitry 310, memory 320, display unit 330, camera 340, sensor 350, audio circuitry 360, wireless fidelity (Wireless Fidelity, wi-Fi) circuitry 370, processor 380, bluetooth circuitry 381, and power supply 390.

The RF circuit 310 may be configured to receive and transmit signals during a message or call, and may receive downlink data from a base station and then transmit the downlink data to the processor 380 for processing; uplink data may be sent to the base station. Typically, RF circuitry includes, but is not limited to, antennas, at least one amplifier, transceivers, couplers, low noise amplifiers, diplexers, and the like.

Memory 320 may be used to store software programs and data. The processor 380 performs various functions of the communication terminal 300 and data processing by running software programs or data stored in the memory 320. Memory 320 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device. The memory 320 stores an operating system enabling the communication terminal 300 to operate. The memory 320 may store an operating system and various application programs, and may also store code for performing the methods described in embodiments of the present application.

The display unit 330 may be used to receive input digital or character information, generate signal inputs related to user settings and function control of the communication terminal 300, and in particular, the display unit 330 may include a touch screen 331 provided at the front of the communication terminal 300, and may collect touch operations on or near the user, such as clicking buttons, dragging scroll boxes, and the like.

The display unit 330 may also be used to display information input by a user or information provided to the user and a graphical user interface (graphical user interface, GUI) of various menus of the communication terminal 300. Specifically, the display unit 330 may include a display 332 provided on the front surface of the communication terminal 300. The display 332 may be configured in the form of a liquid crystal display, light emitting diodes, or the like. The display unit 330 may be used to display various graphical user interfaces described in the present application.

The touch screen 331 may cover the display screen 332, or the touch screen 331 may be integrated with the display screen 332 to implement the input and output functions of the communication terminal 300, and the integrated touch screen may be simply referred to as a touch display screen. The display unit 330 may display the application program and the corresponding operation steps.

The camera 340 may be used to capture still images or video. The object generates an optical image through the lens and projects the optical image onto the photosensitive element. The photosensitive element may be a charge coupled device (charge coupled device, CCD) or a Complementary Metal Oxide Semiconductor (CMOS) phototransistor. The photosensitive elements convert the optical signals to electrical signals, which are then transferred to the processor 380 for conversion to digital image signals.

The communication terminal 300 may further include at least one sensor 350 such as an acceleration sensor 351, a distance sensor 352, a fingerprint sensor 353, a temperature sensor 354. The communication terminal 300 may be further configured with other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, light sensors, motion sensors, and the like.

Audio circuitry 360, speaker 361, microphone 362 can provide an audio interface between a user and communication terminal 300. The audio circuit 360 may transmit the received electrical signal converted from audio data to the speaker 361, and the electrical signal is converted into a sound signal by the speaker 361 and output. The communication terminal 300 may also be configured with a volume button for adjusting the volume of the sound signal. On the other hand, the microphone 362 converts the collected sound signals into electrical signals, which are received by the audio circuit 360 and converted into audio data, which are output to the RF circuit 310 for transmission to, for example, another terminal, or to the memory 320 for further processing. The microphone 362 of the present application can obtain the voice of the user.

Wi-Fi belongs to a short-range wireless transmission technology, and the communication terminal 300 can help a user to send and receive e-mail, browse web pages, access streaming media and the like through the Wi-Fi circuit 370, so that wireless broadband internet access is provided for the user.

The processor 380 is a control center of the communication terminal 300, connects various parts of the entire terminal using various interfaces and lines, and performs various functions of the communication terminal 300 and processes data by running or executing software programs stored in the memory 320 and calling data stored in the memory 320. In some embodiments, processor 380 may include one or more processing units; the processor 380 may also integrate an application processor that primarily processes operating systems, user interfaces, applications, etc., with a baseband processor that primarily processes wireless communications. It will be appreciated that the baseband processor described above may not be integrated into the processor 380. The processor 380 of the present application may run an operating system, an application program, a user interface display, and a touch response, and a processing method according to an embodiment of the present application. In addition, the processor 380 is coupled to the input unit 330 and the display unit 340.

Bluetooth circuit 381 is used for exchanging information with other bluetooth devices having bluetooth circuits through bluetooth protocol. For example, the communication terminal 300 may establish a bluetooth connection with a wearable electronic device (e.g., a smart watch) also provided with a bluetooth circuit through the bluetooth circuit 381, thereby performing data interaction.

The communication terminal 300 also includes a power supply 390 (e.g., a battery) that provides power to the various components. The power supply may be logically connected to the processor 380 through a power management system, so that functions of managing charging, discharging, power consumption, etc. are implemented through the power management system. The communication terminal 300 may also be configured with a power button for powering on and off the terminal, and locking the screen, etc.

Fig. 6 is a software configuration block diagram of the communication terminal 300 according to the embodiment of the present invention.

The layered architecture divides the software into several layers, each with distinct roles and branches. The layers communicate with each other through a software interface. In some embodiments, the Android system is divided into four layers, from top to bottom, an application layer, an application framework layer, an Zhuoyun row (Android run) and system libraries, and a kernel layer, respectively.

The application layer may include a series of application packages.

As shown in fig. 6, the application package may include applications for cameras, gallery, calendar, phone calls, maps, navigation, WLAN, bluetooth, music, video, short messages, etc.

The application framework layer provides an application programming interface (application programming interface, API) and programming framework for application programs of the application layer. The application framework layer includes a number of predefined functions.

As shown in fig. 6, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

The window manager is used for managing window programs. The window manager can acquire the size of the display screen, judge whether a status bar exists, lock the screen, intercept the screen and the like.

The content provider is used to store and retrieve data and make such data accessible to applications. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phonebooks, etc.

The view system includes visual controls, such as controls to display text, controls to display pictures, and the like. The view system may be used to build applications. The display interface may be composed of one or more views. For example, a display interface including a text message notification icon may include a view displaying text and a view displaying a picture.

The telephony manager is used to provide the communication function of the communication terminal 300. Such as the management of call status (including on, hung-up, etc.).

The resource manager provides various resources for the application program, such as localization strings, icons, pictures, layout files, video files, and the like.

The notification manager allows the application to display notification information in a status bar, can be used to communicate notification type messages, can automatically disappear after a short dwell, and does not require user interaction. Such as notification manager is used to inform that the download is complete, message alerts, etc. The notification manager may also be a notification in the form of a chart or scroll bar text that appears on the system top status bar, such as a notification of a background running application, or a notification that appears on the screen in the form of a dialog window. For example, a text message is presented in a status bar, a presentation sound is emitted, the communication terminal vibrates, and an indicator light blinks.

Android run time includes a core library and virtual machines. Android run time is responsible for scheduling and management of the Android system.

The core library consists of two parts: one part is a function which needs to be called by java language, and the other part is a core library of android.

The application layer and the application framework layer run in a virtual machine. The virtual machine executes java files of the application program layer and the application program framework layer as binary files. The virtual machine is used for executing the functions of object life cycle management, stack management, thread management, security and exception management, garbage collection and the like.

The system library may include a plurality of functional modules. For example: surface manager (surface manager), media Libraries (Media Libraries), three-dimensional graphics processing Libraries (e.g., openGL ES), 2D graphics engines (e.g., SGL), etc.

The surface manager is used to manage the display subsystem and provides a fusion of 2D and 3D layers for multiple applications.

Media libraries support a variety of commonly used audio, video format playback and recording, still image files, and the like. The media library may support a variety of audio and video encoding formats, such as MPEG4, h.264, MP3, AAC, AMR, JPG, PNG, etc.

The three-dimensional graphic processing library is used for realizing three-dimensional graphic drawing, image rendering, synthesis, layer processing and the like.

The 2D graphics engine is a drawing engine for 2D drawing.

The kernel layer is a layer between hardware and software. The inner core layer at least comprises a display driver, a camera driver, an audio driver and a sensor driver.

The workflow of the communication terminal 300 software and hardware is exemplified below in conjunction with the media playback scenario.

When touch screen 331 receives a touch operation, a corresponding hardware interrupt is issued to the kernel layer. The kernel layer processes the touch operation into the original input event (including information such as touch coordinates, time stamp of touch operation, etc.). The original input event is stored at the kernel layer. The application framework layer acquires an original input event from the kernel layer, and identifies a control corresponding to the input event. Taking the touch operation as a single click operation, the control corresponding to the single click operation is a control for playing the application icon by the media asset as an example. The media asset playing application calls an interface of the application framework layer, starts the media asset playing application, and then plays the media asset by calling the kernel layer.

In some embodiments, as shown in fig. 7, a user may open an application of a corresponding media playback application by touching an application icon on a user interface.

Based on the display device 200 and the communication terminal 300 described above, the display device 200 can be caused to play the content in the display communication terminal 300 by establishing a screen-cast connection, as shown in fig. 8. In some embodiments, as shown in fig. 9, in order to establish the screen-cast connection relationship, a user may issue a screen-cast connection request through the communication terminal 300 and transmit the screen-cast connection request to the display apparatus 200 through the WiFi network. The display device 200 completes the configuration of the transmission protocol according to the screen connection request, so as to establish a transmission channel of the screen data with the communication terminal 300.

The display device 200 may also implement a screen-drop connection with the communication terminal 300 through different connection means. For example, when the display apparatus 200 and the communication terminal 300 access the same wireless lan, a screen-cast connection may be established based on a WiFi network. For another example, when NFC (Near Field Communication ) components are provided on both the display device 200 and the communication terminal 300, a screen-casting connection relationship may be established through the NFC component. It is obvious that other wired or wireless connection manners may be adopted between the display device 200 and the communication terminal 300, so as to establish a projection screen connection relationship, for example, an RF radio frequency connection, an infrared connection, a cellular network, and other connection manners that can be imagined by those skilled in the art based on the connection manners provided in the foregoing embodiments.

After the screen-cast connection is established, the display device 200 may receive the related data of the communication terminal 300 through the screen-cast data channel, thereby playing the related content in the display device 200. Under different screen data transmission protocols, the screen data acquired by the display device 200 is in different forms. For example, under the DLNA (Digital Living Network Alliance ) protocol, the display device 200 may acquire a link address of a certain resource in the communication terminal 300 through a screen-cast data channel and acquire media content data by accessing the link address, thereby playing the media data. Meanwhile, the display device 200 may also obtain, through the screen-throwing data channel, a control instruction sent by the communication terminal 300, such as fast forward, pause, stop screen throwing, etc., when playing media asset data. For example, after the connection is established through the communication protocol, the communication terminal 300 such as a mobile phone and the display device 200 may perform negotiation between the device function and the network condition to select a suitable audio/video transmission format. After establishing an on-line protocol (Session) of the video stream, a series of real-time streaming protocol (RTSP) control commands are used to control the playback and termination of the video stream.

Under other screen connection protocols such as Miracast screen projection, airplay screen projection, NFC mirror screen projection, music broadcast screen projection, etc., the display device 200 may directly acquire the video data stream from the communication terminal 300. The video data stream may be a video data stream corresponding to a final display screen of the display unit 310 of the communication terminal 300, that is, the video data stream includes a media content screen, a UI interface, and other video screen contents that may be displayed in the display unit 310 in the communication terminal 300. The video data stream may be just video data that can be acquired in the communication terminal 300, such as media video data being played, or the like. At this time, the communication terminal 300 may serve as a signal source of the display device 200, and the display device 200 may continuously acquire video data through the on-screen data channel and display the video data on the display 275 in real time, thereby achieving the effect of displaying the video data in synchronization with the communication terminal 300.

The communication terminal 300 can acquire different screen projection data according to different screen projection protocols. For example, for a system application or directly on the system level, the communication terminal 300 may establish a screen-casting data channel using a Miracast screen-casting, an Airplay screen-casting, or a WiDi screen-casting protocol.

The Miracast screen is a wireless display standard based on Wi-Fi direct connection and established by Wi-Fi alliance. The Miracast screen can be suitable for Android equipment, namely the Android equipment can share video pictures in a Miracast screen-throwing mode. Generally, the intelligent device with the android of more than 5.0 is internally provided with a Miracast protocol, so that the direct wireless screen can be directly carried out through the WiFiP2P function between the screen-throwing device and the WiFi network card of the receiving end device under the condition of no need of installing any software. In addition, miracast screen-throwing also supports audio and video synchronous transmission. However, since the Android devices are too many and too miscellaneous, the Miracast cannot be fully compatible with all the Android devices, and the screen-throwing function based on the Miracast cannot be realized, and malicious or misscreen-throwing cannot be prevented.

The Airplay screen is a wireless playing technology which is put forward by apple corporation and is suitable for wireless screen projection of iOS and Mac systems. The built-in Airplay screen-throwing function of the apple series equipment is matched with a screen-throwing device supporting AirPlay server service for use, the effect is clear and stable, and the protocol has the screen-throwing code function. However, it is necessary to ensure that the apple device and the screen projector are used under the same local area network (same network segment), and cannot be used across network segments and across VLANs.

The WiDi screen-throwing technology is a wireless playing technology which is dominant by Intel and is also based on a WiFi Direct connection technology. The Windows8/10 notebook computer originally supports WiDi screen projection, can directly project the screen without installing any software, and supports the USB reverse control function.

The above three screen projection protocols can acquire not only a video data stream in the communication terminal 300 but also an audio data stream in the communication terminal 300, so that the video data and the audio data can be synchronously transmitted to the display apparatus 200, and the display apparatus 200 can synchronously play video and audio. Since the above-mentioned Miracast screen-throwing protocol and the air-lay screen-throwing protocol require system-level application authority, the Miracast screen-throwing protocol and the air-lay screen-throwing protocol are generally only suitable for system application, or the display device 200 and the communication terminal 300 of the same manufacturer directly establish screen-throwing connection through the built-in rule of the operating system.

The display device 200 may establish a screen-projection connection with the communication terminal based on any of the screen-projection manners provided in the above embodiments.

However, not all the media resource applications can realize the instant screen projection from the communication terminal to the display device, and as part of the media resource suppliers set playing rights to the media resource, part of the media resource applications encode the media resource in the communication terminal, the display device needs to decode the media resource through the same media resource application to play the media resource, and only when the same media resource exists in both the communication terminal and the display device, the screen projection from the communication terminal to the display device can be performed to play the media resource in the media resource application.

FIG. 10 illustrates a schematic diagram of a media asset application drop scene in accordance with an exemplary embodiment. As shown in fig. 10, taking the media asset application as an example, when browsing the media asset in the Y Tube application on the communication terminal, the display device can only decode the media asset through the Y Tube in the display device, and if the Y Tube application does Not exist in the display device, the communication terminal prompts the user that the connection fails, and in some embodiments, the communication terminal prompts the user that the connection fails through a "404Not Found" message. The page being displayed by the display device is unchanged. In the scene, the user can not influence the display device only by operating the communication terminal, and the user does not know how to operate the display device side so as to view the media asset.

In order to solve the problem that in the screen throwing process, when the display equipment lacks a corresponding application and is matched with the application in the communication terminal, no prompt exists on the display equipment, and the communication terminal interface can only prompt the user of the interactive mode of connection failure.

An embodiment of the present application shows a display apparatus including: a display; a controller configured to perform step S1101 to step S1104 as shown in fig. 11;

step S1101: when the communication terminal plays media assets through the first application, receiving an application starting instruction sent by the communication terminal to the display equipment; the method comprises the steps that an application starting instruction is used for starting a second application of a display device end, and the first application is an application used for playing media in a communication terminal; the second application is an application corresponding to the first application in the display device and is used for playing media resources sent by the first application through the screen throwing connection.

In some embodiments, when the communication terminal plays the media asset through the first application, in order to ensure that the user can realize the screen-throwing function, the media asset provider is typically provided with a screen-throwing control on the media asset playing interface, and the user can send an application starting instruction to the display device by clicking the screen-throwing control on the media asset playing interface of the communication terminal so as to establish a connection between the communication terminal and the display device.

It should be noted that, in the process of establishing connection between the display device and the communication terminal, the display device does not have any prompt, and does not represent that the display device does not respond to the start application instruction sent by the communication terminal at all, but there is a response process between the display device and the communication terminal, but the connection is not successfully established finally.

In some embodiments, the step of receiving an application start instruction sent by the communication terminal to the display device is further configured to:

receiving and analyzing an application starting instruction sent by a communication terminal through a dial server; the dialing server is used for providing equipment discovery service; the device discovery service is a service for causing the communication terminal to discover the display device based on an SSDP protocol;

after the dialing server analyzes an application starting instruction sent by the communication terminal, checking whether the second application is legal or not through the dialing server;

and if the second application is legal, executing the step of judging whether the second application exists in the display equipment.

In a specific implementation, the display device establishes a connection with the communication terminal through the DIAL server, and the DIAL server in the application can be a DIAL server for realizing a DIAL screen-throwing protocol. In the process of implementing the DIAL-up protocol, the DIAL-up server may be capable of providing a device discovery service and REST service to the communication terminal; wherein the device discovery service may cause the communication terminal to discover the display device based on the SSDP protocol; REST service can receive and parse contents delivered by the communication terminal to the display device according to the DIAL screen-dropping protocol, for example: media asset information, control instructions, etc.

The SSDP protocol, i.e. the simple service discovery protocol, is an application layer protocol, provides a mechanism for discovering network services for network clients, and is implemented in a multicast manner based on notification and discovery routing.

It should be noted that, in the present application, the device discovery service and the REST service provided by the dial server are independent from each other, and whether the REST service receives and parses the content delivered to the display device by the communication terminal, the device discovery service can operate normally so that the display device can be discovered by the communication terminal through the SSDP protocol. When the communication terminal transmits a REST request to the display device, the display device provides a REST service that receives and parses the REST request to execute the contents of the REST request.

In the embodiment of the application, the REST request is an application starting instruction, and after the display device analyzes the request through the REST service, the dial server verifies the validity of the second application before executing the operation of starting the second application, so as to ensure that the display device is not infringed by illegal application.

In some embodiments, the display device may be connected to the server through a wired network or a wireless network, and query the packet name of the second application to determine whether the second application is legal.

In some embodiments, the display device may compare the package name of the second application with a built-in legal application package name database to determine whether the second application is legal.

Step S1102: and after receiving the application starting instruction, judging whether the second application exists in the display equipment.

In some embodiments, when the second application is determined to be legal, it is indicated that the second application may be used to play media in the display device, where it is further determined whether the second application exists in the display device.

Step S1103: and if the second application exists in the display device, starting the second application in response to the application starting instruction.

In some embodiments, the second application may be a media asset playing application commonly used by the user, or may be a media asset playing application not commonly used by the user, where in a case where the user has downloaded the second application, if the second application is a commonly used application by the user, the second application may be set at a striking position of the display interface, and if the second application is a commonly used application by the user, the second application may be set at a hidden position of the display interface, for example, at the end of an application page.

Step S1104: and if the second application does not exist in the display equipment, downloading the second application.

In some embodiments, when the second application is not present in the display device, the controller is further configured to:

and controlling the dialing server to send a first prompt to the communication terminal according to an SSDP protocol so as to enable a display interface of the communication terminal to display the first prompt, wherein the first prompt is used for prompting a user that the second application is not installed in the display equipment.

In some embodiments, when the second application is not installed in the display device, the dialserver may send a response "application not installed" request to the communication terminal according to the SSDP protocol, and the communication terminal may correspondingly pop up an error prompt to prompt the user that the second application is not installed in the display device.

In some embodiments, the controller performs the step of downloading the second application, and is further configured to:

determining whether the second application is present in an application store of the display device when the second application is not present in the display device;

if the second application exists in the application store of the display device, sending an application download address of the second application to the dial-up server;

And when the dial-up server receives the application download address of the second application, responding to a download instruction of a user to download the second application.

In some embodiments, when the communication terminal pops up the error prompt, the DIAL server sends a search request to an application store of the display device, confirming whether the second application is present in the application store. If the second application exists in the application store, the display device jumps to the application detail page address, and after the DIAL server receives the response address, installation prompt information is displayed on the display device. And when receiving an application downloading instruction of the user, jumping to the application detail page to download the second application.

In some embodiments, when the dial-up server receives the application download address of the second application, the controller is further configured to:

controlling a display interface of the display device to display a second prompt, the second prompt being used to characterize that the second application is not installed on the display device and that the second application is downloadable from an application store of the display device.

Controlling a display interface of the display device to display a third prompt, wherein the third prompt is used for recommending the associated application of the second application to a user; the third prompt comprises a link for downloading the associated application;

and when the user clicks a link for downloading the associated application, downloading the associated application.

controlling a display interface of the display device to display a fourth prompt, wherein the fourth prompt is used for providing a search function for a user to search media resources being played by the communication terminal;

and when the user searches the media asset through the searching function, playing the media asset.

It should be noted that, the above prompts are all for facilitating quick application download to the user for playing the media asset. The first prompt and the second prompt can be set on the same display interface, the first prompt and the third prompt can be set on the same display interface, and the first prompt, the third prompt and the fourth prompt can be set on the same display interface.

Fig. 12 is a schematic diagram illustrating that the first prompt and the second prompt are in the same display interface according to an embodiment of the present application, and as shown in fig. 12, the prompt information of the display interface may include: current screen-drop applications are uninstalled and the applications can be downloaded from the application store.

Fig. 13 is a schematic diagram illustrating that the first prompt and the third prompt are in the same display interface according to an embodiment of the present application, and as shown in fig. 13, the prompt information of the display interface may include: "the application is not online, you can use other associated resources. "

Fig. 14 is a schematic diagram schematically showing that a first prompt, a third prompt and a fourth prompt are in the same display interface according to an embodiment of the present application, and as shown in fig. 14, prompt information of the display interface may include: "the application is not online, you can use other associated resources, and can also use the search function to search video information directly. "

It should be noted that, the prompt message shown by the display interface may set the description text according to the actual situation, including but not limited to the schematic manner shown in the embodiment of the present application, and may also guide the user in various manners such as graphic, video, etc.

receiving an application starting address of a second application sent by the communication terminal through the dial server; wherein the application start addresses of the first application and the second application are the same;

And after the dialing server receives the application starting address, controlling the dialing server to load the media asset through a browser according to the application starting address so as to play the media asset.

In some embodiments, the second applications are browser-based applications, each of the second applications corresponds to a URL address, when the second applications are started, the browser loads a resource file through the corresponding URL address and presents the resource file to the user, when the communication terminal receives a response of "application not installed" from the dialserver, the communication terminal sends an application start address to the display device, and when the dialserver receives the application start address, the browser is used to load the resource file through the URL address of the second application and presents the resource file to the user.

The application also shows an application pairing method, which comprises the following steps:

when the communication terminal plays media assets through a first application, receiving an application starting instruction sent by the communication terminal to the display equipment; the application starting instruction is used for starting a second application of the display equipment end, and the second application is a unique application corresponding to the first application;

judging whether the second application exists in the display equipment or not after receiving the application starting instruction;

If the second application exists in the display device, responding to the application starting instruction to start the second application;

and if the second application does not exist in the display equipment, downloading the second application.

In some embodiments, the receiving an application starting instruction sent by the communication terminal to the display device includes:

receiving and analyzing an application starting instruction sent by the communication terminal through a dial server; the dialing server is used for providing equipment discovery service; the device discovery service is a service for causing the communication terminal to discover the display device based on an SSDP protocol;

It should be understood that, the specific implementation manner of each step in the above application pairing method may refer to the foregoing display device embodiment, which is not described herein. According to the embodiment, the application pairing method can solve the problem that the connection failure can only be prompted under the condition that the pairing between the communication terminal and the display equipment in the existing screen-throwing scene fails, and the application downloading method is timely provided to play the media asset under the condition that the display equipment does not have the corresponding application play media asset.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the application.

The foregoing description, for purposes of explanation, has been presented in conjunction with specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit the embodiments to the precise forms disclosed above. Many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles and the practical application, to thereby enable others skilled in the art to best utilize the embodiments and various embodiments with various modifications as are suited to the particular use contemplated.

Claims

1. A display device, characterized by comprising:

a display;

a communicator configured to establish a screen-drop connection with a communication terminal;

A controller configured to:

when the communication terminal plays media assets through a first application, receiving an application starting instruction sent by the communication terminal to the display equipment; the application starting instruction is used for starting a second application of the display equipment end; the first application is an application for playing media resources in the communication terminal; the second application is an application corresponding to the first application in the display equipment and is used for playing media resources sent by the first application through screen projection connection;

2. The display device according to claim 1, wherein the controller performs the step of receiving a start application instruction transmitted from a communication terminal to the display device, and is further configured to:

3. The display device of claim 2, wherein when the second application is not present in the display device, the controller is further configured to:

4. The display device of claim 2, wherein the controller performs the step of downloading the second application, and is further configured to:

5. The display device of claim 4, wherein when the dial-up server receives the application download address of the second application, the controller is further configured to:

6. The display device of claim 4, wherein when the second application is not present in the display device, the controller is further configured to:

7. The display device of claim 4, wherein when the second application is not present in the display device, the controller is further configured to:

8. The display device of claim 2, wherein when the second application is not present in the display device, the controller is further configured to:

9. An application pairing method, the method comprising:

when the communication terminal plays media assets through the first application, receiving an application starting instruction sent by the communication terminal to the display equipment; the application starting instruction is used for starting a second application of the display equipment end, and the first application is an application used for playing media resources in the communication terminal; the second application is an application corresponding to the first application in the display equipment and is used for playing media resources sent by the first application through screen projection connection;

10. The application pairing method according to claim 9, wherein the receiving the application starting instruction sent by the communication terminal to the display device includes: