CN115022688A

CN115022688A - Display device and media data relay method

Info

Publication number: CN115022688A
Application number: CN202210589505.0A
Authority: CN
Inventors: 付延松; 李金昆
Original assignee: Hisense Visual Technology Co Ltd
Current assignee: Hisense Visual Technology Co Ltd
Priority date: 2022-05-26
Filing date: 2022-05-26
Publication date: 2022-09-06

Abstract

The application provides a display device and a media data relay method, wherein the method can generate a relay instruction based on voice information of a user; the relay instruction comprises a device name; then responding to the relay instruction, and acquiring a local area network equipment list; then according to the device name, selecting receiving end equipment from the local area network equipment list; and finally, sending the media data to receiving end equipment. The relay instruction indicates that the media data of the display device are subjected to data relay; the local area network device list records a display device and a device supporting relay media data. Therefore, the real-time performance of media data relay is improved, and the communication configuration requirement required by the media data relay is reduced.

Description

Display device and media data relay method

Technical Field

The application relates to the technical field of smart home, in particular to a display device and a media data relay method.

Background

Along with the popularization of intelligent terminal equipment and networks, the number of intelligent equipment in families is continuously increased, and the linkage among multiple intelligent equipment is more and more. Therefore, the interest in linkage control among multiple smart devices is increasing.

At present, the linkage control among multiple intelligent devices generally means that media data of an initiating terminal device is relayed to a receiving terminal device, and the receiving terminal device continues to play the media data; in the process of controlling the media data of the initiating terminal device to relay to the receiving terminal device, the initiating terminal device sends a relay request to the receiving terminal device, and the receiving terminal device responds to the relay request and acquires the media data from the cloud server through a protocol channel of the wide area network, so that the media data relay is realized.

However, in the process from the receiving end device receiving the relay request to the acquiring the media data, the time consumption is too large, which results in poor real-time performance of the media data relay, and the required communication configuration requirements (such as bandwidth requirements, network stability requirements, and the like) are high.

Disclosure of Invention

The application provides a display device and a media data relay method, which are used for solving the problems of poor real-time performance and high requirement on required communication configuration in the process of relaying media data of an initiating terminal device to a receiving terminal device.

In a first aspect, the present application provides a display device comprising:

a display configured to present media data;

the memory is used for storing the media data to be relayed;

a communicator configured to establish a communication connection with a local area network service;

a controller configured to:

generating a relay instruction based on voice information of a user, wherein the relay instruction comprises an equipment name; the relay instruction represents that the media data of the display equipment are subjected to data relay;

responding to the relay instruction, and acquiring a local area network equipment list; the local area network device list records the display device and a device supporting relay of the media data;

selecting receiving terminal equipment from a local area network equipment list according to the equipment name;

and sending the media data to the receiving terminal equipment.

In a second aspect, the present application further provides a media data relay method, where the method is applied to a display device;

the method comprises the following steps:

and sending the media data to the receiving terminal equipment.

According to the technical scheme, the display device serves as an initiating terminal device, and the receiving terminal device is selected from the local area network device list according to the device name; the local area network equipment list is an equipment list of local area network services to which display equipment belongs, and the equipment names are obtained based on voice information input by a user; the local area network device list records a display device and a device supporting receiving of media data; that is, the devices recorded in the lan device list belong to the same lan service as the display device.

Because the receiving end device is selected from the local area network device list, the receiving end device and the display device belong to the same local area network service, that is, the receiving end device and the display device are terminal devices in the same local area network.

The media data are sent to the receiving terminal equipment through the same local area network, so that the real-time performance of media data relay is improved, and the communication configuration requirement required by the media data relay is reduced.

Drawings

In order to more clearly explain the technical solution of the present application, the drawings needed to be used in the embodiments are briefly described below, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

Fig. 1 is a schematic view of an application scenario of a display device in an embodiment of the present application;

fig. 2 is a block diagram showing a configuration of a control device according to an embodiment of the present application;

fig. 3 is a block diagram of a hardware configuration of a display device in an embodiment of the present application;

fig. 4 is a block diagram of a software configuration of a display device in an embodiment of the present application;

fig. 5 is a flowchart illustrating a media data relay method according to an embodiment of the present application;

FIG. 6 is a diagram illustrating a media data relay according to an embodiment of the present application;

FIG. 7 is a diagram illustrating a media data relay according to an embodiment of the present application;

FIG. 8 is a diagram illustrating a media data relay according to an embodiment of the present application;

fig. 9 is a flowchart illustrating a media data relay according to an embodiment of the present application.

Detailed Description

To make the purpose and embodiments of the present application clearer, the following will clearly and completely describe the exemplary embodiments of the present application with reference to the attached drawings in the exemplary embodiments of the present application, and it is obvious that the described exemplary embodiments are only a part of the embodiments of the present application, and not all of the embodiments.

It should be noted that the brief descriptions of the terms in the present application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of the present application. These terms should be understood in their ordinary and customary meaning unless otherwise indicated.

The terms "first," "second," "third," and the like in the description and claims of this application and in the above-described drawings are used for distinguishing between similar or analogous objects or entities and not necessarily for describing a particular sequential or chronological order, unless otherwise indicated. It is to be understood that the terms so used are interchangeable under appropriate circumstances.

The terms "comprises" and "comprising," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a product or apparatus that comprises a list of elements is not necessarily limited to all elements expressly listed, but may include other elements not expressly listed or inherent to such product or apparatus.

Fig. 1 is a schematic diagram of an application scenario according to some embodiments of the present application, which is intended to illustrate a scenario in which a plurality of display devices, including but not limited to devices having data transceiving and processing functions and image display functions and/or sound output functions, are present, and a server capable of communicating with the display devices. In the scenario shown in fig. 1, the smart television 200, the mobile device 300, the smart refrigerator 400, the speaker device 500, and the server 600 are included. The intelligent refrigerator 400 includes a media data output device, such as a display screen, an audio player, and the like.

Based on the internet of everything technology, communication connection can be established between multiple display devices in the above scenario, for example, communication is performed between the mobile device 300 and the smart television 200, so as to project an image displayed on the mobile device 300 onto the smart television 200.

The communication protocols for realizing the above-mentioned everything interconnection can include a local area network protocol, a wide area network protocol and a short-range wireless communication protocol which is not limited by a network. Local area network protocols include, but are not limited to, HSP communication protocols; the wide area network includes, but is not limited to, an Artificial Intelligence Internet of Things (AIOT) protocol, and the short range wireless communication protocol includes, but is not limited to, a bluetooth transmission protocol, an infrared transmission protocol.

Based on the difference of the aforementioned communication protocol types, the communication protocol channels of the display device can be divided into local area network protocol channels based on a local area network, wide area network protocol channels based on a wide area network, and other protocol channels. Other protocol channels include bluetooth protocol channels, infrared protocol channels, etc. The display device in the above scenario may support one or more of the aforementioned protocol channels.

The display device can establish a communication connection with the server to perform information interaction with the server, for example, to provide various contents and interaction information to the display device. The display device may be allowed to be communicatively connected via a Local Area Network (LAN), a Wireless Local Area Network (WLAN), and other networks. The server 600 may be a cluster or a plurality of clusters, and may include one or more types of servers.

It should be noted that, in the same scene shown in fig. 1, other display devices may also be included, including but not limited to external devices such as a touch-control integrated device, a projection device, a tablet computer, a computer, and a notebook computer. The number of devices in the same type of terminal is not limited herein.

In some embodiments, control between different display devices may be achieved by the control apparatus 100. As shown in fig. 1, a user may control or operate the smart tv 200 through the control apparatus 100 to perform data communication with the mobile device 300.

In some embodiments, the control apparatus 100 may be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication or bluetooth protocol communication, and other short-distance communication methods, and the display device is controlled by wireless or wired methods. The user may input user instructions via keys on a remote control, voice input, control panel input, etc. to control the display device.

In some embodiments, the display device may be controlled by a manner other than the control apparatus 100, for example, the voice information control of the user may be directly received by a module configured inside the display device for acquiring voice information, or may be received by a voice control device provided outside the display device.

In some embodiments, the display device may not receive instructions using the smart device or control device described above, but may receive user control through touch or gestures, or the like.

In some embodiments, a display device includes a controller, a memory, a communicator, and a display. In some embodiments, the display apparatus further comprises at least one of a tuner demodulator, a detector, an external device interface, an audio output interface, a power supply, a user interface. The controller includes, but is not limited to, a processor, a video processor, an audio processor, an image processor, a Read-Only Memory (ROM), a Random Access Memory (RAM), and the like.

Fig. 2 exemplarily shows a block diagram of a configuration of the control apparatus 100 according to an exemplary embodiment. As shown in fig. 2, the control device 100 includes a controller 110, a communication interface 130, a user input/output interface 140, a memory, and a power supply. The control apparatus 100 may receive an input operation instruction from a user, and convert the operation instruction into an instruction recognizable and responsive to the display device, thereby playing a role in mediating interaction between the user and the display device.

Fig. 3 illustrates a hardware configuration block diagram of a display device according to an exemplary embodiment.

In some embodiments, the display apparatus includes at least one of a tuner demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, a user interface.

In some embodiments the controller comprises a processor, a video processor, an audio processor, a graphics processor, a RAM, a ROM, a first interface to an nth interface for input/output.

In some embodiments, the display 260 includes a display screen component for presenting a picture, and a driving component for driving an image display, a component for receiving an image signal from the controller output, performing display of video content, image content, and a menu manipulation interface, and a user manipulation UI interface.

In some embodiments, the display 260 may be a liquid crystal display, an OLED display, and a projection display, and may also be a projection device and a projection screen.

In some embodiments, communicator 220 is a component for communicating with external devices or servers according to various communication protocol types. For example: the communicator may include at least one of a Wifi module, a bluetooth module, a wired ethernet module, and other network communication protocol chips or near field communication protocol chips, and an infrared receiver. The display apparatus may establish transmission and reception of control signals and data signals with the external control apparatus 100 or the server 400 through the communicator 220.

In some embodiments, the user interface may be configured to receive control signals for controlling the apparatus 100 (e.g., an infrared remote control, etc.).

In some embodiments, the detector 230 is used to collect signals of the external environment or interaction with the outside. For example, detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; alternatively, the detector 230 includes an image collector, such as a camera, which may be used to collect external environment scenes, attributes of the user, or user interaction gestures, or the detector 230 includes a sound collector, such as a microphone, which is used to receive external sounds.

In some embodiments, the external device interface 240 may include, but is not limited to, the following: high Definition Multimedia Interface (HDMI), analog or data high definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, and the like. The interface may be a composite input/output interface formed by the plurality of interfaces.

In some embodiments, the tuner demodulator 210 receives broadcast television signals via wired or wireless reception, and demodulates audio/video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

In some embodiments, the controller 250 and the modem 210 may be located in different separate devices, that is, the modem 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

In some embodiments, the controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in memory. The controller 250 controls the overall operation of the display device. For example: in response to receiving a user command for selecting a UI object to be displayed on the display 260, the controller 250 may perform an operation related to the object selected by the user command.

In some embodiments, the object may be any one of selectable objects, such as a hyperlink, an icon, or other actionable control. The operations related to the selected object are: displaying an operation connected to a hyperlink page, document, image, or the like, or performing an operation of a program corresponding to the icon.

In some embodiments the controller comprises at least one of a Central Processing Unit (CPU), a video processor, an audio processor, a Graphics Processing Unit (GPU), a RAM Random Access Memory (RAM), a ROM (Read-Only Memory), a first to nth interface for input/output, a communication Bus (Bus), and the like.

A CPU processor. For executing operating system and application program instructions stored in the memory, and executing various application programs, data and contents according to various interactive instructions receiving external input, so as to finally display and play various audio-video contents. The CPU processor may include a plurality of processors. E.g. comprising a main processor and one or more sub-processors.

In some embodiments, a graphics processor for generating various graphics objects, such as: icons, operation menus, user input instruction display graphics, and the like. The graphic processor comprises an arithmetic unit, which performs operation by receiving various interactive instructions input by a user and displays various objects according to display attributes; the system also comprises a renderer for rendering various objects obtained based on the arithmetic unit, wherein the rendered objects are used for being displayed on a display.

In some embodiments, the video processor is configured to receive an external video signal, and perform video processing such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, and image synthesis according to a standard codec protocol of the input signal, so as to obtain a signal that can be displayed or played on a directly displayable device.

In some embodiments, the video processor includes a demultiplexing module, a video decoding module, an image synthesis module, a frame rate conversion module, a display formatting module, and the like. The demultiplexing module is used for demultiplexing the input audio and video data stream. And the video decoding module is used for processing the demultiplexed video signal, including decoding, scaling and the like. And the image synthesis module, such as an image synthesizer, is used for performing superposition mixing processing on the GUI signal input by the user or generated by the user and the video image after the zooming processing by the graphics generator so as to generate an image signal for display. And the frame rate conversion module is used for converting the frame rate of the input video. And the display formatting module is used for converting the received video output signal after the frame rate conversion, and changing the signal to be in accordance with the signal of the display format, such as an output RGB data signal.

In some embodiments, the audio processor is configured to receive an external audio signal, decompress and decode the received audio signal according to a standard codec protocol of the input signal, and perform noise reduction, digital-to-analog conversion, and amplification processing to obtain an audio signal that can be played in the speaker.

In some embodiments, a user may enter user commands on a Graphical User Interface (GUI) displayed on display 260, and the user input interface receives the user input commands through the Graphical User Interface (GUI). Alternatively, the user may input the user command by inputting a specific sound or gesture, and the user input interface receives the user input command by recognizing the sound or gesture through the sensor.

In some embodiments, a "user interface" is a media interface for interaction and information exchange between an application or operating system and a user that enables conversion between an internal form of information and a form that is acceptable to the user. A commonly used presentation form of the User Interface is a Graphical User Interface (GUI), which refers to a User Interface related to computer operations and displayed in a graphical manner. It may be an interface element such as an icon, a window, a control, etc. displayed in the display screen of the electronic device, where the control may include a visual interface element such as an icon, a button, a menu, a tab, a text box, a dialog box, a status bar, a navigation bar, a Widget, etc.

Referring to fig. 4, in some embodiments, the system is divided into four layers, which are an Application (Applications) layer (abbreviated as "Application layer"), an Application Framework (Application Framework) layer (abbreviated as "Framework layer"), an Android runtime (Android runtime) and system library layer (abbreviated as "system runtime library layer"), and a kernel layer from top to bottom.

In some embodiments, at least one application program runs in the application program layer, and the application programs may be windows (windows) programs carried by an operating system, system setting programs, clock programs or the like; or an application developed by a third party developer. In particular implementations, the application packages in the application layer are not limited to the above examples.

The framework layer provides an Application Programming Interface (API) and a programming framework for the application. The application framework layer includes a number of predefined functions. The application framework layer acts as a processing center that decides to let the applications in the application layer act. The application program can access the resources in the system and obtain the services of the system in execution through the API interface.

As shown in fig. 4, in the embodiment of the present application, the application framework layer includes a manager (Managers), a Content Provider (Content Provider), and the like, where the manager includes at least one of the following modules: an Activity Manager (Activity Manager) is used for interacting with all activities running in the system; the Location Manager (Location Manager) is used for providing the system service or application with the access of the system Location service; a Package Manager (Package Manager) for retrieving various information related to an application Package currently installed on the device; a Notification Manager (Notification Manager) for controlling display and clearing of Notification messages; a Window Manager (Window Manager) is used to manage icons, windows, toolbars, wallpapers, and desktop components on a user interface.

In some embodiments, the activity manager is used to manage the lifecycle of the various applications as well as general navigational fallback functions, such as controlling exit, opening, fallback, etc. of the applications. The window manager is used for managing all window programs, such as obtaining the size of the display screen, judging whether a status bar exists, locking the screen, intercepting the screen, controlling the change of the display window (for example, reducing the display window, shaking the display, distorting and deforming the display, and the like).

In some embodiments, the system runtime layer provides support for the upper layer, i.e., the framework layer, and when the framework layer is used, the android operating system runs the C/C + + library included in the system runtime layer to implement the functions to be implemented by the framework layer.

In some embodiments, the kernel layer is a layer between hardware and software. As shown in fig. 4, the core layer includes at least one of the following drivers: audio drive, display driver, bluetooth drive, camera drive, WIFI drive, USB drive, HDMI drive, sensor drive (like fingerprint sensor, temperature sensor, pressure sensor etc.) and power drive etc..

In some embodiments, a user may enter a user command on a Graphical User Interface (GUI) displayed on the display, and the user input interface receives the user input command through the Graphical User Interface (GUI). Alternatively, the user may input the user command by inputting a specific sound or gesture, and the user input interface receives the user input command by recognizing the sound or gesture through the sensor. The user interface is an interface (such as a physical key on the display device body, or the like) which can be used for receiving control input.

In some embodiments, a "user interface" is a media interface for interaction and information exchange between an application or operating system and a user that enables conversion between an internal form of information and a form that is acceptable to the user. A common presentation form of a User Interface is a Graphical User Interface (GUI), which refers to a User Interface related to computer operations and displayed in a graphical manner. It may be an interface element such as an icon, a window, a control, etc. displayed in the display screen of the electronic device, where the control may include a visual interface element such as an icon, a button, a menu, a tab, a text box, a dialog box, a status bar, a navigation bar, a Widget, etc.

In some embodiments, when a display device presents certain media data (e.g., pictures, video, audio, etc.), the media data may be pushed to a user-specified display device, i.e., a sink device, in response to user control. So that the receiving end device plays the media data according to the continuation.

For convenience of explanation, in the following embodiments, a display device that initiates a media data relay is referred to as an originating device; a display device that receives media data and continues to play the media data is referred to as a sink device.

For example based on fig. 1, when a user watches a movie (one type of media data) on the smart tv 200, and wants to go to a kitchen for cooking, the smart refrigerator 400 in the kitchen has a display screen, and the user can push the related data of the movie watched on the smart tv 200 to the smart refrigerator 400 by operating the smart tv 200, thereby completing media data relay, so that the smart refrigerator 400 continues to play the movie. Furthermore, the user can cook in the kitchen and continue watching the movie.

For another example, when a user listens to a song on the smart tv 200 in a bedroom, the user needs to go to a living room, and the living room has a sound box device 500. Then, the user can push the data related to the song played on the smart television 200 to the sound box device 500 by operating the smart television 200, so as to complete media data relay, and enable the sound box device 500 to continue playing the song.

In some embodiments, in the process of pushing the media data to the receiving device by the initiating device, the media data generally includes two types of data. One type is online data and the other type is local data. The online data refers to data stored in the server, and the local data refers to data stored in the initiating device. Assuming that there is no network transmission between the application installed on the originating device and the server, the data for running the application, i.e., the data residing in the originating device, is local data.

For example, for content data of a certain video, the online data is the content data of the video stored in the internet; the local data is content data of the video stored in the memory of the originating device.

In some embodiments, when the type of the media data is online data, the initiating device initiates a relay instruction to the server 600, so that the server 600 determines the media data to be relayed, and sends the media data to be relayed to the receiving device through the wan communication protocol, so that the receiving device continues to play the media data, thereby completing data relay.

In some embodiments, when the type of the media data is local data, the initiating device initiates a relay instruction to the server 600, and sends media data to be relayed (e.g. 4k high definition video) to the server 600 through a wide area network communication protocol. The server 600 then sends the media data to be relayed to the receiving end device through the wan communication protocol, so that the receiving end device continues to play the media data, thereby completing the data relay.

In the above method, the configuration requirements of the wide area network communication protocol on bandwidth, network speed, and the like are high, and the real-time performance of pushing the local data to the receiving end device through the wide area network communication protocol is poor.

Therefore, in order to improve the real-time performance of media data relay and reduce the communication configuration required for data relay, in some embodiments, a flowchart of a media data relay method is provided, as shown in fig. 5, which is applicable to a display device performing the media data relay method. The method comprises the following steps:

generating a relay instruction based on voice information of a user; the relay instruction comprises a device name; the relay instruction represents data relay of the media data of the display device. The display device is used as an initiating device, and the receiving device is selected based on the device name.

In some embodiments, the user triggers the relay instruction by sending text information to the receiving end device or touching a display screen of the receiving end device (e.g., a relay control is displayed on the display screen of the receiving end device), so that the receiving end device generates the relay instruction.

In some embodiments, the user sends the voice information to the initiating device, the initiating device sends the voice information to the voice service, and receives a relay instruction fed back by the voice service based on the voice information, so as to obtain the relay instruction.

For example, the user sends voice information to the originating device S1, where the voice information is: an intelligent refrigerator which relays to a kitchen. The originating terminal apparatus S1 transmits the voice information to the voice service after acquiring the voice information, converts the voice information into text information by the voice service, and then generates a relay instruction based on the text information.

In some embodiments, the user sends the voice information to the initiating device, and the voice service in the initiating device performs semantic parsing on the voice information, so as to generate the relay instruction.

For example, the user sends voice information to the initiating device S1, and after acquiring the voice information, the initiating device S1 performs semantic parsing on the voice information by its own voice service to obtain text information; and then generating a relay instruction based on the text information.

After the relay instruction is obtained, a first device function used by the display device when generating the relay instruction is detected. In some embodiments, the display device may detect the first device function through a self-installed application; for example, when the display device generates the relay instruction, it detects that the application in use by the display device is a video playing application, and detects that the first device function is a video playing function according to the video playing function of the video playing application.

In some embodiments, a function registration file is recorded in any one display device; a first device function used by the display device when the relay instruction is generated is detected by the function registration file. The function registration file is generated by the display device by inquiring the device function of the display device and recording the device function.

For example, the originating device S1 has a video playback function S11, an audio playback function S12, an image display function S13, and the like. The function registration file of the originating device S1 has recorded therein: originating device S1-video play function S11, audio play function S12, image display function S13.

After detecting the function of the first equipment, the initiating terminal equipment acquires a local area network equipment list; wherein, the local area network device list records the identifier of the device supporting receiving the media data to be relayed; the list of local area network devices is generated based on the first device function.

In some embodiments, the generation of the list of local area network devices is as follows: the initiating terminal device detects a first device function used when the relay instruction is generated in its own function registration file.

The first device function refers to a device function used by the display device when the relay instruction is generated. For example, when the originating device S1 generates the relay command, it detects that the device function used is the video playback function S11, and the first device function is the video playback function.

Then, the initiating terminal equipment inquires out external terminal equipment with the function of the first equipment from the local area network service; the local area network service records the identification of each external terminal device which is communicated with the local area network service. For example, the local area network service to which the originating device S1 belongs is B, and the local area network service B records therein the identities of external terminal devices such as S2, S3, and the like, which are in communication connection with the local area network service B.

The local area network service determines external terminal equipment which is in communication connection with the local area network service according to a preset heartbeat mechanism, wherein the heartbeat means that a self-defined structure body (a heartbeat packet or a heartbeat frame) is sent at regular time to enable an opposite side to know that the local area network service is online.

The preset heartbeat mechanism is that a heartbeat packet or a heartbeat frame is transmitted between the external terminal equipment and the local area network service within a preset time period. For example, if any external terminal device transmits a heartbeat packet twice within 3 seconds with the lan service B, it is determined that the external terminal device maintains a communication connection with the lan service B, and the device identifier of the external terminal device is recorded in the lan service B, such as S2, S3, and the like.

In some embodiments, for any external terminal device communicatively connected to the local area network service to which the originating device belongs, the local area network service further records a device function of the external terminal device. For example, the device functions of the external terminal device S2 include a video playback function S21 and a voice playback function S22.

In some embodiments, the query manner may be a periodic query, such as querying an external terminal device communicatively connected to the lan service in the lan service every 3 seconds; in some embodiments, the query manner may be a polling traversal, such as traversing 2 times in a traversal manner to query the external terminal device communicatively connected to the lan service.

The external terminal device having the first device function is inquired out of the external terminal devices communicating with the local area network service. As an example based on the above embodiment, if the first device function is the video playback function s11, the external terminal device having the video playback function s11 is queried among the external terminal devices communicating with the lan service.

For example, the external terminal devices communicating with the local area network service collectively include the external terminal device S2 and the external terminal device S3; wherein, the external terminal device S2 has a video playback function S21 and a voice playback function S22; the external terminal apparatus S3 has a voice playback function S31 and a voice playback function S32. Since the video playback function S11 is the same device function as the video playback function S21, the external terminal device having the first device function is the external terminal device S3.

Finally, storing the external terminal equipment with the function of the first equipment and generating a local area network equipment list; for example, the local area network device list has the external terminal device S3 and the external terminal device S4 recorded therein; namely, the external terminal device S3 and the external terminal device S4 have the first device function.

In some embodiments, the initiating device also establishes a communication connection with a wide area network service; the wide area network service is a service of a wide area network to which the originating device belongs, and the wide area network service records an external terminal device having the same user binding information as the originating device. For example, if the user Y binds the terminal device D1 and the terminal device D2 with information such as his mobile phone number and id card, the terminal device D1 and the terminal device D2 have the same user binding information.

After the initiating terminal device obtains the local area network device list, the receiving terminal device is selected from the local area network device list according to the identifier of the initiating terminal device and the device name in the relay instruction. The identifier of the initiating device may be a unique identifier, such as a unique ID of the display device; the device name in the relay instruction is a non-unique identifier, such as a type name of the terminal device (e.g., a smart television, a smart refrigerator, etc.).

In some embodiments, the initiating terminal device queries user binding information of the initiating terminal device according to its own identifier; for example, if the identifier of the originating device S1 is 001, and the user binding information corresponding to the identifier 001 is Y1, the user binding information of the originating device S1 is Y1.

Acquiring user equipment which has the same user binding information as the initiating terminal equipment and has the same equipment name as the initiating terminal equipment from the wide area network service; for example, the user binding information of the originating device S1 is Y1, and the external terminal device having the user binding information Y1 includes an external terminal device S2, an external terminal device S3, and an external terminal device S4; the external terminal device S2 is an intelligent television, the external terminal device S3 is an intelligent refrigerator, the external terminal device S4 is an intelligent refrigerator, and the device name is the intelligent refrigerator, so that the user devices are the external terminal device S3 and the external terminal device S4.

And finally, the initiating terminal equipment selects the receiving terminal equipment from the local area network equipment list according to the unique identification of the user equipment.

In some embodiments, if it is determined that the unique identifier of the user equipment is recorded in the local area network device list, the user equipment is determined as the receiving end device. Based on the foregoing embodiment, for example, assuming that the identifier of the external terminal device S3 is recorded in the local area network device list, the receiving end device is the external terminal device S3.

In some embodiments, if a plurality of selected receiving end devices are provided, the plurality of receiving end devices are respectively prompted to the user, and a further receiving instruction of the user is received, where the further receiving instruction includes an identifier of one of the plurality of receiving end devices. The prompting mode may include a voice prompt and a text prompt, which is not limited herein.

After the receiving end equipment is determined, the initiating end equipment combines the data to be played with the identification of the receiving end equipment to generate the media data to be relayed; the data to be played is local data of the originating device, such as a local XX movie.

For example, the identifier of the receiving end device is S3, the data to be played is "video data XX", and the media data to be relayed is "S3-XX".

Then, the initiating terminal device sends the media data to be relayed to the receiving terminal device in a local area network communication protocol according to the form of data stream. FIG. 6 is a diagram illustrating an exemplary media data relay; as shown in fig. 6, the initiating device is an intelligent television, and the receiving device is an intelligent refrigerator.

After the intelligent television generates the relay instruction, inquiring that the receiving end equipment is the intelligent refrigerator, and sending the media data to be relayed S3-XX to the intelligent refrigerator in a local area network communication protocol according to a data stream form; so that the display screen of the smart refrigerator displays the video data "XX".

In some embodiments, after the media data to be relayed is sent to the receiving end device in a local area network communication protocol according to a data stream form, a playing state fed back by the receiving end device based on the media data to be relayed is obtained; the playing state includes a relay success state or a relay failure state.

If the playing state is relay success, the media data to be relayed is continuously sent to the receiving end equipment by the local area network communication protocol until the receiving end equipment stops receiving the media data to be relayed; and stopping receiving the media data to be relayed if the playing of the media data to be relayed by using the receiving terminal equipment with the function of the first equipment is finished.

Based on fig. 6, fig. 7 is a schematic diagram illustrating an exemplary media data relay; as shown in fig. 7, after the receiving end device acquires the media data to be relayed, the media data to be relayed is analyzed to obtain video data "XX".

Then, the receiving end device plays the video data "XX", and feeds back the playing status to the initiating end device as success in relay. After acquiring the playing state, the initiating terminal device displays the playing state of 'relay success'.

To better state the above technical solution, fig. 8 exemplarily shows a schematic diagram of a media data relay, as shown in fig. 8, a voice assistant, a relay service and a media center are provided in an initiating end device. The voice assistant, the relay service and the media center correspond to software services in the initiating terminal device.

The media center is used for recording the device function of the initiating terminal device registered by the user, generating a function registration file and storing the function registration file to the relay service. For example, the device functions of the initiating device include a video playing function, an audio playing function, and an image displaying function, and the media center generates a function registration file of the initiating device according to the video playing function, the audio playing function, and the image displaying function.

The media center is further configured to combine the media data with the identifier of the receiving end device to generate media data to be relayed, and store the media data to be relayed to the relay service. For example, the identifier of the receiving end device is S2, the data to be played is "XY", and the generated media data to be relayed is "S2-XY".

The voice assistant of the initiating terminal device obtains the voice information sent by the user, for example, the user sends the voice information to the intelligent refrigerator in the kitchen through the voice, and then the voice information is 'relayed to the intelligent refrigerator in the kitchen'.

Then, the voice assistant of the initiating terminal equipment sends the voice information to the voice service through the voice code stream; the voice service is used for carrying out semantic analysis on the voice information and identifying the text information. That is to say, the voice information is converted into text information, and a relay instruction is generated by combining the identifier of the initiating terminal device, and the relay instruction is returned to the initiating terminal device. Wherein, the intelligent refrigerator is the name of the equipment.

After the voice assistant of the initiating terminal equipment receives the relay instruction, the voice assistant of the initiating terminal equipment detects a first equipment function used by the receiving terminal equipment when generating the relay instruction in the relay service when generating the relay instruction; for example, when the originating device S1 generates the relay command, it detects that the device function used is the audio playback function S12, and the first device function is the audio playback function.

Then, the voice assistant of the initiating device acquires the list of the local area network devices according to the relay service. The relay service of the local area network device list receiving terminal device is generated after being inquired through the local area network service.

For example, the local network service to which the initiator device S1 belongs is B, and the local network service B has recorded therein the external terminal device S2 and the external terminal device S3 that are communicatively connected to the local network service. Then, the external terminal device S2 having the audio playing function is queried from the external terminal device S2 and the external terminal device S3 according to the first device function (such as the audio playing function), and a local area network device list is generated according to the identifier of the external terminal device S2.

The voice assistant of the initiating terminal device detects the user binding information of the initiating terminal device, generates a query instruction according to the user binding information of the initiating terminal device and the device name in the relay instruction, and then sends the query instruction to the wide area network service; after receiving the query instruction, the wide area network service queries the user equipment which has the same user binding information and the same equipment name according to the user binding information and the equipment name in the query instruction.

For example, the user binding information of the originating device S1 is Y1, and the external terminal device having the user binding information Y1 includes an external terminal device S2 and an external terminal device S4; the external terminal device S2 is an intelligent television, the external terminal device S4 is an intelligent refrigerator, the device name is the intelligent refrigerator, and the user device is the external terminal device S4.

That is, the process of the initiating device generating the local area network device list and the process of querying the first device may be performed synchronously, and the order of the operation process is not limited herein.

After the initiating terminal device queries the user device and generates the local area network device list, the receiving terminal device is selected according to the identifier of the user device and the identifier of the terminal device recorded in the local area network device list. For example, if the identifier of the external terminal device S4 is recorded in the lan device list, the sink device is the external terminal device S4.

After the initiator device selects the receiver device, the relay service of the initiator device sends the media data to be relayed to the receiver device through the local area network server.

After receiving the media data to be relayed, the relay service of the receiving terminal equipment sends the media data to be relayed to the media center of the receiving terminal equipment, the media center of the receiving terminal equipment selects the application for playing the media data to be relayed, and decodes and plays the media data to be relayed.

After playing the media data to be relayed, the media center of the receiving end device feeds back the playing state (such as success relay) aiming at the media data to be relayed, and the relay service of the receiving end device feeds back the playing state to the initiating end device, thereby realizing the completion of media data relay through the local area network, and further improving the real-time performance of media data relay and reducing the communication configuration required by the media data relay.

To better state the above technical solution, fig. 9 exemplarily shows a flow diagram of media data relay; to increase the simplicity of the schematic, fig. 9 only shows the cloud. The cloud end shown in fig. 9 includes a voice cloud service and a wide area network cloud service, that is, a voice service and a wide area network service.

As shown in fig. 9, the content includes:

s910, registering the device function.

The media center of the initiating terminal device responds to the registration instruction of the user and detects the device function of the initiating terminal device; for example, the device functions of the originating device include a video playback function, an audio playback function, an image display function, and the like.

Wherein, the device functions can be divided into a plurality of levels; for example, the audio playing function is a primary device function, and the secondary device function under the primary device function is to broadcast information such as an author and a date in the audio.

In some embodiments, an application page of an application of each display device may also be detected; the application page displays the functions installed by the initiating terminal equipment and corresponding to the application page, and the equipment functions are determined according to the application. For example, if the originating device has an audio playing application and a video playing application installed therein, the device functions of the originating device include a video playing function and an audio playing function.

Then, the initiating terminal equipment generates a function registration file according to the functions of each equipment and stores the function registration file to the relay service of the initiating terminal equipment; for example, "S2-play audio function-author in broadcast audio" or "S3-image display function" is recorded in the function registration file. The initiating terminal device S2 has a primary device function as a play function, and a secondary device function under the primary device function is an author in the broadcast audio; the originating device S3 has an image display function, and the image display function is a primary device function, i.e., the originating device S3 has no other level device function.

And S920, generating a relay instruction.

The media center of the initiating terminal device acquires voice information of a user and sends the voice information to the voice cloud clothes at the cloud end. Performing semantic analysis on the voice information by using a voice service in the cloud to obtain text information, and feeding the text information back to a media center of the initiating terminal equipment; whereby the media center of the originating device may generate the relay instruction based on the textual information. The relay instruction comprises a device name, and the device name is used for selecting receiving-end equipment.

For example, the voice message is "smart tv relayed to living room"; the device name is "smart television".

And S930, inquiring the user equipment.

The media center of the initiating terminal device detects the user binding information of the initiating terminal device, generates a query instruction according to the user binding information and the device name of the initiating terminal device, and sends the query instruction to the wide area network service of the cloud. And the wide area network service queries the user equipment which has the same user binding information and the same equipment name as the initiating terminal equipment according to the user binding information and the equipment name of the initiating terminal equipment in the query instruction.

For example based on the above embodiment, if the user binding information of the initiating device is Y, the user binding information of the user device is Y, and the device name of the user device is "smart tv".

The wide area network service feeds back the first device to the media center of the initiating device.

And S940, acquiring a local area network device list.

The media center of the initiating terminal device calls a local area network device list from the relay service of the initiating terminal device; the local area network device list is generated by the first device function used when the relay service of the initiating terminal device detects that the initiating terminal device generates the relay instruction in the function registration file of the initiating terminal device, and inquiring the external terminal device with the first device function in the local area network service.

For example, if the first device function is an image display function, the external terminal device having the image display function is inquired in the lan service, and the external terminal device having the image display function is recorded in the lan device list, thereby generating the lan device list.

S950, selecting the receiving end equipment.

And the media center of the initiating terminal equipment selects the receiving terminal equipment according to the identification of the user equipment and the local area network equipment list. For example, the id of the ue S4 is recorded in the lan device list, and the sink device is the ue S4.

S960, recording the identification of the receiving terminal equipment.

After the media center of the initiating terminal device selects the receiving terminal device, the media center records the identification of the receiving terminal device to the media center of the initiating terminal device. For example, based on the above description, if the receiving device is the user device S4, the identifier of the user device S4 is recorded to the media center of the initiating device.

And S970, storing the media data to be relayed.

After acquiring the identifier of the receiving end device, the media center of the initiating end device detects data to be played (i.e., media data), such as data of playing content, playing time point, etc.

Then combining the identification of the receiving terminal equipment and the data to be played to generate the media data to be relayed; based on the above description, for example, the to-be-relayed media data "S4-XU" is generated in combination with the identifier of the receiving end device S4 and the to-be-played data "XU".

And after the generated media data to be relayed are stored, the relay service of the initiating terminal equipment is initiated.

And S980, sending the media data to be relayed.

The initiating terminal device sends the media data to be relayed to the receiving terminal device by a local area network communication protocol according to the form of data stream and based on the identification of the receiving terminal device.

According to the technical scheme, as the display device of the initiating terminal device, the unique identifier of the user device with the same user binding information and the same device name is inquired in the wide area network service according to the user binding information of the initiating terminal device and the device name in the relay instruction; and selecting the receiving terminal equipment from the unique identification of the user equipment according to the local area network equipment list.

The local area network device list is a service of a local area network to which the display device belongs, that is, the receiving end device is the same as the local area network service to which the display device belongs, that is, the receiving end device and the display device are terminal devices in the same local area network.

And then the media data to be relayed is sent to the receiving terminal equipment through the same local area network according to the local area network communication protocol, so that the relay instantaneity of the media data is improved, and the communication configuration requirement required by the relay of the media data is reduced.

The detailed description provided above is only a few examples under the general concept of the present application, and does not constitute a limitation to the scope of the present application. Any other embodiments extended according to the scheme of the present application without inventive efforts will be within the scope of protection of the present application for a person skilled in the art.

Claims

1. A display device, comprising:

a display configured to present media data;

a memory for storing media data;

a controller configured to:

responding to the relay instruction, and acquiring a local area network equipment list; the local area network device list records the display device and a device supporting relaying the media data;

and sending the media data to the receiving terminal equipment.

2. The display device of claim 1, wherein the controller is further configured to:

in the step of generating the relay instruction based on the voice information of the user, the voice information of the user is obtained;

sending the voice information to a voice service;

receiving text information which is semantically converted by a voice service based on the voice information;

and generating the relay instruction according to the text information.

3. The display device of claim 1, wherein the controller is further configured to:

inquiring the display equipment with the data relay equipment function before acquiring the local area network equipment list;

and recording the device function in a function registration file.

4. The display device of claim 3, wherein the controller is further configured to:

in the step of obtaining the local area network device list, detecting a first device function used by the display device when generating a relay instruction in a function registration file;

inquiring external terminal equipment with the first equipment function from a local area network service; the local area network service records the identification of each external terminal device communicated with the local area network service;

and storing the external terminal equipment with the function of the first equipment, and generating and acquiring a local area network equipment list.

5. The display device of claim 4, wherein the controller is further configured to:

in the step of inquiring the external terminal equipment with the first equipment function from the local area network service, inquiring the external terminal equipment communicated with the local area network service in a polling traversal mode;

and inquiring the external terminal equipment with the first equipment function in the external terminal equipment communicated with the local area network service.

6. The display device of claim 1, wherein the communicator is further configured to establish a communication connection with a wide area network service, and wherein the controller is further configured to:

in the step of selecting receiving terminal equipment from a local area network equipment list according to the equipment name, inquiring user binding information of the display equipment;

acquiring user equipment which has the same user binding information with the display equipment and has the same equipment name with the display equipment from a wide area network service;

and selecting the receiving terminal equipment from the local area network equipment list according to the unique identifier of the user equipment.

7. The display device of claim 6, wherein the controller is further configured to:

in the step of selecting the receiving end device from the local area network device list according to the unique identifier of the user device, if the unique identifier of the user device is determined to be recorded in the local area network device list, the user device is determined to be the receiving end device.

8. The display device of claim 1, wherein the controller is further configured to:

in the step of sending the media data to the receiving end equipment, combining the media data with the identification of the receiving end equipment to generate the media data to be relayed;

and sending the media data to be relayed to the receiving terminal equipment in a local area network communication protocol according to a data stream form.

9. The display device of claim 8, wherein the controller is further configured to:

after the media data to be relayed is sent to the receiving end equipment in a local area network communication protocol according to a data stream form, the playing state fed back by the receiving end equipment based on the media data to be relayed is obtained;

if the playing state is relay success, the media data to be relayed is continuously sent to the receiving end equipment by the local area network communication protocol until the receiving end equipment stops receiving the media data to be relayed.

10. A media data relay method is characterized in that the method is applied to a display device;

the method comprises the following steps:

and sending the media data to the receiving terminal equipment.