WO2018090720A1

WO2018090720A1 - Method and apparatus for plug and play screen mirroring

Info

Publication number: WO2018090720A1
Application number: PCT/CN2017/103467
Authority: WO
Inventors: 贺欣
Original assignee: 广州视源电子科技股份有限公司; 广州视睿电子科技有限公司
Priority date: 2016-10-12
Filing date: 2017-09-26
Publication date: 2018-05-24
Also published as: CN107948566A; CN107948566B

Abstract

The present invention relates to a method and apparatus for plug and play screen mirroring. Said method comprises: a first universal serial bus (USB) port of a local terminal providing a plurality of communication protocol ports to a first device, which may be identified by a generic driver of the first device; establishing a USB communication connection between the local terminal and the first device by means of said communication protocol ports and first USB port; when detected that the first USB port has a storage space access instruction from the first device, a preset storage space is presented to the first device, wherein a corresponding application program is saved in said storage space; said application program may be run by the first device so that current audio/visual information of a system of the first device is obtained; if audio/visual data is detected, the audio/visual data is transmitted by means of a second USB port of the local terminal to a second device. The present invention may achieve a plug and play USB screen mirroring experience without requiring a source device to preinstall a corresponding application program.

Description

Method and device for realizing plug and play screen

Technical field

The present invention relates to the field of data transmission technologies, and in particular, to a method and apparatus for implementing plug and play screen transmission.

Background technique

In the traditional screen mode, the PC system as the display source device can actively output its system sound and picture through the HDMI or VGA interface, but cannot actively output through the USB interface. Therefore, in the USB screen mode, an application program needs to be run in advance on the PC side, and the audio and video data of the system is intercepted by the application program, and then sent out through the USB interface, transferred through the USB screen device, and finally decoded to the display terminal. display.

However, for a display source device that does not have a corresponding application installed in advance, it is impossible to transfer its system sound and screen to the display terminal via USB, which brings inconvenience to the user.

Summary of the invention

Based on this, the method and device for implementing the plug-and-play screen provided by the embodiment of the present invention can implement the plug-and-play USB screen-playing experience without the need to display the source device to pre-install the corresponding application program.

An aspect of the present invention provides a method for implementing a plug and play screen, including:

Providing, by the first USB interface of the local end, a plurality of communication protocol ports to the first device, the communication protocol ports being identifiable by the universal drive of the first device; the universal drive providing universal communication between the first device and the standard USB device a communication protocol, the communication protocol port at least comprising: a port for transmitting audio and video data and a port for accessing the storage space;

Establishing a USB communication connection between the local end and the first device by using the communication protocol port and the first USB interface;

Detecting that the first USB interface has information from the first device, detecting a communication protocol port that transmits the information, and if it is a port for accessing the storage space, identifying the information as a storage space access instruction; displaying the preset storage space Providing, to the first device, a corresponding application stored in the storage space, the application program is executable by the first device, and the application program is capable of acquiring current audio and video information of the first device system when the application is running;

If it is detected that the communication protocol port for transmitting the information is a port for transmitting audio and video data, the information is recognized as audio and video data; and the audio and video data is forwarded to the second device through the second USB interface of the local end, indicating The second device plays the corresponding audio and video after receiving the audio and video data.

Another aspect of the present invention provides a method for implementing a plug and play screen, comprising:

Identifying, by a pre-installed universal drive, a communication protocol port provided by a USB device that accesses the USB interface of the first device; the universal driver provides a universal communication protocol for communication between the first device and the standard USB device, the communication protocol port being at least The port includes: a port for transmitting audio and video data and a port for accessing the storage space; the USB device is provided with a first USB interface, and the USB device accesses the first device USB interface through the first USB interface, and Providing, by the first USB interface, a plurality of communication protocol ports that can be recognized by the universal drive of the first device;

Generating a storage space access instruction, adapting the access instruction to the identified port for accessing the storage space, and transmitting the first USB interface to the USB device through the first device USB interface to access the USB device a preset storage space, and running a corresponding application in the preset storage space through the first device system interface to intercept the picture and audio played by the first device;

The intercepted audio and video data is sent to the first USB interface of the USB device through the USB interface of the first device.

Another aspect of the present invention provides an apparatus for implementing a plug and play screen, including:

a port simulation module, configured to provide, by the first USB interface of the local end, a plurality of communication protocol ports, wherein the communication protocol port is recognized by a universal drive of the first device, where the universal drive provides the first device and the standard USB a universal communication protocol for communication between devices, the communication protocol port including at least a port for transmitting audio and video data and a port for accessing a storage space;

a connection establishing module, configured to establish a USB communication connection between the local end and the first device by using the communication protocol port and the first USB interface;

a storage module, configured to detect that the first USB interface has information from the first device, detect a communication protocol port that transmits the information, and if it is a port for accessing the storage space, identify the information as a storage space access instruction; The preset storage space is displayed to the first device, where the corresponding storage program is stored, the application program can be run by the first device, and the application program can acquire the current audio and video of the first device system information;

And the audio/video data forwarding module is configured to: if the communication protocol port that transmits the information is detected as a port for transmitting audio and video data, identify the information as audio and video data; and pass the audio and video data through the local end The two USB interfaces are forwarded to the second device, and the second device is instructed to play the corresponding audio and video after receiving the audio and video data.

a USB identification module, configured to identify, by a pre-installed universal driver, a communication protocol port provided by a USB device that accesses a first device USB interface; the universal driver provides a universal communication protocol for communication between the first device and the standard USB device, The communication protocol port includes at least a port for transmitting audio and video data and a port for accessing the storage space; the USB device is provided with a first USB interface, and the USB device accesses the first USB interface a device USB interface, and through the first USB interface, provides a plurality of communication protocol ports that can be recognized by the universal drive of the first device;

And an audio and video intercepting module, configured to generate a storage space access instruction, adapt the access instruction to the identified port for accessing the storage space, and send the first USB interface to the USB device through the first device USB interface Accessing the preset storage space of the USB device, and running a corresponding application in the preset storage space through the first device system interface to intercept the picture and audio played by the first device;

And an audio and video sending module, configured to send the intercepted audio and video data to the first USB interface of the USB device through a USB interface of the first device.

In the above technical solution, after the screen device is simulated as a standard USB device, and an application for capturing the picture and audio played by the display source device is pre-stored therein, after the screen device is connected to the display source device USB, the display source device passes The application running the storage space of the screen device can complete the data capture of the picture and audio played by the display source device. For the display source device, there is no need to pre-install the corresponding application, and the plug-and-play USB screen is realized, which is beneficial to the promotion of the USB screen application.

DRAWINGS

1 is a schematic diagram of a system environment for implementing a method for implementing plug and play screen transmission according to the present invention;

2 is a schematic flowchart of a method for implementing a plug and play screen according to an embodiment;

3 is a detailed flowchart of a method for implementing a plug and play screen according to an embodiment;

4 is a schematic flowchart of a method for implementing a plug and play screen in a second embodiment;

5 is a detailed flowchart of a method for implementing a plug-and-play screen display by a display source device according to a second embodiment;

6 is a schematic flowchart of a method for implementing a plug and play screen in a third embodiment;

7 is a detailed flowchart of a method for implementing a plug-and-play screen display by a display device of a third embodiment;

FIG. 8 is a schematic structural diagram of an apparatus for implementing a plug-and-play screen according to an embodiment; FIG.

FIG. 9 is a schematic structural diagram of an apparatus for implementing a plug-and-play screen of a second embodiment; FIG.

Fig. 10 is a schematic structural view showing an apparatus for realizing a plug and play screen of the third embodiment.

detailed description

The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

1 is a schematic diagram of a system environment for implementing a method for wired screen transmission of the present invention. As shown in FIG. 1 , the system environment of the embodiment of the present invention includes a display source device (ie, a first device), a screen device (ie, an analog USB device), and a display device (ie, a second device). The display source device may be a device or a system running an operating system such as a PC or a mobile phone, and the system has at least one pre-installed universal driver therein, and the universal driver provides universal communication between the first device and the standard USB device. letter of agreement. The display source device is further provided with at least one USB interface (ie, a USB interface D/third USB interface), and the attribute of the USB interface is set to a USB host. The display device is a device or system running an operating system, such as a large-screen electronic whiteboard, and the display device is further provided with at least one USB interface (ie, a USB interface A/fourth USB interface), and the attribute of the USB interface is set to USB host.

The screen device is provided with a USB interface, a memory, and a processor. The USB interface includes a USB interface C and a USB interface B (ie, a first USB interface and a second USB interface), and the attributes of the two USB interfaces are all USB devices, which are respectively connected to the display source device and the display device. The embedded device can run an embedded operating system and connect to the display source device through the USB interface C. The USB interface B can be an ordinary USB interface for accessing a display device such as a smart tablet (or other large-screen device). , driven and identified by the system of the display device. The screen device can be combined with a USB cable to form a custom special USB cable, or it can be a combination of a USB dongle with a screen device and a standard USB cable. One of the uses of the memory is a downloader that requires a specific program or a specific program for storing a display source device or a display device. The screen device can be powered directly through the USB interface of the display source device or the USB interface of the display device.

FIG. 2 is a schematic flowchart of a method for implementing a plug-and-play screen according to an embodiment. In this embodiment, the method is applied to a screen device as an example. The screen device includes a first USB interface and a second USB interface, respectively, for accessing the display source device and the display device.

As shown in FIG. 2, the method for implementing plug and play screen transmission in this embodiment includes the following steps:

S11. The first device is provided with a plurality of communication protocol ports by using a first USB interface of the local end, where the communication protocol port can be recognized by a universal driver of the first device, where the universal driver provides communication between the first device and the standard USB device. a universal communication protocol, the communication protocol port comprising at least a port for transmitting audio and video data and a port for accessing the storage space;

In this embodiment, the first device is a display source device, and the local device refers to a screen device. The screen device can be emulated into several types of standard USB devices and provides a corresponding communication protocol port. The generic driver pre-installed on the display source device can independently drive different types of standard USB devices. The standard USB device is simulated by the screen device, and the display source device can be recognized by the drive (ie, it can be recognized by the universal drive without a specific universal drive).

In this embodiment, the USB interface is a physical concept, and the communication protocol port is an abstract concept for the USB connection service. Different communication protocol ports can be distinguished by corresponding port identifiers, for example, by port number. In this embodiment, the port identifiers of the communication protocol ports can be recognized by the universal USB driver, and the processor in the screen device also performs various types of information (including audio and video data or control signals) through the port identifier of the communication protocol port. Transceiver, to achieve the transmission and reception of a variety of different types of information through the same physical interface.

Correspondingly, the screen device can be simulated into several types of standard USB devices, which means that the screen device can enumerate several standard USB communication protocol ports based on the common standard USB communication protocol, and different USB communication protocol ports use different port identifiers. Make a distinction.

S12. Establish a USB communication connection between the local end and the first device by using the communication protocol port and the first USB interface.

Preferably, the communication protocol port further includes: a port for transmitting connection information. After the screen device confirms that the USB communication connection between the local end and the first device is established, a connection success instruction may also be generated; the connection success instruction is adapted to the port for transmitting the connection information, and the first USB is passed. The interface is sent to the first device.

S13, detecting that the first USB interface has information from the first device, detecting a communication protocol port that transmits the information, and if the port is used to access the storage space, identifying the information as a storage space access instruction; The space is displayed to the first device, where the storage device stores a corresponding application, the application can be run by the first device, and the application can obtain the current audio and video information of the first device system when the application is running;

S14, if it is detected that the communication protocol port that transmits the information is a port for transmitting audio and video data, the information is recognized as audio and video data; and the audio and video data is forwarded to the second device through the second USB interface of the local end. Instructing the second device to play the corresponding audio and video after receiving the audio and video data.

In this embodiment, the second device is a display device. If the display source device does not have a software program for intercepting the currently displayed display screen and audio, the data stored in the memory of the screen device can be read or used to intercept the current by accessing the memory (storage space) of the screen device. The software program for playing the display screen and audio (the screen device is equivalent to an external USB flash drive), and then the packaged audio and video data is sent to the screen device. This method has lower requirements on the display source device, and even if it does not have the corresponding program installed in advance, the related function can be realized by accessing the memory of the screen device, and for the display source device (for example, a PC), the screen is inserted into the USB interface. When the device is in use, it will see a "removable disk" in the "My Computer" root directory, which is the storage space of the screen device, thereby indicating that the source device can run pre-stored from the "removable disk". The application, in turn, achieves a plug-and-play effect. Pass After the audio and video data received by the screen device from the USB interface C is forwarded to the display device through the USB interface B, the screen display of the audio and video data is realized.

In this embodiment, a corresponding client software is installed in the display source device for receiving a screen operation of the user. As a preferred implementation manner, the communication protocol port further includes: a port for transmitting a manipulation signal. After the connection between the display source device and the screen device is successful, if it is detected that the second USB interface of the screen device has a manipulation signal from the display device, the manipulation signal is adapted to the port for transmitting the control information, and passes The first USB interface of the screen device is sent to the display source device to cause the display source device to perform an action corresponding to the manipulation signal. For example, the screen device converts the received touch event from the display device into a HID touch event, which is simulated as a HID touch device, and submits the HID touch event to the display source device system through the USB interface C, for the display source device system. Said, the way to handle the touch events of HID touch devices as usual. The type of the control signal includes: a touch control signal, an infrared remote control signal, or a mouse/keyboard control signal; and the corresponding action of the control signal may be any action that can be performed on the display source device side.

Based on the above method for implementing plug and play screen transmission, it is not necessary to display the source device to pre-install the corresponding application program, and only need to connect the corresponding screen transmission device, thereby realizing the effect of plug and play.

It can be understood that the port for transmitting the connection information, the port for transmitting the audio and video information in the embodiment may be the HID1 port and the HID2 port respectively, and the port for transmitting the storage space access instruction may be the Mac type port. In other words, in the embodiment, the screen device can enumerate a plurality of communication protocol ports that can be recognized by the display source terminal by the USB interface C according to actual needs, and can be independently driven by the display source terminal. Drivers are used to transmit different types of data information.

FIG. 3 is a detailed flowchart of a method for implementing a plug-and-play screen according to an embodiment; referring to FIG. 3, the steps to be performed by the screen device are as follows:

S101: After the screen device processor program starts running, the screen device is simulated into a standard USB device through the USB interface C, so that the display source device with the universal driver can be driven to recognize it.

S102. Check whether the USB interface C has a display source device that meets the requirements;

Detect whether the USB interface C has a device connection, and if so, further check whether the device is a display source device that meets the requirements.

S103. Detect whether the USB interface B has a display device that meets the requirements;

Detect whether the USB interface B has a device connection, and if so, further check whether the device is a display device that meets the requirements.

It can be understood that the execution order of steps S102 and S103 is not limited to the example in the figure, and step S103 may be performed first, and then step S102 may be performed; or two parts may be simultaneously executed.

S104. Generate a connection success instruction, and confirm that the screen connection is successful.

In this embodiment, the connection success instruction is adapted to a port for transmitting connection information, and is sent to the first device through the USB interface C; meanwhile, the display device is sent through the USB interface B based on a protocol rule predetermined with the display device. A successful connection command is issued.

S105. Check whether the USB interface C has an audio and video file stream from the display source device.

In this embodiment, after detecting that the USB interface C has the information from the display source device, the screen device further detects the communication protocol port that transmits the information, and if it is a port for transmitting audio and video data, the information may be identified as Audio and video data.

In this embodiment, the screen device further provides a port for accessing the storage space to the display source device. Before this step, if the screen device After detecting that the USB interface C has information from the display source device, further detecting the communication protocol port that transmits the information, if the port is used to access the storage space, the information is recognized as a storage space access instruction, and the screen device is pre- The storage space is displayed to the display source device, so that the display source device system interface can be used to intercept the screen and audio played by the display source device by running a corresponding program in the storage space. The storage space of the screen device stores an exe executable program for intercepting audio and video data, or stores download information of the corresponding program. Therefore, the display source device is configured to access the currently played screen or audio by accessing information of the storage space of the screen device.

S106. Forward the audio and video file stream received by the USB interface C to the display device through the USB interface B.

In this embodiment, after the audio and video data received by the screen device, the received audio and video data can be identified according to the communication protocol port used by the screen device, and forwarded to the second device through the USB interface B.

As a preferred embodiment, after the screen device recognizes that the audio and video data is received, it also monitors whether there is a preset trigger control signal for indicating information forwarding, and if the trigger control signal is monitored, The audio and video data is forwarded to the second device through the USB interface B. The trigger control signal may be an operation signal that the preset button of the screen device is pressed, or may be a specific touch control signal generated on the human-machine interface provided by the screen device, that is, the user may trigger through hardware. / Software trigger mode to control the forwarding of audio and video data by the screen device to flexibly control the effect of the screen display.

S107. Detect whether the USB interface C has a return request instruction from the display source device.

In this embodiment, if the screen device detects that the USB interface C has information from the display source device, detecting a communication protocol port that transmits the information, if the port is used for transmitting the return request, the information may be identified as back. Pass the request instruction.

S108. Forward the received backhaul request command to the display device through the USB interface B.

It can be understood that the triggering mechanism for forwarding audio and video data is also applicable to triggering the forwarding of the return request command by the screen device, that is, the hardware trigger mode/software trigger mode can be used to control the transmission device to return the transmission. Request forwarding of instructions.

In this embodiment, the screen transmitting device sends the received backhaul request command to the display device through the USB interface B according to the protocol rule agreed with the display device.

It can be understood that the execution order of steps S105 and S106, steps S107 and S108 is not limited to the example in the figure, and steps S107 and S108 may be performed first, and then steps S105 and S106 may be performed; or both parts are set according to respective time intervals. Execute in parallel.

S109. Detect whether the USB interface B has a control signal from the display device.

In this embodiment, the screen device can detect whether the information received by the USB interface B from the display device is a control signal according to a protocol rule agreed with the display device.

S110, the control signal is adapted to a corresponding port provided by the screen device for displaying the source device for transmitting control information, and sent to the display source device through the USB interface C, so that the display source device performs the control signal. Corresponding actions, thereby achieving the effect of manipulating the display source device on the display device side.

It can be understood that the trigger mechanism for forwarding audio and video data is also applicable to triggering the forwarding of the control signal by the screen device, that is, the hardware trigger mode/software trigger mode can be used to control the forwarding of the control signal by the screen device. To flexibly control the effect of multi-end manipulation.

The wired screen method of the embodiment provides a display source device with a plurality of communication protocol ports through a USB interface of the simulated USB device by emulating a standard USB device in advance to establish a display source device and display through the simulated USB device. a wired screen connection of the device; when detecting that the first USB interface of the simulated USB device has information from the display source device, detecting a communication protocol port transmitting the information, if it is a port for transmitting audio and video data, The information is identified as audio and video data; the audio and video data is forwarded to the display device through the second USB interface of the local end to play the corresponding audio and video on the display device; when the second USB interface of the simulated USB device is detected When there is a manipulation signal from the display device, the manipulation signal is adapted to a port for transmitting control information, and is sent to the display source device through the first USB interface, so that the display source device performs corresponding to the manipulation signal The action thus achieves the effect of manipulating the display source device on the display device side. Based on the analog USB device, the display source device and the display device can realize the transmission and display of audio and video data only by one connection line, and at the same time, realize the return of the control signal of the display device, simplify the wiring of the screen display, and reduce the pair. User wiring is difficult.

4 is a schematic flowchart of a method for implementing plug and play screen transmission in the second embodiment; the embodiment is applicable to a display source device. As shown in FIG. 4, the method for implementing the plug and play screen in this embodiment includes the following steps.

Step S21: The universal driver pre-installed by the first device (display source device) identifies a communication protocol port provided by the currently accessed USB device (screening device), and the universal driver provides a connection between the first device and the standard USB device. a general communication protocol for communication, the communication protocol port at least comprising: a port for transmitting audio and video data and a port for accessing a storage space; the USB device is provided with a first USB interface, and the USB device passes the a USB interface is connected to the first device USB interface, and provides a plurality of communication protocol ports that can be recognized by the universal drive of the first device through the first USB interface;

In this embodiment, the USB interface C of the screen device is connected to the USB interface D of the display source device, and the attribute of the USB interface D is a USB host; the display source device refers to a device used as a display source.

Step S22: Generate a storage space access instruction, adapt the access instruction to the identified port for accessing the storage space, and send the first USB interface to the USB device through the first device USB interface to access a preset storage space of the USB device, and running a corresponding application in the preset storage space by using a first device system interface to intercept a picture and audio played by the first device;

Specifically, for example, generating a storage space access instruction according to a display source device user operation, adapting the access instruction to a port for accessing a storage space, and sending the USB interface C to the USB device to access the USB device Presetting the storage space, and running a corresponding program in the preset storage space through the system interface of the display source device for intercepting the picture and audio played by the display source device. Therefore, for the display source device (for example, a PC), when the USB interface is inserted into the screen device, a "removable disk" is seen in the "My Computer" root directory, that is, the storage space of the screen device. Therefore, the display source device can run the pre-stored application program from the "removable disk", thereby realizing the plug-and-play effect, and the user does not need to pre-install the corresponding program on the display source device, thereby providing convenience for the user.

Step S23: Send the intercepted audio and video data to the first USB interface of the USB device through a corresponding port of the USB interface of the first device, where the corresponding port is adapted to the port for transmitting audio and video data;

In this embodiment, the communication protocol port further includes: a port for transmitting a control signal. The implementation of this embodiment is a plug-and-play screen The method further includes: if detecting that the second USB interface has a manipulation signal from the second device, adapting the manipulation signal to a port for transmitting control information, and transmitting to the first device through the first USB interface, So that the first device performs an action corresponding to the manipulation signal.

Preferably, the display source device user can select whether to need to enable the control signal return function according to actual needs, and if necessary, send a return request to the display device through the screen device after the connection is successfully established, correspondingly, the communication The protocol port also includes a port for transmitting a backhaul request. The specific manner is as follows: obtaining the return request identifier bit information preset by the display source device, determining whether the return request function of the display source device is the open state according to the return request identifier bit information; if yes, generating a return request request; The return request command is adapted to the port for transmitting the return request, and is sent to the USB interface C of the screen device, and sent to the display device through the USB interface B of the screen device.

FIG. 5 is a detailed flowchart of a method for implementing plug and play screen transmission in the second embodiment; referring to FIG. 5, the steps to be performed on the source device are as follows:

S201, running a universal driver for displaying the source device, and detecting, in the background of the system, whether a USB device (ie, a screen device) having a specific ID (PID/VID) conforming to the characteristics of the screen device is accessed;

S202, if yes, confirm whether there is a connection success instruction sent by the screen transmission device;

In this embodiment, the universal driving of the display source device detects the information of the USB interface C from the screen device received by the USB interface D of the display source device, and detects the communication protocol port that transmits the information, if it is used to transmit the connection information. The port can identify whether the information is a connection success command according to the content of the signal.

S203, if received, intercepting a picture and/or audio currently displayed by the display source device, and then compressing or encoding into a specific file format, and then transmitting the image to the connected screen device;

Preferably, the display source device can identify a port provided by the screen device for accessing the storage space, adapt the access instruction to the port for accessing the storage space, and send the USB interface C to the screen device, The preset storage space of the screen device is accessed, and the corresponding program in the preset storage space is run through the system interface of the display source device for intercepting the picture and audio played by the display source device. Therefore, the display source device does not need to install the corresponding program in advance, which provides convenience for the user.

The audio and video data may also be compressed and/or encoded by a program that displays the default storage space of the screen device by displaying the system interface of the source device for transmission. Because the interception code is stored in a U disk emulated by the screen device, after the display source device is connected to the screen device, the USB communication protocol port pre-simulated through the USB interface of the screen device can be viewed and stored in the screen device. The exe program data can intercept the currently playing screen and/or audio (including sound data, including audio adjustment information such as volume and channel) by running the relevant program in the screen device.

S204, confirming whether the return request function switch of the display source device is turned on (the user can select to open or close);

In this embodiment, after the connection is successful, the backhaul request identifier information preset by the display source device may be read to determine whether the current backhaul request function of the display source device is in an open state.

S205, if opened, sending a return request command to the screen device;

If the current backhaul request function of the source device is displayed as being on, generating a return request command according to the user operation; providing from the screen device The port for transmitting the return request is obtained from the communication protocol port of the display source device; and is sent to the USB interface C of the screen device, and sent to the display device through the USB interface B of the screen device.

S206, detecting whether there is a control signal sent back by the USB interface C of the screen transmission device;

S207. If receiving a control signal sent back by the screen device, performing a corresponding response action.

In this embodiment, the USB interface D is detected to receive the information of the USB interface C from the screen device, and the communication protocol port for transmitting the information is detected. If the port is used for transmitting the control signal, the external HID device can be controlled according to the conventional processing. The manner of the signal, identifying the returned control signal, and causing the first device to perform an action corresponding to the manipulation signal.

It can be understood that the execution order of steps S204 and S205 and steps S206-S207 is not limited to the example in the figure, and steps S206-S207 may be performed first, and then steps S204 and S205 may be performed; or both parts are set according to respective time intervals. Execute in parallel.

The wired screen method of the embodiment, after simulating the screen device as a standard USB device, and pre-storing an application for intercepting the screen and audio played by the display source device, after the screen device is connected to the display source device by USB The display source device can complete the data capture of the picture and audio played by the display source device by running the application program of the storage space of the screen device. For the display source device, there is no need to pre-install the corresponding application, and the plug-and-play USB screen is realized, which is beneficial to the promotion of the USB screen application; and the display source device and the display device all need only one connection line. Realizing the transmission and display of audio and video data, and at the same time, realizing the return of the control signal of the display device, realizing the effect of manipulating the display source device on the display device side, simplifying the wiring of the screen display, and reducing the difficulty of wiring the user.

FIG. 6 is a schematic flowchart of a method for implementing plug and play screen transmission according to a third embodiment; the embodiment is applicable to a display device. As shown in FIG. 6, the method for implementing the plug and play screen in this embodiment includes the following steps.

S31. The USB device of the second device is detected to have a USB device access, and the USB device is identified as a USB device that meets the requirements. The USB device is configured with a second USB interface, and the USB device is connected through the second USB interface. Into the second device USB interface;

In this embodiment, it is detected that the USB interface A of the display device has a USB device access, and whether the current access device is the above-mentioned screen device can be determined according to the PID and/or VID of the USB device. The display device refers to a device used to display audio and video information, and the attribute of the USB interface A is a USB host.

S32. If the USB device meets the requirements, receive the audio and video data forwarded by the second USB interface of the USB device through the USB interface of the second device, and play the corresponding audio and video on the second device;

Preferably, after detecting that the USB interface A of the display device has the required screen device access, the connection response signal may also be generated and sent to the USB interface B of the screen device to notify the screen device that the device matches the sound. The requirement of the video transmission display, the screen device can confirm whether the connection is successful according to the information.

In this embodiment, in order to facilitate the transmission, the audio and video data is compressed and/or encoded at the display source end, and the corresponding display device needs to perform corresponding decompression and/or de-compression processing on the audio and video data, and the processing is performed. The corresponding picture and/or audio is then played on the display device.

S33. Acquire a control signal of the first device detected by the second device, and send the control signal to a second USB interface of the USB device, to return to the first device by using the first USB interface of the USB device. Instructing the first device to receive the Performing an action corresponding to the manipulation signal after the signal is manipulated;

The type of the control signal in this embodiment may include: a touch control signal, an infrared remote control signal, or a mouse/keyboard control signal; and the corresponding action of the manipulation signal may be any action that can be performed on the display source device side.

7 is a detailed flowchart of a method for implementing plug and play screen transmission in the third embodiment; referring to FIG. 7, the steps to be performed by the display device are as follows:

S301. After the display device system program runs, it detects whether a USB device (ie, a screen device) with a specific ID is accessed.

In this embodiment, the ID is a USB ID (PID or VID) of the screen device preset in the program; therefore, it is detected whether the screen device is connected to the USB interface A of the display device.

S302. If the transmission of the screen transmission device is detected, the specified format data information is sent through the USB interface A, and the local end of the screen transmission device is notified to meet the requirements of the audio and video transmission display;

S303. Detect whether the audio interface file of the USB interface B of the screen transmission device is transmitted.

Since the display device can make a customized terminal, the signal type transmitted by the USB interface B of the screen device can be automatically recognized. That is, the display device can recognize whether the transmission device sends the audio and video data, or the request for returning the request, or the instruction for successful connection.

In this embodiment, the audio and video file stream is data subjected to compression and encoding processing.

S304. If the audio and video file stream transmitted by the screen transmission device is received, the audio and video data is decompressed or decoded, and the corresponding picture and audio are outputted, thereby realizing the screen display of the display device and the display source device.

Preferably, the display device can also return the preset message to the screen device, and forward it to the display source device through the communication protocol port pre-simulated by the screen device to notify the display source device that the screen display is successful.

S305. Detect whether the USB interface B of the screen transmission device has a control signal return request transmitted.

In this embodiment, the display device can detect information from the USB interface B of the screen device, and identify whether the information identifies whether it is a backhaul request instruction.

S306. If a control signal return request is detected, acquiring a control signal to the display source device detected by the local end, and sending the control signal to the screen transmission device.

The display device acquires a control signal detected by the local end for controlling the display source device by running a preset program, and the signal type includes but is not limited to a touch control signal, an infrared remote control signal, or a mouse/keyboard control signal.

Through the method of implementing the plug-and-play screen in the above embodiment, the screen device can be simulated into several types of USB devices through its USB interface, so that the display source device can be protected from special drive recognition. The screen device also obtains a data stream of the manipulation signal from the display device and adapts it to the corresponding port provided to the display source device for transmission; the code for intercepting the audio and video data is a system interface for displaying the source device, but the corresponding exe program The data can be saved in a USB flash drive emulated by the screen device. The screen device is also emulated as a USB storage device, storing program data or a program downloader for data communication and intercepting audio and video data of the display source device. After the display source device establishes a USB connection with the screen device, the USB storage device simulated by the screen device can be accessed to run the corresponding exe program data for intercepting the screen or audio of the display source device. The display source device may also package the audio and video data according to the agreed protocol by running the corresponding exe program data stored in the screen device, and transmit the corresponding communication protocol port provided to the display source device to the screen transmission device through the screen device, and the screen device The data packet is received from the USB interface C, and the content of the package is not changed, and is directly forwarded to the display device through the USB interface B. The display device receives the data packet, parses the protocol and the content of the packet, and then decodes the audio and video data for playback.

The display device software can also obtain a touch operation signal (or other type of control signal) that the user touches the display source device through the display device, and transmit the signal to the screen transmission device through the USB interface A, and the screen device adapts the operation signal to the corresponding communication. The protocol port is then submitted to the display source device for execution. The display source device identifies the transmitted touch operation signal according to the corresponding communication protocol port provided by the USB interface C of the screen transmission device, and converts it into a manipulation event (the touch operation signal is converted into a touch event. This process is to display the operating system support of the source device. The WINDOWS/MAC has this function), thereby causing the display source device to perform an action corresponding to the manipulation signal.

It should be noted that, for the foregoing method embodiments, for the sake of brevity, they are all described as a series of action combinations, but those skilled in the art should understand that the present invention is not limited by the described action sequence, because In accordance with the present invention, certain steps may be performed in other sequences or concurrently.

Based on the same idea as the method for implementing plug and play screen in the above embodiment, the present invention also provides an apparatus and system for implementing plug and play screen, and the device and system can be used to implement the above-mentioned implementation of plug and play screen. Methods. For the convenience of the description, in the structural diagram of the device and the system embodiment for implementing the plug-and-play screen, only the parts related to the embodiment of the present invention are shown, and those skilled in the art can understand that the illustrated structure does not constitute the device. And the definition of the system may include more or less components than those illustrated, or some components may be combined, or different component arrangements.

FIG. 8 is a schematic structural diagram of an apparatus for implementing a plug-and-play screen according to an embodiment of the present invention. As shown in FIG. 8, the apparatus for implementing a plug-and-play screen in this embodiment includes: a port simulation module 510, The connection establishing module 520, the storage module 530, and the audio and video data forwarding module 540 are detailed as follows:

The port emulation module 510 is configured to provide, by using a first USB interface of the local end, a plurality of communication protocol ports to the first device, where the communication protocol port can be identified by a universal drive of the first device, where the universal drive provides the first device a universal communication protocol for communication with a standard USB device, the communication protocol port comprising at least: a port for transmitting audio and video data and a port for accessing the storage space;

The connection establishing module 520 is configured to establish a USB communication connection between the local end and the first device by using the communication protocol port and the first USB interface;

The storage module 530 is configured to detect that the first USB interface has information from the first device, and detect a communication protocol port that transmits the information. If the port is used to access the storage space, the information is identified as a storage space access. An instruction to display a preset storage space to a first device, wherein the storage space stores a corresponding application, the application program can be run by the first device, and the application program can obtain the current current state of the first device system Audio and video information;

The audio and video data forwarding module 540 is configured to detect that the first USB interface has information from the first device, detect a communication protocol port that transmits the information, and if the port is used for transmitting audio and video data, identify the information. The audio and video data is forwarded to the second device through the second USB interface of the local end, and the second device is instructed to play the corresponding audio and video after receiving the audio and video data;

As a preferred implementation manner, after the audio/video data forwarding module 540 identifies the information as audio and video data, it also monitors whether The preset trigger signal for indicating information forwarding is forwarded to the second device through the second USB interface of the local end if the preset trigger control signal for indicating information forwarding is monitored. The trigger control signal includes an operation signal that the preset control button is pressed, and may further include a touch control signal that is monitored by the preset human-machine interaction interface. This makes it easy for the user to flexibly control the effect of the screen display.

As a preferred embodiment, the communication protocol port further includes: a port for transmitting control information; the device for implementing the plug and play screen in the embodiment may further include: a control signal returning module 550, configured to detect The second USB interface has a control signal from the second device, and the control signal is adapted to a port for transmitting control information, and is sent to the first device through the first USB interface, indicating that the first device is receiving An action corresponding to the manipulation signal is performed after the manipulation signal.

Preferably, the communication protocol port further includes: a port for transmitting connection information; correspondingly, the device for implementing the plug and play screen further comprises: a connection confirmation module (not shown), configured to: A connection success instruction is generated, the connection success instruction is adapted to a port for transmitting connection information, and sent to the first device through the first USB interface.

Preferably, the communication protocol port further includes: a port for transmitting a return request; correspondingly, the device for implementing the plug and play screen further comprises: a backhaul request module (not shown), For detecting that the first USB interface has information from the first device, detecting a communication protocol port that transmits the information, and if it is a port for transmitting the return request, identifying the information as a backhaul request instruction; The return request command is forwarded to the second device through the second USB interface, so that the second device returns its detected control signal to the first device.

In this embodiment, the attributes of the first USB interface and the second USB interface are both USB devices; the first device is a display source device, the second device is a display device; and the port for transmitting audio and video data is hid Type port, the port used to access the storage space is the msc type port.

Fig. 9 is a schematic structural view showing an apparatus for realizing a plug and play screen of the second embodiment. As shown in FIG. 9, the apparatus for implementing the plug and play screen in this embodiment includes:

The USB identification module 610 is configured to identify, by using a pre-installed universal driver, a communication protocol port provided by a USB device that accesses the USB interface of the first device, where the universal driver provides a universal communication protocol for communication between the first device and the standard USB device. The communication protocol port at least includes: a port for transmitting audio and video data and a port for accessing the storage space; the USB device is provided with a first USB interface, and the USB device is accessed through the first USB interface a first device USB interface, and providing, by the first USB interface, a plurality of communication protocol ports that can be recognized by the universal drive of the first device;

The audio and video intercepting module 620 is configured to generate a storage space access instruction, adapt the access instruction to the identified port for accessing the storage space, and send the first USB to the USB device through the first device USB interface The interface is configured to access a preset storage space of the USB device, and run a corresponding application in the preset storage space through the first device system interface to intercept a picture and audio played by the first device;

The audio and video sending module 630 is configured to send to the first USB interface of the USB device through a designated port of the USB interface of the first device, where the designated port is adapted to be used for transmitting audio and video data.

As a preferred embodiment, the communication protocol port further includes a port for transmitting a manipulation signal. The device for implementing the plug and play screen further includes: a control signal receiving module 640, configured to receive information from the first USB interface of the USB device, and detect the transmission The communication protocol port that inputs the information; if it is a port for transmitting the manipulation signal, the information is recognized as a manipulation signal, and the first device performs an action corresponding to the manipulation signal.

In this embodiment, the first device is a display source device; and the attribute of the first USB interface is a USB device.

Fig. 10 is a schematic structural view showing an apparatus for realizing a plug and play screen of the third embodiment. As shown in FIG. 10, the apparatus for implementing the plug and play screen in this embodiment includes:

The USB access detection module 710 is configured to detect that the USB device of the second device has a USB device access, and identify whether the USB device is a USB device that meets the requirements; the USB device is configured with a second USB interface, and the USB device passes The second USB interface is connected to the USB interface of the second device;

The audio and video data receiving module 720 is configured to receive audio and video data forwarded by the second USB interface of the USB device through the second device USB interface, and play the corresponding audio and video on the second device, if the USB device meets the requirements;

And a control signal sending module 730, configured to acquire a control signal of the first device detected by the second device, and send the control signal to a second USB interface of the USB device to pass through the USB device The first USB interface returns to the first device, instructing the first device to perform an action corresponding to the manipulation signal after receiving the manipulation signal.

In this embodiment, the second device is a display device, and the attribute of the second USB interface is a USB device.

It should be noted that, in the implementation of the apparatus for implementing the plug-and-play screen of the above example, the information interaction, the execution process, and the like between the modules/units are based on the same concept as the foregoing method embodiments of the present invention. The technical effects are the same as those of the foregoing method embodiments of the present invention. For details, refer to the description in the method embodiment of the present invention, and details are not described herein again.

In addition, in the implementation of the apparatus for implementing the plug-and-play screen of the above example, the logical division of each functional module is only an example, and the actual application may be implemented according to requirements, for example, according to the configuration requirements of the corresponding hardware or the implementation of the software. Conveniently, the above-mentioned function allocation is completed by different functional modules, that is, the internal structure of the device for realizing the plug and play screen is divided into different functional modules to complete all or part of the functions described above. Each function module can be implemented in the form of hardware or in the form of a software function module.

It will be understood by those skilled in the art that all or part of the processes in the above embodiments may be implemented by a computer program to instruct related hardware, and the program may be stored in a computer readable storage medium as Independent product sales or use. The program, when executed, may perform all or part of the steps of an embodiment of the methods described above. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

In the above embodiments, the descriptions of the various embodiments are all focused, and the parts that are not detailed in a certain embodiment can be referred to the related descriptions of other embodiments. It will be understood that the terms "first", "second", and the like, as used herein, are used herein to distinguish an object, but these are not limited by these terms.

The above-described embodiments are merely illustrative of several embodiments of the invention and are not to be construed as limiting the scope of the invention. It should be noted that a number of variations and modifications may be made by those skilled in the art without departing from the spirit and scope of the invention. Therefore, the scope of the invention should be determined by the appended claims.

Claims

A method for implementing plug and play screening, comprising:

Providing, by the first USB interface of the local end, a plurality of communication protocol ports to the first device, the communication protocol ports being identifiable by the universal drive of the first device; the universal drive providing universal communication between the first device and the standard USB device a communication protocol, the communication protocol port at least comprising: a port for transmitting audio and video data and a port for accessing the storage space;

Establishing a USB communication connection between the local end and the first device by using the communication protocol port and the first USB interface;

Detecting that the first USB interface has information from the first device, detecting a communication protocol port that transmits the information, and if it is a port for accessing the storage space, identifying the information as a storage space access instruction; displaying the preset storage space Providing, to the first device, a corresponding application stored in the storage space, the application program is executable by the first device, and the application program is capable of acquiring current audio and video information of the first device system when the application is running;

If it is detected that the communication protocol port for transmitting the information is a port for transmitting audio and video data, the information is recognized as audio and video data; and the audio and video data is forwarded to the second device through the second USB interface of the local end, indicating The second device plays the corresponding audio and video after receiving the audio and video data.
The method for implementing wired screen transmission according to claim 1, wherein the communication protocol port further comprises: a port for transmitting connection information;

After the USB communication connection between the local device and the first device is established by using the communication protocol port and the first USB interface, the method further includes:

A connection success instruction is generated, the connection success instruction is adapted to a port for transmitting connection information, and sent to the first device through the first USB interface.
The method for implementing a plug and play screen according to claim 1, wherein the communication protocol port further comprises: a port for transmitting a control signal and a port for transmitting a return request;

The first USB interface is detected to have information from the first device, and after detecting the communication protocol port for transmitting the information, the method further includes:

If the port is used for transmitting the backhaul request, the information is identified as a backhaul request instruction; the backhaul request command is forwarded to the second device through the second USB interface, so that the second device returns the detected pair a control signal of the first device;

The method for implementing plug and play screen transmission further includes:

Detecting that the second USB interface has a control signal from the second device, adapting the control signal to a port for transmitting control information, and sending the first USB device to the first device, indicating that the first device is The action corresponding to the manipulation signal is performed after receiving the manipulation signal.
The method for implementing the plug and play screen according to claim 1, wherein the step of forwarding the audio and video data to the second device through the second USB interface of the local end comprises:

If the preset trigger control signal for indicating information forwarding is monitored, the audio and video data is forwarded to the second device through the second USB interface of the local end.
The method for realizing plug and play screen according to claim 4, wherein the trigger control signal comprises an operation signal that the preset control button of the local end is pressed.
The method for implementing the plug and play screen according to any one of claims 1 to 5, wherein the attributes of the first USB interface and the second USB interface are both USB devices;

The first device is a display source device, and the second device is a display device;

The port for transmitting audio and video data is a hid type port, and the port for accessing the storage space is an msc type port.
A method for implementing plug and play screening, comprising:

Identifying, by a pre-installed universal drive, a communication protocol port provided by a USB device that accesses the USB interface of the first device; the universal driver provides a universal communication protocol for communication between the first device and the standard USB device, the communication protocol port being at least The port includes: a port for transmitting audio and video data and a port for accessing the storage space; the USB device is provided with a first USB interface, and the USB device accesses the first device USB interface through the first USB interface, and Providing, by the first USB interface, a plurality of communication protocol ports that can be recognized by the universal drive of the first device;

Generating a storage space access instruction, adapting the access instruction to the identified port for accessing the storage space, and transmitting the first USB interface to the USB device through the first device USB interface to access the USB a preset storage space of the device, and running a corresponding application in the preset storage space through the first device system interface to intercept the picture and audio played by the first device;

The intercepted audio and video data is sent to the first USB interface of the USB device through the USB interface of the first device.
The method for implementing plug and play screen according to claim 7, wherein the communication protocol port further comprises: a port for transmitting a control signal and a port for transmitting a backhaul request;

After the communication protocol port provided by the USB device connected to the USB interface of the first device is identified by the pre-installed universal driver, the following further includes:

Obtaining a backhaul request identifier bit information preset by the first device, determining, according to the backhaul request identifier bit information, whether the backhaul request function of the first device is in an open state; if yes, generating a backhaul request instruction;

And adapting the backhaul request instruction to a port for transmitting a backhaul request, and sending the first USB interface to the USB device through a first device USB interface;

The method for implementing plug and play screen transmission further includes:

Receiving, by the first device USB interface, information from the first USB interface of the USB device, detecting a communication protocol port transmitting the information; if it is a port for transmitting a control signal, identifying the information as a manipulation signal, and The first device performs an action corresponding to the manipulation signal.
The method for implementing plug and play screen according to claim 7, wherein before the intercepting the audio and video data to the communication protocol port, the method further comprises:

Running, by the first device system interface, the identified port for accessing the storage space, the corresponding application in the preset storage space of the USB device, to intercept the picture and audio played by the first device, to intercept the captured audio and video The data is compressed and/or encoded.
The method for implementing the plug and play screen according to any one of claims 7 to 9, wherein the first device is a display source device, and the attribute of the first device USB interface is a USB host; the first USB interface The property is USB device.
An apparatus for implementing plug and play screen transmission, comprising:

a port analog module, configured to provide, by using a first USB interface of the local end, a plurality of communication protocol ports to the first device, where the communication protocol is The port can be identified by a universal drive of the first device, the universal drive providing a universal communication protocol for communication between the first device and the standard USB device, the communication protocol port comprising at least: a port for transmitting audio and video data and The port that accesses the storage space;

a connection establishing module, configured to establish a USB communication connection between the local end and the first device by using the communication protocol port and the first USB interface;

a storage module, configured to detect that the first USB interface has information from the first device, detect a communication protocol port that transmits the information, and if it is a port for accessing the storage space, identify the information as a storage space access instruction; The preset storage space is displayed to the first device, where the corresponding storage program is stored, the application program can be run by the first device, and the application program can acquire the current audio and video of the first device system information;

And the audio/video data forwarding module is configured to: if the communication protocol port that transmits the information is detected as a port for transmitting audio and video data, identify the information as audio and video data; and pass the audio and video data through the local end The two USB interfaces are forwarded to the second device, and the second device is instructed to play the corresponding audio and video after receiving the audio and video data.
The apparatus for implementing plug and play screen according to claim 11, wherein the communication protocol port further comprises: a port for transmitting connection information;

The device for implementing plug and play screen transmission further includes:

And a connection confirmation module, configured to generate a connection success instruction, adapt the connection success instruction to a port for transmitting connection information, and send the first connection to the first device through the first USB interface.
The apparatus for implementing plug and play screen according to claim 11, wherein the communication protocol port further comprises: a port for transmitting a manipulation signal and a port for transmitting a return request;

The backhaul request module is configured to detect that the first USB interface has information from the first device, and detects a communication protocol port that transmits the information; if the port is used for transmitting the return request, the information is identified as a backhaul request command Transmitting the return request command to the second device through the second USB interface, and causing the second device to return the detected control signal to the first device;

The device for implementing plug and play screen transmission further includes:

a control signal returning module, configured to detect that the second USB interface has a control signal from the second device, adapt the control signal to a port for transmitting control information, and send the first to the first USB interface And the device instructing the first device to perform an action corresponding to the control signal after receiving the control signal.
The device for implementing the plug and play screen according to any one of claims 11 to 13, wherein the attributes of the first USB interface and the second USB interface are all USB devices;

The first device is a display source device, and the second device is a display device
An apparatus for implementing plug and play screen transmission, comprising:

a USB identification module, configured to identify, by a pre-installed universal driver, a communication protocol port provided by a USB device that accesses a first device USB interface; the universal driver provides a universal communication protocol for communication between the first device and the standard USB device, The communication protocol port includes at least a port for transmitting audio and video data and a port for accessing the storage space; the USB device is provided with a first USB interface, and the USB device accesses the first USB interface a device USB interface and provided by the first USB interface a plurality of communication protocol ports that are identifiable by the universal drive of the first device;

And an audio and video intercepting module, configured to generate a storage space access instruction, adapt the access instruction to the identified port for accessing the storage space, and send the first USB interface to the USB device through the first device USB interface Accessing the preset storage space of the USB device, and running a corresponding application in the preset storage space through the first device system interface to intercept the picture and audio played by the first device;

And an audio and video sending module, configured to send the intercepted audio and video data to the first USB interface of the USB device through a USB interface of the first device.
The apparatus for implementing plug and play screen according to claim 15, wherein

The first device is a display source device, and the attribute of the first device USB interface is a USB host;

The attribute of the first USB interface is a USB device.