TWI826203B - Computer device and display device - Google Patents

Abstract

A computer device is provided, which includes a storage device and a processing unit. The storage device stores a screen-mirroring program. The processing unit executes the screen-mirroring program to perform the following steps: in response to detecting that a portable electronic device is connected to the computer device, pushing a software archive to the portable electronic device for automatic execution; receiving a video stream from the portable electronic device, wherein the video stream includes a screen image of the portable electronic device; decoding the video stream and performing image rendering on the decoded image to obtain the screen image that is displayed on the display device; receiving an input command from an input device connected to the computer device, and converting the input command to a control command used by the portable electronic device according to a key setting interface corresponding to an application executed by the portable electronic device.

Description

Computer devices and display devices

The present invention relates to screen mirroring technology, and in particular to a computer device and a display device.

Due to the advancement of technology, there are many mobile phone screen projection software on the market that allow users to project the screen of their portable electronic devices onto a monitor or TV. However, traditional mobile phone screen projection software requires users to install related applications on their portable electronic devices. In addition to complicated operating steps, this method also has the problem of updating application versions. In addition, traditional mobile phone screen projection software needs to complete function call synchronization. As a result, when the signal transmission is not completed, the software will wait for the signal to arrive, thus causing a delay in command operations.

Furthermore, traditional mobile phone screen projection software only supports signal transmission, and users cannot define their own keyboard and mouse operation function mapping, which cannot fully support mobile games that are mainly touch and click.

In view of this, the present invention provides a computer device and a display device to solve the above problems.

The invention provides a computer device, which includes a storage device and a processing unit. The storage device is used to store a screen mirroring program. The processing unit is used to execute the screen mirroring program to perform the following steps: in response to detecting that a portable electronic device is connected to the computer device, pushing a software package to the portable electronic device to perform the following steps on the portable electronic device: The device automatically executes the software package; receives a video stream from the portable electronic device, wherein the video stream includes a screen image of the portable electronic device; decodes the video stream and renders the decoded image to obtain the screen image, and playing the screen image on a display device; and receiving input commands from the input device connected to the computer device, and according to the corresponding key setting interface of an application executed by the portable electronic device. The input command is converted into a control command used by the portable electronic device.

In some embodiments, the universal serial bus (USB) debug mode of the portable electronic device is enabled, and the portable electronic device is connected to the computer device through a USB connection.

In some embodiments, the software package executed by the portable electronic device obtains the screen image from the playback image buffer of the portable electronic device and encodes the screen image into the video stream.

In some embodiments, the input instructions include key codes, text codes, mouse signals, scroll signals and touch signals.

In some embodiments, the software package executed by the portable electronic device transmits the control command to the touch screen of the portable electronic device to simulate the touch operation related to the input command, and in the application The program performs the touch operation.

In some embodiments, when the portable electronic device detects that the connection between the portable electronic device and the computer device is disconnected, the portable electronic device automatically deletes the software package.

The present invention further provides a display device, including: a display module; a universal serial bus (USB) hub connected to one or more input devices; and a display controller for performing the following steps: in response to detection Connect a portable electronic device to the USB hub, push a software package to the portable electronic device through the USB hub to automatically execute the software package on the portable electronic device; download the software package from the portable electronic device through the USB hub The portable electronic device receives a video stream, wherein the video stream includes a screen image of the portable electronic device; decodes the video stream and renders the decoded image to obtain the screen image, and displays the screen image on the portable electronic device. Play on the display module; and receive the input command from the input device through the USB hub, and convert the input command into the portable electronic device according to the corresponding key setting interface of an application executed by the portable electronic device. The control instructions used.

In some embodiments, the universal serial bus (USB) debug mode of the portable electronic device is turned on, and the portable electronic device is connected to the display device through a USB connection.

In some embodiments, the software package executed by the portable electronic device obtains the screen image from the playback image buffer of the portable electronic device and encodes the screen image into the video stream.

In some embodiments, the input instructions include key codes, text codes, mouse signals, scroll signals and touch signals.

In some embodiments, the software package executed by the portable electronic device transmits the control command to the touch screen of the portable electronic device to simulate the touch operation related to the input command, and in the application The program performs the touch operation.

In some embodiments, when the portable electronic device detects that the connection between the portable electronic device and the computer device is disconnected, the portable electronic device automatically deletes the software package.

In order to make the above-mentioned objects, features and advantages of the present invention more clearly understandable, a preferred embodiment is given below and described in detail with reference to the accompanying drawings.

Figure 1 is a block diagram of an electronic system according to an embodiment of the present invention. The electronic system 1 includes a computer system 10 and a portable electronic device 30 . The computer system 10 may be, for example, a personal computer or a server equipped with a display device. As shown in FIG. 1 , the computer system 10 includes a computer device 20 and a display device 40 , wherein the computer device 20 is connected to the display device 40 via signals. The computer device 20 includes, for example, a processing unit 110, a graphics processing unit 120, a memory unit 130, a storage device 140, transmission interfaces 150 and 160, a keyboard 171 and a mouse 172. The processing unit 110 , the graphics processing unit 120 , the memory unit 130 , the storage device 140 , the transmission interface 150 , the keyboard 171 and the mouse 172 are coupled to each other through the system bus 111 . The processing unit 110 may be, for example, a central processing unit (CPU), a general-purpose processor, or the like, but the invention is not limited thereto. The graphics processing unit 120 may be, for example, a graphics processing unit on a display card or a graphics processing unit integrated into the processing unit 110 .

The memory unit 130 is a random access memory, such as a dynamic random access memory (DRAM) or a static random access memory (SRAM), but the invention is not limited thereto. The storage device 140 is a non-volatile memory, such as a hard disk drive, a solid-state disk, or a flash memory. , or a read-only memory, but the invention is not limited thereto.

The transmission interface 160 may include a wired transmission interface and/or a wireless transmission interface, where the wired transmission interface may include: a High Definition Multimedia Interface (HDMI), a DisplayPort (DP) interface, an embedded display port ( embedded DisplayPort (eDP), Universal Serial Bus (USB) interface, USB Type-C interface, Thunderbolt (Thunderbolt) interface, Digital Video Interface (DVI), Video Graphics Array (VGA) interface, general purpose Input and output (GPIO) interface, Universal Asynchronous Transceiver and Transmitter (UART) interface, Serial Peripheral Interface (SPI) interface, Integrated Circuit Bus (I2C) interface, or a combination thereof, and the wireless transmission interface may include: Bluetooth ( Bluetooth), WiFi, near field communication (NFC) interface, etc., but the invention is not limited thereto. Peripheral devices 160 include, for example, input devices such as keyboards, mice, and touch pads, but the invention is not limited thereto.

For example, the storage device 140 can store the screen mirroring program 141 and the operating system 142 (such as Windows, Linux, MacOS, etc.), and the processing unit 110 reads the screen mirroring program 141 and the operating system 142 into the memory unit. 130 and execute. For example, the graphics processing unit 120 can perform graphics processing on the application program executed by the processing unit 110 to generate an image signal including one or more images, and transmit the image signal to the display device 40 for playback through the transmission interface 160 . The display device 40 may be, for example, a flat-panel display, a television, a projector, a computer screen, etc., but the invention is not limited thereto.

The portable electronic device 30 includes a processing unit 210, a touch panel 220, a memory unit 230, a storage device 240 and a transmission interface 250. The processing unit 210 may be, for example, a central processing unit (CPU), a general-purpose processor, or the like, but the invention is not limited thereto. The touch panel 220 may include a display panel 221 and a touch module 222, for example. The display panel 221 is, for example, a liquid crystal display panel, a light emitting diode display panel or an organic light emitting diode display panel, but the invention is not limited thereto. The touch module 222 can be, for example, a touch unit array, which can be integrated with the display panel 221 to form the touch panel 220 .

The memory unit 230 is a random access memory, such as a dynamic random access memory (DRAM) or a static random access memory (SRAM), but the invention is not limited thereto. In one embodiment, a part of the memory unit 230 can be used as a display image buffer (display image buffer) 231, which can temporarily store display images generated by the application program 241 executed by the processing unit 210.

The storage device 240 is a non-volatile memory, such as a flash memory or a read-only memory, but the invention is not limited thereto. The storage device 240 stores, for example, an operating system 241 (such as an Android operating system) and an application 242 (such as a mobile game), but the invention is not limited thereto.

In one embodiment, the transmission interfaces 150 and 250 are both USB Type-C interfaces, for example, and the transmission interface 150 of the computer device 20 can be connected to the transmission interface 250 of the portable electronic device 30 through the USB data cable 11 . Therefore, the computer device 20 and the portable electronic device 30 can transmit control commands and video streams to each other through the USB data cable 11, the details of which will be described in detail later.

In some embodiments, the operating system 241 running on the portable electronic device 30 is, for example, the Android operating system, and the computer device 20 executes the screen mirroring program 141 (which can also be called a mobile game assistant program). In addition, assuming that the USB debugging mode on the operating system 241 is turned on, when the user connects the USB data cable 11 to the transmission interface 150 of the computer device 20 and the transmission interface 250 of the portable electronic device 30, the computer device The screen mirroring program 141 running on 20 will detect the portable electronic device 30, and the screen mirroring program 141 will push the software package 232 to the portable electronic device 30 through the USB data cable 11 to use The processing unit 210 of the portable electronic device 30 automatically loads the software package 232 into the memory unit 230 and executes it.

Since the USB debugging mode of the operating system 241 is enabled, the software package 232 can obtain the screen of the video buffer 231 without obtaining the super user or root user rights of the operating system 241 The screen, for example, is a game screen generated by the processing unit 210 executing the application 242 (for example, a mobile game). Then, the software package 232 encodes the obtained screen image into a video stream (for example, through the FFmpeg tool program), and transmits the video stream to the computer device 20 through the USB data cable 11 .

After the computer device 20 receives the video stream from the portable electronic device 30, the processing unit 110 (or the graphics processing unit 120) can decode the video stream (for example, the FFmpeg tool program can also be used) and generate the decoded image. Perform rendering processing (for example, using the OpenGL protocol) to obtain a display image. The processing unit 110 transmits the above display image to the display device 40 through the transmission interface 160 for playback.

In some embodiments, when the software package 232 executed by the portable electronic device 30 detects that the connection (such as a USB connection) between the portable electronic device 30 and the computer device 20 is disconnected, the portable electronic device 30 The electronic device 30 will automatically delete the software package 232 .

Figure 2 is a schematic diagram of converting keyboard and mouse input commands into corresponding control commands according to an embodiment of the present invention. Please refer to Figure 1 and Figure 2 at the same time.

In one embodiment, when the portable electronic device 30 is running the software package 232, the user can also control input operations or touch operations on the portable electronic device 30 through the keyboard 171 and mouse 172 on the computer device 20. control operation. For example, the screen mirroring program 141 can detect and capture input commands related to the keyboard 171, mouse 172, or other input devices (such as a touch screen) on the system bus 111, and convert the input commands into controls. instruction. For example, the above-mentioned input command may include keycode (keycode), text code (text), mouse signal (mouse), scroll signal (scroll), and touch signal (touch), and the screen mirroring program 141 may convert the above-mentioned Input signals from various types of peripheral devices are converted into control instructions 201 to 205, as shown in Figure 2.

For example, the control instruction 201 corresponding to the key code includes type (for example, type=0), Android key code (AndroidKeycode), and Android super state (AndroidMetaState). The control instruction 202 corresponding to the text code includes fields such as type (for example, type=1), length (len), and text. The control instructions 203 corresponding to the mouse signal include type (for example, type=2), AndroidMotionEventButtons, X-axis coordinate (x), Y-axis coordinate (y), width (w), and height (h). The control instructions 204 corresponding to the scroll signal include type (for example, type = 3), X-axis coordinate (x), Y-axis coordinate (y), width (w), height (h), horizontal scroll (hScroll), and vertical scroll. Move (vScroll). The control instruction 205 corresponding to the touch signal includes type (for example, type=5), identifier (id), action (action), X-axis coordinate (x), Y-axis coordinate (y), width (w), height (h) )etc. In the above control instructions, the quantity of each field can be represented by the corresponding number of bytes. For example, the length (len) field in the control instruction 203 can be represented by two bytes, and the text (text) field can be represented by two bytes. It can be represented by five bytes, and so on.

Then, the processing unit 110 transmits the control command to the portable electronic device 30 through the transmission interface 150 and the USB data cable 11 . When the software package 232 running on the portable electronic device 30 receives the control command, the software package 232 can simulate the control command as an input operation or touch on the touch screen 260 of the portable electronic device 30 Operations (such as finger clicks, slides, etc.). Specifically, the software package 232 can transmit the above control instructions to the touch screen file (for example, located in the /dev/input/event directory) of the portable electronic device 30 to simulate the corresponding touch operation.

Figure 3 is a block diagram of a display system according to an embodiment of the present invention.

In one embodiment, the display system 50 includes a display device 300 and a computer device 450 . The display device 300 includes, for example, a display controller 310, a display module 320, a storage unit 330, an image buffer 340, a transmission interface 350, an input interface 360 and a universal serial bus (USB) hub 370. The display controller 310 may be, for example, an application-specific integrated circuit, a system-on-chip, a processor, or a microcontroller, but the present invention does not Limited to this.

In one embodiment, the display controller 310 includes an image scaler 311 and a timing controller 312. The display controller 310 receives an image signal from the computer device 450 or the portable electronic device 30 through the transmission interface 350, and the image scaler 311 can perform image scaling and/or image scaling on the image in the received image signal. The overlay process is performed to match the resolution of the display module 320 , and the image after image scaling (for example, called an output image) is stored in the image buffer 340 . The timing controller 312 controls the display module 320 to read the output image from the image buffer 340 and play it.

In another embodiment, the display controller 310 may include a timing controller 312, and the resolution of the image signal from the computer device 450 or the portable electronic device 30 is consistent with the resolution of the display module 320, so the display control After receiving the image signal from the computer device 450 or the portable electronic device 30, the processor 310 can store the image signal in the image buffer 340 without undergoing image scaling processing. The timing controller 320 can read the output image from the image buffer 340 and control the display module 320 to play the output image.

The display module 320 may be, for example, a liquid crystal panel (including a backlight module), a light-emitting diode panel, an organic light-emitting diode panel, a Cathode ray tube (cathode ray tube), electronic ink (E-Ink) display module, electroluminescent (electroluminescent) display module, plasma (Plasma) display module, projection (projection) display module, quantum dot ( Quantum Dot) display module, but the invention is not limited thereto.

The storage unit 330 may be, for example, a non-volatile memory, such as a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or an electronically erasable programmable read-only memory (EEPROM). The storage unit 330 may be external to the display controller 310 , or may be integrated into the display controller 310 . The storage unit 330 is used to store the firmware 331, firmware 332 and on-screen display (OSD) interface 333 related to the display device 300.

The image buffer 340 may be, for example, a volatile memory (such as a dynamic random access memory) or a non-volatile memory (such as a flash memory), which is used to store images to be played on the display module 320 Output images.

The transmission interface 350 may include a wired transmission interface and/or a wireless transmission interface, where the wired transmission interface may include: a High Definition Multimedia Interface (HDMI), a DisplayPort (DP) interface, an embedded display port ( embedded DisplayPort (eDP), Thunderbolt interface, digital video interface (DVI), video graphics array (VGA) interface, general purpose input and output (GPIO) interface, universal asynchronous transceiver transmitter (UART) interface, serial peripherals interface (SPI) interface, integrated circuit bus (I2C) interface, or a combination thereof, and the wireless transmission interface may include: Bluetooth, WiFi, near field communication (NFC) interface, etc., but the present invention does not Limited to this.

The firmware 331 includes, for example, display settings of the on-screen display interface of the display device 300 and extended display capability identification data (Extended Display Identification Data, EDID) and display settings. The extended display capability identification information includes, for example, the manufacturer, product name, resolution, display frames per second, etc. of the display device 300 . Display settings include, for example, settings such as brightness, contrast, sharpness, and color temperature of the display device 300 .

In one embodiment, the display controller 310 can read the firmware 331 and the OSD interface 333 code stored in the storage unit 330 through a bus (such as an I2C bus), and set relevant display parameters accordingly. . In addition, the display controller 310 can further read the firmware 332 in the storage unit 330 to perform the mobile phone screen projection operation. The firmware 332 includes, for example, related program codes and software packages for mobile phone screen projection.

In some embodiments, the display device 300 is electrically connected to the computer device 450, for example, through a USB connection or other image transmission protocols (such as HDMI, DisplayPort, etc.). In addition, the display controller 310 can also transmit the extended display capability identification data of the display device 300 to the computer device 450 through the transmission interface 350 (for example, it can be an image transmission channel or a data transmission channel), so that the computer device 450 can set the output to be output. The resolution of the image signal and the related synchronization signal. The on-screen display interface 333 includes, for example, an on-screen display menu (OSD Menu) and options, an information display interface (information dashboard), a timer, a counter, a crosshair, a specific symbol, a specific color, a specific text, or a combination thereof, but The present invention is not limited to this.

The input interface 360 is used to control the on-screen display menu of the display device 300 . The input interface 360 may be implemented by, for example, a five-way joystick 362 or five physical buttons 361 to implement instructions such as up, down, left, right, and confirmation.

In one embodiment, the computer device 450 can be connected to the USB hub 370 of the display device 300 through the USB data cable 13 to transmit data to and from the display device 300 . For example, the keyboard 371 and the mouse 372 can be plugged into the USB hub 370, and the display controller 310 can transmit the received input signals of the keyboard 371 and the mouse 372 to the computer device 450 through the USB hub 370 and the USB data cable 13. Perform relevant input operations.

In another embodiment, the display device 300 is not electrically connected to the computer device 450 , which means that the display device 300 performs screen mirroring operations together with the portable electronic device 30 . For example, the portable electronic device 30 can be connected to the USB hub 370 of the display device 300 through the USB data cable 12 .

Similar to the embodiment of FIG. 1 , when the firmware 332 executed by the display controller 310 (for example, including the relevant program code and software package 232 for mobile phone screen projection) determines that the portable electronic device 30 is connected to the display through the USB data cable 12 The USB hub 370 of the device 300 and the display controller 310 push the software package 232 to the portable electronic device 30 through the USB hub 370 and the USB data cable 12, so that the portable electronic device 30 automatically executes the software package 232. The details of the above software package 232 can be referred to the embodiment in Figure 1, and therefore will not be described again here.

Therefore, the software package 232 executed by the portable electronic device 30 can obtain the screen image of the playback image buffer 231 , wherein the screen image is, for example, a game generated by the processing unit 210 executing the application program 242 (for example, a mobile game). picture. Then, the software package 232 encodes the obtained screen image into a video stream (for example, through the FFmpeg tool program), and transmits the video stream to the display device 300 through the USB data cable 12 . After the display device 300 receives the video stream from the portable electronic device 30, the display controller 310 can decode the video stream (for example, the FFmpeg tool program can also be used) and render the decoded image (for example, can Use the OpenGL protocol) to get the display image. The display controller 310 plays the above display image on the display module 320 .

In addition, the firmware 332 executed by the display controller 310 can convert the input commands from the keyboard 371 and the mouse 372 into corresponding control commands, and transmit the control commands to the portable electronic device 30 . The software package 232 running on the portable electronic device 30 can simulate the above control instructions as input operations or touch operations related to the position of the specific function button on the touch screen 220 of the portable electronic device 30 (for example, Finger clicks, slides, etc.).

In some embodiments, when the software package 232 executed by the portable electronic device 30 detects that the connection (such as a USB connection) between the portable electronic device 30 and the display device 300 is disconnected, the portable electronic device 30 The electronic device 30 will automatically delete the software package 232 .

Figure 4 is a schematic diagram of a key setting interface of a screen mirroring program in a specific application according to an embodiment of the present invention. Please refer to Figure 1 and Figure 4 at the same time.

When using the hardware configuration in Figure 1, the screen of the portable electronic device 30 can be projected to the display device 40 for display, and the user can also view the screen through the keyboard 171 and mouse 172 connected to the computer device 20. The application program 242 executed by the portable electronic device 30 performs input operations or touch operations.

For example, the screen mirroring program 141 can set a corresponding key setting interface 400 for the application program 242 executed by the portable electronic device 30 , and different application programs 242 correspond to different key setting interfaces 400 . The key setting interface 400 includes, for example, a plurality of function keys, direction keys, etc., all of which are marked with corresponding keyboard key text. When the user presses a specific button on the keyboard 171 or manipulates the cursor of the mouse 172 to click on a specific function button, the screen mirroring program 141 can convert the above input command into a corresponding control command, and The control command is sent to the portable electronic device 30 . The software package 232 running on the portable electronic device 30 can simulate the above control instructions as input operations or touch operations related to the position of the specific function button on the touch screen 220 of the portable electronic device 30 (for example, Finger clicks, slides, etc.).

When using the hardware configuration in Figure 3, the display device 300 and the portable electronic device 30 perform screen mirroring operations together. For example, the portable electronic device 30 can be connected to the USB hub 370 of the display device 300 through the USB data cable 12 . In this usage scenario, when the user presses a specific button on the keyboard 371 or manipulates the cursor of the mouse 372 to click on a specific function button, the firmware 332 executed by the display controller 310 can change the above The input command is converted into a corresponding control command, and the control command is transmitted to the portable electronic device 30 . The software package 232 running on the portable electronic device 30 can simulate the above control instructions as input operations or touch operations related to the position of the specific function button on the touch screen 220 of the portable electronic device 30 (for example, Finger clicks, slides, etc.).

To sum up, the present invention provides a computer device and a display device that can automatically push a software package to the portable electronic device for automatic execution when the portable electronic device is connected. Therefore, the user does not need to Relevant programs must be installed on the portable electronic device in advance to ensure that the version of the software package installed and executed on the portable electronic device corresponds to the version of the screen mirroring program on the computer device/display device. The software package executed by the portable electronic device can be used to capture the screen of the portable electronic device to encode it into a video stream, and the computer device (or display device) has the ability to decode the video stream and render the image to obtain a usable image. The screen image of the portable electronic device is played on the display device, or the display device can play the screen image on its own display module. When the software package executed by the portable electronic device detects that the connection (such as a USB connection) between the portable electronic device 30 and the computer device (or display device) is disconnected, the portable electronic device automatically Remove the software package so users can use the screen mirroring function more conveniently.

In addition, users can customize the operation function mapping of keyboard and mouse on the screen mirroring program executed by the computer device/display device, so it can more fully support mobile games based on touch and click, and users can use the plug-in to The keyboard/mouse installed on the computer device or the display device can control the application or mobile game executed by the portable electronic device, and can also improve the convenience of control and enhance the user experience.

The use of words such as "first", "second", and "third" in the scope of the patent application are used to modify the elements in the scope of the patent application, and are not used to indicate the order of priority, precedence relationship, or relationship between them. The precedence of one element over another, or the chronological order in which method steps are performed, is only used to distinguish elements with the same name.

Although the present invention is disclosed above in terms of preferred embodiments, they are not intended to limit the scope of the present invention. Anyone with ordinary skill in the art may make slight changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended patent application scope.

1: Electronic system 10:Computer system 11, 12, 13: USB data cable 20: Computer equipment 30: Portable electronic devices 40:Display device 110: Processing unit 111:System bus 120: Graphics processing unit 130:Memory unit 140:Storage device 141:Screen mirroring program 142:Operating system 150, 160, 250: Transmission interface 171:Keyboard 172:Mouse 201-205:Control instructions 210: Processing unit 220:Touch panel 221:Display panel 222:Touch module 230:Memory unit 231: Play image buffer 232:Software package 240:Storage device 241:Operating system 242:Application 300:Display device 310:Display controller 320:Display module 330:Storage unit 331, 332: Firmware 333: Display interface on screen 340:Image buffer 350:Transmission interface 360: Input interface 361:Entity button 362: Five-way control lever 370:USB hub 371:Keyboard 372:Mouse 400: Button setting interface 450:Computer device

Figure 1 is a block diagram of an electronic system according to an embodiment of the present invention. Figure 2 is a schematic diagram of converting keyboard and mouse input commands into corresponding control commands according to an embodiment of the present invention. Figure 3 is a block diagram of a display system according to an embodiment of the present invention. Figure 4 is a schematic diagram of a key setting interface of a screen mirroring program in a specific application according to an embodiment of the present invention.

1: Electronic system

10:Computer system

11:USB data cable

20: Computer equipment

30: Portable electronic devices

40:Display device

110: Processing unit

111:System bus

120: Graphics processing unit

130:Memory unit

140:Storage device

141:Screen mirroring program

142:Operating system

150, 160, 250: Transmission interface

171:Keyboard

172:Mouse

210: Processing unit

220:Touch panel

221:Display panel

222:Touch module

230:Memory unit

231: Play image buffer

232:Software package

240:Storage device

241:Operating system

242:Application

Claims (12)

A computer device including: a storage device for storing a screen mirroring program; A processing unit for executing the screen mirroring program to perform the following steps: In response to detecting that a portable electronic device is connected to the computer device, push a software package to the portable electronic device to automatically execute the software package on the portable electronic device; Receive a video stream from the portable electronic device, wherein the video stream includes a screen image of the portable electronic device; Decode the video stream and render the decoded image to obtain the screen image, and play the screen image on a display device; and Receive an input command from an input device connected to the computer device, and convert the input command into a control command used by the portable electronic device according to a corresponding key setting interface of an application executed by the portable electronic device. The computer device of claim 1, wherein the universal serial bus (USB) debug mode of the portable electronic device is turned on, and the portable electronic device is connected to the computer device through a USB connection. For example, the computer device of claim 2, wherein the software package executed by the portable electronic device obtains the screen image from the playback image buffer of the portable electronic device and encodes the screen image into the video stream. The computer device of claim 1, wherein the input command includes key codes, text codes, mouse signals, scroll signals and touch signals. The computer device of claim 1, wherein the software package executed by the portable electronic device transmits the control command to the touch screen file of the portable electronic device to simulate the touch operation related to the input command, And perform the touch operation in the application. The computer device of claim 1, wherein when the portable electronic device detects that the connection between the portable electronic device and the computer device is disconnected, the portable electronic device automatically deletes the software package. A display device including: a display module; A Universal Serial Bus (USB) hub connected to one or more input devices; and A display controller for performing the following steps: In response to detecting that a portable electronic device is connected to the USB hub, push a software package to the portable electronic device through the USB hub to automatically execute the software package on the portable electronic device; Receive a video stream from the portable electronic device through the USB hub, wherein the video stream includes a screen image of the portable electronic device; Decode the video stream and render the decoded image to obtain the screen image, and play the screen image on the display module; and The input command of the input device is received through the USB hub, and the input command is converted into a control command used by the portable electronic device according to a corresponding key setting interface of an application executed by the portable electronic device. The display device of claim 7, wherein the universal serial bus (USB) debug mode of the portable electronic device is turned on, and the portable electronic device is connected to the display device through a USB connection. For example, the display device of claim 8, wherein the software package executed by the portable electronic device obtains the screen image from the playback image buffer of the portable electronic device and encodes the screen image into the video stream. The display device of claim 7, wherein the input command includes key codes, text codes, mouse signals, scroll signals and touch signals. The display device of claim 7, wherein the software package executed by the portable electronic device transmits the control command to the touch screen file of the portable electronic device to simulate the touch operation related to the input command, And perform the touch operation in the application. The display device of claim 7, wherein when the portable electronic device detects that the connection between the portable electronic device and the computer device is disconnected, the portable electronic device automatically deletes the software package.

Images