CN118210391A - Computer device and display device - Google Patents
Computer device and display device Download PDFInfo
- Publication number
- CN118210391A CN118210391A CN202211613682.4A CN202211613682A CN118210391A CN 118210391 A CN118210391 A CN 118210391A CN 202211613682 A CN202211613682 A CN 202211613682A CN 118210391 A CN118210391 A CN 118210391A
- Authority
- CN
- China
- Prior art keywords
- portable electronic
- electronic device
- screen
- display
- software package
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012545 processing Methods 0.000 claims abstract description 28
- 238000009877 rendering Methods 0.000 claims abstract description 6
- 230000004044 response Effects 0.000 claims abstract description 5
- 230000005540 biological transmission Effects 0.000 description 30
- 230000006870 function Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 8
- 238000000034 method Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 6
- 230000002093 peripheral effect Effects 0.000 description 4
- 239000004973 liquid crystal related substance Substances 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- APTZNLHMIGJTEW-UHFFFAOYSA-N pyraflufen-ethyl Chemical compound C1=C(Cl)C(OCC(=O)OCC)=CC(C=2C(=C(OC(F)F)N(C)N=2)Cl)=C1F APTZNLHMIGJTEW-UHFFFAOYSA-N 0.000 description 2
- 239000002096 quantum dot Substances 0.000 description 2
- 230000008054 signal transmission Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000001934 delay Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Landscapes
- Position Input By Displaying (AREA)
Abstract
A computer device, comprising: a storage device and a processing unit. The storage device is used for storing the screen image program. The processing unit executes a screen mirroring program to perform the steps of: in response to detecting that the portable electronic device is connected to the computer device, pushing the software package to the portable electronic device to automatically execute the software package; receiving a video stream from the portable electronic device, wherein the video stream comprises a screen of the portable electronic device; decoding the video stream and rendering the decoded image to obtain a screen picture for playing on a display device; and receiving an input instruction of an input device connected to the computer device, and converting the input instruction into a control instruction used by the portable electronic device according to a key setting interface corresponding to an application program executed by the portable electronic device.
Description
Technical Field
The present invention relates to a screen mirroring technology, and more particularly, to a computer device and a display device.
Background
Because of technological advances, many mobile phone screen projection software is available in the market for users to project the screen of their portable electronic devices onto a display or television. However, the conventional mobile phone screen projection software requires the user to install the related application program on the portable electronic device, and this method has a problem of updating the version of the application program besides the complicated operation steps required by the user. In addition, the conventional mobile phone picture projection software needs to complete function call synchronization, so that when signal transmission is not completed, the software always waits for the arrival of a signal, and further delays in instruction operation.
Furthermore, the conventional mobile phone screen projection software only supports signal transmission, and a user cannot define the operation function mapping of the keyboard and the mouse by himself, and cannot completely correspond to a mobile phone game mainly including touch control clicking.
Disclosure of Invention
Accordingly, the present invention is directed to a computer device and a display device for solving the above-mentioned problems.
The invention provides a computer device, comprising: a storage device and a processing unit. The storage device is used for storing a screen mirror program. The processing unit is used for executing a screen mirror program to execute the following steps: in response to detecting that a portable electronic device is connected to the computer device, pushing a software package to the portable electronic device to automatically execute the software package on the portable electronic device; receiving a video stream from the portable electronic device, wherein the video stream comprises a screen of the portable electronic device; decoding the video stream, rendering the decoded image to obtain a screen picture, and playing the screen picture on a display device; and receiving an input instruction of an input device connected to the computer device, and converting the input instruction into a control instruction used by the portable electronic device according to a key setting interface corresponding to an application program executed by the portable electronic device.
In some embodiments, a Universal Serial Bus (USB) debug mode of the portable electronic device is turned on, and the portable electronic device is connected to the computer device through a USB connection.
In some embodiments, a software package executed by the portable electronic device obtains a screen from a play-back image buffer of the portable electronic device and encodes the screen into a video stream.
In some embodiments, the input command includes a key code, a text code, a mouse signal, a scroll signal, and a touch signal.
In some embodiments, the software package executed by the portable electronic device transmits a control command to the touch screen of the portable electronic device to simulate a touch operation related to the input command, and the touch operation is executed in the application program.
In some embodiments, when the portable electronic device detects that the connection between the portable electronic device and the computer device is disconnected, the portable electronic device automatically deletes the software package.
The present invention further provides a display device including: a display module; a Universal Serial Bus (USB) hub coupled to one or more input devices; and a display controller for executing the following steps: in response to detecting that a portable electronic device is connected to the USB hub, pushing a software package to the portable electronic device through the USB hub so as to automatically execute the software package in the portable electronic device; receiving a video stream from the portable electronic device through the USB hub, wherein the video stream comprises a screen picture of the portable electronic device; decoding the video stream, rendering the decoded image to obtain a screen picture, and playing the screen picture on a display module; and receiving an input instruction of the input device through the USB hub, and converting the input instruction into a control instruction used by the portable electronic device according to a key setting interface corresponding to an application program executed by the portable electronic device.
In some embodiments, a Universal Serial Bus (USB) debug mode of the portable electronic device is turned on, and the portable electronic device is connected to the display device through a USB connection.
In some embodiments, a software package executed by the portable electronic device obtains a screen from a play-back image buffer of the portable electronic device and encodes the screen into a video stream.
In some embodiments, the input command includes a key code, a text code, a mouse signal, a scroll signal, and a touch signal.
In some embodiments, the software package executed by the portable electronic device transmits a control command to the touch screen of the portable electronic device to simulate a touch operation related to the input command, and the touch operation is executed in the application program.
In some embodiments, when the portable electronic device detects that the connection between the portable electronic device and the computer device is disconnected, the portable electronic device automatically deletes the software package.
Drawings
Fig. 1 is a block diagram of an electronic system in accordance with an embodiment of the present invention.
Fig. 2 is a schematic diagram illustrating the conversion of input commands of a keyboard and a mouse into corresponding control commands according to an embodiment of the invention.
Fig. 3 is a block diagram of a display system according to an embodiment of the invention.
FIG. 4 is a diagram illustrating a screen mirroring process in a key setting interface of a specific application according to an embodiment of the present invention.
Wherein reference numerals are as follows:
1: electronic system
10: Computer system
11. 12, 13: USB data line
20: Computer device
30: Portable electronic device
40: Display device
110: Processing unit
111: System bus
120: Graphics processing unit
130: Memory cell
140: Storage device
141: Screen mirroring program
142: Operating system
150. 160, 250: Transmission interface
171: Keyboard with keyboard body
172: Mouse with mouse body
201-205: Control instructions
210: Processing unit
220: Touch panel
221: Display panel
222: Touch control module
230: Memory cell
231: Playing image buffer
232: Software package
240: Storage device
241: Operating system
242: Application program
300: Display device
310: Display controller
311: Image scaler
312: Time sequence controller
320: Display module
330: Memory cell
331. 332: Firmware
333: On-screen display interface
340: Image buffer
350: Transmission interface
360: Input interface
361: Entity button
362: Five-way control rod
370: USB hub
371: Keyboard with keyboard body
372: Mouse with mouse body
400: Key setting interface
450: Computer device
Detailed Description
In order to make the above objects, features and advantages of the present invention more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Fig. 1 is a block diagram of an electronic system in accordance with an embodiment of the present invention. The electronic system 1 comprises a computer system 10 and a portable electronic device 30. The computer system 10 may be, for example, a personal computer or a server equipped with a display device. As shown in FIG. 1, the computer system 10 includes a computer device 20 and a display device 40, wherein the computer device 20 is signally connected to the display device 40. The computer device 20 includes, for example, a processing unit 110, a graphics processing unit 120, a memory unit 130, a storage device 140, transmission interfaces 150 and 160, a keyboard 171, and a mouse 172. The processing unit 110, the graphics processing unit 120, the memory unit 130, the storage device 140, the transmission interface 150, the keyboard 171, and the mouse 172 are coupled to each other via the system bus 111. The processing unit 110 may be, for example, a Central Processing Unit (CPU), a general-purpose processor (general-purpose processor), etc., but the invention is not limited thereto. The gpu 120 may be, for example, a gpu on a display card or a gpu integrated into the processing unit 110.
The memory unit 130 is a random access memory, such as a Dynamic Random Access Memory (DRAM) or a Static Random Access Memory (SRAM), but the invention is not limited thereto. The storage device 140 is a non-volatile memory (non-volatile memory), such as a hard disk drive (HARD DISK DRIVE), a solid state disk (solid-state disk) (STATE DISK), a flash memory (flash memory), or a read-only memory (read-only memory), but the invention is not limited thereto.
The transmission interface 160 may include a wired transmission interface and/or a wireless transmission interface, wherein the wired transmission interface may include: a high-resolution multimedia interface (High Definition Multimedia Interface, HDMI), a Display (DP) interface, an embedded display (embedded DisplayPort, eDP), a universal serial bus (Universal Serial Bus, USB) interface, a USB Type-C interface, a Lei Li (Thunderbolt) interface, a Digital Video Interface (DVI), a Video Graphics Array (VGA) interface, a general purpose input/output (GPIO) interface, a universal asynchronous receiver/transmitter (UART) interface, a Serial Peripheral Interface (SPI), an integrated circuit bus (I2C) interface, or a combination thereof, and the wireless transmission interface may include: bluetooth (Bluetooth), wiFi, near Field Communication (NFC) interfaces, etc., but the invention is not limited thereto. The peripheral device 160 includes, for example: input devices such as a keyboard, a mouse, and a touch pad, but the present invention is not limited thereto.
For example, the storage device 140 may store the screen image program 141 and the operating system 142 (e.g., windows, linux, macOS, etc.), and the processing unit 110 reads and executes the screen image program 141 and the operating system 142 to the memory unit 130. The gpu 120 may perform graphics processing of an application program executed by the gpu 110 to generate an image signal including one or more images, and transmit the image signal to the display device 40 for playing via the transmission interface 160. The display device 40 may be, for example, a flat panel display, a television, a projector, a computer screen, etc., but the present invention is not limited thereto.
The portable electronic device 30 includes a processing unit 210, a touch panel 220, a memory unit 230, a storage device 240 and a transmission interface 250. The processing unit 210 may be, for example, a Central Processing Unit (CPU), a general-purpose processor (general-purpose processor), etc., but the invention is not limited thereto. The touch panel 220 may include, for example, a display panel 221 and a touch module 222. The display panel 221 is, for example, a liquid crystal display panel, a light emitting diode display panel or an organic light emitting diode display panel, but the present invention is not limited thereto. The touch module 222 may be, for example, an array of touch units, which may be integrated with the display panel 221 into the touch panel 220.
The memory unit 230 is a random access memory, such as a Dynamic Random Access Memory (DRAM) or a Static Random Access Memory (SRAM), but the invention is not limited thereto. In one embodiment, a portion of the memory unit 230 may be used as a play image buffer (DISPLAY IMAGE buffer) 231, which may temporarily store the play image generated by the application 241 executed by the processing unit 210.
The storage device 240 is a non-volatile memory (non-volatile memory), such as a flash memory (flash memory) or a read-only memory (read-only memory), but the invention is not limited thereto. The storage device 240 stores, for example, an operating system 241 (for example, an Android operating system) and an application 242 (for example, a mobile game), but the present invention is not limited thereto.
In an embodiment, the transmission interfaces 150 and 250 are, for example, USB Type-C interfaces, and the transmission interface 150 of the computer device 20 can be connected to the transmission interface 250 of the portable electronic device 30 through the USB data line 11. Therefore, the computer device 20 and the portable electronic device 30 can mutually transmit control commands and video streams through the USB data line 11, and details thereof will be described later.
In some embodiments, the operating system 241 running on the portable electronic device 30 is, for example, an Android operating system, and the computer device 20 executes the screen mirroring program 141 (also referred to as a hand-tour assistant program). In addition, assuming that the USB debug (debugging) mode on the os 241 is on, when the user connects the USB data line 11 to the transmission interface 150 of the computer device 20 and the transmission interface 250 of the portable electronic device 30, the screen image program 141 running on the computer device 20 detects the portable electronic device 30, and the screen image program 141 pushes (pushes) the software package 232 to the portable electronic device 30 through the USB data line 11, so that the processing unit 210 of the portable electronic device 30 automatically loads the software package 232 into the memory unit 230 and executes the software package.
Because the USB debug mode of the OS 241 is on, the software package 232 can obtain the screen of the playback image buffer 231 without obtaining the authority of the super user or root user of the OS 241, wherein the screen is, for example, a game screen generated by the processing unit 210 executing the application 242 (for example, a mobile game). The software package 232 then encodes the acquired screen into a video stream (e.g., via FFmpeg tools) and transmits the video stream to the computer device 20 via the USB data line 11.
When the computer device 20 receives the video stream from the portable electronic device 30, the processing unit 110 (or the graphics processing unit 120) may decode the video stream (e.g., FFmpeg tool may be used as well) and render the decoded image (e.g., openGL protocol may be used) to obtain a display image. The processing unit 110 transmits the display image to the display device 40 for playing through the transmission interface 160.
In some embodiments, when the software package 232 executed by the portable electronic device 30 detects that the connection (e.g. USB connection) between the portable electronic device 30 and the computer device 20 is disconnected, the portable electronic device 30 automatically deletes the software package 232.
Fig. 2 is a schematic diagram illustrating the conversion of input commands of a keyboard and a mouse into corresponding control commands according to an embodiment of the invention. Please refer to fig. 1 and fig. 2 simultaneously.
In one embodiment, when the portable electronic device 30 is running the software package 232, the user can also control the input operation or the touch operation on the portable electronic device 30 through the keyboard 171 and the mouse 172 on the computer device 20. For example, the screen mirroring program 141 may detect and capture input commands on the system bus 111 associated with the keyboard 171, the mouse 172, or other input devices (e.g., a touch screen), and convert the input commands into control commands. For example, the input command may include a key code (keycode), a text code (text), a mouse signal (mouse), a scroll signal (scroll), a touch signal (touch), and the screen mirroring program 141 may convert the peripheral device input signals of the above types into control commands 201 to 205, respectively, as shown in fig. 2.
For example, the control instructions 201 corresponding to the key codes include type (type) (e.g., type=0), android key codes (AndroidKeycode), and An Zhuochao status (AndroidMetaState). The control command 202 corresponding to the text code includes fields such as a type (e.g., type=1), a length (len), and text. The corresponding control command 203 of the mouse signal includes a type (e.g., type=2), an android movement event button (AndroidMotionEventButtons), an X-axis coordinate (X), a Y-axis coordinate (Y), a width (w), and a height (h). The control commands 204 corresponding to the scroll signal include type (e.g., type=3), X axis (X), Y axis (Y), width (w), height (h), horizontal scroll (hScroll) and vertical scroll (vScroll). The control command 205 corresponding to the touch signal includes a type (e.g., type=5), an identifier (id), an action (action), an X-axis (X), a Y-axis (Y), a width (w), a height (h), and so on. In the control instruction, the number of fields may be represented by a corresponding number of bytes (byte), for example, the length (len) field in the control instruction 203 may be represented by two bytes, the text (text) field may be represented by five bytes, and so on.
Then, the processing unit 110 transmits the control command to the portable electronic device 30 through the transmission interface 150 and the USB data line 11. When the software package 232 running on the portable electronic device 30 receives the control command, the software package 232 may simulate the control command as an input operation or a touch operation (such as clicking, sliding, etc. of a finger) on the touch screen 260 of the portable electronic device 30. In detail, the software package 232 may transmit the control command to a touch screen of the portable electronic device 30 (e.g. under a/dev/input/event directory) to simulate a corresponding touch operation.
Fig. 3 is a block diagram of a display system according to an embodiment of the invention.
In one embodiment, the display system 50 includes a display device 300 and a computer device 450. The display device 300 includes, for example, a display controller 310, a display module 320, a storage unit 330, a video buffer 340, a transmission interface 350, an input interface 360, and a Universal Serial Bus (USB) hub 370. The display controller 310 may be, for example, an application-oriented integrated circuit (application-SPECIFIC INTEGRATED circuits), a System-on-chip (System-on-chip), a processor, or a microcontroller (microcontroller), but the present invention is not limited thereto.
In one embodiment, the display controller 310 includes an image scaler (IMAGE SCALER) 311 and a timing controller (timing controller) 312. The display controller 310 receives an image signal from the computer device 450 or the portable electronic device 30 through the transmission interface 350, and the image scaler 311 performs an image scaling process and/or an image superimposition process on the image in the received image signal to conform to the resolution of the display module 320, and stores the image (e.g. referred to as an output image) after the image scaling process in the image buffer 340. The timing controller 312 controls the display module 320 to read the output image from the image buffer 340 and play the output image.
In another embodiment, the display controller 310 may include the timing controller 312, and the resolution of the image signal from the computer device 450 or the portable electronic device 30 is consistent with the resolution of the display module 320, so that the display controller 310 may store the image signal in the image buffer 340 without performing the image scaling process after receiving the image signal from the computer device 450 or the portable electronic device 30. The timing controller 320 can read the output image from the image buffer 340 and control the display module 320 to play the output image.
The display module 320 may be, for example, a liquid crystal (liquid crystal) panel (including a backlight module), a light-emitting diode (light-emitting diode) panel, an organic light-emitting diode (organic light-emitting diode) panel, a cathode ray tube (cathode ray tube), an electronic Ink (E-Ink) display module, an electroluminescent (electroluminescent) display module, a Plasma (Plasma) display module, a projection (projection) display module, a Quantum Dot (Quantum Dot) display module, but the present invention is not limited thereto.
The memory unit 330 may be a nonvolatile memory such as Read Only Memory (ROM), erasable Programmable Read Only Memory (EPROM), electrically Erasable Programmable Read Only Memory (EEPROM). The storage unit 330 may be external to the display controller 310, or may be integrated into the display controller 310, for example. The storage unit 330 is used for storing firmware 331, firmware 332 and an on-screen display (OSD) interface 333 related to the display device 300.
The image buffer 340 may be, for example, a volatile memory (e.g., a dynamic random access memory) or a nonvolatile memory (e.g., a flash memory) for storing the output image to be played on the display module 320.
The transmission interface 350 may include a wired transmission interface and/or a wireless transmission interface, where the wired transmission interface may include: a high-resolution multimedia interface (High Definition Multimedia Interface, HDMI), a Display (DP) interface, an embedded display (embedded DisplayPort, eDP), a Lei Li (Thunderbolt) interface, a Digital Video Interface (DVI), a Video Graphics Array (VGA) interface, a General Purpose Input Output (GPIO) interface, a Universal Asynchronous Receiver Transmitter (UART) interface, a Serial Peripheral Interface (SPI), an integrated circuit bus (I2C) interface, or a combination thereof, and the wireless transmission interface may include: bluetooth (Bluetooth), wiFi, near Field Communication (NFC) interfaces, etc., but the invention is not limited thereto.
The firmware 331 includes, for example, display settings of an on-screen display interface of the display device 300, extended display capability identification data (Extended Display Identification Data, EDID), and display settings. The extended display capability identification data includes, for example, a manufacturer of the display device 300, a product name, a resolution, a display frame number per second, and the like. The display settings include, for example, brightness, contrast, sharpness, color temperature, and the like settings of the display device 300.
In one embodiment, the display controller 310 may read the program codes of the firmware 331 and the OSD interface 333 stored in the storage unit 330 through a bus (e.g., an I2C bus), and set the relevant display parameters accordingly. In addition, the display controller 310 can further read the firmware 332 in the storage unit 330 to perform the mobile phone screen projection operation. Firmware 332 includes, for example, program codes and software packages related to mobile phone screen projection.
In some embodiments, the display device 300 is electrically connected to the computer device 450, such as via a USB connection or other image transmission protocol (e.g., HDMI, displayPort, etc.). In addition, the display controller 310 may also transmit the extended display capability identification data of the display device 300 to the computer device 450 through the transmission interface 350 (for example, an image transmission channel or a data transmission channel) so as to set the resolution of the image signal to be outputted and the related synchronization signal by the computer device 450. The on-screen display interface 333 includes, for example, an on-screen display Menu (OSD Menu) and options, an information display interface (information dashboard), a timer, a counter, a sight line (crosshair), a specific symbol, a specific color, a specific text, or a combination thereof, but the present invention is not limited thereto.
The input interface 360 is used to control the on-screen display menu of the display device 300. The input interface 360 may be implemented, for example, by a five-way joystick 362 or by five physical buttons 361, for example, to implement up, down, left, right, and confirm commands.
In one embodiment, the computer device 450 may be connected to the USB hub 370 of the display device 300 through the USB data line 13 to perform data transmission with the display device 300. The keyboard 371 and the mouse 372 may be inserted into the USB hub 370, for example, and the display controller 310 may transmit the received input signals of the keyboard 371 and the mouse 372 to the computer device 450 through the USB hub 370 and the USB data line 13 for performing related input operations.
In another embodiment, the display device 300 is not electrically connected to the computer device 450, i.e. the display device 300 performs a screen mirroring operation together with the portable electronic device 30. For example, the portable electronic device 30 can be connected to the USB hub 370 of the display device 300 through the USB data line 12.
Similar to the embodiment of fig. 1, when the firmware 332 (e.g., the program code and the software package 232 related to the mobile phone screen projection) executed by the display controller 310 determines that the portable electronic device 30 is connected to the USB hub 370 of the display device 300 through the USB data line 12, the display controller 310 pushes the software package 232 to the portable electronic device 30 through the USB hub 370 and the USB data line 12, so that the portable electronic device 30 automatically executes the software package 232. Details of the software package 232 may refer to the embodiment of fig. 1, and thus are not described herein.
Therefore, the software package 232 executed by the portable electronic device 30 can obtain a screen of the play image buffer 231, wherein the screen is, for example, a game screen generated by the processing unit 210 executing the application 242 (for example, a mobile phone game). Next, the software package 232 encodes the acquired screen into a video stream (e.g. by FFmpeg tool) and transmits the video stream to the display device 300 via the USB data line 12. When the display device 300 receives the video stream from the portable electronic device 30, the display controller 310 may decode the video stream (e.g., FFmpeg tool program may be used as well) and render the decoded image (e.g., openGL protocol may be used) to obtain a display image. The display controller 310 plays the display image on the display module 320.
In addition, the firmware 332 executed by the display controller 310 can convert the input instructions from the keyboard 371 and the mouse 372 into corresponding control instructions, and transmit the control instructions to the portable electronic device 30. The software package 232 running on the portable electronic device 30 can simulate the control command as an input operation or a touch operation (such as clicking, sliding, etc. of a finger) related to a specific function key position on the touch screen 220 of the portable electronic device 30.
In some embodiments, when the software package 232 executed by the portable electronic device 30 detects that the connection (e.g. USB connection) between the portable electronic device 30 and the display device 300 is disconnected, the portable electronic device 30 automatically deletes the software package 232.
FIG. 4 is a diagram illustrating a screen mirroring process in a key setting interface of a specific application according to an embodiment of the present invention. Please refer to fig. 1 and fig. 4 simultaneously.
When the hardware configuration of fig. 1 is used, the screen of the portable electronic device 30 can be projected to the display device 40 for displaying, and the user can also perform an input operation or a touch operation on the application 242 executed by the portable electronic device 30 through the keyboard 171 and the mouse 172 connected to the computer device 20.
For example, the screen mirroring program 141 can set the corresponding key setting interface 400 for the application 242 executed by the portable electronic device 30, and different application 242 corresponds to different key setting interfaces 400. The key setting interface 400 includes, for example, a plurality of function keys, direction keys, etc., each of which has been labeled with a corresponding key text of a keyboard. When the user presses a specific key on the keyboard 171 or manipulates a cursor of the mouse 172 to click on a specific function key, the screen mirroring program 141 converts the input command into a corresponding control command and transmits the control command to the portable electronic device 30. The software package 232 running on the portable electronic device 30 can simulate the control command as an input operation or a touch operation (such as clicking, sliding, etc. of a finger) related to a specific function key position on the touch screen 220 of the portable electronic device 30.
When the hardware configuration of fig. 3 is used, the display device 300 performs a screen mirroring operation together with the portable electronic device 30. For example, the portable electronic device 30 can be connected to the USB hub 370 of the display device 300 through the USB data line 12. In this usage scenario, when the user presses a specific key on the keyboard 371 or manipulates the cursor of the mouse 372 to click on the specific function key, the firmware 332 executed by the display controller 310 may convert the input command into a corresponding control command and transmit the control command to the portable electronic device 30. The software package 232 running on the portable electronic device 30 can simulate the control command as an input operation or a touch operation (such as clicking, sliding, etc. of a finger) related to a specific function key position on the touch screen 220 of the portable electronic device 30.
In summary, the present invention provides a computer device and a display device, which can automatically push a software package to a portable electronic device for automatic execution when the portable electronic device is connected, so that a user does not need to install a related program on the portable electronic device in advance, thereby ensuring that the version of the software package installed and executed by the portable electronic device corresponds to the version of the screen mirror program on the computer device/display device. The software package executed by the portable electronic device can be used for capturing the screen picture of the portable electronic device to encode the screen picture into a video stream, and the computer device (or the display device) has the capability of decoding the video stream and performing image rendering so as to obtain the screen picture of the portable electronic device and play the screen picture on the display device, or the display device can play the screen picture on a display module of the display device. When the software package executed by the portable electronic device detects that the connection (such as a USB connection) between the portable electronic device 30 and the computer device (or the display device) is disconnected, the portable electronic device automatically deletes the software package, so that the user can use the screen mirroring function more conveniently.
In addition, the user can define the operation function mapping of the keyboard and the mouse on the screen mirror program executed by the computer device/the display device, so that the mobile game mainly comprising touch control clicking can be supported more completely, and the user can control the application program or the mobile game executed by the portable electronic device through the keyboard/the mouse inserted on the computer device or the display device, thereby improving the control convenience and increasing the user experience.
In the claims, the terms "first," "second," "third," and the like are used for modifying elements of the claims, and are not intended to indicate a priority order, a precedence relationship or a precedence of one element over another or the temporal order in which steps of a method are performed, but are used to distinguish between elements having the same name.
While the invention has been described with reference to the preferred embodiments, it will be understood by those skilled in the art that various changes and modifications may be made therein without departing from the spirit and scope of the invention as defined in the following claims.
Claims (12)
1. A computer device, comprising:
A storage device for storing a screen image program;
A processing unit for executing the screen mirror program to execute the following steps:
in response to detecting that a portable electronic device is connected to the computer device, pushing a software package to the portable electronic device to automatically execute the software package in the portable electronic device;
Receiving a video stream from the portable electronic device, wherein the video stream comprises a screen of the portable electronic device;
decoding the video stream, rendering the decoded image to obtain the screen picture, and playing the screen picture on a display device; and
Receiving an input instruction of an input device connected to the computer device, and converting the input instruction into a control instruction used by the portable electronic device according to a key setting interface corresponding to an application program executed by the portable electronic device.
2. The computer device of claim 1, wherein a Universal Serial Bus (USB) debug mode of the portable electronic device is on and the portable electronic device is connected to the computer device via a USB connection.
3. The computer device of claim 2, wherein the software package executed by the portable electronic device obtains the screen from a play-out video buffer of the portable electronic device and encodes the screen into the video stream.
4. The computer device of claim 1, wherein the input command includes a key code, a text code, a mouse signal, a scroll signal, and a touch signal.
5. The computer device of claim 1, wherein the software package executed by the portable electronic device transmits the control command to a touch screen of the portable electronic device to simulate a touch operation associated with the input command, and the touch operation is executed in the application program.
6. The computer device of claim 1, wherein the portable electronic device automatically deletes the software package when the portable electronic device detects that the portable electronic device is disconnected from the computer device.
7. A display device, comprising:
A display module;
a Universal Serial Bus (USB) hub coupled to one or more input devices; and
A display controller for performing the steps of:
in response to detecting that a portable electronic device is connected to the universal serial bus hub, pushing a software package to the portable electronic device through the universal serial bus hub so as to automatically execute the software package in the portable electronic device;
Receiving a video stream from the portable electronic device through the universal serial bus hub, wherein the video stream comprises a screen picture of the portable electronic device;
Decoding the video stream, rendering the decoded image to obtain the screen picture, and playing the screen picture on the display module; and
The input instruction of the input device is received through the universal serial bus hub, and the input instruction is converted into a control instruction used by the portable electronic device according to a key setting interface corresponding to an application program executed by the portable electronic device.
8. The display device of claim 7, wherein a Universal Serial Bus (USB) debug mode of the portable electronic device is on and the portable electronic device is connected to the display device via a USB connection.
9. The display device of claim 8, wherein the software package executed by the portable electronic device obtains the screen from a play-out video buffer of the portable electronic device and encodes the screen into the video stream.
10. The display device of claim 7, wherein the input command includes a key code, a text code, a mouse signal, a scroll signal, and a touch signal.
11. The display device of claim 7, wherein the software package executed by the portable electronic device transmits the control command to a touch screen of the portable electronic device to simulate a touch operation associated with the input command, and the touch operation is executed in the application.
12. The display device of claim 7, wherein the portable electronic device automatically deletes the software package when the portable electronic device detects that the portable electronic device is disconnected from the computer device.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211613682.4A CN118210391A (en) | 2022-12-15 | 2022-12-15 | Computer device and display device |
| TW111149730A TWI826203B (en) | 2022-12-15 | 2022-12-23 | Computer device and display device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211613682.4A CN118210391A (en) | 2022-12-15 | 2022-12-15 | Computer device and display device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118210391A true CN118210391A (en) | 2024-06-18 |
Family
ID=90053309
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211613682.4A Pending CN118210391A (en) | 2022-12-15 | 2022-12-15 | Computer device and display device |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN118210391A (en) |
| TW (1) | TWI826203B (en) |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111641849B (en) * | 2014-10-23 | 2022-07-22 | 应用程序动态有限责任公司 | Universal image receiver |
| CN111324327B (en) * | 2020-02-20 | 2022-03-25 | 华为技术有限公司 | Screen projection method and terminal equipment |
| CN115665454A (en) * | 2020-04-24 | 2023-01-31 | 海信视像科技股份有限公司 | Display device and screen projection method |
| CN113556590B (en) * | 2020-04-24 | 2022-11-11 | 海信视像科技股份有限公司 | Method for detecting effective resolution of screen-projected video stream and display equipment |
| CN111340808A (en) * | 2020-05-20 | 2020-06-26 | 北京小米移动软件有限公司 | Screen display monitoring method and device, electronic equipment and storage medium |
| CN114286165B (en) * | 2020-12-21 | 2023-04-25 | 海信视像科技股份有限公司 | Display equipment, mobile terminal, and screen-throwing data transmission method and system |
| CN114286137A (en) * | 2021-08-23 | 2022-04-05 | 海信视像科技股份有限公司 | Mirror image screen projection method, display device and terminal |
-
2022
- 2022-12-15 CN CN202211613682.4A patent/CN118210391A/en active Pending
- 2022-12-23 TW TW111149730A patent/TWI826203B/en active
Also Published As
| Publication number | Publication date |
|---|---|
| TWI826203B (en) | 2023-12-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| TWI673645B (en) | Computer system, display apparatus, and display method of an on-screen-display interface | |
| US7327353B2 (en) | Computer system and related method for calibrating a digitizer without utilizing calibration software | |
| JP7178397B2 (en) | Electronic device and method of triggering macro keys using external input signals | |
| CN118210391A (en) | Computer device and display device | |
| CN111782169A (en) | Computer system, display device and operation method of display interface on screen | |
| KR102519683B1 (en) | electronic device and method of automatically triggering hot key using display image | |
| CN113126860B (en) | Electronic device and display method of display interface on screen | |
| TWI766518B (en) | Display apparatus and method of synchronizing mouse sensitivities | |
| CN113467729B (en) | Electronic device and multi-screen display method | |
| TWI804116B (en) | Display apparatus and method of image rewinding thereof | |
| CN114690917A (en) | Display device and mouse sensitivity synchronization method | |
| TWI766509B (en) | Display apparatus and control method of projected on-screen-display interface | |
| TWI770979B (en) | Method and apparatus for screen displaying | |
| CN116301681A (en) | Display device and picture backtracking method thereof | |
| CN114697622A (en) | Display device and control method of display interface on projection type screen | |
| CN116343711A (en) | Display device and picture burn-in preventing method thereof | |
| CN113515253A (en) | Working method of display adapter, display adapter and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination |