CN110798708A - Screen projection method, device and system for display content of VR equipment - Google Patents
Screen projection method, device and system for display content of VR equipment Download PDFInfo
- Publication number
- CN110798708A CN110798708A CN201910968604.8A CN201910968604A CN110798708A CN 110798708 A CN110798708 A CN 110798708A CN 201910968604 A CN201910968604 A CN 201910968604A CN 110798708 A CN110798708 A CN 110798708A
- Authority
- CN
- China
- Prior art keywords
- image
- screen display
- display image
- screen
- compressed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 69
- 239000000872 buffer Substances 0.000 claims description 24
- 238000004590 computer program Methods 0.000 claims description 23
- 230000006835 compression Effects 0.000 claims description 21
- 238000007906 compression Methods 0.000 claims description 21
- 230000005540 biological transmission Effects 0.000 claims description 15
- 230000015654 memory Effects 0.000 claims description 12
- 238000004422 calculation algorithm Methods 0.000 claims description 9
- 229910052754 neon Inorganic materials 0.000 claims description 9
- GKAOGPIIYCISHV-UHFFFAOYSA-N neon atom Chemical compound [Ne] GKAOGPIIYCISHV-UHFFFAOYSA-N 0.000 claims description 9
- 230000006837 decompression Effects 0.000 claims description 8
- 238000009877 rendering Methods 0.000 claims description 8
- 230000000694 effects Effects 0.000 abstract description 3
- 238000011161 development Methods 0.000 abstract description 2
- 238000012827 research and development Methods 0.000 abstract description 2
- 238000012545 processing Methods 0.000 description 13
- 230000006870 function Effects 0.000 description 8
- 238000013507 mapping Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 230000005291 magnetic effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000003936 working memory Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/44012—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
Abstract
The invention aims to provide a method, a device and a system for projecting display contents of a VR device to other devices. The VR equipment acquires a current display image of a screen according to a preset frame rate; the VR device compresses the acquired screen display image and sends the compressed screen display image to other devices; the other device receiving and decompressing the compressed screen display image; the other device presents the decompressed screen display image. Compared with the prior art, the invention realizes the high-definition real-time screen projection function of the VR equipment. This effectively has improved the efficiency of development debugging at the research and development stage of VR equipment. When carrying out the product demonstration to VR equipment, improved the effect of demonstration, show clearly and low delay, do not have the card pause.
Description
Technical Field
The invention relates to the technical field of VR (virtual reality), in particular to a technology for projecting display content of VR equipment to other equipment.
Background
VR (Virtual Reality) industry is developing more and more rapidly, often requiring that the picture of the VR device is displayed on a large screen. Since the screen resolution of the VR device is very high, generally 3840 × 2160, VR screen data is transmitted through USB or WIFI, which has high requirement on transmission bandwidth.
The current remote screen projection display scheme, such as 360 mobile phone assistant, app and some other Android screen projection software, generally adopts high compression rate to compress into video stream for transmission. These schemes using video compression and transmission may reduce the sharpness of the picture display and even cause mosaics when the picture is switched rapidly. The approach of using video compression when the screen of the VR device is rotated requires re-initializing the video encoder resulting in a projected screen flash that is dark. Some solutions have very large picture delays, up to 500 milliseconds or even higher.
Still some remote screen projection schemes adopt the adb command to transmit to the PC terminal after screen capturing, resulting in a lower frame rate, which also affects the demonstration effect.
Disclosure of Invention
The invention aims to provide a method, a device and a system for projecting display contents of a VR device to other devices.
According to an aspect of the invention, a method for projecting display content of a VR device to other devices is provided, wherein the method comprises the following steps:
the VR equipment acquires a current display image of a screen according to a preset frame rate;
the VR device compresses the acquired screen display image and sends the compressed screen display image to other devices;
the other device receiving and decompressing the compressed screen display image;
the other device presents the decompressed screen display image.
According to an aspect of the present invention, there is also provided a method implemented on a VR device side for projecting display content thereof to other devices, wherein the method includes the following steps:
acquiring a current display image of a screen according to a preset frame rate;
the acquired screen display image is compressed and the compressed screen display image is transmitted to other devices.
According to an aspect of the present invention, there is also provided a method implemented on the other device side for displaying the display content of the VR device in real time, wherein the method includes the following steps:
receiving a compressed screen display image from VR equipment, wherein the screen display image is a current display image of a screen acquired by the VR equipment according to a preset frame rate;
decompressing the compressed screen display image and presenting the decompressed screen display image.
According to an aspect of the present invention, there is also provided a system in which display content of a VR device is projected to other devices, wherein the system includes the VR device and the other devices;
wherein the VR device includes:
the screen capture device is used for acquiring a current display image of the screen according to a preset frame rate;
image compression means for compressing the acquired screen display image;
image transmitting means for transmitting the compressed screen display image to the other device;
wherein the other device comprises:
image receiving means for receiving the compressed screen display image transmitted by the VR device;
image decompression means for decompressing the compressed screen display image;
and the image rendering device is used for presenting the decompressed screen display image.
According to an aspect of the present invention, there is also provided an apparatus implemented on a VR device side for projecting display content thereof to other devices, wherein the apparatus includes:
the screen capture device is used for acquiring a current display image of the screen according to a preset frame rate;
image compression means for compressing the acquired screen display image;
image transmitting means for transmitting the compressed screen display image to the other device.
According to an aspect of the present invention, there is also provided an apparatus for displaying display content of a VR device in real time, implemented on an other device side, wherein the apparatus includes:
the image receiving device is used for receiving the compressed screen display image from the VR equipment;
image decompression means for decompressing the compressed screen display image;
and the image rendering device is used for presenting the decompressed screen display image.
According to an aspect of the present invention, there is also provided a VR device, including a memory and a processor, where the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the method implemented at the VR device for projecting the display content to other devices is implemented.
According to an aspect of the present invention, there is also provided a computer program product, which includes computer program instructions, when the computer program instructions are executed by a VR device, the method implemented at the VR device for projecting the display content thereof to other devices is implemented.
According to an aspect of the present invention, there is also provided a computer-readable storage medium storing computer program instructions, which when executed by a VR device, implement a method implemented at a VR device for projecting display content thereof to other devices as described above.
According to an aspect of the present invention, there is also provided a computer device, including a memory and a processor, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the method for displaying the display content of the VR device in real time implemented at the other device is implemented.
According to an aspect of the present invention, there is also provided a computer program product comprising computer program instructions which, when executed by a computer device, implement a method for real-time display of display content of a VR device implemented at an other device as described above.
According to an aspect of the present invention, there is also provided a computer-readable storage medium storing computer program instructions, which when executed by a computer device, implement a method for real-time displaying display content of a VR device implemented at an other device as described above.
Compared with the prior art, the invention realizes the high-definition real-time screen projection function of the VR equipment. This effectively has improved the efficiency of development debugging at the research and development stage of VR equipment. When carrying out the product demonstration to VR equipment, improved the effect of demonstration, show clearly and low delay, do not have the card pause.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments made with reference to the following drawings:
fig. 1 shows a flowchart of a method for a VR device to screen display content to other devices, according to an embodiment of the invention;
fig. 2 shows a schematic diagram of an apparatus according to an embodiment of the present invention, in which a system in which display content of a VR device is projected to other devices is specifically shown.
The same or similar reference numbers in the drawings identify the same or similar elements.
Detailed Description
Before discussing exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although a flowchart may describe the operations as a sequential process, many of the operations can be performed in parallel, concurrently, or simultaneously. In addition, the order of the operations may be re-arranged. The process may be terminated when its operations are completed, but may have additional steps not included in the figure. The processes may correspond to methods, functions, procedures, subroutines, and the like.
Specific structural and functional details disclosed herein are merely representative and are provided for purposes of describing example embodiments of the present invention. The present invention may, however, be embodied in many alternate forms and should not be construed as limited to only the embodiments set forth herein.
It will be understood that, although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first element may be termed a second element, and, similarly, a second element may be termed a first element, without departing from the scope of example embodiments. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may be present. In contrast, when an element is referred to as being "directly connected" or "directly coupled" to another element, there are no intervening elements present. Other words used to describe the relationship between elements (e.g., "between" versus "directly between", "adjacent" versus "directly adjacent to", etc.) should be interpreted in a similar manner.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used herein, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It should also be noted that, in some alternative implementations, the functions/acts noted may occur out of the order noted in the figures. For example, two figures shown in succession may, in fact, be executed substantially concurrently, or the figures may sometimes be executed in the reverse order, depending upon the functionality/acts involved.
The present invention is described in further detail below with reference to the attached drawing figures.
Fig. 1 shows a flow diagram of a method according to an embodiment of the invention, which particularly shows a process in which display content of a VR device is projected to other devices.
As shown in fig. 1, in step S1, the VR device acquires a current display image of the screen at a predetermined frame rate; in step S2, the VR device compresses the acquired screen display image and transmits the compressed screen display image to the other device; in step S3, the other device receives and decompresses the compressed screen display image; in step S4, the other device presents the decompressed screen display image.
Wherein, the VR device can install and run a specific application to perform the operations performed by the VR device; likewise, other devices may also be equipped to perform the operations described above as being performed by other devices and running a particular application. The aforementioned specific application, typically a program function module, may be represented as an APP.
Here, the other device is typically a computer device, including but not limited to any electronic device coupled with a display device, which can execute a predetermined process such as numerical calculation and/or logic calculation by executing a predetermined program or instruction, and usually includes a processor and a memory, the predetermined process is executed by the processor by executing a program instruction prestored in the memory, or the predetermined process is executed by hardware such as ASIC, FPGA, DSP, or a combination of the two. Computer devices include, but are not limited to, Personal Computers (PCs), laptops, tablets, and the like.
VR devices are typically integrated VR devices, and specifically, VR head displays (virtual reality head-mounted display devices) with independent processors, and have independent computing, input, and output functions.
Referring to fig. 1, in step S1, the VR device acquires a current display image of a screen at a predetermined frame rate; in step S2, the VR device compresses the acquired screen display image and transmits the compressed screen display image to another device, such as a computer device.
Here, the VR device may capture a current display image of the screen directly by capturing the screen image, or may sample the video image.
Further, the VR device may further include two buffers: the first cache region caches the screen display images acquired by the VR equipment, and the second cache region caches the screen display images compressed by the VR equipment.
Specifically, according to a preferred embodiment of the present invention, in step S1, the VR device acquires the current display image of the screen at a predetermined frame rate and buffers the current display image in the first buffer area; in step S2, the VR device fetches the cached screen display image from the first cache region and compresses the screen display image into the second cache region, and then fetches the compressed screen display image from the second cache region and transmits it to the other device.
The above process at the VR device side can be further divided into 3 threads:
thread 1 is used to capture high resolution on-screen display images, for example 30 frames per second, place the captured image data in the first buffer and notify thread 2 to process the data and then continue to capture on-screen display images.
And the thread 2 is used for acquiring the original image data stored by the thread 1, compressing the original image data in the BMP format into high-quality JPEG image data by adopting an NEON accelerated image compression algorithm, storing the JPEG image data into a second cache region, and informing the thread 3 of processing the compressed image data.
Here, by using the image compression algorithm accelerated by the nen, the processing time of image compression can be effectively shortened, and the frame rate (FPS) of the transmission image can be improved.
And the thread 3 is used for acquiring the compressed screen display image stored by the thread 2 and transmitting the compressed screen display image to other remote equipment, such as a PC (personal computer) through a preset socket port.
Preferably, the VR device may further determine whether the current image is the same as the previous image, and if so, do no processing and continue to acquire the screen display image; if not, the current image is compressed.
Specifically, one implementation is for the VR device to compare two screen display images acquired before and after, and when the two are the same, discard the screen display image acquired after (i.e., the current image). The VR device then proceeds to capture the on-screen display image. Therefore, the same picture is prevented from being transmitted to a receiving end, and the load of transmission bandwidth can be effectively reduced.
Another way to achieve this is for the VR device to compare two screen display images acquired before and after, and when the two are different, compress the screen display image acquired after (i.e., the current image). For example, when the thread 1 judges that the current image is different from the previous frame image, the thread 1 puts the current image data into a first cache region and informs the thread 2 to process the data, and then continues to acquire the screen display image; and the thread 2 acquires the original BMP image data stored in the thread 1, compresses the original BMP image data into high-quality JPEG image data by adopting an NEON accelerated image compression algorithm, stores the JPEG image data into a second cache region, and informs the thread 3 of processing the compressed screen image data.
Next, in step S3, the computer device receives and decompresses the compressed screen display image; in step S4, the computer device presents the decompressed screen display image.
Further, the computer device may also include two buffers: the third cache region caches the compressed screen display images received by the computer device, and the fourth cache region caches the decompressed screen display images.
Specifically, according to a preferred embodiment of the present invention, in step S3, the computer device receives the compressed screen display image from the VR device and buffers it in the third buffer, and then fetches the compressed screen display image from the third buffer and decompresses it and stores it in the fourth buffer; in step S4, the computer apparatus fetches the decompressed screen display image from the fourth buffer and presents the decompressed screen display image in its display device.
The above process at the computer device side can also be further divided into 3 threads:
and the thread 1 is used for receiving the screen image data which is transmitted by the VR device and is compressed by NEON, putting the screen image data into a third buffer area, and then informing the thread 2 of processing.
And after receiving the notification of the thread 1, the thread 2 acquires the compressed screen image data from the third cache region, decompresses the compressed screen image data, puts the decompressed screen image data into a fourth cache region, and then notifies the thread 3 to process the decompressed screen image data.
And after receiving the notification of the thread 2, the thread 3 acquires decompressed original screen image data from the fourth cache region, and draws the decompressed original screen image data to a specified window in the display Device through a Graphics Device Interface (GDI).
And the receiving end (computer equipment) receives the compressed image data, decompresses the compressed image data to obtain the original BMP screen image data with high resolution, and then draws the original BMP screen image data on a corresponding window through GDI. Therefore, the receiving end can dynamically adjust the size of the display window or display the full screen according to the size of the resolution of the display.
According to the invention, the VR device can capture the image of the current display content of the VR device in real time and send the image to the computer device, so that the computer device can synchronize the current display content of the VR device.
Here, to reduce the data processing amount and the data transmission amount, the VR device may compare the currently captured image with the last captured image, and when the currently captured image and the last captured image are the same, the VR device does not perform processing, that is, only when the currently displayed content of the VR device changes, the changed displayed image is synchronized to the computer device through the designated socket port. VR equipment passes through TCP with the computer equipment and is connected, can throw the screen to computer equipment with VR's display content fast, and the real-time is higher.
The process shown in fig. 1 may further include a step S0 (not shown) according to an embodiment of the present invention. In step S0, the VR device establishes a TCP connection with the other device.
Specifically, a TCP connection is established between the VR device and the computer device through adb port mapping.
And the VR equipment is connected with the computer equipment through USB or WIFI, and uses an adforward command to perform port mapping under the condition that adb works normally. Accordingly, a TCP connection is established between the VR device and the computer device. The VR device may then transfer images to and from the computer device over the TCP connection.
Fig. 2 shows a schematic apparatus diagram according to an embodiment of the present invention, which particularly shows a system in which the display content of a VR device is projected to other devices.
As shown in fig. 1, the system 20 includes a VR device 21 and other devices (e.g., a computer device 22), the VR device 21 further includes a screen capturing apparatus 211, an image compressing apparatus 212, and an image transmitting apparatus 213, and the computer device 22 further includes an image receiving apparatus 221, an image decompressing apparatus 222, and an image rendering apparatus 223.
On the VR device 21 side, the screen capture device 211 obtains a current display image of the screen at a predetermined frame rate; the image decompression means 212 compresses the screen display image acquired by the screen capture means 211; the image transmission means 213 transmits the compressed screen display image to the computer device 22; subsequently, on the computer device 22 side, the image receiving device 221 receives the compressed screen display image; the image decompression device 222 decompresses the compressed screen display image; the image rendering device 223 renders the decompressed screen display image.
Referring to fig. 2, the screen capture device 211 acquires a currently displayed image of the screen at a predetermined frame rate; the image decompression means 212 compresses the screen display image acquired by the screen capture means 211; the image transmitting means 213 transmits the compressed screen display image to a computer device 22, such as a computer device.
Here, the screen capture device 211 may capture the current display image of the screen directly by capturing the screen image or sampling the video image.
Further, the VR device 21 may further include two buffers: the first cache region caches the screen display images acquired by the VR equipment, and the second cache region caches the screen display images compressed by the VR equipment.
Specifically, according to a preferred embodiment of the present invention, the screen capture device 211 acquires a current display image of the screen at a predetermined frame rate and buffers the current display image into the first buffer area; the image compression means 212 fetches the cached screen display image from the first cache region and compresses the screen display image into the second cache region, and then the image transmission means 213 fetches the compressed screen display image from the second cache region and transmits it to the computer device 22.
The above process can be further divided into 3 threads:
thread 1 is used to capture high resolution on-screen display images, for example 30 frames per second, place the captured image data in the first buffer and notify thread 2 to process the data and then continue to capture on-screen display images.
And the thread 2 is used for acquiring the original image data stored by the thread 1, compressing the original image data in the BMP format into high-quality JPEG image data by adopting an NEON accelerated image compression algorithm, storing the JPEG image data into a second cache region, and informing the thread 3 of processing the compressed image data.
Here, by using the image compression algorithm accelerated by the nen, the processing time of image compression can be effectively shortened, and the frame rate (FPS) of the transmission image can be improved.
And the thread 3 is used for acquiring the compressed screen display image stored by the thread 2 and transmitting the compressed screen display image to other remote equipment, such as a PC (personal computer) through a preset socket port.
Preferably, the screen capture device 211 or other devices of the VR device 21 may also determine whether the current image is the same as the previous frame image, and if so, do nothing and continue to acquire the screen display image; if not, the current image is compressed.
Specifically, in one implementation, the screen capture device 211 compares two screen display images acquired before and after, and discards the screen display image acquired after (i.e., the current image) when the two are the same. Then, the screen capture device 211 continues to acquire the screen display image. Therefore, the same picture can be prevented from being transmitted to a receiving end, and the load of the transmission bandwidth can be effectively reduced.
In another implementation, the screen capture device 211 compares two screen display images acquired before and after, and when the two are different, the image compression device 212 compresses the screen display image acquired after (i.e., the current image). For example, when the thread 1 judges that the current image is different from the previous frame image, the thread 1 puts the current image data into a first cache region and informs the thread 2 to process the data, and then continues to acquire the screen display image; and the thread 2 acquires the original BMP image data stored in the thread 1, compresses the original BMP image data into high-quality JPEG image data by adopting an NEON accelerated image compression algorithm, stores the JPEG image data into a second cache region, and informs the thread 3 of processing the compressed screen image data.
Next, the image receiving device 221 receives the compressed screen display image; the image decompression device 222 decompresses the compressed screen display image; the image rendering device 223 renders the decompressed screen display image.
Further, the computer device 22 may also include two buffers: the third cache region caches the compressed screen display images received by the computer device, and the fourth cache region caches the decompressed screen display images.
Specifically, according to a preferred embodiment of the present invention, the image receiving device 221 receives the compressed screen display image from the VR device and buffers it in the third buffer, and then the image decompressing device 222 fetches the compressed screen display image from the third buffer and decompresses it and stores it in the fourth buffer; the image rendering device 223 takes the decompressed screen display image out of the fourth buffer and presents the decompressed screen display image in its display device.
The above process at the computer device 22 side can also be further divided into 3 threads:
and the thread 1 is used for receiving the screen image data which is transmitted by the VR device and is compressed by NEON, putting the screen image data into a third buffer area, and then informing the thread 2 of processing.
And after receiving the notification of the thread 1, the thread 2 acquires the compressed screen image data from the third cache region, decompresses the compressed screen image data, puts the decompressed screen image data into a fourth cache region, and then notifies the thread 3 to process the decompressed screen image data.
And after receiving the notification of the thread 2, the thread 3 acquires decompressed original screen image data from the fourth cache region, and draws the decompressed original screen image data to a specified window in the display Device through a Graphics Device Interface (GDI).
And the receiving end (computer equipment) receives the compressed image data, decompresses the compressed image data to obtain the original BMP screen image data with high resolution, and then draws the original BMP screen image data on a corresponding window through GDI. Therefore, the receiving end can dynamically adjust the size of the display window or display the full screen according to the size of the resolution of the display.
According to the invention, the VR device can capture the image of the current display content of the VR device in real time and send the image to the computer device, so that the computer device can synchronize the current display content of the VR device.
Here, to reduce the data processing amount and the data transmission amount, the screen capture device 211 of the VR device 21 may compare the currently captured image with the last captured image, and when the two are the same, do no processing, that is, only when the currently displayed content of the VR device changes, the changed displayed image is synchronized to the computer device through the designated socket port.
According to one embodiment of the invention, the VR device 21 shown in fig. 2 further comprises a first connection means (not shown), and the computer device 22 further comprises a second connection means (not shown). The first and second connection means establish a TCP connection between the VR device 21 and the computer device 22.
Specifically, a TCP connection is established between the first connection device and the second connection device through an adb port mapping.
The first connecting device and the second connecting device are connected through USB or WIFI, and port mapping is carried out by using an adb forward command under the condition that the adb works normally. Accordingly, a TCP connection is established between the VR device and the computer device. Subsequently, the first connection means and the second connection means can perform image transmission between the VR device 21 and the computer device 22 through the TCP connection.
It should be noted that the present invention may be implemented in software and/or in a combination of software and hardware, for example, as an Application Specific Integrated Circuit (ASIC), a general purpose computer or any other similar hardware device. In one embodiment, the software program of the present invention may be executed by a processor to implement the steps or functions described above. Also, the software programs (including associated data structures) of the present invention can be stored in a computer readable recording medium, such as RAM memory, magnetic or optical drive or diskette and the like. Further, some of the steps or functions of the present invention may be implemented in hardware, for example, as circuitry that cooperates with the processor to perform various steps or functions.
In addition, some of the present invention can be applied as a computer program product, such as computer program instructions, which when executed by a computer, can invoke or provide the method and/or technical solution according to the present invention through the operation of the computer. Program instructions which invoke the methods of the present invention may be stored on a fixed or removable recording medium and/or transmitted via a data stream on a broadcast or other signal-bearing medium and/or stored within a working memory of a computer device operating in accordance with the program instructions. An embodiment according to the invention herein comprises an apparatus comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein the computer program instructions, when executed by the processor, trigger the apparatus to perform a method and/or solution according to embodiments of the invention as described above.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned. Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the system claims may also be implemented by one unit or means in software or hardware. The terms first, second, etc. are used to denote names, but not any particular order.
Claims (23)
1. A method of a VR device whose display is projected onto another device, wherein the method comprises the steps of:
the VR equipment acquires a current display image of a screen according to a preset frame rate;
the VR device compresses the acquired screen display image and sends the compressed screen display image to other devices;
the other device receiving and decompressing the compressed screen display image;
the other device presents the decompressed screen display image.
2. The method of claim 1, wherein the VR device further comprises two buffers;
the first cache region caches the screen display images acquired by the VR equipment, and the second cache region caches the screen display images compressed by the VR equipment.
3. The method of claim 1 or 2, wherein the other device further comprises two buffers;
the third cache region caches the compressed screen display images received by the other devices, and the fourth cache region caches the screen display images decompressed by the other devices.
4. The method according to claim 3, wherein the presenting step specifically comprises:
and drawing the decompressed screen display image in a specified display window by the other equipment.
5. The method of any of claims 1 to 4, wherein the method further comprises:
and the VR equipment establishes a TCP connection with the other equipment for image transmission.
6. The method according to any one of claims 1 to 5, wherein the compressing step employs a NEON accelerated image compression algorithm.
7. A method implemented at a VR device for projecting its display content to other devices, wherein the method comprises the steps of:
acquiring a current display image of a screen according to a preset frame rate;
the acquired screen display image is compressed and the compressed screen display image is transmitted to other devices.
8. The method of claim 7, wherein the compressing step further comprises:
and comparing the two screen display images acquired before and after the comparison, and compressing the screen display image acquired after the comparison when the two screen display images are different.
9. The method of claim 7, wherein the obtaining step further comprises:
comparing the two screen display images acquired before and after the comparison, and discarding the screen display image acquired after the comparison when the two screen display images are the same.
10. The method of any of claims 7 to 9, wherein the method further comprises:
and establishing a TCP connection with the other equipment for image transmission.
11. The method according to any one of claims 7 to 10, wherein the compressing step employs a NEON accelerated image compression algorithm.
12. A method implemented on an other device side for displaying display content of a VR device in real time, wherein the method comprises the steps of:
receiving a compressed screen display image from VR equipment, wherein the screen display image is a current display image of a screen acquired by the VR equipment according to a preset frame rate;
decompressing the compressed screen display image and presenting the decompressed screen display image.
13. The method of claim 12, wherein the method comprises:
and establishing a TCP connection with the VR device for image transmission.
14. The method according to claim 12 or 13, wherein the compression employs a NEON accelerated image compression algorithm.
15. A system for projecting display content of a VR device to other devices, wherein the system comprises the VR device and the other devices;
wherein the VR device includes:
the screen capture device is used for acquiring a current display image of the screen according to a preset frame rate;
image compression means for compressing the acquired screen display image;
image transmitting means for transmitting the compressed screen display image to the other device;
wherein the other device comprises:
image receiving means for receiving the compressed screen display image transmitted by the VR device;
image decompression means for decompressing the compressed screen display image;
and the image rendering device is used for presenting the decompressed screen display image.
16. An apparatus implemented at a VR device for projecting its display content to other devices, wherein the apparatus comprises:
the screen capture device is used for acquiring a current display image of the screen according to a preset frame rate;
image compression means for compressing the acquired screen display image;
image transmitting means for transmitting the compressed screen display image to the other device.
17. An apparatus for displaying display content of a VR device in real time implemented on an other device side, wherein the apparatus comprises:
the image receiving device is used for receiving the compressed screen display image from the VR equipment;
image decompression means for decompressing the compressed screen display image;
and the image rendering device is used for presenting the decompressed screen display image.
18. A VR device comprising a memory and a processor, wherein the memory stores computer program instructions that, when executed by the processor, implement the method of any of claims 7 to 11.
19. A computer program product comprising computer program instructions to implement the method of any of claims 7 to 11 when executed by a VR device.
20. A computer readable storage medium storing computer program instructions which, when executed by a VR device, implement the method of any of claims 7 to 11.
21. A computer apparatus comprising a memory and a processor, wherein the memory stores computer program instructions that, when executed by the processor, implement the method of any of claims 12 to 14.
22. A computer program product comprising computer program instructions which, when executed by a computer device, implement the method of any one of claims 12 to 14.
23. A computer readable storage medium storing computer program instructions which, when executed by a computer device, implement the method of any one of claims 12 to 14.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910968604.8A CN110798708A (en) | 2019-10-12 | 2019-10-12 | Screen projection method, device and system for display content of VR equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910968604.8A CN110798708A (en) | 2019-10-12 | 2019-10-12 | Screen projection method, device and system for display content of VR equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110798708A true CN110798708A (en) | 2020-02-14 |
Family
ID=69439016
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910968604.8A Pending CN110798708A (en) | 2019-10-12 | 2019-10-12 | Screen projection method, device and system for display content of VR equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110798708A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170214887A1 (en) * | 2014-05-07 | 2017-07-27 | Remote Xccess, LLC | Camera Headband Device And System With Attachable Apparatus |
| CN107613338A (en) * | 2017-09-25 | 2018-01-19 | 中兴通讯股份有限公司 | Video transmission method, server, VR playback terminals and computer-readable recording medium |
| CN109218731A (en) * | 2017-06-30 | 2019-01-15 | 腾讯科技（深圳）有限公司 | The throwing screen method, apparatus and system of mobile device |
| CN109960449A (en) * | 2019-03-22 | 2019-07-02 | 深圳前海达闼云端智能科技有限公司 | A kind of throwing screen display methods and relevant apparatus |
| CN110224981A (en) * | 2019-05-05 | 2019-09-10 | 重庆爱奇艺智能科技有限公司 | A kind of method and apparatus for connecting equipment with target for VR all-in-one machine and being interconnected |
| CN110221798A (en) * | 2019-05-29 | 2019-09-10 | 华为技术有限公司 | A kind of throwing screen method, system and relevant apparatus |
-
2019
- 2019-10-12 CN CN201910968604.8A patent/CN110798708A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170214887A1 (en) * | 2014-05-07 | 2017-07-27 | Remote Xccess, LLC | Camera Headband Device And System With Attachable Apparatus |
| CN109218731A (en) * | 2017-06-30 | 2019-01-15 | 腾讯科技（深圳）有限公司 | The throwing screen method, apparatus and system of mobile device |
| CN107613338A (en) * | 2017-09-25 | 2018-01-19 | 中兴通讯股份有限公司 | Video transmission method, server, VR playback terminals and computer-readable recording medium |
| CN109960449A (en) * | 2019-03-22 | 2019-07-02 | 深圳前海达闼云端智能科技有限公司 | A kind of throwing screen display methods and relevant apparatus |
| CN110224981A (en) * | 2019-05-05 | 2019-09-10 | 重庆爱奇艺智能科技有限公司 | A kind of method and apparatus for connecting equipment with target for VR all-in-one machine and being interconnected |
| CN110221798A (en) * | 2019-05-29 | 2019-09-10 | 华为技术有限公司 | A kind of throwing screen method, system and relevant apparatus |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9665332B2 (en) | Display controller, screen transfer device, and screen transfer method | |
| JP6263830B2 (en) | Techniques for including multiple regions of interest indicators in compressed video data | |
| US10402940B2 (en) | Method and system for accelerating video preview digital camera | |
| US8463020B1 (en) | Centralized immersive image rendering for thin client | |
| US20030017846A1 (en) | Wireless display | |
| KR101698951B1 (en) | System, apparatus and method for sharing a screen having multiple visual components | |
| WO2012079453A1 (en) | Video data processing system and method, cpu, gpu and video monitoring system | |
| CN111625211B (en) | Screen projection method and device, android device and display device | |
| US10554989B2 (en) | Efficient encoding of display data | |
| CA2737842C (en) | Remote user interface | |
| CN105338323A (en) | Video monitoring method and device | |
| US20220058771A1 (en) | Image Processing Method in Remote Control, Device, Apparatus and Program Product | |
| WO2013086530A2 (en) | Method and apparatus for processing partial video frame data | |
| US20090153434A1 (en) | Display assistant system | |
| WO2013134936A1 (en) | Screen recording method, and screen recording control method and device | |
| CN110740352B (en) | SPICE protocol-based difference image display method in video card transparent transmission environment | |
| WO2018032696A1 (en) | Method and system for desktop screenshot control | |
| CN110798708A (en) | Screen projection method, device and system for display content of VR equipment | |
| JP2006246110A (en) | Apparatus and system for transmitting video | |
| US9118955B2 (en) | Stereo image handling device and method | |
| JP2790538B2 (en) | Two-way simple video communication system | |
| US20020154889A1 (en) | Video pre-processing/post-processing method for processing video efficiently and pre-processing/post-processing apparatus using the same | |
| CN110519530B (en) | Hardware-based picture-in-picture display method and device | |
| CN115623217B (en) | Image preprocessing method, device and system | |
| CN113901009A (en) | Multimedia file processing method and device, electronic equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20200214 |