WO2020042494A1

WO2020042494A1 - Method for screenshot of vr scene, device and storage medium

Info

Publication number: WO2020042494A1
Application number: PCT/CN2018/123764
Authority: WO
Inventors: 张向军
Original assignee: 歌尔股份有限公司
Priority date: 2018-08-31
Filing date: 2018-12-26
Publication date: 2020-03-05
Also published as: CN109002248A; CN109002248B

Abstract

Some embodiments of the present application disclose a method for screenshot of a scene, a device and a storage medium. In the present embodiment, at least two reference fields of view are selected in a three-dimensional virtual scene that can be displayed by a VR device, and edges of two adjacent reference fields of view are superimposed; next, virtual scene segments corresponding to the at least two reference fields of view are acquired, and images of the virtual scene segments corresponding to the at least two reference fields of view are spliced; and then the generated VR scene screenshots match the three-dimensional virtual scene actually displayed by the VR device, effectively capturing the real display content of the VR device.

Description

VR scene screenshot method, device and storage medium

This application claims priority from a Chinese patent application filed on August 31, 2018 with the Chinese Patent Office, application number 201811013485.2, and the invention name is "VR scene screenshot method, device, and storage medium", the entire contents of which are incorporated herein by reference. Applying.

Technical field

The present invention relates to the technical field of virtual reality (VR), and in particular, to a method, a device, and a storage medium for capturing a VR scene.

Background technique

Screenshot is a way to capture the displayed content. The user can save the content displayed on the screen as an image file through the screenshot operation. For example, a smartphone can save the content displayed on the screen through the screenshot operation.

At present, when a two-dimensional image display device performs a screenshot operation, usually the content currently displayed on the screen is directly taken as a target screenshot. However, when a three-dimensional image display device, such as a VR device, performs a screenshot operation, since the image displayed on the screen of the VR device is a processed image, the screenshot obtained by directly intercepting the content currently displayed on the screen of the VR device is the same as that of the VR device The virtual scenes shown are generally different.

Summary of the Invention

Some embodiments of the present invention provide a VR scene screenshot method, device and storage medium, which are used to capture the 3D virtual scene actually displayed by the VR device according to the screenshot instruction during the VR device display of the VR scene.

Some embodiments of the present invention provide a VR scene screenshot method, including: in response to a screenshot instruction, selecting at least two reference fields of view in a three-dimensional virtual scene that can be displayed by a VR device, and the edge of the field of view of two adjacent reference fields of view Superimpose; obtain virtual scene segments corresponding to the at least two reference fields of view; perform image stitching on the virtual scene segments corresponding to the at least two reference fields of view to generate a VR scene screenshot.

Optionally, selecting at least two reference fields of view in the three-dimensional virtual scene displayed by the VR device includes: uniformly selecting at least two different user viewpoints in the three-dimensional virtual scene; and viewing the at least two different user viewpoints. The field FOV is set to obtain at least two reference fields of view, where at least one field of view satisfies FOV> 360 ° / N, where N is the number of user viewpoints included in the at least two user viewpoints.

Optionally, uniformly selecting at least two different user viewpoints in the three-dimensional virtual scene includes: obtaining a user's current head position pose data; and determining the head position pose data separately in the three-dimensional virtual scene. Determining the left and / or right eye viewpoints of the user; determining the base user viewpoint based on the left and / or right eye viewpoints of the user; selecting at least uniformly distributed with the base user viewpoint within the three-dimensional virtual scene One viewpoint serves as a secondary user viewpoint.

Optionally, the number of viewpoints of the auxiliary user is two.

Optionally, obtaining virtual scene fragments corresponding to the at least two reference fields of view includes: sequentially performing scene rendering at each of the user viewpoints of the at least two different user viewpoints to render the at least two Virtual scene clips corresponding to each reference field of view.

Optionally, image stitching the virtual scene segments corresponding to the at least two reference fields of view to generate a VR scene screenshot includes: performing edge similarity detection on the virtual scene segments corresponding to the at least two reference fields of view. ; According to the detection result of the edge similarity point detection, the virtual scene segments corresponding to the at least two reference fields of view whose edge similarity is greater than a set threshold are spliced.

Optionally, performing image stitching on the virtual scene segments corresponding to the at least two reference fields of view to generate a VR scene screenshot includes determining the basis from the virtual scene segments corresponding to the at least two reference fields of view. The virtual scene segment corresponding to the user viewpoint is the stitching center; according to the positional relationship between the auxiliary user viewpoint and the basic user viewpoint, determining the relative position of the virtual scene fragment corresponding to the auxiliary user viewpoint and the stitching center; The stitching center and the relative position perform image stitching on the virtual scene segments corresponding to the at least two reference fields of view to generate a screenshot of the VR scene. Further optionally, the method further includes: storing the VR scene screenshot to a designated path of the VR device, and notifying a user that a screen capture operation has been completed and / or displaying the designated path of the VR scene screenshot.

Other embodiments of the present invention further provide a VR device, including: a memory and a processor; wherein the memory is used to store one or more computer instructions; the processor is coupled to the memory and used to execute the memory; One or more computer instructions for performing a VR scene screenshot method provided by the present invention.

Still other embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which when executed can implement the VR scene screenshot method provided by the present invention.

In the VR scene screenshot method provided by some embodiments of the present invention, at least two reference fields of view are selected in a three-dimensional virtual scene that can be displayed by the VR device, and the edges of the two adjacent reference fields of view are superimposed; then, at least two Virtual scene clips corresponding to each reference field of view, and image stitching of at least two virtual scene clips corresponding to each of the reference field of view, and further, the generated VR scene screenshot matches the 3D virtual scene actually displayed by the VR device, effectively capturing Real display content of VR equipment.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to explain the technical solutions in the embodiments of the present application or the prior art more clearly, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are merely It is a part of the drawings of the present application. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without paying creative labor.

FIG. 1a is a schematic diagram of a screenshot process of a two-dimensional display device; FIG.

FIG. 1b is a schematic diagram of a three-dimensional display principle of a VR device; FIG.

FIG. 1c is a schematic diagram of a screen display content of a VR device; FIG.

FIG. 1d is a schematic flowchart of a VR scene screenshot method provided by some embodiments of the present invention; FIG.

2a is a schematic flowchart of a VR scene screenshot method provided by some embodiments of the present invention;

FIG. 2b is a schematic diagram of correspondence between different user viewpoints and VR camera viewpoints provided by some embodiments of the present invention; FIG.

2c is a schematic diagram of splicing a virtual scene segment provided by some embodiments of the invention;

3 is a schematic structural diagram of a VR device according to some embodiments of the present invention;

FIG. 4 is a schematic structural diagram of a VR device according to some embodiments of the present invention.

detailed description

The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

The screenshot operation of the two-dimensional image display device (taking a mobile phone as an example) can be shown in FIG. 1a. The content displayed on the screen of the two-dimensional image display device is the real display content of the two-dimensional image display device at a certain moment. When the two-dimensional image display device receives a screenshot instruction, it can directly capture the real content displayed on the current screen and save it as a picture (for example, a jpg file). But for VR devices, as shown in Figure 1b, the user wearing the VR device is in a 3D virtual space at any time, and the image that the user can see in the 3D space should be in the entire 3D virtual space. image. Unlike the two-dimensional image display device, the content displayed on the screen of the VR device is only a part of the three-dimensional space that the VR device can display at a certain moment, and is usually subjected to distortion processing, as shown in FIG. 1c. Therefore, when a VR device receives a screenshot instruction, simply capturing the content being displayed on the screen cannot meet the actual screenshot requirements. In view of the above technical problems, the present invention provides a solution, which will be described in detail below.

FIG. 1d is a schematic flowchart of a VR scene screenshot method provided by some embodiments of the present invention. As shown in FIG. 1d, the method includes:

Step 101: In response to a screenshot instruction, at least two reference fields of view are selected in a three-dimensional virtual scene that can be displayed by the VR device, and the edges of the two adjacent reference fields of view are superimposed.

Step 102: Obtain a virtual scene segment corresponding to each of the at least two reference fields of view.

Step 103: Perform image stitching on the virtual scene segments corresponding to the at least two reference fields of view to generate a VR scene screenshot.

In this embodiment, the screenshot instruction may be initiated by a user, or may be initiated by a VR device. When the screenshot instruction is initiated by the user, the user can send a screenshot instruction through a specific physical button on the VR device, or send a screenshot instruction through a set hand or head movement, or send a screenshot instruction through a voice wake-up. This implementation Examples do not limit this. When a screenshot instruction is initiated by a VR device, the screenshot instruction may be sent by an application currently running on the VR device, or may be sent by the VR device according to a set screenshot period, depending on the application scenario.

Based on the above description, in order to capture the real display content of the VR device, in response to the screenshot instruction, at least two reference fields of view can be selected within the three-dimensional virtual scene that the VR device can display. The reference field of view is used to simulate the field of view of the user in the three-dimensional virtual scene. Considering the limit value of the human eye field of view (not exceeding 180 °) and the realistic and immersive needs of the three-dimensional virtual scene, at least two reference fields of view may be selected. The at least two reference fields of view can simulate at least two fields of view of the user in the three-dimensional virtual scene. Among the at least two reference fields of view, the edges of the fields of view of two adjacent reference fields of view are superimposed, and the field of view ranges of the at least two reference fields of view can cover the panorama of the three-dimensional virtual scene.

Next, after obtaining at least two reference fields of view, determine virtual scene segments corresponding to the at least two reference fields of view, and perform image stitching on the virtual scene segments corresponding to the at least two reference fields of view, and stitch the resulting VR scenes. The screenshot shows the 3D scene that the VR device is showing when the screenshot instruction is received.

In some embodiments of the present application, at least two reference fields of view are selected in a three-dimensional virtual scene that can be displayed by the VR device, and the edges of the two adjacent reference fields of view are superimposed; then, each of the at least two reference fields of view is obtained correspondingly Virtual scene clips, and image stitching of at least two virtual scene clips corresponding to the reference field of view, and further, the generated VR scene screenshot matches the 3D virtual scene actually displayed by the VR device, effectively capturing the real display content of the VR device .

FIG. 2a is a schematic flowchart of a VR scene screenshot method according to another embodiment of the present invention. As shown in FIG. 2, the method includes:

Step 201: In response to a screenshot instruction, at least two different user viewpoints are uniformly selected in a three-dimensional virtual scene that can be displayed by the VR device.

Step 202: Set the field of view angles FOV of the at least two different user viewpoints to obtain at least two reference fields of view, where at least one field of view satisfies FOV> 360 ° / N, where N is the at least two The number of user viewpoints contained in each user viewpoint.

Step 203: Perform scene rendering in sequence at each of the user viewpoints of the at least two different user viewpoints to render and obtain virtual scene fragments corresponding to the at least two reference viewpoints.

Step 204: Perform edge similarity detection on the virtual scene segments corresponding to the at least two reference fields of view.

Step 205: According to the detection result of the edge similarity detection, among the virtual scene fragments corresponding to the at least two reference fields of view, the virtual scene fragments whose edge similarity is greater than a set threshold are stitched.

Step 206: Store the VR scene screenshot to a designated path of the VR device, and notify the user that the screen capture operation has been completed and / or the designated path of the VR scene screenshot is displayed.

In step 201, the user viewpoint refers to a visual base point in the three-dimensional virtual scene when the user views the three-dimensional virtual scene, and the visual base point generally includes viewing position information and viewing direction information, as shown in viewpoint A and viewpoint shown in FIG. 2b. B and viewpoint C. In the field of VR technology, as a user's viewpoint changes, a VR device can display a virtual scene that matches the user's viewpoint. In some exemplary embodiments, the VR scene is developed by development tools such as Unity 3D. These development tools can create a three-dimensional virtual space, design a three-dimensional virtual scene in the three-dimensional virtual space, and design a VR virtual camera to simulate the user's Eyes, the viewpoint of the VR virtual camera is the user's viewpoint. As shown in FIG. 2b, the viewpoints A, B, and C of the user can be simulated with the viewpoints A ', B', and C 'of the VR virtual camera, respectively.

In this embodiment, selecting different user viewpoints in a three-dimensional virtual scene can simulate different viewing positions and viewing directions of the user, and then based on the different viewing positions and viewing directions, obtain a three-dimensional scene being displayed by the VR device when a screenshot instruction is received. Optionally, uniformly selecting different user viewpoints is beneficial to quickly calculate the size of the field of view corresponding to each viewpoint.

In an exemplary embodiment, a method for uniformly selecting at least two different user viewpoints in a three-dimensional virtual scene includes:

Acquire the current head position and posture data of the user; wherein, the head position and posture data of the user can be obtained according to the inertial detection unit, multi-axis acceleration sensor, gyroscope and other devices installed on the VR device, and details are not described here. Then, according to the head position pose data, the left and / or right eye viewpoints of the user are respectively determined in the three-dimensional virtual scene, and the base user viewpoint is determined according to the left and / or right eye viewpoints of the user.

Optionally, in this embodiment, the user's left-eye viewpoint can be used as the basic user's viewpoint, the user's right-eye viewpoint can also be used as the basic user's viewpoint, and the middle viewpoint of the user's left-eye viewpoint and right-eye viewpoint can also be selected The viewpoint of the basic user is not limited in this embodiment. Then, in the three-dimensional virtual scene, at least one viewpoint uniformly distributed with the viewpoint of the base user is selected as the auxiliary user viewpoint. Optionally, the number of auxiliary user viewpoints may be two, and the two auxiliary user viewpoints may be respectively distributed on both sides of the base user viewpoint. Furthermore, one basic user viewpoint and two auxiliary user viewpoints can provide the highest image rendering efficiency and image stitching efficiency on the premise of ensuring that the subsequent virtual scene fragments completely cover the three-dimensional virtual scene.

In some embodiments, when the number of auxiliary user viewpoints is two, the field of view corresponding to at least one user viewpoint satisfies FOV> 360 ° / N, and N is the number of user viewpoints; or the field of view corresponding to each user viewpoint satisfies FOV > 360 ° / N, where N is the number of user viewpoints.

In addition, in this embodiment, the selected at least two different user viewpoints are associated with the user's current head and pose data, which is beneficial to determine the virtual scene clips that the user is currently watching in the subsequent process, and obtain Screenshots of VR scenes more in line with real viewing progress.

Based on the at least two user viewpoints selected in the above steps, at least two reference fields of view corresponding thereto may be determined. To ensure that the determined at least two reference fields of view can cover the panorama of the three-dimensional virtual scene, step 202 may be performed. In step 202, when the number of viewpoints included in the at least two user viewpoints is N, the field of view of each user viewpoint is greater than N> 360 ° / N. For example, when N = 3, the field of view of each user's viewpoint can be set to 130 ° or 150 °. Furthermore, the edge of the field of view of two adjacent fields of view appears superimposed, which is beneficial for subsequent image stitching.

In practice, in addition to the implementations described in

steps

201 and 202, when selecting the user's viewpoint, you can also choose arbitrarily without considering the uniformity between viewpoints. For example, it can be randomly selected in a three-dimensional virtual scene that can be displayed by a VR device. Select multiple user viewpoints. In this manner, the size of the field of view corresponding to each user's viewpoint can be calculated according to the positional relationship between the user's viewpoint and other user's viewpoints, and the sum of the angles of view corresponding to multiple user viewpoints is greater than 360 °. For example, among the 3 user viewpoints randomly selected, the position deviation between the user viewpoint A and the user viewpoint B is small, and the user viewpoint A and the user viewpoint B have a large position deviation from the user viewpoint C, then the user may be set The viewpoints A and B have smaller field angles, and the user's viewpoint C has a larger field angle. For example, set the field angle corresponding to point A to 90 ° and set the field angle corresponding to point B to 120 °. The field of view corresponding to C is 160 °. In step 203, after determining the fields of view of at least two different user viewpoints, scene rendering may be performed at each of the user points of view of the at least two different user viewpoints in order to render the at least two reference fields of view. Corresponding virtual scene clip.

In some embodiments, taking Unity 3D as an example, when a VR virtual camera is used to simulate a user's eyes, the user's viewpoint can be used as the viewpoint of the VR virtual camera, and the VR virtual camera can be adjusted to each different user's viewpoint in turn, and A 3D virtual scene rendering program is run at each different user viewpoint to render a virtual scene fragment corresponding to the field of view of the user viewpoint.

In step 204, in some embodiments, after acquiring virtual scene segments corresponding to at least two reference fields of view, edge similarity detection is performed on the virtual scene segments.

Optionally, in this step, a picture correlation coefficient method may be used to find virtual scene fragments with similar edges. For example, the edges of each virtual scene segment can be identified separately, and the edge correlation coefficients of two adjacent virtual scene segments are calculated based on the recognition results, and virtual scene segments with similar edges are determined based on the correlation coefficients. Of course, in this embodiment, other picture edge similarity algorithms can also be used to identify virtual scene fragments with similar edges, such as Euclidean distance method, perceptual hashing method, and sliding window-based template matching method. This embodiment includes but Not only that.

In step 205, in some embodiments, in this step, among the virtual scene fragments corresponding to the at least two reference fields of view, the virtual scene fragments whose edge similarity is greater than a set threshold may be stitched. As shown in FIG. 2c, the virtual scene segments in the reference field of view corresponding to the user viewpoints A, B, and C can be stitched, where the overlapping area with the virtual scene segment is a region with similar edges. It should be understood that the method of horizontal splicing is illustrated in FIG. 2c. In practice, the method of vertical splicing or other degrees of splicing may also be included.

In some embodiments, when image stitching is performed, the virtual scene segment corresponding to the user's viewpoint can be used as a center to make the VR scene screenshot obtained by the stitching center on the virtual scene segment that the user is currently watching, which is more in line with the user's actual viewing effect. . Optionally, in this embodiment, the virtual scene segment corresponding to the basic user viewpoint may be first determined as the splicing center from the virtual scene segments corresponding to the at least two reference fields of view; then, according to the auxiliary user viewpoint and the basic user viewpoint, The positional relationship between them determines the relative position of the virtual scene segment corresponding to the auxiliary user's viewpoint and the stitching center; then, according to the stitching center and the relative position, image stitching is performed on the virtual scene segments corresponding to at least two reference fields of view to generate VR Scene screenshot. For example, when the base user's viewpoint is viewpoint B, and the auxiliary user's viewpoints are viewpoints A and C on the left and right sides of viewpoint B, the virtual scene fragment in the field of view corresponding to viewpoint B can be used as the center of the VR screenshot. On both sides of the virtual scene segment in the field of view corresponding to B, the virtual scene segments in the field of view corresponding to viewpoint A and viewpoint C are spliced, respectively.

In step 206, after the image stitching is completed, the spliced VR scene screenshot may also be stored in a specified path of the VR device, and the user is notified that the screen capture operation has been completed and / or the designation of the VR scene screenshot is displayed. path. The method for notifying the user that the screen capture operation has been completed may be a voice mode or a text mode, which is not limited in this embodiment.

In this embodiment, at least two reference fields of view are uniformly selected in the three-dimensional virtual scene that can be displayed by the VR device, and the edges of the two adjacent reference fields of view are superimposed; then, at least two reference fields of view are obtained to correspond to each other Virtual scene clips, and image stitching of at least two reference scenes corresponding to the virtual scene clips according to the edge similarity of the virtual scene clips, and further, the generated VR scene screenshot matches the 3D virtual scene actually displayed by the VR device, which is effective Captured the real display content of VR devices.

It should be noted that the execution subject of each step of the method provided in the foregoing embodiment may be the same device, or the method may also use different devices as execution subjects. For example, the execution subject of steps 201 to 204 may be device A; for example, the execution subject of

steps

201 and 202 may be device A, and the execution subject of step 203 may be device B; and so on.

In addition, some processes described in the above embodiments and drawings include multiple operations that occur in a specific order, but it should be clearly understood that these operations may be performed out of order or in parallel in this document The sequence numbers of operations such as 201 and 202 are only used to distinguish different operations. The sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be performed sequentially or in parallel.

The above embodiment describes a part of the implementation method of the VR scene screenshot method provided by some embodiments of the present invention. This method may be implemented by the VR device shown in FIG. 3. Optionally, the VR device includes: a memory 301, a processor 302, and an input. Device 303 and output device 304.

The memory 301, the processor 302, the input device 303, and the output device 304 may be connected by using a bus or other methods. In the figure, the bus connection is taken as an example. In other connection modes not shown, the memory 301 may be directly coupled to the processor 302, and the input device 303 and the output device 304 may be directly or indirectly connected to the processor 302 through a data line and a data interface. Of course, the above connection mode is only used for illustrative description, and does not constitute any limitation on the protection scope of the embodiment of the present invention.

The memory 301 is used to store one or more computer instructions, and may be configured to store various other data to support operations on the VR device. Examples of such data include instructions for any application or method for operating on a VR device.

The memory 301 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), Programming read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

In some embodiments, the memory 301 may optionally include a memory remotely set relative to the processor 302, and these remote memories may be connected to the AR display device through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

A processor 302 coupled to the memory 301 and configured to execute the one or more computer instructions for: in response to a screenshot instruction, selecting at least two reference fields of view within a three-dimensional virtual scene that can be displayed by the VR device, and adjacent to each other The edges of the two reference fields of view are superimposed; the virtual scene segments corresponding to the at least two reference fields of view are obtained; image stitching is performed on the virtual scene segments corresponding to the at least two reference fields of view to generate a VR scene screenshot .

Further optionally, when at least two reference fields of view are selected in the three-dimensional virtual scene displayed by the VR device, the processor 302 is specifically configured to: uniformly select at least two different user viewpoints in the three-dimensional virtual scene; The FOVs of two different user viewpoints are set to obtain at least two reference FOVs, where at least one FOV satisfies FOV> 360 ° / N, where N is the user viewpoint contained in the at least two user viewpoints Quantity.

Further optionally, when at least two different user viewpoints are uniformly selected in the three-dimensional virtual scene, the processor 302 is specifically configured to: obtain current head position pose data of the user; Determining the left and / or right eye viewpoints of the user in the three-dimensional virtual scene; determining the base user viewpoints according to the left and / or right eye viewpoints of the user; At least one viewpoint with evenly distributed base user viewpoints is used as the auxiliary user viewpoint.

Further optionally, the number of the auxiliary user viewpoints is two.

Further optionally, when acquiring virtual scene fragments corresponding to the at least two reference fields of view, the processor 302 is specifically configured to perform scene rendering at each user viewpoint in the at least two different user viewpoints in sequence, A virtual scene segment corresponding to the at least two reference fields of view is obtained by rendering.

Further optionally, when image stitching is performed on the virtual scene segments corresponding to the at least two reference fields of view to generate VR scene screenshots, the processor 302 is specifically configured to: The scene segments are detected for edge similarity. According to the detection result of the edge similarity detection, virtual scene segments whose edge similarity is greater than a set threshold among the virtual scene segments corresponding to the at least two reference fields of view are stitched.

Further optionally, when image stitching is performed on the virtual scene segments corresponding to the at least two reference fields of view to generate a VR scene screenshot, the processor 302 is specifically configured to: from the virtual scenes corresponding to the at least two reference fields of view Among the fragments, it is determined that the virtual scene fragment corresponding to the basic user viewpoint is a splicing center; and according to a positional relationship between the auxiliary user viewpoint and the basic user viewpoint, determining the virtual scene fragment corresponding to the auxiliary user viewpoint and the Relative position of the stitching center; according to the stitching center and the relative position, image stitching is performed on the virtual scene segments corresponding to the at least two reference fields of view to generate a screenshot of the VR scene.

Further optionally, the processor 302 is further configured to: store the VR scene screenshot to a specified path of the VR device, and notify the user that the screen capture operation has been completed and / or display the designated path of the VR scene screenshot .

The input device 303 can receive inputted numeric or character information, and generate key signal inputs related to user settings and function control of the VR device. The output device 304 may include a display device such as a display screen.

Further, as shown in FIG. 3, the VR device further includes: a power supply component 305. The power source component 305 provides power to various components of the device where the power source component is located. Power components can include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the equipment in which the power components are located.

The above VR device can execute the VR scene screenshot method provided by the embodiment of the present application, and has corresponding function modules and beneficial effects of executing the method. For technical details that are not described in detail in this embodiment, reference may be made to the method provided in this embodiment of the application, and details are not described herein again.

The present invention also provides a computer-readable storage medium storing a computer program, and the computer program, when executed, can implement the steps in the method that the VR device can execute.

The VR device provided by some embodiments of the present invention may be an external head-mounted display device or an integrated head-mounted display device, wherein the external head-mounted display device may be used in conjunction with an external processing system (such as a computer processing system).

FIG. 4 shows a schematic diagram of the internal configuration of the VR device 400 in some embodiments.

The display unit 401 may include a display panel. The display panel is disposed on a side surface of the VR device 400 that faces the user's face, and may be a whole panel, or a left panel and a right panel corresponding to the left and right eyes of the user, respectively. The display panel may be an electroluminescence (EL) element, a liquid crystal display, or a micro-display with a similar structure, or a laser scanning display with a direct retina display or the like.

The virtual image optical unit 402 displays the image displayed by the display unit 401 to the user in an enlarged manner, and allows the user to observe the displayed image in accordance with the enlarged virtual image. As a display image output to the display unit 401, it may be an image of a virtual scene provided from a content reproduction device (Blu-ray disc or DVD player) or a streaming server, or an image of a real scene captured using an external camera 410. In some embodiments, the virtual image optical unit 402 may include a lens unit, such as a spherical lens, an aspherical lens, a Fresnel lens, and the like.

The input operation unit 403 includes at least one operation component for performing an input operation, such as a key, a button, a switch, or other components having similar functions, receives a user instruction through the operation component, and outputs the instruction to the control unit 407.

The status information acquiring unit 404 is configured to acquire status information of a user wearing the VR device 400. The status information obtaining unit 404 may include various types of sensors for detecting status information by itself, and may obtain status information from external devices (such as a smartphone, a wristwatch, and other multifunctional terminals worn by the user) through the communication unit 405. The state information acquisition unit 404 may acquire position information and / or posture information of a user's head. The status information acquisition unit 404 may include one or more of a gyroscope sensor, an acceleration sensor, a global positioning system (GPS) sensor, a geomagnetic sensor, a Doppler effect sensor, an infrared sensor, and a radio frequency field strength sensor. In addition, the status information acquisition unit 404 acquires status information of a user wearing the VR device 400, such as acquiring, for example, an operation status of the user (whether the user is wearing the VR device 400), an action status of the user (such as stationary, walking, running, and the like) State, hand or fingertip posture, eye open or closed state, line of sight, pupil size), mental state (whether the user is immersed in observing the displayed image and the like), and even a physiological state.

The communication unit 405 performs a communication process with an external device, a modulation and demodulation process, and an encoding and decoding process of a communication signal. In addition, the control unit 407 may transmit transmission data from the communication unit 405 to an external device. The communication method can be wired or wireless, such as mobile high-definition link (MHL) or universal serial bus (USB), high-definition multimedia interface (HDMI), wireless fidelity (Wi-Fi), Bluetooth communication or Bluetooth low energy communication, And the IEEE802.11s standard mesh network. In addition, the communication unit 405 may be a cellular wireless transceiver operating according to Wideband Code Division Multiple Access (W-CDMA), Long Term Evolution (LTE), and similar standards.

In some embodiments, the VR device 400 may further include a storage unit, and the storage unit 406 is a mass storage device configured with a solid state drive (SSD) or the like. In some embodiments, the storage unit 406 may store an application program or various types of data. For example, content viewed by a user using the VR device 400 may be stored in the storage unit 406.

In some embodiments, the VR device 400 may further include a control unit and a storage unit (for example, the ROM 407A and the RAM 407B shown in the figure). The control unit 407 may include a computer processing unit (CPU) or other devices with similar functions. In some embodiments, the control unit 407 may be used to execute an application program stored in the storage unit 406, or the control unit 407 may also be used to execute the methods, functions, and circuits disclosed in some embodiments of the present invention.

The image processing unit 408 is used to perform signal processing, such as image quality correction related to the image signal output from the control unit 407, and to convert its resolution to a resolution according to the screen of the display unit 401. Then, the display driving unit 409 sequentially selects each row of pixels of the display unit 401 and sequentially scans each row of pixels of the display unit 401 one by one, thereby providing a pixel signal based on the image signal subjected to signal processing.

In some embodiments, the VR device 400 may further include an external camera. The external cameras 410 may be disposed on the front surface of the main body of the VR device 400, and the external cameras 410 may be one or more. The external camera 410 can acquire three-dimensional information and can also be used as a distance sensor. In addition, a position sensitive detector (PSD) or other type of distance sensor that detects a reflected signal from an object may be used with the external camera 410. The external camera 410 and the distance sensor may be used to detect a body position, posture, and shape of a user wearing the VR device 400. In addition, under certain conditions, the user can directly view or preview the real scene through the external camera 410.

In some embodiments, the VR device 400 may further include a sound processing unit, and the sound processing unit 411 may perform sound quality correction or sound amplification of a sound signal output from the control unit 407, signal processing of an input sound signal, and the like. Then, the sound input / output unit 412 outputs sound to the outside after sound processing and inputs sound from a microphone.

It should be noted that the structure or component shown by the dashed box in FIG. 4 may be independent of the VR device 400, for example, it may be provided in an external processing system (such as a computer system) for use with the VR device 400; or The resulting structure or component may be disposed inside or on the surface of the VR device 400.

Those of ordinary skill in the art can also understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly explain the hardware and software Interchangeability. In the above description, the composition and steps of each example have been described generally in terms of functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

The steps of the method or algorithm described in connection with the embodiments disclosed herein may be directly implemented by hardware, a software module executed by a processor, or a combination of the two. Software modules can be placed in random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or in technical fields Any other form of storage medium known in the art.

It should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply these entities or operations There is any such actual relationship or order among them. Moreover, the terms "including", "comprising", or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or device that includes a series of elements includes not only those elements but also those that are not explicitly listed Or other elements inherent to such a process, method, article, or device. Without more restrictions, the elements defined by the sentence "including a ..." do not exclude the existence of other identical elements in the process, method, article, or equipment including the elements.

Claims

A VR scene screenshot method includes:

In response to the screenshot instruction, select at least two reference fields of view within a three-dimensional virtual scene that can be displayed by the VR device, and the edges of the two adjacent reference fields of view are superimposed;

Acquiring virtual scene fragments corresponding to the at least two reference fields of view;

Image stitching is performed on the virtual scene segments corresponding to the at least two reference fields of view to generate a VR scene screenshot.
The method according to claim 1, wherein selecting at least two reference fields of view within a three-dimensional virtual scene displayed by a VR device comprises:

Uniformly selecting at least two different user viewpoints in the three-dimensional virtual scene;

Setting the field angle FOV of the at least two different user viewpoints to obtain at least two reference fields of view, where at least one field of view satisfies FOV> 360 ° / N, where N is the at least two user viewpoints The number of user viewpoints included.
The method according to claim 2, wherein uniformly selecting at least two different user viewpoints in the three-dimensional virtual scene comprises:

Get the user's current head position pose data;

Determining the left-eye and / or right-eye viewpoints of the user in the three-dimensional virtual scene respectively according to the head position pose data;

Determining the base user viewpoint according to the left eye and / or right eye viewpoint of the user;

At least one viewpoint that is evenly distributed with the basic user viewpoint is selected as the auxiliary user viewpoint within the three-dimensional virtual scene.
The method according to claim 3, wherein the number of the auxiliary user viewpoints is two.
The method according to any one of claims 2-4, wherein acquiring virtual scene segments corresponding to the at least two reference fields of view comprises:

Scene rendering is performed sequentially at each of the user viewpoints of the at least two different user viewpoints to render a virtual scene segment corresponding to the at least two reference viewpoints.
The method according to any one of claims 1-4, wherein performing image stitching on the virtual scene segments corresponding to the at least two reference fields of view to generate a VR scene screenshot includes:

Performing edge similarity detection on the virtual scene segments corresponding to the at least two reference fields of view;

According to the detection result of the edge similarity detection, among the virtual scene fragments corresponding to the at least two reference fields of view, the virtual scene fragments whose edge similarity is greater than a set threshold are stitched.
The method according to claim 3 or 4, wherein performing image stitching on the virtual scene segments corresponding to the at least two reference fields of view to generate a VR scene screenshot includes:

Determining, from the virtual scene fragments corresponding to the at least two reference views, the virtual scene fragments corresponding to the basic user viewpoint as a splicing center;

Determining a relative position of the virtual scene segment corresponding to the auxiliary user viewpoint and the splicing center according to a positional relationship between the auxiliary user viewpoint and the basic user viewpoint;

According to the stitching center and the relative position, image stitching is performed on the virtual scene segments corresponding to the at least two reference fields of view to generate a VR scene screenshot.
The method according to any one of claims 1-4, further comprising:

The screenshot of the VR scene is stored in a designated path of the VR device, and the user is notified that the screen capture operation has been completed and / or the designated path of the VR scene screenshot is displayed.
A VR device, comprising: a memory and a processor;

The memory is used to store one or more computer instructions;

The processor is coupled to the memory, and is configured to execute the one or more computer instructions for executing the VR scene screenshot method according to any one of claims 1-8.
A computer-readable storage medium storing a computer program, wherein when the computer program is executed, the steps in the method according to any one of claims 1-8 can be realized.