WO2018214431A1

WO2018214431A1 - Method and apparatus for presenting scene using virtual reality device, and virtual reality apparatus

Info

Publication number: WO2018214431A1
Application number: PCT/CN2017/112383
Authority: WO
Inventors: 王鹏
Original assignee: 歌尔科技有限公司
Priority date: 2017-05-22
Filing date: 2017-11-22
Publication date: 2018-11-29
Also published as: CN107302694B; CN107302694A

Abstract

The present invention discloses a method and an apparatus for presenting a scene using virtual reality device, and a virtual reality apparatus. The method comprises: obtaining imaging parameters from a left-eye imaging device and a right-eye imaging device, respectively; obtaining a first displacement distance of the left-eye imaging device and a second displacement distance of the right-eye imaging device on the basis of the imaging parameters from the left-eye imaging device and the right-eye imaging device, respectively; adjusting a left-eye lens, a left-eye camera, a right-eye lens and a right-eye camera on the basis of the first displacement distance, the second displacement distance and a predetermined human pupil distance; and rendering a first image previewed at the left-eye camera and a second image simultaneously previewed at the right-eye camera to a left-eye screen and a right-eye screen, respectively.

Description

Method, device and virtual reality device for presenting scene through virtual reality device

Technical field

The present invention relates to the field of virtual reality technologies, and in particular, to a method, a device, and a virtual reality device for presenting a scene through a virtual reality device.

Background technique

In recent years, virtual reality technology has developed rapidly, not only for presenting virtual scenes to provide users with near-real immersion, but also for presenting real-life scenes to provide users with a sense of realism in the real world. Therefore, virtual reality devices such as virtual reality helmets, virtual reality glasses, and the like that present scenes based on virtual reality technology are also attracting more and more users' attention.

Current virtual reality devices are typically used to present scenes by setting a dual camera mode for the left eye camera and the right eye camera. The dual camera of the virtual reality device is different from the general single camera, and the depth of field of the corresponding scene image can be increased when the scene is presented, so that the presented scene image has a stereoscopic effect and enhances the realism when the user uses to view the real world.

When a virtual reality device uses a dual camera to present a scene, it is necessary to introduce a third-party open source library, such as the FFmpeg library, to perform a merge algorithm on each frame image previewed by the left-eye camera and the right-eye camera to achieve a real-world simulation effect. Presented to the user through a virtual reality device.

Summary of the invention

According to a first aspect of the present invention, there is provided a method of presenting a scene by a virtual reality device, the virtual reality device comprising a left eye imaging device and a right eye imaging device, the left eye imaging device a left eye lens, a left eye camera, a left eye screen, and a left lens head are sequentially disposed, and the right eye imaging device is sequentially provided with a right eye lens, a right eye camera, a right eye screen, and a right lens head;

The method includes:

Obtaining an imaging parameter of the left-eye imaging device and an imaging parameter of the right-eye imaging device, where the imaging parameter includes at least a central vertical distance of the lens to the camera in the corresponding imaging device and a refractive angle of the lens;

Obtaining a first offset distance of the left eye imaging device according to an imaging parameter of the left eye imaging device, and a second offset distance of the right eye imaging device according to an imaging parameter of the right eye imaging device;

Adjusting the left eye lens, the left eye camera, the right eye lens, and the right eye camera according to the first offset distance, the second offset distance, and a preset human eye pupil distance, such that the left eye lens and the right eye The center horizontal distance of the eye lens is not greater than the human eyelid distance, the left eye lens is horizontally offset from the left eye camera center by a first offset distance, and the right eye lens is horizontally offset from the right eye camera center by a second offset Moving distance

The first image previewed by the left eye camera is rendered on the left eye screen, and the second image corresponding to the simultaneous preview of the right eye camera is rendered on the right eye screen to present the corresponding scene to the user through the corresponding lens.

According to a second aspect of the present invention, a scene presenting apparatus is provided, which is disposed on a virtual reality device side.

The virtual reality device includes a left eye imaging device and a right eye imaging device, and the left eye imaging device is sequentially provided with a left eye lens, a left eye camera, a left eye screen, and a left lens, and the right eye imaging device is sequentially provided with Right eye lens, right eye camera, right eye screen, and right lens head;

The scene rendering device includes:

a parameter obtaining unit, configured to respectively acquire an imaging parameter of the left-eye imaging device and an imaging parameter of the right-eye imaging device, where the imaging parameter includes at least a central vertical distance of the lens to the camera in the corresponding imaging device and a refractive angle of the lens;

An offset distance acquisition unit, configured to acquire a first offset distance of the left-eye imaging device according to an imaging parameter of the left-eye imaging device, and a second offset distance of the right-eye imaging device according to an imaging parameter of the right-eye imaging device;

a component adjustment unit, configured to adjust the left eye lens, the left eye camera, the right eye lens, and the right eye camera according to the first offset distance, the second offset distance, and a preset human eye distance The center horizontal distance between the left eye lens and the right eye lens is not greater than the human eyelid distance, The left eye lens is horizontally offset from the left eye camera center by a first offset distance and the right eye lens is horizontally offset from the right eye camera center by a second offset distance;

An image rendering unit, configured to render a first image previewed by the left eye camera on the left eye screen, and a second image corresponding to simultaneously previewing the right eye camera on the right eye screen to present a corresponding to the user through the corresponding lens Scene.

According to a third aspect of the present invention, a virtual reality device is provided, comprising:

a left eye imaging device, which is provided with a left eye lens, a left eye camera, a left eye screen, and a left lens head in sequence;

a right-eye imaging device, which is sequentially provided with a right-eye lens, a right-eye camera, a right-eye screen, and a right-eye lens;

Any of the scene presentation devices provided by the second aspect of the present invention.

In an embodiment of the present invention, a method, a device, and a virtual reality device for presenting a scenario by using a virtual reality device may perform a merge algorithm processing on each frame image previewed by the left and right eye cameras without introducing a third-party library, and reduce the virtual reality device. The difficulty of implementation, reduce the size of the program, to avoid the problem of low operating efficiency and high power consumption. Moreover, the image refresh speed is improved when the scene is presented, and the realism of the scene presentation is enhanced. At the same time, it is possible to avoid duplication or overlapping of images when the scene is presented by the virtual reality device, enhance the depth of field effect of the corresponding image, and enhance the realism of the scene presentation. In addition, it can enhance the comfort of users using virtual reality devices. Especially suitable for presenting real scenes in the real world. Other features and advantages of the present invention will become apparent from the Detailed Description of the <RTIgt;

DRAWINGS

The accompanying drawings, which are incorporated in FIG

1 is a block diagram showing an example of a hardware configuration of an electronic device that can be used to implement an embodiment of the present invention.

2 shows a flow diagram of a method of presenting a scene by a virtual reality device in accordance with an embodiment of the present invention.

FIG. 3 illustrates an example of a method of presenting a scene by a virtual reality device according to an embodiment of the present invention. Sub-schematic.

FIG. 4 shows a schematic block diagram of a scene presenting apparatus of an embodiment of the present invention.

FIG. 5 shows a schematic block diagram of a virtual reality scene device of an embodiment of the present invention.

detailed description

Various exemplary embodiments of the present invention will now be described in detail with reference to the drawings. It should be noted that the relative arrangement of the components and steps, numerical expressions and numerical values set forth in the embodiments are not intended to limit the scope of the invention unless otherwise specified.

The following description of the at least one exemplary embodiment is merely illustrative and is in no way

Techniques, methods and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail, but the techniques, methods and apparatus should be considered as part of the specification, where appropriate.

In all of the examples shown and discussed herein, any specific values are to be construed as illustrative only and not as a limitation. Thus, other examples of the exemplary embodiments may have different values.

It should be noted that similar reference numerals and letters indicate similar items in the following figures, and therefore, once an item is defined in one figure, it is not required to be further discussed in the subsequent figures.

FIG. 1 is a block diagram showing a hardware configuration of an electronic device 1000 in which an embodiment of the present invention can be implemented.

In one example, the electronic device 1000 can be a virtual reality helmet or virtual reality glasses or the like. As shown in FIG. 1, the electronic device 1000 may include a processor 1100, a memory 1200, an interface device 1300, a communication device 1400, a display device 1500, an input device 1600, a speaker 1700, a microphone 1800, and the like. The processor 1100 may be a central processing unit CPU, a microprocessor MCU, or the like. The memory 1200 includes, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), a nonvolatile memory such as a hard disk, and the like. The interface device 1300 includes, for example, a USB interface, a headphone jack, and the like. The communication device 1400 can, for example, perform wired or wireless communication, and specifically can include Wifi communication, Bluetooth communication, 2G/3G/4G/5G communication, and the like. Display device 1500 cases Such as LCD screen, touch screen, etc. Input device 1600 can include, for example, a touch screen, a keyboard, a somatosensory input, and the like. The user can input/output voice information through the speaker 1700 and the microphone 1800.

The electronic device shown in Figure 1 is merely illustrative and is in no way meant to limit the invention, its application or use. In an embodiment of the present invention, the memory 1200 of the electronic device 1000 is configured to store instructions for controlling the processor 1100 to operate to perform any of the embodiments provided by the embodiments of the present invention through a virtual reality device. The method of rendering the scene. It will be understood by those skilled in the art that although a plurality of devices are illustrated for electronic device 1000 in FIG. 1, the present invention may relate only to some of the devices therein, for example, electronic device 1000 only relates to processor 1100 and memory 1200. A technician can design instructions in accordance with the disclosed aspects of the present invention. How the instructions control the processor for operation is well known in the art and will not be described in detail herein.

In this embodiment, a method for presenting a scene by using a virtual reality device is implemented on a virtual reality device, where the virtual reality device includes a left eye imaging device and a right eye imaging device, and the left eye imaging device is sequentially disposed with a left An eye lens, a left eye camera, a left eye screen, and a left eye lens, and the right eye imaging device is sequentially provided with a right eye lens, a right eye camera, a right eye screen, and a right eye lens.

Specifically, the left-eye imaging device and the right-eye imaging device in the virtual reality device may be physically separated physical devices, or may be physically or partially integrated and logically separated.

In one example, the virtual reality device can be a virtual reality glasses or a virtual reality helmet.

The method for presenting a scene by using a virtual reality device, as shown in FIG. 2, includes steps S2100 to S2400.

In step S2100, an imaging parameter of the left-eye imaging device and an imaging parameter of the right-eye imaging device are respectively acquired, the imaging parameter including at least a central vertical distance of the lens to the camera in the corresponding imaging device and a refractive angle of the lens;

The imaging parameters may be provided by a manufacturer or manufacturer of the corresponding virtual reality device. And pre-stored in the storage area of the virtual reality device and provides an interface for acquisition, or may be obtained as a product parameter of the virtual reality device by means of a manual or a product official website description, and provided for downloading, etc., which are not enumerated here.

In step S2200, a first offset distance of the left-eye imaging device is acquired according to an imaging parameter of the left-eye imaging device, and a second offset distance of the right-eye imaging device is acquired according to an imaging parameter of the right-eye imaging device.

For example, the imaging parameters of the left-eye rendering device acquired in step S2100 include the lens-to-camera vertical vertical distance h _{1 of the} left-eye imaging device and the refractive angle β _{1 of the} lens, and the imaging parameters of the right-eye rendering device include right-eye imaging. The center vertical distance h ₂ of the lens of the device to the camera and the refractive angle β _{2 of the} lens. Specifically, the steps of obtaining the first offset distance and the second offset distance include:

Calculating a first offset distance w ₁ =h ₁ ×tgβ ₁ according to a center vertical distance h ₁ of the lens of the left eye imaging device to the camera and a refractive angle β _{1 of the} lens;

The second offset distance w ₂ =h ₂ ×tgβ _{2 is} calculated according to the center vertical distance h ₂ of the lens of the right eye imaging device to the camera and the refractive angle β _{2 of the} lens.

In step S2300, the left eye lens, the left eye camera, the right eye lens, and the right eye camera are adjusted according to the first offset distance, the second offset distance, and a preset human eye pupil distance, so that the The center horizontal distance of the left eye lens and the right eye lens is not greater than the human eyelid distance, the left eye lens is horizontally offset from the left eye camera center by a first offset distance, and the right eye lens and the right eye camera center level are Offset the second offset distance.

Through the above adjustment, the left eye lens is horizontally offset from the left eye camera center by a first offset distance and the right eye lens is horizontally offset from the right eye camera center by a second offset distance, which can be prevented from being presented by the virtual reality device. When the scene is repeated or overlapping, the depth of field effect of the image is previewed by the left and right cameras, and the user's comfort is reduced or even fatigued. Therefore, the center horizontal distance of the left-eye lens and the right-eye lens is not greater than the human eyelid distance, and the visual fatigue caused by the user after use may be avoided because the distance between the left and right eye lenses is greater than the lay length, thereby further improving the use comfort.

For example, the center vertical distance of the right lens to right eye camera is h ₁ and the angle of refraction of the right lens is β ₁ , the vertical distance from the center of the left lens to the camera is h ₂ and the angle of refraction of the right lens is β. ₂ , the first offset distance obtained by the step S2200 is w ₁ , and the second offset distance is w ₂ . After the step S2300 is performed, the corresponding components are set as shown in FIG. 3 , wherein the left side shown in FIG. 3 The central horizontal distance of the eye lens and the right eye lens is the human eyelid distance and is merely illustrative. In other examples, it is also possible that the center horizontal distance of the left-eye lens and the right-eye lens is a value close to the human eyelid distance.

The human eyelid distance may be an average distance of a general user selected according to engineering experience or experimental simulation, or may be obtained by an interface or interface provided by the virtual reality device involved in the embodiment for user operation or input. The value is used by the user who actually uses the virtual reality device to set the eyelid distance according to his own needs or application scenarios, so as to realize the personalized distance setting to further improve the comfort of the user using the virtual reality device.

In step S2400, the first image previewed by the left eye camera is rendered on the left eye screen, and the second image corresponding to the simultaneous preview of the right eye camera is rendered on the right eye screen to present the corresponding image to the user through the corresponding lens. Scenes.

Specifically, the scene may be a virtual scene to provide a near-real immersion of the user, or a real scene in the real world, to provide the user with a sense of realism in the real world.

After the left and right eye cameras are set in step S2300, the images obtained by simultaneously previewing the left and right eye cameras respectively in step S2400 are respectively corresponding to the respective left and right eye screens, and the corresponding scenes are directly presented to the user, so that the first scene does not need to be introduced. The three-party library performs a merging algorithm on each frame of the preview of the left and right eye cameras, thereby reducing the difficulty of implementing the virtual reality device, reducing the size of the program, and avoiding the problems of low operating efficiency and high power consumption. Moreover, the image refresh speed is improved when the scene is presented, and the realism of the scene presentation is enhanced. In particular, it is particularly suitable for presenting real world real scenes, providing a sense of solid reality.

Specifically, the rendering operation can be implemented by the rendering function provided by Unity3D. Unity3D is a fully integrated professional game engine developed by Unity Technologies, which can be generally applied to the development of virtual reality device rendering scenarios, and will not be described here.

In this embodiment, each frame image obtained by simultaneously previewing the left and right eye cameras respectively can be respectively rendered on the corresponding left and right eye screens by the rendering function provided by Unity3D to present corresponding scenes.

In an example, the image center of each frame of the image previewed by the left and right eye cameras may be offset before the rendering operation is performed to enhance the depth of field effect of the corresponding image. It makes the corresponding corresponding scene (especially the real scene of the real world) more realistic. Correspondingly, the method for presenting a scenario by using a virtual reality device provided in this example further includes:

Obtaining a first horizontal offset distance based on a center-to-center distance of the lens-to-camera of the left-eye imaging device, a first offset distance, and an image height of the first image, the first being based on the first horizontal offset distance The center of the image is horizontally offset to obtain a new image center, and then the rendering operation is performed;

as well as

Obtaining a second horizontal offset distance according to a center vertical distance of the lens of the right eye imaging device to the camera, a second offset distance, and an image height of the second image, and the second image is based on the second horizontal offset distance The center performs a horizontal offset to obtain a new image center, and then performs the rendering operation.

More specifically, h ₁ is the center vertical distance from the lens of the left eye imaging device to the camera, w ₁ is the first offset distance, and H ₁ is the image height of the first image, and h ₂ is the lens to camera of the right eye imaging device. The center vertical distance, w ₂ is the first offset distance, and H ₂ is the image height of the second image, correspondingly, the first horizontal offset distance is d ₁ =(w ₁ /h ₁ )×(H ₁ /2 The second horizontal offset distance is d ₂ = (w ₂ /h ₂ ) × (H ₂ /2).

In another example, the object other than the outline of the image center object of each frame image previewed by the left and right eye cameras may be blurred or blurred, and then the rendering operation is performed to enhance the correspondence. The depth of field effect of the image makes the corresponding scene (especially the real scene of the real world) more realistic. Correspondingly, the method for presenting a scenario by using a virtual reality device provided in this example further includes:

After determining a center object of the first image based on an image center point of the first image, performing contour detection on the center object to correspond to a center object contour, and the first outside the center object contour After the object included in the image is blurred or blurred, the rendering operation is performed;

After determining a center object of the second image based on an image center point of the second image, performing contour detection on the center object to correspond to a center object contour, and the second outside the center object contour After the object included in the image is blurred or blurred, the rendering operation is performed.

The contour detection can be implemented by the function findContours() provided in OpenCV, which is an open source cross-platform computer vision library, which will not be described here.

In addition, it should be understood that, in a specific application, the image center of each frame image previewed by the left and right eye cameras may be offset, and the object other than the contour of the image center object may be blurred or blurred. The rendering operation is performed to better enhance the depth of field effect of the corresponding image, so that the corresponding corresponding scene is more realistic.

In this embodiment, a scene presentation device 3000 is further provided, as shown in FIG. 3, disposed on the virtual reality device 4000 side, where the scene presentation device 3000 includes a parameter acquisition unit 3100, an offset distance acquisition unit 3200, and component adjustment. The unit 3300 and the image rendering unit 3400, optionally, further include an image center offset unit 3500 and a contour processing unit 3600 for implementing the method for presenting a scene by using a virtual reality device provided in this embodiment, and details are not described herein again.

Specifically, the virtual reality device 4000 includes a left eye imaging device 4100 and a right eye imaging device 4200, which are sequentially provided with a left eye lens 4101, a left eye camera 4102, a left eye screen 4103, and a left lens. 4104. The right-eye imaging device 4200 is sequentially provided with a right-eye lens 4201, a right-eye camera 4202, a right-eye screen 4203, and a right-eye lens 4204.

The scene rendering device 3000 includes:

The parameter obtaining unit 3100 is configured to respectively acquire imaging parameters of the left-eye imaging device and imaging parameters of the right-eye imaging device, where the imaging parameters include at least a central vertical distance of the lens from the corresponding imaging device to the camera and a refractive angle of the lens;

The offset distance acquiring unit 3200 is configured to acquire a first offset distance of the left eye imaging device according to the imaging parameter of the left eye imaging device, and acquire a second offset distance of the right eye imaging device according to the imaging parameter of the right eye imaging device;

The component adjusting unit 3300 is configured to adjust the left eye lens, the left eye camera, the right eye lens, and the right eye camera according to the first offset distance, the second offset distance, and a preset human eye pupil distance, so that The center horizontal distance of the left eye lens and the right eye lens is not greater than the human eyelid distance, the left eye lens is horizontally offset from the left eye camera center by a first offset distance, and the right eye lens and the right eye camera are The center horizontal offset is a second offset distance;

The image rendering unit 3400 is configured to render the first image previewed by the left eye camera on the left eye screen, and the second image corresponding to the right eye camera simultaneously previewed on the right eye screen to present to the user through the corresponding lens Corresponding scene.

Optionally, the offset distance obtaining unit 3200 includes:

Means for calculating a first offset distance w ₁ =h ₁ ×tgβ ₁ according to a center-to-center distance h ₁ of the lens of the left-eye imaging device to the camera and a refractive angle β _{1 of the} lens;

It means _{_₂} = h ₂ × _{tgβ 2} h ₂ for an angle β, and the refractive lens _2, calculating a second offset distance w eye lens imaging apparatus according to the vertical distance from the center of the camera.

Optionally, the scene rendering device 3000 further includes an image center offset unit 3400, configured to:

as well as,

Optionally, the scene rendering device 3000 further includes a contour processing unit 3500, configured to:

After determining a center object of the first image based on an image center point of the first image, performing contour detection on the center object to correspond to a center object contour, and the first outside the center object contour After the object included in the image is blurred or blurred, the rendering operation is performed; and,

It should be understood that the scene presentation device 3000 and the virtual reality device 4000 shown in FIG. 3 The connection relationship is merely illustrative and not a specific limitation. The scene presentation device 3000 may be disposed in or integrated with the virtual reality device 4000, and may be connected to the virtual reality device 4000 through a wireless or wired connection form, and may be executed in a virtual or wired connection manner. This is not listed one by one.

In addition, the left-eye imaging device 4100 and the right-eye imaging device 4200 included in the virtual reality device 4000 illustrated in FIG. 3 are merely illustrative, and do not necessarily mean that the left-eye imaging device 4100 and the right-eye imaging device 4200 are physically separated. The two physical devices, the left-eye imaging device 4100 and the right-eye imaging device 4200, may be logically separated, for example, in a specific implementation, a left-eye lens 4101, a left-eye camera 4102, corresponding to the left-eye imaging device 4100. The left-eye screen 4103, the left-eye lens 4104, and the right-eye lens 4201, the right-eye camera 4202, the right-eye screen 4203, and the right-eye lens 4204 corresponding to the right-eye rendering device 4200 may be integrated or built in the same physical device. The elements are not physically separated to set the corresponding left-eye imaging device 4100 and right-eye imaging device 4200.

In a specific example, the hardware configuration of the scene rendering device 3000 may be the electronic device 1000 as shown in FIG. 1.

Those skilled in the art will also appreciate that the scene rendering device 3000 can be implemented in a variety of ways. For example, the scene rendering device 3000 can be implemented by an instruction configuration processor. For example, the instructions can be stored in the ROM, and when the device is booted, the instructions are read from the ROM into the programmable device to implement the scene rendering device 3000. For example, scene rendering device 3000 can be cured into a dedicated device (eg, an ASIC). The scene rendering device 3000 can be divided into mutually independent units, or they can be implemented together. The scene rendering device 3000 may be implemented by one of the various implementations described above, or may be implemented by a combination of two or more of the various implementations described above.

In this embodiment, a virtual reality device 5000 is further provided, as shown in FIG. 5, including:

a left eye imaging device 5100, the left eye imaging device 5100 is sequentially provided with a left eye lens 5101, a left eye camera 5102, a left eye screen 5103, and a left lens 5104;

a right-eye imaging device 5200, which is sequentially provided with a right-eye lens 5201, a right-eye camera 5202, a right-eye screen 5203, and a right-eye lens 5204;

The virtual display scene presentation device 3000 provided in this embodiment.

Specifically, the virtual reality device 5000 may be a virtual reality helmet or virtual reality glasses or the like. In a specific example, the virtual reality device 5000 can be the electronic device 1000 as shown in FIG.

In addition, the left-eye imaging device 5100 and the right-eye imaging device 5200 included in the virtual reality device 5000 illustrated in FIG. 5 are merely illustrative, and do not necessarily mean that the left-eye imaging device 5100 and the right-eye imaging device 5200 are physically separated. The two-dimensional physical device, the left-eye imaging device 5100 and the right-eye imaging device 5200, may be logically separated. For example, in a specific implementation, the virtual reality device 5000 may include a left-eye lens 5101 and a left eye that are sequentially disposed. The camera 5102, the left-eye screen 5103, the left-eye lens 5104, and the right-eye lens 5201, the right-eye camera 5202, the right-eye screen 5203, and the right-eye lens 5204 of the corresponding sequential device are not physically separated to set the corresponding left eye. The imaging device 5100 and the right eye imaging device 5200.

The embodiment has been described above with reference to the accompanying drawings. In this embodiment, a method, a device, and a virtual reality device for presenting a scene through a virtual reality device are provided, and after performing an adjustment operation on the left and right eye cameras in the virtual reality device, respectively The images previewed by the left and right eye cameras are respectively correspondingly rendered on the corresponding left and right lens heads, so that the third party library is not required to introduce a merge algorithm for each frame of the left and right eye camera previews, thereby reducing the virtual reality. The difficulty of implementing the device reduces the size of the program and avoids the problems of low operating efficiency and high power consumption. Moreover, the image refresh speed is improved when the scene is presented, and the realism of the scene presentation is enhanced. At the same time, it is possible to avoid duplication or overlapping of images when the scene is presented by the virtual reality device, enhance the depth of field effect of the corresponding image, and enhance the realism of the scene presentation. In addition, it can enhance the comfort of users using virtual reality devices. Especially suitable for presenting real scenes in the real world.

The invention can be a system, method and/or computer program product. The computer program product can comprise a computer readable storage medium having computer readable program instructions embodied thereon for causing a processor to implement various aspects of the present invention.

The computer readable storage medium can be a tangible device that can hold and store the instructions used by the instruction execution device. The computer readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer readable storage media (non-exhaustive List) includes: portable computer disk, hard disk, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash), static random access memory (SRAM), portable compression Disk read only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanical encoding device, for example, a punch card or a recessed structure in which a command is stored thereon, and any suitable The combination. A computer readable storage medium as used herein is not to be interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (eg, a light pulse through a fiber optic cable), or through a wire The electrical signal transmitted.

The computer readable program instructions described herein can be downloaded from a computer readable storage medium to various computing/processing devices or downloaded to an external computer or external storage device over a network, such as the Internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium in each computing/processing device .

Computer program instructions for performing the operations of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine related instructions, microcode, firmware instructions, state setting data, or in one or more programming languages. Source code or object code written in any combination, including object oriented programming languages such as Smalltalk, C++, etc., as well as conventional procedural programming languages such as the "C" language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer, partly on the remote computer, or entirely on the remote computer or server. carried out. In the case of a remote computer, the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or wide area network (WAN), or can be connected to an external computer (eg, using an Internet service provider to access the Internet) connection). In some embodiments, the customized electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), can be customized by utilizing state information of computer readable program instructions. Computer readable program instructions are executed to implement various aspects of the present invention.

Aspects of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus, and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowcharts and/or block diagrams can be implemented by computer readable program instructions.

The computer readable program instructions can be provided to a general purpose computer, a special purpose computer, or a processor of other programmable data processing apparatus to produce a machine such that when executed by a processor of a computer or other programmable data processing apparatus Means for implementing the functions/acts specified in one or more of the blocks of the flowcharts and/or block diagrams. The computer readable program instructions can also be stored in a computer readable storage medium that causes the computer, programmable data processing device, and/or other device to operate in a particular manner, such that the computer readable medium storing the instructions includes An article of manufacture that includes instructions for implementing various aspects of the functions/acts recited in one or more of the flowcharts.

The computer readable program instructions can also be loaded onto a computer, other programmable data processing device, or other device to perform a series of operational steps on a computer, other programmable data processing device or other device to produce a computer-implemented process. Thus, instructions executed on a computer, other programmable data processing apparatus, or other device implement the functions/acts recited in one or more of the flowcharts and/or block diagrams.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the invention. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a portion of an instruction that includes one or more components for implementing the specified logical functions. Executable instructions. In some alternative implementations, the functions noted in the blocks may also occur in a different order than those illustrated in the drawings. For example, two consecutive blocks may be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowcharts, and combinations of blocks in the block diagrams and/or flowcharts, can be implemented in a dedicated hardware-based system that performs the specified function or function. Or it can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that implementation by hardware, implementation by software, and implementation by a combination of software and hardware are equivalent.

The embodiments of the present invention have been described above, and the foregoing description is illustrative, not limiting, and not limited to the disclosed embodiments. Numerous modifications and changes will be apparent to those skilled in the art without departing from the scope of the invention. The choice of terms used herein is intended to best explain the principles, practical applications, or technical improvements in the various embodiments of the embodiments, or to enable those of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the invention is defined by the appended claims.

Claims

A method for presenting a scene by using a virtual reality device, implemented on a virtual reality device, wherein

The virtual reality device includes a left eye imaging device and a right eye imaging device, and the left eye imaging device is sequentially provided with a left eye lens, a left eye camera, a left eye screen, and a left lens, and the right eye imaging device is sequentially provided with Right eye lens, right eye camera, right eye screen, and right lens head;

The method includes:

Obtaining an imaging parameter of the left-eye imaging device and an imaging parameter of the right-eye imaging device, where the imaging parameter includes at least a central vertical distance of the lens to the camera in the corresponding imaging device and a refractive angle of the lens;

Obtaining a first offset distance of the left-eye imaging device according to imaging parameters of the left-eye imaging device, and acquiring a second offset distance of the right-eye imaging device according to imaging parameters of the right-eye imaging device;

Adjusting the left eye lens, the left eye camera, the right eye lens, and the right eye camera according to the first offset distance, the second offset distance, and a preset human eye pupil distance, such that the left eye lens and the right eye The center horizontal distance of the eye lens is not greater than the human eyelid distance, the left eye lens is horizontally offset from the left eye camera center by a first offset distance, and the right eye lens is horizontally offset from the right eye camera center by a second offset Moving distance

The first image previewed by the left eye camera is rendered on the left eye screen, and the second image corresponding to the simultaneous preview of the right eye camera is rendered on the right eye screen to present the corresponding scene to the user through the corresponding lens.
The method according to claim 1, wherein the step of acquiring the first offset distance and the second offset distance comprises:

Calculating a first offset distance w 1 =h 1 ×tgβ 1 according to a center vertical distance h 1 of the lens of the left eye imaging device to the camera and a refractive angle β 1 of the lens;

The second offset distance w 2 =h 2 ×tgβ 2 is calculated according to the center vertical distance h 2 of the lens of the right eye imaging device to the camera and the refractive angle β 2 of the lens.
The method according to claim 1 or 2, further comprising:

Obtaining a first horizontal offset distance based on a center-to-center distance of the lens-to-camera of the left-eye imaging device, a first offset distance, and an image height of the first image, the first being based on the first horizontal offset distance The center of the image is horizontally offset to obtain a new image center, and then the rendering operation is performed;

as well as,

Obtaining a second horizontal offset distance according to a center vertical distance of the lens of the right eye imaging device to the camera, a second offset distance, and an image height of the second image, and the second image is based on the second horizontal offset distance The center performs a horizontal offset to obtain a new image center, and then performs the rendering operation.
A method according to any one of claims 1 to 3, characterized in that

The first horizontal offset distance is d 1 = (w 1 /h 1 )×(H 1 /2),

Wherein h 1 is the center vertical distance from the lens of the left eye imaging device to the camera, w 1 is the first offset distance, and H 1 is the image height of the first image;

The second horizontal offset distance is d 2 = (w 2 /h 2 ) × (H 2 /2),

Where h 2 is the center vertical distance from the lens of the right eye imaging device to the camera, w 2 is the first offset distance, and H 2 is the image height of the second image.
The method according to any one of claims 1 to 4, further comprising:

After determining a center object of the first image based on an image center point of the first image, performing contour detection on the center object to correspond to a center object contour, and the first outside the center object contour After the object included in the image is blurred or blurred, the rendering operation is performed; and,

After determining a center object of the second image based on an image center point of the second image, performing contour detection on the center object to correspond to a center object contour, and the second outside the center object contour After the object included in the image is blurred or blurred, the rendering operation is performed.
A scene presenting device, which is disposed on a virtual reality device side,

The virtual reality device includes a left eye imaging device and a right eye imaging device, and the left eye imaging device is sequentially provided with a left eye lens, a left eye camera, a left eye screen, and a left lens, and the right eye imaging device is sequentially provided with Right eye lens, right eye camera, right eye screen, and right lens head;

The scene rendering device includes:

a parameter obtaining unit, configured to respectively acquire an imaging parameter of the left-eye imaging device and an imaging parameter of the right-eye imaging device, where the imaging parameter includes at least a central vertical distance of the lens to the camera in the corresponding imaging device and a refractive angle of the lens;

An offset distance acquisition unit configured to acquire a first offset distance of the left-eye imaging device according to an imaging parameter of the left-eye imaging device, and acquire a second offset distance of the right-eye imaging device according to an imaging parameter of the right-eye imaging device;

a component adjustment unit, configured to adjust the left eye lens, the left eye camera, the right eye lens, and the right eye camera according to the first offset distance, the second offset distance, and a preset human eye distance The center horizontal distance of the left eye lens and the right eye lens is not greater than the human eyelid distance, the left eye lens is horizontally offset from the left eye camera center by a first offset distance, and the right eye lens and the right eye camera center are Horizontally offset by a second offset distance;

An image rendering unit, configured to render a first image previewed by the left eye camera on the left eye screen, and a second image corresponding to simultaneously previewing the right eye camera on the right eye screen to present a corresponding to the user through the corresponding lens Scene.
The device according to claim 6, wherein the offset distance obtaining unit comprises:

Means for calculating a first offset distance w 1 =h 1 ×tgβ 1 according to a center-to-center distance h 1 of the lens of the left-eye imaging device to the camera and a refractive angle β 1 of the lens;

It means 2 = h 2 × tgβ 2 h 2 for an angle β, and the refractive lens 2, calculating a second offset distance w eye lens imaging apparatus according to the vertical distance from the center of the camera.
The device according to claim 6 or 7, further comprising an image center offset unit, configured to:

Obtaining a first horizontal offset distance based on a center-to-center distance of the lens-to-camera of the left-eye imaging device, a first offset distance, and an image height of the first image, the first being based on the first horizontal offset distance The center of the image is horizontally offset to obtain a new image center, and then the rendering operation is performed;

as well as,

Obtaining a second horizontal offset distance according to a center vertical distance of the lens of the right eye imaging device to the camera, a second offset distance, and an image height of the second image, and the second image is based on the second horizontal offset distance The center performs a horizontal offset to obtain a new image center, and then performs the rendering operation.
The device according to any one of claims 6-8, further comprising a contour processing unit for:

After determining a center object of the first image based on an image center point of the first image, performing contour detection on the center object to correspond to a center object contour, and the first outside the center object contour After the object included in the image is blurred or blurred, the rendering operation is performed; and,

After determining a center object of the second image based on an image center point of the second image, performing contour detection on the center object to correspond to a center object contour, and the second outside the center object contour After the object included in the image is blurred or blurred, the rendering operation is performed.
A virtual reality device, comprising:

a left eye imaging device, which is provided with a left eye lens, a left eye camera, a left eye screen, and a left lens head in sequence;

a right eye imaging device, wherein the right eye imaging device is sequentially provided with a right eye lens, a right eye camera, a right eye screen, and a right eye lens;

And any scene rendering device according to any of claims 6-9.