WO2017113729A1 - 360度图像加载方法、加载模块及移动终端 - Google Patents

360度图像加载方法、加载模块及移动终端 Download PDF

Info

Publication number
WO2017113729A1
WO2017113729A1 PCT/CN2016/089567 CN2016089567W WO2017113729A1 WO 2017113729 A1 WO2017113729 A1 WO 2017113729A1 CN 2016089567 W CN2016089567 W CN 2016089567W WO 2017113729 A1 WO2017113729 A1 WO 2017113729A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
texture
data layer
current
degree
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2016/089567
Other languages
English (en)
French (fr)
Inventor
许小飞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Le Holdings Beijing Co Ltd
Leshi Zhixin Electronic Technology Tianjin Co Ltd
Original Assignee
Le Holdings Beijing Co Ltd
Leshi Zhixin Electronic Technology Tianjin Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Le Holdings Beijing Co Ltd, Leshi Zhixin Electronic Technology Tianjin Co Ltd filed Critical Le Holdings Beijing Co Ltd
Priority to US15/236,764 priority Critical patent/US20170186218A1/en
Publication of WO2017113729A1 publication Critical patent/WO2017113729A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—Three-dimensional [3D] image rendering
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—Three-dimensional [3D] image rendering
    • G06T15/005—General purpose rendering architectures
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/122—Improving the three-dimensional [3D] impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/275—Image signal generators from three-dimensional [3D] object models, e.g. computer-generated stereoscopic image signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N2013/40—Privacy aspects, i.e. devices showing different images to different viewers, the images not being viewpoints of the same scene
    • H04N2013/405—Privacy aspects, i.e. devices showing different images to different viewers, the images not being viewpoints of the same scene the images being stereoscopic or three dimensional

Definitions

  • the present patent application relates to the field of image display technologies, and in particular, to a 360-degree image loading method, a loading module, and a mobile terminal.
  • the 360 degree panorama is a virtual reality (VR) technology that can be implemented on a microcomputer platform based on static images. Based on 360-degree panoramic view, people can perform 360-degree panoramic observation on the computer, and through interactive operation, can realize free browsing; thus experience the three-dimensional VR visual world.
  • VR virtual reality
  • the inventor has found in the process of implementing the invention that in the VR scheme based on the Android mobile phone, the implementation of the 360-degree full landscape mode is generally based on C++, and the rendering engine is implemented in the native layer. That is, the Android java layer passes the original two-dimensional image to the native layer, and the native layer renders and draws the original two-dimensional image, and then passes it to the java layer for display. In this process, the Android java layer and the native layer need to pass a large number of JNI (Java Native Interface) standard video broadcast control data; moreover, the JNI standard code is very inconvenient to debug. Therefore, the current design pattern has caused great inconvenience to developers.
  • JNI Java Native Interface
  • the purpose of some embodiments of the present invention is to provide a 360-degree image loading method, a loading module, and a mobile terminal, which avoids the transmission of a large amount of video broadcast control data between two data layers in a 360-degree video display process, and improves the program.
  • the efficiency of development is to provide a 360-degree image loading method, a loading module, and a mobile terminal, which avoids the transmission of a large amount of video broadcast control data between two data layers in a 360-degree video display process, and improves the program.
  • an embodiment of the present invention provides a 360-degree image loading method, including the following steps: a first data layer generates a three-dimensional image; a first data layer acquires a current view; and a first data layer according to the current view Rendering a three-dimensional image within a current viewing angle range to a texture to generate a texture image; a first data layer passes a texture label of the texture image to a second data layer; and a second data layer acquires the texture image according to the texture label And performing inverse distortion processing on the texture image.
  • One embodiment of the present invention provides a computer readable storage medium comprising computer executable instructions that, when executed by at least one processor, cause the processor to perform the above method.
  • An embodiment of the present invention further provides a 360-degree image loading module, including: a three-dimensional image generating unit, a viewpoint acquiring unit, a texture image generating unit, a texture image transmitting unit, and an anti-distortion processing unit; Generating a three-dimensional image; the viewpoint acquiring unit is configured to acquire a current viewpoint; the texture image generating unit is configured to generate a texture image according to the three-dimensional image in the current viewing angle range to generate a texture image; The texture image is passed to a second data layer; the anti-distortion processing unit is configured to perform inverse distortion processing on the texture image.
  • a 360-degree image loading module including: a three-dimensional image generating unit, a viewpoint acquiring unit, a texture image generating unit, a texture image transmitting unit, and an anti-distortion processing unit; Generating a three-dimensional image; the viewpoint acquiring unit is configured to acquire a current viewpoint; the texture image generating unit is configured to generate a texture image according to the three-
  • An embodiment of the present invention further provides a mobile terminal, including: the 360-degree image loading module.
  • the first data layer generates a three-dimensional image, and only transfers the texture label of the texture image in the current viewing angle range to the second data layer;
  • the data layer acquires the texture image according to the texture label, and the pattern The image is subjected to anti-distortion processing. That is, in the 360-degree video display process, only the texture label needs to be transmitted between the first data layer and the second data layer, thereby avoiding the transmission of a large amount of video broadcast control data between the first data layer and the second data layer, thereby improving The efficiency of program development.
  • the first data layer is a java layer and the second data layer is a native layer. That is, the present invention can be implemented based on the Android platform.
  • FIG. 1 is a flow chart of a 360-degree image loading method according to a first embodiment of the present invention
  • FIG. 2 is a flow chart of a 360 degree image loading module in accordance with a second embodiment of the present invention.
  • a first embodiment of the present invention relates to a 360-degree image loading method applied to a mobile terminal.
  • the mobile terminal is a smart phone based on the Android platform. Therefore, the first data layer and the second data layer in this embodiment are respectively a java layer and a native layer. However, this embodiment does not impose any restrictions on the development platform of the smart phone. When the development platform is different, the first data layer and the second data layer will also be different.
  • FIG. 1 The specific flow of the 360-degree image loading method in this embodiment is shown in FIG. 1 .
  • Step 10 The java layer generates a three-dimensional image.
  • the java layer builds a three-dimensional sphere model.
  • the java layer acquires a two-dimensional image pre-stored inside the mobile terminal, and maps the two-dimensional image texture to the three-dimensional sphere model to generate a three-dimensional image.
  • the specific implementation method is: the java layer generates a texture label (such as textureId), and generates a surface texture according to the texture label (SurfaceTexture); the java layer creates an empty surface through the SurfaceTexture to receive the two-dimensional image; the java layer binds the Surface to On a three-dimensional sphere. That is, a three-dimensional image is generated.
  • the generated three-dimensional image can be modified in terms of light, transparency, etc., so that the finally rendered three-dimensional image is more realistic.
  • Step 11 The java layer obtains the current viewpoint. Wherein step 11 comprises the following sub-steps.
  • Sub-step 111 The java layer detects the current pose of the mobile terminal.
  • the spatial orientation of the mobile terminal may be changed; the current posture reflects the spatial orientation of the mobile terminal.
  • the current posture in the present embodiment is characterized by the angular velocity of the mobile terminal.
  • the angular velocity of the mobile terminal includes three angular velocities of the mobile terminal in the X, Y, and Z axis directions.
  • the specific parameters for characterizing the current posture are not limited as long as the spatial orientation of the mobile terminal can be reflected.
  • Sub-step 112 The java layer calculates the current viewpoint according to the current pose.
  • three angles of the Euler angles are calculated according to the three angular velocities of the mobile terminal in the X, Y, and Z axis directions, and the three angles are: yaw, which represents an angle at which the viewpoint rotates around the Y axis; pitch, Indicates the angle at which the viewpoint rotates around the X axis, and roll, which represents the angle at which the viewpoint rotates about the Z axis.
  • yaw matrix::rotateY(yaw);
  • matrix_pitch matrix::rotateX(pitch);
  • matrix_roll matrix::rotateZ(roll). That is, the current viewpoint is substantially represented by three rotation matrices.
  • the current view point is not limited in any way; in other embodiments, the current view point may also be a pre-existing recommended view point in the mobile terminal (representing a better viewing angle), or pre-existing mobile Multiple continuously varying viewpoints within the terminal.
  • Step 12 The java layer renders the three-dimensional image in the current perspective range to the texture according to the current viewpoint to generate a texture image to generate the texture image.
  • step 12 includes the following sub-steps.
  • Sub-step 121 The java layer renders the three-dimensional image in the current perspective range to the frame buffer according to the current viewpoint.
  • the java layer creates a frame buffer object. Secondly, the java layer calculates the vertex coordinates of the three-dimensional image in the current viewing angle range according to the current viewpoint (ie, three rotation matrices), thereby rendering the three-dimensional image in the current viewing angle range into the frame buffer according to the calculated vertex coordinates, wherein The image in the frame buffer is the image displayed on the display.
  • the current viewpoint ie, three rotation matrices
  • Sub-step 122 The java layer renders the three-dimensional image in the frame buffer to the texture to generate a texture image.
  • the java layer generates a new texture label (textureId_new) and generates a surface texture (SurfaceTexture) based on the new texture label (textureId_new).
  • the java layer renders the 3D image in the frame buffer to the surface texture (SurfaceTexture) to generate the texture image.
  • Step 13 The java layer passes the texture label of the texture image to the native layer.
  • the java layer passes the texture label textureId_new of the texture image to the native layer.
  • Step 14 The native layer obtains the texture image according to the texture label, and performs inverse distortion processing on the texture image.
  • the native layer finds the actual physical position of the texture image according to the texture label (textureId_new) of the texture image, and the actual physical position is the frame buffer; the native layer directly performs anti-distortion processing on the texture image in the frame buffer, and the anti-distortion processing After the texture image covers the original texture
  • the image is stored in the frame buffer.
  • the anti-distortion processing is to eliminate the distortion phenomenon that occurs when the user views the image by using the lens.
  • Step 15 The java layer displays the anti-distortion processed texture image in the frame buffer on the display screen.
  • the first data layer (the java layer in this embodiment) implements rendering and rendering to generate a three-dimensional image; the first data layer renders the three-dimensional image in the current viewing angle range to the texture, and
  • the texture label is passed to the second data layer (native layer in this embodiment), and the anti-distortion processing is implemented by the second data layer. That is, in the 360-degree video display process, only the texture label needs to be transmitted between the first data layer and the second data layer, thereby avoiding the transmission of a large amount of video broadcast control data between the first data layer and the second data layer, thereby improving The efficiency of program development.
  • a second embodiment of the present invention relates to a 360-degree image loading module, as shown in FIG. 2, comprising: a three-dimensional image generating unit 10, a viewpoint acquiring unit 11, a texture image generating unit 12, a texture image transmitting unit 13, and an anti-distortion processing unit. 14 and display unit 15.
  • the three-dimensional image generating unit 10 is for generating a three-dimensional image.
  • the viewpoint acquiring unit 11 is configured to acquire a current viewpoint. Specifically, the viewpoint obtaining unit 11 further includes Includes: attitude detection subunit and viewpoint calculation subunit.
  • the attitude detection subunit is configured to detect a current posture of the mobile terminal; the viewpoint calculation subunit is configured to calculate the current viewpoint according to the current posture.
  • the attitude detecting subunit includes a gyroscope.
  • the texture image generating unit 12 is configured to generate a texture image according to rendering a three-dimensional image within a current viewing angle range to a texture.
  • the image transfer unit is for labeling the texture passed by the texture image to the anti-distortion processing unit 14.
  • the anti-distortion processing unit 14 is configured to acquire a texture image according to the texture label and perform inverse distortion processing on the texture image.
  • the display unit 15 is for displaying the texture image after the anti-distortion processing.
  • the present embodiment is a system embodiment corresponding to the first embodiment, and the present embodiment can be implemented in cooperation with the first embodiment.
  • the related technical details mentioned in the first embodiment are still effective in the present embodiment, and are not described herein again in order to reduce repetition. Accordingly, the related art details mentioned in the present embodiment can also be applied to the first embodiment.
  • each module involved in this embodiment is a logic module.
  • a logical unit may be a physical unit, a part of a physical unit, or multiple physical entities. A combination of units is implemented.
  • the present embodiment does not introduce a unit that is not closely related to solving the technical problem proposed by the present invention, but this does not mean that there are no other units in the present embodiment.
  • a third embodiment of the present invention relates to a mobile terminal including the 360-degree image loading module described in the second embodiment.
  • the mobile terminal in this embodiment is a smart phone, but is not limited thereto. .
  • the steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two.
  • Software modules can reside in random access memory (RAM), flash memory, read only memory (ROM), programmable read only memory (PROM), erasable read only memory (PROM), erasable and programmable only Read memory (EPROM), electrically erasable programmable read only memory (EEPROM), registers, hard disk, removable disk, compact disk read only memory (CD-ROM), or any other form known in the art Storage media.
  • the storage medium may be integral to the processor.
  • the processor and the storage medium can reside in an application specific integrated circuit (ASIC).
  • the ASIC can reside in a computing device or user terminal, or the processor and storage medium can reside as discrete components in a computing device or user terminal.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Graphics (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Processing Or Creating Images (AREA)
  • Image Generation (AREA)

Abstract

本发明实施例涉及图像显示技术领域，公开了一种360度图像加载方法、加载模块及移动终端。本发明部分实施例中，360度图像加载方法，包含以下步骤：第一数据层生成三维图像；第一数据层获取当前视点；第一数据层根据所述当前视点将当前视角范围内的三维图像渲染到纹理，以生成纹理图像；第一数据层将所述纹理图像的纹理标号传递至第二数据层；第二数据层根据所述纹理标号获取所述纹理图像，并对所述纹理图像进行反畸变处理。本发明部分实施例提供的360度图像加载方法、加载模块及移动终端，在360度视频显示过程中，避免了第一数据层与第二数据层之间大量视频播控数据的传递，提高了程序开发的效率。

Description

360度图像加载方法、加载模块及移动终端
交叉引用
本申请引用于2015年12月28日递交的第201511022816.5号中国专利申请，其通过引用被全部并入本申请。
技术领域
本专利申请涉及图像显示技术领域，特别涉及一种360度图像加载方法、加载模块及移动终端。
背景技术
360度全景是一种基于静态图像在微机平台上能够实现虚拟现实(VR)技术。基于360度全景，人们能在电脑上进行360度全景观察，而且通过交互操作，可以实现自由浏览；从而体验三维的VR视觉世界。
目前，发明人在实现本发明的过程中发现，在基于安卓手机的VR方案中，360度全景观影方式的实现一般都是基于C++，在native层实现渲染引擎。即，安卓java层将原始的二维图像传递至native层，native层将原始的二维图像渲染和绘制后，再传递至java层进行显示。在这过程中，安卓java层和native层需要传递大量JNI(Java Native Interface)标准的视频播控数据；而且，JNI标准的代码很不方便调试。因此，目前的设计模式给开发者造成了极大的不便。
发明内容
本发明部分实施例的目的在于提供一种360度图像加载方法、加载模块及移动终端，在360度视频显示过程中，避免了两个数据层之间大量视频播控数据的传递，提高了程序开发的效率。
为解决上述技术问题，本发明的实施方式提供了一种360度图像加载方法，包含以下步骤：第一数据层生成三维图像；第一数据层获取当前视点；第一数据层根据所述当前视点将当前视角范围内的三维图像渲染到纹理，以生成纹理图像；第一数据层将所述纹理图像的纹理标号传递至第二数据层；第二数据层根据所述纹理标号获取所述纹理图像，并对所述纹理图像进行反畸变处理。
本发明的一个实施例提供了一种计算机可读存储介质，包括计算机可执行指令，所述计算机可执行指令在由至少一个处理器执行时致使所述处理器执行上述方法。
本发明的实施方式还提供了一种360度图像加载模块，包含：三维图像生成单元、视点获取单元、纹理图像生成单元、纹理图像传递单元、以及反畸变处理单元；所述三维图像生成单元用于生成三维图像；所述视点获取单元用于获取当前视点；所述纹理图像生成单元用于根据将当前视角范围内的三维图像渲染到纹理，以生成纹理图像；所述图像传递单元用于将所述纹理图像传递至第二数据层；所述反畸变处理单元用于对所述纹理图像进行反畸变处理。
本发明的实施方式还提供了一种移动终端，包含：所述的360度图像加载模块。
本发明实施方式相对于现有技术而言，在360度视频显示过程中，第一数据层生成三维图像，并仅将当前视角范围内的纹理图像的纹理标号传递至第二数据层；第二数据层根据所述纹理标号获取所述纹理图像，并对所述纹 理图像进行反畸变处理。即，在360度视频显示过程中，第一数据层与第二数据层之间只需要传递纹理标号，从而避免了第一数据层与第二数据层之间大量视频播控数据的传递，提高了程序开发的效率。并且，由于在第一数据层中完成三维图像的渲染和绘制，即只有第一数据层需要使用渲染和绘制的相关数据而第二数据层无需使用，从而有效避免了第一数据层和第二数据层之间共用渲染和绘制的相关数据，简化了程序开发的复杂度。
在一个实施例中，所述第一数据层是java层，所述第二数据层是native层。即，本发明可基于安卓平台实现。
附图说明
图1是根据本发明第一实施方式的360度图像加载方法的流程图；
图2是根据本发明第二实施方式的360度图像加载模块的流程图。
具体实施方式
为使本发明部分实施例的目的、技术方案和优点更加清楚，下面将结合附图对本发明的各实施方式进行详细的阐述。然而，本领域的普通技术人员可以理解，在本发明各实施方式中，为了使读者更好地理解本申请而提出了许多技术细节。但是，即使没有这些技术细节和基于以下各实施方式的种种变化和修改，也可以实现本申请所要求保护的技术方案。
本发明的第一实施方式涉及一种360度图像加载方法，应用于移动终端。其中，移动终端为基于安卓平台的智能手机，因此，本实施方式中的第一数据层与第二数据层分别为java层与native层。然而，本实施方式对智能手机的开发平台不做任何限制，当开发平台不同时，第一数据层与第二数据层也会随之不同。
本实施方式中的360度图像加载方法的具体流程如图1所示。
步骤10：java层生成三维图像。
首先，java层建立一个三维球体模型。
其次，java层获取移动终端内部预存的二维图像，并将该二维图像纹理贴图至该三维球体模型，以生成三维图像。具体实现方法为：java层生成一个纹理标号(如textureId)，并根据纹理标号生成一个表面纹理(SurfaceTexture)；java层通过SurfaceTexture创建一个空的Surface去接收二维图像；java层将Surface绑定到三维球体上。即，生成了三维图像。
较佳的，纹理贴图后，还可以对生成的三维图像进行光线、透明度等方面的修饰，使得最后呈现的三维图像更加真实。
步骤11：java层获取当前视点。其中，步骤11包含以下子步骤。
子步骤111：java层检测移动终端的当前姿态。
具体而言，用户在使用移动终端时，可能会变换移动终端的空间朝向；当前姿态即反映移动终端的空间朝向。本实施方式中的当前姿态由移动终端的角速度表征。其中，移动终端的角速度包含移动终端在X、Y、Z轴方向上的三个角速度。然而，本实施方式中对表征当前姿态的具体参数不作任何限制，只要能够反映移动终端的空间朝向即可。
子步骤112：java层根据当前姿态计算当前视点。
具体而言，首先，根据移动终端在X、Y、Z轴方向上的三个角速度计算欧拉角的三个角度，三个角度分别为：yaw，表示视点绕Y轴旋转的角度；pitch，表示视点绕X轴旋转的角度，roll，表示视点绕Z轴旋转的角度。其次，根据欧拉角的三个角度，计算三个旋转矩阵matrix_yaw＝matrix::rotateY(yaw)；matrix_pitch＝matrix::rotateX(pitch)；matrix_roll＝matrix::rotateZ(roll)。即，当前视点实质由三个旋转矩阵表示。
需要说明的是，本实施方式对当前视点的获取方式不作任何限制；于其他实施方式中，当前视点也可以为预存在移动终端内的推荐视点(表示较佳的观看角度)、或者预存在移动终端内的多个连续变化的视点。
步骤12：java层根据当前视点将当前视角范围内的三维图像渲染到纹理，以生成纹理图像生成纹理图像。其中，步骤12包含以下子步骤。
子步骤121：java层根据当前视点将当前视角范围内的三维图像渲染到帧缓存。
首先，java层创建一个帧缓存(Frame buffer object)。其次，java层根据当前视点(即三个旋转矩阵)计算出当前视角范围内的三维图像的顶点坐标，从而根据计算出的顶点坐标将当前视角范围内的三维图像渲染到帧缓存内，其中，帧缓存内的图像即为显示屏显示出来的图像。
子步骤122：java层将帧缓存中的三维图像渲染到纹理，以生成纹理图像。
首先，java层生成一个新的纹理标号(textureId_new)，并根据新的纹理标号(textureId_new)生成一个表面纹理(SurfaceTexture)。其次，java层将帧缓存中的三维图像渲染到表面纹理(SurfaceTexture)，以生成纹理图像。
步骤13：java层将纹理图像的纹理标号传递至native层。
即，java层将纹理图像的纹理标号textureId_new传递至native层。
步骤14：native层根据纹理标号获取纹理图像，并对纹理图像进行反畸变处理。
即，native层根据纹理图像的纹理标号(textureId_new)查找到纹理图像的实际物理位置，该实际物理位置即为上述帧缓存；native层直接对帧缓存内的纹理图像进行反畸变处理，反畸变处理后的纹理图像覆盖原来的纹理 图像储存在帧缓存内。其中，反畸变处理是为了消除后续用户使用透镜观看图像时发生的畸变现象，本领域技术人员应当了解反畸变处理的具体方式，此处不再赘述。
步骤15：java层将帧缓存中的反畸变处理后的纹理图像在显示屏上进行显示。
本发明提供的360度图像加载方法，第一数据层(本实施例中为java层)实现渲染与绘制以生成三维图像；第一数据层将当前视角范围内的三维图像渲染到纹理后，并将纹理标号传递至第二数据层(本实施例中为native层)，并由第二数据层实现反畸变处理。即，在360度视频显示过程中，第一数据层与第二数据层之间只需要传递纹理标号，从而避免了第一数据层与第二数据层之间大量视频播控数据的传递，提高了程序开发的效率。并且，由于在第一数据层中完成三维图像的渲染和绘制，即只有第一数据层需要使用渲染和绘制的相关数据而第二数据层无需使用，从而有效避免了第一数据层和第二数据层之间共用渲染和绘制的相关数据，简化了程序开发的复杂度。
上面各种方法的步骤划分，只是为了描述清楚，实现时可以合并为一个步骤或者对某些步骤进行拆分，分解为多个步骤，只要包含相同的逻辑关系，都在本专利的保护范围内；对算法中或者流程中添加无关紧要的修改或者引入无关紧要的设计，但不改变其算法和流程的核心设计都在该专利的保护范围内。
本发明第二实施方式涉及一种360度图像加载模块，如图2所示，包含：三维图像生成单元10、视点获取单元11、纹理图像生成单元12、纹理图像传递单元13、反畸变处理单元14以及显示单元15。
三维图像生成单元10用于生成三维图像。
视点获取单元11用于获取当前视点。具体而言，视点获取单元11还包 含：姿态检测子单元与视点计算子单元。姿态检测子单元用于检测移动终端的当前姿态；视点计算子单元用于根据所述当前姿态计算所述当前视点。其中，姿态检测子单元包含陀螺仪。
纹理图像生成单元12用于根据将当前视角范围内的三维图像渲染到纹理，以生成纹理图像。
图像传递单元用于将纹理图像传递的纹理标号至反畸变处理单元14。
反畸变处理单元14用于根据纹理标号获取纹理图像，并对纹理图像进行反畸变处理。
显示单元15用于显示经反畸变处理后的纹理图像。
不难发现，本实施方式为与第一实施方式相对应的系统实施例，本实施方式可与第一实施方式互相配合实施。第一实施方式中提到的相关技术细节在本实施方式中依然有效，为了减少重复，这里不再赘述。相应地，本实施方式中提到的相关技术细节也可应用在第一实施方式中。
值得一提的是，本实施方式中所涉及到的各模块均为逻辑模块，在实际应用中，一个逻辑单元可以是一个物理单元，也可以是一个物理单元的一部分，还可以以多个物理单元的组合实现。此外，为了突出本发明的创新部分，本实施方式中并没有将与解决本发明所提出的技术问题关系不太密切的单元引入，但这并不表明本实施方式中不存在其它的单元。
本发明第三实施方式涉及一种移动终端，包含第二实施方式所述的360度图像加载模块。本实施方式中的移动终端为智能手机，然并不限于此。。
其中，第二实施方式中提到的相关技术细节在本实施方式中依然有效，在第二实施方式中所能达到的技术效果在本实施方式中也同样可以实现，为了减少重复，这里不再赘述。相应地，本实施方式中提到的相关技术细节也可应用在第二实施方式中。
结合本文中所揭示的实施例而描述的方法或算法的步骤可直接体现于硬件中，由处理器执行的软件模块中或所述两者的组合中。软件模块可驻留在随机存取存储器(RAM)、快闪存储器、只读存储器(ROM)、可编程只读存储器(PROM)，可擦除只读存储器(PROM)、可擦除可编程只读存储器(EPROM)、电可擦除可编程只读存储器(EEPROM)、寄存器、硬盘、可装卸式盘、压缩光盘只读存储器(CD-ROM)或此项技术中已知的任一其他形式的存储媒体。在替代方案中，存储媒体可与处理器成一体式。处理器及存储媒体可驻留在专用集成电路(ASIC)中。ASIC可驻留在计算装置或用户终端中，或者，处理器及存储媒体可作为离散组件驻留在计算装置或用户终端中。
本领域的普通技术人员可以理解，上述各实施方式是实现本发明的具体实施例，而在实际应用中，可以在形式上和细节上对其作各种改变，而不偏离本发明的精神和范围。

Claims (12)

  1. 一种360度图像加载方法，包含以下步骤：
    第一数据层生成三维图像；
    第一数据层获取当前视点；
    第一数据层根据所述当前视点将当前视角范围内的三维图像渲染到纹理，以生成纹理图像；
    第一数据层将所述纹理图像的纹理标号传递至native层；
    第二数据层根据所述纹理标号获取所述纹理图像，并对所述纹理图像进行反畸变处理。
  2. 根据权利要求1所述的360度图像加载方法，其中，所述第一数据层根据所述当前视点将当前视角范围内的三维图像渲染到纹理，以生成纹理图像的步骤，包含以下子步骤：
    第一数据层根据所述当前视点将当前视角范围内的三维图像渲染到帧缓存；
    第一数据层将所述帧缓存中的内容渲染到纹理，以生成纹理图像。
  3. 根据权利要求1或2所述的360度图像加载方法，其中，所述获取当前视点的步骤，包含以下子步骤：
    第一数据层检测移动终端的当前姿态；
    第一数据层根据所述当前姿态计算所述当前视点。
  4. 根据权利要求3所述的360度图像加载方法，其中，所述当前视点至少由移动终端的角速度表征。
  5. 根据权利要求1至4任一项所述的360度图像加载方法，其中，所 述第二数据层对所述纹理图像进行反畸变处理的步骤之后，还包含以下步骤：
    第一数据层显示经所述反畸变处理后的纹理图像。
  6. 根据权利要求1至5任一项所述的360度图像加载方法，其中，所述第一数据层是java层，所述第二数据层是native层。
  7. 一种360度图像加载模块，包含：三维图像生成单元、视点获取单元、纹理图像生成单元、纹理图像传递单元以及反畸变处理单元；
    所述三维图像生成单元用于生成三维图像；
    所述视点获取单元用于获取当前视点；
    所述纹理图像生成单元用于根据将当前视角范围内的三维图像渲染到纹理，以生成纹理图像；
    所述图像传递单元用于将所述纹理图像的纹理标号传递至所述反畸变处理单元；
    所述反畸变处理单元根据所述纹理标号获取所述纹理图像，并对所述纹理图像进行反畸变处理。
  8. 根据权利要求7所述的360度图像加载模块，其中，所述视点获取单元还包含：姿态检测子单元与视点计算子单元；
    所述姿态检测子单元用于检测移动终端的当前姿态；
    所述视点计算子单元用于根据所述当前姿态计算所述当前视点。
  9. 根据权利要求8所述的360度图像加载模块，其中，所述姿态检测子单元包含陀螺仪。
  10. 根据权利要求7至9任一项所述的360度图像加载模块，其中，所述360度图像显示模块还包含：显示单元；
    所述显示单元用于显示经所述反畸变处理后的纹理图像。
  11. 一种移动终端，包含权利要求7至10中任意一项所述的360度图像加载模块。
  12. 一种计算机可读存储介质，包括计算机可执行指令，所述计算机可执行指令在由至少一个处理器执行时致使所述处理器执行如权利要求1-6任一项所述的方法。
PCT/CN2016/089567 2015-12-28 2016-07-10 360度图像加载方法、加载模块及移动终端 Ceased WO2017113729A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US15/236,764 US20170186218A1 (en) 2015-12-28 2016-08-15 Method for loading 360 degree images, a loading module and mobile terminal

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201511022816.5A CN105898272A (zh) 2015-12-28 2015-12-28 360度图像加载方法、加载模块及移动终端
CN201511022816.5 2015-12-28

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15/236,764 Continuation US20170186218A1 (en) 2015-12-28 2016-08-15 Method for loading 360 degree images, a loading module and mobile terminal

Publications (1)

Publication Number Publication Date
WO2017113729A1 true WO2017113729A1 (zh) 2017-07-06

Family

ID=57002568

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/089567 Ceased WO2017113729A1 (zh) 2015-12-28 2016-07-10 360度图像加载方法、加载模块及移动终端

Country Status (2)

Country Link
CN (1) CN105898272A (zh)
WO (1) WO2017113729A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109155078A (zh) * 2018-08-01 2019-01-04 深圳前海达闼云端智能科技有限公司 样本图像的集合的生成方法、装置、电子设备和存储介质

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106358036B (zh) * 2016-08-31 2018-05-08 杭州当虹科技有限公司 一种以预设视角观看虚拟现实视频的方法
CN106934763B (zh) * 2017-04-17 2023-08-22 北京灵起科技有限公司 全景摄像机、行车记录仪、图像处理方法和装置
CN109271117A (zh) * 2017-07-17 2019-01-25 北京海鲸科技有限公司 一种图像显示方法、装置及设备
CN108282648B (zh) * 2018-02-05 2020-11-03 北京搜狐新媒体信息技术有限公司 一种vr渲染方法、装置、穿戴式设备及可读存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040095385A1 (en) * 2002-11-18 2004-05-20 Bon-Ki Koo System and method for embodying virtual reality
JP2005063041A (ja) * 2003-08-08 2005-03-10 Olympus Corp ３次元モデリング装置、方法、及びプログラム
CN102483859A (zh) * 2009-09-29 2012-05-30 索尼计算机娱乐公司 全景图像显示装置和全景图像显示方法
CN103929536A (zh) * 2014-03-31 2014-07-16 广东明创软件科技有限公司 提高图片处理回显速度的方法及其移动终端
CN103955960A (zh) * 2014-03-21 2014-07-30 南京大学 一种基于单幅输入图像的图像视点变换方法
CN104867175A (zh) * 2015-06-02 2015-08-26 孟君乐 一种虚拟效果图实景展示装置及其实现方法

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102957748A (zh) * 2012-11-07 2013-03-06 广东威创视讯科技股份有限公司 三维场景动态更新方法和系统
CN103279382B (zh) * 2013-04-27 2016-12-28 北京微云即趣科技有限公司 原生方式访问资源的方法、Java端、原生端及系统
CN103617027B (zh) * 2013-10-29 2015-07-29 合一网络技术(北京)有限公司 基于安卓系统的图像渲染引擎构建方法及系统

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040095385A1 (en) * 2002-11-18 2004-05-20 Bon-Ki Koo System and method for embodying virtual reality
JP2005063041A (ja) * 2003-08-08 2005-03-10 Olympus Corp ３次元モデリング装置、方法、及びプログラム
CN102483859A (zh) * 2009-09-29 2012-05-30 索尼计算机娱乐公司 全景图像显示装置和全景图像显示方法
CN103955960A (zh) * 2014-03-21 2014-07-30 南京大学 一种基于单幅输入图像的图像视点变换方法
CN103929536A (zh) * 2014-03-31 2014-07-16 广东明创软件科技有限公司 提高图片处理回显速度的方法及其移动终端
CN104867175A (zh) * 2015-06-02 2015-08-26 孟君乐 一种虚拟效果图实景展示装置及其实现方法

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109155078A (zh) * 2018-08-01 2019-01-04 深圳前海达闼云端智能科技有限公司 样本图像的集合的生成方法、装置、电子设备和存储介质
CN109155078B (zh) * 2018-08-01 2023-03-31 达闼机器人股份有限公司 样本图像的集合的生成方法、装置、电子设备和存储介质

Also Published As

Publication number Publication date
CN105898272A (zh) 2016-08-24

Similar Documents

Publication Publication Date Title
US20170186219A1 (en) Method for 360-degree panoramic display, display module and mobile terminal
WO2017113731A1 (zh) 360度全景显示方法、显示模块及移动终端
CN109743626B (zh) 一种图像显示方法、图像处理方法和相关设备
US9135678B2 (en) Methods and apparatus for interfacing panoramic image stitching with post-processors
CN103247072B (zh) 基于安卓系统实现三维旋转界面的方法及装置
CN114842120B (zh) 一种图像渲染处理方法、装置、设备及介质
WO2018119786A1 (zh) 一种显示数据处理方法及装置
WO2017113729A1 (zh) 360度图像加载方法、加载模块及移动终端
CN114419226A (zh) 全景渲染方法、装置、计算机设备和存储介质
CN114663632B (zh) 基于空间位置的光照显示虚拟物体的方法及设备
CN114549289A (zh) 图像处理方法、装置、电子设备和计算机存储介质
CN109448050A (zh) 一种目标点的位置的确定方法及终端
CN111161398B (zh) 一种图像生成方法、装置、设备及存储介质
WO2025077567A1 (zh) 三维模型输出方法、装置、设备及计算机可读存储介质
CN114049403B (zh) 一种多角度三维人脸重建方法、装置及存储介质
CN114758100A (zh) 显示方法、装置、电子设备和计算机可读存储介质
CN108364324B (zh) 图像数据处理方法、装置及电子终端
CN106909223B (zh) 基于3d场景的摄像头朝向修正方法及装置
EP4485357A2 (en) Image processing method and apparatus, electronic device, and storage medium
CN111862342A (zh) 增强现实的纹理处理方法、装置、电子设备及存储介质
CN114663615B (zh) 电子地图显示方法及装置、电子设备
CN120013780B (zh) 场景渲染方法、装置、ar设备及存储介质
US11910068B2 (en) Panoramic render of 3D video
CN110310325A (zh) 一种虚拟测量方法、电子设备及计算机可读存储介质
US20170186218A1 (en) Method for loading 360 degree images, a loading module and mobile terminal

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16880527

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16880527

Country of ref document: EP

Kind code of ref document: A1