CN111885310A

CN111885310A - Panoramic data processing method, processing equipment and playing equipment

Info

Publication number: CN111885310A
Application number: CN202010901729.1A
Authority: CN
Inventors: 王士博; 沈靖程
Original assignee: Shenzhen Pisofttech Technology Co ltd
Current assignee: Shenzhen Pisofttech Technology Co ltd
Priority date: 2020-08-31
Filing date: 2020-08-31
Publication date: 2020-11-03

Abstract

The embodiment of the invention discloses a panoramic data processing method, a panoramic data processing device and a panoramic data playing device, which are used for reducing the data capacity of an image while keeping the definition of the image when the image is watched. The method provided by the embodiment of the invention comprises the following steps: the processing equipment acquires a panoramic plane image, the panoramic plane image comprises a plane fixation position, a first image area and a second image area, the plane fixation position is a position for a user to fix, and the first image area is closer to the plane fixation position than the second image area. Then, the processing device reduces the data capacity of the second image area on the panoramic plane image to obtain the target panoramic plane image. The plane watching position is a position for the user to watch, so that when the target panoramic plane image is displayed, a first image area close to the position for the user to watch is clearer, the definition of the image is kept, a second image area far away from the position for the user to watch is lower in definition but small in data capacity, and the integral data capacity of the image is reduced.

Description

Panoramic data processing method, processing equipment and playing equipment

Technical Field

The present invention relates to the field of image data processing, and in particular, to a panoramic data processing method, a panoramic data processing device, and a panoramic data playing device.

Background

A PANORAMIC image (english name: PANORAMIC PHOTO, or PANORAMA) generally refers to a photograph taken in conformity with a normal effective visual angle of both eyes of a human (about 90 degrees horizontally, 70 degrees vertically) or a full scene range including more than a visual angle of the residual light of both eyes (about 180 degrees horizontally, 90 degrees vertically), or even 360 degrees.

Since the panoramic image includes a large field of view, the amount of information is large, and thus, the data capacity of the panoramic image is large, which results in that the panoramic image is not favorable for storage, transmission, encoding and decoding, and the like.

Disclosure of Invention

The embodiment of the invention provides a panoramic data processing method, a panoramic data processing device and a panoramic data playing device, which are used for reducing the data capacity of an image while maintaining the definition of the image when the image is watched.

To achieve the purpose, the embodiment of the invention adopts the following technical scheme:

a panoramic data processing method, comprising:

the method comprises the steps that a processing device obtains a panoramic plane image, wherein the panoramic plane image comprises a plane fixation position, a first image area and a second image area, the plane fixation position is a position for a user to fix, and the first image area is closer to the plane fixation position than the second image area;

the processing device reduces a data capacity of the second image area on the panoramic planar image to obtain a target panoramic planar image, wherein the first image area is higher in definition than the second image area on the target panoramic planar image, and the first image area is larger in data capacity than the second image area.

Optionally, the processing device acquires a panoramic plane image, and includes:

the processing equipment acquires a panoramic spherical image;

and the processing equipment projects the panoramic spherical image onto a plane to obtain a panoramic plane image.

Optionally, the method further comprises:

the processing equipment acquires target viewing direction data, and the target viewing direction data is used for representing a target viewing direction;

before the processing device projects the panoramic spherical image onto a plane to obtain a panoramic plane image, the method further includes:

and the processing equipment takes the sphere center of the panoramic spherical image as an origin, extends the target viewing direction outwards according to the target viewing direction data, and determines the intersection point of the target viewing direction and the panoramic spherical image as a panoramic fixation position, wherein the planar fixation position is obtained when the panoramic fixation position is projected to a plane.

Optionally, the processing device obtains a panoramic spherical image, and includes:

the processing equipment acquires a panoramic spherical image through a camera on the processing equipment, the optical center of the processing equipment is the spherical center of the panoramic spherical image, the optical center of the processing equipment represents the geometric center of a geometric figure formed by the optical centers of the cameras of the processing equipment, and the processing equipment is provided with at least two cameras;

the method further comprises the following steps:

the processing equipment acquires playing end viewing direction data sent by playing equipment, wherein the playing end viewing direction data is used for indicating a viewing direction represented in a coordinate system where the playing equipment is located;

the processing equipment acquires an attitude angle when the processing equipment acquires the panoramic spherical image;

the processing device acquires target viewing direction data, and comprises:

the processing equipment calculates to obtain target viewing direction data according to the playing end viewing direction data and the attitude angle, wherein the target viewing direction data and the panoramic spherical image are both represented in a coordinate system where the processing equipment is located;

after the processing device reduces the data capacity of the second image area on the panoramic planar image to obtain a target panoramic planar image, the method further comprises:

and the processing equipment sends the target panoramic planar image to the playing equipment so that the playing equipment displays the target panoramic planar image.

Optionally, the projecting, by the processing device, the panoramic spherical image onto a plane to obtain a panoramic plane image includes:

the processing equipment projects the panoramic spherical image onto a plane by using an equidistant rectangular projection mode to obtain a panoramic plane image;

the processing device reducing a data capacity of the second image area on the panoramic planar image to obtain a target panoramic planar image, comprising:

the processing device reduces pixels of the second image area on the panoramic planar image to obtain a target panoramic planar image, wherein the first image area has more pixels than the second image area on the target panoramic planar image.

Optionally, along a target direction, the sharpness of the target panoramic plane image decreases in a linear or step-like manner;

along the target direction, the data capacity of the target panoramic plane image is reduced in a linear or step-like manner;

wherein the target direction is a direction pointing from the planar gaze location to an exterior of the target panoramic planar image on the target panoramic planar image.

The embodiment of the invention also adopts the following technical scheme:

a panoramic data processing method, comprising:

the method comprises the steps that a playing device obtains a target panoramic plane image sent by a processing device, wherein the target panoramic plane image is obtained by reducing the data capacity of a second image area on the panoramic plane image, the panoramic plane image comprises a plane fixation position, a first image area and the second image area, the plane fixation position is a position for a user to fix, and the first image area is closer to the plane fixation position than the second image area; on the target panoramic plane image, the first image area is higher in definition than the second image area, and the data capacity of the first image area is larger than that of the second image area;

the playing device projects the target panoramic plane image to obtain a target panoramic image;

and the playing equipment displays the target panoramic image.

Optionally, before the playing device acquires the target panoramic plane image sent by the processing device, the method further includes:

the playing device acquires playing end viewing direction data, wherein the playing end viewing direction data is used for indicating a viewing direction represented in a coordinate system where the playing device is located;

and the playing device sends the playing end viewing direction data to the processing device, so that the processing device determines the plane fixation position according to the playing end viewing direction data.

The embodiment of the invention also adopts the following technical scheme:

a processing device, comprising:

the system comprises an acquisition module, a display module and a display module, wherein the acquisition module is used for acquiring a panoramic plane image, the panoramic plane image comprises a plane fixation position, a first image area and a second image area, the plane fixation position is a position for a user to fix, and the first image area is closer to the plane fixation position than the second image area;

a processing module, configured to reduce a data capacity of the second image area on the panoramic planar image to obtain a target panoramic planar image, where, on the target panoramic planar image, the first image area has a higher definition than the second image area, and the first image area has a larger data capacity than the second image area.

The embodiment of the invention also adopts the following technical scheme:

a playback device comprising:

an obtaining module, configured to obtain a target panoramic planar image sent by a processing device, where the target panoramic planar image is obtained by reducing a data capacity of a second image area on a panoramic planar image, the panoramic planar image includes a planar gaze position, a first image area, and the second image area, the planar gaze position is a position for a user to gaze, and the first image area is closer to the planar gaze position than the second image area; on the target panoramic plane image, the first image area is higher in definition than the second image area, and the data capacity of the first image area is larger than that of the second image area;

the projection module is used for projecting the target panoramic plane image to obtain a target panoramic image;

and the display module is used for displaying the target panoramic image.

According to the technical scheme, the embodiment of the invention has the following advantages:

the embodiment of the invention provides a panoramic data processing method, wherein in the panoramic data processing method, a processing device acquires a panoramic plane image, the panoramic plane image comprises a plane fixation position, a first image area and a second image area, wherein the plane fixation position is a position for a user to fix, and the first image area is closer to the plane fixation position than the second image area. Then, the processing device reduces a data capacity of a second image area on the panoramic planar image on which the first image area is higher in definition than the second image area to obtain a target panoramic planar image, the first image area being larger in data capacity than the second image area. Therefore, the plane watching position is the position watched by the user, so that when the target panoramic plane image is displayed, the first image area close to the position watched by the user is clearer, the definition of watching the image is kept, the second image area far away from the position watched by the user is lower in definition but small in data capacity, and the integral data capacity of the image is reduced.

Drawings

Fig. 1 is a flowchart of a method of processing panoramic data according to an embodiment of the present invention;

fig. 2 is a flowchart of a method of processing panoramic data according to an embodiment of the present invention;

fig. 3 is a flowchart of a method of processing panoramic data according to an embodiment of the present invention;

FIG. 4 is a schematic view of a panoramic plan image according to the embodiment shown in FIG. 3;

FIG. 5 is a schematic view of the panoramic planar image of FIG. 4 after adjustment;

FIG. 6 is a schematic diagram of a division of the schematic diagram shown in FIG. 5;

FIG. 7 is a schematic diagram of a panoramic planar image of an object according to the embodiment shown in FIG. 3;

fig. 8 is a schematic structural diagram of a processing apparatus according to an embodiment of the present invention;

fig. 9 is a schematic structural diagram of a playing device according to an embodiment of the present invention.

Detailed Description

Fig. 1 is a flowchart of a method for processing panoramic data according to an embodiment of the present invention. The panoramic data processing method can be applied to a playing device.

Referring to fig. 1, a panoramic data processing method according to an embodiment of the present invention includes:

step 101: and the playing device acquires the target panoramic plane image sent by the processing device.

Wherein the target panoramic planar image is derived by reducing a data capacity of the second image area on the panoramic planar image.

The panoramic planar image includes a planar gaze location, a first image region, and a second image region. The planar gaze location is a location at which the user gazes, and the first image region is closer to the planar gaze location than the second image region. On the target panoramic plane image, the first image area has higher definition than the second image area, and the data capacity of the first image area is larger than that of the second image area.

The processing device is provided with a target panoramic planar image, and the processing device can send the target panoramic planar image to the playing device, specifically, the processing device can send the target panoramic planar image to the playing device through a wireless transmission mode.

In the embodiment of the present invention, the target panoramic plane image is obtained by reducing the data capacity of the second image area on the panoramic plane image. The panoramic planar image includes a planar gaze location, a first image region, and a second image region, such that the planar gaze location, the first image region, and the second image region are also included on the target panoramic planar image.

On the panoramic planar image and the target panoramic planar image, the first image area is closer to the planar gaze position than the second image area. The plane fixation position is a position for users to fix, and the definition of the first image area is higher than that of the second image area on the target panoramic plane image, so that when the users watch the target panoramic plane image, the first image area with higher definition falls into the visual field range of the users more than the second image area, and the panoramic image which can be watched by the users can maintain a certain definition requirement. On the other hand, the first image area has a larger data capacity than the second image area, and the data capacity of the second image area is subjected to reduction processing, so that the overall data capacity of the target panoramic flat image is reduced, and the target panoramic flat image is facilitated to be transmitted, codec-processed, and stored on a network.

It should be understood that the playing device in the embodiment of the present invention includes, but is not limited to, a PC-side panoramic player, a mobile phone-side panoramic player, a VR glasses panoramic player, and the like, and this is not particularly limited in the embodiment of the present invention.

It should be understood that the processing device of the embodiment of the present invention may be an electronic device such as a panoramic camera.

Step 102: and the playing equipment projects the target panoramic plane image to obtain a target panoramic image.

The target panoramic planar image is a planar image, and in order to perform panoramic display, the target panoramic planar image needs to be projected to obtain a target panoramic image.

The target panoramic image may be a spherical panoramic image, a cubic panoramic image, or a cylindrical panoramic image, and the like, which is not specifically limited in this embodiment of the present invention.

Step 103: and the playing equipment displays the target panoramic image.

The playing device acquires the target panoramic image, and then the playing device displays the target panoramic image for the user to watch. And when the user watches the target panoramic image, the watching position of the user is concentrated on the projection position of the plane watching position on the target panoramic image.

In summary, in the panoramic data processing method according to the embodiment of the present invention, the playing device obtains the target panoramic planar image sent by the processing device, where the target panoramic planar image is obtained by reducing the data capacity of the second image area on the panoramic planar image, the panoramic planar image includes a planar gaze position, a first image area and a second image area, the planar gaze position is a position for the user to gaze, and the first image area is closer to the planar gaze position than the second image area; on the target panoramic plane image, the first image area has higher definition than the second image area, and the data capacity of the first image area is larger than that of the second image area. And the playing equipment projects the target panoramic plane image to obtain a target panoramic image. Then, the playback device presents the target panoramic image. In the target panoramic plane image, the plane fixation position is the position for the user to fix, so that a first image area close to the position for the user to fix is clearer, the definition of the image to be fixed is kept, a second image area far away from the position for the user to fix is lower in definition but small in data capacity, and the integral data capacity of the image is reduced.

Fig. 2 is a flowchart of a method for processing panoramic data according to an embodiment of the present invention. The panorama data processing method shown in fig. 2 may be applied to a processing device.

Referring to fig. 2, the panoramic data processing method according to the embodiment of the present invention includes:

step 201: the processing device acquires a panoramic planar image.

The panoramic plane image comprises a plane fixation position, a first image area and a second image area. The planar gaze location is a location at which the user gazes, and the first image region is closer to the planar gaze location than the second image region.

The panoramic plane image is an image obtained by projecting the panoramic image onto a plane, and the panoramic image is projected onto the plane, so that the panoramic plane image is beneficial to processing and transmitting the image.

And on the panoramic plane image, when the panoramic plane image is displayed for a user to watch, the plane watching position is a position for the user to watch.

Step 202: the processing device reduces a data capacity of a second image area on the panoramic planar image to obtain a target panoramic planar image.

On the target panoramic plane image, the first image area has higher definition than the second image area, and the data capacity of the first image area is larger than that of the second image area.

The processing device processes the panoramic planar image, specifically, reduces the data capacity of the second image area on the panoramic planar image, so as to obtain the target panoramic planar image from the panoramic planar image.

The data capacity of the second image area is reduced so that the first image area has a larger data capacity than the second image area, and accordingly, the first image area can have higher definition than the second image area. On the other hand, since the data capacity of the second image area is reduced, the overall data capacity of the target panoramic flat image is reduced, and the target panoramic flat image is more convenient for transmission, codec processing, and storage on the network.

There are various implementations in which the processing device acquires the panoramic planar image, for example, the processing device acquires the panoramic planar image from another device, or the processing device generates the panoramic planar image.

In a specific implementation manner, the step of acquiring, by the processing device, the panoramic plane image specifically includes: the processing device acquires the panoramic spherical image. And the processing equipment projects the panoramic spherical image onto a plane to obtain a panoramic plane image.

Specifically, the processing device may obtain a panoramic spherical image sent by another device, or the processing device is provided with a camera, and the processing device acquires the panoramic spherical image through the camera on the processing device. After the processing device acquires the panoramic spherical image, the panoramic spherical image can be projected onto a plane to obtain a panoramic plane image. The panoramic plane image is a panoramic image displayed on a plane.

Optionally, the panoramic data processing method according to the embodiment of the present invention further includes: the processing device obtains target viewing direction data. Wherein the target viewing direction data is used to represent a target viewing direction.

Before the step of projecting the panoramic spherical image onto a plane by the processing device to obtain the panoramic plane image, the panoramic data processing method according to the embodiment of the present invention further includes: and the processing equipment takes the sphere center of the panoramic spherical image as an origin, and extends the target viewing direction outwards according to the target viewing direction data to determine the intersection point of the target viewing direction and the panoramic spherical image as the panoramic fixation position. Wherein the planar gaze location is obtained from the panoramic gaze location as projected onto the plane.

Specifically, the panoramic spherical image is a spherical image with a center as a sphere center, and the panoramic spherical image surrounds the sphere center. The processing equipment analyzes the data of the target viewing direction, the target viewing direction is extended outwards from the center of the panoramic spherical image to form a ray, and the intersection point of the extended target viewing direction and the panoramic spherical image is the panoramic fixation position. The panoramic gaze location represents a location at which the user gazes at the panoramic spherical image. Because the panoramic fixation position is positioned on the panoramic spherical image, when the processing equipment projects the panoramic spherical image onto a plane to obtain a panoramic plane image, the panoramic fixation position is also projected onto the plane fixation position on the panoramic plane image. In this way, the processing device obtains the planar gaze position from the target viewing direction data.

In summary, in the panoramic data processing method according to the embodiment of the present invention, the processing device obtains the panoramic planar image, where the panoramic planar image includes a planar gaze location, a first image area, and a second image area, where the planar gaze location is a location at which the user gazes, and the first image area is closer to the planar gaze location than the second image area. Then, the processing device reduces a data capacity of a second image area on the panoramic planar image on which the first image area is higher in definition than the second image area to obtain a target panoramic planar image, the first image area being larger in data capacity than the second image area. Therefore, the plane watching position is the position watched by the user, so that when the target panoramic plane image is displayed, the first image area close to the position watched by the user is clearer, the definition of watching the image is kept, the second image area far away from the position watched by the user is lower in definition but small in data capacity, and the integral data capacity of the image is reduced.

Fig. 3 is a flowchart of a method for processing panoramic data according to an embodiment of the present invention. The panorama data processing method shown in fig. 3 can be implemented based on the panorama data processing method of the embodiment shown in fig. 1 and the panorama data processing method of the embodiment shown in fig. 2.

Referring to fig. 3, the panoramic data processing method according to the embodiment of the present invention includes:

step 301: the playing device obtains the viewing direction data of the playing terminal.

The data of the viewing direction of the playing end is used for indicating the viewing direction represented in the coordinate system where the playing device is located. The viewing direction of the playing end is the direction of sight line when the user views the panoramic image through the playing device. There are various ways to represent the viewing direction data of the playing end, for example, the viewing direction data of the playing end is a vector representing the direction.

The specific implementation manner of the playing device for acquiring the viewing direction data of the playing end is various, for example, a user inputs viewing direction information to the playing device, or a touch screen is arranged on the playing device, and after the user clicks the touch screen, the playing device determines the viewing direction according to the position where the user clicks the touch screen, so as to calculate the viewing direction data of the playing end.

In a specific implementation manner, a gyroscope is arranged on the playing device, and the playing device acquires the viewing direction data of the playing end through the gyroscope.

For example, the playing device is Virtual Reality technology (VR) glasses, when a user wears the VR glasses, a gyroscope on the VR glasses acquires an orientation vector, where the orientation vector represents a viewing direction of the user when the user views a panoramic image displayed on the VR glasses, that is, the orientation vector is data of a viewing direction of the playing terminal.

It should be understood that the playing device according to the embodiment of the present invention is a device capable of playing a panoramic image, such as a computer, a mobile phone, VR glasses, and the like provided with a panoramic player. In some embodiments, the playback device of the embodiments of the present invention may also play a panoramic video.

Step 302: the playing device sends the viewing direction data of the playing end to the processing device.

After the playing device acquires the playing end viewing direction data, the playing device sends the playing end viewing direction data to the processing device, so that the processing device determines the plane fixation position according to the playing end viewing direction data. The specific implementation of the processing device using the viewing direction data of the playing end can be seen from the following detailed description.

The connection between the playback device and the processing device may be in various manners, such as wired connection or wireless connection.

Before the playing device acquires the target panoramic planar image sent by the processing device, the panoramic data processing method according to the embodiment of the present invention executes step 301 and step 302, so that the target panoramic planar image acquired by the playing device from the processing device can be displayed according to the viewing direction of the user on the playing device, and meets the viewing requirement of the user.

It should be understood that, in the embodiment of the present invention, the specific implementation manner of step 301 and step 302 may be: the playing device acquires the viewing direction data of the playing end in real time, and sends the viewing direction data of the playing end to the processing device in real time.

Through the sending operation of the playing device, the processing device can acquire the viewing direction data of the playing terminal sent by the playing device. For example, the playing device is VR glasses, a gyroscope on the VR glasses acquires the viewing direction data of the playing end, and then the playing device sends the viewing direction data of the playing end to the far-end panoramic camera in real time.

Step 303: the processing equipment acquires the panoramic spherical image through a camera on the processing equipment.

Wherein, the optical center of the processing equipment is the spherical center of the panoramic spherical image.

The optical center of the processing apparatus represents the geometric center of a geometric figure composed of the optical centers of the cameras of the processing apparatus. Wherein, the processing equipment is provided with at least two cameras. In other words, each camera of the processing device has an optical center, and the optical centers of the cameras of the processing device are connected to form a geometric figure, which is the optical center of the processing device.

The panoramic spherical image is a panoramic image with a spherical structure. For example, in the present example, when the processing device is a panoramic camera, the distance between the subject and the optical center of the camera of the panoramic camera is d, i.e., the object distance of the panoramic camera. And taking the optical center of a camera of the panoramic camera as a spherical center and d as a radius to form a spherical surface, and projecting the shot object onto the spherical surface to obtain a panoramic image, namely the panoramic spherical image.

In the embodiment of the invention, the processing equipment is provided with the camera, and a panoramic image, in particular a panoramic spherical image, can be acquired through the camera.

The processing device is, for example, a panoramic camera, and two or more cameras are arranged on the panoramic camera, and images can be acquired through the cameras, and then the images acquired by the cameras are spliced to obtain a panoramic spherical image.

It should be understood that step 303 is one of specific implementations of the processing device obtaining the panoramic spherical image, in this case, the processing device may be a device capable of acquiring the panoramic spherical image, such as a panoramic camera, and in other embodiments, the processing device may also obtain the panoramic spherical image from other devices, which is not limited in this embodiment of the present invention.

It should be understood that the execution order of step 302 and step 303 is not limited by the embodiment of the present invention.

Step 304: the processing equipment acquires an attitude angle when the processing equipment acquires the panoramic spherical image.

When the processing equipment collects the panoramic spherical image through the camera on the processing equipment, the processing equipment acquires the attitude angle of the processing equipment. Wherein the attitude angle represents spatial attitude information of the processing device, such as an attitude azimuth quaternion.

There are various ways for the processing device to obtain the attitude angle, for example, a gyroscope is provided on the processing device, and the processing device obtains the attitude angle through the gyroscope.

For example, the processing device is a panoramic camera with a gyroscope disposed thereon, and the panoramic camera uses the gyroscope to obtain the current attitude angle of the panoramic camera.

Step 305: and the processing equipment calculates to obtain target viewing direction data according to the viewing direction data and the attitude angle of the playing end.

And the target viewing direction data and the panoramic spherical image are both represented in a coordinate system where the processing equipment is located. The target viewing direction data is used to represent a target viewing direction.

The processing device acquires the playing end viewing direction data sent by the playing device, wherein the playing end viewing direction data is used for indicating the viewing direction represented in the coordinate system where the playing device is located. Because the data of the viewing direction of the playing end is obtained on the playing device, the representation of the data of the viewing direction of the playing end is represented in the coordinate system where the playing device is located. In step 303, the processing device captures the panoramic spherical image, such that the panoramic spherical image is represented in the coordinate system in which the processing device is located.

In order to combine the viewing direction data of the playing end with the panoramic spherical image, the two data need to be converted into the same coordinate system for representation. In the embodiment of the invention, the two are represented in the coordinate system where the processing equipment is located. Specifically, the angle of the viewing direction data of the playing end is adjusted through the attitude angle, the viewing direction data of the playing end is represented in the coordinate system where the processing device is located, and the target viewing direction data is obtained, in other words, the target viewing direction data is the viewing direction data of the playing end represented in the coordinate system where the processing device is located. In this way, the target viewing direction data and the panoramic spherical image are both represented in the coordinate system where the processing device is located, so that the two can be combined.

In the embodiment of the present invention, the target viewing direction data is used to represent a target viewing direction. The target viewing direction data and the playback-side viewing direction data both represent the same viewing direction, but both are represented under different coordinate systems.

For example, in step 305, the playing-side viewing direction data is a direction vector V1, which is expressed in the coordinate system where the playing device is located, the attitude angle is the attitude azimuth quaternion Q of the panoramic camera, and the target viewing direction data is a direction vector V2, where V2 is V1 · Q. V2 is represented in the coordinate system in which the processing apparatus is located.

It should be understood that step 305 is one of specific implementations of the step of the processing device acquiring the target viewing direction data.

Step 306: and the processing equipment takes the sphere center of the panoramic spherical image as an origin, and extends the target viewing direction outwards according to the target viewing direction data to determine the intersection point of the target viewing direction and the panoramic spherical image as the panoramic fixation position.

Because the target viewing direction data and the panoramic spherical image are both represented in the coordinate system where the processing device is located, the target viewing direction data and the panoramic spherical image can be used in combination.

Specifically, since the target viewing direction data is used to represent the target viewing direction, the processing device extends the target viewing direction to the outside of the panoramic spherical image with the center of the sphere of the panoramic spherical image as the origin to form a ray, the ray represents the target viewing direction, and the ray and the panoramic spherical image intersect at an intersection point, which is the panoramic fixation position. The panoramic gaze location represents a location at which a user gazes on the panoramic spherical image when viewing the panoramic spherical image within the panoramic spherical image, the user's field of view encompassing the panoramic gaze location.

For example, the target viewing direction data is a direction vector V2, and a ray is obtained by extending along the direction of the direction vector V2 with the center of sphere of the panoramic spherical image as the origin, and the ray intersects with the spherical surface of the panoramic spherical image at a point P, which is the panoramic fixation position.

The target viewing direction data and the playing end viewing direction data represent the same viewing direction, namely the direction of the sight line of the user when the user views the panoramic image through the playing device, so that the panoramic fixation position is the position of the user using the playing device to fix on the panoramic spherical image.

Step 307: and the processing equipment projects the panoramic spherical image onto a plane to obtain a panoramic plane image.

The panoramic plane image comprises a plane fixation position, a first image area and a second image area. The plane fixation position is a position for a user to fix, and the first image area is closer to the plane fixation position than the second image area.

The plane fixation position is obtained when the panoramic fixation position is projected to the plane, specifically, the intersection point of the target viewing direction and the panoramic spherical image is the panoramic fixation position, the panoramic fixation position is on the panoramic spherical image, and in the process that the processing equipment projects the panoramic spherical image onto the plane to obtain the panoramic plane image, the panoramic fixation position is also projected on the plane to form the plane fixation position on the panoramic plane image.

Optionally, step 307 specifically includes: the processing equipment projects the panoramic spherical image onto a plane by using an equidistant rectangular projection (Equirectangular) mode to obtain a panoramic plane image.

For example, the processing device is a panoramic camera, and after the panoramic camera stitches the acquired images into a panoramic spherical image, the panoramic camera projects the panoramic spherical image onto a plane by using an equidistant rectangular projection manner, so as to obtain a panoramic plane image as shown in fig. 4. Where point 400 in fig. 4 represents a planar gaze location, point 400 results from the panoramic gaze location being projected onto the plane. The point 400 is a point that the user is looking right when using a playback device that is VR glasses.

In some embodiments of the present invention, after the panoramic planar image shown in fig. 4 is obtained, it may be processed, for example, to adjust the panoramic planar image so that the planar gaze location 400 is centered in the panoramic planar image, as shown in fig. 5. Fig. 6 is a schematic diagram of dividing the regions of fig. 5, and referring to the panoramic planar image shown in fig. 6, the planar gaze location is a point 400, the first image region is a region 402, the second image region is a region 401 and a region 403, and the region 402 is closer to the planar gaze location 400 than the region 401 and the region 403.

In the example of fig. 6, a region 401, a region 402, and a region 403 divide the panoramic plane image into three equal parts in the horizontal direction.

In the embodiment of the present invention, the implementation manner of the step of acquiring the panoramic plane image by the processing device is as follows: the processing equipment acquires the panoramic spherical image, and projects the panoramic spherical image onto a plane to obtain a panoramic plane image. The specific implementation process is shown in the above description.

It should be understood that the foregoing description of the embodiment of the present invention describes one implementation manner of the planar gaze location, and other implementation manners of the planar gaze location of the embodiment of the present invention are also possible, for example, the playing device sends positioning information to the processing device, and then the processing device positions the planar gaze location on the panoramic planar image according to the positioning information. Or the processing equipment positions out the plane watching position on the panoramic plane image according to the preset position information.

Step 308: the processing device reduces a data capacity of a second image area on the panoramic planar image to obtain a target panoramic planar image.

In order to reduce the data capacity of the panoramic image as a whole, the processing device reduces the data capacity of the second image area on the panoramic plane image when the target panoramic plane image is obtained from the panoramic plane image. The data capacity of the target panoramic plane image is less than the data capacity of the panoramic plane image due to the reduction in the data capacity of the second image area.

In the embodiment of the invention, the first image area and the second image area are sequentially transmitted on the panoramic spherical image, the panoramic plane image and the target panoramic plane image. The plane fixation position on the panoramic plane image is obtained by projecting the panoramic fixation position on the panoramic spherical image, and the plane fixation position is transmitted to the target panoramic plane image from the panoramic plane image. On the panoramic spherical image, the first image area is closer to the panoramic gaze position than the second image area, in other words, the distance between the first image area and the panoramic gaze position is smaller than the distance between the second image area and the panoramic gaze position along the surface of the panoramic spherical image.

There are various specific implementations of the processing device for reducing the data capacity of the second image area on the panoramic plane image, and optionally, step 308 specifically includes: the processing device reduces pixels of a second image area on the panoramic planar image to obtain a target panoramic planar image, wherein the first image area has more pixels than the second image area on the target panoramic planar image.

Specifically, the reduction of the pixels of the second image region can be achieved by reducing the area of the second image region. At this time, the first image area is closer to the panoramic gaze position than the second image area on the panoramic spherical image. The first image area is larger in area than the second image area on the target panoramic plane image, so that the first image area has more pixels than the second image area, and the first image area is clearer than the second image area.

The sharpness of the second image area is reduced due to the reduced data capacity of the second image area. At this time, on the target panoramic planar image, the first image area has higher definition than the second image area, so that the area close to the planar gazing position has higher definition and the area far from the planar gazing position has lower definition, for example, the panoramic planar image shown in fig. 6 is processed so that the area 401 and the area 403 are blurred and the area 402 is clearer, and at this time, the area near the planar gazing position 400 has higher definition and the area far from the planar gazing position 400 has lower definition.

For example, in the target panoramic planar image shown in fig. 6, the areas of the region 401 and the region 403 are compressed in the horizontal direction so that the areas of the region 401 and the region 403 are reduced to one tenth of the original areas in the horizontal direction, thereby obtaining a target panoramic planar image fig. 7. As shown in fig. 7, the region 402 near the planar gaze location 400 has high sharpness, and the

regions

401 and 403 distant from the planar gaze location 400 have low sharpness. The processing of fig. 7 is 40% of the area of the original fig. 6, and the whole data capacity is reduced, so that the transmission on the network can greatly save the network bandwidth.

It should be understood that the first image area is closer to the planar gaze location than the second image area on the panoramic planar image. There are various ways to divide the first image area and the second image area, for example, as shown in fig. 6, the first image area surrounds the planar gaze position, and the two second image areas are respectively located at the left and right sides of the first image area. In other embodiments, the second image areas may be respectively located at upper and lower sides of the first image area. Or the first image area and the second image area are arranged in a nine-square grid, the first image area is positioned in the center, the plane fixation position is positioned in the middle of the first image area, and the second image area is positioned at the periphery of the nine-square grid and arranged around the first image area.

In the embodiment of the present invention, there are various implementations of the distribution of the sharpness and the data capacity of the target panoramic plane image, for example, the sharpness of the target panoramic plane image decreases in a linear or stepwise manner along the target direction. Along the target direction, the data capacity of the target panoramic plane image decreases in a linear or stepwise fashion. Wherein the target direction is a direction pointing from the planar gaze position to outside of the target panoramic planar image on the target panoramic planar image.

In the embodiment of the invention, on the target panoramic plane image, the definition and the data capacity of the same area are in direct proportion, in other words, the data capacity is large if the definition is high; otherwise, the definition is low, and the data capacity is small.

The target direction is a direction pointing from the planar gaze position to the outside of the target panoramic planar image on the target panoramic planar image. Since the first image area is closer to the planar gaze position than the second image area, in some embodiments the target direction is a direction pointing from the planar gaze position to the first image area and the second image area in that order.

In the embodiment of the present invention, the processing device may reduce the pixels of the second image region by reducing the area of the second image region, thereby achieving a reduction in the data capacity of the second image region. At this time, on the target panoramic planar image, the first image region is closer to the planar gaze position than the second image region, and the first image region has a larger area than the second image region. In the embodiment of the invention, the processing device projects the panoramic spherical image onto a plane to obtain a panoramic plane image. The target panoramic planar image is obtained by processing the panoramic planar image. Therefore, the target panoramic plane image can be obtained according to the panoramic spherical image, and the panoramic fixation position on the panoramic spherical image corresponds to the plane fixation position on the target panoramic plane image. In this case, the area of the panoramic spherical image on the target panoramic plane image is reduced after the image is subjected to projection and reduction processing as the area is farther from the panoramic gaze position. On the contrary, the area of the panoramic spherical image closer to the panoramic fixation position is larger than the area of the target panoramic plane image after the image is subjected to projection and reduction processing.

Step 309: the processing device sends the target panoramic planar image to the playing device.

And the processing equipment reduces the data capacity of a second image area on the panoramic planar image to obtain a target panoramic planar image, and then sends the target panoramic planar image to the playing equipment so that the playing equipment displays the target panoramic planar image.

In a specific transfer operation, the processing device may perform encoding processing on the target panoramic planar image, for example, processing into an H264 format, and then, the processing device transmits the encoded target panoramic planar image to the playback device. Since the data capacity of the target panoramic planar image is reduced relative to the data capacity of the panoramic planar image, the computational pressure of the processing device on the encoding processing of the target panoramic planar image is reduced.

Step 310: and the playing equipment projects the target panoramic plane image to obtain a target panoramic image.

After the playing device acquires the target panoramic planar image sent by the processing device, the playing device can project the target panoramic planar image to obtain the target panoramic image.

As described above, the target panoramic planar image is obtained by reducing the data capacity of the second image area on the panoramic planar image, the panoramic planar image includes the planar gaze position, the first image area, and the second image area, the planar gaze position is a position on which the user gazes, and the first image area is closer to the planar gaze position than the second image area. On the target panoramic plane image, the first image area has higher definition than the second image area, and the data capacity of the first image area is larger than that of the second image area.

The target panoramic planar image is a panoramic image represented in a planar manner, and for a user to view conveniently, the target panoramic planar image needs to be projected to obtain a target panoramic image in a stereoscopic space. There are various specific projection manners, for example, projecting the target panoramic planar image onto a spherical surface, a cylindrical side surface, or a cube, and the like, which is not limited in this embodiment of the present invention.

And the target panoramic plane image comprises a plane fixation position, and the plane fixation position is a position for a user to fix. When the target panoramic plane image is projected, the plane fixation position is projected to the target panoramic image, and when a user watches the target panoramic image, the sight of the user directly faces the projection position of the plane fixation position on the target panoramic image. After the target panoramic image is obtained through projection, the first image area is still closer to the plane fixation position than the second image area on the target panoramic image. The first image area has a higher resolution than the second image area, and the first image area has a larger data capacity than the second image area.

It should be understood that, after the playing device acquires the target panoramic planar image sent by the processing device, if the target panoramic planar image is subjected to encoding processing, the playing device performs decoding processing on the target panoramic planar image, because the method of the embodiment of the present invention reduces the data capacity of the target panoramic planar image, the decoding calculation pressure of the playing device is low.

Step 311: and the playing equipment displays the target panoramic image.

And after the playing equipment acquires the displayable target panoramic image, the playing equipment displays the target panoramic image for the user to watch.

The playback device presents the target panoramic image in a variety of ways, such as on a screen on the playback device.

As described above, when the user views the target panoramic image, the line of sight of the user is directly facing the projection position of the planar gaze position on the target panoramic image. And the first image region is closer to the planar gaze location than the second image region. The first image area is of higher definition than the second image area, so that the user mainly views the first image area, which is of higher definition. And a second image region, which is further from the user's field of view than the first image region, is less sharp, thus having less impact on the user's viewing experience. The area of the target panoramic image within the visual field of the user is clearer.

For example, the panoramic camera sends the target panoramic plane image shown in fig. 7 to VR glasses. When the VR glasses enlarge the area 401 and the area 403 of the target panoramic planar image shown in fig. 7 in the horizontal direction to the area shown in fig. 6, the data capacity of the area 401 and the area 403 is reduced, and the area 401 and the area 403 are enlarged and have low sharpness. And then, projecting the amplified target panoramic plane image onto a panoramic spherical surface by VR glasses to obtain a target panoramic image. Then, VR glasses show the target panoramic image. When a user watches a target panoramic image by using VR glasses, the projection position of the plane watching position 400 on the target panoramic plane image is the position mainly watched by the user, and the definition is higher near the projection position of the plane watching position 400, namely the position of the area 402, and the area 402 is the position mainly watched by the user. The regions of the projected position, i.e., region 401 and region 403, that are far from the planar gaze position 400 are less sharp, but the user's field of view is far from these regions, so that the less

sharp regions

401 and 403 have less of an impact on the user's viewing experience.

In the embodiment of the invention, when the target panoramic plane image is displayed, the first image area close to the gazing position of the user is clearer, the definition of the image when the user watches the image is kept, and the second image area far away from the gazing position of the user has low definition but small data capacity, so that the whole data capacity of the image is reduced.

And adjusting the size and shape of the first image area and the second image area according to the visual field range of the user when the user watches on the playing device, or dividing more areas of the target panoramic plane image so as to adjust the definition and the data capacity of the areas according to the distance relative to the plane fixation position. Therefore, when a user watches the displayed target panoramic plane image on the playing equipment, the definition of the image watched by the user is similar to that of the traditional panoramic image playing mode within the visual field range watched by the user. But outside the viewing field of the user, i.e. the place on the displayed target panoramic plane image which cannot be viewed by the user, the definition is low and the data capacity is small. Therefore, the overall data capacity of the panoramic image is reduced while the user experience of watching definition is not influenced, so that the transmission bandwidth of the panoramic image is reduced, and the pressure of a playing end on decoding the panoramic image is also reduced.

It should be understood that, in the embodiment of the present invention, there may be a plurality of image areas on the target panoramic plane image, and the relationship between the definition and the data capacity between adjacent ones of the image areas may be the same as the relationship between the definition and the data capacity between the first image area and the second image area.

The panoramic data processing method of the embodiment of the invention can transmit a single panoramic image between the playing device and the processing device, and can also transmit the viewing direction data of the playing end and the target panoramic plane image in real time, so that the panoramic video can be displayed on the playing device, and the panoramic live broadcast can be realized. Therefore, the panoramic data processing method provided by the embodiment of the invention can be applied to one-to-one live system scenes for realizing remote control of the robot or remote command through equipment such as a panoramic camera.

Fig. 8 is a schematic structural diagram of a processing device according to an embodiment of the present invention, where the processing device is configured to execute the method executed by the processing device according to the above embodiment.

Referring to fig. 8, the processing apparatus of the embodiment of the present invention includes:

an obtaining module 801, configured to obtain a panoramic planar image, where the panoramic planar image includes a planar gaze location, a first image area, and a second image area, where the planar gaze location is a location at which a user gazes, and the first image area is closer to the planar gaze location than the second image area;

a processing module 802, configured to reduce a data capacity of a second image area on the panoramic planar image to obtain a target panoramic planar image, where, on the target panoramic planar image, a first image area has a higher definition than the second image area, and the data capacity of the first image area is larger than the data capacity of the second image area.

Optionally, the acquisition module 801 includes an acquisition unit 803 and a projection unit 804;

an obtaining unit 803, configured to obtain a panoramic spherical image;

and the projection unit 804 is used for projecting the panoramic spherical image onto a plane to obtain a panoramic plane image.

Optionally, the processing device further comprises a direction obtaining module 805 and a determining module 806;

a direction obtaining module 805, configured to obtain target viewing direction data, where the target viewing direction data is used to indicate a target viewing direction;

the determining module 806 is configured to use a sphere center of the panoramic spherical image as an origin, extend the target viewing direction outwards according to the target viewing direction data, and determine an intersection point of the target viewing direction and the panoramic spherical image as a panoramic gaze position, where the planar gaze position is obtained when the panoramic gaze position is projected onto a plane.

Optionally, the obtaining unit 803 is further configured to collect the panoramic spherical image through a camera on the processing device, where an optical center of the camera of the processing device is a spherical center of the panoramic spherical image;

the processing device further comprises a target acquisition module 807;

a target obtaining module 807, configured to obtain playing end viewing direction data sent by the playing device, where the playing end viewing direction data is used to indicate a viewing direction indicated in a coordinate system where the playing device is located, and obtain an attitude angle when the processing device collects the panoramic spherical image;

the direction obtaining module 805 is further configured to calculate, according to the viewing direction data and the attitude angle of the playing end, target viewing direction data, where the target viewing direction data and the panoramic spherical image are both represented in a coordinate system where the processing device is located;

the processing device further comprises a sending module 808;

a sending module 808, configured to send the target panoramic planar image to the playback device, so that the playback device displays the target panoramic planar image.

Optionally, the projection unit 804 is further configured to project the panoramic spherical image onto a plane by using an equidistant rectangular projection manner, so as to obtain a panoramic plane image;

the processing module 802 is further configured to reduce pixels of a second image area on the panoramic planar image to obtain a target panoramic planar image, where the first image area has more pixels than the second image area on the target panoramic planar image.

Optionally, along the target direction, the sharpness of the target panoramic planar image decreases in a linear or stepped manner;

wherein the target direction is a direction pointing from the planar gaze position to outside of the target panoramic planar image on the target panoramic planar image.

In summary, in the processing device of the embodiment of the present invention, the obtaining module 801 obtains a panoramic plane image, where the panoramic plane image includes a plane gaze location, a first image area, and a second image area, where the plane gaze location is a location at which a user gazes, and the first image area is closer to the plane gaze location than the second image area. The processing module 802 reduces the data capacity of the second image area on the panoramic planar image to obtain the target panoramic planar image. On the target panoramic plane image, the first image area has higher definition than the second image area, and the data capacity of the first image area is larger than that of the second image area. Therefore, the plane watching position is the position watched by the user, so that when the target panoramic plane image is displayed, the first image area close to the position watched by the user is clearer, the definition of watching the image is kept, the second image area far away from the position watched by the user is lower in definition but small in data capacity, and the integral data capacity of the image is reduced.

Fig. 9 is a schematic structural diagram of a playing device according to an embodiment of the present invention, where the playing device is configured to execute the method executed by the playing device according to the foregoing embodiment.

Referring to fig. 9, the playback device according to the embodiment of the present invention includes:

an obtaining module 901, configured to obtain a target panoramic planar image sent by a processing device, where the target panoramic planar image is obtained by reducing data capacity of a second image area on the panoramic planar image, the panoramic planar image includes a planar gaze position, a first image area, and a second image area, the planar gaze position is a position for a user to gaze, and the first image area is closer to the planar gaze position than the second image area; on the target panoramic plane image, the definition of a first image area is higher than that of a second image area, and the data capacity of the first image area is larger than that of the second image area;

a projection module 902, configured to project the target panoramic planar image to obtain a target panoramic image;

and a display module 903 for displaying the target panoramic image.

Optionally, the playing device further includes a direction obtaining module 904 and a sending module 905;

a direction obtaining module 904, configured to obtain viewing direction data of a playing end, where the viewing direction data of the playing end is used to indicate a viewing direction represented in a coordinate system where the playing device is located;

the sending module 905 is configured to send the data of the viewing direction of the playing end to the processing device, so that the processing device determines the planar gaze position according to the data of the viewing direction of the playing end.

In summary, in the playback device according to the embodiment of the present invention, the obtaining module 901 obtains the target panoramic plane image sent by the processing device. The target panoramic plane image is obtained by reducing the data capacity of a second image area on the panoramic plane image, the panoramic plane image comprises a plane fixation position, a first image area and a second image area, the plane fixation position is a position for a user to fix, and the first image area is closer to the plane fixation position than the second image area; on the target panoramic plane image, the first image area has higher definition than the second image area, and the data capacity of the first image area is larger than that of the second image area. The projection module 902 projects the target panoramic planar image to obtain a target panoramic image. Then, the presentation module 903 presents the target panoramic image. In the target panoramic plane image, the plane fixation position is the position for the user to fix, so that a first image area close to the position for the user to fix is clearer, the definition of the image to be fixed is kept, a second image area far away from the position for the user to fix is lower in definition but small in data capacity, and the integral data capacity of the image is reduced.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims

1. A panoramic data processing method, comprising:

2. The panoramic data processing method according to claim 1,

the processing device acquires a panoramic plane image, and comprises:

the processing equipment acquires a panoramic spherical image;

3. The panoramic data processing method according to claim 2,

the method further comprises the following steps:

4. The panoramic data processing method according to claim 3,

the processing equipment acquires the panoramic spherical image and comprises:

the method further comprises the following steps:

the processing device acquires target viewing direction data, and comprises:

5. The panoramic data processing method according to claim 2,

the processing device projects the panoramic spherical image onto a plane to obtain a panoramic plane image, and the method comprises the following steps:

6. The panoramic data processing method according to claim 1,

along the target direction, the definition of the target panoramic plane image is reduced in a linear or step-shaped manner;

7. A panoramic data processing method, comprising:

and the playing equipment displays the target panoramic image.

8. The panoramic data processing method according to claim 7,

before the playing device acquires the target panoramic plane image sent by the processing device, the method further includes:

9. A processing device, comprising:

10. A playback device, comprising:

and the display module is used for displaying the target panoramic image.