WO2024093763A1

WO2024093763A1 - Panoramic image processing method and apparatus, computer device, medium and program product

Info

Publication number: WO2024093763A1
Application number: PCT/CN2023/126583
Authority: WO
Inventors: 张君培
Original assignee: 影石创新科技股份有限公司
Priority date: 2022-11-01
Filing date: 2023-10-25
Publication date: 2024-05-10
Also published as: CN115760551A

Abstract

The present application relates to a panoramic image processing method and apparatus, a computer device, a medium and a program product. The method comprises: acquiring observation viewing angles at which a panoramic image is observed at a plurality of preset moments, respectively; determining a first viewing angle change range according to a plurality of observation viewing angles; determining an actual viewing angle sampling range of the panoramic image according to a blur intensity parameter and the first viewing angle change range; in the actual viewing angle sampling range, sequentially extracting N observation viewing angles, and obtaining from the panoramic image N observed images corresponding to the N observation viewing angles, wherein N is a positive integer; and performing calculation on the basis of the N observed images, so as to obtain a target blurred image. By using the method, the blurring effect of adding an afterimage to an image, and the bionic human vision effect can be realized.

Description

Panoramic image processing method, device, computer equipment, medium and program product

Technical Field

The present application relates to the field of image processing technology, and in particular to a panoramic image processing method, device, computer equipment, medium and program product.

Background technique

When observing the landscape with the human eye, due to the phenomenon of persistence of vision, the electrochemical phenomenon of the retina will cause the vision to have a certain reaction time. This means that when the human eye sees a picture at a certain moment, the picture will not disappear from the brain's vision for a short period of time; therefore, in the process of rapid changes in the human eye's perspective, the scenery seen by the human eye is often blurred and has afterimages.

For any panoramic image, the viewing angle is constantly changing during the camera movement, which causes the observed plane image to constantly change as well. Each change in the plane image is equivalent to a momentary scene seen by the human eye.

However, when general camera movement editing is performed on panoramic materials, each frame in the camera movement process is very clear, and the resulting edited image is also very clear, which is inconsistent with the real human eye vision. Therefore, how to make the resulting edited image have a blur effect with an additional afterimage and a visual effect that mimics the human eye has become a technical problem that technicians in this field need to solve urgently.

Summary of the invention

Based on this, it is necessary to provide a panoramic image processing method, device, computer equipment, medium and program product that can add a blur effect of residual image and a visual effect of bionic human eye to the image to address the above technical problems.

In a first aspect, the present application provides a panoramic image processing method. The method comprises:

Obtaining observation angles of the panoramic image at multiple preset moments;

Determining a first viewing angle variation range according to the plurality of viewing angles;

Determining an actual viewing angle sampling range of the panoramic image according to a blur intensity parameter and the first viewing angle variation range;

In the actual viewing angle sampling range, N observation viewing angles are sequentially extracted, and N observation images corresponding to the N observation viewing angles in the panoramic image are obtained; wherein N is a positive integer;

A target blurred image is obtained by calculation based on the N observed images.

In one embodiment, the first viewing angle change range is a viewing angle change range between a first moment and a target moment and/or a viewing angle change range between the target moment and a second moment; wherein the first moment, the target moment and the second moment are moments observed sequentially in the preset moments, and the time interval between the first moment and the target moment is equal to the time interval between the target moment and the second moment;

The step of determining the actual viewing angle sampling range of the panoramic image according to the blur intensity parameter and the first viewing angle variation range includes:

Among the multiple observation angles, marking the target observation angle corresponding to the target moment as a second sampling angle;

Calculate a first sampling angle of view according to the target observation angle of view, the first observation angle of view corresponding to the first moment, and the blur intensity parameter;

Calculate a third sampling angle of view according to the target observation angle of view, the second observation angle of view corresponding to the second moment, and the blur intensity parameter;

The perspective change range from the first sampling perspective to the second sampling perspective and/or the perspective change range from the second sampling perspective to the third sampling perspective is marked as the actual perspective sampling range.

In one embodiment, the actual viewing angle sampling range is a viewing angle variation range from the first sampling viewing angle to the second sampling viewing angle and a viewing angle variation range from the second sampling viewing angle to the third sampling viewing angle;

The step of sequentially extracting the N observation viewing angles in the actual viewing angle sampling range and obtaining N observation images corresponding to the N observation viewing angles in the panoramic image comprises:

In the range of the viewing angle change from the first sampling viewing angle to the second sampling viewing angle, n1 observation viewing angles are uniformly selected in sequence; wherein n1=N/2, and n1 is a positive integer;

In the range of the viewing angle variation from the second sampling viewing angle to the third sampling viewing angle, n2 observation viewing angles are uniformly extracted in sequence; wherein n2=N/2, and n2 is a positive integer;

Combine n1 observation perspectives and n2 observation perspectives to form N observation perspectives. The panoramic image is sampled from observation perspectives to obtain N observation images.

In one embodiment, the step of calculating the blurred target image based on the N observed images includes:

Setting a corresponding weight for each of the observed images;

A weighted summation process is performed according to the pixel values of each observed image and the corresponding weights to obtain the target blurred image.

In one embodiment, setting a corresponding weight for each of the observed images includes:

According to the extraction order of the observation images, the initial value of each observation image is set; wherein the initial value of each observation image is increased in sequence according to the extraction order;

Each of the initial values is normalized to obtain a weight corresponding to each of the observed images.

In one embodiment, performing weighted summation processing according to the pixel values of each observed image and the corresponding weights to obtain the target blurred image includes:

Obtaining an initial pixel value of each pixel in each of the observed images;

The initial pixel value of each pixel point in each observation image and the corresponding weight are weighted summed to obtain the target pixel value of each pixel point in the target blurred image.

In a second aspect, the present application also provides a panoramic image processing device. The device comprises:

An observation angle acquisition module is used to acquire observation angles for observing the panoramic image at multiple preset moments;

A first viewing angle variation range determining module, configured to determine a first viewing angle variation range according to the plurality of viewing angles;

An actual viewing angle sampling range determining module, used to determine an actual viewing angle sampling range of the panoramic image according to a blur intensity parameter and the first viewing angle variation range;

An observation image acquisition module, used to sequentially extract N observation viewing angles in the actual viewing angle sampling range, and obtain N observation images corresponding to the N observation viewing angles in the panoramic image; wherein N is a positive integer;

The target blurred image calculation module is used to calculate the target blurred image according to the N observation images.

In a third aspect, the present application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor implementing the steps of the embodiment of the first aspect when executing the computer program.

In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the embodiment of the first aspect are implemented.

In a fifth aspect, the present application further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the embodiment of the first aspect are implemented.

The above-mentioned panoramic image processing method, device, computer equipment, medium and program product obtain the observation angles of the panoramic image at multiple preset moments, and then determine the first angle variation range according to the observation angle, and then determine the actual angle sampling range of the panoramic image according to the blur intensity parameter and the first angle variation range, and then sequentially extract N observation angles in the actual angle sampling range, and obtain N observation images corresponding to the N observation angles in the panoramic image, and finally calculate the target blurred image according to the N observation images. The technical solution of the present application determines the first angle variation range according to the observation angle to simulate the change of the human eye's observation angle, and then sequentially extracts N observation angles in the actual angle sampling range, and obtains N observation images corresponding to the N observation angles in the panoramic image, so as to facilitate the calculation of the target blurred image according to the N observation images, so as to generate a blurred image observed by simulating the human eye's observation angle during the change process, thereby achieving the blur effect of adding afterimages to the clipped image, and adding the visual effect of bionic human eyes.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a diagram showing an application environment of a panoramic image processing method according to an embodiment;

FIG2 is a schematic diagram of a flow chart of a panoramic image processing method in one embodiment;

FIG3 is a schematic diagram of a flow chart of calculating a first viewing angle variation range and a second viewing angle variation range in one embodiment;

FIG4 is a schematic diagram of a flow chart of calculating an actual viewing angle sampling range in one embodiment;

FIG5 is a schematic diagram of a process for calculating a blurred target image in one embodiment;

FIG6 is a schematic flow chart of a panoramic image processing method in another embodiment;

FIG7 is a structural block diagram of a panoramic image processing device in one embodiment;

FIG. 8 is a diagram showing the internal structure of a computer device in one embodiment.

Detailed ways

In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

The panoramic image processing method provided in the embodiment of the present application can be applied in the application environment shown in FIG. 1. The terminal 102 communicates with the server 104 through a network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on a cloud or other network server. The terminal 102 obtains the observation angles of the panoramic image at multiple preset moments, and then determines the first angle variation range according to the multiple observation angles; then determines the actual angle sampling range of the panoramic image according to the blur intensity parameter and the first angle variation range, and then sequentially extracts N observation angles in the actual angle sampling range, and obtains N observation images corresponding to the N observation angles in the panoramic image, and finally calculates the target blurred image according to the N observation images. The terminal 102 can be various electronic devices with shooting and image processing functions such as personal computers, laptops, smart phones, tablet computers, cameras, etc. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

In some embodiments, as shown in FIG. 2 , a panoramic image processing method is provided, which is described by taking the method applied to the terminal 102 in FIG. 1 as an example, including the following steps:

Step 202: Obtain observation angles of the panoramic image at a plurality of preset moments.

The panoramic image may be a directly input image or a video frame in a panoramic video. It should be noted that if the panoramic image is a video frame in a panoramic video, each video frame in the panoramic video is processed using the panoramic image processing method involved in this embodiment.

It should be noted that the panoramic image can be an image stitched together from multiple sub-plane images, or can be taken by an electronic device with shooting capabilities, and this application does not impose specific restrictions on this. For example, a panoramic image can be taken by a shooting device with front and rear dual fisheye lenses.

The preset time may refer to a pre-set time.

The observation angle can refer to the panoramic image that the observer can observe when observing the panoramic image. The angle formed by the light rays drawn from the two ends of the panorama at the optical center of the observer's eyes. The smaller the size of the panorama image and the farther the distance from the observer, the smaller the viewing angle. It should be understood that at different times, the viewing angle of the panorama image of the observer is equal in value, but the specific images observed by the observer may be different.

For example, through a specific mapping relationship, the pixel information in the panoramic image can be first converted to a spherical surface of a three-dimensional coordinate system, and the observer is at the center of the sphere. At any time, the observer can only observe a part of the content on the sphere, so the angle formed by the light rays from both ends of the part at the center of the observer's eyes is the observation angle.

When an image whose camera movement process is known and completed within a time sequence needs to be processed, the observation angle of the image at each moment in the time sequence is known. That is, in this embodiment, the observation angle of the panoramic image at each moment is known, that is, the observation angle of the panoramic image at multiple preset moments is known.

In some embodiments, the panoramic image may be an image pre-stored in a server or other storage device. For example, the panoramic image is an image pre-stored in a server, and the panoramic image stored in the server may be obtained through a network or other communication methods, and the observation angles of the panoramic image at multiple preset moments may be obtained.

In some embodiments, the panoramic image may be an image captured in real time by an electronic device with a shooting capability. For example, a camera is used to perform panoramic shooting of a target object to form a panoramic image, and the captured panoramic image is transmitted to a terminal (the terminal may be a personal computer, a laptop, etc.) through a network or other communication methods. The terminal receives the captured panoramic image and obtains the shooting angle of the camera shooting the target object at each time, thereby obtaining the observation angles for observing the panoramic image at multiple preset times.

Step 204: determine a first viewing angle variation range according to multiple viewing angles.

The first viewing angle variation range may refer to a viewing angle variation range of an observer observing the panoramic image at a plurality of preset moments.

Exemplarily, the sequential change of multiple viewing angles in time sequence may be taken as the first viewing angle change range.

For example, suppose there are five preset moments, which occur in sequence and are named a, b, c, d, and e. The observation angle corresponding to moment a to the observation angle corresponding to moment e can be constructed as The range of the first perspective.

In one embodiment, it is assumed that there are three preset moments, which occur sequentially in time sequence and are named as moment a, moment b and moment c, respectively, wherein moment a occurs before moments b and c in time sequence, moment b occurs between moments a and c in time sequence, and moment c occurs after moments a and b in time sequence. Each preset moment corresponds to an observation angle for observing the panoramic image, and the first angle variation range may be the angle variation range between moment a and moment b and the angle variation range between moment b and moment c.

The technical solution of the embodiment of the present application can sense the movement state and direction of the lens by determining the range of change of the first perspective according to the sequential changes of the observation perspective, thereby simulating the observation of the human eye, so that the subsequently generated blurred image is more in line with the actual observation effect of the human eye, thereby improving the accuracy of the generated bionic human eye visual effect.

In one embodiment, the first viewing angle change range can be the viewing angle change range between the first moment and the target moment and/or the viewing angle change range between the target moment and the second moment; wherein the first moment, the target moment, and the second moment are moments observed sequentially in the preset moments, and the time interval between the first moment and the target moment is equal to the time interval between the target moment and the second moment.

For example, it is assumed that among multiple preset moments, the time intervals between the moments are equal, there are two moments a1 and a2 in sequence before the target moment, and there are two moments b1 and b2 in sequence after the target moment.

When the first moment is moment a1, the second moment is moment b1, and the time interval between moment a1 and the target moment is equal to the time interval between the target moment and moment b1. Then the first viewing angle range can be the viewing angle change range between moment a1 and the target moment and the viewing angle change range between the target moment and moment b1.

When the first moment is moment a2, the second moment is moment b2, and the time interval between moment a2 and the target moment is equal to the time interval between the target moment and moment b2. Then the first viewing angle range can be the viewing angle change range between moment a2 and the target moment and the viewing angle change range between the target moment and moment b2, wherein the time interval between moment a2 and the target moment is greater than the time interval between moment a1 and the target moment.

The technical solution of the embodiment of the present application sets the time interval between the first moment and the target moment to be equal to the time interval between the target moment and the second moment, so that the first viewing angle can be changed within a certain range. It can change with the selection of the first moment and the second moment, so as to facilitate changing the value of the first viewing angle variation range, and then facilitate changing the accuracy of the subsequent actual viewing angle sampling range, thereby improving the adaptability of the panoramic image processing method.

In one embodiment, referring to FIG. 3 , step 204 includes but is not limited to the following steps:

Step 302: Determine, from among multiple observation angles, a first observation angle corresponding to a first moment, a target observation angle corresponding to a target moment, and a second observation angle corresponding to a second moment.

In some embodiments, the target observation angle corresponding to the target moment is represented by _At , the first observation angle corresponding to the first moment can be represented by At _-1 , and the second observation angle corresponding to the second moment can be represented by At ₊₁ .

When the observation angle at each moment in the time sequence is known, the first observation angle corresponding to the first moment is obtained according to the first moment and the known observation angle. Similarly, the target observation angle corresponding to the target moment and the second observation angle corresponding to the second moment can be obtained.

Step 304: Determine a second viewing angle variation range according to the first viewing angle and the target viewing angle.

The second viewing angle variation range can be represented by the first observation viewing angle and the target observation viewing angle. For example, the second viewing angle variation range can be represented by: At _-1 → _At , that is, the viewing angle variation range from the first moment to the target moment can be represented by: At _-1 → _At .

In some embodiments, after the first observation angle of view and the target observation angle of view are determined, the observer's angle of view change range can be used as the second angle of view change range during the process of converting the first observation angle of view to the target observation angle of view.

Step 306: Determine a third viewing angle variation range according to the target viewing angle and the second viewing angle.

The third viewing angle variation range can be represented by the target viewing angle and the second viewing angle. For example, the third viewing angle variation range can be represented by: _At → At ₊₁ , that is, the viewing angle variation range between the target moment and the second moment can be represented by: _At → At ₊₁ .

In some embodiments, after the target observation perspective and the second observation perspective are determined, the observer's perspective change range in the process of switching from the target observation perspective to the second observation perspective can be used as the third perspective change range, and the first perspective change range can be the perspective change range after the second perspective change range and the third perspective change range are combined.

In particular, when the target time is the initial time (that is, the target time is the 0th time), The corresponding first viewing angle change range is the viewing angle change range from the target moment to the second moment. When the target moment is the end moment (ie, the target moment is the last moment in the time sequence), the corresponding first viewing angle change range is the viewing angle change range from the first moment to the target moment.

For example, if the observer rotates parallel to the object at a uniform angular velocity, the first viewing angle variation range can be obtained as follows:

Obtain the observation angle of view corresponding to the observer at the first moment, the observation angle corresponding to the target moment, and the observation angle at the second moment, then the first angle of view change range can be: obtain the first time difference between the target moment and the first moment, when the first time difference is large, multiply the first time difference by the angular velocity of rotation to obtain the angle of rotation, and add the rotation angle to the observation angle corresponding to the first moment to obtain the second angle of view change range; then obtain the second time difference between the target moment and the second moment, when the second time difference is large, multiply the second time difference by the angular velocity of rotation to obtain the angle of rotation, and add the rotation angle to the observation angle corresponding to the target moment to obtain the third angle of view change range; after determining the second angle of view change range and the third angle of view change range, the first angle of view change range can be determined.

Step 206: Determine the actual viewing angle sampling range of the panoramic image according to the blur intensity parameter and the first viewing angle variation range.

Among them, the blur intensity parameter may refer to a sampling intensity parameter for sampling a panoramic image. The blur intensity parameter is a pre-set adjustable parameter. The blur intensity parameter may be pre-set by a user or automatically set by a processor. The specific value of the blur intensity parameter may be set according to the specific situation. This application does not impose any specific restrictions on this. The blur intensity parameter may be represented by K, where K∈[0,1]. When the blur intensity parameter K is closer to 0, the corresponding actual viewing angle sampling range is smaller; when the blur intensity parameter is closer to 1, the corresponding actual viewing angle sampling range is larger.

The actual viewing angle sampling range may refer to a range of viewing angle sampling for a panoramic image.

In some embodiments, the actual viewing angle sampling range can be controlled by controlling the value of the blur strength parameter.

For example, the second viewing angle variation range and the third viewing angle variation range are combined, and then the blur intensity parameter value is taken as 0.5, then the actual viewing angle sampling range is half of the sum of the second viewing angle variation range and the third viewing angle variation range, that is, the actual viewing angle sampling range is half of the first viewing angle variation range. In particular, When the second viewing angle variation range and the third viewing angle variation range are equal in value, the actual viewing angle sampling range is equal in value to the second viewing angle variation range, and the actual viewing angle sampling range can be obtained by taking half of the second viewing angle variation range and half of the third viewing angle variation range. It should be understood that other methods can also be used to obtain the actual viewing angle sampling range, and this application does not make specific restrictions on this.

Step 208, sequentially extracting N observation viewing angles in the actual viewing angle sampling range, and obtaining N observation images corresponding to the N observation viewing angles in the panoramic image; wherein N is a positive integer.

The observed image may refer to an image obtained by observing the panoramic image from an observation perspective by an observer.

In the actual viewing angle sampling range obtained above, N observation viewing angles are sequentially extracted from the panoramic image, and the observation image corresponding to each observation viewing angle in the panoramic image is obtained to obtain N observation images.

For example, an unequal sampling method may be adopted to sequentially and unequally extract N observation viewing angles from the panoramic image within the actual viewing angle sampling range to obtain N observation images. For example, a first viewing angle is randomly extracted from the panoramic image, and then a second viewing angle is extracted from the panoramic image after the first viewing angle at a randomly obtained sampling interval.

Exemplarily, an equal sampling method may be adopted to equally extract N observation viewing angles for the panoramic image within the actual viewing angle sampling range, and then obtain N observation images corresponding to each observation viewing angle in the panoramic image.

Step 210, calculating and obtaining a blurred target image based on the N observed images.

The target blurred image may refer to an image obtained by processing the panoramic image and adding an afterimage blur effect and a bionic human eye perspective effect.

In some embodiments, the target blurred image can be obtained by performing weighted summation on N observed images.

In the above-mentioned panoramic image processing method, the first perspective change range is determined according to the observation perspective to simulate the change of the human eye's observation perspective, and then N observation perspectives are sequentially extracted in the actual perspective sampling range, and N observation images corresponding to the N observation perspectives in the panoramic image are obtained, so as to facilitate the calculation of the target blurred image based on the N observation images to generate a blurred image observed by simulating the human eye's observation perspective during the change process, thereby achieving the blur effect of adding afterimages to the clipped image and adding the visual effect of the bionic human eye.

In some embodiments, as shown in FIG. 4 , step 206 includes but is not limited to the following steps:

Step 402: Among multiple observation perspectives, mark the target observation perspective corresponding to the target moment as a second sampling perspective.

Wherein, B ₁ can be used to represent the second sampling angle of view, then B ₁ =A _t , that is, the observation angle of view corresponding to the target moment is taken as the second sampling angle of view corresponding to the target moment.

Step 404, calculating a first sampling viewing angle according to the target observation viewing angle, the first observation viewing angle corresponding to the first moment and the blur intensity parameter.

Among them, B ₀ can be used to represent the first sampling angle of view, so the first sampling angle of view B ₀ can be represented by the following formula (1), and the formula (1) is specifically:
B ₀ = _At -( _At -At _-1 )*K (1)

Wherein, K in formula (1) represents the blur intensity parameter, _At represents the target observation angle corresponding to the target moment, and At _-1 represents the first observation angle corresponding to the first moment.

Substituting the preset blur intensity parameter, target observation angle and first observation angle into formula (1), the first sampling angle can be calculated.

Step 406, calculating a third sampling angle of view according to the target observation angle of view, the second observation angle of view corresponding to the second moment and the blur intensity parameter.

In some embodiments, the third sampling angle of view may be represented by B ₂ , and the third sampling angle of view B ₂ may be represented by the following formula (2), where formula (2) is specifically:
_B2 = _At + ( _{At + 1} - _At ) * K (2)

In formula (2), K represents the blur intensity parameter, _At represents the target observation angle corresponding to the target moment, and At ₊₁ represents the second observation angle corresponding to the second moment.

Substituting the preset blur intensity parameter, target observation angle and second observation angle into formula (2), the third sampling angle can be calculated.

Step 408: Mark the viewing angle change range from the first sampling viewing angle to the second sampling viewing angle and/or the viewing angle change range from the second sampling viewing angle to the third sampling viewing angle as the actual viewing angle sampling range.

In some embodiments, the actual perspective sampling range can be the perspective change range from the first sampling perspective to the second sampling perspective, or the perspective change range from the second sampling perspective to the third sampling perspective, or the perspective change range from the first sampling perspective to the second sampling perspective and the perspective change range from the second sampling perspective to the third sampling perspective.

For example, when the actual viewing angle sampling range is the viewing angle change range from the first sampling viewing angle to the second sampling viewing angle and the viewing angle change range from the second sampling viewing angle to the third sampling viewing angle, the actual viewing angle sampling range can be expressed as B ₀ →B ₁ →B _2. Among them, B ₀ →B ₁ represents the change from the first sampling viewing angle to the second sampling viewing angle; B ₁ →B ₂ represents the change from the second sampling viewing angle to the third sampling viewing angle.

In this embodiment, the actual viewing angle sampling range can be controlled by controlling the first viewing angle variation range and the blur intensity parameter.

In one embodiment, when the target moment is the initial moment (i.e., the target moment is the 0th moment), the corresponding first perspective change range is the perspective change range from the target moment to the second moment, and the actual perspective sampling range is the perspective change range from the second sampling perspective to the third sampling perspective. When the target moment is the end moment (i.e., the target moment is the last moment in the sequence), the corresponding first perspective change range is the perspective change range from the first moment to the target moment, and the actual perspective sampling range is the perspective change range from the first sampling perspective to the third sampling perspective.

In some embodiments, the actual viewing angle sampling range is the viewing angle variation range from the first sampling viewing angle to the second sampling viewing angle and the viewing angle variation range from the second sampling viewing angle to the third sampling viewing angle. Step 208 includes but is not limited to the following steps: within the viewing angle variation range from the first sampling viewing angle to the second sampling viewing angle, n1 observation viewing angles are uniformly extracted in sequence; wherein n1=N/2, n1 is a positive integer; within the viewing angle variation range from the second sampling viewing angle to the third sampling viewing angle, n2 observation viewing angles are uniformly extracted in sequence; wherein n2=N/2, n2 is a positive integer; n1 observation viewing angles and n2 observation viewing angles are combined to form N observation viewing angles, and the panoramic image is sampled according to the extracted N observation viewing angles to obtain N observation images.

Exemplarily, an even sampling method may be adopted to perform even sampling within the viewing angle variation range from the first sampling viewing angle to the second sampling viewing angle to obtain n1 observation viewing angles.

For example, the aforementioned _B0 can be used to represent the first sampling perspective, _B1 can be used to represent the second sampling perspective, and _B2 can be used to represent the third sampling perspective. Then, within the perspective variation range of _B0 → _B1 , n1 perspectives are equally sampled, which can be represented by the following formula (3):

In formula (3), n1 represents the number of extracted viewing angles, n1=N/2, n1 is an integer, _Ci represents the viewing angle extracted within the viewing angle variation range of _B0 → _B1 , i=1,2,…,n1.

Similarly, within the viewing angle variation range of B ₁ →B ₂ , n2 viewing angles are equally extracted, which can be expressed by the following formula (4). Formula (4) is specifically:

In formula (4), n2 represents the number of extracted viewing angles, n2=N/2, n2 is an integer, _Dj represents the viewing angle extracted within the viewing angle variation range of _B1 → _B2 , j=1,2,…,n2.

For example, when N=30, 15 temporally coherent observation viewing angles are sequentially extracted within the viewing angle variation range of B ₀ →B ₁ , and 15 temporally coherent observation viewing angles are sequentially extracted within the viewing angle variation range of B ₁ →B ₂ .

In summary, in the actual viewing angle sampling range B ₀ →B ₁ →B ₂ , N observation viewing angles that are coherent in time sequence are extracted, and then, the panoramic image is sampled according to the extracted N observation viewing angles to obtain an observation image corresponding to each observation viewing angle.

In one embodiment, when the actual viewing angle sampling range is the viewing angle variation range from the first sampling viewing angle to the second sampling viewing angle or the viewing angle variation range from the second sampling viewing angle to the third sampling viewing angle, N observation viewing angles can be directly extracted from the actual viewing angle sampling range.

As shown in FIG. 5 , in some embodiments, step 210 includes but is not limited to the following steps:

Step 502: Set corresponding weights for each observed image.

Step 504, performing weighted sum processing according to the pixel values of each observed image and the corresponding weights to obtain a target blurred image.

Exemplarily, a weight is set for each of the N observation images. For example, W ₁ represents the weight corresponding to the first observation image in the time sequence, W ₂ represents the weight corresponding to the second observation image in the time sequence, and so on, W _N represents the weight corresponding to the Nth observation image in the time sequence.

It should be noted that the sum of the weights is set to 1, that is Where i=1,2…,N.

After determining the weight corresponding to each observation image, a weighted summation process is performed according to the pixel value of each observation image and the corresponding weight to obtain the target blurred image.

In some embodiments, step 502 includes but is not limited to the following steps: setting the initial value of each observation image according to the extraction order of the observation images; wherein the initial value of each observation image increases in the extraction order; and normalizing each initial value to obtain the weight corresponding to each observation image.

In this embodiment, by setting increasing weights, the proportion of observation images corresponding to observation angles that are earlier in the time sequence is smaller, and the proportion of observation images corresponding to observation angles that are later in the time sequence is larger, so that the camera blur effect has a strong sense of direction, allowing the user to feel the specific movement method, thereby improving the user experience.

For example, let the initial value _Wi = i, and then normalize _Wi to obtain a weight that increases monotonically in time series. That is, let _W1 = 1, _W2 = 2, ..., _WN = N, and then normalize each initial value to obtain a weight that increases monotonically in time series, and then perform weighted summation based on the pixel values of each observed image and the normalized weight to obtain the target blurred image.

In some embodiments, step 504 includes but is not limited to the following steps: obtaining the initial pixel value corresponding to each pixel point in each observed image; performing weighted summation processing on the initial pixel value of each pixel point in each observed image and the corresponding weight to obtain the target pixel value of each pixel point in the target blurred image.

The target blurred image may be composed of a number of pixels. After the pixel value of each pixel is determined, the target blurred image is determined.

First, the initial pixel value corresponding to each pixel point in each observation image is obtained, and then the initial pixel value corresponding to each observation image is weighted using the weight to obtain the weighted pixel value, and then the sum of the weighted pixel values of the pixel points at the same position in each observation image is obtained to obtain the target pixel value. After determining the target pixel value corresponding to each pixel point of the observation image, the target blurred image is obtained.

For example, the final target blurred image is represented by I, and the target pixel value of the first row and first column pixel in the target blurred image I is represented by I ₁₁ , then I ₁₁ can be calculated by the following formula (5), which is specifically:

In formula (5), a _i represents the initial pixel value in the first row and first column of the i-th observed image among N observed images.

Similarly, the target pixel values of other pixels in the target blurred image are calculated in the same manner as in formula (5) to obtain the final target blurred image.

In some embodiments, as shown in FIG6 , the panoramic image processing method includes but is not limited to the following steps:

Step 602: Obtain observation angles of the panoramic image at a plurality of preset moments.

Step 604, determining a first perspective change range based on multiple observation perspectives; the first perspective change range is a perspective change range between a first moment and a target moment and/or a perspective change range between a target moment and a second moment; wherein the first moment, the target moment and the second moment are moments that are observed sequentially in preset moments, and the time interval between the first moment and the target moment is equal to the time interval between the target moment and the second moment.

Step 606: Among the multiple observation perspectives, mark the target observation perspective corresponding to the target moment as the second sampling perspective.

Step 608, calculating a first sampling viewing angle according to the target observation viewing angle, the first observation viewing angle corresponding to the first moment and the blur intensity parameter.

Step 610, calculating a third sampling angle of view according to the target observation angle of view, the second observation angle of view corresponding to the second moment and the blur intensity parameter.

Step 612: Mark the viewing angle change range from the first sampling viewing angle to the second sampling viewing angle and/or the viewing angle change range from the second sampling viewing angle to the third sampling viewing angle as the actual viewing angle sampling range.

Step 614, within the range of the viewing angle change from the first sampling viewing angle to the second sampling viewing angle, n1 observation viewing angles are uniformly sampled in sequence; wherein n1=N/2, and n1 is a positive integer.

Step 616, within the range of the viewing angle variation from the second sampling viewing angle to the third sampling viewing angle, n2 observation viewing angles are equally selected in sequence; wherein n2=N/2, and n2 is a positive integer.

Step 618: n1 observation angles and n2 observation angles are combined to form N observation angles, and the panoramic image is sampled according to the extracted N observation angles to obtain N observation images.

Step 620, setting the initial value of each observation image according to the extraction order of the observation images; wherein the initial value of each observation image increases in sequence according to the extraction order.

Step 622, normalize each initial value to obtain the weight corresponding to each observed image.

Step 624, performing weighted summation processing according to the pixel values of each observed image and the corresponding weights to obtain a target blurred image.

It should be noted that the embodiments of steps 602 to 624 may refer to the aforementioned specific steps.

It should be understood that although the steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps. Furthermore, at least a portion of the steps in the flowcharts of the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but may be performed at different times, and the execution order of these steps or stages is not necessarily sequential, but may be performed in turn or alternately with other steps or at least a portion of the steps or stages in other steps.

Based on the same inventive concept, an embodiment of the present application also provides a panoramic image processing device for implementing the panoramic image processing method involved above.

In some embodiments, as shown in FIG. 7 , a panoramic image processing device is provided, including: an observation viewing angle acquisition module 702, a first viewing angle variation range determination module 704, an actual viewing angle sampling range determination module 706, an observation image acquisition module 708, and a target blurred image calculation module 710, wherein:

The observation angle acquisition module 702 is used to acquire the observation angles for observing the panoramic image at multiple preset moments.

The first viewing angle variation range determining module 704 is configured to determine the first viewing angle variation range according to multiple viewing angles.

The actual viewing angle sampling range determining module 706 is used to determine the actual viewing angle sampling range of the panoramic image according to the blur intensity parameter and the first viewing angle variation range.

The observation image acquisition module 708 is used to sequentially extract N observation viewing angles in the actual viewing angle sampling range, and obtain N observation images corresponding to the N observation viewing angles in the panoramic image; wherein N is a positive integer.

The target blurred image calculation module 710 is used to calculate the target blurred image according to the N observation images.

In some embodiments, the first viewing angle change range is the viewing angle change range between the first moment and the target moment and/or the viewing angle change range between the target moment and the second moment; wherein the first moment, the target moment and the second moment are moments observed sequentially in the preset moments, and the time interval between the first moment and the target moment is equal to the time interval between the target moment and the second moment.

The actual viewing angle sampling range determination module 706 includes:

The second sampling viewing angle determining unit is used to mark the target observation viewing angle corresponding to the target moment as the second sampling viewing angle among multiple observation viewing angles.

The first sampling viewing angle calculation unit is used to calculate the first sampling viewing angle according to the target observation viewing angle, the first observation viewing angle corresponding to the first moment and the blur intensity parameter.

The third sampling viewing angle calculation unit is used to calculate the third sampling viewing angle according to the target observation viewing angle, the second observation viewing angle corresponding to the second moment and the blur intensity parameter.

The first marking unit is used to mark the viewing angle change range from the first sampling viewing angle to the second sampling viewing angle and/or the viewing angle change range from the second sampling viewing angle to the third sampling viewing angle as the actual viewing angle sampling range.

In some embodiments, the actual viewing angle sampling range is the viewing angle variation range from the first sampling viewing angle to the second sampling viewing angle and the viewing angle variation range from the second sampling viewing angle to the third sampling viewing angle. The observation image acquisition module 708 includes:

The first viewing angle extraction unit is used to equally extract n1 observation viewing angles in sequence within the viewing angle variation range from the first sampling viewing angle to the second sampling viewing angle; wherein n1=N/2, and n1 is a positive integer.

The second viewing angle extraction unit is used to equally extract n2 observation viewing angles in sequence within the viewing angle variation range from the second sampling viewing angle to the third sampling viewing angle; wherein n2=N/2, and n2 is a positive integer.

The image sampling unit is used to combine n1 observation viewing angles and n2 observation viewing angles to form N observation viewing angles, and to sample the panoramic image according to the extracted N observation viewing angles to obtain N observation images.

In some embodiments, the target blurred image calculation module 710 includes:

The weight setting unit is used to set a corresponding weight for each observation image.

The weighted summation unit is used to perform weighted summation processing according to the pixel values of each observation image and the corresponding weights to obtain a target blurred image.

In some embodiments, the weight setting unit includes:

The initial value setting subunit is used to set the initial value of each observation image according to the extraction order of the observation images; wherein the initial value of each observation image increases in sequence according to the extraction order.

The normalization processing subunit is used to normalize each initial value to obtain the weight corresponding to each observed image.

In some embodiments, the weighted summation unit comprises:

The initial pixel value acquisition subunit is used to obtain the initial pixel value corresponding to each pixel point in each observed image.

The weighted processing subunit is used to calculate the initial pixel value and the corresponding weight of each pixel in each observation image. The target pixel value of each pixel in the target blurred image is obtained by performing weighted summation processing.

Each module in the panoramic image processing device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

In some embodiments, a computer device is provided, which may be a terminal, and its internal structure diagram may be shown in FIG8. The computer device includes a processor, a memory, a communication interface, a display unit, and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a panoramic image processing method is implemented. The display unit of the computer device may be a liquid crystal display or an electronic ink display, and the input device of the computer device may be a touch layer covered on the display unit, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

Those skilled in the art will understand that the structure shown in FIG. 8 is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

In some embodiments, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of the above-mentioned panoramic image processing method when executing the computer program.

In some embodiments, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the panoramic image processing method are implemented.

In some embodiments, a computer program product is provided, including a computer program, which implements the steps of the above-mentioned panoramic image processing method when executed by a processor.

A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchain, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, etc., but are not limited to this.

The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

A panoramic image processing method, characterized in that the method comprises:

Obtaining observation angles of the panoramic image at multiple preset moments;

Determining a first viewing angle variation range according to the plurality of viewing angles;

Determining an actual viewing angle sampling range of the panoramic image according to a blur intensity parameter and the first viewing angle variation range;

In the actual viewing angle sampling range, N observation viewing angles are sequentially extracted, and N observation images corresponding to the N observation viewing angles in the panoramic image are obtained; wherein N is a positive integer;

A target blurred image is obtained by calculation based on the N observed images.
The method according to claim 1 is characterized in that the first viewing angle change range is the viewing angle change range between a first moment and a target moment and/or the viewing angle change range between the target moment and a second moment; wherein the first moment, the target moment and the second moment are moments observed sequentially in the preset moments, and the time interval between the first moment and the target moment is equal to the time interval between the target moment and the second moment;

The step of determining the actual viewing angle sampling range of the panoramic image according to the blur intensity parameter and the first viewing angle variation range includes:

Among the multiple observation angles, marking the target observation angle corresponding to the target moment as a second sampling angle;

Calculate a first sampling angle of view according to the target observation angle of view, the first observation angle of view corresponding to the first moment, and the blur intensity parameter;

Calculate a third sampling angle of view according to the target observation angle of view, the second observation angle of view corresponding to the second moment, and the blur intensity parameter;

The perspective change range from the first sampling perspective to the second sampling perspective and/or the perspective change range from the second sampling perspective to the third sampling perspective is marked as the actual perspective sampling range.
The method according to claim 2, characterized in that the actual viewing angle sampling range is a viewing angle change range from the first sampling viewing angle to the second sampling viewing angle and a viewing angle change range from the second sampling viewing angle to the third sampling viewing angle;

The step of sequentially extracting the N observation viewing angles in the actual viewing angle sampling range and obtaining N observation images corresponding to the N observation viewing angles in the panoramic image comprises:

In the range of the viewing angle change from the first sampling viewing angle to the second sampling viewing angle, n1 observation viewing angles are uniformly selected in sequence; wherein n1=N/2, and n1 is a positive integer;

In the range of the viewing angle variation from the second sampling viewing angle to the third sampling viewing angle, n2 observation viewing angles are uniformly extracted in sequence, where n2=N/2, and n2 is a positive integer;

The n1 observation viewing angles and the n2 observation viewing angles are combined to form N observation viewing angles, and the panoramic image is sampled according to the extracted N observation viewing angles to obtain N observation images.
The method according to any one of claims 1 to 3, characterized in that the step of calculating the target blurred image based on the N observed images comprises:

Setting a corresponding weight for each of the observed images;

A weighted summation process is performed according to the pixel values of each observed image and the corresponding weights to obtain the target blurred image.
The method according to claim 4, characterized in that the step of setting a corresponding weight for each of the observed images comprises:

According to the extraction order of the observation images, the initial value of each observation image is set; wherein the initial value of each observation image is increased in sequence according to the extraction order;

Each of the initial values is normalized to obtain a weight corresponding to each of the observed images.
The method according to claim 4 is characterized in that the step of performing weighted summation processing according to the pixel values of each observed image and the corresponding weights to obtain the target blurred image comprises:

Obtaining an initial pixel value of each pixel in each of the observed images;

The initial pixel value of each pixel point in each observation image and the corresponding weight are weighted summed to obtain the target pixel value of each pixel point in the target blurred image.
A panoramic image processing device, characterized in that the device comprises:

An observation angle acquisition module is used to acquire observation angles for observing the panoramic image at multiple preset moments;

A first viewing angle variation range determining module, configured to determine a first viewing angle variation range according to the plurality of viewing angles;

An actual viewing angle sampling range determining module, used to determine an actual viewing angle sampling range of the panoramic image according to a blur intensity parameter and the first viewing angle variation range;

An observation image acquisition module, used to sequentially extract N observation viewing angles in the actual viewing angle sampling range, and obtain N observation images corresponding to the N observation viewing angles in the panoramic image; wherein N is a positive integer;

The target blurred image calculation module is used to calculate the target blurred image according to the N observation images.
A computer device comprises a memory and a processor, wherein the memory stores a computer program, and wherein the processor implements the steps of any one of the methods of claims 1 to 6 when executing the computer program.
A computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 6 are implemented.
A computer program product, comprising a computer program, characterized in that when the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 6 are implemented.