WO2019096323A1 - Fisheye image processing method and apparatus, and electronic device - Google Patents

Abstract

Embodiments of the present application provide a fisheye image processing method and apparatus, and an electronic device. The method comprises: mapping a fisheye image onto a three-dimensional cylindrical surface to obtain a three-dimensional cylindrical image; and after receiving a cylindrical surface unfolding instruction, mapping the three-dimensional cylindrical image to obtain an image of an unfolded surface. In the present solution, if a user wants to see an image area at a back viewing angle, the image of the unfolded surface can be seen simply by sending a cylindrical surface unfolding instruction without the need for manually rotating a three-dimensional cylinder, the image of the unfolded surface comprising the image area at a large viewing angle, and therefore, the image display effect is improved.

Description

Fisheye image processing method, device and electronic device

The present application claims priority to Chinese Patent Application No. 200911156272.0, entitled "A Fisheye Image Processing Method, Apparatus, and Electronic Device", filed on November 20, 2017, the entire contents of which are incorporated by reference. In this application.

Technical field

The present application relates to the field of image processing technologies, and in particular, to a fisheye image processing method, apparatus, and electronic device.

Background technique

At present, fisheye images have been widely used due to their large viewing angles. At present, the solution for processing the fisheye image generally includes: presetting the mapping relationship between the fisheye image coordinate system and the three-dimensional hemispherical coordinate system, and mapping the acquired fisheye image onto the three-dimensional hemispherical surface according to the mapping relationship, A three-dimensional image with a three-dimensional effect, such a three-dimensional hemispherical image has image deformation, and the visual effect is poor.

In the above solution, after the three-dimensional image is obtained, only the three-dimensional image area of the front view of the three-dimensional hemisphere can be displayed to the user. If the user wants to see the image area of the back view of the three-dimensional hemisphere, the three-dimensional hemisphere can only be manually rotated, and the display effect is better. difference.

Summary of the invention

An object of the embodiments of the present application is to provide a fisheye image processing method, apparatus, and electronic device to improve visual effects and display effects.

To achieve the above objective, an embodiment of the present application provides a fisheye image processing method, including:

Obtaining a fisheye image to be processed;

According to a mapping relationship between a preset fisheye image coordinate point and a three-dimensional cylindrical coordinate point, the fisheye image to be processed is mapped onto the three-dimensional cylindrical surface to obtain a three-dimensional cylindrical image;

After receiving the cylindrical surface expansion command, the three-dimensional cylindrical image is mapped according to a mapping relationship between the preset three-dimensional cylindrical coordinate point and the developed surface image coordinate point to obtain a developed surface image.

Optionally, after receiving the cylindrical surface expansion instruction, mapping the three-dimensional cylindrical image to the expansion surface according to a mapping relationship between the preset three-dimensional cylindrical coordinate point and the expansion surface image coordinate point, and obtaining the expansion surface image Can include:

After receiving the cylindrical surface expansion instruction, determining an expansion duration, and a first mapping relationship between the target expansion surface image and the three-dimensional cylindrical image;

Determining, according to the expansion duration and the first mapping relationship, a second mapping relationship between the expansion surface image and the three-dimensional cylindrical image at each time before reaching the expansion duration;

For each time before the expansion time is reached, the three-dimensional cylindrical image is mapped according to the second mapping relationship corresponding to the time, to obtain the time expansion surface image;

When the expansion time is reached, the three-dimensional cylindrical image is mapped according to the first mapping relationship to obtain the target expansion surface image.

Optionally, after receiving the cylindrical surface expansion instruction, the three-dimensional cylindrical image is mapped according to a mapping relationship between the preset three-dimensional cylindrical coordinate point and the developed surface image coordinate point, and the expanded surface image is obtained, which may include :

After receiving the cylindrical expansion command, start the timer and determine the expansion time;

Determining a time value stored in the timer at the current moment;

Determining whether the stored time value is less than the expansion time;

If the value is smaller than the third mapping relationship corresponding to the stored time value, the three-dimensional cylindrical image is mapped to obtain an expanded surface image corresponding to the stored time value; wherein, each of the expansion time is greater than or equal to 0 The time values respectively correspond to a third mapping relationship, and a third mapping relationship is: a mapping relationship between the three-dimensional cylindrical coordinate points and a coordinate value of the expanded curved image corresponding to a time value;

If it is greater than or equal to, the three-dimensional cylindrical image is mapped according to a mapping relationship between the preset three-dimensional cylindrical coordinate point and the developed planar image coordinate point, and an expanded planar image is obtained.

Optionally, the coordinate value of the three-dimensional cylindrical coordinate point includes: a first coordinate value, a second coordinate value, and a third coordinate value, where coordinate values of the expanded surface image coordinate point corresponding to the one time value include: fourth coordinate a value, a fifth coordinate value, and a sixth coordinate value;

Mapping the three-dimensional cylindrical image according to the third mapping relationship corresponding to the stored time value may include:

Calculating a ratio of the time value to the expansion time for each stored time value;

Calculating the mapped fourth coordinate value according to the first coordinate value, the arc length between the first coordinate value and the preset reference point, and the ratio; wherein the arc length is according to the viewpoint angle and the Determining the radius of the three-dimensional cylinder;

Determining the second coordinate value as the fifth coordinate value to be mapped;

The mapped sixth coordinate value is calculated according to the third coordinate value, the radius of the three-dimensional cylinder, and the ratio.

Optionally, the calculating the mapped fourth coordinate value according to the first coordinate value, the arc length between the first coordinate value and the preset reference point, and the ratio may include:

The fourth coordinate value to be mapped = the first coordinate value + the ratio * [sign (first coordinate value) * the arc length between the first coordinate value and the preset reference point - the first coordinate value];

An arc length between the first coordinate value and a preset reference point = a radius of the three-dimensional cylinder * (π - viewpoint angle);

The viewpoint angle = tan ^-1 (absolute value of the first coordinate value / third coordinate value);

The calculating the mapped sixth coordinate value according to the third coordinate value, the radius of the three-dimensional cylinder, and the ratio may include:

The sixth coordinate value to be mapped = the third coordinate value + the ratio * (- the radius of the three-dimensional cylinder - the third coordinate value).

Optionally, the coordinate value of the three-dimensional cylindrical coordinate point includes: a first coordinate value, a second coordinate value, and a third coordinate value, where the coordinate value of the developed planar image coordinate point includes: a seventh coordinate value, an eighth coordinate Value and ninth coordinate value;

And mapping the three-dimensional cylindrical image according to the mapping relationship between the preset three-dimensional cylindrical coordinate point and the expanded planar image coordinate point, and obtaining the expanded planar image, which may include:

Calculating the mapped seventh coordinate value according to the first coordinate value and an arc length between the first coordinate value and the preset reference point; wherein the arc length is according to the viewpoint angle and the three-dimensional cylinder Radius determination;

Determining the second coordinate value as the eighth coordinate value to be mapped;

The inverse of the radius of the three-dimensional cylinder is determined as the ninth coordinate value to which the mapping is made.

Optionally, the calculating the mapped seventh coordinate value according to the first coordinate value and the arc length between the first coordinate value and the preset reference point may include:

a seventh coordinate value=sign (first coordinate value)* an arc length between the first coordinate value and the preset reference point;

An arc length between the first coordinate value and a preset reference point = a radius of the three-dimensional cylinder * (π - viewpoint angle);

The viewpoint angle = tan ^-1 (absolute value of the first coordinate value / third coordinate value).

Optionally, after the obtaining the three-dimensional cylindrical image, the method further includes:

Receiving a rotation cylinder command sent by the user; determining a rotation angle corresponding to the rotation cylinder instruction; rotating the three-dimensional cylinder image by the rotation angle by using a center line of the three-dimensional cylindrical image as an axis;

or,

Receiving a viewpoint movement instruction sent by the user; determining a viewpoint angle and a depth of field corresponding to the viewpoint movement instruction; and adjusting the three-dimensional cylindrical image according to the determined viewpoint angle and depth of field.

Optionally, after the obtaining the three-dimensional cylindrical image, the method further includes:

Receiving a zoom instruction sent by the user;

Determining a target depth of field corresponding to the scaling instruction;

The depth of field of the three-dimensional cylindrical image is adjusted to the target depth of field.

Optionally, after receiving the scaling instruction sent by the user, the method further includes:

Start the timer and determine the zoom duration;

The adjusting the depth of field of the three-dimensional cylindrical image to the target depth of field may include:

Determining a time value stored in the timer at the current moment;

Determining whether the stored time value is less than the scaling time;

If the value is less than, the transition depth of field corresponding to the stored time value is calculated, wherein each time value greater than or equal to 0 and less than the zoom duration corresponds to a transition depth of field, and the transition depth of field is located at the current depth of field of the three-dimensional cylindrical image and the target Between the depths of field; adjusting the depth of field of the three-dimensional cylindrical image to the transition depth of field;

If it is greater than or equal to, the depth of field of the three-dimensional cylindrical image is adjusted to the target depth of field.

Optionally, the calculating the transition depth of field corresponding to the stored time value may include:

Transition Depth of Field = stored time value / said zoom duration * (target depth of field - current depth of field of the 3D cylindrical image) + current depth of field of the 3D cylindrical image.

Optionally, after mapping the three-dimensional cylindrical image to the expansion surface according to the mapping relationship between the preset three-dimensional cylindrical coordinate point and the expansion surface image coordinate point, and obtaining the expansion surface image, the method further includes:

Determining, according to the mapping relationship, that the expanded edge line of the expanded plane image is mapped to a corresponding position in the three-dimensional cylindrical image;

Determining, in the three-dimensional cylindrical image, an area in which the location is located according to a preset area division rule;

Copying the determined area to obtain a duplicate image;

The duplicate image is added to the expanded plane image according to the mapping relationship.

In order to achieve the above objective, an embodiment of the present application further provides a fisheye image processing apparatus, wherein the apparatus includes:

Obtaining a module, configured to obtain a fisheye image to be processed;

a first mapping module, configured to map the image of the fisheye to be processed to a three-dimensional cylindrical surface according to a mapping relationship between a preset fisheye image coordinate point and a three-dimensional cylindrical coordinate point, to obtain a three-dimensional cylindrical image;

The second mapping module is configured to map the three-dimensional cylindrical image according to a mapping relationship between the preset three-dimensional cylindrical coordinate point and the developed surface image coordinate point after receiving the cylindrical surface expansion instruction, to obtain a developed surface image.

Optionally, the second mapping module is specifically configured to:

After receiving the cylindrical surface expansion instruction, determining an expansion duration, and a first mapping relationship between the target expansion surface image and the three-dimensional cylindrical image;

Determining, according to the expansion duration and the first mapping relationship, a second mapping relationship between the expansion surface image and the three-dimensional cylindrical image at each time before reaching the expansion duration;

For each time before the expansion time is reached, the three-dimensional cylindrical image is mapped according to the second mapping relationship corresponding to the time, to obtain the time expansion surface image;

When the expansion time is reached, the three-dimensional cylindrical image is mapped according to the first mapping relationship to obtain the target expansion surface image.

Optionally, the second mapping module may include:

a timing sub-module, configured to start a timer after receiving a cylindrical expansion command, and determine an expansion time;

a first determining submodule, configured to determine a time value stored in the timer at a current moment;

a determining sub-module, configured to determine whether the stored time value is less than the expansion time; if not, triggering the first mapping sub-module, if greater than or equal to, triggering the second mapping sub-module;

a first mapping sub-module, configured to map the three-dimensional cylindrical image according to a third mapping relationship corresponding to the stored time value, to obtain an expanded surface image corresponding to the stored time value; wherein, greater than or equal to 0 and less than Each time value of the expansion time length corresponds to a third mapping relationship, and a third mapping relationship is: a mapping relationship between the coordinate points of the three-dimensional cylindrical coordinate points and the expanded surface image corresponding to a time value;

The second mapping sub-module is configured to map the three-dimensional cylindrical image according to a mapping relationship between the preset three-dimensional cylindrical coordinate points and the developed planar image coordinate points, to obtain an expanded planar image.

Optionally, the coordinate value of the three-dimensional cylindrical coordinate point includes: a first coordinate value, a second coordinate value, and a third coordinate value, where coordinate values of the expanded surface image coordinate point corresponding to the one time value include: a fourth coordinate a value, a fifth coordinate value, and a sixth coordinate value;

The first mapping submodule may be specifically configured to:

Calculating a ratio of the time value to the expansion time for each stored time value;

Calculating the mapped fourth coordinate value according to the first coordinate value, the arc length between the first coordinate value and the preset reference point, and the ratio; wherein the arc length is according to the viewpoint angle and the Determining the radius of the three-dimensional cylinder;

Determining the second coordinate value as the fifth coordinate value to be mapped;

The mapped sixth coordinate value is calculated according to the third coordinate value, the radius of the three-dimensional cylinder, and the ratio.

Optionally, the first mapping submodule is specifically configured to:

Calculating a ratio of the time value to the expansion time for each stored time value;

Calculate using the following formula:

The fourth coordinate value to be mapped = the first coordinate value + the ratio * [sign (first coordinate value) * the arc length between the first coordinate value and the preset reference point - the first coordinate value];

An arc length between the first coordinate value and a preset reference point = a radius of the three-dimensional cylinder * (π - viewpoint angle);

The viewpoint angle = tan ^-1 (absolute value of the first coordinate value / third coordinate value);

Determining the second coordinate value as the fifth coordinate value to be mapped;

The sixth coordinate value to be mapped = the third coordinate value + the ratio * (- the radius of the three-dimensional cylinder - the third coordinate value).

Optionally, the coordinate value of the three-dimensional cylindrical coordinate point includes: a first coordinate value, a second coordinate value, and a third coordinate value, where the coordinate value of the developed planar image coordinate point includes: a seventh coordinate value, an eighth coordinate Value and ninth coordinate value;

The second mapping submodule may be specifically configured to:

Calculating the mapped seventh coordinate value according to the first coordinate value and an arc length between the first coordinate value and the preset reference point; wherein the arc length is according to the viewpoint angle and the three-dimensional cylinder Radius determination;

Determining the second coordinate value as the eighth coordinate value to be mapped;

The inverse of the radius of the three-dimensional cylinder is determined as the ninth coordinate value to which the mapping is made.

Optionally, the second mapping sub-module may be specifically configured to: perform calculation by using the following formula:

a seventh coordinate value=sign (first coordinate value)* an arc length between the first coordinate value and the preset reference point;

An arc length between the first coordinate value and a preset reference point = a radius of the three-dimensional cylinder * (π - viewpoint angle);

The viewpoint angle = tan ^-1 (absolute value of the first coordinate value / third coordinate value);

Determining the second coordinate value as the eighth coordinate value to be mapped;

The inverse of the radius of the three-dimensional cylinder is determined as the ninth coordinate value to which the mapping is made.

Optionally, the device further includes: a first receiving module, a first determining module, and a rotating module, or the device further includes: a second receiving module, a second determining module, and a first adjusting module, where

a first receiving module, configured to receive a rotating cylinder command sent by a user;

a first determining module, configured to determine a rotation angle corresponding to the rotating cylinder instruction;

a rotation module, configured to rotate the three-dimensional cylindrical image by the rotation angle by using a center line of the three-dimensional cylindrical image as an axis;

a second receiving module, configured to receive a viewpoint moving instruction sent by the user;

a second determining module, configured to determine a viewpoint angle and a depth of field corresponding to the viewpoint moving instruction;

The first adjustment module is configured to adjust the three-dimensional cylindrical image according to the determined viewpoint angle and depth of field.

Optionally, the device may further include:

a third receiving module, configured to receive a scaling instruction sent by the user;

a third determining module, configured to determine a target depth of field corresponding to the scaling instruction;

And a second adjustment module, configured to adjust a depth of field of the three-dimensional cylindrical image to the target depth of field.

Optionally, the device may further include:

a timing module, configured to start a timer after receiving a zoom instruction sent by the user, and determine a zoom duration;

a fourth determining module, configured to determine a time value stored in the timer at a current moment;

a judging module, configured to determine whether the stored time value is less than the zooming time length; if not, triggering the third adjusting module, if greater than or equal to, triggering the fourth adjusting module;

The third adjustment module includes: a calculation submodule and an adjustment submodule,

a calculation sub-module, configured to calculate a transition depth of field corresponding to the stored time value, wherein each time value greater than or equal to 0 and less than the zoom duration corresponds to a transition depth of field, and the transition depth of field is located at a current depth of field of the three-dimensional cylindrical image Between the target depths of field;

Adjusting a submodule for adjusting a depth of field of the three-dimensional cylindrical image to the transition depth of field;

And a fourth adjustment module, configured to adjust a depth of field of the three-dimensional cylindrical image to the target depth of field.

Optionally, the calculating sub-module may be specifically configured to: perform calculation by using the following formula:

Transition Depth of Field = stored time value / said zoom duration * (target depth of field - current depth of field of the 3D cylindrical image) + current depth of field of the 3D cylindrical image.

Optionally, the device may further include:

a fifth determining module, configured to determine, according to the mapping relationship, that an expanded edge line of the expanded plane image is mapped to a corresponding position in the three-dimensional cylindrical image;

a sixth determining module, configured to determine an area where the location is located in the three-dimensional cylindrical image according to a preset area dividing rule;

a copy module for copying the determined area to obtain a duplicate image;

And a adding module, configured to add the duplicate image to the expanded plane image according to the mapping relationship.

To achieve the above objective, an embodiment of the present application further provides an electronic device, including a processor and a memory;

a memory for storing a computer program;

The processor is configured to implement any of the above-described fisheye image processing methods when executing a program stored on the memory.

In order to achieve the above object, an embodiment of the present application further discloses an executable program code for being executed to execute any of the above-described fisheye image processing methods.

Applying the embodiment of the present application, the fisheye image is mapped onto the three-dimensional cylindrical surface to obtain a three-dimensional cylindrical image, and after receiving the cylindrical surface expansion instruction, the three-dimensional cylindrical image is mapped to obtain a developed surface image; in the present scheme, the image of the three-dimensional cylindrical image is obtained. It is better than the hemisphere image. If you want to see the image area of the back view, you don't need to manually rotate the 3D cylinder. You only need to send a cylindrical expansion command to see the expansion surface image. The expansion surface image contains The image area of the large viewing angle, therefore, improves the display effect of the image.

DRAWINGS

In order to more clearly illustrate the embodiments of the present application and the technical solutions of the prior art, the following description of the embodiments and the drawings used in the prior art will be briefly introduced. Obviously, the drawings in the following description are only Some embodiments of the application may also be used to obtain other figures from those of ordinary skill in the art without departing from the scope of the invention.

FIG. 1 is a schematic flowchart of a fisheye image processing method according to an embodiment of the present application;

2 is a schematic diagram of a fisheye image in the embodiment of the present application;

FIG. 3 is a schematic diagram of coordinate mapping in the embodiment of the present application;

4 is a schematic diagram of a three-dimensional cylindrical image in an embodiment of the present application;

5 is a top view of a three-dimensional cylindrical image expanded into a planar image in the embodiment of the present application;

6 is a schematic diagram of a process of expanding a three-dimensional cylindrical image into a planar image according to an embodiment of the present application;

FIG. 7 is a top view showing another three-dimensional cylindrical image expanded into a planar image in the embodiment of the present application; FIG.

FIG. 8 is a schematic diagram of transformation of a three-dimensional cylindrical image rotated at different angles according to an embodiment of the present application; FIG.

FIG. 9 is a schematic diagram showing transformation of different depth of field of a three-dimensional cylindrical image in an embodiment of the present application; FIG.

FIG. 10 is a schematic structural diagram of a fisheye image processing apparatus according to an embodiment of the present application;

FIG. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed ways

In order to make the objects, technical solutions, and advantages of the present application more comprehensible, the present application will be further described in detail below with reference to the accompanying drawings. It is apparent that the described embodiments are only a part of the embodiments of the present application, and not all of them. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without departing from the inventive scope are the scope of the present application.

In order to solve the above technical problem, the embodiment of the present application provides a fisheye image processing method, device, and electronic device. The method and device can be applied to any electronic device having an image processing function, such as a computer, or a fisheye camera, or a server connected to a fisheye camera, or a handheld device connected to a fisheye camera, etc., which is not limited. .

A fisheye image processing method provided by an embodiment of the present application is described in detail below.

FIG. 1 is a schematic flowchart of a fisheye image processing method according to an embodiment of the present application, including:

S101: Acquire an image of a fisheye to be processed.

S102: According to a mapping relationship between a preset fisheye image coordinate point and a three-dimensional cylindrical coordinate point, the fisheye image to be processed is mapped onto the three-dimensional cylindrical surface to obtain a three-dimensional cylindrical image.

S103: After receiving the cylindrical surface expansion command, the three-dimensional cylindrical image is mapped according to a mapping relationship between the preset three-dimensional cylindrical coordinate point and the developed surface image coordinate point to obtain a developed surface image.

Applying the embodiment shown in FIG. 1 of the present application, the fisheye image is mapped onto the three-dimensional cylindrical surface to obtain a three-dimensional cylindrical image, and after receiving the cylindrical surface expansion instruction, the three-dimensional cylindrical image is mapped to obtain a developed surface image; in the scheme, three-dimensional The image of the cylindrical image is better than the hemisphere image. If you want to see the image area of the back view, you don't need to manually rotate the 3D cylinder. You only need to send the cylindrical expansion command to see the expansion surface image. The image contains an image area with a large viewing angle, thus improving the image display effect.

The embodiment shown in Figure 1 will be described in detail below:

S101: Acquire an image of a fisheye to be processed.

For example, the image to be processed can be as shown in FIG. 2, and FIG. 2 is an image collected by the fisheye camera.

S102: According to a mapping relationship between a preset fisheye image coordinate point and a three-dimensional cylindrical coordinate point, the fisheye image to be processed is mapped onto the three-dimensional cylindrical surface to obtain a three-dimensional cylindrical image.

The fisheye image is a two-dimensional image, and the fisheye image coordinate system is also a two-dimensional coordinate system. For example, as shown in FIG. 3, the fisheye image coordinate system is represented as a uv coordinate system. The three-dimensional cylindrical image is a three-dimensional image, and the three-dimensional cylindrical coordinate system is a three-dimensional coordinate system. As shown in FIG. 3, the three-dimensional cylindrical coordinate system is represented as an xyz coordinate system.

Assume that the coordinate value of a fisheye image coordinate point p in the fisheye image coordinate system is (u, v), and the coordinate value of a three-dimensional cylindrical coordinate point p' in the three-dimensional cylindrical coordinate system is (x, y, z). The height of the three-dimensional cylinder is h, the radius is r, and the line connecting the fisheye image coordinate point p and the center of the cylinder bottom is in the xz plane. The angle between the projection line and the x-axis is α, and the mapping relationship can be:

α=u*2π, x=r*cos(α), y=h*v,z=r*sin(α)

According to the mapping relationship, the fisheye image coordinate points in the fisheye image to be processed are mapped to the three-dimensional cylindrical coordinate points in the three-dimensional cylindrical coordinate system, and a three-dimensional cylindrical image is obtained, and the three-dimensional cylindrical image can be as shown in FIG.

S103: After receiving the cylindrical surface expansion command, the three-dimensional cylindrical image is mapped according to a mapping relationship between the preset three-dimensional cylindrical coordinate point and the developed surface image coordinate point to obtain a developed surface image.

For example, the unfolding surface image may be a planar image, a wavy image, a curved image of a preset curvature, or the like, which is not limited in detail. Intuitively, the three-dimensional cylindrical image is mapped to obtain a developed surface image, that is, the three-dimensional cylindrical image is expanded, and the height of the developed surface image can be the same as the height of the three-dimensional cylindrical image.

Taking the unfolded surface image as a planar image as an example, as shown in FIG. 5, FIG. 5 is a plan view of expanding a three-dimensional cylindrical image into a planar image, and it is assumed that point B in FIG. 5 is a point in the three-dimensional cylindrical image, point B. The point mapped to the plane image is point E, the point O is the reference point for the expansion of the three-dimensional cylindrical image, the coordinates of point B are (x, y, z), and the coordinates of point E are (sign(x)*s, y, -r), s is the arc length of the OB. That is to say, the mapping relationship between the three-dimensional cylindrical coordinate points and the plane image coordinate points is: mapping (x, y, z) to the mapping relationship of (sign(x)*s, y, -r). According to the mapping relationship, the points of the three-dimensional cylindrical image are mapped into the planar image, and a planar image is obtained.

As an implementation manner, an electronic device (executing body, hereinafter referred to as an electronic device) that executes the present solution may include a display screen, or may be communicably connected to a display device; a three-dimensional cylindrical image may be displayed on a display screen or a display device. Expand to the intermediate process of expanding the face image.

In this case, S103 can include:

Step 1: After receiving the cylindrical surface expansion instruction, determine the expansion duration, and the first mapping relationship between the target expansion surface image and the three-dimensional cylindrical image.

The expansion time is the length of time from the receipt of the cylindrical expansion command to the end of the expansion. The expansion time can be preset; or the expansion time can be determined according to the cylindrical expansion command. For example, the execution body is a touch screen device, and the cylindrical surface expansion instruction can be a user gesture, and the expansion time is determined according to the magnitude of the user gesture amplitude; for example, the execution body is a computer, and the cylindrical surface expansion instruction can be an instruction for the user to click the mouse, according to the user click The frequency of the mouse determines the length of the expansion, and so on, and is not limited.

The target expansion surface image is an expansion surface image obtained after the expansion is completed. The shape of the target expansion surface image may be various, such as a planar image, a wavy image, a curved image of a preset curvature, and the like, which are not limited.

For example, the shape of the target expansion surface image may be preset, and the mapping relationship between the three-dimensional cylindrical coordinate point and the target development surface of the preset shape may be calculated. In the embodiment of the present application, a plurality of mapping relationships are mentioned. To distinguish the description, the mapping relationship between the target expansion surface image and the three-dimensional cylindrical image is referred to as a first mapping relationship.

Alternatively, the shape of the target development surface image may be determined according to the cylindrical surface expansion command. For example, the button 1 corresponds to a plane image, and the button 2 corresponds to a wave image. If the user clicks the button 1, the target expansion surface image is a planar image. If the user clicks the button 2, the target expansion surface image is a wave image. For example, it is detected that the user clicks the mouse to indicate that the target expansion surface image is a flat image, and the user is detected that the user clicks on the mouse twice, indicating that the target expansion surface image is a wave image, and so on, and is not enumerated one by one.

If it is determined that the target expanded image is a planar image, the mapping relationship between the three-dimensional cylindrical coordinate point and the planar image is calculated as the first mapping relationship; if the target expanded image is determined to be a wavy image, the three-dimensional cylindrical coordinate point and the wavy shape are calculated. The mapping relationship of the image as the first mapping relationship.

Step 2: Determine, according to the expansion duration and the first mapping relationship, a second mapping relationship between the expansion surface image and the three-dimensional cylindrical image at each time before reaching the expansion duration.

For example, the expansion of the image from the three-dimensional cylindrical image to obtain the target expansion surface image can be a uniform expansion process. In this way, linear interpolation between the three-dimensional cylindrical image and the target expansion surface image may be performed according to the expansion duration, that is, linear interpolation between the three-dimensional cylindrical coordinate point included in the first mapping relationship and the target expansion surface image coordinate point .

Assuming that the target expansion surface image is a planar image, still taking FIG. 5 as an example, the first mapping relationship is: mapping (x, y, z) to the mapping relationship of (sign(x)*s, y, -r) . Assuming that the expansion time is T, linear interpolation between the three-dimensional cylindrical image and the planar image is performed. The second mapping relationship corresponding to each moment in T is: mapping (x, y, z) to {x+t*sign( x) The mapping relationship of *sx, y, t*(-rz)}. Where t is a process quantity in T, or t is the percentage of the expansion process. For example, T is 1 second, and the expansion process has continued for 0.5 seconds at the current moment, then t is 50%.

Step 3: For each time before the expansion time is reached, the three-dimensional cylindrical image is mapped according to the second mapping relationship corresponding to the time, and the time expansion surface image is obtained.

In step 2, a second mapping relationship corresponding to each moment in the T is obtained. For each moment in the T, the three-dimensional cylindrical image is mapped according to the second mapping relationship corresponding to the moment, and each of the Ts is obtained. The expanded face image of the moment, so that is presented to the user is a dynamic expansion process.

Step 3: When the deployment duration is reached, the three-dimensional cylindrical image is mapped according to the first mapping relationship to obtain the target expansion surface image.

After reaching T, the expansion is completed, and the image of the target expansion surface is displayed to the user. Therefore, according to the first mapping relationship, the three-dimensional cylindrical image is mapped to obtain a target expansion surface image.

Alternatively, the "intermediate process of displaying the three-dimensional cylindrical image as a developed surface image" may also be implemented by the following embodiments, and S103 may include:

After receiving the cylindrical expansion command, start the timer and determine the expansion time;

Determining a time value stored in the timer at the current moment;

Determining whether the stored time value is less than the expansion time;

If the value is smaller than the third mapping relationship corresponding to the stored time value, the three-dimensional cylindrical image is mapped to obtain an expanded surface image corresponding to the stored time value; wherein, each of the expansion time is greater than or equal to 0 The time values respectively correspond to a third mapping relationship, and a third mapping relationship is: a mapping relationship between the three-dimensional cylindrical coordinate points and a coordinate value of the expanded curved image corresponding to a time value;

If it is greater than or equal to, the three-dimensional cylindrical image is mapped according to a mapping relationship between the preset three-dimensional cylindrical coordinate point and the developed planar image coordinate point, and an expanded planar image is obtained.

In the present embodiment, a case where the target expanded image is a planar image will be described as an example. As described above, the expansion time is the length of time from the reception of the cylindrical surface expansion command to the end of the expansion. The expansion time can be preset; or the expansion time can be determined according to the cylindrical expansion command.

For convenience of description, the expansion time is denoted by T, and the time value stored in the timer is denoted as T'. That is to say, within the T after receiving the cylindrical surface expansion instruction is the expansion process, and after T is the process of displaying the surface of the target expansion surface. The expansion process is dynamic. The content of the screen at each moment in the expansion process is different. The content of the screen is the expanded surface image, as shown in Figure 6. The presentation process for a flat image is static, only the planar image is shown, but if other instructions are received during the presentation, other instructions are executed.

For example, after receiving the cylindrical expansion command, the timer is started, and the T' value in the timer gradually increases from 0 to 0; if T' is smaller than T, the 3D cylindrical image needs to be mapped to the expanded surface. On the image, different T' values correspond to different mapping relationships, and the expanded surface images at different times are different. In order to distinguish the description, the mapping relationship between the coordinate points of the developed curved surface image corresponding to a T' value of the three-dimensional cylindrical coordinate point is referred to as a third mapping relationship.

It is assumed that the coordinate values of the three-dimensional cylindrical coordinate points include: a first coordinate value, a second coordinate value, and a third coordinate value, and the expanded surface image coordinate points corresponding to the one time value include: a fourth coordinate value and a fifth coordinate value And sixth coordinate value;

And mapping the three-dimensional cylindrical image according to the third mapping relationship corresponding to the ratio, including:

Calculating, for each stored time value, a ratio of the time value to the expansion time, that is, a ratio of T' to T, denoted by t, t is a process quantity in T, or t is an expansion process Percentage

Calculating the mapped fourth coordinate value according to the first coordinate value, the arc length between the first coordinate value and the preset reference point, and the ratio; wherein the arc length is according to the viewpoint angle and the Determining the radius of the three-dimensional cylinder;

Determining the second coordinate value as the fifth coordinate value to be mapped;

The mapped sixth coordinate value is calculated according to the third coordinate value, the radius of the three-dimensional cylinder, and the ratio.

For example, as shown in FIG. 7, FIG. 7 is similar to FIG. 5 except that the point P in the display process is added, the point B is a point in the three-dimensional cylindrical image, and the point B is mapped to the point in the plane image as E. Point, point O is the preset reference point, point P is a point in the process of expanding point B to point E. The coordinate values of point P include: fourth coordinate value, fifth coordinate value and sixth coordinate value; point B The coordinates are (x, y, z), the coordinates of point E are (sign(x)*s, y, -r), and s is the arc length of OB.

In Fig. 7, the first coordinate value is x, the second coordinate value is y, the third coordinate value is z, the coordinate of the three-dimensional cylindrical coordinate point is (x, y, z), and the arc between x and the preset reference point The length of the arc is OB:

In this embodiment, the fourth coordinate value is calculated according to the arc length between x, x and the preset reference point and t, and specifically, the arc of the fourth coordinate value=x+t*[sign(x)*OB Length -x], arc length of OB = r * (π - viewpoint angle), viewpoint angle = tan ^-1 (|x|/z);

During the expansion process, the y value is unchanged, and the fifth coordinate value = y;

The sixth coordinate value is calculated according to z, r, t, and specifically, the sixth coordinate value = z + t * (-r-z).

In this way, the mapping relationship of t at each moment in the interval (0, 1) can be calculated. According to the mapping relationship at each moment, the expanded surface at each moment can be obtained, that is, the three-dimensional cylinder is displayed. The dynamic process of expanding the image to a flat image shows that the effect is good.

If t is greater than or equal to 1, the expansion process is completed, and only the expanded planar image is displayed. Specifically, it is assumed that the coordinate values of the three-dimensional cylindrical coordinate points include: a first coordinate value, a second coordinate value, and a third coordinate value, and the coordinate values of the expanded planar image coordinate point include: a seventh coordinate value, an eighth coordinate value, and Ninth coordinate value;

And mapping the three-dimensional cylindrical image according to a mapping relationship between the preset three-dimensional cylindrical coordinate point and the developed planar image coordinate point, to obtain an expanded planar image, including:

Calculating the mapped seventh coordinate value according to the first coordinate value and an arc length between the first coordinate value and the preset reference point; wherein the arc length is according to the viewpoint angle and the three-dimensional cylinder Radius determination;

Determining the second coordinate value as the eighth coordinate value to be mapped;

The inverse of the radius of the three-dimensional cylinder is determined as the ninth coordinate value to which the mapping is made.

In Fig. 7, the first coordinate value is x, the second coordinate value is y, the third coordinate value is z, the coordinate of the three-dimensional cylindrical coordinate point is (x, y, z), and the arc between x and the preset reference point The arc length of the length OB, the coordinate value of the E point includes: a seventh coordinate value, an eighth coordinate value, and a ninth coordinate value:

In this embodiment, the seventh coordinate value is calculated according to the arc length between x, x and the preset reference point, specifically, the arc length of the seventh coordinate value=sign(x)*OB, and the arc length of the OB= r*(π-viewpoint angle), viewpoint angle=tan ^-1 (|x|/z);

Eightth coordinate value = y;

The ninth coordinate value is the opposite of r, and the ninth coordinate value = -r.

According to the mapping relationship, the points of the three-dimensional cylindrical image are mapped into the developed planar image, and the developed planar image is obtained.

Here is a specific example:

Assuming that the fisheye image to be processed shown in FIG. 2 is acquired, the fisheye image coordinate system is a uv coordinate system. As shown in Fig. 3, the mapping relationship between the fisheye image coordinate points and the three-dimensional cylindrical coordinate points is: α=u*2π, x=r*cos(α), y=h*v, z=r*sin(α) According to the mapping relationship, the fisheye image to be processed is mapped onto the three-dimensional cylindrical surface to obtain a three-dimensional cylindrical image. The coordinate point of the fisheye image to be processed is (u, v), and the coordinate point of the three-dimensional cylindrical image is (x, y, z).

The cylindrical surface expansion command is received, the timer is started, and the time value stored in the timer is recorded as T'. Assume that the target expansion surface image is a planar image, assuming that the expansion time is T.

After receiving the cylindrical surface expansion instruction, determining the time value T′ stored in the timer at each current moment, if T′ is less than T, determining that the third mapping relationship corresponding to T′ is: (x, y, z Mapped to the mapping relationship of {x+t*sign(x)*sx,y,t*(-rz)}; where (x,y,z) is the coordinate point of the 3D cylindrical image, {x+t* Sign(x)*sx,y,t*(-rz)} is the coordinate point of the expanded surface image corresponding to the T', t=T'/T, s=r*(π-α), α=tan ^-1 (|x|/z). According to the third mapping relationship, the three-dimensional cylindrical image is mapped to obtain an expanded curved surface image.

If T' is greater than or equal to T, the determined mapping relationship is a mapping relationship that maps (x, y, z) to (sign(x)*s, y, -r); according to the mapping relationship, the three-dimensional cylindrical image is performed. Map to get the expanded planar image.

In an optional embodiment of the present application, the unfolding process is dynamic, and the content of the screen at each moment in the unfolding process is different, and the content of the screen is an expanded curved image, as shown in FIG. 6. The presentation process for a flat image is static, only the planar image is shown, but if other instructions are received during the presentation, other instructions are executed. For example, if a signal of a double click of the right mouse button is received, a full expansion instruction is generated, and the three-dimensional cylindrical image is expanded into a planar image. In the process of displaying the planar image, upon receiving the restoration instruction, the image is restored to the three-dimensional cylindrical image.

As an implementation manner, after S102 and before S103, the method may further include:

Receiving a rotation cylinder command sent by the user; determining a rotation angle corresponding to the rotation cylinder instruction; and rotating the three-dimensional cylinder image by the rotation angle by using a center line of the three-dimensional cylindrical image as an axis.

For example, the display screen included in the electronic device or the display device connected to the electronic device may be a touch screen, and the user may send a rotating cylinder instruction by making a gesture on the touch screen, for example, if the electronic device detects that the user slides left and right on the touch screen The gesture indicates that the rotary cylinder command was received.

The specific correspondence between the gesture and the command content can be set according to the actual situation. For example, it can be set to slide to the left to rotate clockwise, to the right to rotate counterclockwise, and to set the corresponding relationship between the amplitude of the slide and the angle of rotation. The angle of rotation can be positive or negative, for example, clockwise rotation is positive and counterclockwise rotation is negative.

Assuming that the user's gesture of sliding to the left on the touch screen is detected, and the rotation angle corresponding to the gesture is determined to be 20 degrees clockwise, or +20 degrees, the center line of the three-dimensional cylindrical image is taken as the axis, and the three-dimensional cylindrical image is smoothed. The hour hand turns 20 degrees.

Fig. 8 is a schematic diagram showing the transformation of the three-dimensional cylindrical image at different angles.

As an implementation manner, after S102 and before S103, the method may further include:

Receiving a zoom instruction sent by the user;

Determining a target depth of field corresponding to the scaling instruction;

The depth of field of the three-dimensional cylindrical image is adjusted to the target depth of field.

For example, the display screen included in the electronic device or the display device connected to the electronic device may be a touch screen, and the user may send a zoom instruction by making a gesture on the touch screen, for example, if the electronic device detects that the user pinches on the touch screen The gesture indicates that the zoom-out instruction is received, and if the gesture of the user's finger separation on the touch screen is detected, it indicates that the zoom-in instruction is received.

The image is reduced, that is, the image is far away from the angle of view, the depth of field becomes larger, the image is enlarged, that is, the distance between the image and the angle of view becomes closer, and the depth of field becomes smaller; the specific correspondence between the gesture and the command content can be set according to the actual situation, for example, the finger pinch can be set. The correspondence between the amplitude and the depth of field is large, and the correspondence between the amplitude of the finger separation and the depth of field is small, and so on, and is not limited.

Assuming that the user's gesture of finger pinching on the touch screen is detected and the target depth of field corresponding to the gesture is determined, the depth of field of the three-dimensional cylindrical image is adjusted to the target depth of field. Fig. 9 is a schematic diagram showing the transformation of different depths of a three-dimensional cylindrical image.

As an implementation manner, an intermediate process of adjusting the depth of field may be displayed on a display screen of the electronic device or a display device connected to the electronic device. Specifically, after receiving the zoom instruction sent by the user, the timer may be started, and the zoom is determined. duration;

Adjusting the depth of field of the three-dimensional cylindrical image to the target depth of field includes:

Determining a time value stored in the timer at the current moment;

Determining whether the stored time value is less than the scaling time;

If the value is less than, the transition depth of field corresponding to the stored time value is calculated, wherein each time value greater than or equal to 0 and less than the zoom duration corresponds to a transition depth of field, and the transition depth of field is located at the current depth of field of the three-dimensional cylindrical image and the target Between the depths of field; adjusting the depth of field of the three-dimensional cylindrical image to the transition depth of field;

If it is greater than or equal to, the depth of field of the three-dimensional cylindrical image is adjusted to the target depth of field.

The zoom duration is the length of time from the receipt of the zoom command to the end of the zoom. The zoom duration can be preset; or the zoom duration can be determined based on the zoom command. For example, the execution subject is a touch screen device, and the zoom instruction may be a user gesture, and the zoom duration is determined according to the magnitude of the user gesture amplitude.

Assuming that the zoom duration is 1 second, the depth of field adjustment process is within 1 second after receiving the zoom instruction, and the depth of field in this process is referred to as the transition depth of field, and the transition depth at each moment in the depth of field adjustment process is different.

As an embodiment, the transition depth of field = the stored time value / the zoom duration * (target depth of field - the current depth of field of the three-dimensional cylindrical image) + the current depth of field of the three-dimensional cylindrical image.

In the embodiment, the current depth of field transitions to the target depth of field, the image adjustment process is smoother, and the visual effect is better.

As an implementation manner, after S102 and before S103, the method may further include:

Receiving a viewpoint movement instruction sent by the user;

Determining a viewpoint angle and a depth of field corresponding to the viewpoint movement instruction;

The three-dimensional cylindrical image is adjusted according to the determined viewpoint angle and depth of field.

For example, the display screen included in the electronic device or the display device connected to the electronic device may be a touch screen, and the user may send a viewpoint moving instruction by making a gesture on the touch screen, for example, if the electronic device detects that the user slides up and down in the touch screen. The gesture indicates that the viewpoint movement instruction is received.

After the viewpoint is moved, the depth of field and the angle of the viewpoint usually change. The specific correspondence between the gesture and the command content can be set according to the actual situation. For example, the relationship between the amplitude of the upward sliding and the change of the depth of field and the angle of the viewpoint can be set, and the downward relationship can be set. The corresponding relationship between the amplitude of the sliding and the change of the depth of field and the angle of the viewpoint, etc., is not limited in detail.

According to the correspondence relationship, the viewpoint angle and the depth of field to be reached are determined; the viewpoint angle of the three-dimensional cylindrical image is adjusted to the determined viewpoint angle, and the depth of field of the three-dimensional cylindrical image is adjusted to the determined depth of field.

As an implementation manner, an initial depth of field of the three-dimensional cylindrical image may be set, that is, a depth of field when the three-dimensional cylindrical image is obtained after S102 is obtained, and then after receiving the viewpoint movement instruction or the zoom instruction, the initial Adjust based on the depth of field.

For example, the initial depth of field = (h / 2) / [tan (fov / 2)], fov represents the field of view, the field of view can be set according to the actual situation, for example, can be set between 120 degrees and 180 degrees The value.

In this embodiment, the process of determining the mapping relationship and the process of mapping the image according to the mapping relationship may be performed in a GPU (Graphics Processing Unit), thereby improving the calculation speed and reducing the CPU usage. .

As an implementation manner, after S103, the method may further include:

Determining, according to the mapping relationship, that the expanded edge line of the unfolding plane image is mapped to a corresponding position in the three-dimensional cylindrical image; determining, according to a preset region dividing rule, the region where the location is located Copying the determined area to obtain a duplicate image; adding the copied image to the expanded plane image according to the mapping relationship.

The expanded edge line can be as shown in Fig. 6. It can be understood that if there is a portrait at the position where the expanded edge line is located, the portrait image is split into two parts in the expanded plane image, and the display effect is poor. In the embodiment, the area where the edge line is expanded is copied, and the copied image is added to the expanded surface image. If there is a portrait at the position where the edge line is expanded, the area where the portrait is located is copied, and the image is copied. It is added to the expansion surface image. Specifically, a copy image can be added to each edge position of the expansion surface image, so that both edge positions of the expansion surface image are complete portraits, which improves the display effect.

For example, a contoured image is mapped to a corresponding position in the three-dimensional cylindrical image as a center line, and an image area of a preset size is determined, the area is copied, and the copy area is separately added to the left and right sides of the expanded image.

For another example, determining the position (the expanded edge line is mapped to the corresponding position in the three-dimensional cylindrical image) to the left within the preset distance, copying the area, and adding the copied area to the right edge of the expanded image, determining The position is preset to the right within the distance, the area is copied, and the copied area is added to the left edge of the expanded-surface image.

Corresponding to the above method embodiment, the embodiment of the present application further provides a fisheye image processing apparatus. As shown in FIG. 10, the apparatus includes:

The obtaining module 1001 is configured to obtain a fisheye image to be processed;

The first mapping module 1002 is configured to map the image of the fisheye to be processed onto the three-dimensional cylindrical surface according to a mapping relationship between the preset fisheye image coordinate point and the three-dimensional cylindrical coordinate point, to obtain a three-dimensional cylindrical image;

The second mapping module 1003 is configured to map the three-dimensional cylindrical image according to a mapping relationship between a preset three-dimensional cylindrical coordinate point and a developed surface image coordinate point after receiving the cylindrical surface expansion command, to obtain a developed surface image.

As an implementation manner, the second mapping module 1003 may be specifically configured to:

After receiving the cylindrical surface expansion instruction, determining an expansion duration, and a first mapping relationship between the target expansion surface image and the three-dimensional cylindrical image;

Determining, according to the expansion duration and the first mapping relationship, a second mapping relationship between the expansion surface image and the three-dimensional cylindrical image at each time before reaching the expansion duration;

For each time before the expansion time is reached, the three-dimensional cylindrical image is mapped according to the second mapping relationship corresponding to the time, to obtain the time expansion surface image;

When the expansion time is reached, the three-dimensional cylindrical image is mapped according to the first mapping relationship to obtain the target expansion surface image.

As an embodiment, the second mapping module 1003 may include: a timing submodule, a first determining submodule, a determining submodule, a first mapping submodule, and a second mapping submodule (not shown), where

a timing sub-module, configured to start a timer after receiving a cylindrical expansion command, and determine an expansion time;

a first determining submodule, configured to determine a time value stored in the timer at a current moment;

a determining sub-module, configured to determine whether the stored time value is less than the expansion time; if not, triggering the first mapping sub-module, if greater than or equal to, triggering the second mapping sub-module;

a first mapping sub-module, configured to map the three-dimensional cylindrical image according to a third mapping relationship corresponding to the stored time value, to obtain an expanded surface image corresponding to the stored time value; wherein, greater than or equal to 0 and less than Each time value of the expansion time length corresponds to a third mapping relationship, and a third mapping relationship is: a mapping relationship between the coordinate points of the three-dimensional cylindrical coordinate points and the expanded surface image corresponding to a time value;

The second mapping sub-module is configured to map the three-dimensional cylindrical image according to a mapping relationship between the preset three-dimensional cylindrical coordinate points and the developed planar image coordinate points, to obtain an expanded planar image.

In one embodiment, the coordinate values of the three-dimensional cylindrical coordinate points include: a first coordinate value, a second coordinate value, and a third coordinate value, and the coordinate values of the developed curved surface image coordinate points corresponding to the one time value include: Four coordinate values, fifth coordinate values, and sixth coordinate values;

The first mapping submodule is specifically configured to:

Calculating a ratio of the time value to the expansion time for each stored time value;

Calculating the mapped fourth coordinate value according to the first coordinate value, the arc length between the first coordinate value and the preset reference point, and the ratio; wherein the arc length is according to the viewpoint angle and the Determining the radius of the three-dimensional cylinder;

Determining the second coordinate value as the fifth coordinate value to be mapped;

The mapped sixth coordinate value is calculated according to the third coordinate value, the radius of the three-dimensional cylinder, and the ratio.

As an implementation manner, the first mapping submodule is specifically configured to:

Calculating a ratio of the time value to the expansion time for each stored time value;

Calculate using the following formula:

The fourth coordinate value to be mapped = the first coordinate value + the ratio * [sign (first coordinate value) * the arc length between the first coordinate value and the preset reference point - the first coordinate value];

An arc length between the first coordinate value and a preset reference point = a radius of the three-dimensional cylinder * (π - viewpoint angle);

The viewpoint angle = tan ^-1 (absolute value of the first coordinate value / third coordinate value);

Determining the second coordinate value as the fifth coordinate value to be mapped;

The sixth coordinate value to be mapped = the third coordinate value + the ratio * (- the radius of the three-dimensional cylinder - the third coordinate value).

In one embodiment, the coordinate value of the three-dimensional cylindrical coordinate point includes: a first coordinate value, a second coordinate value, and a third coordinate value, and the coordinate value of the developed planar image coordinate point includes: a seventh coordinate value, Eight coordinate values and ninth coordinate values;

The second mapping submodule may be specifically configured to:

Calculating the mapped seventh coordinate value according to the first coordinate value and an arc length between the first coordinate value and the preset reference point; wherein the arc length is according to the viewpoint angle and the three-dimensional cylinder Radius determination;

Determining the second coordinate value as the eighth coordinate value to be mapped;

The inverse of the radius of the three-dimensional cylinder is determined as the ninth coordinate value to which the mapping is made.

As an implementation manner, the second mapping submodule may be specifically configured to:

Calculate using the following formula:

a seventh coordinate value=sign (first coordinate value)* an arc length between the first coordinate value and the preset reference point;

An arc length between the first coordinate value and a preset reference point = a radius of the three-dimensional cylinder * (π - viewpoint angle);

The viewpoint angle = tan ^-1 (absolute value of the first coordinate value / third coordinate value);

Determining the second coordinate value as the eighth coordinate value to be mapped;

The inverse of the radius of the three-dimensional cylinder is determined as the ninth coordinate value to which the mapping is made.

As an embodiment, the device may further include: a first receiving module, a first determining module, and a rotating module (not shown), wherein

a first receiving module, configured to receive a rotating cylinder command sent by a user;

a first determining module, configured to determine a rotation angle corresponding to the rotating cylinder instruction;

And a rotation module, configured to rotate the three-dimensional cylindrical image by the rotation angle by using a center line of the three-dimensional cylindrical image as an axis.

As an embodiment, the device may further include: a second receiving module, a second determining module, and a first adjusting module (not shown), where

a second receiving module, configured to receive a viewpoint moving instruction sent by the user;

a second determining module, configured to determine a viewpoint angle and a depth of field corresponding to the viewpoint moving instruction;

The first adjustment module is configured to adjust the three-dimensional cylindrical image according to the determined viewpoint angle and depth of field.

As an embodiment, the device may further include: a third receiving module, a third determining module, and a second adjusting module (not shown), where

a third receiving module, configured to receive a scaling instruction sent by the user;

a third determining module, configured to determine a target depth of field corresponding to the scaling instruction;

And a second adjustment module, configured to adjust a depth of field of the three-dimensional cylindrical image to the target depth of field.

As an embodiment, the device may further include: a timing module, a fourth determining module, a determining module, a third adjusting module, and a fourth adjusting module (not shown), where

a timing module, configured to start a timer after receiving a zoom instruction sent by the user, and determine a zoom duration;

a fourth determining module, configured to determine a time value stored in the timer at a current moment;

a judging module, configured to determine whether the stored time value is less than the zooming time length; if not, triggering the third adjusting module, if greater than or equal to, triggering the fourth adjusting module;

The third adjustment module includes: a calculation submodule and an adjustment submodule,

a calculation sub-module, configured to calculate a transition depth of field corresponding to the stored time value, wherein each time value greater than or equal to 0 and smaller than the second preset value corresponds to a transition depth of field, and the transition depth of field is located in the three-dimensional cylindrical image When the foreground is deep and the target depth of field;

Adjusting a submodule for adjusting a depth of field of the three-dimensional cylindrical image to the transition depth of field;

And a fourth adjustment module, configured to adjust a depth of field of the three-dimensional cylindrical image to the target depth of field.

As an implementation manner, the calculating submodule may be specifically used to:

Calculate using the following formula:

Transition Depth of Field = stored time value / said zoom duration * (target depth of field - current depth of field of the 3D cylindrical image) + current depth of field of the 3D cylindrical image.

As an implementation manner, the apparatus may further include: a fifth determining module, a sixth determining module, a copying module, and an adding module (not shown), where

a fifth determining module, configured to determine, according to the mapping relationship, that an expanded edge line of the expanded plane image is mapped to a corresponding position in the three-dimensional cylindrical image;

a sixth determining module, configured to determine an area where the location is located in the three-dimensional cylindrical image according to a preset area dividing rule;

a copy module for copying the determined area to obtain a duplicate image;

And a adding module, configured to add the duplicate image to the expanded plane image according to the mapping relationship.

Applying the embodiment shown in FIG. 10 of the present application, the fisheye image is mapped onto the three-dimensional cylindrical surface to obtain a three-dimensional cylindrical image, and after receiving the cylindrical surface expansion instruction, the three-dimensional cylindrical image is mapped to obtain a developed surface image; in the scheme, three-dimensional The visual effect of the cylindrical image is better than that of the hemisphere image, and if the user wants to see the image area of the back view, there is no need to manually rotate the three-dimensional cylinder, and only need to send a cylindrical expansion command to see the expansion surface image, the expansion surface The image contains an image area with a large viewing angle, thus improving the image display effect.

The embodiment of the present application further provides an electronic device, as shown in FIG. 11, including a processor 1101 and a memory 1102.

a memory 1102, configured to store a computer program;

The processor 1101 is configured to implement any of the above-described fisheye image processing methods when executing the program stored on the memory 1102.

The memory mentioned in the above electronic device may include a random access memory (RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

The above processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; or may be a digital signal processing (DSP), dedicated integration. Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component.

The embodiment of the present application further provides a computer readable storage medium, where the computer readable storage medium stores a computer program, and when the computer program is executed by the processor, implements any of the above-described fisheye image processing methods.

The embodiment of the present application also provides an executable program code for being executed to execute any of the above-described fisheye image processing methods.

It should be noted that, in this context, relational terms such as first and second are used merely to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply such entities or operations. There is any such actual relationship or order between them. Furthermore, the term "comprises" or "comprises" or "comprises" or any other variations thereof is intended to encompass a non-exclusive inclusion, such that a process, method, article, or device that comprises a plurality of elements includes not only those elements but also Other elements, or elements that are inherent to such a process, method, item, or device. An element defined by the phrase "comprising a ..." does not exclude the presence of additional elements in the process, method, article, or device that comprises the element.

The various embodiments in the present specification are described in a related manner, and the same or similar parts between the various embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the fisheye image processing device embodiment shown in FIG. 10, the electronic device embodiment shown in FIG. 11, the computer readable storage medium embodiment, and the above-described executable program code embodiment are basically similar. In the embodiment of the fisheye image processing method shown in FIG. 1-9, the description is relatively simple, and the relevant parts can be referred to the description of the embodiment of the fisheye image processing method shown in FIG. 1-9.

The above is only the preferred embodiment of the present application, and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc., which are made within the spirit and principles of the present application, should be included in the present application. Within the scope of protection.

Claims (27)

1. A fisheye image processing method, the method comprising:

   Obtaining a fisheye image to be processed;

   According to a mapping relationship between a preset fisheye image coordinate point and a three-dimensional cylindrical coordinate point, the fisheye image to be processed is mapped onto the three-dimensional cylindrical surface to obtain a three-dimensional cylindrical image;

   After receiving the cylindrical surface expansion command, the three-dimensional cylindrical image is mapped according to a mapping relationship between the preset three-dimensional cylindrical coordinate point and the developed surface image coordinate point to obtain a developed surface image.
2. The method according to claim 1, wherein after receiving the cylindrical surface expansion command, the three-dimensional cylindrical image is mapped according to a mapping relationship between a preset three-dimensional cylindrical coordinate point and a developed surface image coordinate point. Go to the expansion surface and get the expanded surface image, including:

   After receiving the cylindrical surface expansion instruction, determining an expansion duration, and a first mapping relationship between the target expansion surface image and the three-dimensional cylindrical image;

   Determining, according to the expansion duration and the first mapping relationship, a second mapping relationship between the expansion surface image and the three-dimensional cylindrical image at each time before reaching the expansion duration;

   For each time before the expansion time is reached, the three-dimensional cylindrical image is mapped according to the second mapping relationship corresponding to the time, to obtain the time expansion surface image;

   When the expansion time is reached, the three-dimensional cylindrical image is mapped according to the first mapping relationship to obtain the target expansion surface image.
3. The method according to claim 1, wherein after receiving the cylindrical surface expansion command, the three-dimensional cylindrical image is performed according to a mapping relationship between a preset three-dimensional cylindrical coordinate point and an expansion surface image coordinate point. Map to get the expanded face image, including:

   After receiving the cylindrical expansion command, start the timer and determine the expansion time;

   Determining a time value stored in the timer at the current moment;

   Determining whether the stored time value is less than the expansion time;

   If the value is smaller than the third mapping relationship corresponding to the stored time value, the three-dimensional cylindrical image is mapped to obtain the expanded surface image corresponding to the stored time value; wherein, each of the expansion time is greater than or equal to 0 The time values respectively correspond to a third mapping relationship, and a third mapping relationship is: a mapping relationship between the three-dimensional cylindrical coordinate points and a coordinate value of the expanded curved image corresponding to a time value;

   If it is greater than or equal to, the three-dimensional cylindrical image is mapped according to a mapping relationship between the preset three-dimensional cylindrical coordinate point and the developed planar image coordinate point, and an expanded planar image is obtained.
4. The method according to claim 3, wherein the coordinate values of the three-dimensional cylindrical coordinate points comprise: a first coordinate value, a second coordinate value, and a third coordinate value, and the expanded surface image coordinates corresponding to the one time value The coordinate values of the point include: a fourth coordinate value, a fifth coordinate value, and a sixth coordinate value;

   Mapping, by the root, the three-dimensional cylindrical image according to the third mapping relationship corresponding to the stored time value, including:

   Calculating a ratio of the time value to the expansion time for each stored time value;

   Calculating the mapped fourth coordinate value according to the first coordinate value, the arc length between the first coordinate value and the preset reference point, and the ratio; wherein the arc length is according to the viewpoint angle and the Determining the radius of the three-dimensional cylinder;

   Determining the second coordinate value as the fifth coordinate value to be mapped;

   The mapped sixth coordinate value is calculated according to the third coordinate value, the radius of the three-dimensional cylinder, and the ratio.
5. The method according to claim 4, wherein said calculating the mapped number according to said first coordinate value, an arc length between said first coordinate value and a preset reference point, and said ratio Four coordinate values, including:

   The fourth coordinate value to be mapped = the first coordinate value + the ratio * [sign (first coordinate value) * the arc length between the first coordinate value and the preset reference point - the first coordinate value];

   An arc length between the first coordinate value and a preset reference point = a radius of the three-dimensional cylinder * (π - viewpoint angle);

   The viewpoint angle = tan ^-1 (absolute value of the first coordinate value / third coordinate value);

   And calculating, according to the third coordinate value, the radius of the three-dimensional cylinder, and the ratio, the mapped sixth coordinate value, including:

   The sixth coordinate value to be mapped = the third coordinate value + the ratio * (- the radius of the three-dimensional cylinder - the third coordinate value).
6. The method according to claim 3, wherein the coordinate values of the three-dimensional cylindrical coordinate points comprise: a first coordinate value, a second coordinate value, and a third coordinate value, and the coordinate values of the expanded planar image coordinate point include : a seventh coordinate value, an eighth coordinate value, and a ninth coordinate value;

   And mapping the three-dimensional cylindrical image according to a mapping relationship between the preset three-dimensional cylindrical coordinate point and the developed planar image coordinate point, to obtain an expanded planar image, including:

   Calculating the mapped seventh coordinate value according to the first coordinate value and an arc length between the first coordinate value and the preset reference point; wherein the arc length is according to the viewpoint angle and the three-dimensional cylinder Radius determination;

   Determining the second coordinate value as the eighth coordinate value to be mapped;

   The inverse of the radius of the three-dimensional cylinder is determined as the ninth coordinate value to which the mapping is made.
7. The method according to claim 6, wherein the calculating the mapped seventh coordinate value according to the first coordinate value and the arc length between the first coordinate value and the preset reference point ,include:

   a seventh coordinate value=sign (first coordinate value)* an arc length between the first coordinate value and the preset reference point;

   An arc length between the first coordinate value and a preset reference point = a radius of the three-dimensional cylinder * (π - viewpoint angle);

   The viewpoint angle = tan ^-1 (absolute value of the first coordinate value / third coordinate value).
8. The method according to claim 1, wherein after the obtaining the three-dimensional cylindrical image, the method further comprises:

   Receiving a rotation cylinder command sent by the user; determining a rotation angle corresponding to the rotation cylinder instruction; rotating the three-dimensional cylinder image by the rotation angle by using a center line of the three-dimensional cylindrical image as an axis;

   or,

   Receiving a viewpoint movement instruction sent by the user; determining a viewpoint angle and a depth of field corresponding to the viewpoint movement instruction; and adjusting the three-dimensional cylindrical image according to the determined viewpoint angle and depth of field.
9. The method according to claim 1, wherein after the obtaining the three-dimensional cylindrical image, the method further comprises:

   Receiving a zoom instruction sent by the user;

   Determining a target depth of field corresponding to the scaling instruction;

   The depth of field of the three-dimensional cylindrical image is adjusted to the target depth of field.
10. The method according to claim 9, wherein after receiving the scaling instruction sent by the user, the method further comprises: starting a timer and determining a zooming duration;

    Adjusting the depth of field of the three-dimensional cylindrical image to the target depth of field includes:

    Determining a time value stored in the timer at the current moment;

    Determining whether the stored time value is less than the scaling time;

    If the value is less than, the transition depth of field corresponding to the stored time value is calculated, wherein each time value greater than or equal to 0 and less than the zoom duration corresponds to a transition depth of field, and the transition depth of field is located at the current depth of field of the three-dimensional cylindrical image and the target Between the depths of field; adjusting the depth of field of the three-dimensional cylindrical image to the transition depth of field;

    If it is greater than or equal to, the depth of field of the three-dimensional cylindrical image is adjusted to the target depth of field.
11. The method according to claim 10, wherein the calculating the transition depth of field corresponding to the stored time value comprises:

    Transition Depth of Field = stored time value / said zoom duration * (target depth of field - current depth of field of the 3D cylindrical image) + current depth of field of the 3D cylindrical image.
12. The method according to claim 1, wherein the three-dimensional cylindrical image is mapped onto the expansion surface according to a mapping relationship between the preset three-dimensional cylindrical coordinate point and the developed surface image coordinate point, and the expansion surface is obtained. After the image, it also includes:

    Determining, according to the mapping relationship, that the expanded edge line of the expanded plane image is mapped to a corresponding position in the three-dimensional cylindrical image;

    Determining, in the three-dimensional cylindrical image, an area in which the location is located according to a preset area division rule;

    Copying the determined area to obtain a duplicate image;

    The duplicate image is added to the expanded plane image according to the mapping relationship.
13. A fisheye image processing apparatus, characterized in that the apparatus comprises:

    Obtaining a module, configured to obtain a fisheye image to be processed;

    a first mapping module, configured to map the image of the fisheye to be processed to a three-dimensional cylindrical surface according to a mapping relationship between a preset fisheye image coordinate point and a three-dimensional cylindrical coordinate point, to obtain a three-dimensional cylindrical image;

    The second mapping module is configured to map the three-dimensional cylindrical image according to a mapping relationship between the preset three-dimensional cylindrical coordinate point and the developed surface image coordinate point after receiving the cylindrical surface expansion instruction, to obtain a developed surface image.
14. The device according to claim 13, wherein the second mapping module is specifically configured to:

    After receiving the cylindrical surface expansion instruction, determining an expansion duration, and a first mapping relationship between the target expansion surface image and the three-dimensional cylindrical image;

    Determining, according to the expansion duration and the first mapping relationship, a second mapping relationship between the expansion surface image and the three-dimensional cylindrical image at each time before reaching the expansion duration;

    For each time before the expansion time is reached, the three-dimensional cylindrical image is mapped according to the second mapping relationship corresponding to the time, to obtain the time expansion surface image;

    When the expansion time is reached, the three-dimensional cylindrical image is mapped according to the first mapping relationship to obtain the target expansion surface image.
15. The apparatus according to claim 13, wherein the second mapping module comprises:

    a timing sub-module, configured to start a timer after receiving a cylindrical expansion command, and determine an expansion time;

    a first determining submodule, configured to determine a time value stored in the timer at a current moment;

    a determining sub-module, configured to determine whether the stored time value is less than the expansion time; if not, triggering the first mapping sub-module, if greater than or equal to, triggering the second mapping sub-module;

    a first mapping sub-module, configured to map the three-dimensional cylindrical image according to a third mapping relationship corresponding to the stored time value, to obtain an expanded surface image corresponding to the stored time value; wherein, greater than or equal to 0 and less than Each time value of the expansion time length corresponds to a third mapping relationship, and a third mapping relationship is: a mapping relationship between the coordinate points of the three-dimensional cylindrical coordinate points and the expanded surface image corresponding to a time value;

    The second mapping sub-module is configured to map the three-dimensional cylindrical image according to a mapping relationship between the preset three-dimensional cylindrical coordinate points and the developed planar image coordinate points, to obtain an expanded planar image.
16. The apparatus according to claim 15, wherein the coordinate values of the three-dimensional cylindrical coordinate points comprise: a first coordinate value, a second coordinate value, and a third coordinate value, and the expanded surface image coordinates corresponding to the one time value The coordinate values of the point include: a fourth coordinate value, a fifth coordinate value, and a sixth coordinate value;

    The first mapping submodule is specifically configured to:

    Calculating a ratio of the time value to the expansion time for each stored time value;

    Calculating the mapped fourth coordinate value according to the first coordinate value, the arc length between the first coordinate value and the preset reference point, and the ratio; wherein the arc length is according to the viewpoint angle and the Determining the radius of the three-dimensional cylinder;

    Determining the second coordinate value as the fifth coordinate value to be mapped;

    The mapped sixth coordinate value is calculated according to the third coordinate value, the radius of the three-dimensional cylinder, and the ratio.
17. The device according to claim 16, wherein the first mapping submodule is specifically configured to:

    Calculating a ratio of the time value to the expansion time for each stored time value;

    Calculate using the following formula:

    The fourth coordinate value to be mapped = the first coordinate value + the ratio * [sign (first coordinate value) * the arc length between the first coordinate value and the preset reference point - the first coordinate value];

    An arc length between the first coordinate value and a preset reference point = a radius of the three-dimensional cylinder * (π - viewpoint angle);

    The viewpoint angle = tan ^-1 (absolute value of the first coordinate value / third coordinate value);

    Determining the second coordinate value as the fifth coordinate value to be mapped;

    The sixth coordinate value to be mapped = the third coordinate value + the ratio * (- the radius of the three-dimensional cylinder - the third coordinate value).
18. The apparatus according to claim 15, wherein the coordinate values of the three-dimensional cylindrical coordinate points comprise: a first coordinate value, a second coordinate value, and a third coordinate value, and the coordinate values of the expanded planar image coordinate point include : a seventh coordinate value, an eighth coordinate value, and a ninth coordinate value;

    The second mapping submodule is specifically configured to:

    Calculating the mapped seventh coordinate value according to the first coordinate value and an arc length between the first coordinate value and the preset reference point; wherein the arc length is according to the viewpoint angle and the three-dimensional cylinder Radius determination;

    Determining the second coordinate value as the eighth coordinate value to be mapped;

    The inverse of the radius of the three-dimensional cylinder is determined as the ninth coordinate value to which the mapping is made.
19. The device according to claim 18, wherein the second mapping sub-module is specifically configured to perform calculation by using the following formula:

    a seventh coordinate value=sign (first coordinate value)* an arc length between the first coordinate value and the preset reference point;

    An arc length between the first coordinate value and a preset reference point = a radius of the three-dimensional cylinder * (π - viewpoint angle);

    The viewpoint angle = tan ^-1 (absolute value of the first coordinate value / third coordinate value);

    Determining the second coordinate value as the eighth coordinate value to be mapped;

    The inverse of the radius of the three-dimensional cylinder is determined as the ninth coordinate value to which the mapping is made.
20. The device according to claim 13, wherein the device further comprises: a first receiving module, a first determining module, and a rotating module, or the device further comprises: a second receiving module, a second determining module, and a first adjustment module, wherein

    a first receiving module, configured to receive a rotating cylinder command sent by a user;

    a first determining module, configured to determine a rotation angle corresponding to the rotating cylinder instruction;

    a rotation module, configured to rotate the three-dimensional cylindrical image by the rotation angle by using a center line of the three-dimensional cylindrical image as an axis;

    a second receiving module, configured to receive a viewpoint moving instruction sent by the user;

    a second determining module, configured to determine a viewpoint angle and a depth of field corresponding to the viewpoint moving instruction;

    The first adjustment module is configured to adjust the three-dimensional cylindrical image according to the determined viewpoint angle and depth of field.
21. The device according to claim 13, wherein the device further comprises:

    a third receiving module, configured to receive a scaling instruction sent by the user;

    a third determining module, configured to determine a target depth of field corresponding to the scaling instruction;

    And a second adjustment module, configured to adjust a depth of field of the three-dimensional cylindrical image to the target depth of field.
22. The device of claim 21, wherein the device further comprises:

    a timing module, configured to start a timer after receiving a zoom instruction sent by the user, and determine a zoom duration;

    a fourth determining module, configured to determine a time value stored in the timer at a current moment;

    a judging module, configured to determine whether the stored time value is less than the zooming time length; if not, triggering the third adjusting module, if greater than or equal to, triggering the fourth adjusting module;

    The third adjustment module includes: a calculation submodule and an adjustment submodule,

    a calculation sub-module, configured to calculate a transition depth of field corresponding to the stored time value, wherein each time value greater than or equal to 0 and less than the zoom duration corresponds to a transition depth of field, and the transition depth of field is located at a current depth of field of the three-dimensional cylindrical image Between the target depths of field;

    Adjusting a submodule for adjusting a depth of field of the three-dimensional cylindrical image to the transition depth of field;

    And a fourth adjustment module, configured to adjust a depth of field of the three-dimensional cylindrical image to the target depth of field.
23. The device according to claim 22, wherein the calculation sub-module is specifically configured to perform calculation by using the following formula:

    Transition Depth of Field = stored time value / said zoom duration * (target depth of field - current depth of field of the 3D cylindrical image) + current depth of field of the 3D cylindrical image.
24. The device according to claim 13, wherein the device further comprises:

    a fifth determining module, configured to determine, according to the mapping relationship, that an expanded edge line of the expanded plane image is mapped to a corresponding position in the three-dimensional cylindrical image;

    a sixth determining module, configured to determine an area where the location is located in the three-dimensional cylindrical image according to a preset area dividing rule;

    a copy module for copying the determined area to obtain a duplicate image;

    And a adding module, configured to add the duplicate image to the expanded plane image according to the mapping relationship.
25. An electronic device, comprising: a processor and a memory;

    a memory for storing a computer program;

    The processor, when executed to execute a program stored on the memory, implements the method steps of any of claims 1-12.
26. A computer readable storage medium, wherein the computer readable storage medium stores a computer program, the computer program being executed by a processor to implement the method steps of any of claims 1-12.
27. An executable program code, characterized in that the executable program code is operative to perform the method steps of any of claims 1-12.

Images