WO2024002023A1 - Method and apparatus for generating panoramic stereoscopic image, and electronic device - Google Patents

Abstract

Provided in the embodiments of the present invention are a method and apparatus for generating a panoramic stereoscopic image, and an electronic device. The method comprises: according to a plurality of acquired fisheye images, generating a left-eye longitude and latitude image and a right-eye longitude and latitude image, which correspond to each fisheye image; extracting first stitching areas between a plurality of left-eye longitude and latitude images, so as to obtain a plurality of first stitching areas, and performing image fusion on the first stitching areas, so as to generate a left-eye panoramic image; extracting second stitching areas between a plurality of right-eye longitude and latitude images, so as to obtain a plurality of second stitching areas, and performing image fusion on the second stitching areas, so as to generate a right-eye panoramic image; and generating a panoramic stereoscopic image according to the left-eye panoramic image and the right-eye panoramic image, thereby eliminating a fault or ghosting phenomenon.

Description

Panoramic stereoscopic image generation method, device and electronic equipment 【Technical field】

Embodiments of the present invention relate to the field of image processing, and in particular, to a method, device and electronic device for generating a panoramic stereoscopic image.

【Background technique】

With the development of 3D stereo vision technology and panoramic cameras, users' requirements for 3D stereo vision experience continue to increase. Panoramic stereo images can record 360° images, giving users a strong sense of reality and three-dimensionality. Therefore, they are widely used in travel, live broadcast, There are wide application prospects in many industries such as film and television and real estate.

At present, panoramic stereoscopic images are mostly obtained by splicing multiple images. There are obvious faults or ghosting phenomena in the splicing area of the image. This phenomenon is especially obvious when there is a close-up view in the splicing area, causing the panoramic stereoscopic image to appear. Distortion, thereby reducing the realism of the panoramic stereoscopic image.

[Content of the invention]

In view of this, embodiments of the present invention provide a method, device and electronic device for generating a panoramic stereoscopic image, which are used to eliminate faults or ghosting phenomena in the splicing area of the panoramic stereoscopic image, so as to reduce the picture distortion of the panoramic stereoscopic image.

In a first aspect, embodiments of the present invention provide a method for generating a panoramic stereoscopic image. The method includes:

Based on the multiple acquired fisheye images, generate the left eye longitude and latitude image and the right eye longitude and latitude image corresponding to each fisheye image;

Extract first splicing areas between multiple left-eye longitude and latitude images to obtain multiple first splicing areas, and perform image fusion on each of the first splicing areas to generate a left-eye panoramic image;

Extract second splicing areas between multiple right eye latitude and longitude images to obtain multiple second splicing areas, and perform image fusion on each of the second splicing areas to generate a right eye panoramic image;

A panoramic stereoscopic image is generated based on the left eye panoramic image and the right eye panoramic image.

In a possible implementation, generating a left eye longitude and latitude image and a right eye longitude and latitude image corresponding to each fisheye image based on the acquired multiple fisheye images includes:

A left-eye view splicing block area is divided on the set left-eye blank longitude and latitude map. In the left-eye view splicing block area, the longitude and latitude map of the fisheye image is expanded according to the projection relationship between the fisheye image and the longitude and latitude map. , generate the left eye latitude and longitude image corresponding to the fisheye image;

A right-eye view splicing block area is divided on the set right-eye blank longitude and latitude map. In the right-eye view splicing block area, the longitude and latitude map of the fisheye image is expanded according to the projection relationship between the fisheye image and the longitude and latitude map. , generate the right eye latitude and longitude image corresponding to the fisheye image.

In a possible implementation, the left-eye view splicing block area is divided on the set left-eye blank longitude and latitude map, including:

According to the projection relationship between the fisheye image and the latitude and longitude map, the center point of the fisheye image is projected onto the blank latitude and longitude map of the left eye to form a projection of the center point, and the meridian where the projection of the center point is located is defined as the left eye Blank latitude and longitude map centerline;

The meridian with a set angle value to the left of the center line of the left eye blank longitude and latitude map from the left eye blank longitude and latitude map is used as the left eye viewing angle centerline;

In the left-eye blank longitude and latitude map, the left-eye viewing angle center line is the viewing field angle center line, and the image area with the set viewing angle is taken as the left-eye viewing angle splicing block area.

In a possible implementation, the right eye perspective splicing block area is divided on the set right eye blank longitude and latitude map, including:

According to the projection relationship between the fisheye image and the longitude and latitude map, the center point of the fisheye image is projected onto the blank longitude and latitude map of the right eye to form a projection of the center point, and the meridian where the projection of the center point is located is defined as the right eye Blank latitude and longitude map centerline;

The meridian with a set angle value to the right of the center line of the right eye blank longitude and latitude map from the right eye blank longitude and latitude map is used as the right eye viewing angle centerline;

In the right eye blank latitude and longitude map, the right eye viewing angle center line is the viewing field angle center line, and the image area with the set viewing angle is taken as the right eye viewing angle splicing block area.

In a possible implementation, performing image fusion on each of the first splicing areas to generate a left-eye perspective panoramic image includes:

Project multiple first splicing areas onto the unit sphere to obtain multiple first areas to be spliced on the unit sphere;

Map the multiple first areas to be spliced on the unit sphere to the cylinder corresponding to the unit sphere respectively to obtain the cylinder areas corresponding to the multiple first areas to be spliced;

Calculate the optical flow field between the left image and the right image of the cylindrical area corresponding to the plurality of first areas to be spliced;

According to the optical flow field between the left image and the right image of each cylindrical area, the images of each first splicing area are fused to generate the left eye panoramic image.

In a possible implementation, performing image fusion on each of the second splicing areas to generate a right-eye perspective panoramic image includes:

Project multiple second splicing areas onto the unit sphere to obtain multiple second areas to be spliced on the unit sphere;

Map the multiple second areas to be spliced on the unit sphere to the cylinder corresponding to the unit sphere respectively, to obtain the cylinder areas corresponding to the multiple second areas to be spliced;

Calculate the optical flow field between the left image and the right image of the cylindrical area corresponding to the multiple second areas to be spliced;

According to the optical flow field between the left image and the right image of each cylindrical area, the images of each second splicing area are fused to generate the right eye panoramic image.

In a possible implementation, generating a panoramic stereoscopic image based on the left eye panoramic image and the right eye panoramic image includes:

Reproject the left-eye panoramic image to generate the left-eye visible picture;

Reproject the right-eye panoramic image to generate the right-eye visible picture;

The left-eye viewable picture and the right-eye viewable picture are synchronized to generate the panoramic stereoscopic image.

In a second aspect, embodiments of the present invention provide a device for generating a panoramic stereoscopic image, including:

Fisheye image acquisition module, used to acquire multiple fisheye images;

The longitude and latitude image generation module is used to generate the left eye longitude and latitude image and the right eye longitude and latitude image corresponding to each fisheye image based on the multiple acquired fisheye images;

The fusion module is used to extract the first splicing area between multiple left-eye longitude and latitude images to obtain multiple first splicing areas, perform image fusion on each of the first splicing areas to generate a left-eye panoramic image; extract multiple right-eye panoramic images. A plurality of second splicing areas are obtained from the second splicing areas between the eye longitude and latitude images, and image fusion is performed on each of the second splicing areas to generate a right eye panoramic image;

A generating module, configured to generate a panoramic stereoscopic image based on the left eye panoramic image and the right eye panoramic image.

In a third aspect, embodiments of the present invention provide an electronic device. The electronic device includes a plurality of fisheye lenses for acquiring fisheye images. The electronic device further includes a memory and a processor, and the memory is used to store Information including program instructions, the processor is used to control the execution of the program instructions, and when the program instructions are loaded and executed by the processor, the panoramic stereoscopic image generation method in the first aspect or any possible implementation of the first aspect is implemented. A step of.

In a fourth aspect, embodiments of the present invention provide a storage medium that includes a stored program, wherein when the program is running, the electronic device where the stored medium is located is controlled to execute the first aspect or any possibility of the first aspect. The steps of the method for generating panoramic stereoscopic images.

In the technical solution provided by the embodiment of the present invention, according to the multiple acquired fisheye images, the left eye longitude and latitude image and the right eye longitude and latitude image corresponding to each fisheye image are generated, and the first splicing between the multiple left eye longitude and latitude images is extracted. area, generate a left eye panoramic image by image fusion of the first splicing area, extract the second splicing area between multiple right eye longitude and latitude images, generate a right eye panoramic image by image fusion of the second splicing area, according to the left eye The panoramic image and the right-eye panoramic image generate a panoramic stereoscopic image, which can eliminate the faults or ghosting phenomena in the splicing area of the panoramic stereoscopic image, and can reduce the picture distortion of the panoramic stereoscopic image, thus improving the realism of the panoramic stereoscopic image.

[Picture description]

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required to be used in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention and are not relevant to the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without exerting any creative effort.

Figure 1 is a flow chart of a method for generating a panoramic stereoscopic image provided by an embodiment of the present invention;

Figure 2 is a flow chart of a method for generating a left eye longitude and latitude image and a right eye longitude and latitude image provided by an embodiment of the present invention;

Figures 3a to 3b are schematic diagrams of generating a left eye latitude and longitude image according to an embodiment of the present invention;

Figures 3c to 3d are schematic diagrams of generating a right eye longitude and latitude image according to an embodiment of the present invention;

4a to 4b are schematic diagrams of projecting the first splicing area onto a unit sphere in an embodiment of the present invention;

Figures 5a to 5c are schematic diagrams of mapping the first splicing area to a cylinder in an embodiment of the present invention;

Figure 6 is a schematic diagram of a presentation method of a panoramic stereoscopic image generated in an embodiment of the present invention.

Figure 7 is a schematic structural diagram of a device for generating a panoramic stereoscopic image provided by an embodiment of the present invention;

Figure 8 is a schematic diagram of an electronic device provided by an embodiment of the present invention.

【Detailed ways】

In order to better understand the technical solution of the present invention, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

It should be clear that the described embodiments are only some, but not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

The terminology used in the embodiments of the present invention is only for the purpose of describing specific embodiments and is not intended to limit the present invention. As used in this embodiment and the appended claims, the singular forms "a,""the" and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise.

It should be understood that the term "and/or" used in this article is only an association relationship describing related objects, indicating that there can be three relationships, for example, A and/or B, which can mean: A alone exists, and A and A exist simultaneously. B, there are three situations of B alone. In addition, the character "/" in this article generally indicates that the related objects are an "or" relationship.

In order to solve the technical problem in the prior art that there are obvious faults or ghosting phenomena in the spliced area images of panoramic stereoscopic images, embodiments of the present invention provide a method for generating panoramic stereoscopic images. The method for generating a panoramic image can be applied to electronic devices. In other words, each step in the method for generating a panoramic stereoscopic image can be performed by an electronic device. Among them, the electronic device can include multiple fisheye lenses, and the fisheye lens can be an ultra-wide-angle fisheye lens. The number of fisheye lenses in the electronic device can be set according to product design needs, as long as the field of view of all fisheye lenses is ensured ( The sum of Field of View (FOV for short) can cover the entire 360° panoramic stereoscopic image, that is, the number n of fisheye lenses needs to satisfy: n*FOV>=360°, where FOV is the size of a single fisheye lens Field of view, n is a positive integer greater than or equal to 2. When the difference between n*FOV and 360° (i.e. n*FOV-360°) is larger, the FOV of adjacent fisheye lenses is larger, which is more conducive to seamless splicing during the generation of panoramic stereoscopic images. When the FOV of the fisheye lens is constant, due to the larger distortion of the imaging edge of the fisheye lens, the more fisheye lenses there are, the farther away the picture actually used for panoramic rendering by each fisheye lens is from the edge of the fisheye lens, and the quality of the picture will be reduced. The higher. In the embodiment of the present invention, as an optional solution, the number of fisheye lenses can be 6 or 8. Electronic equipment using 6 or 8 fisheye lenses can make the difference between n*FOV and 360° larger. , which is conducive to seamless splicing during the generation of panoramic stereoscopic images.

In the embodiment of the present invention, the electronic device includes but is not limited to a panoramic camera, and the electronic device can be applied to robots, vehicles, drones, etc.

Figure 1 is a flow chart of a method for generating a panoramic stereoscopic image provided by an embodiment of the present invention. As shown in Figure 1, the method includes:

Step 10: Based on the acquired multiple fisheye images, generate a left eye longitude and latitude image and a right eye longitude and latitude image corresponding to each fisheye image.

Each fisheye lens in the electronic device can capture a fisheye image, and multiple fisheye lenses can acquire multiple fisheye images, so that the electronic device can acquire multiple fisheye images through the multiple fisheye lenses set up. eye image. For example, take the electronic device including 6 fisheye lenses as an example. The 6 fisheye lenses are fisheye lens No. 1 to fisheye lens No. 6 respectively. The multiple fisheye images acquired include 6 fisheye images. Then this step It is necessary to generate the left eye longitude and latitude image and the right eye longitude and latitude image corresponding to each of the 6 fisheye images.

In embodiments of the present invention, the projection relationship between the fisheye image and the latitude and longitude map can be established based on the calibration parameters of the fisheye lens, where the calibration parameters of the fisheye lens can include internal parameters and external parameters of the camera.

Then step 10 may specifically include: using the calibration parameters of the fisheye lens and generating a left eye longitude and latitude image and a right eye longitude and latitude image corresponding to each fisheye image based on the acquired multiple fisheye images.

As an optional solution, Figure 2 is a flow chart of a method for generating a left eye longitude and latitude image and a right eye longitude and latitude image provided by an embodiment of the present invention. As shown in Figure 2, step 10 may specifically include:

Step S1: Divide the left eye perspective splicing block area on the set left eye blank longitude and latitude map. In the left eye perspective splicing block area, expand the longitude and latitude map of the fisheye image according to the projection relationship between the fisheye image and the longitude and latitude map. Generate the left eye latitude and longitude image corresponding to the fisheye image.

In the embodiment of the present invention, a corresponding left eye blank longitude and latitude map is set for each fisheye image. Taking six fisheye images as an example for description, the six set left eye blank longitude and latitude maps are respectively the left eye blank longitude and latitude map I1 to I6.

Figures 3a to 3b are schematic diagrams of generating a left eye longitude and latitude image according to an embodiment of the present invention. As shown in Figure 3a and Figure 3b, in step S1, a left eye perspective splicing block is divided on the set left eye blank longitude and latitude image. Area, specifically may include:

Step S11: According to the projection relationship between the fisheye image and the latitude and longitude map, project the center point of the fisheye image onto the blank latitude and longitude map of the left eye to form a projection of the center point, and define the meridian where the projection of the center point is located as the blank latitude and longitude map of the left eye. Centerline.

The center point of the fisheye image is projected onto the left eye blank longitude and latitude maps I1 to I6 respectively, and the center line of the left eye blank longitude and latitude map of each left eye blank longitude and latitude map I1 to I6 is obtained.

Step S12: Use the meridian with a set angle value to the left of the center line of the left-eye blank longitude and latitude map as the left-eye viewing angle center line.

As an alternative, set the angle value α=360°/N/2, where N is the number of fisheye lenses. Taking N=6 as an example, set the angle value α to 30°.

In the embodiment of the present invention, Figures 3a and 3b take the left eye blank longitude and latitude map I1 and the left eye blank longitude and latitude map I2 as examples for description. As shown in Figure 3a, the meridian 30° to the left of the center line of the left-eye blank longitude and latitude map I1 is used as the left-eye viewing angle center line. As shown in Figure 3b, the meridian 30° to the left of the center line of the left-eye blank longitude and latitude map I2 is used as the left-eye viewing angle center line. By analogy, the meridian 30° to the left of the center line of the left eye blank longitude and latitude map I6 is used as the left eye viewing angle center line.

Step S13: In the left-eye blank longitude and latitude map, use the center line of the left eye viewing angle as the center line of the viewing angle and take the image area with the set viewing angle as the left eye viewing angle splicing block area.

In the embodiment of the present invention, the field of view angle β is set to be greater than 60°. As an alternative, the field of view angle β is set to be greater than or equal to 80° and less than or equal to 100°.

Since the seamless splicing method needs to be used to splice and fuse the splicing blocks in the embodiment of the present invention, the seamless splicing method requires adjacent splicing block areas to have a certain overlapping FOV in order to perform image matching and alignment operations during the splicing process. Therefore, The FOV must be greater than 60°. Only when it is greater than 60° can adjacent splicing block areas be guaranteed to have overlapping FOVs. For example, when the FOV of a splicing block is 80°, there is a 20° overlapping FOV between adjacent splicing block areas; when the FOV of a splicing block area is 100°, there is a 40° FOV between adjacent splicing block areas. Coincident FOV. The larger the overlapping FOV is, the more conducive it is to the realization of seamless splicing. As shown in Figure 3a and Figure 3b, taking the field of view angle β as 100° as an example.

In the embodiment of the present invention, as shown in Figure 3a and Figure 3b, the shape of the left-eye view splicing block area is a rectangle. In practical applications, the shape of the left-eye view splicing block area can also adopt other shapes, as long as the splicing block area can cover the required FOV. For example, the boundary of the left-eye view splicing block area can be a curve.

As shown in Figure 3a, in the left-eye blank longitude and latitude map I1, the center line of the left eye perspective is the center line of the field of view, and an image with a set field of view angle of 100° is taken as the left eye view splicing block area L1. As shown in Figure 3b, in the left-eye blank longitude and latitude map I2, the center line of the left-eye viewing angle is the center line of the viewing angle, and the image with the set viewing angle of 100° is used as the left-eye viewing angle splicing block area L2. By analogy, in the left eye blank longitude and latitude map I6, the center line of the left eye view angle is the center line of the field of view, and the image with a set field of view angle of 100° is used as the left eye view splicing block area. Domain L6.

Furthermore, as shown in Figure 3a, the fisheye image is expanded into a latitude and longitude map in the left eye view splicing block area L1 to generate a left eye longitude and latitude image A1 corresponding to the fisheye image. As shown in Figure 3b, the longitude and latitude map of the fish-eye image is expanded in the left-eye view splicing block area L2, and a left-eye longitude and latitude image A2 corresponding to the fish-eye image is generated. By analogy, the longitude and latitude map of the fisheye image is expanded in the left eye view splicing block area L6 to generate the left eye longitude and latitude image A6 corresponding to the fisheye image.

Step S2: Divide the right eye perspective splicing block area on the set right eye blank latitude and longitude map. In the right eye perspective splicing block area, expand the longitude and latitude map of the fisheye image according to the projection relationship between the fisheye image and the longitude and latitude map. Generate the right eye latitude and longitude image corresponding to the fisheye image.

In the embodiment of the present invention, a corresponding right eye blank longitude and latitude map is set for each fisheye image. Taking six fisheye images as an example for description, the set six right eye blank longitude and latitude maps are respectively the right eye blank longitude and latitude map M1 to M6.

Figures 3c to 3d are schematic diagrams of generating a right eye longitude and latitude image according to an embodiment of the present invention. As shown in Figure 3c and Figure 3d, in step S1, right eye perspective splicing blocks are divided on the set right eye blank longitude and latitude image. Area, specifically may include:

Step S21: According to the projection relationship between the fisheye image and the latitude and longitude map, project the center point of the fisheye image onto the blank latitude and longitude map of the right eye to form a projection of the center point, and define the meridian where the projection of the center point is located as the blank latitude and longitude map of the right eye. Centerline.

Project the center point of the fisheye image to the right-eye blank longitude and latitude maps M1 to M6 respectively, and obtain the center line of the right-eye blank longitude and latitude map of each right-eye blank longitude and latitude map M1 to M6.

Step S22: Use the meridian with a set angle value to the right of the center line of the blank latitude and longitude map of the right eye as the center line of the right eye viewing angle.

As an alternative, set the angle value α=360°/N/2, where N is the number of fisheye lenses. Taking N=6 as an example, set the angle value α to 30°.

In the embodiment of the present invention, Figures 3c and 3d take the right eye blank longitude and latitude map M1 and the right eye blank longitude and latitude map M2 as examples for description. As shown in Figure 3c, the longitude 30° to the right of the center line of the right eye blank longitude and latitude map M1 is used as the right eye viewing angle center line. As shown in Figure 3d, place the right eye The meridian 30° to the right of the center line of the blank longitude and latitude map M2 of the right eye's blank longitude and latitude map is used as the center line of the right eye's visual angle. By analogy, the longitude 30° to the right of the center line of the right eye's blank longitude and latitude map M6 is used as the center line of the right eye's viewing angle.

Step S23: In the right-eye blank longitude and latitude map, use the center line of the right eye viewing angle as the center line of the viewing angle and take the image area with the set viewing angle as the right eye viewing angle splicing block area.

In the embodiment of the present invention, the field of view angle β is set to be greater than 60°. As an alternative, the field of view angle β is set to be greater than or equal to 80° and less than or equal to 100°.

Since the seamless splicing method needs to be used to splice and fuse the splicing blocks in the embodiment of the present invention, the seamless splicing method requires adjacent splicing block areas to have a certain overlapping FOV in order to perform image matching and alignment operations during the splicing process. Therefore, The FOV must be greater than 60°. Only when it is greater than 60° can adjacent splicing block areas be guaranteed to have overlapping FOVs. For example, when the FOV of a splicing block is 80°, there is a 20° overlapping FOV between adjacent splicing block areas; when the FOV of a splicing block area is 100°, there is a 40° FOV between adjacent splicing block areas. Coincident FOV. The larger the overlapping FOV is, the more conducive it is to the realization of seamless splicing. As shown in Figure 3a and Figure 3b, taking the field of view angle β as 100° as an example.

In the embodiment of the present invention, as shown in Figure 3c and Figure 3d, the shape of the right-eye view splicing block area is a rectangle. In practical applications, the shape of the right-eye view splicing block area can also adopt other shapes, as long as the splicing block area can cover the required FOV. For example, the boundary of the right-eye view splicing block area can be a curve.

As shown in Figure 3c, in the right eye blank longitude and latitude map M1, the center line of the right eye viewing angle is the center line of the field of view, and an image with a set field of view angle of 100° is used as the right eye viewing angle splicing block area R1. As shown in Figure 3d, in the right eye blank longitude and latitude map M2, the center line of the right eye viewing angle is the center line of the field of view, and an image with a set field of view angle of 100° is used as the right eye viewing angle splicing block area R2. By analogy, in the right eye blank longitude and latitude map M6, the center line of the right eye viewing angle is the center line of the field of view, and an image with a set field of view angle of 100° is taken as the right eye viewing angle splicing block area R6.

Furthermore, as shown in Figure 3c, the fish-eye image is expanded into a latitude and longitude map in the right-eye view splicing block area R1 to generate a right-eye longitude and latitude image B1 corresponding to the fish-eye image. As shown in Figure 3d, the longitude and latitude map of the fisheye image is expanded in the right eye view splicing block area R2' to generate the right eye longitude and latitude corresponding to the fisheye image. Image B2. By analogy, the longitude and latitude map of the fisheye image is expanded in the right eye view splicing block area M6 to generate the right eye longitude and latitude image B6 corresponding to the fisheye image.

Step 12: Extract first splicing areas between multiple left-eye longitude and latitude images to obtain multiple first splicing areas, and perform image fusion on each first splicing area to generate a left-eye panoramic image.

In this embodiment of the present invention, step 12 may specifically include:

Step 122: Extract first splicing areas between multiple left-eye longitude and latitude images to obtain multiple first splicing areas.

Specifically, the first splicing areas between n left-eye longitude and latitude images A1-An are respectively extracted to obtain N first splicing areas S1-Sn. Wherein, the first splicing area between two adjacent left-eye longitude and latitude maps is all or part of the overlapping area between two adjacent left-eye view splicing block areas. For example, n=6, then the first splicing areas between the six left-eye longitude and latitude images A1-A6 are respectively extracted, and six first splicing areas S1-S6 are obtained.

Step 124: Perform image fusion on each first splicing area to generate a left-eye panoramic image.

In this embodiment of the present invention, step 124 may specifically include:

Step 1242: Project multiple first splicing areas onto the unit sphere to obtain multiple first areas to be spliced on the unit sphere.

Perform spherical projection on the n first splicing areas S1-Sn, and project the n first splicing areas S1-Sn onto the unit sphere to obtain n first areas to be spliced J1-Jn on the unit sphere. 4a to 4b are schematic diagrams of projecting the first splicing area onto the unit sphere in the embodiment of the present invention, as shown in FIG. 4b. FIG. 4b shows the first splicing area Sn. As shown in FIG. 4a, the first splicing area is The area Sn is projected onto the unit sphere to obtain the first area Jn to be spliced on the unit sphere.

Step 1244: Map the plurality of first areas to be spliced on the unit sphere to the cylinder surfaces corresponding to the unit sphere respectively, to obtain cylinder areas corresponding to the plurality of first areas to be spliced.

The plurality of first regions J1-Jn to be spliced on the unit sphere are respectively mapped onto the cylinder corresponding to the unit sphere, thereby obtaining cylinder regions T1-Tn corresponding to the plurality of first regions to be spliced. Figures 5a to 5c are schematic diagrams of mapping the first splicing area to a cylinder in an embodiment of the present invention. As shown in Figure 5a, for example, the first area Jn to be spliced on the unit sphere is mapped to the cylinder surface, and the cylinder area corresponding to the first area Jn to be spliced is obtained. Tn. Figure 5b shows the cylindrical area Tn corresponding to the first area Jn to be spliced, and Figure 5c shows the cylindrical area Tn in the expanded state.

Step 1246: Calculate the optical flow field between the left image and the right image of the cylindrical area corresponding to the plurality of first regions to be spliced.

In this step, the optical flow field between the left image and the right image of the N cylindrical regions T1-Tn can be calculated respectively.

Among them, calculating the optical flow field between the left image and the right image of the N cylindrical regions T1-Tn may specifically include:

Calculate the optical flow field f _L→R (x, y) from the left image to the right image in the cylindrical area and the optical flow field f _R→L (x, y) from the right image to the left image;

According to the distance between the pixels of the cylindrical area and the left and right boundaries of the cylindrical area, the weight value α of the optical flow field f _L→R (x, y) from the left image to the right image and the right image to the left are adaptively adjusted. The weight value β of the optical flow field f _R→L (x, y) of the image is used to obtain the optical flow field between the left image and the right image of the adjusted cylindrical area.
f _L→R (x,y)＝α·f _L→R (x,y) Formula 1
f _R→L (x,y)＝β·f _R→L (x,y) Formula 2

In order to achieve seamless splicing, for the left image of the cylindrical area, the optical flow field should be smaller as it is closer to the left boundary of the cylindrical area; while for the right image of the cylindrical area, it should be closer to the right boundary of the cylindrical area. The smaller the light field value, therefore, set two weight values α and β, where α∈[0,1] and β∈[0,1]. In Formula 1, the α value gradually increases from left to right along with the area of the left image; in Formula 2, the β value gradually decreases along with the area of the right image from left to right.

Step 1248: Fusion of the images of each first splicing area according to the optical flow field between the left image and the right image of each cylindrical area to generate a left-eye panoramic image.

In the embodiment of the present invention, the optical flow field between the left image and the right image of the cylindrical area can be used to fuse the images in the first splicing area of the left eye longitude and latitude images to achieve seamless splicing of the first splicing area. , thus achieving seamless splicing of left eye latitude and longitude images.

As an optional solution, step 1248 may specifically include:

Step A1. On the longitude and latitude map, according to the distance between the pixels in the area and the left and right boundaries of the area distance, adaptively adjust the weight value λ' corresponding to the left image and the weight value μ' corresponding to the right image when the left image and the right image are fused.

Step A2: According to the projection relationship between the cylinder and the longitude and latitude image, back-project the optical flow field between the left image and the right image of the cylinder area onto the longitude and latitude image, and obtain the left image and the first splicing area of the left eye longitude and latitude image. Optical flow field between right images.

Wherein, the optical flow field between the left image and the right image in the cylindrical area may include the optical flow field f _L→R (x, y) from the left image to the right image and the optical flow field f _R→ from the right image to the left image. _L (x, y), then after back-projection, the optical flow field between the left image and the right image in the first splicing area of the left eye longitude and latitude image includes the optical flow field f' from the left image to the right image _L→R (x, y) and the optical flow field f' _R→L (x, y) from the right image to the left image.

Step A3: According to the weight value λ' corresponding to the left image and the weight value μ' corresponding to the right image and the optical flow field between the left image and the right image of the first splicing area, use Formula 3 to calculate the left image of the first splicing area. The image S ^L and the right image S ^R are image fused, and the fused fusion area S ^B is obtained to obtain the left eye panoramic image.
S ^B (x,y)＝λ′·S ^L (x+f′ ^x _(L→R) (x,y),y+f′ ^y _(L→R) (x,y))+
μ′·S ^R (x+f _′ ^x _(R→L) (x,y),y+f _′ ^y _(R→L) (x,y)) Formula 3

Among them, S ^B (x, y) represents the value of the pixel point (x, y) in the fusion area S ^B , S ^L (x, y) represents the value of the pixel point (x, y) in the left image S ^L , S ^R (x,y) represents the value of the pixel point (x,y) in the right image S ^R , λ' is the weight value corresponding to the left image, μ ^' is the weight value corresponding to the right image, λ ^' ∈[0,1], μ ^' ∈[0,1]. For the fusion area close to the left image, the corresponding pixel value of the left image should have a greater weight, so the λ' value changes from large to small as the distance from the left boundary of S ^L increases. For the fusion area close to the right image, the corresponding pixel value of the right image should have a greater weight, so the μ' value changes from large to small as the distance from the right boundary of ^SR increases.

At this point, after all the fusion areas S ^B are obtained, the left-eye panoramic image is obtained, and the obtained left-eye panoramic image is a seamlessly spliced longitude and latitude map.

In the solution shown in steps A1 to A3, first the optical flow field between the left image and the right image is back-projected onto the latitude and longitude image, and then the image of the first area to be spliced is directly performed on the latitude and longitude image. Fusion to obtain the left eye panoramic image.

As an optional solution, step 1248 may specifically include:

Step B1: According to the distance between the pixels of the cylindrical area and the left and right boundaries of the cylindrical area, adaptively adjust the weight value λ corresponding to the left image and the weight value μ corresponding to the right image when the left image and the right image are fused. .

Step B2. In the cylindrical area, according to the weight value λ corresponding to the left image and the weight value μ corresponding to the right image and the optical flow field between the left image and the right image of the first to-be-stitched area, use Formula 4 to calculate the first The left image ^TL and the right image ^TR of the area to be spliced are image fused to obtain the fused fusion area ^TB .
T ^B (x, y)=λ′·T ^L (x+f′ _x(L→R) (x, y), y+f′ _y(L→R) (x, y))
+μ′·T ^R (x+f′ _x(R→L) (x, y), y+f′ _y(R→L) (x, y)) Formula 4

Among them, T ^B (x, y) represents the value of the pixel point (x, y) in the fusion area T ^B , T ^L (x, y) represents the value of the pixel point in the left image T ^L , T ^R (x, y) Represents the value of the pixel point in the right image T ^R , λ is the weight value corresponding to the left image, μ is the weight value corresponding to the right image, λ∈[0, 1], μ∈[0, 1]. For the fusion area close to the left image, the corresponding pixel value of the left image should have a greater weight, so the λ value changes from large to small as the distance from the left boundary of the cylindrical area increases. For the fusion area close to the right image, the corresponding pixel value of the right image should have a greater weight, so the μ value changes from large to small as the distance from the right boundary of the cylindrical area increases.

In this step, the images of each first region to be spliced are fused respectively until all the images of the first regions to be spliced are fused and all the fusion regions T ^B are obtained.

Step B3: According to the projection relationship between the cylinder and the longitude and latitude image, back-project all the fusion areas T ^B into the longitude and latitude image to obtain the left eye panoramic image.

In this step, since the fusion area T ^B of the cylinder is seamless, the left-eye panoramic image obtained after back-projecting it onto the latitude and longitude map is also seamless.

In the solution shown in steps B1 to B3, the image of the first region to be spliced is first fused on the cylinder, and then the fused region is back-projected onto the longitude and latitude image to achieve the image of the first splicing region. Fusion is performed to obtain the left eye panoramic image.

Step 14: Extract second splicing areas between multiple right eye latitude and longitude images to obtain multiple second splicing areas, and perform image fusion on each second splicing area to generate a right eye panoramic image.

In this embodiment of the present invention, step 14 may specifically include:

Step 142: Extract second splicing areas between multiple right eye longitude and latitude images to obtain multiple second splicing areas.

Step 144: Perform image fusion on each second splicing area to generate a right-eye panoramic image.

In this embodiment of the present invention, step 144 may specifically include:

Step 1442: Project the plurality of second splicing areas onto the unit sphere to obtain multiple second areas to be spliced on the unit sphere.

Step 1444: Map the plurality of second regions to be spliced on the unit sphere to the cylinder surfaces corresponding to the unit sphere, respectively, to obtain cylinder regions corresponding to the plurality of second regions to be spliced.

Step 1446: Calculate the optical flow field between the left image and the right image of the cylindrical area corresponding to the plurality of second areas to be spliced.

Step 1448: Fusion of the images of each second splicing area according to the optical flow field between the left image and the right image of each cylindrical area to generate a right-eye panoramic image.

In the embodiment of the present invention, for the detailed description of steps 142 to 144, please refer to the above detailed description of steps 122 to 124, which will not be described again here.

Step 16: Generate a panoramic stereoscopic image based on the left eye panoramic image and the right eye panoramic image.

In this embodiment of the present invention, step 16 may specifically include:

Step 161: Reproject the left-eye panoramic image to generate a left-eye visible picture.

The left-eye viewable picture is a panoramic stereoscopic left-eye viewable picture, and the left-eye viewable picture is an image including the left eye's visible range.

Step 162: Reproject the right-eye panoramic image to generate a left-eye viewable image.

The right-eye viewable picture is a panoramic stereoscopic right-eye viewable picture, and the right-eye viewable picture includes images within the right eye's visible range.

In steps 161 and 162, reprojection may include multiple projection methods such as plane projection, curved surface projection, or spherical projection.

Specifically, through the established projection model, the left-eye panoramic image and the right-eye panoramic image are re-projected according to the orientation of the user's eyes to generate left-eye visible images and right-eye visible images respectively.

Step 163: Synchronize the left eye viewable image and the right eye viewable image to generate a stereoscopic panoramic image.

Figure 6 is a schematic diagram of a presentation method of a panoramic stereoscopic image generated in an embodiment of the present invention. As shown in Figure 6, the left window is the left eye viewable picture obtained by reprojecting the left eye panoramic image, and the right window is the right eye The panoramic image is re-projected and the right-eye viewable image is obtained. The pictures in the left and right windows have real parallax, thus achieving a three-dimensional display effect. The pictures in the left and right windows can change in real time according to the direction of the user's eyes, and can present pictures from every angle in the original left-eye panoramic image and right-eye panoramic image, thereby achieving a panoramic three-dimensional display effect.

In the technical solution provided by the embodiment of the present invention, according to the multiple acquired fisheye images, the left eye longitude and latitude image and the right eye longitude and latitude image corresponding to each fisheye image are generated, and the first splicing between the multiple left eye longitude and latitude images is extracted. area, generate a left eye panoramic image by image fusion of the first splicing area, extract the second splicing area between multiple right eye longitude and latitude images, generate a right eye panoramic image by image fusion of the second splicing area, according to the left eye The panoramic image and the right-eye panoramic image generate a panoramic stereoscopic image, which can eliminate the faults or ghosting phenomena in the splicing area of the panoramic stereoscopic image, and can reduce the picture distortion of the panoramic stereoscopic image, thus improving the realism of the panoramic stereoscopic image.

Figure 7 is a schematic structural diagram of a panoramic stereoscopic image generation device provided by an embodiment of the present invention. As shown in Figure 7, the device includes: a fisheye image acquisition module 11, a latitude and longitude image generation module 12, a fusion module 13 and a generation module. 14. Wherein, the fisheye image acquisition module 11 may include multiple fisheye lenses.

The fisheye image acquisition module 11 is used to acquire multiple fisheye images. The longitude and latitude image generation module 12 is used to generate a left eye longitude and latitude image and a right eye longitude and latitude image corresponding to each fisheye image based on the multiple acquired fisheye images; the fusion module 13 is used to extract the third image between the multiple left eye longitude and latitude images. A splicing area is used to obtain multiple first splicing areas, and image fusion is performed on each of the first splicing areas to generate a left eye panoramic image; second splicing areas between multiple right eye longitude and latitude images are extracted to obtain multiple second splicing areas. area, perform image fusion on each of the second splicing areas to generate a right-eye panoramic image; the generating module 14 is configured to generate a panoramic stereoscopic image according to the left-eye panoramic image and the right-eye panoramic image.

In the embodiment of the present invention, the longitude and latitude image generation module 12 is specifically used to divide the left eye view splicing block area on the set left eye blank longitude and latitude map. In the left eye view splicing block area, the fisheye image is converted into the longitude and latitude map. projection relationship, expand the longitude and latitude map of the fisheye image, and generate the left eye longitude and latitude image corresponding to the fisheye image; divide the right eye perspective splicing block area on the set right eye blank longitude and latitude map, and divide the right eye perspective splicing block area on the right eye In the eye view splicing block area, according to the projection relationship between the fisheye image and the longitude and latitude map, the longitude and latitude map of the fisheye image is expanded to generate the right eye longitude and latitude image corresponding to the fisheye image.

In the embodiment of the present invention, the latitude and longitude image generation module 12 is specifically configured to project the center point of the fisheye image onto the blank latitude and longitude map of the left eye to form a projection of the center point based on the projection relationship between the fisheye image and the latitude and longitude map. The meridian where the projection of the center point is located is defined as the center line of the left eye blank longitude and latitude map; the meridian with a set angle value to the left of the left eye blank longitude and latitude map center line of the left eye blank longitude and latitude map is used as the left eye viewing angle center line; in the left eye blank longitude and latitude map, the left eye viewing angle center line is the viewing field angle center line, and the image area with the set viewing angle is taken as the left eye viewing angle splicing block area.

In the embodiment of the present invention, the latitude and longitude image generation module 12 is specifically configured to project the center point of the fisheye image onto the blank latitude and longitude map of the right eye to form a projection of the center point based on the projection relationship between the fisheye image and the latitude and longitude map. The meridian where the projection of the center point is located is defined as the center line of the right eye blank longitude and latitude map; the meridian with a set angle value to the right of the right eye blank latitude and longitude map from the right eye blank longitude and latitude map is used as the right eye viewing angle center line; in the right eye blank longitude and latitude map, the right eye viewing angle center line is the viewing field angle center line, and the image area with the set viewing angle is taken as the right eye viewing angle splicing block area.

In the embodiment of the present invention, the fusion module 13 is specifically used to project multiple first splicing areas onto the unit sphere to obtain multiple first areas to be spliced on the unit sphere; Map them to the cylinder corresponding to the unit sphere respectively to obtain the cylinder areas corresponding to the multiple first areas to be spliced; calculate the optical flow field between the left image and the right image of the cylinder areas corresponding to the multiple first areas to be spliced. ; According to the optical flow field between the left image and the right image of each cylindrical area, the images of each first splicing area are fused to generate the left eye panoramic image.

In the embodiment of the present invention, the fusion module 13 is specifically used to project multiple second splicing areas onto the unit sphere to obtain multiple second areas to be spliced on the unit sphere; Map them to the cylinder corresponding to the unit sphere respectively, and obtain the cylinders corresponding to multiple second areas to be spliced. area; calculate the optical flow field between the left image and the right image of the cylindrical area corresponding to the multiple second areas to be spliced; according to the optical flow field between the left image and the right image of each cylindrical area, for each The images of the second splicing area are fused to generate the right eye panoramic image.

In the embodiment of the present invention, the generation module 14 is specifically configured to re-project the left-eye panoramic image to generate the left-eye viewable picture; re-project the right-eye panoramic image to generate the right-eye viewable picture. Picture; synchronize the left-eye visible picture and the right-eye visible picture to generate the panoramic stereoscopic image.

In the technical solution provided by the embodiment of the present invention, the panoramic stereoscopic image is generated by the panoramic stereoscopic image generating device in the embodiment of the present invention, which can eliminate the faults or ghosting phenomena that appear in the splicing area of the panoramic stereoscopic image, and can reduce the appearance of the panoramic stereoscopic image. Distortion, thus improving the realism of panoramic stereoscopic images.

An embodiment of the present invention provides a storage medium. The storage medium includes a stored program. When the program is running, the device where the storage medium is located is controlled to execute each step of the embodiment of the method for generating a panoramic stereoscopic image. For detailed description, please refer to the above panoramic view. An embodiment of a stereoscopic image generation method.

An embodiment of the present invention provides an electronic device. The electronic device includes a plurality of fisheye lenses for acquiring fisheye images. The electronic device also includes a memory and a processor. The memory is used to store information including program instructions, and the processor is used to control Execution of the program instructions. When the program instructions are loaded and executed by the processor, each step of the embodiment of the above-mentioned method for generating a panoramic stereoscopic image is implemented. For detailed description, please refer to the embodiment of the above-mentioned method for generating a panoramic stereoscopic image.

FIG. 8 is a schematic diagram of an electronic device provided by an embodiment of the present invention. The fisheye lens is not specifically shown in FIG. 8 . As shown in FIG. 8 , the electronic device 30 of this embodiment includes: a processor 31 , a memory 32 and a computer program 33 stored in the memory 32 and executable on the processor 31 . When the computer program 33 is executed by the processor 31 To avoid duplication, the methods for generating panoramic stereoscopic images in the implementation embodiment will not be described one by one here. Alternatively, when the computer program is executed by the processor 31, the functions of each model/unit in the device for generating panoramic stereoscopic images in the embodiment are implemented. To avoid duplication, they will not be described one by one here.

The electronic device 30 includes, but is not limited to, a processor 31 and a memory 32 . Those skilled in the art can understand that FIG. 8 is only an example of the electronic device 30 and does not constitute a limitation on the electronic device 30. To include more or less components than shown in the figures, or to combine certain components, or to use different components, for example, the electronic device 30 may also include input and output devices, network access devices, buses, etc.

The so-called processor 31 can be a central processing unit (Central Processing Unit, CPU), or other general-purpose processor, digital signal processor (Digital Signal Processor, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

The memory 32 may be an internal storage unit of the electronic device 30 , such as a hard disk or memory of the electronic device 30 . The memory 32 can also be an external storage device of the electronic device 30, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SD) card, a flash memory card (Flash) equipped on the electronic device 30. Card) etc. Further, the memory 32 may also include both an internal storage unit of the server 30 and an external storage device. Memory 32 is used to store computer programs and other programs and data required by the server. The memory 32 may also be used to temporarily store data that has been output or is to be output.

Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be described again here.

In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined. Either it can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the coupling or direct coupling or communication connection between each other shown or discussed may be through some interfaces, and the indirect coupling or communication connection of the devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place, or they may not be physically separated. Can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional units.

The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium and includes a number of instructions to cause a computer device (which can be a personal computer, server, or network device, etc.) or processor (Processor) to execute the methods described in various embodiments of the present invention. Some steps. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program code. .

The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the present invention. within the scope of protection.

Claims (10)

1. A method for generating panoramic stereoscopic images, which is characterized by including:

   Based on the multiple acquired fisheye images, generate the left eye longitude and latitude image and the right eye longitude and latitude image corresponding to each fisheye image;

   Extract first splicing areas between multiple left-eye longitude and latitude images to obtain multiple first splicing areas, and perform image fusion on each of the first splicing areas to generate a left-eye panoramic image;

   Extract second splicing areas between multiple right eye latitude and longitude images to obtain multiple second splicing areas, and perform image fusion on each of the second splicing areas to generate a right eye panoramic image;

   A panoramic stereoscopic image is generated based on the left eye panoramic image and the right eye panoramic image.
2. The method according to claim 1, characterized in that generating a left eye longitude and latitude image and a right eye longitude and latitude image corresponding to each fisheye image based on the acquired multiple fisheye images includes:

   A left-eye view splicing block area is divided on the set left-eye blank longitude and latitude map. In the left-eye view splicing block area, the longitude and latitude map of the fisheye image is expanded according to the projection relationship between the fisheye image and the longitude and latitude map. , generate the left eye latitude and longitude image corresponding to the fisheye image;

   A right-eye view splicing block area is divided on the set right-eye blank longitude and latitude map. In the right-eye view splicing block area, the longitude and latitude map of the fisheye image is expanded according to the projection relationship between the fisheye image and the longitude and latitude map. , generate the right eye latitude and longitude image corresponding to the fisheye image.
3. The method according to claim 2, characterized in that dividing the left eye perspective splicing block area on the set left eye blank longitude and latitude map includes:

   According to the projection relationship between the fisheye image and the latitude and longitude map, the center point of the fisheye image is projected onto the blank latitude and longitude map of the left eye to form a projection of the center point, and the meridian where the projection of the center point is located is defined as the left eye Blank latitude and longitude map centerline;

   The meridian with a set angle value to the left of the center line of the left eye blank longitude and latitude map from the left eye blank longitude and latitude map is used as the left eye viewing angle centerline;

   In the left-eye blank longitude and latitude map, the left-eye viewing angle center line is the viewing field angle center line, and the image area with the set viewing angle is taken as the left-eye viewing angle splicing block area.
4. The method according to claim 1, characterized in that dividing the right eye perspective splicing block area on the set right eye blank longitude and latitude map includes:

   According to the projection relationship between the fisheye image and the longitude and latitude map, the center point of the fisheye image is projected onto the blank longitude and latitude map of the right eye to form a projection of the center point, and the meridian where the projection of the center point is located is defined as the right eye Blank latitude and longitude map centerline;

   The meridian with a set angle value to the right of the center line of the blank latitude and longitude map of the right eye from the center line of the blank latitude and longitude map of the right eye is used as the center line of the right eye viewing angle;

   In the right eye blank latitude and longitude map, the right eye viewing angle center line is the viewing field angle center line, and the image area with the set viewing angle is taken as the right eye viewing angle splicing block area.
5. The method according to claim 1, characterized in that, performing image fusion on each of the first splicing areas to generate a left-eye perspective panoramic image includes:

   Project multiple first splicing areas onto the unit sphere to obtain multiple first areas to be spliced on the unit sphere;

   Map the multiple first areas to be spliced on the unit sphere to the cylinder corresponding to the unit sphere respectively to obtain the cylinder areas corresponding to the multiple first areas to be spliced;

   Calculate the optical flow field between the left image and the right image of the cylindrical area corresponding to the plurality of first areas to be spliced;

   According to the optical flow field between the left image and the right image of each cylindrical area, the images of each first splicing area are fused to generate the left eye panoramic image.
6. The method according to claim 1, characterized in that, performing image fusion on each of the second splicing areas to generate a right-eye perspective panoramic image includes:

   Project multiple second splicing areas onto the unit sphere to obtain multiple second areas to be spliced on the unit sphere;

   Map the multiple second areas to be spliced on the unit sphere to the cylinder corresponding to the unit sphere respectively, to obtain the cylinder areas corresponding to the multiple second areas to be spliced;

   Calculate the optical flow field between the left image and the right image of the cylindrical area corresponding to the plurality of second areas to be spliced;

   According to the optical flow field between the left image and the right image of each cylinder area, for each second splicing The regional images are fused to generate the right eye panoramic image.
7. The method of claim 1, wherein generating a panoramic stereoscopic image based on the left eye panoramic image and the right eye panoramic image includes:

   Reproject the left-eye panoramic image to generate the left-eye visible picture;

   Reproject the right-eye panoramic image to generate the right-eye visible picture;

   The left-eye viewable picture and the right-eye viewable picture are synchronized to generate the panoramic stereoscopic image.
8. A device for generating panoramic stereoscopic images, which is characterized by including:

   Fisheye image acquisition module, used to acquire multiple fisheye images;

   The longitude and latitude image generation module is used to generate the left eye longitude and latitude image and the right eye longitude and latitude image corresponding to each fisheye image based on the multiple acquired fisheye images;

   The fusion module is used to extract the first splicing area between multiple left-eye longitude and latitude images to obtain multiple first splicing areas, perform image fusion on each of the first splicing areas to generate a left-eye panoramic image; extract multiple right-eye panoramic images. A plurality of second splicing areas are obtained from the second splicing areas between the eye longitude and latitude images, and image fusion is performed on each of the second splicing areas to generate a right eye panoramic image;

   A generating module, configured to generate a panoramic stereoscopic image based on the left eye panoramic image and the right eye panoramic image.
9. An electronic device, characterized in that the electronic device includes a plurality of fisheye lenses for acquiring fisheye images; the electronic device further includes a memory and a processor, and the memory is used to store information including program instructions, The processor is used to control the execution of program instructions, which are characterized in that, when the program instructions are loaded and executed by the processor, the steps of the panoramic stereoscopic image generation method described in any one of claims 1 to 7 are implemented.
10. A storage medium, characterized in that the storage medium includes a stored program, wherein when the program is run, the electronic device where the stored medium is located is controlled to execute the panoramic stereoscopic image according to any one of claims 1 to 7. Generate the steps of the method.