WO2021012856A1

WO2021012856A1 - Method for photographing panoramic image

Info

Publication number: WO2021012856A1
Application number: PCT/CN2020/097404
Authority: WO
Inventors: 沈峘; 张佩泽; 程百顺; 王汉全
Original assignee: 南京泓众电子科技有限公司
Priority date: 2019-02-14
Filing date: 2020-06-22
Publication date: 2021-01-28
Also published as: WO2021012855A1; CN110248079A; WO2021012854A1; CN110213475A; CN109743489A; CN209897158U; CN110248078A

Abstract

Disclosed in the present invention is a method for photographing a panoramic image, comprising the following steps: connecting a lens module to a rotating mechanism to ensure that the distance between a lens node and the rotation center of the rotating mechanism is less than or equal to 3 cm; setting the parameters of rotation degrees and number of rotations for the rotating mechanism; rotating the rotating mechanism according to the set parameters to drive the lens module to rotate to different angular positions, so that the lens of the lens module acquires regional images of a current field of view until all the regional images required for splicing a panoramic image are acquired; and synthesizing the plurality of regional images into a panoramic image by an image synthesis device. In the present invention, the distance between the lens node and the rotation center of the rotating mechanism is less than or equal to 3 cm, such that the image parallax can be effectively reduced, thereby not only improving the imaging quality of the final panoramic image, but also lowering the requirements and costs for software and algorithms.

Description

A method for shooting panoramic images

Technical field

The present invention relates to the field of camera technology, in particular to the technical field of panoramic image shooting and generation.

Background technique

The panorama shows the surrounding environment as much as possible through wide-angle representation methods and forms such as paintings, photos, videos, and three-dimensional models. 360 panorama, that is, by capturing the image information of the entire scene with a professional camera or using modeling software to render the pictures, using the software to combine the pictures, and playing them with a special player, that is, the plane photos or computer modeling pictures are turned into 360 degrees Full view, used for virtual reality browsing, simulating a two-dimensional plan into a real three-dimensional space and presenting it to the viewer. The panoramic image records the information of the 360-degree space. When the panoramic image is displayed, you can drag the image screen to browse. The feature of 360-degree full recording and unique viewing experience makes the panoramic image in many fields, such as hotel display, real estate display, Car exhibitions are widely used, and viewers can learn about the actual situation of hotels, etc. without leaving home. The panoramic display greatly facilitates people's lives.

At present, there are two main methods for shooting panoramic images in the industry. One is to set up a SLR on a pan-tilt, and use a professional photographer to manually turn the pan-tilt to shoot images from multiple angles, which are processed by professional stitching software on the PC side. A panoramic image is obtained. The panoramic image captured in this way has high quality, but the shooting process and the post-stitching process are relatively cumbersome and the shooting efficiency is low. Another way is to use a panoramic camera to shoot, which has high shooting efficiency. For multi-purpose all-in-one cameras, the shooting quality is high, but the price is expensive, basically more than 10,000 yuan. For binoculars, the shooting efficiency is high and the price is low. However, the binocular all-in-one cameras use fisheye lenses, and the edge areas of the fisheye images with poor image quality and more noise are used when stitching. Therefore, the final panoramic image quality of the stitching is not high.

The above shooting methods are ultimately obtained by acquiring images at different azimuth angles, and stitching multiple sets of images to obtain a panoramic image according to the correlation characteristics of the overlapping areas between adjacent images. However, due to the existence of parallax, there will be differences in the images in the overlapping area, resulting in uneven stitching, and abnormal stitching will bring discomfort to viewing. For example, the patent publication number CN107369129A discloses a method of calculating the optical flow field between adjacent images to achieve the purpose of aligning the overlap area point by point, but this method can only be used to deal with the situation where the parallax is small. When it is larger, because the objects in the overlap area are different, the point-by-point alignment will inevitably lead to stretching and distortion, making the straight line on the seam twisted and split, etc.; the patent publication number CN103685886A discloses An image capture system that changes the direction of the light path by setting a reflective surface between two sets of wide-angle lenses placed back to back, and reflects the light to an imaging sensor that is different from the optical axis of the wide-angle lens. There is no imaging sensor between the two wide-angle lenses. , Thus reducing the distance between the two wide-angle lens nodes, thereby reducing the parallax, making the seams aligned better.

Summary of the invention

Purpose of the invention: The purpose of the present invention is to address the shortcomings of the prior art and provide a panoramic image shooting method, which adopts a single-lens rotation method to collect images. By reasonably setting the distance between the lens node and the rotation center, the image can be effectively reduced Parallax is helpful for seamless stitching in the later stage.

Technical solution: The panoramic image shooting method of the present invention includes the following steps:

(1) Determine the node of the lens module and connect the lens module with a rotating mechanism to ensure that the distance between the lens node and the center of rotation of the rotating mechanism is less than or equal to 3cm;

(2) Set the rotation angle and number of rotation parameters of the rotation mechanism according to the size of the panoramic image and the field of view of the lens module, and ensure that the field of view of the lens module at each shooting angle has the same field as the previous shooting angle. Overlap area

(3) The rotating mechanism rotates according to the set parameters to drive the lens module to rotate to different angular positions. The lens of the lens module collects the regional image of the current field of view until the collection of all the regional images required for stitching the panoramic image is completed;

(4) The multiple area images acquired by the lens module are sent to the image synthesis device, and the image synthesis device synthesizes the multiple area images into a panoramic image.

A further preferred technical solution of the present invention is that when the lens field angle of the lens module is greater than 180 degrees, the rotation angle parameter of the rotation mechanism includes the rotation angle of the lens module to a different longitude position or a rotation to a different latitude position. Angle; the number of rotation parameters of the rotating mechanism include the number of rotations of the lens module to different longitude positions or the number of rotations to different latitude positions;

When the lens field angle of the lens module is less than or equal to 180 degrees, the rotation angle parameters of the rotation mechanism include the rotation angle of the lens module to different longitude positions and the rotation angle to different latitude positions; The number of rotation parameters include the number of rotations of the lens module to different longitude positions and the number of rotations to different latitude positions;

The longitude position corresponds to the heading angle of the lens, and the latitude position corresponds to the pitch angle of the lens.

Preferably, the rotation angle and the number of rotation parameters of the rotating mechanism include the rotation angle and the number of rotation calculations in the longitude and latitude directions. In the longitude direction, the horizontal field of view of the lens is θ_hor, and the two adjacent longitude positions are set to measure the lens The overlapping field of view angle of light is dθ_hor, and the value of dθ_hor is greater than 0. When the lens rotates from the first longitude position to the second longitude position, the rotation angle is θ_hor-dθ_hor, and the horizontal field of view angle of the panoramic image is φ , Φ≤360 degrees, the number of horizontal rotations of the lens is calculated as φ/(θ_hor-dθ_hor)-1, if the calculated value is a decimal, then round up and add 1;

In the latitude direction, the vertical field of view of the lens is θ_ver, and the overlapping field of view of two adjacent latitude images is set to dθ_ver. If the value of dθ_ver is greater than 0, the lens will rotate from the first latitude position to the second latitude position When the angle of rotation is θ_ver-dθ_ver, set the vertical angle of view of the panoramic image to φ, φ≤360 degrees, then the number of vertical rotations of the lens is calculated as φ/(θ_ver-dθ_ver)-1, if the calculated value is For decimals, round up and add 1.

Preferably, when the angle of view of the lens module of the lens module is greater than 180 degrees, in step (4), when multiple area images are synthesized into a panoramic image, two area images taken at adjacent angles are selected to overlap with the image center. The associated features of the regions are spliced and synthesized.

Preferably, in step (1), when connecting the lens module and the rotating mechanism, the lens node of the lens module is coincident with the rotation center of the rotating mechanism.

Preferably, in step (3), when the rotation mechanism drives the lens module to rotate, the position sensor detects the rotation angle of the lens module and compares it with the set rotation angle. The rotation angle of the lens module is not transferred to the theoretical design. Value, control the rotating mechanism to continue to move until it reaches the set position.

Preferably, in step (3), when the lens collects the area image of the current field of view, the attitude sensor is used to obtain the attitude angle data corresponding to the lens module of the area image at the time of collection. When the panoramic image is synthesized, the panoramic image is The image is corrected to the horizontal position.

Preferably, in step (1), the method for determining the node of the lens module is specifically:

Slide the lens module on the guide rod. The lens module can be moved back and forth along the guide rod. The lens optical axis of the lens module is parallel to the guide rod. Fix the guide rod on the tripod and adjust the guide rod to a horizontal position; Place two vertically placed reference objects in front of the lens module, called the first reference object and the second reference object, respectively, so that the lens optical axis of the lens module vertically passes through the center line of the first reference object and the second reference object ; Rotate the guide rod and the lens module in the horizontal plane around the tripod, and watch the relative position of the first reference object and the second reference object through the image preview stream of the lens module. If the rotation is in progress, the first reference object and the second reference object If the objects do not overlap, move the lens module back and forth along the guide bar until the first reference object and the second reference object in the preview stream always overlap during the rotation; mark the vertical axis of the tripod on the lens of the lens module The intersection with the lens, this intersection is the node of the lens.

Beneficial effects: In the present invention, images are collected in a single-lens rotation mode, and finally spliced and synthesized into a panoramic image. First, the hardware cost of the shooting scheme is greatly reduced, the system structure is simplified, and it has the advantages of small size, light weight, and portability; In addition, since the multiple regional images stitched and synthesized into a panoramic image are captured by the same lens, the consistency between the images of each region is relatively high; in the present invention, the distance between the lens node and the rotation center of the rotating mechanism is less than or equal to 3 cm, which can effectively reduce The small image parallax not only improves the imaging quality of the final panoramic image, but also reduces the requirements and costs for software and algorithms.

Description of the drawings

FIG. 1 is a schematic diagram of shooting in the longitude direction of the lens module in the embodiment;

FIG. 2 is a schematic diagram of shooting in the latitude direction of the lens module in the embodiment;

FIG. 3 is a schematic diagram of a method for determining a node of a lens module in an embodiment;

Figure 4 is a schematic diagram of different areas of a fisheye image.

Detailed ways

The technical solution of the present invention will be described in detail below with the accompanying drawings, but the protection scope of the present invention is not limited to the embodiments.

Embodiment: A method for shooting panoramic images includes the following steps:

(1) Determine the node of the lens module and connect the lens module to a rotating mechanism to ensure that the distance between the lens node and the center of rotation of the rotating mechanism is less than or equal to 3cm. When conditions permit, the best choice It overlaps the lens node of the lens module with the rotation center of the rotating mechanism.

The method for determining the node of the lens module is specifically as follows:

Slide the lens module 1 on the guide rod 2. The lens module 1 can be moved back and forth along the guide rod 2. The lens optical axis of the lens module 1 is parallel to the guide rod 2, and the guide rod 2 is fixedly installed on the tripod 3. , Adjust the guide rod 2 to the horizontal position; respectively place two vertical reference objects in front of the lens module 1, called the first reference object 4 and the second reference object 5, so that the lens optical axis of the lens module 1 is vertical Pass through the center lines of the first reference object 4 and the second reference object 5; rotate the guide rod 2 and the lens module 1 in the horizontal plane around the tripod 3, and watch the first reference object 4 and the second reference object 4 and the second reference object 4 and the second reference object through the image preview stream of the lens module 1. The relative position of the reference object 5, if the first reference object 4 and the second reference object 5 do not overlap during the rotation process, move the lens module 1 back and forth along the guide rod 2 until the first reference object in the preview stream is rotated. The reference object 4 and the second reference object 5 always remain overlapped; mark the intersection of the vertical axis of the tripod 3 and the lens on the lens of the lens module 1, and this intersection is the node of the lens.

When the lens field angle of the lens module is greater than 180 degrees, the lens module can only be rotated in one direction to reduce the number of photographs. Therefore, the rotation angle parameter of the rotation mechanism is the rotation angle of the lens module to different longitude positions or Rotation angles to different latitude positions; the rotation number parameter of the rotation mechanism is the number of rotations of the lens module to different longitude positions or the number of rotations to different latitude positions;

When the field of view of the lens module is less than or equal to 180 degrees, the lens module needs to be rotated in two directions. Therefore, the rotation angle parameters of the rotation mechanism include the rotation angle and rotation of the lens module to different longitude positions. Rotation angles to different latitude positions; the rotation number parameters of the rotation mechanism include the number of rotations of the lens module to different longitude positions and the number of rotations to different latitude positions.

The above longitude position corresponds to the heading angle of the lens, and the latitude position corresponds to the pitch angle of the lens.

The specific rotation parameters are determined by calculation, including the rotation angle and the number of rotations of longitude and latitude:

In the longitude direction, the horizontal field of view of the lens is θ_hor, and the overlapping field of view of the lens metering at two adjacent longitude positions is set to dθ_hor. If the value of dθ_hor is greater than 0, the lens will rotate from the first longitude position to the second In the longitude position, the rotation angle is θ_hor-dθ_hor, and the horizontal field of view angle of the panoramic image is φ, φ≤360 degrees, then the number of horizontal rotations of the lens is calculated as φ/(θ_hor-dθ_hor)-1, if the calculated value is For decimals, round up and add 1;

(3) The rotating mechanism rotates according to the set parameters to drive the lens module to rotate to different angular positions. The lens of the lens module collects the regional image of the current field of view until the collection of all the regional images required for stitching the panoramic image is completed.

When the actual rotating mechanism rotates, there will be a certain deviation. In order to achieve the theoretical design value, a position sensor is required. The position sensor detects the rotation angle of the lens module and compares it with the set rotation angle. If the angle does not turn to the theoretical design value, control the rotating mechanism to continue to move until it reaches the set position.

(4) The multiple area images acquired by the lens module are sent to the image synthesis device, and the image synthesis device synthesizes the multiple area images into a panoramic image. When the field of view of the lens module is greater than 180 degrees, because the brightness of the edge image will be less than the brightness of the center of the image during the imaging of the fisheye lens, vignetting will appear. The edge area is the ring area p shown in Figure 4, if When the edge images participate in the stitching and fusion, dark shadows will appear in the panoramic image, and the sharpness of the edge pixels of the fisheye image is lower than that of the center image. Therefore, in order to obtain high-quality panoramic images, try to select the overlapping area close to the center of the image when stitching. , Such as area q.

In addition, when taking images, an attitude sensor needs to be set up, and the attitude angle data corresponding to the lens module when the image of a certain area is acquired through the attitude sensor. The attitude angles include pitch angle, roll angle, and heading angle. If the lens is placed obliquely during shooting, the originally vertical wall may appear to be inclined in the panorama shot. When a panoramic photo is played in a panoramic player and the angle of the panoramic image is changed for display, because the screen content is not vertical, the rotation of the screen makes it difficult for human eyes to understand. Therefore, when synthesizing the panoramic image, the panoramic image is corrected to the horizontal position according to the attitude angle data.

As mentioned above, although the present invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the present invention itself. Various changes in form and details can be made without departing from the spirit and scope of the present invention as defined by the appended claims.

Claims

A method for shooting panoramic images is characterized in that it comprises the following steps:

(1) Determine the node of the lens module and connect the lens module with a rotating mechanism to ensure that the distance between the lens node and the center of rotation of the rotating mechanism is less than or equal to 3cm;

(2) Set the rotation angle and number of rotation parameters of the rotation mechanism according to the size of the panoramic image and the field of view of the lens module, and ensure that the field of view of the lens module at each shooting angle has the same field as the previous shooting angle. Overlap area

(3) The rotating mechanism rotates according to the set parameters to drive the lens module to rotate to different angular positions. The lens of the lens module collects the regional image of the current field of view until the collection of all the regional images required for stitching the panoramic image is completed;

(4) The multiple area images acquired by the lens module are sent to the image synthesis device, and the image synthesis device synthesizes the multiple area images into a panoramic image.
The method for shooting a panoramic image according to claim 1, wherein when the lens field angle of the lens module is greater than 180 degrees, the rotation angle parameter of the rotation mechanism includes the rotation angle of the lens module to different longitude positions Or the angle of rotation to different latitude positions; the number of rotation parameters of the rotation mechanism includes the number of rotations of the lens module to different longitude positions or the number of rotations to different latitude positions;

When the lens field angle of the lens module is less than or equal to 180 degrees, the rotation angle parameters of the rotation mechanism include the rotation angle of the lens module to different longitude positions and the rotation angle to different latitude positions; The number of rotation parameters include the number of rotations of the lens module to different longitude positions and the number of rotations to different latitude positions;

The longitude position corresponds to the heading angle of the lens, and the latitude position corresponds to the pitch angle of the lens.
The panoramic image shooting method according to claim 2, wherein the rotation angle and the number of rotation parameters of the rotating mechanism include the rotation angle and the number of rotation calculations in the longitude and latitude directions, and the horizontal field of view of the lens in the longitude direction The angle is θ_hor, set the overlapping field of view angle of the lens metering at two adjacent longitude positions to dθ_hor. The value of dθ_hor is greater than 0. When the lens rotates from the first longitude position to the second longitude position, the rotation angle is θ_hor -dθ_hor, suppose the horizontal field of view angle of the panoramic image is φ, φ≤360 degrees, then the number of horizontal rotations of the lens is calculated as φ/(θ_hor-dθ_hor)-1, if the calculated value is a decimal, then round up and add 1;

In the latitude direction, the vertical field of view of the lens is θ_ver, and the overlapping field of view of two adjacent latitude images is set to dθ_ver. If the value of dθ_ver is greater than 0, the lens will rotate from the first latitude position to the second latitude position When the angle of rotation is θ_ver-dθ_ver, set the vertical angle of view of the panoramic image to φ, φ≤360 degrees, then the number of vertical rotations of the lens is calculated as φ/(θ_ver-dθ_ver)-1, if the calculated value is For decimals, round up and add 1.
The method for shooting a panoramic image according to claim 2, wherein when the field of view of the lens module is greater than 180 degrees, in step (4), when multiple regional images are combined into a panoramic image, select Two regional images taken from adjacent angles are stitched together to synthesize the associated features of the overlapping area near the center of the image.
The method for shooting a panoramic image according to claim 1, wherein in step (1), when the lens module and the rotating mechanism are connected, the lens node of the lens module is coincident with the rotation center of the rotating mechanism.
The method for shooting panoramic images according to claim 1, wherein in step (3), when the rotation mechanism drives the rotation of the lens module, the position sensor detects the rotation angle of the lens module and compares it with the set rotation angle By comparison, when the rotation angle of the lens module does not turn to the theoretical design value, the rotation mechanism is controlled to continue to move until it reaches the set position.
The panoramic image shooting method according to claim 1, characterized in that, in step (3), when the lens collects an area image of the current field of view, the attitude sensor obtains the attitude of the area image corresponding to the lens module at the time of collection. Angle data, when synthesizing a panoramic image, the panoramic image is corrected to a horizontal position based on the data.
The method for shooting a panoramic image according to claim 1, wherein in step (1), the method for determining the node of the lens module is specifically:

Slide the lens module on the guide rod. The lens module can be moved back and forth along the guide rod. The lens optical axis of the lens module is parallel to the guide rod. Fix the guide rod on the tripod and adjust the guide rod to a horizontal position; Place two vertically placed reference objects in front of the lens module, called the first reference object and the second reference object, respectively, so that the lens optical axis of the lens module vertically passes through the center line of the first reference object and the second reference object ; Rotate the guide rod and the lens module in the horizontal plane around the tripod, and watch the relative position of the first reference object and the second reference object through the image preview stream of the lens module. If the rotation is in progress, the first reference object and the second reference object If the objects do not overlap, move the lens module back and forth along the guide bar until the first reference object and the second reference object in the preview stream always overlap during the rotation; mark the vertical axis of the tripod on the lens of the lens module The intersection with the lens, this intersection is the node of the lens.