WO2011095026A1

WO2011095026A1 - Method and system for photography

Info

Publication number: WO2011095026A1
Application number: PCT/CN2010/078856
Authority: WO
Inventors: 苏红宏; 王兆祥
Original assignee: 华为终端有限公司
Priority date: 2010-02-02
Filing date: 2010-11-18
Publication date: 2011-08-11
Also published as: CN102143305B; CN102143305A

Abstract

A method and system for photography are provided. Said method comprises: each photography equipment in a photography device shoots an object respectively, said photography device includes multi photography equipments, the optical centers of said multi photography equipments coincide, and there is an overlap region between the images shot by two adjacent photography equipments; imaging by splicing the images shot by each photography equipment in the photography device.

Description

The present invention claims the priority of the Chinese Patent Application entitled "Camera Method and System", filed on February 2, 2010, the Chinese Patent Office, Application No. 201010105422. In this application. The present invention relates to the field of image acquisition technologies, and in particular, to an imaging method and system. BACKGROUND OF THE INVENTION In the production of high-resolution movie and television, telepresence, and the like, it is often necessary to take high-resolution and large-scale images. Due to the limited resolution and shooting range of a single camera, in order to meet the shooting needs, multiple cameras are usually used for shooting, and then the images captured by each camera are stitched together.

In the prior art, there are two schemes for shooting with two cameras. In the first scheme, two cameras are placed in parallel, and the images captured by the two cameras are stitched and imaged, and the images captured by the two cameras overlap at different depths. different. In the second scheme, the equivalent optical center of one camera is coincident with the optical center of the other camera through the mirror, and the shooting edges of the two cameras are also aligned.

In the process of implementing the present invention, the inventors have found that at least the following problems exist in the prior art:

In the first scheme, the images captured by the two cameras have different overlapping areas at different depths, and it is difficult to splicing the images that are aligned at each depth level; in the second scheme, the two cameras have no overlapping regions, and there are obvious stitching seams. In addition, if there are differences between the two cameras, when shooting images, the brightness and/or chromaticity of the images captured by the two cameras will be different. There will be obvious differences when stitching together, and it is difficult to combine the two without overlapping areas. Alignment of images, it is also difficult to determine and eliminate brightness and/or chromaticity differences by image processing, which is not conducive to splicing imaging of images. Summary of the invention

Embodiments of the present invention provide an imaging method and system, and a plurality of imaging devices capable of capturing an overlapping area when an optical center is contracted, which is advantageous for splicing imaging of an image.

The technical solution adopted by the embodiment of the present invention is:

A camera method includes:

Each of the imaging devices in the imaging device respectively captures an object, and the imaging device includes a plurality of imaging devices, The imaging devices are optically coincident, and there are overlapping regions between the captured images of two adjacent imaging devices;

The captured images of the respective imaging devices in the imaging device are spliced and imaged.

A camera system comprising:

An image capturing device, wherein the image capturing device includes a plurality of image capturing devices, wherein the plurality of image capturing devices respectively capture an object, the plurality of image capturing devices are optically coincident, and an image of overlapping images between adjacent two image capturing devices has an overlap region;

The processing device is configured to perform splicing imaging on the captured image of each imaging device in the imaging device.

In the imaging method and system of the embodiment of the present invention, the imaging device includes a plurality of imaging devices, and the plurality of imaging devices are optically coincident. When the captured images of the imaging devices in the imaging device are spliced and imaged, the overlapping regions can be conveniently used. Aligning the images to fuse the brightness and/or chromaticity of the spliced image is beneficial to the analysis processing during image splicing; since there is an overlapping area between the captured images of two adjacent imaging devices, the splicing seam pair can be easily eliminated The effect of image stitching improves image quality. BRIEF DESCRIPTION OF THE DRAWINGS In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings to be used in the embodiments or the description of the prior art will be briefly described below, and obviously, in the following description The drawings are only some of the embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative work.

1 is a flowchart of an image capturing method according to Embodiment 1 of the present invention;

2 is a flowchart of an image capturing method according to Embodiment 2 of the present invention;

2a is a schematic diagram of a brightness fitting curve of each pixel in an overlapping area according to Embodiment 2 of the present invention; FIG. 3 is a schematic structural diagram of a camera system according to Embodiment 3 of the present invention;

4 is a schematic structural diagram of a camera system according to Embodiment 4 of the present invention;

FIG. 5 is a schematic structural diagram of an image pickup apparatus according to Embodiment 4 of the present invention; FIG.

FIG. 6 is a schematic structural diagram of an image pickup apparatus according to Embodiment 5 of the present invention. The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of them. Example. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention. In order to make the advantages of the technical solutions of the present invention clearer, the present invention will be described in detail below with reference to the accompanying drawings and embodiments.

Embodiment 1

This embodiment provides an imaging method. As shown in FIG. 1, the method includes:

101. Each of the image capturing apparatuses in the image capturing apparatus respectively captures an object, the image capturing apparatus includes a plurality of image capturing apparatuses, and the plurality of image capturing apparatuses have an optical center coincidently, and an overlapping area exists between the captured images of the adjacent two image capturing apparatuses;

102. Perform splicing imaging on the captured image of each imaging device in the imaging device.

In the imaging method of the embodiment of the present invention, the imaging device includes a plurality of imaging devices, and the plurality of imaging devices are optically coincident. When the captured images of the imaging devices in the imaging device are spliced and imaged, the overlapping regions can be used to conveniently perform images. The alignment and the fusion of the brightness and/or chromaticity of the stitched image are beneficial to the analysis processing during image stitching; since there are overlapping regions between the captured images of two adjacent camera devices, the stitching can be easily eliminated. The impact is enhanced to improve image quality.

Embodiment 2

As shown in FIG. 2, the imaging method includes:

201. Each of the imaging devices in the imaging device respectively captures an object, and the imaging device includes a plurality of imaging devices, wherein the plurality of imaging devices are optically coincident, and there is an overlapping region between the captured images of the adjacent two imaging devices.

202. Align the overlapping regions between the captured images of the two adjacent imaging devices to obtain a stitched image.

Wherein, the overlapping regions between the captured images of the two adjacent imaging devices can be aligned, and the images captured by the two imaging devices can be analyzed to find the paired feature points of the overlapping regions, and the images are aligned based on the paired feature points (position) Correction). The SIFT (Scale-Invariant Feature Transform) method can be used to determine the pairing feature points.

203. Determine a brightness and/or chromaticity correction relationship of the overlapping area according to the brightness value and/or the chromaticity value of the partial image or the pixel in the overlapping area.

Determining the brightness and/or chrominance correction relationship of the overlap region can be obtained by calculating the luminance and/or chrominance values of a plurality of images/pixels and then fitting the calibration curve. Taking the brightness calculation as an example, if 10 points of the overlapping area are selected, the brightness values of the 10 points in the captured images of the two imaging devices are:

According to this brightness value, a linear fitting curve can be obtained by least squares method, as shown in Fig. 2a, the line The fitted curve of sex has the following functional relationship:

y = 0.987 + 7.184

Wherein, X represents a luminance value of a captured image of the first imaging apparatus, and y represents a luminance value of a captured image of the second imaging apparatus.

204. Correct the brightness and/or chromaticity difference of the stitched image according to the brightness and/or chromaticity correction relationship of the overlap region.

Correcting the stitched image according to the brightness and/or chromaticity correction relationship of the overlap region obtained in step 203, and correcting the captured image of another imaging device based on the captured image of one imaging device, or simultaneously capturing two images The captured image of the device is processed.

Assuming that the luminance value of the captured image of the second imaging apparatus is kept unchanged, the luminance value of each point of the image captured by the first imaging apparatus is adjusted according to the fitting curve function relationship obtained in step 203, that is,

0.9871 + 7.184

Where ; is the adjusted brightness and η is the brightness before adjustment.

The adjustment method of chromaticity is similar to the brightness. Of course, it is also possible to directly construct a correspondence table between each pixel point and the brightness value without fitting the function relationship, and adjust the method by looking up the table.

The brightness value and/or chrominance value correction can be analyzed based on one or more frames of images. After the correction relationship is calculated, the correction relationship is directly used in subsequent images to correct, and the paired feature points are not extracted and analyzed in real time.

At the two edges of the overlapping area, the imaging quality of a certain camera may not be very good. Therefore, at the edge of the overlapping area, the image of one camera can be selected as the main reference for correction, that is, the image is obtained by weighting in the overlapping area. You can use the following formula to weight:

F(P) = α Έ _λ {ρ)+ β Έ ₂ (Ρ)

In the above formula, F(P) represents an image signal finally synthesized at any point in the overlap region, the image signal may be an RGB value or a YUV value, and F^P) represents an image signal of the P point output by the camera C1, F ₂ (P) represents the image signal of point P output by camera C2, α and ? are weighting coefficients, generally " = at the edge of the overlapping area, α and can be set according to the edge condition, for example: " = 2, β can be taken = 0.

Further, when it is required to output the captured image through projection, if each imaging device corresponds to one projection output, the fusion region of the projection stitching is set to coincide with the overlapping region of the captured image, ax F^P) and y9 x F ₂ (P) As the input of two adjacent projections respectively, the image is naturally restored when the output is superimposed, and the stitching fusion is performed without projection stitching.

In the imaging method of the embodiment of the present invention, the imaging device includes a plurality of imaging devices, and the plurality of imaging devices are optically coincident. When the captured image of each imaging device in the imaging device is spliced and imaged, the overlapping region can be used to conveniently align the image and correct the brightness and/or chromaticity of the spliced image according to the paired feature points in the overlapping region, which is advantageous for Analysis processing when images are stitched; Since there are overlapping regions between the captured images of two adjacent imaging devices, the influence of the stitching seam on the image stitching can be easily eliminated, and the image quality is improved.

Embodiment 3

This embodiment provides a camera system. As shown in FIG. 3, the system includes:

The imaging device 31 includes a plurality of imaging devices, wherein the plurality of imaging devices respectively acquire captured images, the plurality of imaging devices are optically coincident, and the captured images of the adjacent two imaging devices overlap The processing device 32 is configured to perform splicing imaging on the captured image of each imaging device in the imaging device 31. In the imaging system of the embodiment of the present invention, the imaging device includes a plurality of imaging devices, and the plurality of imaging devices are optically coincident. When the captured images of the imaging devices in the imaging device are spliced and imaged, the overlapping regions can be used to conveniently perform images. The alignment and the fusion of the brightness and/or chromaticity of the stitched image are beneficial to the analysis processing during image stitching; since there are overlapping regions between the captured images of two adjacent camera devices, the stitching can be easily eliminated. The impact is enhanced to improve image quality.

Embodiment 4

This embodiment provides a camera system. As shown in FIG. 4, the system includes:

The imaging device 41 includes a plurality of imaging devices, the plurality of imaging devices respectively acquiring captured images, the plurality of imaging devices having optical centers coincident, and overlapping images of adjacent two imaging devices The processing device 42 is configured to perform splicing imaging of the captured image of each imaging device in the imaging device 41. Taking an imaging device including two imaging devices as an example, as shown in FIG. 5, the imaging device includes: two cameras C1 and C2, on the mid-perpendicular line of the line segment formed by the optical center of C1 and the optical center of C2 , the mirror AD is set, wherein the AC segment is a total reflection mirror surface, and the CD segment is a semi-reflex lens surface. Through the reflection of the BD segment, the virtual optical center of C1 (the image of the C1 optical center with respect to the AD surface) coincides with the optical center 0 of C2.

PQRS is the subject. When the light emitted by the QR segment passes through the semi-reflex lens surface CD, part of the light is reflected into C1, and some of the light passes through the transmission into C2. The QR segment will be imaged in both cameras, but the brightness will be lower than other regions. . Therefore, C1 can capture the PR segment of the object, and C2 can capture the QS segment of the object, eventually forming the overlapping region QR.

The mirror AD can be selected from glass as the substrate. The AC segment is coated with a total reflection film (such as aluminum) to form total reflection. The CD segment is coated with a very thin semi-reflective film (also aluminum), which can pass through a part. Light can also reflect a portion of the light. Preferably, the transmittance and reflectance of the CD segment light are both close to 50%. Since the coating on the mirror AD is very thin and the substrate is the same, the gap between the total reflection film and the semi-reflective film is very small and does not substantially affect. Imaging. In order to reduce the effect of refraction on imaging, the substrate selects materials with a low refractive index, such as: light glass, plexiglass, resin, and the thickness of the substrate is small.

As shown in Figure 5, in order to avoid interference, the JK surface needs to be occluded (opaque), and there is no object in the CDJK area. The position of the CDs C1 and C2 and the half-reflex lens surface CD are adjusted so that the object is located at EBDJH. Area (E, H points can be extended).

The method described in the second embodiment can be used for the splicing process of the captured images of the cameras C1 and C2, and details are not described herein again.

Further, as shown in FIG. 4, the processing device 42 may include:

The splicing module 421 is configured to align the overlapping areas between the captured images of the two adjacent imaging devices to obtain a spliced image.

Further, as shown in FIG. 4, the processing device 42 may further include:

The correction module 422 is configured to correct a brightness and/or a chromaticity difference of the stitched image according to the overlapping area. The correction module 422 can include:

a determining unit 4221, configured to determine a brightness and/or a chromaticity correction relationship of the overlapping area according to the brightness value and/or the chromaticity value of the partial image or the pixel point in the overlapping area;

The correcting unit 4222 is configured to correct the brightness and/or chromaticity difference of the mosaic image according to the brightness and/or chromaticity correction relationship of the overlapping area.

In the present embodiment, when determining the brightness correction relationship of the overlap region, the transmittance and the reflectance of the half-reflex lens surface CD are calculated as an example, and the transmittance and reflectance of the half-reflex lens surface CD are calculated. When not equal, the brightness value is divided by the corresponding transmittance or reflectance before calculation. Specifically, for the captured image of the camera C1, it is necessary to divide the luminance value of the overlap region by the reflectance of the semi-reflex lens surface CD; for the captured image of the camera C2, it is necessary to divide the luminance value of the overlap region by The transmittance of the semi-reflex lens surface CD.

In the imaging system of the embodiment of the present invention, the imaging device includes a plurality of imaging devices, and the plurality of imaging devices are optically coincident. When the captured images of the imaging devices in the imaging device are spliced and imaged, the overlapping regions can be used to conveniently perform images. The alignment and the fusion of the brightness and/or chromaticity of the stitched image are beneficial to the analysis processing during image stitching; since there are overlapping regions between the captured images of two adjacent camera devices, the stitching can be easily eliminated. The impact is enhanced to improve image quality.

In the present embodiment, the cameras C 1 and C2 can be replaced with imaging and photosensitive devices such as a lens and a CCD sensor.

Embodiment 5 The embodiment provides an imaging system, and the system includes the devices, modules and units as shown in FIG. 4 in the fourth embodiment.

Taking an image pickup apparatus including three image pickup apparatuses as an example, as shown in FIG. 6, the image pickup apparatus includes: a convex lens L and a concave mirror ^ point 0 is an optical center of the convex lens L, the concave mirror M is a spherical mirror, and the concave mirror M The center of the circle is also at 0, since the light passing through 0 passes through the spherical mirror and the direction is unchanged (reflected along the original path), so 0 is actually the equivalent optical center of the concave mirror M. 1, 2, and 3 are three photosensitive devices, wherein the photosensitive surface of 1 faces the convex lens L, and the photosensitive faces of 2 and 3 face the concave mirror M. The object AF will be imaged separately on 1, 2, 3, and the object space of the image is corresponding to EF, BE, AC, such that there is an overlap region DE between 1 and 2, and an overlap region BC between 2 and 3.

Under the conditions of paraxial shooting, the imaging conditions of the convex lens and the spherical mirror can be expressed by the formula (1): convex lens)

丄 + =丄 (concave mirror) (1)

In the above formula: the object distance of the convex lens L, ^ is the object distance of the concave mirror M, the image distance of the convex lens L, the image distance of the concave mirror M, / ₂ is the object focal length of the concave mirror M, 'for the convex lens The image focal length of L.

As shown in Figure 6, the assignment of each of the above parameters is as shown in equation (2):

s; = VI

s _l = -U\

s ₂ ' =-V2

s ₂ =~U2

f ₂ = ~R^

U2=R+ m

(2)

Substituting the values of the parameters in equation (2) into equation (1) yields equation (3):

(3)

In equation (3), when + ₂ ) = ? When the imaging ratios of the convex lens L and the concave mirror M are the same, when the photosensitive devices 1, 2, 3 are located in the same image plane (that is, the photosensitive device 2 and the photosensitive devices 1, 3 are placed back to back, respectively), it can be concluded that:

Therefore, when the optical center of the convex lens L coincides with the center (center) of the concave mirror M, and the ratio of the focal length of the convex lens L to the focal length of the concave mirror M is, the same imaging plane will be imaged, that is: (V _{l +} V ₂ ) = R , at this time, the imaging position on the photosensitive device 2 at any point in the space is exactly the same as the position on the photosensitive device 1 or 3.

As can be seen from Fig. 6, when ( + ₂ ) < ?, the image point on the photosensitive device 1 or 3 can be mapped to the imaging plane of the photosensitive device 2 by a simple linear transformation.

In order to make the photosensitive device 2 and the photosensitive devices 1 and 3 have the same brightness, it is necessary to make the light of the convex lens L and the concave mirror M equivalent, assuming that L and M are both circular, the diameter of the convex lens L is 0, and the diameter of the concave mirror M is For d, when the object distance is much larger than the image distance, the relationship between the convex lens L and the aperture of the concave mirror M can be approximately determined as: D = d.

In the actual optical system, D and d are the pupil size corresponding to the convex lens L and the concave mirror M. Due to the influence of light transmittance, reflectivity, optical path and shooting distance, it may be appropriately adjusted according to actual conditions, or may be The signal collected by the sensor is corrected by: F; = h F ₂ .

Wherein, F ₂ is an image signal collected by the photosensitive device 2, and A is a proportional coefficient, which can be determined by testing in an actual optical system, and is an adjusted image signal. Of course, it is also possible to adjust the image signals of the photosensitive devices 1 and 3.

The splicing process of the captured images of the camera photographic devices 1, 2, and 3 can be performed by the method described in the second embodiment, and details are not described herein again.

In the present embodiment, the convex lens L and the concave mirror M may also be equivalent convex lenses and equivalent concave mirrors composed of a plurality of optical devices.

In this embodiment, the CCD can also be replaced with a camera by adding some optical components. For example: Remove 3 photosensitive devices, and place the image on the photosensitive devices 1 and 3 to other positions by placing the camera behind the photosensitive device 2, and place the camera at other positions, which is equivalent to the formation of the lens and the concave mirror. The real image can be changed to a camera group with overlapping centers of light and overlapping areas, which will not be described here.

The camera system provided by the embodiment of the invention can implement the method embodiment provided above. The imaging method and system provided by the embodiments of the present invention can be applied to photographing high-resolution or wide-range images, but is not limited thereto. A person skilled in the art can understand that all or part of the process of implementing the above embodiment method can be completed by a computer program to instruct related hardware, and the program can be stored in a computer readable storage medium. In execution, the flow of an embodiment of the methods as described above may be included. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM). The above is only the specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any change or replacement that can be easily conceived by those skilled in the art within the technical scope of the present invention is All should be covered by the scope of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

Rights request

1. A camera system, comprising:

An image capturing device, the image capturing device includes a plurality of image capturing devices, wherein the plurality of image capturing devices respectively acquire captured images, the plurality of image capturing devices are optically coincident, and there are overlapping regions between the captured images of the adjacent two image capturing devices;

2. The system according to claim 1, wherein the imaging device comprises: at least two imaging devices, in an adjacent two imaging devices, an optical center of the first imaging device and a second imaging device a mirror surface is disposed on a mid-perpendicular line of the line segment formed by the optical center connection, wherein the mirror surface includes two segments: a total reflection mirror surface and a half-reflex lens surface; the optical center of the first imaging device passes through the total reflection mirror surface and a half The reverse half lens surface forms a virtual optical center, and the virtual optical center coincides with the optical center of the second imaging device;

The first imaging device and the second imaging device form an overlapping region through the semi-reflex lens surface.

The system according to claim 2, wherein an extension line of the line segment formed by the optical center of the first imaging device and the first end point of the semi-reflex lens surface is connected to the second camera An edge of the field of view of the device forms a first intersection, and an extension of the line segment formed by the optical center of the first imaging device and the second end of the semi-reflex surface is formed with the edge of the field of view of the second imaging device a second intersection, the system is configured to prevent light outside the corresponding plane of the first intersection and the second intersection from entering the first imaging device;

There is no object in the closed area consisting of the first end point, the second end point, the second point of intersection and the first point of intersection.

4. The system according to claim 1, wherein the imaging device comprises: at least two imaging devices, wherein in the two adjacent imaging devices, the first imaging device is composed of a convex lens and a first imaging device, The second imaging device is composed of a concave mirror and an optical axis of the convex lens, the optical axis of the convex lens being on the same line as the optical axis of the concave mirror, and the optical center of the convex lens coincides with the optical center of the concave mirror The first imaging device and the second imaging device are photographed by forming a overlapping area by a convex lens and a concave mirror.

5. The system of claim 4 wherein the first imaging device and the second imaging device are photosensitive devices or cameras.

6. The system according to claim 5, wherein the first imaging device and the second imaging device are photosensitive devices, a photosensitive surface of the first imaging device faces the convex lens, and a photosensitive surface of the second imaging device faces The concave mirror has an overlap region between the projection of the first imaging device on a plane of the second imaging device and the second imaging device.

7. The system of claim 6 wherein the back side of the first imaging device is in contact with the back side of the second imaging device.

The system according to any one of claims 1 to 7, wherein the processing device comprises: a splicing module for aligning overlapping regions between captured images of two adjacent imaging devices, Stitch the image.

The system according to claim 8, wherein the processing device further comprises:

And a correction module, configured to correct a brightness and/or a chromaticity difference of the stitched image according to the overlapping area.

10. The system according to claim 9, wherein the correction module comprises:

a determining unit, configured to determine a brightness and/or a chromaticity correction relationship of the overlapping area according to the brightness value and/or the chromaticity value of the partial image or the pixel point in the overlapping area;

And a correction unit, configured to correct a brightness and/or a chromaticity difference of the stitched image according to a brightness and/or a chromaticity correction relationship of the overlap region.

11. An imaging method, comprising:

Each of the imaging devices in the imaging device respectively captures an object, the imaging device includes a plurality of imaging devices, the plurality of imaging devices are optically coincident, and there are overlapping regions between the captured images of the adjacent two imaging devices;

12. The method according to claim 11, wherein the splicing and imaging the captured image of each of the imaging devices in the imaging device comprises:

The overlapping areas between the captured images of the adjacent two imaging devices are aligned to obtain a stitched image.

The method according to claim 12, wherein after the aligning the image is obtained by aligning the overlapping regions between the captured images of the two adjacent imaging devices, the method further includes:

The brightness and/or chromaticity difference of the stitched image is corrected according to the overlap region.

14. The method according to claim 13, wherein the correcting the brightness and/or chrominance difference of the stitched image according to the overlapping area comprises:

Determining a brightness and/or a chromaticity correction relationship of the overlap region according to a luminance value and/or a chrominance value of a partial image or a pixel in the overlap region;

The brightness and/or chromaticity difference of the stitched image is corrected based on the brightness and/or chromaticity correction relationship of the overlap region.