WO2018082220A1

WO2018082220A1 - Panoramic camera and depth information obtaining method

Info

Publication number: WO2018082220A1
Application number: PCT/CN2017/073983
Authority: WO
Inventors: 孙其民; 李炜
Original assignee: 深圳市掌网科技股份有限公司
Priority date: 2016-11-03
Filing date: 2017-02-17
Publication date: 2018-05-11
Also published as: CN108377327A; CN108377327B

Abstract

The present invention provides a panoramic camera and a depth information obtaining method. The panoramic camera comprises multiple combination type photographing mechanisms (1). The combination type photographing mechanism comprises a first camera (11), a second camera (12), a first processor (13), and a shutter cable module (14), and further comprises a light intensity adjustment module (16) for providing illumination for the second camera. The first processor is further used for controlling a focal length parameter, an aperture parameter, and a shutter parameter of the first camera and the second camera, obtaining a first image photographed by the first camera and multiple second images photographed by the second camera in multiple light intensities, and further obtaining depth information of image areas in the first image according to the correspondence between the light intensity provided by the light intensity adjustment module, and an image depth photographed by the second camera in the light intensity. The technical solutions of the present invention of the panoramic camera and the depth information obtaining method are ingenious and are highly practical.

Description

Description Title: Panoramic camera and depth information acquisition method

[0001] The present invention relates to the field of imaging devices, and in particular, to a panoramic camera and a depth information acquiring method.

Background technique

[0002] With the development of virtual reality technology, higher requirements are placed on the interaction between human and virtual reality scenes. Currently, images of virtual reality scenes based on realistic scenes are captured by a panoramic camera, such as a google map. However, the panoramic image captured by the existing panoramic camera does not have depth information, and the spatial depth and depth are poor, which can only satisfy the requirements for panoramic shooting.

technical problem

[0003] The present invention provides a panoramic camera and a depth information acquisition method for a panoramic image captured by an existing panoramic camera that does not have depth information and has a spatial depth and stereoscopic difference, which can only satisfy the requirement for panoramic photography.

Problem solution

Technical solution

[0004] The present invention provides a panoramic camera including a plurality of combined imaging mechanisms; the plurality of combined imaging mechanisms are distributed on a circumference centered on a point, and the plurality of combined imaging mechanisms are mounted on one The combined camera mechanism includes a first camera, a second camera, a first processor, and a shutter line module; the first camera and the second camera capture the same area; the combined camera mechanism further includes the first The processor is electrically connected to the light intensity adjustment module for providing additional illumination to the second camera; the first processor is further electrically connected to the first camera, the second camera and the shutter release module, respectively, for controlling the first camera and a focal length parameter, an aperture parameter, and a shutter parameter of the second camera, and acquiring, by the trigger of the shutter line module, the first image captured by the first camera and the plurality of images provided by the second camera and the light intensity adjusting module a plurality of second images under light intensity; also for comparing the first image and the plurality of second images, and according to the light intensity adjustment module Correspondence between the image depth for a second intensity of light captured by the camera in the light intensity, thereby acquiring the depth information of each image f π regions in the first image. [0005] In the above panoramic camera of the present invention, the shutter release module includes a first shutter line for triggering the first camera to acquire the first image, and a second shutter line for triggering the second camera to acquire the second image; A shutter line and a second shutter line are electrically connected to the first processor, respectively.

[0006] In the above panoramic camera of the present invention, a plurality of second images captured by the second camera under the N-type light intensity provided by the light intensity adjusting module are denoted as Fl, F2, F3, ..., F(n-1), Fn, ..., F(N-1), FN; n< N; the first image is denoted as F0;

[0007] The first processor is configured to compare F0 and F1, determine an image region in the first image that can be covered by F1; according to the light intensity provided by the light intensity adjustment module and the image depth captured by the second camera at the light intensity Corresponding relationship between the image depth corresponding to the illumination corresponding to F1; determining the image depth corresponding to the illumination corresponding to F1 as the depth information of the image region that can be covered by F1 in the first image;

[0008] for comparing F1 and F2, determining an image region in the first image that can be covered by F2 but not covered by F1; according to the light intensity provided by the light intensity adjustment module and the second camera is photographed at the light intensity Corresponding relationship between image depths determines image depth corresponding to illumination corresponding to F2; determining image depth corresponding to illumination corresponding to F2 as depth information of image regions in the first image that can be covered by F2 but not covered by F1

[0009] for comparing F(nl) and Fn, determining an image region in the first image that can be covered by Fn but not covered by F(nl); according to the light intensity provided by the light intensity adjustment module and the second camera Corresponding relationship between image depths captured under the light intensity determines an image depth corresponding to the illumination corresponding to Fn; determining an image depth corresponding to the illumination corresponding to Fn as being capable of being covered by Fn but not being F (in the first image) Nl) depth information of the covered image area;

[0010] for comparing F(N1) and FN, determining an image region in the first image that can be covered by the FN but not covered by F(N1); according to the light intensity provided by the light intensity adjustment module and the second camera Corresponding relationship between image depths captured under the light intensity determines an image depth corresponding to the illumination corresponding to the FN; determining an image depth corresponding to the illumination corresponding to the FN as being capable of being covered by the FN but not being F (in the first image) Nl) depth information of the image area covered.

[0011] In the above panoramic camera of the present invention, the light intensity adjusting module is configured to provide a plurality of light intensity in a plurality of consecutive segments, and the second camera is configured to respectively capture a plurality of second colors in the plurality of turns image.

[0012] The above panoramic camera of the present invention further includes a second processor, and the second processor is respectively combined with a plurality of The first processor of the imaging mechanism is electrically connected to form a panoramic image by splicing a plurality of first images respectively captured by the first cameras of the plurality of combined imaging mechanisms.

[0013] In the above panoramic camera of the present invention, the light intensity adjusting module is a flash lamp or an external light source.

[0014] The present invention also provides a depth information acquisition method based on the above-mentioned panoramic camera, which is characterized in that it comprises the following steps:

[0015] Step S1: acquiring, in advance, a correspondence between a light intensity provided by the light intensity adjustment module and an image depth captured by the second camera at the light intensity;

[0016] Step S2, using the first camera to capture the first image; and using the second camera to capture the plurality of second images under the plurality of light intensity provided by the light intensity adjustment module;

[0017] Step S3, comparing the first image and the plurality of second images, and according to the correspondence between the light intensity provided by the light intensity adjusting module and the image depth captured by the second camera at the light intensity, thereby obtaining the first Depth information for each image area in an image.

[0018] In the above-described depth information acquisition method of the present invention, a plurality of second images captured by the second camera under the N-type light intensity provided by the light intensity adjustment module are recorded as F1 and F2. F3, ..., F(nl), Fn, ..., F(N-1), FN; the first image is denoted as F0; step S3 further comprises:

[0019] Step S31, comparing F0 and F1, determining an image region in the first image that can be covered by F1; according to the light intensity provided by the light intensity adjustment module and the image depth of the second camera captured under the light intensity The corresponding relationship determines the image depth corresponding to the illumination corresponding to F1; determines the image depth corresponding to the illumination corresponding to F1 as the depth information of the image region that can be covered by F1 in the first image;

[0020] Step S32, comparing F1 and F2, determining an image region in the first image that can be covered by F2 but not covered by F1; according to the light intensity provided by the light intensity adjustment module and the second camera shooting at the light intensity The correspondence between the image depths determines the image depth corresponding to the illumination corresponding to F2; the image depth corresponding to the illumination corresponding to F2 is determined as the depth of the image region in the first image that can be covered by F2 but not covered by F1 Information

[0021] Step S33, comparing F(nl) and Fn, determining an image region in the first image that can be covered by Fn but not covered by F(nl); according to the light intensity provided by the light intensity adjustment module and the second camera Corresponding relationship between image depths captured under the light intensity determines an image depth corresponding to the illumination corresponding to Fn; determining an image depth corresponding to the illumination corresponding to Fn as being capable of being covered by Fn but not being F in the first image (nl) covered image Depth information of the area;

[0022] Step S34, comparing F(N1) and FN, determining an image region in the first image that can be covered by the FN but not covered by F(N1); according to the light intensity provided by the light intensity adjustment module and the second camera Corresponding relationship between image depths captured under the light intensity determines an image depth corresponding to the illumination corresponding to the FN; determining an image depth corresponding to the illumination corresponding to the FN as being capable of being covered by the FN but not being F in the first image (Nl) Depth information of the image area covered.

[0023] In the above method for acquiring depth information, in step S2, the light intensity adjustment module is configured to provide multiple light intensities in a plurality of consecutive segments, and the second camera is used in the plurality of segments Multiple second images are taken separately.

Advantageous effects of the invention

Beneficial effect

[0024] The panoramic camera of the present invention determines depth information of an image region in an image by using a proportional relationship between the light intensity of the image capturing pupil and the area of the effective area of the image. The panoramic camera and the depth information acquisition method of the present invention are technically ingenious and practical.

Brief description of the drawing

DRAWINGS

1 shows a schematic diagram of a panoramic camera of an embodiment of the present invention;

2 is a functional block diagram of the panoramic camera shown in FIG. 1.

Embodiments of the invention

[0027] The technical problem to be solved by the present invention is that the panoramic image captured by the existing panoramic camera does not have depth information, and the spatial depth is poor in stereoscopic appearance, which can only meet the requirements for panoramic shooting. The technical idea proposed by the present invention with respect to this technical problem is to determine the depth information of the image region in the image by using the proportional relationship between the light intensity of the image capturing image and the effective area of the image.

The present invention will be further described in detail with reference to the accompanying drawings and specific embodiments. FIG. 1 and 2, FIG. 1 shows a schematic diagram of a panoramic camera according to an embodiment of the present invention. FIG. 2 is a schematic diagram showing the functional modules of the panoramic camera shown in FIG. 1. The panoramic camera includes a plurality of combined imaging mechanisms 1; the three combined imaging mechanisms 1 shown in FIG. 1 may be part or all of a plurality of combined imaging mechanisms 1; Disposed on a circumference centered on one point, and the plurality of combined camera mechanisms 1 are mounted on a fixed bracket (not shown); each combined camera mechanism 1 is responsible for photographing one of the 360° fields of view Angle (this angle depends on the orientation of the combined camera mechanism 1).

Further, as shown in FIG. 2, each of the combined camera mechanisms includes a first camera 11 and a second camera 12, and the specifications of the first camera 11 and the second camera 12 are the same. The first camera 11 and the second camera 12 are the same; the first camera 11 and the second camera 12 may be disposed on the left or the right, or may be arranged in the upper or lower position, or in other positional relationships; as shown in FIG. 2, the combination The camera mechanism further includes a first processor 13 and a shutter line module 14; the combined camera mechanism 1 further includes a light intensity adjustment module 16 electrically connected to the first processor 13 for providing additional illumination to the second camera 12; A processor 13 is further electrically connected to the first camera 11 , the second camera 12 and the shutter line module 14 for controlling the focal length parameter, the aperture parameter and the shutter parameter of the first camera 1 1 and the second camera 12 , and The triggering of the shutter release module 14 acquires the first image captured by the first camera 11 and the plurality of second images captured by the second camera 12 under various light intensities provided by the light intensity adjustment module 16; And comparing the first image and the plurality of second images according to the light intensity provided by the light intensity adjusting module 16 and the image captured by the second camera 12 at the light intensity Correspondence between the degree, thereby acquiring the depth information of each image region in the first image. Here, the three-dimensional reconstruction of the virtual reality scene can be realized according to the depth information of each image region in the first image and by the existing depth image registration method. In addition, in the embodiment, the light-emphasizing module 16 is a flash, and the light intensity used by the second camera 12 is realized by the flash compensation illumination. It can be understood that in other embodiments, the light intensity employed by the second camera 12 can also be achieved by an external light source. In this embodiment, the shutter release module 14 includes a first shutter line 141 for triggering the first camera 11 to acquire the first image, and a second shutter line 142 for triggering the second camera 12 to acquire the second image; A shutter release 141 and a second shutter release 142 are electrically connected to the first processor 13, respectively.

[0031] The present invention also provides a method for acquiring object depth information based on the above panoramic camera, comprising the following steps:

[0032] Step S1, pre-acquiring the light intensity provided by the light intensity adjusting module 16 and the second camera 12 to shoot under the light intensity Corresponding relationship between image depths taken;

[0033] Step S2, using the first camera 11 to capture the first image; and using the second camera 12 to capture a plurality of second images under various light intensities provided by the light intensity adjustment module 16; The light intensity adjustment module 16 provides a plurality of light intensities in a plurality of consecutive segments, and the second camera (12) performs a shooting task by capturing a second image in the plurality of segments. At the same time, in order to more accurately determine the depth information of the image area, the light intensity can be more precisely defined.

[0034] Step S3, comparing the first image and the plurality of second images, and according to the correspondence between the light intensity provided by the light intensity adjusting module 16 and the image depth captured by the second camera 12 under the light intensity, thereby The depth information of each image area in the first image is acquired. In this step, the light intensity is proportional to the effective area of the effective image captured by the second camera 12, that is, the stronger the light intensity, the greater the depth of the image captured by the second camera 12 at the light intensity.

[0035] In order to facilitate the explanation of step S3, in the present embodiment, a plurality of first images taken by the second camera 12 under the N-type light intensity provided by the light intensity adjusting module 16 will be used. The two images are correspondingly denoted as Fl, F2, F3, ..., F(n-1), Fn, ..., F(N-1), FN; the first image is denoted as F0; step S3 further includes

[0036] Step S31, comparing F0 and F1, determining an image region in the first image that can be covered by F1; according to the light intensity provided by the light intensity adjustment module 16 and the image depth captured by the second camera 12 at the light intensity Corresponding relationship between the image depth corresponding to the illumination corresponding to F1; determining the image depth corresponding to the illumination corresponding to F1 as the depth information of the image region that can be covered by F1 in the first image; Here, the meaning of the overlay It means that there is an intersection; the image area covered by the second image refers to an image area in the first image corresponding to the image area whose resolution of the second image reaches the first image level. The evaluation of the sharpness of the image can be performed by spatial domain parameter variance, entropy, and frequency domain modulation transfer function MTF, etc., or manually by using multiple evaluators.

[0037] Step S32, comparing F1 and F2, determining an image region in the first image that can be covered by F2 but not covered by F1; according to the light intensity provided by the light intensity adjustment module 16 and the light intensity of the second camera 12 Corresponding relationship between image depths taken under the next image determines the image depth corresponding to the illumination corresponding to F2; determining the image depth corresponding to the illumination corresponding to F2 as the image region in the first image that can be covered by F2 but not covered by F1 Depth information [0038] Step S33, comparing F(nl) and Fn, determining an image region in the first image that can be covered by Fn but not covered by F(nl); according to the light intensity provided by the light intensity adjustment module 16 and the second Corresponding relationship between image depths captured by the camera 12 under the light intensity determines an image depth corresponding to the illumination corresponding to Fn; determining an image depth corresponding to the illumination corresponding to Fn as being capable of being covered by Fn but not in the first image Depth information of an image area covered by F(nl);

[0039] Step S34, comparing F(N1) and FN, determining an image region in the first image that can be covered by the FN but not covered by F(N1); according to the light intensity provided by the light intensity adjustment module 16 and the second Corresponding relationship between image depths captured by the camera 12 under the light intensity determines an image depth corresponding to the illumination corresponding to the FN; determining an image depth corresponding to the illumination corresponding to the FN as being capable of being covered by the FN but not in the first image Depth information of the image area covered by F(Nl).

[0040] Thus, step S31 - step S34 is performed by the first processor 13 and completed; by step S31 - step S34, depth information of each image region in the first image can be obtained. Here, the depth information of the object, the space, or the person occupying the image area can be obtained from the depth information of each image area.

[0041] Further, the panoramic camera further includes a second processor 15 electrically connected to the first processor 13 of the plurality of combined camera mechanisms 1 for pairing The plurality of first images respectively captured by the first camera 11 of the imaging unit 1 are spliced to form a panoramic image.

[0042] A preliminary model of the virtual reality can be established by the panoramic camera and the depth information acquisition method, and thus can provide a more realistic stereoscopic feeling and a spatial distance sense for the virtual reality scene. The technical solution of the present invention can be applied to the virtual reality real-world application in the three-dimensional modeling information generated by the real environment; compared with the traditional stereo image information, the three-dimensional modeling information provided by the proposal with depth information can be more Users provide a true sense of spatial distance. In turn, the interactive technology of virtual reality can be combined to realize the interaction between people and the environment.

Industrial applicability

[0043] The panoramic camera of the present invention determines depth information of an image region in an image by using a proportional relationship between the light intensity of the image capturing pupil and the area of the effective area of the image. The panoramic camera and the depth information acquisition method of the present invention are technically ingenious and practical.

Claims

Claim

[Claim 1] A panoramic camera, comprising: a plurality of combined image pickup mechanisms (1); the plurality of combined image pickup mechanisms (1) are distributed on a circumference centered on a point, and the plurality of The combined camera mechanism (1) is mounted on a fixed bracket; the combined camera mechanism (1) includes a first camera (11), a second camera (12), a first processor (13) and a shutter release module (14); the first camera (11) and the second camera (12) capture the same area; the combined camera mechanism (1) further includes an electrical connection with the first processor (13) for the second camera (12) A light intensity adjustment module (16) providing additional illumination; the first processor (13) is also electrically connected to the first camera (11), the second camera (12) and the shutter release module (14), respectively. Controlling a focus parameter, an aperture parameter, and a shutter parameter of the first camera (11) and the second camera (12), and acquiring a first image captured by the first camera (11) via triggering of the shutter line module (14) And by the second shot a plurality of second images captured by the head (12) under various light intensities provided by the light intensity adjusting module (16); also for comparing the first image and the plurality of second images, and adjusting the module according to the light intensity (16) Corresponding relationship between the provided light intensity and the depth of the image captured by the second camera (12) at the light intensity, thereby acquiring depth information of each image region in the first image.

[Claim 2] The panoramic camera according to claim 1, wherein the shutter release module (14) includes a first shutter line (141) for triggering the first camera (11) to acquire the first image, and A second shutter line (142) for triggering the second camera (12) to acquire the second image. The first shutter line (141) and the second shutter line (142) are electrically connected to the first processor (13), respectively.

[Claim 3] The panoramic camera according to claim 1, wherein the light intensity adjustment module (16)

The plurality of second images captured by the second camera (12) under the N light intensity provided from small to large are denoted as Fl, F2, F3, ..., F(n-1), Fn, . .., F(N-1), FN; n< N; the first image is denoted as F0;

The first processor (13) is configured to compare F0 and F1, determine an image area in the first image that can be covered by F1; according to the light intensity provided by the light intensity adjustment module (16) and the second camera (12) a correspondence between image depths captured under the light intensity determines an image depth corresponding to the illumination corresponding to F1; determining an image depth corresponding to the illumination corresponding to F1 as being capable of being covered by F1 in the first image Depth information of the image area;

For comparing F1 and F2, determining an image area in the first image that can be covered by F2 but not covered by F1; according to the light intensity provided by the light intensity adjustment module (16) and the second camera (12) at the light intensity Corresponding relationship between image depths taken under the next image determines the image depth corresponding to the illumination corresponding to F2; determining the image depth corresponding to the illumination corresponding to F2 as the image region in the first image that can be covered by F2 but not covered by F1 Depth information

For comparing F(n-1) and Fn, determining an image region in the first image that can be covered by Fn but not covered by F(n-1); according to the intensity provided by the light intensity adjustment module (16) Corresponding relationship between the image depths captured by the second camera (12) under the light intensity determines the image depth corresponding to the illumination corresponding to Fn; determining the image depth corresponding to the illumination corresponding to Fn as the first image can be Depth information of an image area covered by Fn but not covered by F(nl); for comparing F(N-1) and FN, determining that the first image can be covered by FN but not by F(N-1) An image area; determining an image depth corresponding to the illumination corresponding to the FN according to a correspondence between a light intensity provided by the light intensity adjustment module (16) and an image depth captured by the second camera (12) at the light intensity; The image depth corresponding to the illumination corresponding to the FN is determined as the depth information of the image region in the first image that can be covered by the FN but not covered by F(N1).

The panoramic camera according to claim 3, wherein the light intensity adjusting module (16) is configured to provide a plurality of light intensities in a plurality of consecutive segments, and the second camera (12) is used in the plurality of frames A plurality of second images are respectively taken in the interval.

The panoramic camera according to claim 1, further comprising a second processor (15), the second processor (15) and the first processor of the plurality of combined camera mechanisms (1) 13) An electrical connection for splicing a plurality of first images respectively captured by the first cameras (11) of the plurality of combined imaging mechanisms (1) to form a panoramic image. The panoramic camera of claim 1 wherein the light intensity adjustment module (16) is a flash or an external light source. [Claim 7] The method for acquiring depth information of a panoramic camera according to claim 1, comprising the steps of:

Step S1, pre-acquiring the correspondence between the light intensity provided by the light intensity adjusting module (16) and the image depth captured by the second camera (12) under the light intensity; Step S2, shooting with the first camera (11) a first image; and using the second camera (12) to capture a plurality of second images under various light intensity provided by the light intensity adjustment module (16); step S3, comparing the first image and the plurality of second images, and According to the correspondence between the light intensity provided by the light intensity adjusting module (16) and the image depth captured by the second camera (12) under the light intensity, the depth information of each image region in the first image is acquired.

[Claim 8] The depth information acquisition method according to claim 7, wherein the second camera (12) is used under the N-type light intensity provided by the light intensity adjustment module (16) The plurality of second images taken are correspondingly denoted as F1, F2, F3, ..., F(n-1), Fn, ..., F(Nl), FN; n<N; the first image is denoted as F0; S3 also includes:

Step S31, comparing F0 and Fl, determining an image region in the first image that can be covered by F1; according to the light intensity provided by the light intensity adjustment module (16) and the image captured by the second camera (12) under the light intensity Corresponding relationship between depths determines an image depth corresponding to illumination corresponding to F1; determining an image depth corresponding to illumination corresponding to F1 as depth information of an image region that can be covered by F1 in the first image;

Step S32, comparing F1 and F2, determining an image region in the first image that can be covered by F2 but not covered by F1; according to the light intensity provided by the light intensity adjustment module (16) and the second camera (12) in the light The correspondence between the depths of the images taken under the strong image determines the image depth corresponding to the illumination corresponding to F2; the image depth corresponding to the illumination corresponding to F2 is determined as the image in the first image that can be covered by F2 but not covered by F1 Depth information of the region; Step S33, comparing F(nl) and Fn, determining an image region in the first image that can be covered by Fn but not covered by F(n-1); according to the light intensity adjustment module (16) Corresponding relationship between the light intensity and the depth of the image captured by the second camera (12) at the light intensity determines the image depth corresponding to the illumination corresponding to Fn; determining the image depth corresponding to the illumination corresponding to Fn as the first image Depth information of an image region that can be covered by Fn but not covered by F(nl) Step S34, comparing F(N1) and FN, determining an image region in the first image that can be covered by the FN but not covered by F(N1); according to the light intensity provided by the light intensity adjustment module (16) and the second camera (12) a correspondence between image depths captured under the light intensity determines an image depth corresponding to the illumination corresponding to the FN; determining an image depth corresponding to the illumination corresponding to the FN as being capable of being covered by the FN in the first image but The depth of the image area that is not covered by F(Nl) is f π.

[Claim 9] The depth information acquisition method according to claim 7, wherein in step S2

The intensity adjustment module (16) is configured to provide a plurality of light intensities in a plurality of consecutive segments, and the second camera (12) is configured to respectively capture a plurality of second images in the plurality of segments.