CN1979334A - Image pick-up device of multiple-lens camera for producing full-view image - Google Patents

Abstract

For simplifying the stitching algorithm of generating horizontal panorama, a video pickup device of the invention comprises plural lenses and locating mechanism, where the locating mechanism aligns the intersections of FOV (Field Of View) of all the lenses in the vertical direction. Therefore, the parallax can not exist in the video picked up by the camera system and the stitching points of the objects in different distances are kept constant.

Description

Image pick-up in order to the multiple-lens camera system that produces full-view image

Technical field

The present invention relates to a kind of image pick-up.More particularly, the present invention relates to a kind of image pick-up in order to the multiple-lens camera system that produces full-view image.This image pick-up can be located a plurality of camera lenses in the multiple camera system, is implemented in special IC (ASIC, the Application Specific IntegratedCircuit) solution so that will simply splice operation method.

Background technology

The generation of full-view image needs a plurality of cameras to take simultaneously usually and then uses image processor to come resultant image.Perhaps, also can use single camera in conjunction with pan (panning) thus motor carry out repeatedly to take and the image of each gained spliced and form static full-view image.For example, disclosed full-view image pick device is to adopt pan motor to catch wide-angle image in Jap.P. 11-008845 number and 11-018003 number.Yet pan motor has increased the cost and the size of camera system.Therefore, hope can produce panoramic video with simpler mechanism and simpler splicing operation method.

Summary of the invention

Image pick-up of the present invention is that the point of crossing, visual angle of all camera lenses is aimed at, thereby a kind of fixedly capturing video system of splice point that has is provided, and wherein can produce panoramic video with the simple splicing of ASIC solution enforcement cheaply operation method.

For reaching above-mentioned purpose, the invention provides the image pick-up of a kind of multiple-lens camera system, comprise: N camera lens, wherein the horizontal view angle of each camera lens is respectively HFOV _i(i=1,2 ..., N); And detent mechanism, wherein this detent mechanism rotates θ in the horizontal direction with each camera lens _iDegree (0＜θ _i＜HFOV _i, i=1,2 ..., N-1) and be positioned the top of another camera lens, and this detent mechanism aligns the point of crossing, visual angle of each camera lens in vertical direction.

According to a scheme of the present invention, above-mentioned detent mechanism is with each camera lens  that tilts in vertical direction _iDegree (0＜ _i＜VFOV _i, i=1,2 ..., N).

According to another aspect of the present invention, above-mentioned θ ₁=θ ₂=θ ₃=... θ _N-1

According to another scheme of the present invention, the aggregate level visual angle that an above-mentioned N camera lens is reached equals

$\underset{i=1}{\overset{N}{\mathrm{\Σ}}}\mathrm{HFO}{V}_i-\underset{i=1}{\overset{N}{\mathrm{\Σ}}}(\mathrm{HFO}{V}_{i+1}-{\mathrm{\θ}}_i).$

Description of drawings

Fig. 1 is the synoptic diagram according to N camera lens of image pick-up of the present invention.

Fig. 2 is the synoptic diagram of rotation of lens angle and horizontal view angle (HFOV).

Fig. 3 is the figure of the point of crossing, visual angle of expression camera lens.

Fig. 4 is a figure, the horizontal parallax that its expression is caused by the misalignment (misalignment) of point of crossing, visual angle.

Fig. 5 (a) and Fig. 5 (b) are the figure that is illustrated respectively in the change of the image overlap ratio of near objects and distant objects under the situation of misalignment.

Fig. 6 is a figure, the situation that the point of crossing, visual angle of its expression camera lens is aimed at.

Fig. 7 is a figure, the image offset when camera lens is not tilted in its expression in vertical direction.

Fig. 8 is a figure, the image that its expression is aimed at by the camera lens that tilts in vertical direction.

Fig. 9 is the block diagram of expression according to multiple-lens camera of the present invention system.

The main element symbol description

110 camera lens parts

110A, 110B, 110C camera lens

120A, 120B, 120C soft arranging wire

130 image processing logical blocks

131 multiple-lens image processors

132 splicing logics

133 image processors

134 video encoders

135 mpeg encoders

136 sockets

Embodiment

Fig. 1 with 2,3 and N camera lens be example, expression is according to the image pick-up of multiple-lens camera of the present invention system.This configurations of lenses is to reach by detent mechanism according to the present invention.This detent mechanism can be the part of video-telephone system, in order to produce the wide-angle image of the angle limits that surpasses single lens.Cooperate a kind of ASIC that implements the simple concatenation operation method according to image pick-up of the present invention, thereby the camera system of a kind of low cost, small size and super wide-angle can be provided.

Below, will principle of the present invention be described simply referring to figs. 2 to Fig. 8.

Image pick-up of the present invention comprises N camera lens and detent mechanism.This detent mechanism rotates θ in the horizontal direction with each camera lens _iDegree (0＜θ _i＜HFOV _i, i=1,2 ..., N-1) and be positioned the top of another camera lens.

Fig. 2 is the synoptic diagram of rotation of lens angle and horizontal view angle (HFOV).Suppose in the camera system N camera lens (i=1,2,3 ..., N) have HFOV respectively _iHorizontal view angle and each rotation of lens angle be θ _i(i=1,2,3 ..., N-1), then the aggregate level visual angle HFOVt of system equals $\underset{i=1}{\overset{N}{\mathrm{\Σ}}}\mathrm{HFO}{V}_i-\underset{i=1}{\overset{N}{\mathrm{\Σ}}}(\mathrm{HFO}{V}_{i+1}-{\mathrm{\θ}}_i)$ 。Suppose the HFOV of each camera lens _iBe HFOV and each rotation angle θ _iBe θ, then total HFOVt of system equals HFOV*N-(HFOV-θ) * (N-1).For example, as N=2 (promptly having two camera lenses), HFOV1=HFOV2=60 °, θ ₁In the time of=30 °, total HFOVt equals 90 °; And as N=11 (promptly having 11 camera lenses), HFOV _i=60 ° (i=1,2,3 ..., 11), θ _i=30 ° (i=1,2,3 ..., 10) time, total HFOVt equals 360 °.

Importance of the present invention is that the image of catching can produce panorama sketch with a kind of simple splicing operation method, and its operation method can be used ASIC enforcement cheaply.Because the point of crossing, visual angle of each camera lens is aligned in this system, so that the splice point of the object of different distance is kept is constant.In addition, the rotation angle between each camera lens of camera system can be fixed, so can calculate splice point in calibration (calibration) process when camera is made.Therefore, ASIC need not change owing to the distance of object and each picture is all dynamically calculated splice point.Thereby therefore can reduce the cost that the required calculated amount of splicing reduces ASIC significantly.

In the following description, with the relation between explaining splice point and the point of crossing, visual angle being aimed at.

Fig. 3 represents the point of crossing, visual angle of single lens.Fig. 4 describes because the splicing problem that misalignment caused of point of crossing, visual angle.In the drawings, stpn represents the splice point of near objects; Stpf represents the splice point of distant objects; Dn represents the distance between point of crossing, visual angle and the near objects; Df represents the distance between point of crossing, visual angle and the distant objects; Dth represents the distance of FOV plotted point between the point of crossing, visual angle of single lens and two camera lenses; Wn represents the horizontal view angle width of near objects; Wf represents the horizontal view angle width of distant objects; α represents the angle between border, overlay region and the splice point; And HFOV represents the horizontal view angle.As shown in Figure 4, under the situation of misalignment, within distance D th, there is no image overlap.Suppose that making the center of image overlap is splice point, then also skew thereupon of splice point when the distance between camera and object changes.

Fig. 5 (a) and Fig. 5 (b) are illustrated respectively in the change of the image overlap ratio of near objects and distant objects under the situation of misalignment.Comparison by two figure can find out that the image overlap part (oblique line part) of the near objects shown in Fig. 5 (a) is significantly less than the image overlap part (oblique line part) of the distant objects shown in Fig. 5 (b).

Splice point changes and can be tried to achieve by following arithmetic expression:

For near objects:

$\mathrm{stpn}=2\mathrm{Dn}*\tan \left(\frac{\mathrm{HFO}{V}_i}{2}\right)-(\mathrm{Dn}-\mathrm{Dth})*\tan \mathrm{\α}$

$\mathrm{Wn}=2\mathrm{Dn}*\tan \left(\frac{\mathrm{HFO}{V}_i}{2}\right)$

The number percent of near objects splice point in image is:

$\frac{\mathrm{stpn}}{\mathrm{Wn}}=\frac{2\mathrm{Dn}*\tan \left(\frac{\mathrm{HFO}{V}_i}{2}\right)-(\mathrm{Dn}-\mathrm{Dth})*\tan \mathrm{\α}}{2\mathrm{Dn}*\tan \left(\frac{\mathrm{HFO}{V}_i}{2}\right)}$

For distant objects:

$\mathrm{stpf}=2\mathrm{Df}*\tan \left(\frac{\mathrm{HFO}{V}_i}{2}\right)-(\mathrm{Df}-\mathrm{Dth})*\tan \mathrm{\α}$

$\mathrm{Wf}=2\mathrm{Df}*\tan \left(\frac{\mathrm{HFO}{V}_i}{2}\right)$

The number percent of distant objects splice point in image is:

$\frac{\mathrm{stpf}}{\mathrm{Wf}}=\frac{2\mathrm{Df}*\tan \left(\frac{\mathrm{HFO}{V}_I}{2}\right)-(\mathrm{Df}-\mathrm{Dth})*\tan \mathrm{\α}}{2\mathrm{Df}*\tan \left(\frac{\mathrm{HFO}{V}_i}{2}\right)}$

Therefore,

Fig. 6 represents the situation that the point of crossing, visual angle of camera lens is aligned.In the case, no matter object distance how, splice point all keeps identical.This situation can be explained in following arithmetic expression:

For near objects:

$\mathrm{stpn}=2\mathrm{Dn}*\tan \left(\frac{\mathrm{HFO}{V}_i}{2}\right)-\mathrm{Dn}*\tan \mathrm{\α}$

$\mathrm{Wn}=2\mathrm{Dn}*\tan \left(\frac{\mathrm{HFO}{V}_i}{2}\right)$

The number percent of near objects splice point in image is:

$\frac{\mathrm{stpn}}{\mathrm{Wn}}=\frac{2\mathrm{Dn}*\tan \left(\frac{\mathrm{HFO}{V}_i}{2}\right)-\mathrm{Dn}*\tan \mathrm{\α}}{2\mathrm{Dn}*\tan \left(\frac{\mathrm{HFO}{V}_i}{2}\right)}=\frac{2\tan \left(\frac{\mathrm{HFO}{V}_i}{2}\right)-\tan \mathrm{\α}}{2\tan \left(\frac{\mathrm{HFO}{V}_i}{2}\right)}$

For distant objects:

$\mathrm{stpf}=2\mathrm{Df}*\tan \left(\frac{\mathrm{HFO}{V}_i}{2}\right)-\mathrm{Df}*\tan \mathrm{\α}$

$\mathrm{Wf}=2\mathrm{Df}*\tan \left(\frac{\mathrm{HFO}{V}_i}{2}\right)$

The number percent of distant objects splice point in image is:

$\frac{\mathrm{stpf}}{\mathrm{Wf}}=\frac{2\mathrm{Df}*\tan \left(\frac{\mathrm{HFO}{V}_i}{2}\right)-\mathrm{Df}*\tan \mathrm{\α}}{2\mathrm{Df}*\tan \left(\frac{\mathrm{HFO}{V}_i}{2}\right)}=\frac{2\tan \left(\frac{\mathrm{HFO}{V}_i}{2}\right)-\tan \mathrm{\α}}{2\tan \left(\frac{\mathrm{HFO}{V}_i}{2}\right)}$

Therefore,

$\frac{\mathrm{stpn}}{\mathrm{Wn}}=\frac{\mathrm{stpf}}{\mathrm{Wf}}$

In addition, if the perpendicular displacement at visual angle is arranged, then can be offset to some extent by the image that each camera lens is caught.Fig. 7 explains because the image that perpendicular displacement caused at visual angle does not overlap (non-coinciding).Superposed part does not need to be pruned (cropped) and just can form final full-view image.When N is big more, just there are many more parts to be pruned.For head it off, detent mechanism of the present invention each camera lens  that tilts in vertical direction _iDegree (0＜ _i＜VFOV _i, i=1,2 ..., N).Fig. 8 explains the result who tilts each camera lens in vertical direction and obtain.Attention point of crossing, visual angle when the inclination camera lens is still kept and is aligned.

Therefore, image pick-up of the present invention can provide the image with constant splice point, thereby has simplified the complexity of splicing operation method.

Next, will a embodiment according to multiple-lens camera of the present invention system be described with reference to the block diagram of figure 9.For making instructions of the present invention more succinct and clear, main camera lens part and the relevant image processing program that the multiple-lens camera system is described below described, and the detailed description of having omitted other parts of general camera system.

As shown in Figure 9, camera lens part 110 comprises three camera lens 110A, 110B and 110C, and wherein camera lens 110B is disposed at the top of camera lens 110A and counterclockwise in the horizontal direction deflection angle θ (not being shown in the figure); Camera lens 110C then is disposed at the top of camera lens 110B and the θ of deflection equal angular more counterclockwise in the horizontal direction.The signal of video signal of being caught by camera lens 110A, 110B and 110C sees through soft arranging wire (FFC, Flexible Flat Cable) 120A, 120B and 120C respectively and is conveyed into image processing logical blocks 130 with further execution signal Processing.Image processing logical blocks 130 comprises multiple-lens image processor 131, splicing logical one 32, image processor 133, video encoder 134, mpeg encoder 135 and socket 136.

At first, multiple-lens image processor 131 will carry out rough handling from the signal of video signal that camera lens 110A, 110B and 110C send, and its groundwork is to reduce the difference between the image that each camera lens catches.Signal of video signal after rough handling is sent to splicing logical one 32 respectively.Splicing logical one 32 just carries out the mathematics conversion to each image and merges, thereby each image is seamlessly synthesized single image.This single image is transferred into image processor 133 to carry out traditional image processing.Can be by video encoder 134 coded and be presented at arbitrarily on the display device through image that image processor 133 was handled.Perhaps, through image that image processor 133 was handled also can be by mpeg encoder 135 compressions so that be stored in arbitrarily in the memory storage.At last, the image that is compressed through mpeg encoder 135 can be transferred into world-wide web by socket 136.

The splicing operation method is the part of the maximum calculated amount of loss when producing full-view image.(for example, 30fps), a kind of solution of ASIC cheaply is not sufficient to reach the performance of upgrading splice point in per 1/30 second for the video of high frame frequency (frame rate).The invention discloses a kind of simple and feasible mechanism, a plurality of camera lenses are caught have the image of constant splice point, thereby a kind of low cost, super wide-angle and undersized camera system are provided.

Claims (4)

1. the image pick-up of a multiple-lens camera system comprises:

N camera lens, wherein the horizontal view angle of each camera lens is respectively HFOV _i(i=1,2 ..., N); And

Detent mechanism, wherein this detent mechanism rotates θ in the horizontal direction with each camera lens _iDegree (0＜θ _i＜HFOV _i, i=1,2 ..., N-1) and be positioned the top of another camera lens, and this detent mechanism aligns the point of crossing, visual angle of each camera lens in vertical direction.

2. image pick-up as claimed in claim 1, wherein this detent mechanism is with each camera lens  that tilts in vertical direction _iDegree (0＜ _i＜VFOV _i, i=1,2 ..., N).

3. image pick-up as claimed in claim 1, wherein θ ₁=θ ₂=θ ₃=...=θ _N-1

4. image pick-up as claimed in claim 1, wherein this N camera lens aggregate level visual angle of reaching equals $\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{\mathrm{HFOV}}_i-\underset{i=1}{\overset{N}{\mathrm{\Σ}}}({\mathrm{HFOV}}_{i+1}-{\mathrm{\θ}}_i).$

Images