CN101032161A - Camera module and electronic apparatus provided with it - Google Patents

Camera module and electronic apparatus provided with it Download PDF

Info

Publication number
CN101032161A
CN101032161A CN 200580032738 CN200580032738A CN101032161A CN 101032161 A CN101032161 A CN 101032161A CN 200580032738 CN200580032738 CN 200580032738 CN 200580032738 A CN200580032738 A CN 200580032738A CN 101032161 A CN101032161 A CN 101032161A
Authority
CN
China
Prior art keywords
parallax
mentioned
image pickup
pickup signal
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN 200580032738
Other languages
Chinese (zh)
Other versions
CN100546345C (en
Inventor
饭岛友邦
石川隆
沼井郁夫
早川正人
高井亚季
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Panasonic Holdings Corp
Original Assignee
Matsushita Electric Industrial Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Matsushita Electric Industrial Co Ltd filed Critical Matsushita Electric Industrial Co Ltd
Publication of CN101032161A publication Critical patent/CN101032161A/en
Application granted granted Critical
Publication of CN100546345C publication Critical patent/CN100546345C/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Studio Devices (AREA)

Abstract

A camera module capable of being thinned and obtaining a clear image over the entire image area independent of a distance to an object. The camera module comprises a plurality of lens units each containing at least one lens, a plurality of imaging areas corresponding to the plurality of lens units one-to-one and provided respectively with light reception surfaces almost vertical to the light axes of the corresponding lens units, an imaging signal input unit that receives a plurality of imaging signals respectively output from the plurality of imaging areas, a block dividing unit that divides into a plurality of blocks at least one imaging signal out of the plurality of imaging signals, a parallax computing unit that computes, for each block, inter-image parallaxes formed respectively by the plurality of lens units, and a parallax correcting unit that corrects the plurality of imaging signals based on the parallaxes and synthesizes an image.

Description

Video camera module and electronic equipment with it
Technical field
The present invention relates to small-sized and slim, as to have auto-focus controlled function video camera module and electronic equipment with this video camera module.
Background technology
As video camera module in the past, the video camera module of putting down in writing in the TOHKEMY 2001-78213 communique is arranged.Figure 47 is the cutaway view that is illustrated in the structure of the video camera module of putting down in writing in the TOHKEMY 2001-78213 communique.
In Figure 47, camera system 9010 is that the light from object is imaged on optical processing system on the shooting face of imaging apparatus 9120 via aperture 9110 and imaging lens system 9100.Aperture 9110 has 3 circular open 9110a, 9110b, 9110c.The object light that incides respectively from opening 9110a, 9110b, 9110c on the light entrance face 9100e that leads imaging lens system 9100 penetrates from 3 lens section 9100a, 9100b, the 9100c of imaging lens system 9100, forms 3 object pictures on the photographic plane of imaging apparatus 9120.
On the planar portions 9100d of the light emergence face of imaging lens system 9100, be formed with photomask.On the light entrance face 9100c of imaging lens system 9100, be formed with light transmissive 3 optical light filter 9052a, the 9052b, the 9052c that make different mutually wave-length coverages.In addition, on imaging apparatus 9120, on 3 camera watch region 9120a, 9120b corresponding respectively to 3 lens section 9100a, 9100b of imaging lens system 9100,9100c, 9120c, also be formed with light transmissive 3 optical light filter 9053a, the 9053b, the 9053c that make different mutually wave-length coverages.
Optical light filter 9052a and optical light filter 9053a have the spectral-transmission favtor characteristic of main transmit green (representing with G), optical light filter 9052b and optical light filter 9053b have the spectral-transmission favtor characteristic of main transmits red (representing with R), and then optical light filter 9052c and optical light filter 9053c have the spectral-transmission favtor characteristic of main transmission blueness (representing with B).Therefore, camera watch region 9120a has photosensitivity to green light (G), and camera watch region 9120b has photosensitivity to red light (R), and camera watch region 9120c has photosensitivity to blue light (B).
In having the video camera module of so a plurality of imaging lens systems, if from the video camera module to object the variable in distance of (subject), the space that then a plurality of imaging lens systems are respectively formed at a plurality of object pictures on the shooting face of imaging apparatus 9120 changes.
In the video camera module of in TOHKEMY 2001-78213 communique, putting down in writing, with virtual subject distance D [m] as the photography view angle theta of a plurality of imaging systems [°] function and when being defined as D=1.4/ (tan (θ/2)), the optical axis of setting a plurality of imaging systems at interval so that the variation between the space of the space of a plurality of object pictures of object when being in virtual subject distance D [m] and a plurality of object pictures when being in infinity, than the 2 times little of pel spacing of benchmark image signal.Promptly, even the video camera module put down in writing in the TOHKEMY 2001-78213 communique is owing to set optical axis at interval so that the identical image processing of image processing with the photography of the object that is suitable for being in virtual subject distance D [m] is most carried out in the object photography that is in infinity, the difference of the space of the two articles picture on the photographic plane is also than the 2 times little of pel spacing of reference signal, so the color fringe that is in the object picture of infinity can be suppressed at admissible level.
In recent years, the portable equipments such as portable phone that are mounted with video camera are popularized gradually.Along with miniaturization, slimming and the high performance of these portable equipments, require the video camera module of miniaturization, slimming and high performance.For example, requirement has the auto-focus controlled function, landscape photography (the roughly photography of the object of infinity) and personage photography (photography of the object of common several meters distances) not only can be carried out, macroshot (photography of the object of several centimetres to tens centimetres distance) can also be carried out.
But, in the video camera module of in TOHKEMY 2001-78213 communique, putting down in writing, by have a plurality of lens section 9100a, 9100b, 9100c have realized slimming.But this video camera module does not in the past have the auto-focus controlled function.In addition, virtual subject distance D [m] is to set on the basis of considering personage's photography.Therefore, in this camera assembly in the past,, can not tackle macroshot though can tackle landscape photography and personage's photography.
Video camera module in the past as the unsolved problem that solves the video camera module of putting down in writing in the TOHKEMY 2001-78213 communique has the video camera module of putting down in writing in the TOHKEMY 2002-330332 communique.Figure 48 represents by an example of the image of the video camera module photography of putting down in writing in the TOHKEMY 2002-330332 communique and the zonule of cutting apart.For photographic picture, its central portion k0 and its peripheral surveyed area k1, k2, k3, k4 are set.And obtain parallax p0, p1, p2, p3, the p4 of each surveyed area.And, from these parallaxes, extract the parallax of prescribed limit, at the parallax that is extracted a plurality of situations is arranged, the parallax in the nearer zone of chosen distance central authorities.Carry out the correction of photographic picture integral body again with selected parallax.
In the video camera module of in TOHKEMY 2002-330332 communique, putting down in writing in the past, the part of image of photography is made as a plurality of zonules, obtain these regional parallaxes, from these parallaxes, select 1, based on the whole photographs of parallax correction of this selection.For example, as shown in figure 48, leave the personage M about 2m from video camera and leave under the situation that the personage N about 10m exists together in the central authorities of photographic picture in the end of photographic picture, the personage M that will be positioned at photographic picture central authorities regards subject as, revises whole photographs based on the parallax p0 among the k0 of this zone.At this moment, because the parallax of personage M is identical with selected parallax p0,, in the zone of personage M, obtain distinct image so can revise the influence of parallax.But, because the parallax of personage N is different with selected parallax p0, the influence that can not revise parallax, so, can not become distinct image in the region generating form and aspect inequality of personage N.
Summary of the invention
The present invention makes in view of the above problems, and though purpose provide a kind of can miniaturization, how slimming subject distance can both obtain the video camera module of distinct image in the general image zone.
Video camera module of the present invention has: a plurality of lens sections comprise 1 lens respectively at least; A plurality of camera watch regions, corresponding one by one with above-mentioned a plurality of lens sections, have sensitive surface respectively with respect to the optical axis direction approximate vertical of pairing said lens portion; The image pickup signal input part, a plurality of image pickup signals that input is exported respectively from above-mentioned a plurality of camera watch regions; The piece cutting part is divided into a plurality of with at least 1 image pickup signal in above-mentioned a plurality of image pickup signals; The parallax operational part utilizes above-mentioned image pickup signal, the parallax between the picture that the above-mentioned a plurality of lens sections of each above-mentioned computing are formed respectively; And the parallax correction portion, according to the above-mentioned a plurality of image pickup signals of above-mentioned parallax correction, and composograph.
Relevant program of the present invention is the program of the action of the above-mentioned image processing part of control in following video camera module, and this video camera module has: a plurality of lens sections comprise 1 lens respectively at least; A plurality of camera watch regions, corresponding one by one with above-mentioned a plurality of lens sections, and have sensitive surface respectively with respect to the optical axis direction approximate vertical of pairing said lens portion; The image pickup signal input part, a plurality of image pickup signals that input is exported respectively from above-mentioned a plurality of camera watch regions; And image processing part, carrying out the image processing of the image pickup signal imported, said procedure is characterised in that above-mentioned image processing part is carried out: the piece dividing processing is divided into a plurality of with at least 1 image pickup signal in above-mentioned a plurality of image pickup signals; The parallax calculation process is utilized above-mentioned image pickup signal, the parallax between the picture that the above-mentioned a plurality of lens sections of each above-mentioned computing are formed respectively; And the parallax correcting process, according to the above-mentioned a plurality of image pickup signals of above-mentioned parallax correction, and composograph.
Relevant program recorded medium of the present invention is to record recording medium said procedure, that can be read by computer.
The invention effect
The present invention makes in view of the above problems, can miniaturization no matter can provide, how slimming and subject distance can both obtain the video camera module of distinct image in the general image zone.
Description of drawings
Fig. 1 is the cutaway view of structure of the video camera module of the relevant embodiments of the present invention 1 of expression.
Fig. 2 is the vertical view of lens of the video camera module of relevant embodiments of the present invention 1.
Fig. 3 is the vertical view of circuit part of the video camera module of relevant embodiments of the present invention 1.
Fig. 4 is the vertical view of colour filter of the video camera module of relevant embodiments of the present invention 1.
Fig. 5 is the vertical view of IR filter of the video camera module of relevant embodiments of the present invention 1.
Fig. 6 is the figure that is used for illustrating the position of the object picture that is in infinity in the video camera module of relevant embodiments of the present invention 1.
Fig. 7 is the figure of position that is used for illustrating the object picture of the position that is in limited distance in the video camera module of relevant embodiments of the present invention 1.
Fig. 8 (A) be explanation in the video camera module of relevant embodiments of the present invention 1, the figure of the image in when focusing and the relation of contrast evaluation of estimate, Fig. 8 (B) is the image of explanation in the video camera module of relevant embodiments of the present invention 1, during non-focusing the and the figure of the relation of contrast evaluation of estimate.
Fig. 9 is the figure of the relation of the lens position of video camera module of the relevant embodiments of the present invention 1 of explanation and contrast evaluation of estimate.
Figure 10 is the block diagram of the video camera module of relevant embodiments of the present invention 1.
Figure 11 is the flow chart of action of the video camera module of the relevant embodiments of the present invention 1 of expression.
Figure 12 is the figure of coordinate of image pickup signal of the video camera module of the relevant embodiments of the present invention 1 of explanation.
Figure 13 (A) is the original image of rim detection of the video camera module of the relevant embodiments of the present invention 1 of explanation, Figure 13 (B) be the rim detection of rim detection of video camera module of the relevant embodiments of the present invention 1 of explanation with the image of contrast evaluation of estimate, Figure 13 (C) is the image at edge of rim detection of the video camera module of the relevant embodiments of the present invention 1 of explanation.
Figure 14 be explanation in the video camera module of relevant embodiments of the present invention 1, the figure in the computing zone of parallax evaluation of estimate.
Figure 15 is the figure of the relation of the parallax of video camera module of the relevant embodiments of the present invention 1 of explanation and parallax evaluation of estimate.
Figure 16 (A) is the explanation actuator operated amount of contrast evaluation of estimate when maximum and figure of the relation of contrast evaluation of estimate in the video camera module of relevant embodiments of the present invention 1, when not revising, the actuator operated amount that Figure 16 (B) is an explanation positive side revised contrast evaluation of estimate when maximum and the figure of the relation of contrast evaluation of estimate, Figure 16 (C) is the actuator operated amount of the revised contrast evaluation of estimate of explanation minus side during for maximum and the figure of the relation of contrast evaluation of estimate.
The actuator operated amount that Figure 17 (A) is the contrast evaluation of estimate of explanation in the video camera module of relevant embodiments of the present invention 1, when having carried out further positive side correction, before this positive side correction when maximum and the figure of the relation of contrast evaluation of estimate, Figure 17 (B) be when having carried out further positive side correction, this positive side revised contrast evaluation of estimate actuator operated amount during for maximum and the figure of the relation of contrast evaluation of estimate.
The actuator operated amount that Figure 18 (A) is the contrast evaluation of estimate of explanation in the video camera module of relevant embodiments of the present invention 1, when having carried out further minus side correction, before this minus side correction when maximum and the figure of the relation of contrast evaluation of estimate, Figure 18 (B) be when having carried out further minus side correction, the actuator operated amount of the revised contrast evaluation of estimate of this minus side during and the figure of the relation of contrast evaluation of estimate for maximum.
Figure 19 is the cutaway view of structure of the video camera module of the relevant embodiments of the present invention 2 of expression.
Figure 20 is the block diagram of the video camera module of relevant embodiments of the present invention 2.
Figure 21 is the flow chart of action of the video camera module of the relevant embodiments of the present invention 2 of expression.
Figure 22 is the figure of coordinate of image pickup signal of the video camera module of the relevant embodiments of the present invention 2 of explanation.
Figure 23 is the cutaway view of structure of the video camera module of the relevant embodiments of the present invention 3 of expression.
Figure 24 is the block diagram of the video camera module of relevant embodiments of the present invention 3.
Figure 25 is the flow chart of action of the video camera module of the relevant embodiments of the present invention 3 of expression.
Figure 26 is the flow chart of action of the auto-focus control of the relevant embodiments of the present invention 3 of expression.
Figure 27 is the flow chart of action of the parallax correction of the relevant embodiments of the present invention 3 of expression.
Figure 28 is the explanation figure that piece is cut apart in the video camera module of relevant embodiments of the present invention 3.
Figure 29 is the figure in explanation computing zone of parallax evaluation of estimate in the video camera module of relevant embodiments of the present invention 3.
Figure 30 is the figure that the computing zone of correlation is used in explanation parallax precision evaluation in the video camera module of relevant embodiments of the present invention 3.
Figure 31 (A) is the figure of explanation state before the cutting apart again of piece in the video camera module of relevant embodiments of the present invention 3, and Figure 31 (B) is the figure of explanation state after the cutting apart again of piece in the video camera module of relevant embodiments of the present invention 3.
Figure 32 is the figure that the relation of correlation and parallax correcting mode sign is used in explanation parallax precision evaluation in the video camera module of relevant embodiments of the present invention 3.
Figure 33 is the figure that the relation of contrast evaluation of estimate and parallax correcting mode sign is used in explanation parallax precision evaluation in the video camera module of relevant embodiments of the present invention 3.
Figure 34 is the low relevant flow chart of using the action of parallax correction portion of the relevant embodiments of the present invention 3 of expression.
Figure 35 (A) is the low relevant original image of using the rim detection of parallax correction portion of the video camera module of the relevant embodiments of the present invention 3 of explanation, Figure 35 (B) be the rim detection of low relevant rim detection with the parallax correction portion of video camera module of the relevant embodiments of the present invention 3 of explanation with the image of contrast evaluation of estimate, Figure 35 (C) is the image at edge of low relevant rim detection with the parallax correction portion of the video camera module of the relevant embodiments of the present invention 3 of explanation.
Figure 36 is the cutaway view of structure of the video camera module of the relevant embodiments of the present invention 4 of expression.
Figure 37 is the block diagram of the video camera module of relevant embodiments of the present invention 4.
Figure 38 (A)～(D) is the figure of field picture of the video camera module of the relevant embodiments of the present invention 4 of explanation.
Figure 39 is the flow chart of action of the video camera module of the relevant embodiments of the present invention 4 of expression.
Figure 40 is the sequential chart of action of the video camera module of the relevant embodiments of the present invention 4 of expression.
Figure 41 (A)～(C) is the vertical view of preferred disposition example of colour filter of schematically representing the video camera module of relevant embodiments of the present invention 4.
Figure 42 is the cutaway view of structure of the video camera module of the relevant embodiments of the present invention 5 of expression.
Figure 43 is the block diagram of the video camera module of relevant embodiments of the present invention 5.
Figure 44 is the flow chart of action of the video camera module of the relevant embodiments of the present invention 5 of expression.
Figure 45 (A)～(C) is the figure that the background of the video camera module of the relevant embodiments of the present invention 5 of explanation is replaced.
Figure 46 (A) and Figure 46 (B) are the figure of an example of the electronic equipment of the relevant embodiments of the present invention 6 of expression.
Figure 47 is the cutaway view of structure of representing in the past the video camera module of (putting down in writing in the TOHKEMY 2001-78213 communique).
Figure 48 is the figure of an example of the image of the video camera module shooting (put down in writing in the TOHKEMY 2002-330332 communique) by in the past of expression.
Embodiment
Video camera module of the present invention has: a plurality of lens sections comprise 1 lens respectively at least; A plurality of camera watch regions, corresponding one by one with above-mentioned a plurality of lens sections, have sensitive surface respectively with respect to the optical axis direction approximate vertical of pairing said lens portion; The image pickup signal input part, a plurality of image pickup signals that input is exported respectively from above-mentioned a plurality of camera watch regions; The piece cutting part is divided into a plurality of with at least 1 image pickup signal in above-mentioned a plurality of image pickup signals; The parallax operational part utilizes above-mentioned image pickup signal, the parallax between the picture that the above-mentioned a plurality of lens sections of each above-mentioned computing are formed respectively; And the parallax correction portion, according to the above-mentioned a plurality of image pickup signals of above-mentioned parallax correction, and composograph.
The position of object picture relatively changes according to the subject distance.That is, if the subject distance diminishes, then parallax becomes big.Therefore, if the different a plurality of subjects of adjusting the distance are photographed simultaneously, then parallax is all different to each subject.According to video camera module of the present invention, the parallax of each piece of computing, so that after the influence of parallax reduces, it is synthetic to carry out image according to the parallax correction image pickup signal of this each piece.Thus, though the different a plurality of subjects of adjusting the distance make a video recording simultaneously, also can suitably revise each subject parallax, obtain in the general image zone, having reduced the distinct image of the influence of parallax.
In the video camera module of the invention described above, preferably, also have: actuator changes the relative distance of above-mentioned a plurality of lens section and above-mentioned a plurality of camera watch regions; And focal point control portion, according to above-mentioned parallax control actuator.
According to this preferred construction, owing to make actuator action, utilize auto-focus control to focus, so can carry out auto-focus control at high speed by 1 time actuator action according to parallax.
In the video camera module of the invention described above, preferably, also have the contrast operational part, this contrast operational part is according at least 1 image pickup signal computing contrast in above-mentioned a plurality of image pickup signals; Above-mentioned focal point control portion controls above-mentioned actuator according to above-mentioned parallax and above-mentioned contrast.
According to this preferred construction, because dual-purpose auto-focus control and high-precision auto-focus control at a high speed, so can be at high speed and carry out auto-focus accurately and control based on contrast based on parallax.
In the video camera module of the invention described above, preferably, above-mentioned focal point control portion repeatedly controls above-mentioned actuator, controls above-mentioned actuator according to above-mentioned parallax the 1st time, controls above-mentioned actuator according to above-mentioned contrast after the 2nd time.
According to this preferred construction, make actuator action according to parallax at first, carry out auto-focus control.Then, make actuator action, carry out auto-focus control according to contrast.Control owing to the actuator action by 1 time is focused, so at a high speed based on the auto-focus of parallax.On the other hand, owing to directly judge focusing according to image, precision is better so not influenced by the deviation etc. of actuator based on the control of the auto-focus of contrast.Thereby, initial utilize auto-focus control carrying out at high speed coarse adjustment, then utilize based on the auto-focus control of contrast and carry out high-precision fine adjustment, so can be at high speed and carry out auto-focus control accurately based on parallax.
In the video camera module of the invention described above, preferably, the operational ton the when study of above-mentioned focal point control portion is controlled above-mentioned actuator according to above-mentioned contrast to above-mentioned actuator.
According to this preferred construction,, revise the such study of operational ton function according to the actuator operated amount of in auto-focus control, carrying out based on contrast.Thus, even deviation that actuator is arranged etc. also can more correctly be revised the operational ton function, so the first auto-focus control based on parallax becomes more accurate, the fine adjustment number of times that next time, the later auto-focus based on contrast was controlled reduces, so can realize auto-focus control more at a high speed.
In the video camera module of the invention described above, preferably, above-mentioned cutting part detects the profile of a plurality of image-regions according to above-mentioned at least 1 image pickup signal, above-mentioned at least 1 image pickup signal is divided into a plurality of, so that utilize above-mentioned profile to be divided into above-mentioned a plurality of image-region.
According to this preferred construction, detect profile, be divided into piece, so that after the influence of parallax reduces, it is synthetic to carry out image based on the parallax correction image pickup signal of each piece.Thus, can suitably carry out blocking, so can access the distinct image that in the general image zone, has reduced the influence of parallax.
In the video camera module of the invention described above, preferably, additional outlines parallax operational part, this profile parallax operational part is obtained profile parallax as the parallax of above-mentioned profile according to above-mentioned image pickup signal; Above-mentioned parallax operational part is according to above-mentioned profile parallax, the parallax between the picture that the above-mentioned a plurality of lens sections of each above-mentioned computing are formed respectively.
According to this preferred construction, detect the parallax of profile, according to the parallax of this parallax, with after the influence that reduces parallax, carry out image and synthesize according to this parallax correction image pickup signal to each piece computing parallax.Thereby, can access the distinct image that in the general image zone, has reduced the influence of parallax.
In the video camera module of the invention described above, preferably, above-mentioned cutting part is divided into rectangular-shaped a plurality of with above-mentioned at least 1 image pickup signal.
According to this preferred construction,, with after the influence that reduces parallax, carry out image and synthesize according to this parallax correction image pickup signal to the parallax of each piece computing parallax.Thereby, can access the distinct image that in the general image zone, has reduced the influence of parallax.
In the video camera module of the invention described above, preferably, also have parallax evaluation of estimate operational part, this parallax evaluation of estimate operational part is represented at least 1 parallax precision evaluation value of the precision of above-mentioned parallax to each above-mentioned a plurality of computing according to above-mentioned image pickup signal; Above-mentioned parallax correction portion is revised above-mentioned a plurality of image pickup signal according to above-mentioned parallax and above-mentioned parallax precision evaluation value to above-mentioned a plurality of each, and composograph.
Sometimes include the different a plurality of objects of subject distance in the piece of cutting apart, in the case, parallax is all different to each object picture.In this preferred construction, judge how many precision the parallax to each piece computing has, according to the modification method of this precision change image pickup signal.Thus, can be with optimal modification method to each piece correction image, so can access the distinct image that in the general image zone, has reduced the influence of parallax.
In the video camera module of the invention described above, preferably, whether above-mentioned parallax correction portion is divided at least 2 with this piece according to above-mentioned parallax precision evaluation value to each above-mentioned a plurality of decision, in being judged as the piece that to cut apart, according to the synthesized parallax image of the piece after cutting apart.
According to this preferred construction, judge how many precision the parallax that each piece is calculated has, do not have the piece of precision to establish and do to have mixed a plurality of parallaxes, be divided at least two pieces.Thus, can be always with optimal size correction image, so can access the distinct image that in the general image zone, has reduced the influence of parallax.
In the video camera module of the invention described above, preferably, above-mentioned parallax evaluation of estimate operational part is represented the 1st parallax precision evaluation value of the size of contrast according at least 1 image pickup signal in above-mentioned a plurality of image pickup signals to each above-mentioned a plurality of computing.
According to this preferred construction, according to contrast computing evaluation how many precision the parallax that each piece calculates is had, change the modification method of image pickup signal according to contrast.Thus, can be with optimal modification method to each piece correction image, so can access the distinct image that in the general image zone, has reduced the influence of parallax.
In the video camera module of the invention described above, preferably, the utilization of above-mentioned parallax evaluation of estimate operational part is divided into above-mentioned a plurality of above-mentioned image pickup signal, to each above-mentioned a plurality of computing the 2nd parallax precision evaluation value, how much relevant the picture that the image drift that the said lens portion that the 2nd parallax precision evaluation value representation general is at least two forms has respectively been moved behind the above-mentioned parallax is.
According to this preferred construction, how many precision the parallax that the computing evaluation calculates each piece have, moved the picture of parallax relevant what, change the modification method of image pickup signal according to being correlated with.Thus, can be with optimal modification method to each piece correction image, so can access the distinct image that in the general image zone, has reduced the influence of parallax.
In the video camera module of the invention described above, preferably, above-mentioned parallax correction portion is for above-mentioned a plurality of each, hour this piece is divided at least two in above-mentioned the 2nd parallax precision evaluation value, in the piece after cutting apart according to the synthesized parallax image of the piece after cutting apart.
According to this preferred construction, utilize expression to move relevant what the 2nd parallax precision evaluation value of the picture of parallax, estimate the parallax that each piece is calculated and have how many precision.And, the 2nd parallax precision evaluation value is little, promptly be judged as relevant less piece and be divided at least two pieces.Thus, can be always with optimal size correction image, so can access the distinct image that in the general image zone, has reduced the influence of parallax.
In the video camera module of the invention described above, preferably, above-mentioned parallax evaluation of estimate operational part, according at least 1 image pickup signal in above-mentioned a plurality of image pickup signals, the 1st parallax precision evaluation value of each above-mentioned a plurality of computing being represented the size of contrast, and utilize and be divided into above-mentioned a plurality of above-mentioned image pickup signal, to above-mentioned a plurality of each, computing represents that image drift that near few two said lens portion forms respectively moved relevant what the 2nd parallax precision evaluation value of picture behind the above-mentioned parallax; Above-mentioned parallax correction portion is for above-mentioned a plurality of each, and big and above-mentioned the 2nd parallax precision evaluation value hour in above-mentioned the 1st parallax precision evaluation value is divided into two at least with this piece, in the piece after cutting apart according to the synthesized parallax image of the piece after cutting apart.
According to this preferred construction, utilize the 1st parallax precision evaluation value and the expression of the size of expression contrast to move relevant what the 2nd parallax precision evaluation value of the picture of parallax, how many precision the parallax that each piece evaluation is calculated has.And, be greatly that the big and little piece that promptly turns down mutually of the 2nd parallax precision evaluation value of contrast is divided at least two pieces with being judged as the 1st parallax precision evaluation value.Thus, can be always with optimal size correction image, so can access the distinct image that in the general image zone, has reduced the influence of parallax.
In the video camera module of the invention described above, preferably, above-mentioned image pickup signal input part is imported above-mentioned a plurality of image pickup signal by each of a plurality of; Above-mentioned parallax operational part is respectively to each parallax of above-mentioned of above-mentioned a plurality of computings.
According to this preferred construction, by the parallax computing is carried out in each field, even when the subject of photography motion because of under the different situation of different each the images of the photography time of each, owing to correctly obtain the parallax of each, can utilize this synthesized parallax image, so can access the distinct image that in the general image zone, has reduced the influence of parallax.
In the video camera module of the invention described above, more preferably, also has colour filter, the filter that this colour filter is corresponding one by one with above-mentioned a plurality of lens sections, dispose a plurality of colors; Corresponding in above-mentioned a plurality of lens sections, with at least two lens section of above-mentioned scanning direction configured in parallel, dispose homochromy filter.
According to this preferred structure, though when the subject of photography motion because of under the different situation of different each the images of the photography time of each, also can more correctly obtain the parallax of each.
In the video camera module of the invention described above, preferably, also have other image preservation portions that preserve other images different with photographs; Above-mentioned parallax correction portion will be according to above-mentioned parallax correction image and above-mentioned other images combinations behind the above-mentioned image pickup signal.
According to this preferred construction, by will can from revised image, correctly extracting image, so can be with these clear picture ground combinations according to parallax revised image of institute and the combination of other images.
In the video camera module of the invention described above, the more little mode of ratio that preferably, above-mentioned parallax correction portion is big more according to above-mentioned parallax, revised big and above-mentioned more other images of ratio of the image behind the above-mentioned image pickup signal makes up.
According to this preferred construction, by will can from revised image, correctly extracting the image of the bigger part of parallax, so can be with these clear picture ground combinations according to parallax revised image of institute and the combination of other images.
In addition, relevant electronic equipment of the present invention is characterised in that to have the video camera module of the invention described above.
Relevant program of the present invention is the program of the action of the above-mentioned image processing part of control in following video camera module, and this video camera module has: a plurality of lens sections comprise 1 lens respectively at least; A plurality of camera watch regions, corresponding one by one with above-mentioned a plurality of lens sections, and have sensitive surface respectively with respect to the optical axis direction approximate vertical of pairing said lens portion; The image pickup signal input part, a plurality of image pickup signals that input is exported respectively from above-mentioned a plurality of camera watch regions; And image processing part, carrying out the image processing of the image pickup signal imported, said procedure is characterised in that above-mentioned image processing part is carried out: the piece dividing processing is divided into a plurality of with at least 1 image pickup signal in above-mentioned a plurality of image pickup signals; The parallax calculation process is utilized above-mentioned image pickup signal, the parallax between the picture that the above-mentioned a plurality of lens sections of each above-mentioned computing are formed respectively; And the parallax correcting process, according to the above-mentioned a plurality of image pickup signals of above-mentioned parallax correction, and composograph.
In addition, relevant program recorded medium of the present invention is to record the program recorded medium that reads said procedure, that can carry out computer.
According to program of the present invention or program recorded medium, the parallax of each piece of computing, so that after the influence of parallax reduces, it is synthetic to carry out image according to the parallax correction image pickup signal of this each piece.Thus,, also can suitably revise the parallax of each subject, can access the distinct image that in the general image zone, has reduced the influence of parallax even when the different a plurality of subjects of adjusting the distance are made a video recording simultaneously.
Below, utilize accompanying drawing that concrete execution mode of the present invention is described.
(execution mode 1)
The video camera module of relevant embodiments of the present invention 1 can obtain distinct image by detecting edge (edge) and being divided into piece, carrying out the parallax correction based on the parallax of each piece in the general image zone.In addition, carry out fine adjustment based on the auto-focus control of high accuracy contrast, can carry out auto-focus control with high-speed, high precision ground by utilizing auto-focus control carrying out coarse adjustment, utilization based on parallax.In addition, the fine adjustment amount is learnt, improved the precision of the coarse adjustment of next time.
Video camera module with reference to the relevant embodiments of the present invention 1 of description of drawings.
Fig. 1 is the cutaway view of structure of the video camera module of the relevant embodiments of the present invention 1 of expression.In Fig. 1, video camera module 101 has lens module portion 110 and circuit part 120.
Lens module portion 110 has lens barrel 111, upper cap glass 112, lens 113, actuator fixed part 114 and actuator movable part 115.Circuit part 120 has substrate 121, encapsulation 122, imaging apparatus 123, cap glass 124 and system LSI (below be designated as SLSI) 125.
Lens barrel 111 is cylindric, and inner wall surface thereof is the black of frosted for the diffuse reflection that prevents light, and resin injection is shaped forms lens barrel 111.Upper cap glass 112 is discoid, is formed by transparent resin, is fixed on the upper surface of lens barrel 111 by bonding agent etc., and its surface is provided with the diaphragm of the damage that brings such as prevent to rub and prevents the antireflection film of reflection of incident light.
Fig. 2 is the vertical view of lens 113 of the video camera module of relevant embodiments of the present invention 1.Lens 113 are roughly discoid, are formed by glass or transparent resin, dispose the 1st lens section 113a, the 2nd lens section 113b, the 3rd lens section 113c and the 4th lens section 113d with the chessboard point-like.Along the configuration direction of the 1st～the 4th lens section 113a～113d, set X-axis and Y-axis as shown in Figure 2.In the 1st lens section 113a, the 2nd lens section 113b, the 3rd lens section 113c and the 4th lens section 113d, penetrate 4 pictures of imaging on imaging apparatus 123 to imaging apparatus 123 sides from the light of object side incident.
Actuator fixed part 114 is fixed on the internal face of lens barrel 111 by bonding agent etc.Actuator movable part 115 is fixed on the outer peripheral edges of lens 113 by bonding agent etc.Actuator fixed part 114 and actuator movable part 115 constitute voice coil motor.Actuator fixed part 114 has permanent magnet (not shown) and ferromagnetism body yoke (not shown), and actuator movable part 115 has coil (not shown).Actuator movable part 115 is by relative actuator fixed part 114 resiliency supported of elastomer (not shown).By the coil electricity to actuator movable part 115, actuator movable part 115 relatively moves for actuator fixed part 114, and lens 113 change with the relative distance of imaging apparatus 123 along optical axis.
Substrate 121 is made of resin substrate, and the bottom surface of upper surface and lens barrel 111 is fixing by bonding agent etc. in contact.Like this, lens module 110 and circuit part 120 are fixed, and constitute video camera module 101.
Encapsulation 122 is made of the resin with metal terminal, in the inboard of lens barrel 111, its metal terminal portion is waited by welding be fixed on the upper surface of substrate 121.Imaging apparatus 123 comprises the 1st imaging apparatus 123a, the 2nd imaging apparatus 123b, the 3rd imaging apparatus 123c, reaches the 4th imaging apparatus 124d.The 1st imaging apparatus 123a, the 2nd imaging apparatus 123b, the 3rd imaging apparatus 123c, and the 4th imaging apparatus 124d be respectively ccd sensor or the such solid-state imager of cmos sensor, be configured to respectively, make their center of sensitive surface roughly consistent with the optical axis center of the 1st lens section 113a, the 2nd lens section 113b, the 3rd lens section 113c and the 4th lens section 113d respectively, and the optical axis approximate vertical of the sensitive surface that makes each imaging apparatus and corresponding lens section.
The 1st imaging apparatus 123a, the 2nd imaging apparatus 123b, the 3rd imaging apparatus 123c, and each terminal of the 4th imaging apparatus 123d be connected by wire-bonded, by metal wire 127 on the metal terminal of bottom of inboard of encapsulation 122, electrically be connected with SLSI125 via substrate 121.The light that penetrates from the 1st lens section 113a, the 2nd lens section 113b, the 3rd lens section 113c and the 4th lens section 113d be imaged on respectively the 1st imaging apparatus 123a, the 2nd imaging apparatus 123b, the 3rd imaging apparatus 123c, and each sensitive surface of the 4th imaging apparatus 123d on, be exported to SLSI125 by the electrical information of photodiode after the information conversion of light.
Fig. 3 is the vertical view of circuit part 120 of the video camera module of relevant embodiments of the present invention 1.Cap glass 124 is tabulars, is formed by transparent resin, is fixed on the upper surface of encapsulation 122 by bonding grade.On the upper surface of cap glass 124, dispose the 1st colour filter 124a, the 2nd colour filter 124b, the 3rd colour filter 124c, the 4th colour filter 124d, and light shielding part 124e by evaporation etc.In addition, on the lower surface of cap glass 124, it is (not shown to dispose infrared beam cut filter by evaporation etc.Below be designated as the IR filter).
Fig. 4 is the performance plot of colour filter of the video camera module of relevant embodiments of the present invention 1, and Fig. 5 is the performance plot of IR filter of the video camera module of relevant embodiments of the present invention 1.The 1st colour filter 124a has the branch optical transmission characteristics that mainly makes green transmission that the G of Fig. 4 represents, the 2nd colour filter 124b has the branch optical transmission characteristics that mainly makes blue transmission that the B of Fig. 4 represents, the 3rd colour filter 124c has the branch optical transmission characteristics that mainly makes red transmission that the R of Fig. 4 represents, the 4th colour filter has the branch optical transmission characteristics that mainly makes green transmission that the G of Fig. 4 represents.In addition, the IR filter has the branch optical transmission characteristics that infrared light is cut off that the IR of Fig. 5 represents.
Thereby, penetrate from the bottom of the 1st lens section 113a from the object light of the top incident of the 1st lens section 113a, mainly make green transmission by the 1st colour filter 124a and IR filter, be imaged on the light accepting part of the 1st imaging apparatus 123a, so the 1st imaging apparatus 123a is subjected to light to the green composition in the object light.In addition, penetrate from the bottom of the 2nd lens section 113b from the object light of the top incident of the 2nd lens section 113b, mainly make blue transmission by the 2nd colour filter 124b and IR filter, be imaged on the light accepting part of the 2nd imaging apparatus 123b, so the 2nd imaging apparatus 123b is subjected to light to the blue composition in the object light.In addition, penetrate from the bottom of the 3rd lens section 113c from the object light of the top incident of the 3rd lens section 113c, mainly make red transmission by the 3rd colour filter 124c and IR filter, be imaged on the light accepting part of the 3rd imaging apparatus 123c, so the 3rd imaging apparatus 123c is subjected to light to the red composition in the object light.And then, penetrate from the bottom of the 4th lens section 113d from the object light of the top incident of the 4th lens section 113d, mainly make green transmission by the 4th colour filter 124d and IR filter, be imaged on the light accepting part of the 4th imaging apparatus 123d, so the 4th imaging apparatus 123d is subjected to light to the green composition in the object light.
SLSI125 drives imaging apparatus 123 by the energising of the coil of method control actuator movable part 115 described later, and input is carried out various image processing from the electrical information of imaging apparatus 123, communicates with last bit CPU, to outside output image.
The relation of subject distance and parallax then, is described.The video camera module of relevant embodiments of the present invention 1 is owing to have 4 lens sections (the 1st lens section 113a, the 2nd lens section 113b, the 3rd lens section 113c and the 4th lens section 113d), so the relative position of 4 object pictures that 4 lens sections form respectively changes according to the subject distance.
Fig. 6 is the figure that is used for illustrating the position of the object picture that is in infinity in the video camera module of relevant embodiments of the present invention 1.In Fig. 6,, only represent the 1st lens section 113a, the 1st imaging apparatus 123a, the 2nd lens section 113b, the 2nd imaging apparatus 123b in order to simplify.From the light of the object 10 of infinity, to the incident light L1 of the 1st lens section 113a with parallel to the incident light L2 of the 2nd lens section 113b.Therefore, distance between the 1st lens section 113a and the 2nd lens section 113b and the object on the 1st imaging apparatus 123a equate as the distance between the 11b with object on the 2nd imaging apparatus 123b as 11a.
Here, the optical axis of the 1st lens section 113a, the optical axis of the 2nd lens section 113b, the optical axis of the 3rd lens section 113c and the optical axis of the 4th lens section 113d are configured to, respectively with the center of the sensitive surface of the center of the sensitive surface of the center of the sensitive surface of the 1st imaging apparatus 123a, the 2nd imaging apparatus 123b, the 3rd imaging apparatus 123c, and the center of the sensitive surface of the 4th imaging apparatus 123d roughly consistent.Thereby, the 1st imaging apparatus 123a, the 2nd imaging apparatus 123b, the 3rd imaging apparatus 123c, and the center of each sensitive surface of the 4th imaging apparatus 123d and be imaged on relative position relation between the object picture of the infinity on each sensitive surface respectively are identical for all imaging apparatuss.Promptly there is not parallax.
Fig. 7 is the figure of position that is used for illustrating the object picture of the position that is in limited distance in the video camera module of relevant embodiments of the present invention 1.In Fig. 7,, only represent the 1st lens section 113a, the 1st imaging apparatus 123a, the 2nd lens section 113b, the 2nd imaging apparatus 123b in order to simplify.From the light of the object 12 of limited distance, to the incident light L1 of the 1st lens section 113a with not parallel to the incident light L2 of the 2nd lens section 113b.Thereby, and the 1st lens section 113a compares with the distance between the 2nd lens section 113b, the object on the 1st imaging apparatus 123a is longer as the distance between the 11b as the object on 11a and the 2nd imaging apparatus 123b.Therefore, the 1st imaging apparatus 123a, the 2nd imaging apparatus 123b, the 3rd imaging apparatus 123c, and the center of each sensitive surface of the 4th imaging apparatus 123d be imaged on each sensitive surface respectively on the object picture of limited distance between relative position relation all different for each imaging apparatus.Parallax is promptly arranged.
If be set to object as 12 distance (subject distance) for A, the distance of establishing the 1st lens section 113a and the 2nd lens section 113b are D, the focal length of establishing lens section 113a, 113b is f, then the length on the both sides that clip the right angle by Fig. 7 is that right-angled triangle and the length on the both sides that clip the right angle of A, D are that the right-angled triangle of f, Δ is similar, thus, the parallax Δ is represented like that as shown in the formula (1).Between other lenses portion, same relation is also set up.Like this, corresponding to the subject distance, the relative position of 4 object pictures that 4 lens section 113a, 113b, 113c, 113d form respectively changes.For example, if the subject distance A diminishes, then the parallax Δ becomes big.
Δ＝f·D/A ……(1)
The relation of contrast and focal length then, is described.
Fig. 8 (A) be explanation in the video camera module of relevant embodiments of the present invention 1, the figure of the image in when focusing (when focal adjustments is consistent) and the relation of contrast evaluation of estimate, Fig. 8 (B) is the figure of explanation relation of the image of (when focal adjustments is inconsistent) and contrast evaluation of estimate in the video camera module of relevant embodiments of the present invention 1, during non-the focusing.
The left figure separately of Fig. 8 (A) and Fig. 8 (B) be to a left side half for white, right half image during for the rectangle photography of black.Shown in the left figure of Fig. 8 (A), when focusing, the profile of photographs is clear, and contrast is bigger.On the other hand, shown in the left figure of Fig. 8 (B), when non-focusing, the soft edge of photographs, contrast is less.
Result when the right figure separately of Fig. 8 (A) and Fig. 8 (B) represents the information signal effect bandpass optical filter (BPF) of left figure.Transverse axis is the X-direction position, and the longitudinal axis is the output valve after the BPF.Shown in the right figure of Fig. 8 (A), when focusing, the signal amplitude behind the BPF is bigger, and shown in the right figure of Fig. 8 (B), the signal amplitude when non-focusing behind the BPF is less.Here, the signal amplitude behind this BPF is defined as expression contrast what contrast evaluation of estimate greatly.So shown in the right figure of Fig. 8 (A), the contrast evaluation of estimate is bigger when focusing, the contrast evaluation of estimate is less when non-focusing.
Fig. 9 is the figure of the relation of the lens position of video camera module of the relevant embodiments of the present invention 1 of explanation and contrast evaluation of estimate.To the photography of certain object the time, at the distance of lens 113 and imaging apparatus 123 hour (during z1), owing to be non-focusing, so the contrast evaluation of estimate is less.If increase the distance of lens 113 and imaging apparatus 123 gradually, then the contrast evaluation of estimate becomes greatly gradually, and (during z2) contrast evaluation of estimate becomes greatly when focusing.And then, if increase the distance (during z3) of lens 113 and imaging apparatus 123 gradually, becoming non-focusing, the contrast evaluation of estimate diminishes.Like this, when focusing, the contrast evaluation of estimate becomes greatly.
The action of the video camera module of relevant embodiments of the present invention 1 then, is described.Figure 10 is the block diagram of the video camera module of relevant embodiments of the present invention 1.SLSI125 has systems control division 131, imaging apparatus drive division 132, image pickup signal input part 133, actuator operated amount efferent 134, image processing part 135 and input and output portion 136.In addition, circuit part 120 has amplifier 126 except above-mentioned structure.
Amplifier 126 will with the coil that imposes on actuator movable part 115 from the corresponding voltage of output of actuator operated amount efferent 134.
Central ProcessingUnit), formation such as memory systems control division 131 is by CPU (central arithmetic processing apparatus:, control whole SLSI125.Imaging apparatus drive division 132 is made of logical circuit etc., produces the signal that drives imaging apparatus 123, will impose on imaging apparatus 123 corresponding to this voltage of signals.
Image pickup signal input part 133 comprises the 1st image pickup signal input part 133a, the 2nd image pickup signal input part 133b, the 3rd image pickup signal input part 133c, reaches the 4th image pickup signal input part 133d formation.The 1st image pickup signal input part 133a, the 2nd image pickup signal input part 133b, the 3rd image pickup signal input part 133c, and the 4th image pickup signal input part 133d is respectively CDS circuit (the correlated double sample circuit: Correlated Double SamplingCircuit) that is connected in series, AGC (automatic gain controller: Automatic Gain Controller), ADC (analog-to-digital converter: Analog Digital Converter) constitute, respectively with the 1st imaging apparatus 123a, the 2nd imaging apparatus 123b, the 3rd imaging apparatus 123c, and the 4th imaging apparatus 123d connects, be transfused to the signal of telecommunication from them, by the CDS circuit fixedly noise remove, regulate gain by AGC, be transformed to digital value by ADC from analog signal, be written in the memory of systems control division 131.
Actuator operated amount efferent 134 is by DAC (digital-to-analog converter: DigitalAnalog Converter) constitute output and the voltage corresponding voltage signal that will apply the coil of actuator movable part 115.
Image processing part 135 by logical circuit or DSP or comprise them both and constitute, utilize the memorizer information of systems control division 131, various image processing are carried out in program control according to the rules.Image processing part 135 has rim detection portion 141, piece cutting part 142, parallax operational part 143, parallax correction portion 144, parallax formula auto-focus control part 145, actuator control part 146, contrast formula auto-focus control part 147, auto-focus control contrast evaluation of estimate operational part 148 and actuator operated flow function correction portion 149.
Input and output portion 136 carry out with last bit CPU (not shown) between communicate by letter and exterior display device (not shown) output image signals such as bit CPU, external memory storage (not shown) and liquid crystal upwards.
Figure 11 is the flow chart of action of the video camera module of the relevant embodiments of the present invention 1 of expression.Systems control division 131 by SLSI125 makes video camera module 101 move like that according to this flow chart.
In step S100, begin action.For example, the situation that last bit CPU (not shown) detection shutter release button etc. is pressed, via the beginning of input and output portion 136 pairs of video camera modules command action, thus, video camera module 101 begins action.Follow execution in step S110.
In step S110, carry out photography.By the order of systems control division 131, imaging apparatus drive division 132 is exported the signal that is used for carrying out electronic shutter and transmission at any time.The 1st image pickup signal input part 133a, the 2nd image pickup signal input part 133b, the 3rd image pickup signal input part 133c, and the signal Synchronization ground input of the 4th image pickup signal input part 133d and 132 generations of imaging apparatus drive division is as the 1st imaging apparatus 123a, the 2nd imaging apparatus 123b, the 3rd imaging apparatus 123c, and the image pickup signal of the analog signal of each image of the 4th imaging apparatus 123d output, by CDS fixedly noise remove, automatically regulate input gain by AGC, by ADC analog signal is transformed to digital value, become the 1st image pickup signal I1 (x, y), the 2nd image pickup signal I2 (x, y), the 3rd image pickup signal I3 (x, y), and the 4th image pickup signal I4 (x, y), digital value is written in the memory of specified address of systems control division 131.Figure 12 is the figure of coordinate of image pickup signal of the video camera module of the relevant embodiments of the present invention 1 of explanation.I1 (x, y) the 1st image pickup signal of y on the x, vertical direction on the expression horizontal direction.The pixel count longitudinally of the image of being imported is H, and horizontal pixel count is L, and total pixel number is H * L, and x is from 0 to L-1 variation, and y changes from 0 to H-1.The 2nd image pickup signal I2 (x, y), the 3rd image pickup signal I3 (x, y), the 4th image pickup signal I4 (x, y) too.That is, I2 (x, y), I3 (x, y) and I4 (x, y) expression is the 2nd image pickup signal of y on x, the vertical direction on the horizontal direction, the 3rd image pickup signal, and the 4th image pickup signal respectively.The pixel count longitudinally of the image of being imported is H, and horizontal pixel count is L, and total pixel number separately is H * L, and x is from 0 to L-1 variation, and y changes from 0 to H-1.Then, execution in step S121.
In step S121 and step S122, rim detection portion 141 utilizes data on the memory of systems control division 131, detects the edge.And, this result is written in the memory of systems control division 131.Details is as follows.
In step S121, computing rim detection contrast evaluation of estimate.This computing is only carried out the 1st image pickup signal.Computing laplacian as following formula (2), will as following formula (3), act on again three-dimensionally value behind the LPF (optical low-pass filter) be made as rim detection with contrast evaluation of estimate C2 (x, y).Figure 13 (A) is the original image of rim detection of the video camera module of the relevant embodiments of the present invention 1 of explanation, Figure 13 (B) be the rim detection of rim detection of video camera module of the relevant embodiments of the present invention 1 of explanation with the image of contrast evaluation of estimate, Figure 13 (C) is the image at edge of rim detection of the video camera module of the relevant embodiments of the present invention 1 of explanation.(x, y), it is such then to become Figure 13 (B) if through type (2) and formula (3) are calculated the rim detection of the original image of Figure 13 (A) contrast evaluation of estimate C2.In addition, Figure 13 (B) uses the bigger position of absolute value of black expression (3).Follow execution in step S122.
C1(x，y)＝I1(x-1，y)+I1(x+1，y)+I1(x，y-1)+I1(x，y+1)-4I1(x，y)
……(2)
C2(x，y)＝C1(x-1，y-1)+C1(x，y-1)+C1(x+1，y-1)
+C1(x-1，y)+C1(x，y)+C1(x+1，y)
+C1(x-1，y+1)+C1(x，y+1)+C1(x+1，y+1) ……(3)
In step S122, detect the edge.Figure 13 (C) is the image at edge of rim detection of the video camera module of the relevant embodiments of the present invention 1 of explanation.(x, 0 intersection point y) (value is changed to positive point from just being changed to negative point and value from bearing) can detect the such edge of Figure 13 (C) to rim detection by detecting Figure 13 (B) with Contrast Detection value C2.Then, execution in step S130.
In step S130, piece cutting part 142 utilizes the data on the memory of systems control division 131 to be divided into piece.Then, its result is written in the memory of systems control division 131.As Figure 13 (C), to by the zone of surrounded by edges such as B0, B1 ..., Bi ..., Bn gives number like that.In addition, survey and disappear, also can utilize expansion algorithm and contraction algorithm to revise the edge for the flase drop that prevents the edge that noise etc. causes.Follow execution in step S140.
In step S140, parallax operational part 143 utilizes the data on the memory of systems control division 131, the parallax value of each piece of computing.Then, be written in the memory of systems control division 131.At first, to each piece (B0, B1 ..., Bi ..., Bn) computing parallax evaluation of estimate (R0 (k), R1 (k) ..., Ri (k) ..., Rn (k), k=0,1 ..., m).Figure 14 be explanation in the video camera module of relevant embodiments of the present invention 1, the figure in the computing zone of parallax evaluation of estimate.(also representing as I1) zone of being represented by Bi is i the piece of obtaining in step S130 according to the 1st image pickup signal I1.The zone of being represented by I4 is that Bi is moved k, moves zone behind the k along the y direction along the x direction.And, for the whole image pickup signal I1 in each zone (x, y), (x, y), the absolute value difference summation shown in the following formula of computing (4) is as parallax evaluation of estimate Ri (k) for I4.
Ri(k)＝∑∑|I1(x，y)-I4(x-k，y-k)| ……(4)
This parallax evaluation of estimate Ri (k) represents the 1st picture signal I1 of i piece Bi and leaves respectively along x, y direction (what kind of k, the 4th picture signal I4 in zone k) have be correlated with, expression value more little relevant big more (very approximate).Figure 15 is the figure of the relation of the parallax of video camera module of the relevant embodiments of the present invention 1 of explanation and parallax evaluation of estimate.As shown in figure 15, parallax evaluation of estimate Ri (k) changes according to the value of k, has minimum when k=Δ i.I the piece Bi that this expression makes the 4th picture signal I4 moves (Δ i ,-Δ i) respectively and the picture signal of the piece that obtains and the 1st picture signal I1 have relevant (the most similar) of maximum along x, y direction.Thereby the parallax for the 1st image pickup signal I1 of i piece and the x between the 4th image pickup signal I4, y direction is (Δ i, Δ i) as can be known.Below, this Δ i is called the parallax value Δ i of i piece Bi.Like this, obtain the parallax value Δ i of all Bi from i=0 to i=n.Then, execution in step S151.
In step S151 and step S152, the image that the output preview is used.
In step S151, parallax correction portion 144 is utilized the data on the memory of systems control division 131, utilizes the parallax value corresponding with this piece that each piece is carried out after the parallax correction, and it is synthetic to carry out image.Then, its result is written in the memory of systems control division 131.Because the 1st imaging apparatus 123a and the 4th imaging apparatus 123d mainly are subjected to light to the green composition of object light, so the 1st image pickup signal I1 and the 4th image pickup signal I4 are the information signals of the green composition of object light.In addition, because the 2nd image pickup signal 123b mainly is subjected to light to the blue composition of object light, so the 2nd image pickup signal I2 is the information signal of the blue composition of object light.And then, because the 3rd imaging apparatus 123c mainly is subjected to light to the red composition of object light, so the 3rd image pickup signal I3 is the information signal of the red composition of object light.Because the parallax of prediction the 1st imaging apparatus 123a and the 4th imaging apparatus 123d is (Δ i, Δ i), so as following formula (5), with remarked pixel coordinate (x, G (the x of the green intensity y), y) be made as the 1st image pickup signal I1 (x, y) average with the 4th image pickup signal I4 (x-Δ i, y-Δ i).In addition because the parallax of prediction the 1st imaging apparatus 123a and the 2nd imaging apparatus 123b is (Δ i, 0), so as following formula (6), with the remarked pixel coordinate (x, y) B of the intensity of the blueness in (x, y) be made as the 2nd image pickup signal I2 (x-Δ i, y).And then, because the parallax of prediction the 1st imaging apparatus 123a and the 3rd imaging apparatus 123c is (0, Δ i), so as following formula (7), will represent that (x, y) (x y) is made as the 3rd image pickup signal I3 (x, y-Δ i) to the R of the red intensity in.In addition, as parallax value Δ i, use to comprise pixel coordinate (x, the parallax value Δ i of piece Bi y).Then, execution in step S152.
G(x，y)＝[I1(x，y)+I4(x-Δi，y-Δi)]/2 ……(5)
B(x，y)＝I2(x-Δi，y) ……(6)
R(x，y)＝I3(x，y-Δi) ……(7)
In step S152, output image.Input and output portion 136 with the data on the memory of systems control division 131 be G (x, y), B (x, y), (x y) exports to bit CPU (not shown) and exterior display device (not shown) to R.In addition, also can export for example output such as brightness, color difference signal, replace G (x, y), B (x, y), R (x, y).In addition, also can export value after the image processing such as white balance or γ correction.Then, carry out S161.
In step S161, step S162 and S163, use the auto-focus control of parallax value.
In step S161, parallax formula auto-focus control part 145 is selected auto-focus control piece according to the data on the memory of systems control division 131.And, its result is written in the memory of systems control division 131.Select near at least 1 piece (for example 3 piece Bj1, Bj2, and Bj3) the center of image-region for example.In addition, it is near the central authorities that these pieces might not need, and user that also can the mirror operation video camera (for example by the sensor viewpoint direction) such as the meanings selects piece.Then, execution in step S162.
In step S162, the position command of computing actuator.If above-mentioned example, then as following formula (8), with piece Bj1, Bj2, and parallax value Δ j1, the Δ j2 of Bj3, and Δ j3 on average be made as auto-focus control with parallax value Δ af.In addition, also can come suitable weighting according to the area of piece or the information that does not more approach center etc.And then, the position command Xact of computing actuator as following formula (9).In addition, position command Xact represents that the focusing position with the infinity picture is a benchmark, the instruction of the position between subject.Then, execution in step S163.
Δaf＝(Δj1+Δj2+Δj3)/3 ……(8)
Xact＝kx·Δaf ……(9)
In step S163, actuator control part 146 utilizes the operational ton functional operation actuator operated amount shown in the following formula (10) (applying voltage to what the coil of actuator movable part 115 applied) Vact.In addition, in order to learn operational ton function described later, the actuator operated amount is preserved as Vact0.Then, execution in step S164.
Vact＝ka·Xact+kb ……(10)
Vact0＝Vact ……(10’)
In step S164, make actuator action.The voltage signal that 134 changes of actuator operated amount efferent are exported is so that becoming Vact via the amplifier 126 after-applied voltages of the coil (not shown) of actuator movable part 115 of giving.Then, execution in step S171.
In step S171, step S172, step S173, step S174, step S175 and step S176, contrast formula auto-focus control part 147 utilizes the auto-focus control of contrast.As above-mentioned shown in Figure 9, the contrast evaluation of estimate becomes greatly at the focusing position place.Utilize this principle, search contrast evaluation of estimate is great actuator operated amount.
In step S171, make the auto-focus control contrast evaluation of estimate that does not have the correction of actuator operated amount.Step S171 is made of step S171c and step S171d.Execution in step S171c at first.
In step S171c, carry out shooting.This action is same with step S110, but the auto-focus by the S161 selection that also can only send in the image pickup signal I1 of the 1st imaging apparatus 123a is controlled the image pickup signal I1 that uses piece.In the case, compare, can shorten the delivery time with transmitting whole image pickup signals.Follow execution in step S171d.
In step S171d, auto-focus control utilizes the data on the memory of systems control division 131 with contrast evaluation of estimate operational part 148, makes auto-focus and controls and use the contrast evaluation of estimate.Only the 1st image pickup signal I1 that relevant auto-focus is controlled with piece carries out this computing.The absolute value of computing laplacian as following formula (11), as following formula (12), act on LPF (optical low-pass filter) three-dimensionally, it is on average obtained auto-focus contrast evaluation of estimate C5 as following formula (13) in auto-focus is controlled with piece.Here, N is C4 (x, y) the number of auto-focus control in the piece.Then, as following formula (14), contrast evaluation of estimate C5 at this moment is made as C50, is written in the memory of systems control division 131.Then, execution in step S172.
C3(x，y)＝|I1(x-1，y)+I1(x+1，y)+I1(x，y-1)+I1(x，y+1)-4I1(x，y)|
……(11)
C4(x，y)＝C3(x-1，y-1)+C3(x，y-1)+C3(x+1，y-1)
+C3(x-1，y)+C3(x，y)+C3(x+1，y)
+C3(x-1，y+1)+C3(x，y+1)+C3(x+1，y+1) ……(12)
C5＝∑C4(x，y)/N ……(13)
C50＝C5 ……(14)
In step S172, the auto-focus control contrast evaluation of estimate when making the positive side correction of actuator operated amount.Step S172 by step S172a, step S172b, step S172c, and step S172d constitute.Execution in step S172a at first.
In step S172a, the actuator operated amount Vact of the value of actuator operated amount as following formula (15) when not revising (applying voltage) after Vact0 adds dVact during as positive side correction to what the coil of actuator fixed part 114 applied, as following formula (16), be kept in the memory as Vactp.Then, execution in step S172b, step S172c, and step S172d.
Vact＝Vact0+dVact ……(15)
Vactp＝Vact ……(16)
In step S172b, make actuator action, in step S172c, carry out photography, in step S172d, make auto-focus control contrast evaluation of estimate.Step S172b carries out the action same with step S164.In addition, step S172c, step S172d carry out the action same with step S171c, step S171d, as following formula (17), contrast evaluation of estimate C5 as the revised auto-focus of positive side contrast evaluation of estimate C5p, is written in the memory of systems control division 131.Then, execution in step S173.
C5p＝C5 ……(17)
In step S173, the auto-focus control contrast evaluation of estimate when making the minus side correction of actuator operated amount.Step S173 by step S173a, step S173b, step S173c, and step S173d constitute.Execution in step S173a at first.
In step S173a, the actuator operated amount Vact of the value of actuator operated amount when as following formula (18), never revising (applying voltage) after Vact0 deducts dVact during as the minus side correction to what the coil of actuator fixed part 114 applied, as following formula (19), be kept in the memory as Vactn.Then, execution in step S173b, step S173c, and step S173d.
Vact＝Vact0-dVact ……(18)
Vactn＝Vact ……(19)
In step S173b, make actuator action, in step S173c, carry out photography, in step S173d, make auto-focus control contrast evaluation of estimate.Carry out and step S172b, step S172c, the same action of step S172d, as following formula (20), contrast evaluation of estimate C5 as the revised auto-focus of minus side contrast evaluation of estimate C5n, is written in the memory of systems control division 131.Then, execution in step S174.
C5n＝C5 ……(20)
In step S174, estimate focal point control contrast evaluation of estimate.Relatively which C50, C5p, C5n are that maximum is carried out branch according to.
When C50 is maximum, carry out following action, then execution in step S181.The figure of the actuator operated amount that Figure 16 (A) is the contrast evaluation of estimate of explanation in the video camera module of relevant embodiments of the present invention 1, when do not have revising when maximum and the relation of contrast evaluation of estimate.Shown in Figure 16 (A), do not provide the roughly maximum of auto-focus control owing to there is the actuator operated amount Vact0 that revises, so as following formula (21), making final actuator operated amount Vactf is Vact0 with contrast value C5.
Vactf=Vact0 (when C50 is maximum) ... (21)
When C5p is maximum, carry out following action, then execution in step S175.Figure 16 (B) be explanation in the video camera module of relevant embodiments of the present invention 1, the actuator operated amount of the positive revised contrast evaluation of estimate of side when maximum and the figure of the relation of contrast evaluation of estimate.Shown in Figure 16 (B), because positive side revised actuator operated amount Vactp or further provide the roughly maximum of auto-focus control with contrast evaluation of estimate C5 to the revised actuator operated amount of positive side Vact, so further with the actuator operated amount to positive side correction, the search maximum.In addition, as following formula (22), C5p as previous value, is preserved as C5pp.
C5pp＝C5p ……(22)
When C5n is maximum, carry out following action, then execution in step S176.Figure 16 (C) be explanation in the video camera module of relevant embodiments of the present invention 1, the actuator operated amount of the revised contrast evaluation of estimate of minus side when maximum and the figure of the relation of contrast evaluation of estimate.Shown in Figure 16 (C), minus side revised actuator operated amount Vactn or further provide auto-focus control roughly maximum with contrast evaluation of estimate C5 to the revised actuator operated amount of minus side Vact, so further with the actuator operated amount to the minus side correction, the search maximum.In addition, as following formula (23), C5n as previous value, is preserved as C5np.
C5np＝C5p ……(23)
In step S175, the auto-focus contrast evaluation of estimate when making and estimating further just side correction.Step S175 by step S175a, step S175b, step S175c, step S175d, and step S175e constitute.Execution in step S175a at first.
In step S175a, as following formula (24) to the value of the revised actuator operated amount of positive side last time (applying voltage) after Vactp adds dVact to what the coil of actuator fixed part 114 applied the actuator operated amount Vact during as positive side correction, as following formula (25), be kept in the memory as Vactp.Then, execution in step S175b, step S175c, and step S175d.
Vact＝Vactp+dVact ……(24)
Vactp＝Vact ……(25)
In step S175b, make actuator action, in step S175c, carry out photography, in step S175d, make auto-focus control contrast evaluation of estimate.Carry out and step S172b, step S172c, the same action of step S172d, as following formula (26), contrast evaluation of estimate C5 as the revised auto-focus of positive side contrast evaluation of estimate C5p, is written in the memory of systems control division 131.Then, execution in step S175e.
C5p＝C5 ……(26)
In step S175e, estimate focal point control contrast evaluation of estimate.Relatively which C5pp, C5p are that maximum is carried out branch according to.
When C5pp is maximum, carry out following action, then execution in step S181.The figure of the actuator operated amount that Figure 17 (A) is the contrast evaluation of estimate of explanation in the video camera module of relevant embodiments of the present invention 1, when having carried out further positive side correction, before this positive side correction when maximum and the relation of contrast evaluation of estimate.Shown in Figure 17 (A), because the revised actuator operated amount of positive side Vactp-dVact last time provides the roughly maximum of auto-focus control with contrast evaluation of estimate C5, so as following formula (27), final actuator operated amount Vactf is made as Vactp-dVact.
Vactf=Vactp-dVact (when C5pp is maximum) ... (27)
When C5p is maximum, carry out following action, then execution in step S175a.Figure 17 (B) be explanation in the video camera module of relevant embodiments of the present invention 1, when having carried out further positive side correction, this positive side revised contrast evaluation of estimate actuator operated amount when maximum and the figure of the relation of contrast evaluation of estimate.Shown in Figure 17 (B), owing to work as inferior positive side revised actuator operated amount Vactp or further provide the roughly maximum of auto-focus control with contrast evaluation of estimate C5 to the revised actuator operated amount of positive side Vact, so further with the actuator operated amount to positive side correction, the search maximum.In addition, as following formula (28), C5p as previous value, is preserved as C5pp.
C5pp＝C5p ……(28)
In step S176, the auto-focus contrast evaluation of estimate when making and estimating further minus side correction.Step S176 by step S176a, step S176b, step S176c, step S176d, and step S176e constitute.Execution in step S176a at first.
In step S176a, as following formula (29) from the actuator operated amount Vact of the value of the revised actuator operated amount of minus side last time (applying voltage) after Vactp deducts dVact during as the minus side correction to what the coil of actuator fixed part 114 applied, as following formula (30), be kept in the memory as Vactn.Then, execution in step S176b, step S176c, and step S176d.
Vact＝Vactn-dVact ……(29)
Vactn＝Vact ……(30)
In step S176b, make actuator action, in step S176c, carry out photography, in step S176d, make auto-focus control contrast evaluation of estimate.Carry out and step S172b, step S172c, the same action of step S172d, as following formula (31), contrast evaluation of estimate C5 as the revised auto-focus of minus side contrast evaluation of estimate C5n, is written in the memory of systems control division 131.Then, execution in step S176e.
C5n＝C5 ……(30)
In step S176e, estimate focal point control contrast evaluation of estimate.Relatively which C5np, C5n are that maximum is carried out branch according to.
When C5np is maximum, carry out following action, then execution in step S181.The figure of the actuator operated amount that Figure 18 (A) is the contrast evaluation of estimate of explanation in the video camera module of relevant embodiments of the present invention 1, when having carried out further minus side correction, before this minus side correction when maximum and the relation of contrast evaluation of estimate.Shown in Figure 18 (A), because the revised actuator operated amount of minus side Vactp+dVact last time provides the roughly maximum of auto-focus control with contrast evaluation of estimate C5, so as following formula (32), final actuator operated amount Vactf is made as Vactp+dVact.
Vactf=Vactp+dVact (when C5np is maximum) ... (32)
When C5n is maximum, carry out following action, then execution in step S176a.Figure 18 (B) be explanation in the video camera module of relevant embodiments of the present invention 1, when having carried out further minus side correction, the actuator operated amount of the revised contrast evaluation of estimate of this minus side when maximum and the figure of the relation of contrast evaluation of estimate.Shown in Figure 18 (B), owing to work as inferior minus side revised actuator operated amount Vactn or further provide the roughly maximum of auto-focus control with contrast evaluation of estimate C5 to the revised actuator operated amount of minus side Vact, so further with the actuator operated amount to the minus side correction, the search maximum.In addition, as following formula (33), C5n as previous value, is preserved as C5np.
C5np＝C5n ……(33)
In step S181, step S182, step S183, step S184 and step S185, carry out the output of final photography and image.
In step S181, as following formula (34), set by the above-mentioned final actuator operated amount Vactf that obtains as actuator operated amount Vact.Then, execution in step S182.
Vact＝Vactf ……(34)
In S182, make actuator action.This step and step S164 similarly move.Follow execution in step S183.
In step S183, photograph.This step and step S110 similarly move.Follow execution in step S184.
In step S184, after the utilization parallax value corresponding with this piece carried out the parallax correction to each piece, carry out image and synthesize.This step and step S151 similarly move.Follow execution in step S185.
In step S185, carry out image output.This step and step S152 similarly move.Follow execution in step S190.
In step S190, actuator operated flow function correction portion 149 is based on the correction value correction operational ton function of contrast formula auto-focus control.That is, as following formula (35), revise the coefficient k b of operational ton function, the value that change writes in the memory of systems control division 131.When the photography of next time, utilize this value.Then, execution in step S199.
kb＝kb+kc·(Vactf-Vact0) ……(35)
End process in step S199.
Constitute, move like that by above, have following effect.
In the video camera module of the 1st execution mode, as step S161, step S162, step S163, step S164, make actuator position instruction Xact with parallax value Δ af according to auto-focus control, computing actuator operated amount Vact, coil to actuator fixed part 114 applies voltage, make actuator action, carry out auto-focus control.Thus, owing to focus, can carry out auto-focus control at high speed by 1 time actuator action.
In addition, in the video camera module of the 1st execution mode, at first, in step S161, step S162, step S163, step S164, make actuator action, carry out auto-focus control according to parallax.Then, in step S171, step S172, step S173, step S174, step S175, step S176, make actuator action, carry out auto-focus control according to contrast.Owing to focus, be at a high speed so control based on the auto-focus of parallax by 1 time actuator action.On the other hand, owing to directly judge focusing according to image, so not influenced by the deviation etc. of actuator, precision is better.Thereby, owing to auto-focus control carrying out coarse adjustment, carry out fine adjustment, so can and carry out auto-focus accurately and control with high speed by auto-focus control based on high-precision contrast based on parallax at a high speed.
In addition, in the video camera module of the 1st execution mode, in step S190,, revise the such study of coefficient k b of operational ton function according to the actuator operated amount of in auto-focus control, revising (Vactf-Vact0) based on contrast.Thus, even deviation that has actuator etc. also can more correctly be revised the operational ton function, auto-focus control based on parallax becomes more accurate, and the number of times of the fine adjustment that the auto-focus based on contrast that can reduce next time is controlled is so realized auto-focus control more at a high speed.
Such suc as formula (1), corresponding to the subject distance A, the relative position of 4 object pictures that the 1st～the 4th lens section 113a～113d forms respectively changes.That is, if the subject distance A diminishes, then the parallax Δ becomes big.Therefore, when the different a plurality of subjects of adjusting the distance were photographed simultaneously, the parallax Δ was all different to each subject.In the video camera module of the 1st execution mode, be piece with the general image Region Segmentation in step S130, the parallax of each piece of computing in step S140 is in step S184, carry out image by parallax and synthesize and carry out the parallax correction, so that reduce the influence of parallax according to each piece.Thus,, also can suitably revise the parallax of each subject, can access the distinct image that in the general image zone, has reduced parallax effects even when the different a plurality of subjects of adjusting the distance are photographed simultaneously.
In addition, in the video camera module of the 1st execution mode, in step S121, step S122, detect the edge, in step S130, be divided into piece.Thus, can suitably carry out blocking, so can access the distinct image that in the general image zone, has reduced parallax effects.
In addition, in the video camera module of the 1st execution mode, former state is utilized the parallax after the computing, but also can suitably limit.According to lens peculiarity, the subject distance A than certain the value hour, it is not distinct that image becomes.Therefore, as long as this value is set at the minimum value of subject distance A, just determined the maximum of parallax Δ.The parallax bigger than this value also can be used as error and ignores.In addition, under these circumstances, also can adopt the little value of parallax evaluation of estimate the 2nd as parallax.
In addition, in the video camera module of the 1st execution mode, according to the 1st image pickup signal I1 (main expression is green) and the 4th image pickup signal I4 (main expression is green) computing parallax, but the present invention is not limited to this.For example, so owing to the subject of purple green composition less contain more blue composition, red composition can not situation according to the 1st image pickup signal I1 (main expression is green) and the 4th image pickup signal I4 (main expression is green) computing under, also can be according to the 2nd image pickup signal I2 (main expression is blue) and the 3rd image pickup signal I3 (mainly representing redness) computing parallax.In addition, if can not be according to the 1st parallax signals I1 (main expression green) the 4th image pickup signal I4 (main expression is green) computing parallax and can not be according to the 2nd image pickup signal I2 (main expression is blue) and the 3rd image pickup signal I3 (main expression redness) computing parallax, then regard the influence that does not have parallax as, can think no parallax.
In addition, when the video camera module with the 1st execution mode is loaded in the video camera, by the 1st～the 4th imaging apparatus 123a～123d is configured to: make the 2nd imaging apparatus 123a be configured in upside, the 3rd imaging apparatus 123c is configured in downside, upside becomes blue responsive, downside is become red sensitive thus, so can make the color reproduction of picture with scenes more natural.
In addition, when two outstanding extreme values are arranged, also can adopt a bigger side's parallax in the parallax evaluation of estimate.Owing to contain subject and background in such piece, subject distance and back pitch are from different, so extreme value occurs two.Because less from comparing the subject distance with back pitch, so compare with the parallax of background, the parallax of subject is bigger.Here, if adopt a bigger side's parallax,, can reduce the direct parallax that influences the subject of image quality though can not reduce the influence of the parallax of background.
In addition, the timing of image output is not limited to above-mentioned situation, also can suitably carry out preview output.For example, also can after the photography of step S110, export the image that does not have the parallax correction.In addition, in the piece of step S130 is cut apart, also can when cutting apart 1 piece, reflect its result, upgrade preview screen.
In addition, the video camera module of execution mode 1 makes actuator action according to parallax in S164, make actuator action according to contrast then in S171, S172, S173, S175, S182, but also can only make actuator action according to parallax.This be because, under the situation of using the darker lens of the depth of focus since in the distance of lens and imaging apparatus some errors of permission, so do not need to make actuator action improve precision according to contrast.
In addition, the video camera module of execution mode 1 is revised the correction function in S190.Operational ton when promptly basis makes actuator action based on contrast is learnt, but also can omit this step.Because the actuator of the video camera module of execution mode 1 is a voice coil-type, the amount of movement of lens changes according to variations in temperature and posture change, so by study, precision can improve significantly, but under the situation of using stepping motor, because the variation of variations in temperature and the caused lens moving of posture change is less, so also can omit study.
In addition, though also can be the situation that the focal point control of carrying out parallax does not make actuator action.For example, the distance of also can be focusing during in advance as the photography infinity with the relative distance of lens and imaging apparatus, hour regarding as at parallax does not need to make actuator action, and does not make actuator action.
In addition, also can suitably carry out piece repeatedly and cut apart, and the additional action of upgrading piece.For example, in execution mode 1, after also can in S183, having carried out photographing, carry out S121, S122, reach S130, upgrade piece, in step S184 thereafter, carry out the parallax correction.
(execution mode 2)
The video camera module of relevant embodiments of the present invention 2 utilizes the parallax at the Contrast Detection edge of a plurality of images, parallax according to this parallax computing general image zone, it is synthetic and carry out the parallax correction to carry out image according to this parallax, so that reduce the influence of parallax, obtain in the general image zone, having reduced the distinct image of the influence of parallax thus.
Describe with reference to the video camera module of accompanying drawing relevant embodiments of the present invention 2.
Figure 19 is the cutaway view of structure of the video camera module of the relevant embodiments of the present invention 2 of expression.Same with execution mode 1 except the SLSI225 of the circuit part 220 of video camera module 201, for giving identical label and omit explanation to them with the same parts of execution mode 1.
Figure 20 is the block diagram of the video camera module of relevant embodiments of the present invention 2.SLSI225 has systems control division 231, imaging apparatus drive division 132, image pickup signal input part 133, actuator operated amount efferent 134, image processing part 235 and input and output portion 136.In addition, circuit part 220 also has amplifier 126 except above-mentioned structure.
Systems control division 231 is made of CPU, memory etc., controls whole SLSI225.
Image processing part 235 is by logical circuit or DSP or comprise both and constitute, and utilizes the memorizer information of systems control division 231 to carry out various image processing.Image processing part 235 has contrast operational part 241, auto-focus control part 242, edge parallax operational part 243, general image regional disparity operational part 244, parallax correction portion 245.
Imaging apparatus drive division 132, image pickup signal input part 133, actuator operated amount efferent 134, input and output portion 136, and amplifier 126 same with execution mode 1, omit explanation.
Figure 21 is the flow chart of action of the video camera module of the relevant embodiments of the present invention 2 of expression.By the systems control division 231 of SLSI225, video camera module 201 is moved as this flow chart.
In step S200, begin action.For example, detect the situation that shutter release button etc. is pressed by last bit CPU (not shown), via the beginning of input and output portion 136 pairs of video camera modules command action, video camera module 201 begins action thus.Then, execution in step S211.
Step S211, step S212, step S213, step S214, step S215, step S216, and step S217 in, carry out photography repeatedly, computing contrast evaluation of estimate is preserved photography signal and contrast evaluation of estimate.
In step S211, as following formula (36), with counter i initialization.Then, execution in step S212.
i＝0 ……(36)
In step S212, make actuator position instruction Xact.In addition, position command Xact represent the focusing position with the infinity picture be benchmark, towards the instruction of the position of subject direction.Then, execution in step S213.
Xact＝kx2·i ……(37)
In step S213, as following formula (38),, utilize controlled quentity controlled variable functional operation actuator controlled quentity controlled variable Vact according to actuator position instruction Xact.Then, execution in step S214.
Vact＝ka·Xact+kb ……(38)
In step S214, make actuator action.This action is same with the action of the step S162 of execution mode 1, omits explanation.Then, execution in step S215.
In step S215, photograph, preserve image pickup signal.By the order of systems control division 231, imaging apparatus drive division 132 is exported the signal that is used for carrying out electronic shutter and transmission at any time.The 1st image pickup signal input part 133a, the 2nd image pickup signal input part 133b, the 3rd image pickup signal input part 133c, and the signal Synchronization ground that the 4th image pickup signal input part 133d and imaging apparatus drive division 132 are produced is imported as the 1st imaging apparatus 123a, the 2nd imaging apparatus 123b, the 3rd imaging apparatus 123c, and the image pickup signal of the analog signal of each image of being exported of the 4th imaging apparatus 123d, by CDS fixedly noise remove, automatically regulate input gain by AGC, by ADC analog signal is transformed to digital value, become the 1st image pickup signal I1 (i, x, y), the 2nd image pickup signal I2 (i, x, y), the 3rd image pickup signal I3 (i, x, y), and the 4th image pickup signal I4 (i, x y), is written to digital value in the memory of specified address of systems control division 231.Figure 22 is the figure of coordinate of image pickup signal of the video camera module of the relevant embodiments of the present invention 2 of explanation.I1 (i, x, y) the 1st image pickup signal of y on x, the vertical direction on the expression horizontal direction of photographing for the i time.The pixel count longitudinally of the image of being imported is H, and horizontal pixel count is L, and total pixel number is H * L, and x is from 0 to L-1 variation, and y changes from 0 to H-1.In addition, i is a counter, and expression is the image of the i+1 time photography.The 2nd image pickup signal I2 (i, x, y), the 3rd image pickup signal I3 (i, x, y), the 4th image pickup signal I4 (i, x, y) too.That is, I2 (i, x, y), I3 (i, x, y) and I4 (y) expression is the 2nd image pickup signal, the 3rd image pickup signal of y on the x, vertical direction on the horizontal direction of the i+1 time photography, reaches the 4th image pickup signal respectively for i, x.The pixel count longitudinally of the image of being imported is H, and horizontal pixel count is L, and each total pixel number is H * L, and x is from 0 to L-1 variation, and y changes from 0 to H-1.In addition, i is a counter, and expression is the image of the i+1 time photography.Then, execution in step S216.
In step S216, contrast operational part 241 utilizes the data on the memory of systems control division 231, the computing of degree of comparing evaluation of estimate and preservation.This computing is only carried out the 1st image pickup signal I1.Computing laplacian as following formula (39), will as following formula (40), act on again three-dimensionally value behind the LPF (optical low-pass filter) be made as contrast evaluation of estimate C2 (i, x, y).And, be written in the memory of systems control division 231.Follow execution in step S217.
C1(i，x，y)＝I1(i，x-1，y)+I1(i，x+1，y)+I1(i，x，y-1)+I1(i，x，y+1)-4I1(i，x，y)
……(39)
C2(i，x，y)＝C1(i，x-1，y-1)+C1(i，x，y-1)+C1(i，x+1，y-1)
+C1(i，x-1，y)+C1(i，x，y)+C1(i，x+1，y)
+C1(i，x-1，y+1)+C1(i，x，y+1)+C1(i，x+1，y+1)……(40)
In step S217, check the photography number, carry out branch.At first, as following formula (41), make counter i increase by 1.If counter i is littler than photography number Np, then follow execution in step S212.The number Np if counter i equals to photograph then follows execution in step S221.
i＝i+1 ……(41)
In step S221 and step S222, auto-focus control part 242 utilizes the data on the memory of systems control division 231, is chosen in the image that focusing position is photographed.
In step S221, make the auto-focus piece.OBL that the central authorities of image-region are neighbouring as the auto-focus piece.In addition, this piece might not be near the central authorities, and user's that also can the mirror operation video camera (for example by the sensor viewpoint direction) such as the meanings made piece.Then, execution in step S222.
In step S222, be chosen in the image that focusing position is photographed.As explanation among Fig. 9 like that, the contrast of the image of being photographed at focusing position is for very big.According to this principle, be chosen in the image that focusing position is photographed.At first, as following formula (42), (auto-focus y) is used contrast evaluation of estimate C3 with on average controlling as auto-focus in the piece for i, x with the contrast evaluation of estimate C2 in each photography.In addition, ∑ is represented the summation of auto-focus in the piece.Then, will provide peaked i among the C3 (i) as focusing Counter Value ip, the image at expression focusing position place.Then, execution in step S230.
C3(i)＝|∑C2(i，x，y)|/N ……(42)
Ip: in C3 (i), provide peaked i ... (43)
In step S230, edge parallax operational part 243 utilizes the data on the memory of systems control division 231, detects the edge, detects its parallax.Same with the step S122 of execution mode 1, utilize the image at focusing position place, detect contrast evaluation of estimate C2 (ip, x, 0 crosspoint y) (value is from just being changed to negative point, and value is changed to positive point from bearing).
Then, to the point on the edge (xe, ye), set with (xe ye) be rectangular of center, to the general image after photographing as following formula (44), with contrast evaluation of estimate C2 (i, x, the summation of absolute value y) as C4 (i, xe, ye).Here, the summation in the ∑ rectangular blocks representing to set.And according to principle illustrated in fig. 9, as following formula (45), (xe, (xe ye) is made as and provides C4 (i, xe, peaked i ye) ipe of the focusing position of ye) locating with the point of expression on the edge.Here, utilize formula (9), formula (37), (xe, (xe ye) is expressed as following formula (46) to the parallax Δ of ye) locating with the point on the edge.Then, execution in step S240.
C4(i，xe，ye)＝∑|C2(i，x，y)| ……(44)
Ipe (xe, ye): (i, xe provide peaked i in ye) to C4 ... (45)
Δ(xe，ye)＝(kx2/kx)ipe(xe，ye) ……(46)
In step S240, general image regional disparity operational part 244 utilizes the data on the memory of systems control division 231, according to the point on the edge (xe, the parallax Δ of ye) locating (xe, ye) the parallax Δ in the computing general image zone (x, y).As by the parallax Δ on the roughly the same edge (xe, ye) the parallax Δ of area surrounded (x, y), adopt parallax Δ on this edge (xe, ye).Then, execution in step S250.
In step S250, parallax correction portion 245 is utilized the data on the memory of systems control division 231, and (x y) carries out after the parallax correction parallax Δ in utilizing whole zone, and it is synthetic to carry out image.And its result is written in the memory of systems control division 231.Because the 1st imaging apparatus 123a and the 4th imaging apparatus 123d mainly are subjected to light to the green composition of object light, so the 1st image pickup signal I1 and the 4th image pickup signal I4 are the information signals of the green composition of object light.In addition, because the 2nd imaging apparatus 123b mainly is subjected to light to the blue composition of object light, so the 2nd image pickup signal I2 is the information signal of the blue composition of object light.And then, because the 3rd imaging apparatus 123c mainly is subjected to light to the red composition of object light, so the 3rd image pickup signal I3 is the information signal of the red composition of object light.Owing to predict (x, y) parallax of the 1st imaging apparatus 123a in and the 4th imaging apparatus 123d be (Δ (and x, y), Δ (x, y)) is so shown in following formula (47), to represent (x, y) (x y) is made as the 1st image pickup signal I1 (ip to the G of the green intensity in, x, y) and the 4th image pickup signal I4 (ip, the x-Δ (x, y), y-Δ (x, y)) average.In addition owing to predict (x, y) parallax of the 1st imaging apparatus 123a in and the 2nd imaging apparatus 123b be (Δ (and x, y), 0), so as following formula (48), will represent (x, y) B (x of the intensity of the blueness in, y) be made as the 2nd image pickup signal I2 (ip, the x-Δ (x, y), y).And then, owing to predict (x, y) parallax of the 1st imaging apparatus 123a in and the 3rd imaging apparatus 123c be (0, Δ (x, y)), so as following formula (49), will represent (x, y) R (x of the red intensity in, y) be made as the 3rd image pickup signal I3 (ip, x, y-Δ (x, y)).Then, execution in step S260.
G(x，y)＝[I1(ip，x，y)+I4(ip，x-Δ(x，y)，y-Δ(x，y))]/2……(47)
B(x，y)＝I2(ip，x-Δ(x，y)，y) ……(48)
R(x，y)＝I3(ip，x，y-Δ(x，y)) ……(49)
In step S260, output image.The S152 of this step and execution mode 1 similarly moves.Then, carry out S299.
In step S299, end process.
Constitute, move like that by above, have following effect.
Utilize the parallax at the Contrast Detection edge of a plurality of images,, carry out image according to this parallax and synthesize and carry out the parallax correction, so that reduce the influence of parallax according to the parallax in this parallax computing general image zone.Thus, can access the distinct image that in the general image zone, has reduced the influence of parallax.
In addition, in the video camera module of execution mode 2, the contrast evaluation of estimate of utilization photography is repeatedly obtained the parallax at edge, but the present invention is not limited to this.For example, also can set with the point on the edge of obtaining is that the piece at center, the parallax of obtaining this piece by the method for using in execution mode 1 adopt this parallax.In addition, also the several points on the edge can be merged, not be as point but come the computing parallax as line.In addition, also the edge of wire can be cut apart at end points and breakout and made a plurality of line segments, its parallax of computing.In addition, also can make comprise this line segment around piece, obtain the parallax of this piece by the method for in execution mode 1, using.In addition, also can be with parallax of obtaining according to the contrast evaluation of estimate and the parallax combination of obtaining by the method for in execution mode 1, using.For example, also can use the coefficient of average parallax or study actuator operated flow function.
In addition, in execution mode 1 and execution mode 2, be divided into piece by marginal information, but the present invention is not limited to this.For example also can be divided into a plurality of rectangular blocks.
(execution mode 3)
The video camera module of relevant embodiments of the present invention 3 is divided into piece with camera watch region, the parallax of each piece of computing, according to the contrast value of each piece computing with moved the relevant of the picture behind the parallax, estimate parallax.When relevant big and contrast is big, judge that the parallax of being obtained is correct, carry out common parallax correction according to parallax.And, in relevant less and contrast hour, judge that the parallax of being obtained for low precision, carries out the parallax correction that low contrast is used.And then, when relevant less and contrast is big, thus be judged as because to contain the parallax that the subject of a plurality of distances obtains incorrect, and piece is cut apart again.Like this, in the general image zone, obtain distinct image.
Below, describe with reference to the video camera module of accompanying drawing relevant embodiments of the present invention 3.
Figure 23 is the cutaway view of structure of the video camera module of the relevant embodiments of the present invention 3 of expression.Except the SLSI325 of the circuit part 320 of video camera module 301, same with execution mode 1, for giving identical label and omit explanation to them with the same parts of execution mode 1.
Figure 24 is the block diagram of the video camera module of relevant embodiments of the present invention 3.SLSI325 has systems control division 331, imaging apparatus drive division 132, image pickup signal input part 133, actuator operated amount efferent 134, image processing part 335, reaches input and output portion 136.In addition, circuit part 320 also has amplifier 126 except above-mentioned structure.
DigitalSignal Processor) or comprise both and constitute image processing part 335 is by logical circuit or DSP (digital signal processor:, utilize the memorizer information of systems control division 331 to carry out various image processing.Image processing part 335 has auto-focus control part 341, piece cutting part 342, parallax operational part 343, related operation portion 344, contrast operational part 345, piece, and cutting part 346, common parallax correction portion 347, low contrast are used parallax correction portion 349 with parallax correction portion 348 and low being correlated with again.
Figure 25 is the flow chart of action of the video camera module of the relevant embodiments of the present invention 3 of expression.By the systems control division 331 of SLSI325, video camera module 301 is moved as this flow chart.
In step S3000, begin action.For example, detect the situation that shutter release button (not shown) etc. is pressed by last bit CPU (not shown), via the beginning of 136 pairs of video camera module 301 command action of input and output portion, video camera module 301 begins action.Then, execution in step S3100.
In step S3100, auto-focus control part 341 is carried out auto-focus control.Figure 26 is the flow chart of action of the auto-focus control of the relevant embodiments of the present invention 3 of expression.The details of the action of the flowcharting step S3100 of Figure 26.
In the natural focal point control of step S3100, execution in step S3121 at first.
In step S3121, i is initialized as 0 with counter.Then, execution in step S3122.
In step S3122, the position command of computing actuator.As following formula (50), utilize counter i computing actuator position instruction Xact.In addition, position command Xact represents that the focusing position with the infinity picture is benchmark, is the instruction of positive position towards the subject direction.Here, kx is the value of setting.Then, execution in step S3123.
Xact＝kx·i ……(50)
In step S3123, utilize the operational ton functional operation actuator operated amount shown in the following formula (51) (applying voltage) Vact to what the coil of actuator movable part 115 applied.Here, ka, kb are respectively the values of setting.Then, execution in step S3124.
Vact＝ka·Xact+kb ……(51)
In step S3124, make actuator action.The voltage signal that 134 changes of actuator operated amount efferent are exported is so that become Vact via the amplifier 126 after-applied voltages of the coil (not shown) of actuator movable part 115 of giving.Then, execution in step S3125.
In step S3125, carry out photography.By the order of systems control division 331, imaging apparatus drive division 132 is exported the signal that is used for carrying out electronic shutter and transmission at any time.The 1st image pickup signal input part 133a, the 2nd image pickup signal input part 133b, the 3rd image pickup signal input part 133c, and the signal Synchronization ground input of the 4th image pickup signal input part 133d and 132 generations of imaging apparatus drive division is as the 1st imaging apparatus 123a, the 2nd imaging apparatus 123b, the 3rd imaging apparatus 123c, and the image pickup signal of the analog signal of each image of being exported of the 4th imaging apparatus 123d, by CDS fixedly noise remove, automatically regulate input gain by AGC, by ADC analog signal is transformed to digital value, become the 1st image pickup signal I1 (x, y), the 2nd image pickup signal I2 (x, y), the 3rd image pickup signal I3 (x, y), and the 4th image pickup signal I4 (x, y), digital value is written in the memory of specified address of systems control division 331.Figure 12 is the figure of coordinate of image pickup signal of the video camera module of the relevant embodiments of the present invention 1 of explanation.I1 (x, y) the 1st image pickup signal of y on the x, vertical direction on the expression horizontal direction.The pixel count longitudinally of the image of being imported is H, and horizontal pixel count is L, and total pixel number is H * L, and x is from 0 to L-1 variation, and y changes from 0 to H-1.The 2nd image pickup signal I2 (x, y), the 3rd image pickup signal I3 (x, y), the 4th image pickup signal I4 (x, y) too.That is, I2 (x, y), I3 (x, y) and I4 (x, y) expression is the 2nd image pickup signal of y on x, the vertical direction on the horizontal direction, the 3rd image pickup signal, and the 4th image pickup signal respectively.The pixel count longitudinally of the image of being imported is H, and horizontal pixel count is L, and total pixel number is H * L, and x is from 0 to L-1 variation, and y changes from 0 to H-1.Then, execution in step S3216.
In step S3126, set auto-focus control piece.With near the piece of the rectangular-shaped zone the center of image-region as auto-focus control usefulness.In addition, this piece might not be near the central authorities, and user that also can the mirror operation video camera (for example by the sensor viewpoint direction) such as the meanings sets piece.In addition, also can not single piece but select a plurality of, in a plurality of, use auto-focus control described later average with the contrast evaluation of estimate.In addition, also can select at least 1 piece as auto-focus control piece then a plurality of middle computing auto-focus control described later contrast evaluations of estimate.Then, execution in step S3127.
In step S3127, utilize the data on the memory of systems control division 331, make auto-focus control contrast evaluation of estimate.This computing is that the auto-focus control of the 1st image pickup signal I1 is carried out with the pixel of piece.Formula described as follows (52) is such, computing as 2 rank differential of x direction and y direction and the absolute value of laplacian, as following formula (53), act on LPF (optical low-pass filter three-dimensionally, Low Pass Filter), it in controlling with piece, auto-focus is on average obtained auto-focus contrast evaluation of estimate C3 as following formula (54).Here, Naf is the pixel count of auto-focus control with piece.Then, execution in step S3128.
C1(x，y)＝|I1(x-1，y)+I1(x+1，y)+I1(x，y-1)+I1(x，y+1)-4I1(x，y)|
……(52)
C2(x，y)＝C1(x-1，y-1)+C1(x，y-1)+C1(x+1，y-1)
+C1(x-1，y)+C1(x，y)+C1(x+1，y)
+C1(x-1，y+1)+C1(x，y+1)+C1(x+1，y+1) ……(53)
C3＝∑C2(x，y)/Naf ……(54)
In step S3128, as following formula (55), contrast evaluation of estimate C3 as C3 (i), is written in the memory of systems control division 331.Then, carry out S3129.
C3(i)＝C3 ……(55)
In step S3129, as following formula (56), counter i is added 1.Then, execution in step S3130.
i＝i+1 ……(56)
In step S3130, counter i and threshold value Saf are compared, carry out branch according to its result.Than threshold value Saf hour (comparative result of step S3130 is a "Yes"), follow execution in step S3122 at counter i.On the other hand, be threshold value Saf when above (comparative result of step S3130 is a "No") at counter i, then execution in step S3140.Like this, in step S3121, counter i is initialized as 0, in step S3129, counter i is added 1, in step S3130, carry out branch, thus the processing from S3122 to S3128 is repeated Saf time by counter i.
In step S3140, the evaluation of degree of comparing evaluation of estimate C3.As shown in Figure 9, at the focusing position place, contrast evaluation of estimate C3 becomes maximum.As following formula (57), will provide this peaked Counter Value i and be made as and provide the peaked Counter Value iaf of contrast.Then, execution in step S3151.
Iaf=provides the peaked i of C3 ... (57)
In step S3151, the position command of computing actuator.As following formula (58), utilize the position command Xact that provides the peaked Counter Value iaf of contrast computing actuator.In addition, position command Xact represents that the focusing position with the infinity picture is benchmark, is the instruction of positive position towards the subject direction.Then, execution in step S3152.
Xact＝kx·iaf ……(58)
In step S3152, utilize operational ton functional operation actuator operated amount (applying voltage) Vact to what the coil of actuator movable part 115 applied.This action is same with step S3123, omits explanation.Then, execution in step S3153.
In step S3153, make actuator action.This action is same with step S3124, omits explanation.Then, execution in step S3160.
In step S3160, finish auto-focus control, get back to main program.Thereby, then carry out the step S3200 of Figure 25.
In step S3200, carry out the parallax correction.Figure 27 is the flow chart of action of the parallax correction of the relevant embodiments of the present invention 1 of expression.The details of the action of the flowcharting step S3200 of Figure 27.
In the error correction of step S3200, execution in step S3220 at first.
In step S3220, carry out photography.Utilize the order of systems control division 131, imaging apparatus drive division 132 is exported the signal that is used for carrying out electronic shutter and transmission at any time.The 1st image pickup signal input part 133a, the 2nd image pickup signal input part 133b, the 3rd image pickup signal input part 133c, and the signal Synchronization ground input of the 4th image pickup signal input part 133d and 132 generations of imaging apparatus drive division is as the 1st imaging apparatus 123a, the 2nd imaging apparatus 123b, the 3rd imaging apparatus 123c, and the image pickup signal of the analog signal of each image of being exported of the 4th imaging apparatus 123d, by CDS fixedly noise remove, automatically regulate input gain by AGC, by ADC analog signal is transformed to digital value, become the 1st image pickup signal I1 (x, y), the 2nd image pickup signal I2 (x, y), the 3rd image pickup signal I3 (x, y), and the 4th image pickup signal I4 (x, y), digital value is written in the memory of specified address of systems control division 331.Figure 12 is the figure of coordinate of image pickup signal of the video camera module of the relevant embodiments of the present invention 1 of explanation.I1 (x, y) the 1st image pickup signal of y on the x, vertical direction on the expression horizontal direction.The pixel count longitudinally of the image of being imported is H, and horizontal pixel count is L, and total pixel number is H * L, and x is from 0 to L-1 variation, and y changes from 0 to H-1.The 2nd image pickup signal I2 (x, y), the 3rd image pickup signal I3 (x, y), the 4th image pickup signal I4 (x, y) too.That is, I2 (x, y), I3 (x, y) and I4 (x, y) expression is the 2nd image pickup signal of y on x, the vertical direction on the horizontal direction, the 3rd image pickup signal, and the 4th image pickup signal respectively.The pixel count longitudinally of the image of being imported is H, and horizontal pixel count is L, and each total pixel number is H * L, and x is from 0 to L-1 variation, and y changes from 0 to H-1.Then, execution in step S3230.
In step S3230, piece cutting part 342 utilizes the data on the memory of systems control division 331, carries out piece and cuts apart.Then, its result is written in the memory of systems control division 331.Figure 28 is the explanation figure that piece is cut apart in the video camera module of relevant embodiments of the present invention 3.As shown in figure 28, with the 1st image pickup signal I1 along horizontal partition be M piece, longitudinally be divided into N piece, amount to and to be divided into M * N, each piece is represented with Bi.Here, i changes from 0 to M * N-1.Then, execution in step S3240.
In step S3240, parallax operational part 343 utilizes the data on the memory of systems control division 331, the parallax value of each piece of computing.Then, be written in the memory of systems control division 331.At first, to each piece (B0, B1 ..., Bi ..., BMN-1) computing parallax evaluation of estimate (R0 (k), R1 (k) ..., Ri (k) ..., R MN-1 (k), k=0,1 ..., kmax).Figure 29 be explanation in the video camera module of relevant embodiments of the present invention 3, the figure in the computing zone of parallax evaluation of estimate.(also representing as I1) zone of being represented by Bi is i the piece of obtaining according to the 1st image pickup signal I1 at step S3230.The zone of being represented by I4 is that Bi is moved k, moves zone behind the k along the y direction along the x direction.And, for each regional general image signal I1 (x, y), (x-k, y-k), the absolute value difference summation shown in the following formula of computing (59) is used as parallax evaluation of estimate Ri (k) to I4.
Ri(k)＝∑∑|I1(x，y)-I4(x-k，y-k)| ……(59)
This parallax evaluation of estimate Ri (k) represents the 1st picture signal I1 of i piece Bi and leaves respectively along x, y direction (what kind of k, the 4th picture signal I4 in zone k) have be correlated with, and expresses: value more little relevant big more (very approximate).As shown in figure 15, parallax evaluation of estimate Ri (k) changes according to the value of k, has minimum when k=Δ i.I the piece Bi that this expression makes the 4th picture signal I4 moves (Δ i ,-Δ i) respectively and the picture signal that obtains and the 1st picture signal I1 have relevant (the most similar) of maximum along x, y direction.Thereby, be (Δ i, Δ i) for the 1st image pickup signal I1 of i piece Bi and the x of the 4th image pickup signal I4, the parallax of y direction as can be known.Below, with this Δ (i, j) be called i piece B (i, parallax value Δ j) (i, j).Like this, from i=0 to i=M * N-1, obtain the parallax value Δ i of Bi.Then, execution in step S3251.
In step S3251, related operation portion 344 utilizes the data on the memory of systems control division 331, the parallax precision evaluation correlation of each piece of computing.And write results in the memory of systems control division 331.Figure 30 is the figure that the computing zone of correlation is used in explanation parallax precision evaluation in the video camera module of relevant embodiments of the present invention 3.(also representing as I1) zone of being represented by Bi is i the piece of obtaining according to the 1st image pickup signal I1 at step S3230.The zone of being represented by I4 is that Bi is moved Δ i, moves zone behind the Δ i along the y direction along the x direction.And (x y), I4 (x-Δ i, y-Δ i), as following formula (60), is made as parallax precision evaluation correlation R2i for each regional general image signal I1.Here, the ∑ ∑ is represented the summation in the piece Bi, and avg represents the summation averaging in the piece Bi.Then, execution in step S3252.
R2i＝∑∑{I1(x，y)-avg(I1(x，y))}
×{I4(x-Δi，y-Δi)-avg(I4(x-Δi，y-Δi))}
/√∑∑{I1(x，y)-avg(I1(x，y))} 2
/√∑∑{I4(x-Δi，y-Δi)-avg(I4(x-Δi，y-Δi))} 2 ……(60)
In step S3252, select the parallax correcting mode of each piece Bi with correlation R2i according to the parallax precision evaluation.As following formula (61), when the parallax precision evaluation is bigger than threshold value R20 with correlation R2i, 1 of the common parallax correction of expression is updated among the parallax correcting mode sign Fi, be saved in the memory of systems control division 331.Then, execution in step S3261.
Fi＝1 (R2i＞R20) ……(61)
In step S3261, contrast operational part 345 utilizes the data on the memory of systems control division 331, the parallax precision evaluation contrast value of each piece of computing.And be written in the memory of systems control division 331.As following formula (62), computing as 2 rank differential of x direction and y direction and the absolute value of laplacian, as following formula (63), act on LPF (optical low-pass filter three-dimensionally, Low Pass Filter), it is on average obtained the parallax precision evaluation with contrast evaluation of estimate C6i as following formula (64), be written in the memory of systems control division 331 in each piece Bi.Here, NBi is the pixel count of piece Bi.In addition, the contrast evaluation of estimate is not limited to from formula (62) to formula (64), as long as it is just passable to show contrast, for example also can in formula (62), replace laplacian and use 1 subdifferential, or be not the absolute value of laplacian and use x, y 2 rank differential separately absolute value and.In addition, also can use the LPF of formula (63) form in addition.And then, also can use the average standardization that removes grade of the size of I1 to the result of formula (64).Then, execution in step S3262.
C4(x，y)＝|I1(x-1，y)+I1(x+1，y)+I1(x，y-1)+I1(x，y+1)-4I1(x，y)|
……(62)
C5(x，y)＝C1(x-1，y-1)+C1(x，y-1)+C1(x+1，y-1)
+C1(x-1，y)+C1(x，y)+C1(x+1，y)
+C1(x-1，y+1)+C1(x，y+1)+C1(x+1，y+1) ……(63)
C6i＝∑C5(x，y)/NBi ……(64)
In step S3262, select the parallax correcting mode of each piece Bi with contrast evaluation of estimate C6i according to the parallax precision evaluation.Here, in step S3262, only in step S3252,1 piece that is updated among the parallax correcting mode sign Fi not being estimated.Therefore, in step S3261, in the piece Bi of parallax correcting mode sign Fi=1, also can omit the computing of parallax precision evaluation with contrast evaluation of estimate C6i.As following formula (65), use contrast evaluation of estimate C6i than threshold value C60 hour in the parallax precision evaluation, the expression low contrast is updated among the parallax correcting mode sign Fi with 2 of parallax correction, be saved in the memory of systems control division 331.In addition, as following formula (66), be threshold value C60 when above with contrast evaluation of estimate C6i in the parallax precision evaluation, will expression low relevantly be updated among the parallax correcting mode sign Fi with 3 of parallax correction is saved in the memory of systems control division 331.Then, execution in step S3271.
Fi=2 (C6i＜C60 and be not Fi=1) ... (65)
Fi=3 (C6i 〉=C60 and be not Fi=1) ... (66)
In step S3271, the size according to the size of parallax correcting mode sign and piece judges whether and need carry out cutting apart again of piece, carries out branch according to its judged result.At the parallax correcting mode sign Fi of at least 1 piece is 3 and when the pixel count of minimum piece is bigger than S0 in all piece Bi, is judged as and need carries out the cutting apart again of piece (judged result of step S3271 for " be "), then execution in step S3272.All parallax correcting mode sign Fi be 1 or 2 o'clock or among all piece Bi the pixel count of minimum piece be that S0 is when following, being judged as does not need to carry out the cutting apart again of piece (judged result of step S3271 is a "No"), then, execution in step S3280).
In step S3272, piece cutting part 346 again utilizes data on the memory of systems control division 331, and piece is cut apart again.Figure 31 (A) is the figure of explanation state before the cutting apart again of piece in the video camera module of relevant embodiments of the present invention 3, and Figure 31 (B) is the figure of explanation state after the cutting apart again of piece in the video camera module of relevant embodiments of the present invention 3.In Figure 31, for simply, the expression piece be 4, will be in parallax correcting mode sign Fi substitution low relevant 3 the piece B2 (Figure 31 (A)) of expression with the parallax correction be divided into the example (Figure 31 (B)) of 4 piece B2, B4, B5, B6.The number (2) and the untapped number (4,5,6) of the piece of number after desirable cutting apart of the piece after expression is cut apart piece B2 again.Then, carry out S3240.
Like this, the piece that will hang down relevant (Fi=3) is cut apart (step S3272) again, computing is with respect to the parallax (step S3240) of the piece after cutting apart again, computing is also estimated relevant (step S3251, step S3252), computing is also estimated contrast (step S3261, step S3262), judge whether piece cut apart again (step S3271), carry out the piece of low relevant (Fi=3) is cut apart again the action of (step S3272) more repeatedly, the parallax correcting mode sign Fi that becomes S0 or all piece up to the size of piece minimum in all pieces becomes 1 or 2.In addition, from step S3240 to step S3272, because the size by having changed piece cutting apart again of piece and the number of piece, so can suitably change the action of these steps.
By above processing, become moment (promptly beginning the moment of execution in step S3280) of N in the judged result of above-mentioned step S3271, with the common parallax correction of expression 1, the expression low contrast with the parallax correction 2 or represent low relevantly be updated among the parallax correcting mode sign Fi at all piece Bi that are included in the piece of cutting apart among the step S3272 with 3 of parallax correction.In step S3280, usually parallax correction portion 347, low contrast are utilized data on the memory of systems control division 331 with parallax correction portion 348, low being correlated with parallax correction portion 349, each piece is carried out the corresponding parallax correction of value with parallax correcting mode sign Fi, composograph is written to its result in the memory of systems control division 331.
In step S3280, in the piece of parallax correcting mode sign Fi=1, the parallax precision evaluation is bigger with correlation R2i.Figure 32 is the figure that the relation of correlation R2i and parallax correcting mode sign Fi is used in explanation parallax precision evaluation in the video camera module of relevant embodiments of the present invention 3.The 1st image pickup signal I1 (x in i piece Bi, y) with the 4th image pickup signal I4 (the x-Δ i that has moved parallax Δ i along x direction, y direction, y-Δ i) under the Yi Zhi situation, the parallax precision evaluation of obtaining in step S3251 is 1 with correlation R2i, when both are random noise under the like that different randomly situation, the parallax precision evaluation of obtaining in step S3251 is 0 with correlation R2i, be at both under the situation of the relation as black white reverse, the parallax precision evaluation of obtaining in step S3251 is-1 with correlation R2i.Like this, when both are approximate the parallax precision evaluation big and approach 1 with correlation R2i, when both were not approximate, the parallax precision evaluation was less and depart from 1 with correlation R2i.Here, shown in figure 32, threshold value R20 is set at approaches 1 value (for example 0.9), thus, when the parallax precision evaluation is big with correlation R2i, can judge that (x y) is similar to the 4th image pickup signal I4 (x-Δ i, y-Δ i) that has moved parallax Δ i the 1st interior image pickup signal I1 of i piece Bi.This precision that is illustrated in parallax Δ i among i the piece Bi is higher, and the 1st image pickup signal I1 (x, y) with moved the subject that only has same distance between the 4th image pickup signal I4 (x-Δ i, y-Δ i) of parallax Δ i.Therefore, by the 4th image pickup signal I4 is moved Δ i, moves Δ i along the y direction along the x direction, can reproduce i the image pickup signal among the piece Bi.Because the 2nd image pickup signal I2 only has the influence of parallax on the x direction, the 3rd image pickup signal I3 only has the influence of parallax on the y direction, so, by the 2nd image pickup signal I2 is moved Δ i along the x direction and can reproduce the image pickup signal of the blueness among the piece Bi, by the 3rd image pickup signal I3 is moved Δ i along the y direction and can reproduce the image pickup signal of the redness among the piece Bi.
So in the piece of parallax correcting mode sign Fi=1, usually parallax correction portion 347 is utilized the data on the memory of systems control division 331, utilize the parallax value corresponding that each piece is carried out carrying out image and synthesizing after the parallax correction with this piece.And its result is written in the memory of systems control division 331.Because the 1st imaging apparatus 123a and the 4th imaging apparatus 123d mainly are subjected to light to the green composition of object light, so the 1st image pickup signal I1 and the 4th image pickup signal I4 are the information signals of the green composition of object light.In addition, because the 2nd imaging apparatus 123b mainly is subjected to light to the blue composition of object light, so the 2nd image pickup signal I2 is the information signal of the blue composition of object light.And then, because the 3rd imaging apparatus 123c mainly is subjected to light to the red composition of object light, so the 3rd image pickup signal I3 is the information signal of the red composition of object light.In i piece Bi, because predicting the parallax of the 1st imaging apparatus 123a and the 4th imaging apparatus 123d is (Δ i, Δ i), so as following formula (67), with the remarked pixel coordinate (x, y) (x y) is made as the 1st image pickup signal I1 (x to the G of the green intensity in, y) and the 4th image pickup signal I4 (x-Δ i, y-Δ i) average.By being averaged like this, can reduce the influence of random noise.In addition, be (Δ i, 0) owing to predict the parallax of the 1st imaging apparatus 123a and the 2nd imaging apparatus 123b, thus as following formula (68), with the remarked pixel coordinate (x, y) B of the intensity of the blueness in (x, y) be made as the 2nd image pickup signal I2 (x-Δ i, y).And then, be (0, Δ i) owing to predict the parallax of the 1st imaging apparatus 123a and the 3rd imaging apparatus 123c, so as following formula (69), will represent that (x, y) (x y) is made as the 3rd image pickup signal I3 (x, y-Δ i) to the R of the red intensity in.
G(x，y)＝[I1(x，y)+I4(x-Δi，y-Δi)]/2 ……(67)
B(x，y)＝I2(x-Δi，y) ……(68)
R(x，y)＝I3(x，y-Δi) ……(69)
In the piece of parallax correcting mode sign Fi=2, the parallax precision evaluation is less with correlation R2i in step S3280, and the parallax precision evaluation is less with contrast evaluation of estimate C6i.Figure 33 is the figure that the relation of contrast evaluation of estimate and parallax correcting mode sign is used in explanation parallax precision evaluation in the video camera module of relevant embodiments of the present invention 3.The 1st image pickup signal I1 (x in i piece Bi, y) expression clearly, when being the bigger image of deep or light variation, the parallax precision evaluation of obtaining in step S3261 becomes big with contrast evaluation of estimate C6, the 1st image pickup signal I1 (x in i piece Bi, y) expression fuzzy, when being the less image of deep or light variation, the parallax precision evaluation of obtaining in step S3261 approaches 0 with contrast evaluation of estimate C6.Here, as shown in figure 33, setting threshold C60, thus, in the parallax precision evaluation with the contrast evaluation of estimate hour, (x, y) Biao Shi image is the image of deep or light less, shortage variation to the 1st image pickup signal I1 in i piece Bi.Therefore, big with respect to the noise composition change of signal component, the precision that can judge parallax Δ i is relatively poor.In addition, owing to be deep or light less image,, there is not the problem in the practicality even in the image that reproduces, contain some errors yet.So, mainly based on the 1st image pickup signal I1 (x, information regeneration color y).In addition, also can based on the 2nd image pickup signal I2 (x, y), the 3rd image pickup signal I3 (x, y) or the 4th image pickup signal I4 (x, information regeneration color y).But, compare with red, blueness, visual sensitivity is the highest to green, so be preferably based on the information of green composition with object light the 1st image pickup signal I1 (x, y) or the 4th image pickup signal I4 (x, y).In addition because the precision of parallax Δ i is relatively poor, so preferred use separately the 1st image pickup signal I1 (x, y) or the 4th image pickup signal I4 (x, y).
So in the piece of parallax correcting mode sign Fi=2, low contrast utilizes the data on the memory of systems control division 331 with parallax correction portion 348, utilize the parallax value corresponding that each piece is carried out the parallax correction with this piece after, it is synthetic to carry out image.And its result is written in the memory of systems control division 331.Because the 1st imaging apparatus 123a and the 4th imaging apparatus 123d mainly are subjected to light to the green composition of object light, so the 1st image pickup signal I1 and the 4th image pickup signal I4 are the information signals of the green composition of object light.In addition, because the 2nd imaging apparatus 123b mainly is subjected to light to the blue composition of object light, so the 2nd image pickup signal I2 is the information signal of the blue composition of object light.And then, because the 3rd imaging apparatus 123c mainly is subjected to light to the red composition of object light, so the 3rd image pickup signal I3 is the information signal of the red composition of object light.In i piece Bi, the parallax that predicts the 1st imaging apparatus 123a and the 4th imaging apparatus 123d is (Δ i, Δ i), but it is bigger to be contemplated to error, so as following formula (70), with remarked pixel coordinate (x, the G of the green intensity y) (x, y) be made as the 1st image pickup signal I1 (x, y).In addition, as following formula (71), (x y) is made as that (x y) multiply by value behind the proportionality coefficient kB to the 1st image pickup signal I1 to the B of the intensity that expression is blue.Here, proportionality coefficient kB is a constant.In addition, proportionality coefficient kB is each piece Bi and changing relatively also, also can be: the 2nd image pickup signal I2 (x, the value of the central authorities of piece Bi y) is with respect to the 1st image pickup signal I1 (x, the ratio of the value of the central authorities of piece Bi y), perhaps moved the 2nd image pickup signal I2 (the x-Δ i of parallax Δ i along the x direction, the value of the central authorities of piece Bi y) is with respect to the 1st image pickup signal I1 (x, the ratio of the value of the central authorities of piece Bi y), perhaps the 2nd image pickup signal I2 (x of piece Bi, y) on average with respect to the 1st image pickup signal I1 (x, y) average ratio, perhaps piece Bi's has moved the 2nd image pickup signal I2 (the x-Δ i of parallax Δ i along the x direction, y) on average with respect to the 1st image pickup signal I1 (x, average ratio y).And then as following formula (72), (x y) is made as that (x y) multiply by value behind the proportionality coefficient kR to the 1st image pickup signal I1 with the R of expression red intensity.Here, proportionality coefficient kR is a constant.In addition, proportionality coefficient kR is each piece Bi and changing relatively also, also can be: the 3rd image pickup signal I3 (x, the value of the central authorities of piece Bi y) is with respect to the 1st image pickup signal I1 (x, the ratio of the value of the central authorities of piece Bi y), perhaps moved the 3rd image pickup signal I3 (x of parallax Δ i along the y direction, the value of the central authorities of piece Bi y-Δ i) is with respect to the 1st image pickup signal I1 (x, the ratio of the value of the central authorities of piece Bi y), perhaps the 3rd image pickup signal I3 (x of piece Bi, y) on average with respect to the 1st image pickup signal I1 (x, y) average ratio, perhaps piece Bi's has moved the 3rd image pickup signal I3 (x of parallax Δ i along the y direction, y-Δ i) on average with respect to the 1st image pickup signal I1 (x, average ratio y).In addition, because it is deep or light less, if so even consider the also imperceptible situation of some errors that contains, same in the time of then also can be with parallax correcting mode sign Fi=1, be (Δ i owing to predict the parallax of the 1st imaging apparatus 123a and the 2nd imaging apparatus 123b, 0), so such suc as formula (68), with remarked pixel coordinate (x, y) B (x of the intensity of the blueness in, y) be made as the 2nd image pickup signal I2 (x-Δ i, y), and then, be (0 owing to predict the parallax of the 1st imaging apparatus 123a and the 3rd imaging apparatus 123c, Δ i), so such suc as formula (69), will represent (x, y) R (x of the red intensity in, y) be made as the 3rd image pickup signal I3 (x, y-Δ i).
G(x，y)＝I1(x，y) ……(70)
B(x，y)＝I1(x，y)*kB ……(71)
R(x，y)＝I1(x，y)*kR ……(72)
In the piece of parallax correcting mode sign Fi=3, the parallax precision evaluation is less with correlation R2i in step S3280, and the parallax precision evaluation is bigger with contrast evaluation of estimate C6i.Shown in figure 32, threshold value R20 is set at approaches 1 value (for example 0.9), thus, in the parallax precision evaluation hour with correlation R2i, can judge that (x y) is not similar to the 4th image pickup signal I4 (x-Δ i, y-Δ i) that has moved parallax Δ i the 1st interior image pickup signal I1 of i piece Bi.This is illustrated in the subject that contains a plurality of different subject distances among i the piece Bi, and parallax Δ i can not corresponding all subjects.
So in the piece of parallax correcting mode sign Fi=3, the data on the low relevant memory that utilizes systems control division 331 with parallax correction portion 349 further utilize the edge that piece is divided into a plurality of again, carry out the parallax correction, and carry out image and synthesize.And its result is written in the memory of systems control division 331.Figure 34 is the low relevant flow chart of using the action of parallax correction portion 349 of the relevant embodiments of the present invention 3 of expression.
In step S3310, begin low relevant action with parallax correction portion 349.Then, execution in step S3320.
In step S3320, computing rim detection contrast evaluation of estimate.This computing is only carried out the 1st image pickup signal I1.As following formula (73), the computing laplacian, again will be as following formula (74) with act on three-dimensionally behind the LPF (optical low-pass filter) value as rim detection with contrast evaluation of estimate C8 (x, y).Figure 35 (A) is the original image of low relevant rim detection with parallax correction portion 349 of the video camera module of the relevant embodiments of the present invention 3 of explanation, and Figure 35 (B) is that the rim detection of low relevant rim detection with parallax correction portion 349 of video camera module of the relevant embodiments of the present invention 3 of explanation is with the image of contrast evaluation of estimate.(x, y), it is such then to become Figure 35 (B) if the rim detection of the original image of Figure 35 (A) contrast evaluation of estimate C8 is calculated in through type (73), (74).In addition, use the bigger position of absolute value of black expression (74) among Figure 35 (B).Then, execution in step S3330.
C7(x，y)＝I1(x-1，y)+I1(x+1，y)+I1(x，y-1)+I1(x，y+1)-4I1(x，y)
……(73)
C8(x，y)＝C7(x-1，y-1)+C7(x，y-1)+C7(x+1，y-1)
+C7(x-1，y)+C7(x，y)+C7(x+1，y)
+C7(x-1，y+1)+C7(x，y+1)+C7(x+1，y+1) ……(74)
In step S3330, detect the edge.Figure 35 (C) is the image at edge of low relevant rim detection with parallax correction portion 349 of the video camera module of the relevant embodiments of the present invention 3 of explanation.(x, 0 crosspoint y) (value is changed to positive point from just being changed to negative point and value from bearing) can detect the such edge of Figure 35 (C) by the rim detection that detects Figure 35 (B) contrast evaluation of estimate C8.Then, execution in step S3340.
In step S3340, piece is cut apart again.Shown in Figure 35 (C), for by the zone of surrounded by edges such as Bi, Bj, Bj+1 ... Bj+n gives label like that.In addition, j, j+1 ..., j+n is the number that does not use as the number of expression piece.Here, in Figure 35, in order simply to represent piece Bi is divided into again the example of 5 pieces, n=3.In addition, survey or disappear, also can utilize expansion algorithm or contraction algorithm to revise the edge for the flase drop that prevents the edge that noise etc. brings.Then, execution in step S3350.
In step S3350, the parallax value of each piece of computing.This action is same with step S3240, omits explanation.Then, execution in step S3360.
In step S3360, revise parallax, composograph.This action is same with the action of common parallax correction portion 347, omits explanation.Then, execution in step S3370.
In step S3370, finish low relevant action with the parallax correction.
More than, make step S3280 action, then execution in step S3290.
In step S3290, finish the parallax correction, return to main program.That is, then carry out step S3800 shown in Figure 25.
In step S3800, output image.Input and output portion 136 will as the G of the data on the memory of systems control division 331 (x, y), B (x, y), (x y) exports to bit CPU (not shown) and exterior display device (not shown) to R.In addition, also can export for example output of brightness, carrier chrominance signal etc., replace G (x, y), B (x, y), R (x, y).In addition, also can export value after the image processing such as white balance or γ correction.And then, also can export the data of having carried out such non-reversible compression such as reversible compression or JPEG.In addition, also can export a plurality of of them.Then, carry out S3900.
In step S3900, tenth skill.
By constituting as described above, moving, has following effect.
Such suc as formula (1), according to the subject distance A, the relative position of 4 object pictures that the 1st～the 4th lens section 113a～113d forms respectively changes.That is, if the subject distance A diminishes, then the parallax Δ becomes big.Therefore, at the same time during the different a plurality of subject of photo distance, for each subject, parallax Δ difference.In the video camera module of execution mode 3, in step S3230, be piece with the general image Region Segmentation, the parallax of each piece of computing in step S3240, in step S3280, carry out image and synthesize and carry out the parallax correction, so that reduce the influence of parallax according to the parallax of each piece.Thus, even when the different a plurality of subjects of adjusting the distance are photographed simultaneously, also can suitably revise each subject parallax, obtain in the general image zone, having reduced the distinct image of the influence of parallax.
In the video camera module of execution mode 3, in step S3230, be rectangular-shaped piece with the general image Region Segmentation, the parallax of each piece of computing in step S3240, in step S3280, carry out image and synthesize and carry out the parallax correction, so that reduce the influence of parallax according to the parallax of each piece.Thus, even when the different a plurality of subjects of adjusting the distance are photographed simultaneously, also can suitably revise each subject parallax, obtain in the general image zone, having reduced the distinct image of the influence of parallax.
In addition, the piece after cutting apart in step S3230 sometimes contains the situation of the different a plurality of objects of subject distance, in the case, and for each object picture, parallax difference.In the video camera module of execution mode 3, computing parallax precision evaluation correlation R2i in step S3251, in step S3252, estimate the precision of parallax Δ i with correlation R2i by estimating the parallax precision evaluation, computing parallax precision evaluation contrast evaluation of estimate C6i in step S3261, in step S3262, estimate the precision of parallax Δ i with contrast evaluation of estimate C6i by estimating the parallax precision evaluation, set parallax correcting mode sign Fi, the mode of decision parallax correction.Thus, can be by optimal method to each piece correction image, so can access the distinct image that in the general image zone, has further reduced the influence of parallax.
In addition, in the video camera module of execution mode 3, computing parallax precision evaluation contrast evaluation of estimate C6i in step S3261, in step S3262, use the size of the contrast of contrast evaluation of estimate C6i according to the parallax precision evaluation of each piece, judge how many precision the parallax Δ i to each piece computing has, when contrast is big, make parallax correcting mode sign Fi=1, in step S3280, carry out common parallax correction, in contrast hour, make parallax correcting mode sign Fi=2, in step S3280, carry out the parallax correction that low contrast is used.Thus, can be by optimal method to each piece correction image, so can access the distinct image that in the general image zone, has further reduced the influence of parallax.
In addition, in the video camera module of execution mode 3, computing parallax precision evaluation correlation R2i in step S3251.The parallax precision evaluation is to represent that the parallax of each piece computing is had one of parallax precision evaluation value of how many precision with correlation R2i, is to represent how many relevant values the picture that has moved parallax has.The parallax precision evaluation when big, is carried out common parallax correction with correlation R2i in step S3280 in step S3252, and in the parallax precision evaluation with correlation R2i hour, hang down in step S3280 is correlated with uses the parallax correction.Thus, can be by optimal method to each piece correction image, so can access the distinct image that in the general image zone, has further reduced the influence of parallax.
In addition, in the video camera module of execution mode 3, computing parallax precision evaluation correlation R2i in step S3251.The parallax precision evaluation is to represent that the parallax of each piece computing is had one of parallax precision evaluation value of how many precision with correlation R2i, is to represent how many relevant values the picture that has moved parallax has.When the parallax precision evaluation is big with correlation R2i, in step S3280, carry out common parallax correction, in the parallax precision evaluation hour with correlation R2i, in hanging down of step S3272 and step S3340 is correlated with parallax correction portion 349, piece is cut apart again, each piece after cutting apart is again carried out the parallax correction.That is, cut apart again being judged as low relevant piece.Thus, can be by optimal method to each piece correction image, so can access the distinct image that in the general image zone, has further reduced the influence of parallax.
In addition, in the video camera module of execution mode 3, computing parallax precision evaluation correlation R2i in step S3251.The parallax precision evaluation is to represent that the parallax of each piece computing is had one of parallax precision evaluation value of how many precision with correlation R2i, is to represent how many relevant values the picture that has moved parallax has.The precision evaluation of computing parallax is with contrast evaluation of estimate C6i in step S3261, and the parallax precision evaluation is to represent that the parallax of each piece computing is had one of parallax precision evaluation value of how many precision with contrast evaluation of estimate C6i, is what value of expression contrast height.When the parallax precision evaluation uses contrast evaluation of estimate C6i big with correlation R2i than big and parallax precision evaluation, in step S3280, carry out common parallax correction, the parallax precision evaluation with correlation R2i hour and the parallax precision evaluation with contrast evaluation of estimate C6i hour, in step S3280, carry out low contrast parallax correction, the parallax precision evaluation with correlation R2i less and parallax precision evaluation with contrast evaluation of estimate C6i when big, at step S3272, and step S3340's is low relevant with in the parallax correction portion 349 piece being cut apart again, carries out the parallax correction to cutting apart each piece afterwards again.That is,, then piece is cut apart again if be judged as low relevant and high-contrast.Thus, can be by optimal method to each piece correction image, so can access the distinct image that in the general image zone, has further reduced the influence of parallax.
In addition, in execution mode 3, the timing of image output is not limited to above-mentioned, also suitably preview output.For example, in execution mode 3, the output midway that yet can carry out auto-focus control in step S3100 does not have the image of parallax correction.In addition, also can after step S3271, only carry out the parallax correction, other pieces are not carried out the parallax correction, and upgrade preview screen the piece that can carry out the parallax correction.
In addition, in execution mode 3, (x y), detects the edge according to 0 crosspoint, but is not limited to this in step S3330 with contrast evaluation of estimate C8 suc as formula (74) such computing rim detection in step S3320.For example, also can make the histogram in the piece, minimum is made as threshold value, carry out piece by binaryzation and cut apart.Here, also can have a plurality of threshold values.In addition, also can utilize the absolute value of 1 subdifferential to carry out rim detection.
(execution mode 4)
Embodiments of the present invention 4 are under the situation of reading in of interlocking, and each is obtained parallax, and to carry out image synthetic, and each composograph is synthetic and become final image.Like this, by each is carried out the parallax computing, even when the subject of photography motion because of under the different situation of different each the images of the photography time of each, also can correctly obtain the parallax of each, utilize this synthesized parallax image, so can access the distinct image that in the general image zone, has further reduced the influence of parallax.
Below, describe with reference to the video camera module of accompanying drawing relevant embodiments of the present invention 4.
Figure 36 is the cutaway view of structure of the video camera module of the relevant embodiments of the present invention 4 of expression.Except the SLSI425 of the circuit part 420 of video camera module 401, same with execution mode 1, for giving identical label and omit explanation to them with the same parts of execution mode 1.
Figure 37 is the block diagram of the video camera module of relevant embodiments of the present invention 4.SLSI425 has systems control division 431, imaging apparatus drive division 132, image pickup signal input part 133, actuator operated amount efferent 134, image processing part 435, reaches input and output portion 136.In addition, circuit part 420 also has amplifier 126 except above-mentioned structure.
DigitalSignal Processor) or comprise both and constitute image processing part 435 is by logical circuit or DSP (digital signal processor:, utilize the memorizer information of systems control division 431 to carry out various image processing.Image processing part 435 has auto-focus control part 341, piece cutting part 342, parallax operational part 343, related operation portion 344, contrast operational part 345, piece, and cutting part 346, common parallax correction portion 347, low contrast are used parallax correction portion 349 and two field picture preparing department 451 with parallax correction portion 348, low being correlated with again.
Figure 38 (A)～Figure 38 (D) is the figure of field picture of the video camera module of the relevant embodiments of the present invention 4 of explanation.Integral image alternately disposes the 1st field picture and the 2nd field picture shown in Figure 38 (A).At first transmit the image pickup signal that constitutes the 1st field picture, then transmit the image pickup signal that constitutes the 2nd field picture.In common black and white image, by they are come reproduced image integral body every 1 row alternate configurations.Shown in Figure 38 (B), be L at the width of integral image (two field picture), during highly for H, shown in Figure 38 (C) and Figure 38 (D), the 1st field picture, and the width of the 2nd field picture be respectively L, highly be H/2.
Figure 39 is the flow chart of action of the video camera module of the relevant embodiments of the present invention 4 of expression.Figure 40 is the sequential chart of action of the video camera module of the relevant embodiments of the present invention 4 of expression.By the systems control division 431 of SLSI425, video camera module 401 moves shown in this flow chart and sequential chart like that.
In step S4000, begin action.For example, detect the situation that shutter release button (not shown) etc. is pressed by last bit CPU (not shown), via the beginning of 136 pairs of video camera module 401 command action of input and output portion, video camera module 401 begins action.Then, execution in step S4100.
In step S4100, auto-focus control part 341 is carried out auto-focus control.This action is same with the step S3100 of execution mode 3, omits explanation.But, as shown in figure 40, owing to only transmitting the 1st, only using the 1st field picture, so can suitably apply change.Therefore, compare with the situation of transmitted image integral body (the 1st and the 2nd), transmitting the needed time becomes roughly half, so can correspondingly shorten the time of auto-focus control.Then, execution in step S4200.
In step S4200, carry out the 1st parallax correction.This action is same with the step S3200 of execution mode 3, omits explanation.But, as shown in figure 40, only transmit the 1st, only use the 1st field picture, and suitably applied change so that make the red intensity of the 1st of expression Rf1 (x, y), the Gf1 (x of expression green intensity, y) and the Bf1 of the blue intensity of expression (x, y).Then, execution in step S4300.
In step S4300, carry out the 2nd parallax correction.This action is same with the step S3200 of execution mode 3, omits explanation.But, as shown in figure 40, only transmit the 2nd, only use the 2nd field picture, and suitably applied change so that make the red intensity of the 2nd of expression Rf2 (x, y), the Gf2 (x of expression green intensity, y) and the Bf2 of the blue intensity of expression (x, y).Then, execution in step S4400.
In step S4400, make two field picture (integral image).As following formula (75), Rf1 (the x of the red intensity of the 1st frame will be represented, y) R (x of the red intensity of the even number line of conduct expression two field picture, 2*y), as following formula (76), with the expression the 2nd frame red intensity Rf2 (x, y) as the expression two field picture odd-numbered line red intensity R (x, 2*y+1).In addition, as following formula (77), Gf1 (the x of the green intensity of the 1st frame will be represented, y) G (x of the green intensity of the even number line of conduct expression two field picture, 2*y), as following formula (78), the Gf2 (x of the green intensity of the 2nd frame will be represented, y) as the expression two field picture odd-numbered line green intensity G (x, 2*y+1).In addition, as following formula (79), the Bf1 (x of intensity with the blueness of expression the 1st frame, y) B (x of the intensity of the blueness of the even number line of conduct expression two field picture, 2*y), as following formula (80), with the Bf2 (x of intensity of the blueness of expression the 2nd frame, y) as the expression two field picture odd-numbered line blueness intensity B (x, 2*y+1).Here, x is from 0 to L-1 variation, and y changes from 0 to H/2-1.Then, execution in step S4800.
R(x，2*y)＝Rf1(x，y) ……(75)
R(x，2*y+1)＝Rf2(x，y) ……(76)
G(x，2*y)＝Gf1(x，y) ……(77)
G(x，2*y+1)＝Gf2(x，y) ……(78)
B(x，2*y)＝Bf1(x，y) ……(79)
B(x，2*y+1)＝Bf2(x，y) ……(80)
In step S4800, output image.This action is same with execution mode 3, omits explanation.Then, carry out S4900.
In step S4900, tenth skill.
By constituting as described above and move, the video camera module of execution mode 4 has the effect same with execution mode 3.
In addition, the video camera module of execution mode 4 is obtained the 1st parallax in the situation of interlocking and reading in step S4200, carry out the parallax correction according to the parallax of being obtained, and the construction drawing picture (Rf1, Gf1, Bf1).In addition, in step S4300, obtain the 2nd parallax, carry out the parallax correction according to the parallax of being obtained, come the construction drawing picture (Rf2, Gf2, Bf2).And then, in step S4400, synthesize each composograph, become final image.Like this, by the parallax computing is carried out in each field, even when the subject of photography motion, under the different situation of different each images, also can correctly obtain the parallax of each because of the photography time of each.Thus, can utilize this synthesized parallax image, so can access the distinct image that in the general image zone, has further reduced the influence of parallax.
In addition, the quantity of the field of the video camera module of execution mode 4 is 2, but is not limited to this.Also can be more (for example 3,4).
In addition, the video camera module of execution mode 4 is made each image, but also can prepare the zone of two field picture in advance and directly substitution.In addition, also can keep each parallax information etc., fabricating yard image not, and directly make two field picture.
In addition, the video camera module of execution mode 4 also can as among Figure 41 (A) and Figure 41 (B) respectively shown in, the 1st colour filter 124a, the 2nd colour filter 124b, the 3rd colour filter 124c, and the 4th colour filter 124d in, have the branch optical transmission characteristics that mainly makes green transmission along two adjacent colour filters of directions X.In addition, in Figure 41 (A)～Figure 41 (C), in the drawings the color of each colour filter transmission is represented with the mark of R, G, B.Structure according to Figure 41 (A) or Figure 41 (B), in imaging apparatus 123, the composition of the green in the object light is subjected to light along two adjacent imaging apparatuss of directions X (i.e. the 1st imaging apparatus 123a and the 2nd imaging apparatus 123b or the 3rd imaging apparatus 123c and the 4th imaging apparatus 123d).
Perhaps, shown in Figure 41 (C), also can dispose the 1st colour filter 124a, the 2nd colour filter 124b, the 3rd colour filter 124c, reach the 4th colour filter 124d, make along two adjacent colour filters of directions X (being the 2nd colour filter 124b and the 3rd colour filter 124c in this embodiment) to have the branch optical transmission characteristics that mainly makes green transmission.In addition, shown in Figure 41 (C), disposing like that under the situation of 4 colour filters, with the 1st lens section 113a in the lens 113, the 2nd lens section 113b, the 3rd lens section 113c, and the 4th lens section 113d be configured to, make optical axis consistent with the center of these colour filters.Equally, with the 1st imaging apparatus 123a in the imaging apparatus 123, the 2nd imaging apparatus 123b, the 3rd imaging apparatus 123c, and the 4th imaging apparatus 123d also with lens section 113 in the configuration of each lens section dispose with matching.Here, suitably apply change, so that obtain parallax according to the 1st image pickup signal I1 and the 2nd image pickup signal I2.
In the video camera module 401 of present embodiment,, following advantage is arranged by adopting such color filter arrangement shown in Figure 41 (A)～41 (C).Since by the 1st image pickup signal I1 and the 2nd image pickup signal I2 relatively obtain parallax, so the direction that parallax takes place is the x direction, can not cross between each.Therefore, can obtain more accurate parallax.
In addition, the example of the only preferred color filter arrangement of Figure 41 (A)～41 (C) is not defined in this with embodiments of the present invention.For example, B also can be opposite with the position of the colour filter of R.
(execution mode 5)
Embodiments of the present invention 5 are less with parallax, be that the bigger piece of subject distance is identified as background, with other images displacements such as the background image of preserving in advance.Like this,, can from revised image, correctly extract the image of the bigger part of parallax, so can be with these clear picture ground combinations by making up according to image and other images that parallax is revised.
Below, describe with reference to the video camera module of accompanying drawing relevant embodiments of the present invention 5.
Figure 42 is the cutaway view of structure of the video camera module of the relevant embodiments of the present invention 5 of expression.Except the SLSI525 of the circuit part 520 of video camera module 501, same with execution mode 1, for giving identical label and omit explanation to them with the same parts of execution mode 1.
Figure 43 is the block diagram of the video camera module of relevant embodiments of the present invention 5.SLSI525 has systems control division 531, imaging apparatus drive division 132, image pickup signal input part 133, actuator operated amount efferent 134, image processing part 435, input and output portion 136, reaches background image preservation portion 551.In addition, circuit part 520 also has amplifier 126 except above-mentioned structure.
DigitalSignal Processor) or comprise both and constitute image processing part 535 is by logical circuit or DSP (digital signal processor:, utilize the memorizer information of systems control division 531 to carry out various image processing.Image processing part 535 has auto-focus control part 341, piece cutting part 342, parallax operational part 343, related operation portion 344, contrast operational part 345, piece, and cutting part 346, common parallax correction portion 347, low contrast are used parallax correction portion 349 and background image replacement section 552 with parallax correction portion 348, low being correlated with again.
Background image preservation portion 551 is made of RAM or the such rewritable memory of flash memories, stores image information, can suitably rewrite from the outside via input and output portion 136 and systems control division 531.
Figure 44 is the flow chart of action of the video camera module of the relevant embodiments of the present invention 5 of expression.By the systems control division 531 of SLSI525, video camera module 501 moves shown in this flow chart like that.
In step S5000, begin action.For example, detect the situation that shutter release button (not shown) etc. is pressed by last bit CPU (not shown), via the beginning of 136 pairs of video camera module 501 command action of input and output portion, video camera module 501 begins action.Then, execution in step S5100.
In step S5100, auto-focus control part 341 is carried out auto-focus control.This action is same with the step S3100 of execution mode 3, omits explanation.Then, execution in step S5200.
In step S5200, carry out the parallax correction.This action is same with the step S3200 of execution mode 3, omits explanation.Then, execution in step S5300.
In step S5300, the displacement background.As following formula (81), (82), (83), the pixel that parallax Δ i is bigger than threshold value Δ sh is not carried out the change of image.On the other hand, as following formula (84), (85), (86), parallax Δ i be the following pixel of threshold value Δ sh be replaced into the image that is stored in the background image preservation portion 551 (Rback, Gback, Bback).In addition, Δ i is illustrated in the parallax of the piece Bi that obtains among the step S5200.Then, execution in step S5800.
R(x，y)＝R(x，y) (Δi＞Δsh) ……(81)
G(x，y)＝G(x，y) (Δi＞Δsh) ……(82)
B(x，y)＝B(x，y) (Δi＞Δsh) ……(83)
R(x，y)＝Rback(x，y) (Δi≤Δsh) ……(84)
G(x，y)＝Gback(x，y) (Δi≤Δsh) ……(85)
B(x，y)＝Bback(x，y) (Δi≤Δsh) ……(86)
In step S5800, output image.This action is same with execution mode 3, omits explanation.Then, carry out S5900.
In step S5900, tenth skill.
By constituting as described above and move, the video camera module of execution mode 5 has the effect same with execution mode 3.
In addition, embodiments of the present invention 5 are less with parallax in step S5300, be that the bigger piece Bi of subject distance is identified as background, with the background image of preserving in advance (Rback, Gback, Bback) displacement.Like this,, can from revised image, correctly extract the image of the bigger part of parallax, so can be with these clear picture ground combinations by making up according to image and other images that parallax is revised.Figure 45 (A)～Figure 45 (C) is the figure that the background of the video camera module of the relevant embodiments of the present invention 5 of explanation is replaced.According to execution mode 5, for example under the situation that the bust that to the such background of Figure 45 (A) is the mountain is photographed, the parallax Δ i on the mountain of the big and background of the parallax Δ i of bust is less.Here, if the threshold value Δ sh of parallax is set between the parallax on the parallax of bust and mountain, then by image and the background image (Figure 45 (B) that is kept at the sea in the background image preservation portion with the mountain of background, Rback, Gback, Bback) displacement shown in Figure 45 (C), can be made the image of background for the bust in sea.
(execution mode 6)
One execution mode of Figure 46 (A) and the relevant electronic equipment of the present invention of Figure 46 (B) expression.Shown in Figure 46 (A) and Figure 46 (B), portable phone 600 as the band video camera of an execution mode of relevant electronic equipment of the present invention has loud speaker 601, antenna 602, LCD 603, key portion 605, microphone 606, utilizes hinge 604 and can fold.In addition, portable phone 600 is built-in with lens module 110 in the rear side of LCD 603 shown in Figure 46 (B), can carry out the photography of rest image and dynamic image.
In addition, relevant electronic equipment of the present invention is not limited to portable phone, and the PDA that also can be used as vehicle mounted video camera, digital camera and band video camera waits and implements.
More than, several embodiments of the present invention has been described, but they when enforcement of the present invention, can carry out various changes nothing but illustration as follows.
For example, 5 video camera module from execution mode 1 to execution mode, the parallax that the former state utilization calculates, but also can suitably limit.According to lens peculiarity, the subject distance A than certain the value hour, it is not distinct that image becomes.Therefore, if this value is set at the minimum value of subject distance A, then set the maximum of parallax Δ.The parallax bigger than this value can be used as error and ignores.In addition, under these circumstances, also can adopt the little value of parallax evaluation of estimate the 2nd as parallax.
In addition, 5 the video camera module from execution mode 1 to execution mode, according to the 1st image pickup signal I1 (main expression is green) and the 4th image pickup signal I4 (main expression green) computing parallax, but the present invention is not limited to this.For example composition is less contains more blue composition, red composition owing to the subject of purple is green, so sometimes can not be according to the 1st image pickup signal I1 (main expression is green) and the 4th image pickup signal I4 (main expression green) computing.In the case, also can be according to the 2nd image pickup signal I2 (main expression is blue) and the 3rd image pickup signal I3 (main expression is red) computing parallax.In addition, if can not be according to the 1st parallax signals I1 (main expression green) and the 4th image pickup signal I4 (main expression is green) computing parallax and can not be according to the 2nd image pickup signal I2 (main expression is blue) and the 3rd image pickup signal I3 (mainly representing redness) computing parallax, then regard the influence that does not have parallax as, can think no parallax.
In addition, will be from execution mode 1 to execution mode 5 video camera module when being loaded into the video camera, by the 1st～the 4th imaging apparatus 123a～123d is configured to, make the 2nd imaging apparatus 123a when photography configuration become a side of upside, become a side of downside when making the 3rd imaging apparatus 123c be configured in photography, thereby upside becomes blue sensitivity, downside is become red sensitive, so can make the color reproduction of picture with scenes more natural.
In addition, when two outstanding extreme values are arranged, also can adopt a bigger side's parallax in the parallax evaluation of estimate.Owing to contain subject and background in such piece, subject distance and back pitch are from different, so extreme value occurs two.Because less from comparing the subject distance with back pitch, so compare with the parallax of background, the parallax of subject is bigger.Here, if adopt a bigger side's parallax,, can reduce the direct parallax that influences the subject of image quality though can not reduce the influence of the parallax of background.
In addition, from execution mode 1 to execution mode 5, imaging apparatus 123 comprises the 1st imaging apparatus 123a, the 2nd imaging apparatus 123b, the 3rd imaging apparatus 123c, reaches the 4th imaging apparatus 123d, and image pickup signal input part 133 comprises the 1st image pickup signal input part 133a, the 2nd image pickup signal input part 133b, the 3rd image pickup signal input part 133c, reaches the 4th image pickup signal input part 133d.But, also can constitute imaging apparatus 123 with 1 imaging apparatus, on the different position on its sensitive surface, form formed 4 pictures of the 1st～the 4th lens section 113a～113d.In addition, image pickup signal input part 133 also can be made of 1 the image pickup signal input part that has been transfused to from the signal of an imaging apparatus 123.In the case, as long as select appropriate area the data from the memory that is positioned over systems control division 131,231,331,431,531, be used as the 1st image pickup signal I1, the 2nd image pickup signal I2, the 3rd image pickup signal I3, the 4th image pickup signal I4 get final product.
In addition, from execution mode 1 to execution mode 5, as parallax evaluation of estimate Ri (k) and use formula (4), (59) such difference absolute value summation, but be not limited to this.For example also can use: the summation of the square value of difference, perhaps according to the 1st image pickup signal I1 in the piece on average carry out differentiated value and according to the 4th image pickup signal I4 to the summation of on average carrying out the square value of the difference between the differentiated value in the piece, perhaps according to the 1st image pickup signal I1 in the piece on average carry out differentiated value and according to the 4th image pickup signal I4 to the summation of on average carrying out the square value of the difference between the differentiated value in the piece, perhaps will according to the 1st image pickup signal I1 in the piece on average carry out differentiated value and according to the 4th image pickup signal I4 to the summation of on average carrying out the square value of the difference between the differentiated value in the piece, with the square root of the summation of the square value that on average carries out differentiated value in the piece being removed according to the 1st image pickup signal I1, and use value after according to the 4th image pickup signal I4 the square root of the summation of the square value that on average carries out differentiated value in the piece being removed.
In addition, as parallax evaluation of estimate Ri (k), also can use formula (60) or according to the 1st image pickup signal I1 in the piece on average carry out differentiated value and according to the 4th image pickup signal I4 to the summation of on average carrying out the multiplied result between the differentiated value in the piece.But, the parallax evaluation of estimate Ri (k) of the from the 1st to the 5th execution mode relevant when big more (approximate more) more little, so as shown in figure 15, provide minimizing k and provide parallax Δ i, but when use formula (60) etc., relevant big more (approximate more) value is big more, so the k that need carry out providing maximum is as the suitable change of parallax Δ i etc.
In addition, from execution mode 1 to execution mode 5, be configured to, the center with the optical axis of the 1st lens section 113a, the 2nd lens section 113b, the 3rd lens section 113c and the 4th lens section 113d of making links mutually and the rectangle that obtains is square, but is not limited to this.The length of the x direction of this rectangle also can be different with the length of y direction.In the case, when carrying out the parallax correction when for example in step S140 and step S3240, obtaining parallax, in step S151, step S184, step S250 and step S3280 etc., need suitably change.That is, be not to use identical k on x direction and y direction, but keep the length ratio ground change k of the x direction length and the y direction of above-mentioned rectangle.
In addition, from execution mode 1 to execution mode 5, parallax is obtained as integer, but is not limited to this.Also can in step S140, step S240 and step S3240, parallax be obtained decimal, in step S184, step S250 and step S3280, utilize straight-line interpolation to carry out the parallax correction respectively by straight-line interpolation etc.
In addition, 5, also can omit focal point control, in structure, not comprise actuator ground and carry out piece and cut apart and the parallax correction from execution mode 1 to execution mode.Owing under the situation of utilizing the very dark lens of the depth of focus, in the distance of lens and imaging apparatus, allow bigger error, so do not need to make actuator action.
In addition, from execution mode 1 to execution mode 5, the 1st image pickup signal I1 represents that mainly green composition, the 2nd image pickup signal I2 represent that mainly blue composition, the 3rd image pickup signal I3 represent that mainly red composition, the 4th image pickup signal I4 mainly represent green composition, relatively the 1st image pickup signal I1 and the 4th image pickup signal I4 detect parallax and carry out the parallax correction, but are not limited to this.For example, also can change the design of lens and colour filter, change, represent that mainly green composition, the 3rd image pickup signal I3 represent that mainly blue composition, the 4th image pickup signal I4 mainly represent red composition so that the 1st image pickup signal I1 mainly represents green composition, the 2nd image pickup signal I2.In the case, need following change: as following formula (87), change the parallax evaluation coefficient, as following formula (88), (89), (90), carry out parallax correction etc.
Ri(k)＝∑∑|I1(x，y)-I2(x-k，y)| ……(87)
G(x，y)＝[I1(x，y)+I2(x-Δi，y)]/2 ……(88)
B(x，y)＝I3(x，y-Δi) ……(89)
R(x，y)＝I4(x-Δi，y-Δi) ……(90)
In addition, also can change to, make the 1st image pickup signal I1 represent that mainly green composition, the 2nd image pickup signal I2 represent that mainly blue composition, the 3rd image pickup signal I3 represent that mainly green composition, the 4th image pickup signal I4 mainly represent red composition.In the case, need following change: as following formula (91), change the parallax evaluation coefficient, as following formula (92), (93), (94), carry out parallax correction etc.
Ri(k)＝∑∑|I1(x，y)-I3(x，y-k)| ……(91)
G(x，y)＝[I1(x，y)+I3(x，y-Δi)]/2 ……(92)
B(x，y)＝I2(x-Δi，y) ……(93)
R(x，y)＝I4(x-Δi，y-Δi) ……(94)
In addition,, with lattice-shaped 4 camera watch regions are being set, but are being not limited to this 5 from execution mode 1 to execution mode.For example, also can be configured to make the 1st imaging apparatus, the 2nd imaging apparatus, the 3rd imaging apparatus, the 4th imaging apparatus to become on 1 straight line, change so that the 1st image pickup signal I1 mainly represents blue composition, the 2nd image pickup signal I2 and represent that mainly green composition, the 3rd image pickup signal I3 represent that mainly green composition, the 4th image pickup signal I4 mainly represent red composition.In the case, need following change: as following formula (95), change the parallax evaluation coefficient, as following formula (96), (97), (98), carry out parallax correction etc.
Ri(k)＝∑∑|I2(x，y)-I3(x-k，y)| ……(95)
G(x，y)＝[I2(x，y)+I3(x-Δi，y)]/2 ……(96)
B(x，y)＝I1(x+Δi，y) ……(97)
R(x，y)＝I4(x-2*Δi，y) ……(98)
In addition,, be provided with 4 camera watch regions, but be not limited to this 5 from execution mode 1 to execution mode.For example, also can have 3 camera watch regions, be configured to make the 1st imaging apparatus, the 2nd imaging apparatus, the 3rd imaging apparatus to become on 1 straight line, change so that the 1st image pickup signal I1 mainly represents blueness, the 2nd image pickup signal I2 and represent that mainly green, the 3rd image pickup signal I3 mainly represent redness.In the case, need following change: as following formula (99), (100), change the parallax evaluation coefficient, as following formula (101), make green composition G, the parallax Δ i that utilization is obtained by formula (99), as following formula (102), make blue composition B, the parallax Δ i that utilization is obtained by formula (100) makes red composition R as following formula (103), carry out parallax correction etc.In addition, also can use the such form of formula (60) to come replacement formula (99), (100).
Ri(k)＝∑∑|I2(x，y)-I1(x+k，y)| ……(99)
Ri(k)＝∑∑|I2(x，y)-I3(x-k，y)| ……(100)
G(x，y)＝I2(x，y) ……(101)
B(x，y)＝I1(x+Δi，y) ……(102)
R(x，y)＝I3(x-*Δi，y) ……(103)
In addition, also can have 3 camera watch regions, the 2nd imaging apparatus is configured on the right side (the positive position of x axle) of the 1st imaging apparatus, the 3rd imaging apparatus is configured on the downside (the positive position of y axle) of the 1st imaging apparatus, changes so that the 1st image pickup signal I1 mainly represents green, the 2nd image pickup signal I2 and represent that mainly blueness, the 3rd image pickup signal I3 mainly represent redness.In the case, need following change, as following formula (104), (105), change the parallax evaluation coefficient, as following formula (106), make green composition G, the parallax Δ i that utilization is obtained by formula (104) makes blue composition B as following formula (107), utilize the parallax Δ i that is obtained by formula (105), as following formula (108), make red composition R, carry out parallax correction etc.In addition, also can use the such form of formula (60) to come replacement formula (104), (105).
Ri(k)＝∑∑|I1(x，y)-I1(x-k，y)| ……(104)
Ri(k)＝∑∑|I1(x，y)-I3(x，y-k)| ……(105)
G(x，y)＝I1(x，y) ……(106)
B(x，y)＝I2(x-Δi，y) ……(107)
R(x，y)＝I3(x，y-Δi) ……(108)
At execution mode 1, from execution mode 3 to execution mode 5 the video camera module, utilize the parallax Δ i in the piece to carry out the parallax correction, but also can utilize the parallax Δ i of each piece determine each pixel the parallax Δ (x, y), (x y) carries out the parallax correction to utilize the parallax Δ of this each pixel.
Industrial applicibility
Video camera module of the present invention since be can miniaturization, slimming have an auto-focus So the video camera module of function is for the portable phone with camera function, digital still Video camera, vehicle mounted video camera and monitor with video camera etc. to have practicality.

Claims (21)

1, a kind of video camera module has:
A plurality of lens sections comprise 1 lens respectively at least;
A plurality of camera watch regions, corresponding one by one with above-mentioned a plurality of lens sections, have sensitive surface respectively with respect to the optical axis direction approximate vertical of pairing said lens portion;
The image pickup signal input part, a plurality of image pickup signals that input is exported respectively from above-mentioned a plurality of camera watch regions;
The piece cutting part is divided into a plurality of with at least 1 image pickup signal in above-mentioned a plurality of image pickup signals;
The parallax operational part utilizes above-mentioned image pickup signal, the parallax between the picture that the above-mentioned a plurality of lens sections of each above-mentioned computing are formed respectively; And
The parallax correction portion, according to the above-mentioned a plurality of image pickup signals of above-mentioned parallax correction, and composograph.
2, video camera module as claimed in claim 1 also has:
Actuator changes the relative distance of above-mentioned a plurality of lens section and above-mentioned a plurality of camera watch regions; And
Focal point control portion is according to above-mentioned parallax control actuator.
3, video camera module as claimed in claim 2,
Also have the contrast operational part, this contrast operational part is according at least 1 image pickup signal computing contrast in above-mentioned a plurality of image pickup signals;
Above-mentioned focal point control portion controls above-mentioned actuator according to above-mentioned parallax and above-mentioned contrast.
4, video camera module as claimed in claim 3, above-mentioned focal point control portion repeatedly controls above-mentioned actuator, controls above-mentioned actuator according to above-mentioned parallax the 1st time, controls above-mentioned actuator according to above-mentioned contrast after the 2nd time.
5, video camera module as claimed in claim 3, the operational ton the when study of above-mentioned focal point control portion is controlled above-mentioned actuator according to above-mentioned contrast to above-mentioned actuator.
6, video camera module as claimed in claim 5, above-mentioned cutting part detects the profile of a plurality of image-regions at least according to above-mentioned 1 image pickup signal, above-mentioned at least 1 image pickup signal is divided into a plurality of, so that utilize above-mentioned profile to be divided into above-mentioned a plurality of image-region.
7, video camera module as claimed in claim 6,
Additional outlines parallax operational part, this profile parallax operational part is obtained profile parallax as the parallax of above-mentioned profile according to above-mentioned image pickup signal;
Above-mentioned parallax operational part is according to above-mentioned profile parallax, the parallax between the picture that the above-mentioned a plurality of lens sections of each above-mentioned computing are formed respectively.
8, video camera module as claimed in claim 1, above-mentioned cutting part is divided into rectangular-shaped a plurality of with above-mentioned at least 1 image pickup signal.
9, video camera module as claimed in claim 1,
Also have parallax evaluation of estimate operational part, this parallax evaluation of estimate operational part is represented at least 1 parallax precision evaluation value of the precision of above-mentioned parallax to each above-mentioned a plurality of computing according to above-mentioned image pickup signal;
Above-mentioned parallax correction portion is revised above-mentioned a plurality of image pickup signal according to above-mentioned parallax and above-mentioned parallax precision evaluation value to above-mentioned a plurality of each, and composograph.
10, video camera module as claimed in claim 9, above-mentioned parallax correction portion is according to above-mentioned parallax precision evaluation value, whether each above-mentioned a plurality of decision is divided at least 2 with this piece, in being judged as the piece that to cut apart, according to the synthesized parallax image of the piece after cutting apart.
11, video camera module as claimed in claim 9, above-mentioned parallax evaluation of estimate operational part are represented the 1st parallax precision evaluation value of the size of contrast according at least 1 image pickup signal in above-mentioned a plurality of image pickup signals to each above-mentioned a plurality of computing.
12, video camera module as claimed in claim 9, the utilization of above-mentioned parallax evaluation of estimate operational part is divided into above-mentioned a plurality of above-mentioned image pickup signal, to each above-mentioned a plurality of computing the 2nd parallax precision evaluation value, how much relevant the picture that the image drift that the said lens portion that the 2nd parallax precision evaluation value representation general is at least two forms has respectively been moved behind the above-mentioned parallax is.
13, video camera module as claimed in claim 11, above-mentioned parallax correction portion is for above-mentioned a plurality of each, hour this piece is divided at least two in above-mentioned the 2nd parallax precision evaluation value, in the piece after cutting apart according to the synthesized parallax image of the piece after cutting apart.
14, video camera module as claimed in claim 10,
Above-mentioned parallax evaluation of estimate operational part,
According at least 1 image pickup signal in above-mentioned a plurality of image pickup signals, each above-mentioned a plurality of computing is represented the 1st parallax precision evaluation value of the size of contrast, and
Utilization is divided into above-mentioned a plurality of above-mentioned image pickup signal, and to above-mentioned a plurality of each, computing represents that image drift that near few two said lens portion forms respectively moved relevant what the 2nd parallax precision evaluation value of picture behind the above-mentioned parallax;
Above-mentioned parallax correction portion is for above-mentioned a plurality of each, and big and above-mentioned the 2nd parallax precision evaluation value hour in above-mentioned the 1st parallax precision evaluation value is divided into two at least with this piece, in the piece after cutting apart according to the synthesized parallax image of the piece after cutting apart.
15, video camera module as claimed in claim 1,
Above-mentioned image pickup signal input part is imported above-mentioned a plurality of image pickup signal by each of a plurality of;
Above-mentioned parallax operational part is respectively to each parallax of above-mentioned of above-mentioned a plurality of computings.
16, video camera module as claimed in claim 15,
Also have colour filter, this colour filter is corresponding one by one with above-mentioned a plurality of lens sections, disposes the filter of a plurality of colors;
Corresponding in above-mentioned a plurality of lens sections, with at least two lens section of above-mentioned scanning direction configured in parallel, dispose homochromy filter.
17, video camera module as claimed in claim 1,
Also have other image preservation portions that preserve other images different with photographs;
Above-mentioned parallax correction portion will be according to above-mentioned parallax correction image and above-mentioned other images combinations behind the above-mentioned image pickup signal.
18, video camera module as claimed in claim 17, the more little mode of ratio that above-mentioned parallax correction portion is big more according to above-mentioned parallax, revised big and above-mentioned more other images of ratio of the image behind the above-mentioned image pickup signal makes up.
19, a kind of electronic equipment has each described video camera module in the claim 1～18.
20, a kind of program, the action of the above-mentioned image processing part of control in following video camera module, this video camera module has: a plurality of lens sections comprise 1 lens respectively at least; A plurality of camera watch regions, corresponding one by one with above-mentioned a plurality of lens sections, and have sensitive surface respectively with respect to the optical axis direction approximate vertical of pairing said lens portion; The image pickup signal input part, a plurality of image pickup signals that input is exported respectively from above-mentioned a plurality of camera watch regions; And image processing part, carry out the image processing of the image pickup signal imported,
Said procedure is characterised in that, above-mentioned image processing part is carried out:
The piece dividing processing is divided into a plurality of with at least 1 image pickup signal in above-mentioned a plurality of image pickup signals;
The parallax calculation process is utilized above-mentioned image pickup signal, the parallax between the picture that the above-mentioned a plurality of lens sections of each above-mentioned computing are formed respectively; And
The parallax correcting process, according to the above-mentioned a plurality of image pickup signals of above-mentioned parallax correction, and composograph.
21, a kind of program recorded medium records the program of controlling the action of above-mentioned image processing part in following video camera module, and this video camera module has: a plurality of lens sections comprise 1 lens respectively at least; A plurality of camera watch regions, corresponding one by one with above-mentioned a plurality of lens sections, and have sensitive surface respectively with respect to the optical axis direction approximate vertical of pairing said lens portion; The image pickup signal input part, a plurality of image pickup signals that input is exported respectively from above-mentioned a plurality of camera watch regions; And image processing part, carry out the image processing of the image pickup signal imported,
The said procedure recording medium is characterised in that,
Said procedure is carried out above-mentioned image processing part:
The piece dividing processing is divided into a plurality of with at least 1 image pickup signal in above-mentioned a plurality of image pickup signals;
The parallax calculation process is utilized above-mentioned image pickup signal, the parallax between the picture that the above-mentioned a plurality of lens sections of each above-mentioned computing are formed respectively; And
The parallax correcting process, according to the above-mentioned a plurality of image pickup signals of above-mentioned parallax correction, and composograph.
CNB2005800327385A 2004-09-27 2005-09-27 Video camera module and electronic equipment with it Expired - Fee Related CN100546345C (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
JP279794/2004 2004-09-27
JP2004279794 2004-09-27
JP371849/2004 2004-12-22

Publications (2)

Publication Number Publication Date
CN101032161A true CN101032161A (en) 2007-09-05
CN100546345C CN100546345C (en) 2009-09-30

Family

ID=38716340

Family Applications (1)

Application Number Title Priority Date Filing Date
CNB2005800327385A Expired - Fee Related CN100546345C (en) 2004-09-27 2005-09-27 Video camera module and electronic equipment with it

Country Status (1)

Country Link
CN (1) CN100546345C (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101738840B (en) * 2008-11-21 2012-10-03 索尼株式会社 Image pickup apparatus
CN105208259A (en) * 2014-06-17 2015-12-30 中兴通讯股份有限公司 Method for optimizing camera autofocus and camera
CN106791340A (en) * 2017-03-22 2017-05-31 蓝思科技股份有限公司 A kind of CCD camera assembly containing diaphragm
CN112584136A (en) * 2020-12-23 2021-03-30 上海艾为电子技术股份有限公司 Position calibration module, calibration method, electronic device, calibration apparatus, and storage medium

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101738840B (en) * 2008-11-21 2012-10-03 索尼株式会社 Image pickup apparatus
CN105208259A (en) * 2014-06-17 2015-12-30 中兴通讯股份有限公司 Method for optimizing camera autofocus and camera
CN105208259B (en) * 2014-06-17 2019-12-03 中兴通讯股份有限公司 The method and camera of camera auto-focusing optimization
CN106791340A (en) * 2017-03-22 2017-05-31 蓝思科技股份有限公司 A kind of CCD camera assembly containing diaphragm
CN112584136A (en) * 2020-12-23 2021-03-30 上海艾为电子技术股份有限公司 Position calibration module, calibration method, electronic device, calibration apparatus, and storage medium

Also Published As

Publication number Publication date
CN100546345C (en) 2009-09-30

Similar Documents

Publication Publication Date Title
CN1189773C (en) Photographical lens system
CN1219395C (en) Camera apparatus and camera method
CN1306303C (en) Camera, camera system and lens apparatus
CN1305006C (en) Method and system for profviding promated information to image processing apparatus
CN100339735C (en) Zoom lens and imaging device
CN1306305C (en) Zoom lens and electronic still camera using it
CN1162727C (en) Varifocus lens and optical apparatus with the same vari focus lens
CN1834717A (en) Image pickup lens, image pickup apparatus, and mobile terminal provided with image pickup apparatus
CN1949833A (en) Imaging device with a two-dimensional photodetector
CN1265227C (en) Optical imaging system and reader therewith
CN101042466A (en) Zoom lens, imaging device and camera device and mobile information terminal using the zoom lens
CN1893558A (en) Electronic jitter correction device
CN1574898A (en) Image taking apparatus
CN1867940A (en) Imaging apparatus and image processing method therefor
CN1732682A (en) Image processing device and image processing program
CN1576939A (en) Image pick-up lens, image pick-up unit, and mobile terminal provided with this image pick-up unit
CN1900755A (en) Zoom lens and image pickup apparatus
CN1749798A (en) Zoom lens and image pickup apparatus
CN1977205A (en) Zoom lens system, imaging device, and camera
CN1512211A (en) Zoom lens system
CN1451987A (en) Small camera lens, camera unit and portable terminal provided with them
CN1475828A (en) Picture pickup lens, picture pickup device and portable terminal apparatus
CN1576941A (en) Imaging lens and imaging device
CN1774031A (en) Image processing apparatus and image processing method as well as computer program
CN1864087A (en) Image pickup lens and image pickup device

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
C17 Cessation of patent right
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20090930

Termination date: 20120927