CN108605094A - Image processing apparatus, photographic device and image processing method - Google Patents
Image processing apparatus, photographic device and image processing method Download PDFInfo
- Publication number
- CN108605094A CN108605094A CN201780010867.7A CN201780010867A CN108605094A CN 108605094 A CN108605094 A CN 108605094A CN 201780010867 A CN201780010867 A CN 201780010867A CN 108605094 A CN108605094 A CN 108605094A
- Authority
- CN
- China
- Prior art keywords
- image processing
- image
- restoration disposal
- transfer function
- modulation transfer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012545 processing Methods 0.000 title claims abstract description 68
- 238000003672 processing method Methods 0.000 title claims abstract description 28
- 238000003384 imaging method Methods 0.000 claims abstract description 104
- 238000012546 transfer Methods 0.000 claims abstract description 59
- 238000011084 recovery Methods 0.000 claims abstract description 51
- 230000003287 optical effect Effects 0.000 claims description 47
- 230000008859 change Effects 0.000 claims description 10
- 238000005286 illumination Methods 0.000 claims description 6
- 230000006870 function Effects 0.000 description 62
- 230000004075 alteration Effects 0.000 description 34
- 238000003860 storage Methods 0.000 description 19
- 238000000034 method Methods 0.000 description 14
- 238000007689 inspection Methods 0.000 description 13
- 230000006866 deterioration Effects 0.000 description 12
- 230000008569 process Effects 0.000 description 10
- 238000013461 design Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 7
- QSHDDOUJBYECFT-UHFFFAOYSA-N mercury Chemical compound [Hg] QSHDDOUJBYECFT-UHFFFAOYSA-N 0.000 description 5
- 229910052753 mercury Inorganic materials 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 3
- 238000001914 filtration Methods 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 230000033228 biological regulation Effects 0.000 description 2
- 230000000295 complement effect Effects 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 238000009792 diffusion process Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000001228 spectrum Methods 0.000 description 2
- 206010010071 Coma Diseases 0.000 description 1
- 241001269238 Data Species 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 201000009310 astigmatism Diseases 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000008717 functional decline Effects 0.000 description 1
- 230000003760 hair shine Effects 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 229910052751 metal Inorganic materials 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 210000001747 pupil Anatomy 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/40—Picture signal circuits
- H04N1/409—Edge or detail enhancement; Noise or error suppression
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
- H04N23/81—Camera processing pipelines; Components thereof for suppressing or minimising disturbance in the image signal generation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
- H04N25/61—Noise processing, e.g. detecting, correcting, reducing or removing noise the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"
- H04N25/615—Noise processing, e.g. detecting, correcting, reducing or removing noise the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4" involving a transfer function modelling the optical system, e.g. optical transfer function [OTF], phase transfer function [PhTF] or modulation transfer function [MTF]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20004—Adaptive image processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20004—Adaptive image processing
- G06T2207/20008—Globally adaptive
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
- Studio Devices (AREA)
- Facsimile Image Signal Circuits (AREA)
Abstract
The present invention provides a kind of image processing apparatus, photographic device and the image processing method of the photographed images that can obtain desirable image quality.The image processing part (35) to play a role as image processing apparatus has：Point is as restoration disposal portion (51), for by using the image pickup part shooting subject with the lens unit (12) comprising lens (16) and image-forming component (26) and from the image data that image-forming component (26) obtains, put as restoration disposal using the recovery filter of the point spread function based on lens unit (12)；And judegment part (53), when the modulation transfer function put as the modulation transfer function under defined spatial frequency that restoration disposal is contributed and changed according to imaging conditions is less than threshold value, it is determined as by putting as restoration disposal portion (51) are put as restoration disposal, point is only put when judegment part (53) is determined as being put as restoration disposal as restoration disposal as restoration disposal portion (51).
Description
Technical field
The present invention relates to a kind of image processing apparatus, photographic device and image processing methods more particularly to one kind to be based on
The point spread function of imaging optical system carries out the image shot by imaging optical system the technology of restoration disposal.
Background technology
In the shot object image shot by imaging optical system, aberration and diffraction caused by imaging optical system etc.
Influence, sometimes find point subject have small extension point diffusion phenomena.Indicate imaging optical system to point light source
The function of response be referred to as point spread function (PSF：Point Spread Function), it is known as left and right camera shooting figure
The characteristic of the resolution deterioration (fuzzy) of picture.
For the photographed images of image quality deterioration due to diffusion phenomena, based on PSF points as restoration disposal, thus, it is possible to
The image quality of enough photographed images for restoring (recovery) deterioration.The point is following processing as restoration disposal, that is, finds out camera optical in advance
Degradation characteristic (point as characteristic) caused by aberration of system (lens and aperture) etc., and by using with the point as characteristic pair
Image procossing that the recovery filter answered carries out and eliminate or reduce the processing that the point of photographed images is spread.
In the past, as this point is carried out as the photographic device of restoration disposal, there is the photographic device described in patent document 1.
Following interesting purport has been recorded in patent document 1：About the photographed images obtained by photographic device, even if accurately
In the case of correcting each aberrations such as spherical aberration, coma aberration, curvature of the image, the astigmatism of imaging optical system, also due to
Deteriorate depending on the diffraction phenomena of f-number (F numbers), although also, aberration can pass through and improve imaging optical system
Imaging performance and improved, but due to unavoidable physical phenomenon, that is, diffraction, to the bad of the image caused by diffraction
It is important topic that change, which is corrected,.
Also, following interesting purport is recorded in patent document 1：In the F numbers of F16 or more, aberration is reduced by aperture, and spread out
The influence penetrated becomes leading.
Therefore, it in the photographic device described in patent document 1, is generated when being only specified value or more (small aperture) by f-number
Diffraction (so-called small aperture fuzzy) as point as the object of restoration disposal, f-number when shoot photographed images is provides
When being worth above, the point of photographed images is carried out as restoration disposal using a recovery filter corresponding with the f-number.
Also, small aperture is fuzzy to be not dependent on image height (position of image) depending on the wavelength of f-number and light, therefore
Can be fuzzy to reducing small aperture using a recovery filter in an image, thus, it is possible to reduce data volume and operation
Amount.
Conventional art document
Patent document
Patent document 1：Japanese Unexamined Patent Publication 2014-150423 bulletins
Invention content
The invention technical task to be solved
In photographic device described in patent document 1, since small aperture is only obscured the object as point as restoration disposal,
Therefore the spherical aberration that can not restore imaging optical system, especially by big spherical surface possessed by the light near pupil outer rim
The deterioration of image quality caused by aberration.Therefore, the imaging optical system being applicable in the photographic device described in patent document 1 needs
It being asked using the few imaging optical system of deterioration of the image quality caused by spherical aberration, and there are imaging optical system is expensive
Topic.
The present invention is to complete in light of this situation, and desirable image quality can be obtained its purpose is to provide a kind of
Image processing apparatus, photographic device and the image processing method of photographed images.
Means for solving technical task
To achieve the goals above, the image processing apparatus involved by a kind of mode of the invention has：It puts at as recovery
Reason portion is obtained for by using the image pickup part shooting subject with imaging optical system and image-forming component from image-forming component
Image data, put as restoration disposal using the recovery filter of the point spread function based on imaging optical system；And
Judegment part, when putting as the modulation transfer function under defined spatial frequency that restoration disposal is contributed and changed according to imaging conditions
Modulation transfer function when being less than threshold value, be determined as by putting as restoration disposal portion put as restoration disposal, point is as restoration disposal
It is only put as restoration disposal when judegment part is determined as being put as restoration disposal in portion.
Point becomes as the modulation transfer function under defined spatial frequency that restoration disposal is contributed according to imaging conditions
It is dynamic, and when the modulation transfer function changed according to imaging conditions is less than threshold value, judegment part is determined as being put as restoration disposal.
Moreover, point when judegment part is determined as being put as restoration disposal as only being put as restoration disposal, therefore energy in restoration disposal portion
Enough photographed images by deterioration restore the photographed images of (recovery) at desirable image quality.Also, point only exists as restoration disposal portion
Put when judegment part is determined as being put as restoration disposal as restoration disposal (that is, be set as when modulation transfer function be threshold value with
Can obtain the photographed images of desirable image quality when upper and without putting as restoration disposal), therefore can reduce as at recovery
Operation cost of the data volume of the recovery filter used in reason etc. with point as restoration disposal.
In image processing apparatus involved by the another way of the present invention preferably, when will be based on the sampling of image-forming component
When frequency is set as fs, it is specified that spatial frequency be spatial frequency in the range of 0.25fs to 0.3fs.This is because when passing through
When modulation transfer function under spatial frequency of the point as restoration disposal and in the range of 0.25fs to 0.3fs is got higher, with other skies
Between modulation transfer function in frequency field the case where getting higher compare, the image quality of photographed images improves.
In image processing apparatus involved by the another mode of the present invention preferably, the tune when being 0 by spatial frequency
When modulation trnasfer function is set as 100%, threshold value is the value in the range of 10% to 50%.
In image processing apparatus involved by the another mode of the present invention, imaging conditions are to constitute imaging optical system
The f-number of aperture.The spherical aberration of imaging optical system changes according to the f-number of aperture, as a result, letter is transmitted in modulation
Number changes also according to f-number.Also, when f-number becomes specified value or more (small aperture), generate small aperture it is fuzzy and
Modulation transfer function declines.Therefore, the f-number of aperture is one of the imaging conditions for changing modulation transfer function.
In image processing apparatus involved by the another mode of the present invention preferably, judegment part is according to the f-number of aperture
To differentiate whether the modulation transfer function changed according to imaging conditions is less than threshold value.Judegment part is according to change modulation transfer function
The f-number of the aperture of one of parameter, differentiates whether modulation transfer function is less than threshold value indirectly.
In image processing apparatus involved by the another mode of the present invention preferably, judegment part is according to based on recovery filtering
Filter gain under the defined spatial frequency of device differentiates whether the modulation transfer function changed according to imaging conditions is less than
Threshold value.In general, modulation transfer function more declines, become bigger based on the filter gain for restoring filter.Therefore, judegment part root
According to based on the filter gain for restoring filter, differentiate whether modulation transfer function is less than threshold value indirectly.
In image processing apparatus involved by the another mode of the present invention, imaging conditions illuminate subject
The type of light source or the wavelength of light source.The spherical aberration of imaging optical system changes according to wavelength, as a result, modulation is transmitted
Function changes also according to wavelength, therefore type of light source illuminated to subject etc. is to change modulation transfer function
One of imaging conditions.
In image processing apparatus involved by the another mode of the present invention preferably, as what is changed according to imaging conditions
Modulation transfer function, judegment part use modulation biography corresponding with the type of light source or the wavelength of light source that are illuminated to subject
Delivery function.This is because the wavelength (wavelength for being appointed as the light source of the type of light source) of light source is to change modulation transfer function
One of parameter.
The present invention another mode involved by image processing apparatus in preferably, judegment part according to image-forming component at
Differentiate whether the modulation transfer function changed according to imaging conditions is low as the modulation transfer function in the predetermined region on picture
In threshold value.In general, modulation transfer function is on imaging picture and uneven, therefore preferably using the predetermined region on imaging picture
In modulation transfer function.
Preferably, it is specified that letter is transmitted in the modulation in region in image processing apparatus involved by the another mode of the present invention
Number is the modulation transfer function in the region that the image height being imaged on picture is 50% or more.In general, in modulation transfer function, with image height
Region (central area) less than 50% is compared, and the region (neighboring area) that image height is 50% or more reduces, therefore preferably uses
Reduce the modulation transfer function in region.This is because (at least being modulated as restoration disposal to restore the photographed images of deterioration by putting
Transmission function reduce region image) image quality.
Preferably, it is specified that letter is transmitted in the modulation in region in image processing apparatus involved by the another mode of the present invention
Number is the modulation transfer function of the picture entirety of imaging picture.
Preferably, it is specified that letter is transmitted in the modulation in region in image processing apparatus involved by the another mode of the present invention
Number is the modulation transfer function of any number of points of imaging picture.Letter is transmitted in the modulation of any number of points as imaging picture
Number, such as can be set as the picture entirety for being imaged picture being divided into the modulation of the center of the segmentation area of multiple regions
Transmission function.
In image processing apparatus involved by the another mode of the present invention preferably, when the light for constituting imaging optical system
When circle is at least open aperture, the modulation transfer function of imaging optical system is less than threshold value.As imaging optical system, by adopting
Modulation transfer function when with aperture being open aperture is less than the imaging optical system of threshold value, even if with to be open light in aperture
Modulation transfer function has carried out the imaging optical system of optical design also above the mode of threshold value and has compared in the case of circle, Neng Goushe
For the high and cheap imaging optical system of the degree of freedom of optical design.
Photographic device involved by the another mode of the present invention has the image processing apparatus described in any of the above-described mode
And image pickup part.
Photographic device involved by the another mode of the present invention is preferably used as industrial camera.
Image processing method involved by the another mode of the present invention includes the following steps：For by using with camera shooting
Optical system and the image pickup part shooting subject of image-forming component and from image-forming component obtain image data, using based on shooting light
The recovery filter of the point spread function of system is put as the step of restoration disposal；And when point is as restoring contributed rule
Modulation transfer function under fixed spatial frequency and when being less than threshold value according to the modulation transfer function that imaging conditions change, is determined as
Put as the step of restoration disposal, in being put as the step of restoration disposal, be only determined as in the step of being differentiated into
It is put as restoration disposal using recovery filter when row point is as restoration disposal.
In image processing method involved by the another mode of the present invention preferably, when will be based on the sampling of image-forming component
When frequency is set as fs, it is specified that spatial frequency be spatial frequency in the range of 0.25fs to 0.3fs.
In image processing method involved by the another mode of the present invention preferably, the tune when being 0 by spatial frequency
When modulation trnasfer function is set as 100%, threshold value is the value in the range of 10% to 50%.
In image processing method involved by the another mode of the present invention preferably, imaging conditions are to constitute camera optical
The f-number of the aperture of system.
In image processing method involved by the another mode of the present invention preferably, in the step of being differentiated, according to
The f-number of aperture differentiates whether the modulation transfer function changed according to imaging conditions is less than threshold value.
In image processing method involved by the another mode of the present invention preferably, in the step of being differentiated, according to
It is transmitted according to the modulation that imaging conditions change based on the filter gain under the defined spatial frequency for restoring filter to differentiate
Whether function is less than threshold value.
Invention effect
According to the present invention, only carried out when the modulation transfer function changed according to imaging conditions is less than threshold value at picture recovery
Reason, therefore the photographed images of deterioration can be restored to the photographed images of (recovery) at desirable image quality, also, when modulation is transmitted
Be set as having obtained the photographed images of desirable image quality when function is more than threshold value and without putting as restoration disposal, therefore can
Point is reduced as the data volume of recovery filter etc. that is used in restoration disposal and point are as the operation cost of restoration disposal.
Description of the drawings
Fig. 1 is the block diagram of the functional structure example for the photographic device for indicating to connect with computer
Fig. 2 is the block diagram for the configuration example for indicating camera controller shown in Fig. 1
Fig. 3 is the block diagram for the 1st embodiment for indicating the image processing part in camera controller shown in Figure 2
Fig. 4 is the chart for indicating to be directed to an example that each of spherical aberration and F values combine calculated MTF
Fig. 5 is indicated with point as the corresponding filter gains of the ratio between MTF before the MTF and restoration disposal after restoration disposal
Chart
Fig. 6 is chart corresponding with Fig. 4, and indicates chart of the point as the MTF after restoration disposal
Fig. 7 is to indicate to show to be directed to another chart that each of spherical aberration and F values combine calculated MTF
Fig. 8 is the block diagram for the 2nd embodiment for indicating the image processing part in camera controller shown in Figure 2
Fig. 9 is the flow chart for the 1st embodiment for indicating image processing method
Figure 10 is the flow chart for the 2nd embodiment for indicating image processing method
Figure 11 is the flow chart for the 3rd embodiment for indicating image processing method
Specific implementation mode
Hereinafter, with reference to the accompanying drawings to image processing apparatus according to the present invention, photographic device and image processing method
Embodiment illustrates.In following implementation, as an example, to being applicable in this in the photographic device for NI Vision Builder for Automated Inspection
The case where invention, illustrates.
Fig. 1 is to indicate and computer (PC：Personal Computer (personal computer)) connection photographic device 10
The block diagram of functional structure example.
For industrial camera, it is desirable that widen the depth of field, while reducing the f-number (F values) of aperture as far as possible, thus obtain at a high speed
The focused image of object range is taken, and uses the chance that will produce such as the fuzzy F values of that small aperture of common camera lens
Seldom.
Photographic device 10 shown in FIG. 1 is the industrial camera used in NI Vision Builder for Automated Inspection, mainly by constituting image pickup part
Lens unit 12 and image-forming component 26, camera controller 28, input/output interface 32 and lighting device 40 are constituted.
Lens unit 12 has the imaging optical system being made of lens 16 and aperture 18 and controls the camera optical system
The optical system operational portion 20 of system.Optical system operational portion 20 include adjustment lens 16 focusing position manual operation unit and
The aperture driving portion of aperture 18 is driven by the control signal added from camera controller 28.
Lens for the photographic device used in NI Vision Builder for Automated Inspection, it is important that the lens performances such as resolution, brightness
It is uniform in face.As the design method for realizing this performance, the method for the f-number (F values) of limited f-stop, and
As lens prescriptions, it is also necessary to the brightness of lens.
On the other hand, for the photographic device used in NI Vision Builder for Automated Inspection, consider from the necessity of the depth of field, for
The range of the aperture used in practicality (is more than the F values of open aperture and less than the small aperture of generation usually using the aperture of midband
The aperture of the F value ranges of the F values of fuzzy small aperture).
Therefore, the lens 16 of the photographic device 10 are (to include the F of the aperture used in practicality under defined imaging conditions
Value) emphasis in a manner of the photographed images of the desirable image quality required in the inspection, measurement or positioning that can shoot product etc.
Change and has carried out the lens of lens design.For the photographed images shot for using the lens 16, taken the photograph in addition to defined
Image quality deteriorates and cannot meet desirable image quality (photographed images for becoming deterioration) under imaging conditions other than slice part, but bad
The photographed images of change can restore (recovery) as restoration disposal by aftermentioned point.Also, for the focal length of the lens of this example 16
(f) for, f=16mm.
Aperture 18 is made of multiple aperture blades, and by aperture from F1.4 (open F value) to F22 with 1AV (Aperture
Value (f-number)) increment carry out 10 grades control.
By CMOS, ((complementary metal aoxidizes Complementary Metal-Oxide Semiconductor image-forming component 26
Object semiconductor)) image sensor of type constitutes.In addition, image-forming component 26 is not limited to CMOS type, can also be XY address types or
The image sensor of CCD (Charge Coupled Device (charge coupling device)) type.Also, the image-forming component 26 of this example
Sample frequency (fs) be 90/mm.
Lighting device 40 is the device illuminated to the subject of the imaging area based on photographic device 10, such as is had
The multiple light sources such as the rgb light source of mercury lamp, infrared light sources and red (R), green (G), blue (B), by the kind for selecting illuminating source
Class, to be illuminated to subject by the illumination light of desirable wavelength.In this example, lighting device 40 can be selected suitably
Following light shines, that is, by e ray spectrums (wavelength X=546.070nm) as peak wavelength by opening mercury lamp
Light or infrared light by opening infrared light sources by twice of the wavelength (1092.14nm) of e rays as peak wavelength.
Also, by selecting a light source from rgb light source or with appropriately combined opening light source, can also send out with desirable peak value
The light of wavelength.
As shown in Fig. 2, camera controller 28 has：The conduct device control unit 34 in each portion of Comprehensive Control photographic device 10
Function；And the image data to being sent from image-forming component 26 carries out the conduct image processing part of image procossing (from image
Manage device) 35 function, detailed content is illustrated later.
The image data that image procossing has been carried out by camera controller 28 is sent to calculating by input/output interface 32
In machine 60 etc..The format of the image data exported from camera controller 28 is not particularly limited, when being dynamic image, Ke Yishe
For MPEG (Moving Picture Experts Group (Motion Picture Experts Group)), the formats such as H.264, be still image
When, JPEG (Joint Photographic Experts Group (joint photographic experts group)), TIFF (Tagged can be set as
Image File Format (tagged image file format)) etc. formats.And it is possible to export without being based on image processing part
The initial data (RAW data) of 35 image procossing.Moreover, such as so-called Exif (Exchangeable Image File
Format (exchangeable image file format)) like that, camera controller 28 can also be by heading message (camera shooting date/time, type
Number, pixel number, F values, the type of light source etc.), multiple related datas such as main image data and thumbnail image data establish each other
Correspondence and be configured to an image file, and export the image file.
Computer 60 is the part for constituting the part that various products are carried out with the NI Vision Builder for Automated Inspection checked etc., is passed through
The input/output interface 32 and computer input and output portion 62 of photographic device 10 are connect with photographic device 10, and are received and filled from camera shooting
Set the data class such as the image data that 10 send.64 Comprehensive control computer 60 of computer control, to coming from photographic device 10
Image data carry out image procossing, and captured product is checked etc., also, to itself and the work by network connection
Communication between the various devices such as industry robot (not shown) is controlled.
Computer 60 has display 66, process content and inspection result in computer control 64 etc. as needed and
It is shown in display 66.User such as operations keyboard on one side confirm the display of display 66 while at the input units (illustration omitted),
So as to which data and order are input in computer control 64.User can be to computer 60 and and computer as a result,
The photographic device 10 of 60 connections is controlled.
Each controller (camera controller 28, computer control 64) has the circuit class needed for control process, such as has
Standby central processing unit (CPU (Central Processing Unit) etc.) and memory etc..
[image processing apparatus]
1st embodiment ＞ of ＜ image processing parts
Fig. 3 is the block diagram for the 1st embodiment for indicating the image processing part 35 in camera controller 28 shown in Figure 2.
Image processing part 35 shown in Fig. 3 is mainly by putting as (switching is opened for restoration disposal portion 51, judegment part 53 and switching part
Close) 54 compositions.In addition, image processing part 35 other than putting as restoration disposal portion 51, is also equipped with migration processing, at gray correction
The various processing units such as reason and emphasizing contour processing, are omitted in Fig. 3.
It reads and takes turns from image-forming component 26 (Fig. 1) and change the image data of digital signal into and be added to the of switching switch 54
1 input 54A and point are as restoration disposal portion 51 (restoring filter process portion 51A).
Point is as restoration disposal portion 51 is mainly by recovery filter process portion 51A, recovery filter storage part 51B and recovery
Filter selector 51C is constituted.
Restore the image data before being added to restoration disposal in the input of the side of filter process portion 51A and due to lens
16 spherical aberration etc. and image quality deterioration photographed images image data, be added in the input of the other side from be stored in restore
The recovery filter that multiple recovery filters of filter storage part 51B are suitably selected by restoring filter selector 51C.
It restores filter process portion 51A and is directed to inputted image data, added using from recovery filter selector 51C
Recovery filter put as restoration disposal, and calculate the image data put as restoration disposal.That is, restoring filter
Processing unit 51A, which carries out the regulation core size in inputted image data centered on dealing with objects pixel, (to be and restores filtering
The identical core size of device, such as 7 × 7 etc.) image data and the convolution algorithm that restores filter, and calculate and carried out as multiple
The image data of original place reason.
It has been carried out a little by restoring filter process portion 51A as the image data of restoration disposal is added to switching switch 54
In 2nd input 54B.
Judegment part 53 is to discriminate whether to need to carry out by putting as restoration disposal portion 51 carries out the image data inputted
Point is as the part of restoration disposal, if being determined as being put as restoration disposal, output will switch the movable contact pin of switch 54
54C switches to the switching signal (for example, high level signal) of the 2nd input 54B, and is to discriminate whether to need to carry out by picture again
The point that former processing unit 51 carries out as restoration disposal part, if being determined as to be put as restoration disposal, export by
The movable contact pin 54C of switching switch 54 switches to the switching signal (for example, low level signal) of the 1st input 54A.In addition, for
The detailed content of judegment part 53 is illustrated later.
[restoring filter]
Then, the recovery filter for being stored in recovery filter storage part 51B is illustrated.
In general, in the point spread function based on expression imaging optical system (lens 16, aperture 18) to the response of point light source
(PSF) Convolution-type wiener (Wiener) filter can be utilized in the recovery of the blurred picture carried out.With reference to by PSF (x, y) into
Optical transfer function (the OTF of Fourier transform is gone：Optical Transfer Function) and signal-to-noise ratio (SNR：
Signal to Noise Ratio) information, can be calculate by the following formula restore filter frequency characteristic d (ωx,ωy)。
[numerical expression 1]
Wherein, H (ωx,ωy) indicate OTF, H*(ωx,ωy) indicate its complex conjugate.Also, SNR (ωx,ωy) indicate letter
It makes an uproar ratio.
The design for restoring the filter coefficient of filter is to select coefficient value so that the frequency characteristic of filter is closest to institute
The optimization problem of desired wiener frequency characteristic can suitably calculate filter coefficient by any known method.
In addition, PSF becomes according to imaging conditions such as wavelength (peak wavelength), image height and the focal lengths of F values, illumination light
It is dynamic, therefore restore filter needs and calculated using the PSF changed according to imaging conditions.Also, it can also use and indicate
Modulation transfer function (the MTF of the amplitude component of OTF：Modulation Transfer Function) it is filtered to calculate to restore
Device, to replace OTF.
It restores in filter storage part 51B and imaging conditions is stored in association according to corresponding with multiple imaging conditions
The calculated multiple recovery filters of PSF.Be stored in the recovery filter storage part 51B of this example with the F values of aperture 18 and
The corresponding recovery filter of wavelength (peak wavelength) of the type or light source of the light source of lighting device 40, but it is not limited to this,
It can further generate and store recovery filter corresponding with image height.
It restores in filter selector 51C from camera controller 28 (Fig. 1) added with the current aperture for indicating aperture 18
The light source used when indicating shooting in the information (F value informations) of value (F values) and the multiple light sources used by lighting device 40
The information (light source information) of type restores filter selector 51C according to the F value informations and light source inputted from camera controller 28
Information reads recovery filter corresponding with these information from filter storage part 51B is restored, and the recovery read is filtered
Device is exported to recovery filter process portion 51A.In addition, light source information is not limited to the type for the light source for indicating to use when shooting
Information can also be the information of the wavelength (peak wavelength) for the light source for indicating to use when shooting.
It is added with F value informations and light source information from camera controller 28 in judegment part 53, judegment part 53 discriminates whether to need
To being carried out according to the image data of F value informations and light source information input by putting the point carried out as restoration disposal portion 51 as recovery
Reason, and as described above export switching signal corresponding with result is differentiated to switching switch 54.
Then, to whether needing to put as the specific method of discrimination of restoration disposal is said captured image data
It is bright.
The resolution deterioration (fuzzy) of photographed images is driven by the point spread function (PSF) of lens, also, PSF is main
Spherical aberration depending on lens.
The multinomial of wavefront aberrations as approximate imaging optical system is utilized and uses polar Ze Nike
(Zernike) multinomial, the 9th expression spherical aberration of zernike polynomial (Fritz zernike polynomial).
Now, for using lens that focal length f is 16mm, will be with e ray spectrums (wavelength X=546.070nm) for peak value
The mercury lamp of wavelength is as lighting source and the different various lens of spherical aberration, for each by Fritz zernike polynomial
The spherical aberration and F values of 9th expression, calculate MTF.
Wherein, the sample frequency (fs) based on image-forming component 26 is 90/mm, in the range of MTF is 0.25fs to 0.3fs
Defined spatial frequency (in this example, 0.25fs) under value.In addition, by the spatial frequency in the range of 0.25fs to 0.3fs
Under MTF be as the reasons why evaluation object, the MTF under spatial frequency within the scope of this be suitble to image quality is evaluated, and
And put as restoration disposal contributes to the spatial frequency area of image quality.
Fig. 4 is the chart indicated for spherical aberration and an example of the calculated MTF of each combination of F values.
Shown in Fig. 4 spherical aberration be 0 λ, 0.2 λ ..., 10 kinds of lens of 5 λ it is every with 10 grades of F values of F1.4~F22
The MTF of a combination.
Here, MTF shown in Fig. 4 is the value under the defined spatial frequency (0.25fs) on optical axis, and it is into pixel
The MTF in predetermined region on the imaging picture of part 26 is but it is also possible to be region in addition to this.
The image height that the MTF of predetermined region can be set as on imaging picture is 50% (by the center for being imaged picture to being imaged picture
The distance of the quadrangle in face is set as 50% when 100%) more than region MTF.This is because usually image height it is higher when MTF more
It is low, and preferably using the MTF for reducing region.
Moreover, other examples of the MTF as predetermined region, may be used the MTF of the picture entirety of imaging picture, and
The MTF of any number of points of imaging picture may be used.
In addition, the MTF as predetermined region, using the MTF of predetermined region typical value (for example, average value, median or
Mode etc.).
As shown in figure 4, F1.4 is open F value, as F values increase (aperture openings reduction) from open F value, MTF increases, if
F values become F5.6 or more, then MTF is gradually reduced.It is because small aperture is fuzzy that MTF, which is gradually reduced, if F values become F5.6 or more
It occupies an leading position than spherical aberration.In addition, in F22, it can not correctly be calculated since MTF is too low, MTF be set as
0。
Consider from the necessity of the depth of field, the photographic device 10 used in NI Vision Builder for Automated Inspection is by the aperture used in practicality
Aperture of the range as midband, for lens 16, even if in the aperture of midband without point as restoration disposal,
Also the mode emphasis of the photographed images of the desirable image quality required in inspection of product etc. can be shot and carried out lens
Design.
On the other hand, as lens prescriptions, it is also necessary to the brightness of lens, therefore for example, even if open aperture, it is also desirable to energy
The photographed images of the desirable image quality required in inspection of product etc. are enough obtained, at this point, restoring as restoration disposal by putting
The photographed images of deterioration.
In addition, the lens 16 of this example have carried out lens design in such a way that spherical aberration becomes 2 λ, but not limited to this.
Also, the bigger lens of spherical aberration, lens design is easier, and lens are also cheaper.
Regulation on the imaging picture of the MTF and image-forming component 26 changed according to the imaging conditions wavelength of light source (F values)
When MTF in region is less than threshold value, judegment part 53 is determined as being put as restoration disposal.
In this example, when the MTF when being 0 by spatial frequency is set as 100%, threshold value is set as 50%, but threshold value and unlimited
In the value, such as it can also use the value in the range of 10% to 50%.
As shown in figure 4, when the lens 16 and the illumination light that is sent out from lighting device 40 that be 2 λ of spherical aberration are by e ray light
When composing light as peak wavelength, MTF is less than when due to the F values of aperture 18 being F1.4 (open F value), F1.8, F11 and F16
50%, therefore judegment part 53 is determined as being put as restoration disposal.
That is, instead of comparing MTF and threshold value, what judegment part 53 can will be changed according to imaging conditions the wavelength of light source (F values)
Whether MTF less than threshold value 50% is substituted for F values to be differentiated, in this example, when F values are F2.0~F8 (aperture is midband),
It is determined as without putting as restoration disposal, F values are determined as being put as restoration disposal in range in addition to this.In addition, this
In example, F22 is set as the F values not used in inspection of product etc..
Also, Fig. 5 is to indicate multiple as shown in the MTF and Fig. 4 after restoration disposal (after correction) with point shown in Fig. 6
The chart of the corresponding filter gains of the ratio between MTF before the reason of original place.
It is found that MTF is more reduced from the comparison of Fig. 4 and Fig. 5, in order to approach target MTF and filter gain take it is bigger
Value.
Therefore, instead of comparing MTF and threshold value, whether the MTF changed according to imaging conditions can be less than threshold by judegment part 53
Value 50% is substituted for filter gain to be differentiated.That is, as shown in figure 5, when filter gain be 2 or more and 5 or less when, by
It is less than threshold value 50% in MTF, therefore judegment part 53 is determined as being put as restoration disposal.
In addition, in Fig. 6, filter gain for convenience of description, to all lens and F values show a little as restoration disposal after
MTF, but actually put as restoration disposal portion 51 only is being determined as that the imaging conditions as restoration disposal should be put by judegment part 53
Shi Jinhang points are as restoration disposal, therefore recovery filter storage part 51B only should put the camera shooting item as restoration disposal
Recovery filter corresponding with the imaging conditions is stored under part, thus, it is possible to reduce the data for the recovery filter that should be stored
Amount.Also, point as restoration disposal portion 51 only when being determined as to be put the imaging conditions as restoration disposal by judegment part 53 into
Row point as restoration disposal, therefore with no matter how imaging conditions are put as restoration disposal all images when compared with,
Operation cost can be reduced.
Fig. 7 is to indicate to show to be directed to another chart that each of spherical aberration and F values combine calculated MTF.Separately
Outside, MTF shown in Fig. 7 is only different from an example shown in Fig. 4 in the type of light source, as the light source of illumination light, shows
In relation to using feelings of twice of the wavelength (1092.14nm) by e rays shown in Fig. 4 as the infrared light sources of peak wavelength
Condition.
If the wavelength of light source is different, the refractive index in lens is different, as a result, MTF is also different.Shown in Fig. 4
The comparison of MTF shown in MTF and Fig. 7 is it is found that as light source when shooting, if using twice of the wavelength with e rays
Infrared light, then with use by using mercury lamp as the e rays of light source as the light of peak wavelength situation compared with, MTF reduce.
In the case of lens 16 of this example for having carried out lens design in a manner of becoming 2 λ by spherical aberration, if using will
Light source of the e rays as peak wavelength, and F values are reduced to F2, then MTF becomes 50% or more threshold value, and if being had using sending out
There are the infrared light sources of the infrared light of twice of wavelength of e rays, even if F values are then reduced to F2, MTF will not become threshold value
50% or more, it needs to be reduced to F2.8 or more.
Therefore, for judegment part 53, as the MTF changed according to imaging conditions, the aperture used when using according to shooting
The specified MTF of the type (or wavelength of light source) of the light source of 18 F values and the illumination light sent out from lighting device 40, and according to this
Whether MTF discriminates whether to be put as restoration disposal less than threshold value 50%.Alternatively, it is also possible to pass through F values or filter gain
Differentiated, to replace being judged by MTF.
2nd embodiment ＞ of ＜ image processing parts
Fig. 8 is the block diagram for the 2nd embodiment for indicating the image processing part 35 in camera controller 28 shown in Figure 2.
In addition, a pair part identical with the 1st embodiment shown in Fig. 3 marks the same symbol, and description is omitted.
The image processing part 35 of 2nd embodiment shown in Fig. 8 mainly by put as restoration disposal portion 52, judegment part 53 and
Switch switch 54 to constitute.
Point replaces a little as restoration disposal portion 52 has point spread function storage part 52B and restores filter generating unit 52C
As the recovery filter storage part 51B and recovery filter selector 51C in restoration disposal portion 51, implement in this regard with the 1st
The point of the image processing part 35 of mode is different as restoration disposal portion 51.
Point is corresponding multiple multiple as imaging conditions such as recovery filter storage part 51B storage and the F values in restoration disposal portion 51
Former filter, in contrast, point spread function storage part 52B storages are for generating multiple sources for restoring filter, expression
Point spread function (PSF) of the imaging optical system (lens 16, aperture 18) to the response of point light source.It is deposited in addition, restoring filter
Storage portion 51B stores PSF corresponding with the imaging conditions only in the case where that should be put the imaging conditions as restoration disposal, thus, it is possible to
Enough reduce the data volume for the PSF that should be stored.
It restores in filter generating unit 52C from camera controller 28 (Fig. 1) added with the current F values for indicating aperture 18
The light source information of the type of the light source used when indicating shooting in F value informations and the multiple light sources used by lighting device 40,
Filter generating unit 52C is restored according to the F value informations and light source information inputted from camera controller 28, is stored from point spread function
Portion 52B reads PSF corresponding with these information, and is generated according to the PSF read and restore filter.
That is, restoring filter generating unit 52C according to the OTF that the PSF read has been carried out to Fourier transform and setting in advance
Fixed SNR finds out the frequency characteristic d (ω x, ω y) for for example restoring filter as shown in [numerical expression 1] formula, and is carried out
Inverse fourier transform and generate Convolution-type Wiener filter (restore concerning filter).
The recovery filter produced in this way, which is added to, restores filter process portion 52A, carries out image data herein and answers
The convolution algorithm of former filter.
In addition, point spread function storage part 52B is stored with PSF, but it can also store and PSF has been subjected to Fourier transform
OTF indicates that the MTF of the amplitude component of OTF replaces PSF.
[image processing method]
1st embodiment ＞ of ＜ image processing methods
Then, according to flow chart shown in Fig. 9 to the 1st embodiment of image processing method according to the present invention into
Row explanation.
In Fig. 9, the judegment part 53 (Fig. 3, Fig. 8) of image processing part 35 obtains imaging conditions from camera controller 28 and (indicates
The kind of the light source used when indicating shooting in the F value informations of current F values and the multiple light sources of lighting device 40 of aperture 18
The light source information of class) (step S10).
Then, judegment part 53 obtains imaging optical system (lens 16, aperture 18) corresponding with the imaging conditions obtained
MTF stipulated that spatial frequency (for example, the spatial frequency of a quarter of the sample frequency (fs) based on image-forming component 26
(0.25fs)) under MTF (step S12).For example, judegment part 53 has storage part, which stores shows in Fig. 4 and Fig. 7
Shooting when each type (wavelength X=546.070nm, λ=1092.14nm) of light source for the using and lens 16 with this example
The corresponding each F values of spherical aberration (2 λ) MTF, by from the storage part read MTF corresponding with current imaging conditions come
Obtain MTF.
Judegment part 53 differentiates whether the MTF obtained as described above is less than threshold value (50%), ("Yes" when less than the threshold value
The case where), it is determined as carrying out by putting the point carried out as restoration disposal portion 51 or 52 (Fig. 3, Fig. 8) as restoration disposal, and be transitioned into
Step S16 (step S14).That is, if judegment part 53 is determined as being put as restoration disposal, can make a little as restoration disposal portion 51
Or 52 work, and export by switch switch 54 movable contact pin 54C switch to the 2nd input 54B switching signal.
In step s 16, point is as restoration disposal portion 51 or 52 is only when judegment part 53 is determined as being put as restoration disposal
It is put as restoration disposal, and will have been put as the image data of restoration disposal is exported to the 2nd input of switching switch 54
54B.As a result, the image data put as restoration disposal is exported by switching switch 54.
On the other hand, when the MTF obtained is threshold value (50%) or more (the case where "No"), judegment part 53 is determined as not
It carries out by putting the point carried out as restoration disposal portion 51 or 52 as restoration disposal, and terminates present treatment (step S14).At this point, differentiating
The movable contact pin 54C for switching switch 54 is switched to the switching signal of the 1st input 54A by the output of portion 53, and by switching switch 54
It is input to the untreated image data that do not put as restoration disposal of the 1st input 54A.If also, judegment part 53 is sentenced
Then Wei not preferably it prevent a little as restoration disposal portion 51 or 52 is from working, to generate operation cost without putting as restoration disposal.
2nd embodiment ＞ of ＜ image processing methods
Figure 10 is the flow chart for the 2nd embodiment for indicating image processing method according to the present invention.In addition, Figure 10
In, pair step identical as the 1st embodiment shown in Fig. 9 marks identical step number, and description is omitted.
Differentiate whether put as restoration disposal according to the size of MTF in 1st embodiment, but the shown in Fig. 10 2nd
In embodiment, discriminate whether to be put instead of MTF as restoration disposal, this point according to the F values obtained in step slo
It is different.
That is, the judegment part 53 of the 2nd embodiment differentiates whether the F values obtained in step slo are F2 or more and F8 or less
(F2≤F values≤F8) is determined as when for F2≤F values≤F8 (the case where "Yes") without putting as restoration disposal, and terminates this
It handles (step S20).
On the other hand, when the F values obtained in step slo are less than F2 or more than F8 (the case where "No"), judegment part 53
It is determined as being put as restoration disposal, and is transitioned into step S16 (step S20).
In addition, the range (F2≤F values≤F8) of the F values differentiated in step S20 is the type and mercury lamp (wavelength X of light source
=546.070nm) corresponding range (referring to F values corresponding with the spherical aberration of 2 λ of Fig. 4), and when the type of light source is infrared
When linear light source (wavelength X=1092.14nm), the ranges of the F values differentiated in step S20 become F2.8≤F values≤F8 (with reference to
The corresponding F values of spherical aberration of 2 λ of Fig. 7).
3rd embodiment ＞ of ＜ image processing methods
Figure 11 is the flow chart for the 3rd embodiment for indicating image processing method according to the present invention.In addition, Figure 11
In, pair step identical as the 1st embodiment shown in Fig. 9 marks identical step number, and description is omitted.
In Figure 11, the judegment part 53 of the 3rd embodiment obtains recovery corresponding with the imaging conditions obtained in step slo
Filter filter gain (namely based on restore filter point as under the defined spatial frequency after restoration disposal MTF with
The ratio between the MTF under defined spatial frequency before restoration disposal) (with reference to step S22, corresponding with the spherical aberration of 2 λ of Fig. 6
Filter gain).
Then, judegment part 53 differentiates whether the filter gain obtained in step S22 is 2 or more and 5 or less (2≤filters
Wave device gain≤5), when for 2≤filter gain≤5 (the case where "Yes"), it is determined as being put as restoration disposal, and transition
To step S16 (step S24).This is because when for 2≤filter gain≤5, MTF corresponding with the spherical aberration of 2 λ is small
In threshold value (50%) (with reference to figure 4), to which image data is unsatisfactory for the desirable image quality required in inspection of product etc..
On the other hand, when the filter gain obtained in step S22 is less than 2 or more than 5 (the case where "No"), differentiate
Portion 53 is determined as without putting as restoration disposal, and terminates present treatment (step S24).When filter gain is less than 2, with 2 λ's
The corresponding MTF of spherical aberration becomes threshold value (50%) or more (with reference to figure 4), and image data meets desirable image quality, therefore nothing
It need to be put as restoration disposal.Also, when filter gain is more than 5, overcorrect may be led to by force by restoring intensity, therefore this
Without putting as restoration disposal in example, but can also be put as restoration disposal using the weak recovery filter of intensity is restored.
Wherein, as imaging conditions when to product check etc., the imaging conditions that filter gain is more than 5 are not preferably used.
[other]
In present embodiment, the photographic device 10 used in NI Vision Builder for Automated Inspection is illustrated, but photographic device 10
Purposes be not limited to machine vision purposes.Also, it can also be suitable for general digital camera, DV, monitoring
In video camera etc., and effect is easy to get in the camera being commonly used in the range of small aperture fuzzy F values do not occur.
Also, in present embodiment, image processing part 35 (Fig. 2, Fig. 3, figure in the camera controller 28 of photographic device 10
8) it plays a role as image processing apparatus according to the present invention, but it is not limited to this, when from photographic device 10 to computer
It, can also be so that the image processing part (image processing apparatus) of computer 60 plays and camera controller when 60 transmission RAW data
The mode of 35 identical function of image processing part in 28 constitutes the image processing part of computer 60.At this point, when computer 60 is obstructed
When crossing the control imaging conditions of camera controller 28, camera controller 28 needs to send the imaging conditions such as F values together with RAW data
To the image processing part of computer 60.
Moreover, the point of present embodiment has used Convolution-type Wiener filter as in restoration disposal, as filter is restored,
But it is not limited to this, such as can also use the recovery filter of spatial frequency area shown in [numerical expression 1] formula.At this point, needing
Fourier transform is carried out to the image data inputted, the image data of the spatial frequency area of Fourier transform will have been carried out
It is multiplied with the recovery filter of spatial frequency area, and multiplied result is subjected to inverse fourier transform.
Also, the present invention is not limited to the above embodiments, without departing from the spirit of the invention within the scope, certain energy
Enough carry out various modifications.
Symbol description
10- photographic devices, 12- lens units, 16- lens, 18- apertures, 20- optical system operationals portion, 26- is at pixel
Part, 28- camera controllers, 32- input/output interfaces, 34- device control unit, 35- image processing parts, 40- lighting devices, 51,
For 52- points as restoration disposal portion, 51A, 52A- restore filter process portion, and 51B- restores filter storage part, and 51C- restores filtering
Device selector, 52B- point spread function storage parts, 52C- restore filter generating unit, 53- judegment parts, 54- switching switches, 54A-
1st input, 54B- the 2nd are inputted, the movable contact pin of 54C-, 60- computers, 62- computer input and output portions, the control of 64- computers
Device, 66- displays, S10~S24- steps, f- focal lengths, λ-wavelength.
Claims (21)
1. a kind of image processing apparatus, has：
Point is as restoration disposal portion, for by using the image pickup part shooting subject with imaging optical system and image-forming component
The image data obtained from the image-forming component, uses the recovery filter of the point spread function based on the imaging optical system
It is put as restoration disposal；And
Judegment part, when putting modulation transfer function under the defined spatial frequency contributed as restoration disposal and according to imaging conditions
When the modulation transfer function changed is less than threshold value, it is determined as by the point as restoration disposal portion is put as restoration disposal,
The point only carries out the point as restoring as restoration disposal portion when the judegment part is determined as being put as restoration disposal
Processing.
2. image processing apparatus according to claim 1, wherein
When the sample frequency based on the image-forming component is set as fs, the defined spatial frequency is 0.25fs to 0.3fs
In the range of spatial frequency.
3. image processing apparatus according to claim 1 or 2, wherein
When modulation transfer function when being 0 by spatial frequency is set as 100%, in the range of the threshold value is 10% to 50%
Value.
4. image processing apparatus according to any one of claim 1 to 3, wherein
The imaging conditions are the f-number for the aperture for constituting the imaging optical system.
5. image processing apparatus according to claim 4, wherein
The judegment part transmits letter to differentiate according to the f-number of the aperture according to the modulation that the imaging conditions change
Whether number is less than the threshold value.
6. image processing apparatus according to any one of claim 1 to 4, wherein
The judegment part differentiates according to the filter gain under the defined spatial frequency based on the recovery filter
Whether the modulation transfer function changed according to the imaging conditions is less than the threshold value.
7. image processing apparatus according to any one of claim 1 to 6, wherein
The imaging conditions are the type of the light source illuminated to the subject or the wavelength of the light source.
8. image processing apparatus according to claim 7, wherein
As the modulation transfer function changed according to the imaging conditions, the judegment part use with to the subject into
The corresponding modulation transfer function of wavelength of the type or the light source of the light source of row illumination.
9. image processing apparatus according to any one of claim 1 to 8, wherein
The judegment part is sentenced according to the modulation transfer function in the predetermined region on the imaging picture of the image-forming component
Whether the modulation transfer function not changed according to the imaging conditions is less than the threshold value.
10. image processing apparatus according to claim 9, wherein
The modulation transfer function in the predetermined region is the region that the image height on the imaging picture is 50% or more
Modulation transfer function.
11. image processing apparatus according to claim 9, wherein
The modulation transfer function in the predetermined region is the modulation transfer function of the picture entirety of the imaging picture.
12. image processing apparatus according to claim 9, wherein
The modulation transfer function in the predetermined region is the modulation transfer function of any number of points of the imaging picture.
13. image processing apparatus according to any one of claim 1 to 12, wherein
When the aperture for constituting the imaging optical system is at least open aperture, the modulation of the imaging optical system passes
Delivery function is less than the threshold value.
14. a kind of photographic device has the image processing apparatus described in any one of claim 1 to 13 and the camera shooting
Portion.
15. photographic device according to claim 14 is used as industrial camera.
16. a kind of image processing method comprising following steps：
For by using the image pickup part shooting subject with imaging optical system and image-forming component and from the image-forming component
The image data of acquisition is carried out using the recovery filter of the point spread function based on the imaging optical system at a picture recovery
The step of reason；And
When picture restores the modulation transfer function under the defined spatial frequency contributed and according to described in imaging conditions variation
When modulation transfer function is less than threshold value, it is determined as carrying out the point as the step of restoration disposal,
During the point is carried out as the step of restoration disposal, only it is determined as carrying out the point in described the step of being differentiated as multiple
It is put as restoration disposal using the recovery filter when managing original place.
17. image processing method according to claim 16, wherein
When the sample frequency based on the image-forming component is set as fs, the defined spatial frequency is 0.25fs to 0.3fs
In the range of spatial frequency.
18. image processing method according to claim 16 or 17, wherein
When modulation transfer function when being 0 by spatial frequency is set as 100%, in the range of the threshold value is 10% to 50%
Value.
19. according to claim 16 to 18 any one of them image processing method, wherein
The imaging conditions are the f-number for the aperture for constituting the imaging optical system.
20. image processing method according to claim 19, wherein
In described the step of being differentiated, differentiated according to described in imaging conditions variation according to the f-number of the aperture
Whether modulation transfer function is less than the threshold value.
21. according to claim 16 to 19 any one of them image processing method, wherein
In described the step of being differentiated, according to based on the filter under the defined spatial frequency for restoring filter
Gain differentiates whether the modulation transfer function changed according to the imaging conditions is less than the threshold value.
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2016036196 | 2016-02-26 | ||
| JP2016-036196 | 2016-02-26 | ||
| PCT/JP2017/006419 WO2017146060A1 (en) | 2016-02-26 | 2017-02-21 | Image processing device, image capturing device, and image processing method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108605094A true CN108605094A (en) | 2018-09-28 |
| CN108605094B CN108605094B (en) | 2019-08-06 |
Family
ID=59686195
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201780010867.7A Active CN108605094B (en) | 2016-02-26 | 2017-02-21 | Image processing apparatus, photographic device and image processing method |
Country Status (4)
| Country | Link |
|---|---|
| US (1) | US10699387B2 (en) |
| JP (1) | JP6426323B2 (en) |
| CN (1) | CN108605094B (en) |
| WO (1) | WO2017146060A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112767242A (en) * | 2020-12-09 | 2021-05-07 | 佛山职业技术学院 | Image processing method and device based on frequency domain deconvolution and electronic equipment |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107534733B (en) * | 2015-04-23 | 2019-12-20 | 富士胶片株式会社 | Image pickup apparatus, image processing method of the same, and medium |
| CN117795974A (en) * | 2021-08-05 | 2024-03-29 | 三星电子株式会社 | Electronic device and method for supporting image data deblurring |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104854858A (en) * | 2012-09-26 | 2015-08-19 | 富士胶片株式会社 | Image processing device, imaging device, computer, and program |
| CN104956661A (en) * | 2013-02-01 | 2015-09-30 | 佳能株式会社 | Image pickup apparatus, image processing apparatus, image processing method, and non-transitory computer-readable storage medium |
| CN104995911A (en) * | 2013-02-15 | 2015-10-21 | 富士胶片株式会社 | Image processing device, image capture device, filter generating device, image restoration method, and program |
| CN105009579A (en) * | 2013-03-04 | 2015-10-28 | 富士胶片株式会社 | Restoration filter generation device and method, image processing device and method, imaging device, program, and recording medium |
| CN105009168A (en) * | 2013-03-04 | 2015-10-28 | 富士胶片株式会社 | Restoration filter generation device and method, image processing device and method, imaging device, program, and recording medium |
| CN105075239A (en) * | 2013-03-18 | 2015-11-18 | 富士胶片株式会社 | Restoration filter generation device and method, image processing device and method, image capturing apparatus, program, and recording medium |
Family Cites Families (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4743553B2 (en) * | 2008-09-29 | 2011-08-10 | 京セラ株式会社 | Lens unit, imaging device, and electronic device |
| WO2011096237A1 (en) * | 2010-02-08 | 2011-08-11 | パナソニック株式会社 | Imaging device |
| JP5455728B2 (en) * | 2010-03-17 | 2014-03-26 | キヤノン株式会社 | Imaging apparatus, image processing apparatus, and image processing method |
| JP5264968B2 (en) * | 2011-08-08 | 2013-08-14 | キヤノン株式会社 | Image processing apparatus, image processing method, imaging apparatus, and image processing program |
| JP5656926B2 (en) * | 2012-06-22 | 2015-01-21 | キヤノン株式会社 | Image processing method, image processing apparatus, and imaging apparatus |
| US20160198942A1 (en) * | 2012-12-31 | 2016-07-14 | Amo Development, Llc | Vergence weighting systems and methods for treatment of presbyopia and other vision conditions |
| JP5653464B2 (en) | 2013-02-01 | 2015-01-14 | キヤノン株式会社 | Imaging apparatus, image processing apparatus, image processing method, image processing program, and storage medium |
| WO2015015966A1 (en) * | 2013-08-01 | 2015-02-05 | 富士フイルム株式会社 | Imaging device, imaging method, and image processing device |
| WO2015064264A1 (en) * | 2013-10-31 | 2015-05-07 | 富士フイルム株式会社 | Image processing device, image capture device, parameter generating method, image processing method, and program |
| JP6027287B2 (en) | 2014-03-28 | 2016-11-16 | 富士フイルム株式会社 | Image processing apparatus, imaging apparatus, image processing method, and image processing program |
-
2017
- 2017-02-21 WO PCT/JP2017/006419 patent/WO2017146060A1/en active Application Filing
- 2017-02-21 JP JP2018501710A patent/JP6426323B2/en active Active
- 2017-02-21 CN CN201780010867.7A patent/CN108605094B/en active Active
-
2018
- 2018-08-23 US US16/110,238 patent/US10699387B2/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104854858A (en) * | 2012-09-26 | 2015-08-19 | 富士胶片株式会社 | Image processing device, imaging device, computer, and program |
| CN104956661A (en) * | 2013-02-01 | 2015-09-30 | 佳能株式会社 | Image pickup apparatus, image processing apparatus, image processing method, and non-transitory computer-readable storage medium |
| CN104995911A (en) * | 2013-02-15 | 2015-10-21 | 富士胶片株式会社 | Image processing device, image capture device, filter generating device, image restoration method, and program |
| CN105009579A (en) * | 2013-03-04 | 2015-10-28 | 富士胶片株式会社 | Restoration filter generation device and method, image processing device and method, imaging device, program, and recording medium |
| CN105009168A (en) * | 2013-03-04 | 2015-10-28 | 富士胶片株式会社 | Restoration filter generation device and method, image processing device and method, imaging device, program, and recording medium |
| CN105075239A (en) * | 2013-03-18 | 2015-11-18 | 富士胶片株式会社 | Restoration filter generation device and method, image processing device and method, image capturing apparatus, program, and recording medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112767242A (en) * | 2020-12-09 | 2021-05-07 | 佛山职业技术学院 | Image processing method and device based on frequency domain deconvolution and electronic equipment |
| CN112767242B (en) * | 2020-12-09 | 2023-01-24 | 佛山职业技术学院 | Image processing method and device based on frequency domain deconvolution and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| US10699387B2 (en) | 2020-06-30 |
| CN108605094B (en) | 2019-08-06 |
| US20180365812A1 (en) | 2018-12-20 |
| JP6426323B2 (en) | 2018-11-21 |
| WO2017146060A1 (en) | 2017-08-31 |
| JPWO2017146060A1 (en) | 2018-11-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7016835B2 (en) | Image processing method, image processing device, image processing system, learned weight manufacturing method, and program | |
| JP5284537B2 (en) | Image processing apparatus, image processing method, image processing program, and imaging apparatus using the same | |
| US8830351B2 (en) | Image processing method and image processing apparatus for image restoration to reduce a detected color shift | |
| US8477206B2 (en) | Image processing apparatus and image processing method for performing image restoration using restoration filter | |
| JP4986965B2 (en) | Image processing method, image processing apparatus, image processing program, and imaging apparatus | |
| US20110193997A1 (en) | Image processing method, image processing apparatus, and image pickup apparatus | |
| JP2004343519A (en) | Imaging apparatus and image processing apparatus | |
| US8228401B2 (en) | Image capturing and correction apparatus, image capturing and correction method, and medium storing a program | |
| CN107534727A (en) | Image processing apparatus, camera device, image processing method and image processing program | |
| CN108605094B (en) | Image processing apparatus, photographic device and image processing method | |
| JP5479187B2 (en) | Image processing apparatus and imaging apparatus using the same | |
| JP4891301B2 (en) | Image processing method, image processing apparatus, and imaging apparatus | |
| CN107852456A (en) | Image processing apparatus, camera device, image processing method and program | |
| CN107534728A (en) | Image processing apparatus, camera device, image processing method and program | |
| JP4818956B2 (en) | Imaging apparatus and method thereof | |
| CN108605095A (en) | Image processing apparatus, photographic device and image processing method | |
| US20240144536A1 (en) | Image processing device, image processing method, and storage medium | |
| JP2017126952A (en) | Image processing unit, imaging apparatus, and image processing program | |
| WO2011121761A1 (en) | Image processing apparatus and image capturing apparatus using same | |
| CN115428435A (en) | Information processing device, imaging device, information processing method, and program | |
| JP2009159536A (en) | Imaging device, image processing method, and information code reader | |
| JP2012090310A (en) | Image processing method, image processing apparatus, image pickup apparatus, and program | |
| JP2016092676A (en) | Image processing apparatus, imaging apparatus, camera system and image processing system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |