JP2009206831A - Imaging apparatus, method of generating image, and electronic equipment - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide: an imaging apparatus capable of simplifying an optical system, reducing costs, and obtaining a satisfactory restored image; a method of generating an image; and electronic equipment. <P>SOLUTION: An imaging device 200 has a function for artificially expanding depth of field by driving a dual frequency drive type liquid crystal element (lens) for instantaneously switching a far side focus and a near side focus by applying the dual frequency drive type liquid crystal element (liquid crystal lens) to change a focus position to an optical system 210. An image processor 240 includes a function for generating an image expanding depth of field by compositing a plurality of images obtained by continuous imaging, composes the images after normalizing the difference in the size of an object image among the plurality of images occurring because of variations in an angle of field following a change in focus positions, performs the composition by averaging the data of the plurality of images, or performs the composition by selecting one having the highest contract among the plurality of images for each part of an image. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a digital still camera, a mobile phone camera, a mobile information terminal camera, an image inspection apparatus, an industrial camera for automatic control, an image generation method, and an electronic apparatus using an image sensor and having an optical system. It is about.

In response to the digitization of information, which has been rapidly developing in recent years, the response in the video field is also remarkable.
In particular, as symbolized by a digital camera, a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) sensor, which is a solid-state image sensor, is used in most cases instead of a conventional film.

  As described above, an imaging lens device using a CCD or CMOS sensor as an imaging element is for taking an image of a subject optically by an optical system and extracting it as an electrical signal by the imaging element. In addition to a digital still camera, It is used in video cameras, digital video units, personal computers, mobile phones, personal digital assistants (PDAs), image inspection devices, industrial cameras for automatic control, and the like.

FIG. 16 is a diagram schematically illustrating a configuration and a light flux state of a general imaging lens device.
The imaging lens device 1 includes an optical system 2 and an imaging element 3 such as a CCD or CMOS sensor.
In the optical system, the object side lenses 21 and 22, the diaphragm 23, and the imaging lens 24 are sequentially arranged from the object side (OBJS) toward the image sensor 3 side.

In the imaging lens device 1, as shown in FIG. 16, the best focus surface is matched with the imaging device surface.
17A to 17C show spot images on the light receiving surface of the imaging element 3 of the imaging lens device 1.

Further, an imaging apparatus has been proposed in which a light beam is regularly dispersed by a phase plate (Wavefront Coding optical element) and restored by digital processing to enable imaging with a deep depth of field (for example, non-patent literature). 1, 2, and patent documents 1 to 5).
In addition, an automatic exposure control system for a digital camera that performs filter processing using a transfer function has been proposed (see, for example, Patent Document 6).
“Wavefront Coding; jointly optimized optical and digital imaging systems”, Edward R. Dowski, Jr., Robert H. Cormack, Scott D. Sarama. “Wavefront Coding; A modern method of achieving high performance and / or low cost imaging systems”, Edward R. Dowski, Jr., Gregory E. Johnson. USP 6,021,005 USP 6,642,504 USP 6,525,302 USP 6,069,738 JP 2003-235794 A JP 2004-153497 A

In the imaging devices proposed in the above-mentioned documents, all of them are based on the assumption that the PSF (Point-Spread-Function) when the above-described phase plate is inserted into a normal optical system is constant, When the PSF changes, it is extremely difficult to realize an image with a deep depth of field by convolution using a subsequent kernel.
Therefore, apart from a single-focus lens, a zoom system, an AF system, or the like has a great problem in adopting due to the high accuracy of the optical design and the associated cost increase.
In other words, in the conventional imaging apparatus, proper convolution calculation cannot be performed, and astigmatism and coma that cause a shift of a spot (SPOT) image at the time of wide or tele (Tele). Therefore, an optical design that eliminates various aberrations such as zoom chromatic aberration is required.
However, the optical design that eliminates these aberrations increases the difficulty of optical design, causing problems such as an increase in design man-hours, an increase in cost, and an increase in size of the lens.

  It is an object of the present invention to provide an imaging apparatus, an image generation method, and an electronic apparatus that can simplify an optical system, can reduce costs, and can obtain a good image.

  An imaging apparatus according to a first aspect of the present invention includes an imaging optical system, a dual-frequency drive type liquid crystal element disposed at any position on the optical axis of the imaging optical system, and a subject via the imaging optical system. An image pickup device for picking up an image and a drive unit for the dual-frequency drive type liquid crystal element are provided, and the drive unit drives the dual-frequency drive type liquid crystal device to change the state during photographing.

  Preferably, the drive unit changes the focal position by changing the lens power while changing the focal length of the dual frequency drive type liquid crystal element.

  Preferably, the driving unit changes a wavefront aberration best position by modulating a wavefront, with a focal length of the dual-frequency driving type liquid crystal element unchanged.

  Preferably, the image processing unit includes an image signal processing unit that processes an image signal output from the imaging element, and the image signal processing unit combines a plurality of images obtained by continuously capturing images with the image processing unit. Generate an image with extended depth of field.

  Preferably, the composition is performed after normalizing the difference in the size of the subject image between the plurality of images caused by the change in the angle of view accompanying the change in the focal position by image processing.

  Preferably, the image signal processing unit performs the synthesis by averaging the data of the plurality of images.

  Preferably, the image signal processing unit selects the one having the highest contrast among the plurality of images for each image portion and performs the synthesis.

  Preferably, the imaging optical system has a diaphragm, and the dual-frequency driving liquid crystal element is disposed at a location adjacent to the diaphragm.

  Preferably, the dual-frequency driving liquid crystal element is disposed at a location adjacent to a surface having a higher power than that of the entire imaging optical system.

  Preferably, the dual frequency drive liquid crystal element has an infrared cut filter function.

  An image generation method according to a second aspect of the present invention is a method in which an imaging optical system is continuously photographed while changing a focal position via a two-frequency drive type liquid crystal element, and a plurality of images obtained by photographing are synthesized to be captured. Generate an image with extended depth of field.

  According to a third aspect of the present invention, there is provided an electronic apparatus having an imaging device, the imaging device being an imaging optical system and a dual frequency drive type disposed at any position on the optical axis of the imaging optical system. A liquid crystal element; an image sensor that captures an image of a subject via the imaging optical system; and a drive unit that drives the dual-frequency drive type liquid crystal element. The element is driven to change the state.

  According to the present invention, there is an advantage that the optical system can be simplified, the cost can be reduced, and a good image can be obtained.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

FIG. 1 is an external view showing an example of an information code reading device as an electronic apparatus to which an imaging device according to an embodiment of the present invention is applied.
2A to 2C are diagrams illustrating examples of information codes.
FIG. 3 is a block diagram illustrating a configuration example of an imaging apparatus applied to the information code reading apparatus of FIG.

As shown in FIG. 1, the information code reader 100 according to the present embodiment has a main body 110 connected to a processing device such as an electronic register (not shown) via a cable 111, for example, a reflectance printed on a reading object 120. It is a device that can read the information code 121 such as a different symbol or code.
When the reading start switch 112 formed on the main body 110 is operated, the information code reading device 100 has a function of using the trigger as a trigger to read the information code, for example, if the decoding determination is possible, the decoding result. Have.

  As an information code to be read, for example, a one-dimensional bar code 122 such as a JAN code as shown in FIG. 2A, a stack-type CODE 49 as shown in FIGS. 2B and 2C, Alternatively, a two-dimensional barcode 123 such as a matrix type QR code can be used.

In the information code reading apparatus 100 according to the present embodiment, an illumination light source (not shown) and an imaging apparatus 200 as shown in FIG.
The imaging apparatus 200 according to the present embodiment applies a dual-frequency drive type liquid crystal element (liquid crystal lens) to the optical system 210 in order to change the focal position, so that the conventional optical system that does not have the dual-frequency drive type liquid crystal element. Also, it is possible to extend the depth in a pseudo manner.

In the present embodiment, the imaging apparatus 200 changes the state by driving a dual-frequency drive type liquid crystal element during shooting.
Specifically, the depth of field is pseudo-expanded by driving a dual-frequency drive type liquid crystal element (lens) and instantaneously switching between the far side focus and the near side focus.
In addition, the imaging apparatus 200 has a function of changing the focal position by changing the lens power while changing the focal length of the dual-frequency driving type liquid crystal element.
That is, switching between the far side focus and the near side focus by the dual frequency drive type liquid crystal element has a function of changing the focal position of the liquid crystal lens by changing the focal length of the liquid crystal lens and changing the lens power.
In addition, the imaging apparatus 200 has a function of changing the wavefront aberration best position by modulating the wavefront while keeping the focal length of the dual-frequency drive type liquid crystal element unchanged.
That is, the switching between the far side focus and the near side focus by the dual frequency drive type liquid crystal element has a function of changing the focus position by modulating the wavefront while keeping the focal length of the liquid crystal lens unchanged.

Then, the imaging device 200 generates an image in which the depth of field is expanded by combining a plurality of images obtained by continuously capturing images.
The imaging apparatus 200 performs composition after normalizing the difference in the size of the subject image between a plurality of images caused by the change in the angle of view accompanying the change in the focal position by image processing.
The imaging apparatus 200 performs synthesis by averaging the data of the plurality of images.
In addition, the imaging apparatus 200 performs composition by selecting the one having the highest contrast among a plurality of images for each part of the image.
In other words, the imaging apparatus 200 has a function of correcting the angle-of-view variation that occurs when the far-side focus and the near-side focus are switched by the dual-frequency drive type liquid crystal element by coordinate conversion or image clipping.
In addition, the imaging apparatus 200 has a function of storing a plurality of images in a memory, performing image processing, and outputting during two-frequency driving.

  As shown in FIG. 3, the imaging device 200 of the information code reader 100 includes an optical system 210, an imaging device 220, an analog front end unit (AFE) 230, an image processing device 240, a camera signal processing unit 250, and an image display memory 260. , An image monitoring device 270, an operation unit 280, an exposure control device 290, and a drive unit 300.

The optical system 210 supplies an image obtained by photographing the subject object OBJ to the image sensor 220.
FIG. 4 is a diagram illustrating a first configuration example of an optical system using a dual-frequency drive type liquid crystal element.
FIG. 5 is a diagram showing a second configuration example of an optical system using a dual-frequency drive type liquid crystal element.

As shown in FIG. 4, the optical system 210A of the first configuration example includes, for example, a first lens 211 arranged in order from the object side OBJS, a dual-frequency drive type liquid crystal element 212, a second lens 113, and a third lens 214. , Aperture 215, fourth lens 216, and fifth lens 217.
The optical system 210A of the first configuration example includes a dual-frequency drive type liquid crystal element 212 with a lens surface having a strong power (in the example of FIG. 4, the image surface side surface of the first lens 211 or the object side surface of the second lens 212). ) Is arranged adjacent to.

As shown in FIG. 5, the optical system 220B of the second configuration example includes, for example, a first lens 211, a second lens 113, a third lens 214, a diaphragm 215, and a dual-frequency drive type that are arranged in order from the object side OBJS. A liquid crystal element 212, a fourth lens 216, and a fifth lens 217 are included.
The optical system 210 </ b> A of the first configuration example is an example in which the dual frequency drive type liquid crystal element 212 is disposed at a position adjacent to the stop 215.

  In the optical system 200 </ b> A and the optical system 200 </ b> B, the fourth lens 216 and the fifth lens 217 are cemented and function as an imaging lens for forming an image on the image sensor 220.

As will be described in detail below, in the present embodiment, a pseudo change is made by switching the focus position of the far side focus, the near side focus and the lens unit at a high speed by the high-speed molecular orientation change of the dual frequency drive type liquid crystal element 212. To get output with extended depth.
Hereinafter, the dual frequency drive type liquid crystal element 212 will be described in detail.

In the optical system 210 of the imaging apparatus 200 according to the present embodiment, a dual frequency drive type liquid crystal element (lens) 212 is used to change the focal position.
As shown in FIGS. 4 and 5, the arrangement position of the dual-frequency driving type liquid crystal element (lens) 212 is preferably adjacent to the surface of the lens having high power or adjacent to the diaphragm 215. .
The merit of the lens surface adjacent to a strong surface is that the fluctuation of the principal point position accompanying the power fluctuation of the dual-frequency drive type liquid crystal element 212 can be increased. The fact that the principal point position is moved more greatly than when the lens surface is adjacent to a weak surface, in other words, a longer shooting distance can be taken.

As an advantage of being adjacent to the diaphragm 215, the light beam is most converged in the vicinity of the diaphragm, and the diameter of the dual frequency drive type liquid crystal element 212 can be reduced.
Furthermore, since the fluctuation area of the dual-frequency drive type liquid element 212 is reduced, it is possible to increase the speed.
Since the aperture 215 has the most converged light, it can be reduced by reducing the diameter of the dual-frequency driving type liquid crystal element 213, and more preferably has an IR effect, so that it is not necessary to prepare an IR cut glass separately. This leads to cost reduction. An IR (infrared) cut function can be included in the dual-frequency drive type liquid crystal element 212 to make the dual-frequency drive type liquid crystal element 212 multifunctional. In order to reduce the influence of the angular characteristics of IR, it is preferable to be adjacent to the stop.

  FIG. 6 shows the time change of the power of the dual-frequency drive type liquid crystal element (lens).

The liquid crystal elements (lenses) can each have a variable focal length by applying a voltage to control the alignment state of the liquid crystal molecules.
In particular, the operating principle of the dual frequency drive type liquid crystal element (lens) can be changed from the convex lens to the flat plate to the concave lens by changing the orientation of the liquid crystal by an electric field.
Here, since the dual-frequency driving type liquid crystal element 212 can invert the polarity of dielectric anisotropy depending on the frequency of the applied voltage, the orientation of the liquid crystal can be changed by alternately applying a high-frequency electric field and a low-frequency electric field. .
Therefore, the effective refractive index can be changed with the electric field always applied, and the response speed can be increased only by increasing the electric field.
Further, in practice, the electric field is binary as shown in FIG. 6, but the lens power is continuous without being a binary value.

  FIG. 7 is a diagram for explaining the change in the angle of view accompanying the change in the focal length.

As shown in FIG. 7, when the power of the dual frequency drive type liquid crystal element (lens) 212 changes, the focal length of the lens system changes accordingly, and the angle of view changes.
If the angle of view fluctuates, the signals overlap and the image is blurred. In a moving image, of course, even a still image, if the frame rate is slow, images with different angles of view overlap and produce a sweet image.

  FIGS. 8A to 8C are diagrams for explaining the angle of view variation in more detail, and FIG. 8A shows a state in which a person at a close distance is in focus, and FIG. B) shows a state where a tree at a medium distance is in focus, and FIG. 8C shows a state where a tree at a long distance is in focus.

When the continuous shooting is performed while switching and driving the dual-frequency driving type liquid crystal element (lens) 212, the focused portions in the screen in each image are different as shown in FIGS.
When the focus position is on the near side, as shown in FIG. 8A, the person H in front is in focus, and the trees T and mountains M farther than that are blurred images.
However, when the focus position changes to a medium distance or a long distance due to the change of the dual frequency drive type liquid crystal element (lens) 212, the trees and the mountains are best focused, as shown in FIGS. 8B and 8C. The subject at other distances is a blurred image.
Therefore, as shown by broken lines BL in FIGS. 8A to 8C, the entire screen is divided into blocks, and an image with the timing at which each block is focused is used for the entire screen. It is possible to obtain an image that is in focus, that is, deep.

Specifically, in FIG. 8, for example, when the focus state of each image in a block in which the person H is captured is compared with, for example, a contrast value, the case of FIG. 8A is the highest, and FIG. It becomes lower in order of C).
Therefore, the block in which the person H is shown uses the image of FIG.
Next, regarding the block in which the M tree T is shown, the image in FIG. 8B has the highest contrast, and in FIGS. 8A and 8C, the contrast is lower.
Therefore, the block shown in FIG. 8B is used as the block in which the tree is shown.
Similarly, in the block in which the mountain M is shown, the image in FIG. 8C has the highest contrast, and decreases in the order of FIGS. 8B and 8A.
Therefore, the block in which the mountain M is shown uses the image of FIG.
In this way, when an image having the highest contrast for each block is selected and combined, an image having a deep depth can be obtained as a result.

  Even in this method, when there is a change in the angle of view accompanying a change in the focal length, it is necessary to normalize the image size by image processing.

FIG. 9 is a diagram illustrating a state in which the angle of view variation accompanying the change in focal length is corrected.
In FIG. 9, the solid line indicates an image after correction, and the broken line indicates an image before correction.

In order to prevent the state shown in FIG. 7, in FIG. 9, for example, a coordinate conversion method is used to perform image processing so that the same angle of view is always obtained.
As another method, for example, when the angle of view variation is small and the pixel shift is not performed, a method of averaging a plurality of images can be employed.

FIGS. 10A to 10C are views for explaining focus adjustment by a dual-frequency drive type liquid crystal element (lens).
FIG. 10A shows a focus state at low or high frequency, and FIG. 10B shows a case where the shape of the focus different from that in FIG. 10A is changed so as to change the focal length. Shows a case where the shape of the focus different from that in FIG. 10A is changed so that the wavefront aberration best position changes.

Here, a method of changing the dual-frequency drive type liquid crystal element (lens) will be described with reference to FIGS.
In the first case, as shown in FIG. 10B, the focus position is adjusted while changing the focal length of the dual-frequency drive type liquid crystal element (lens) 212. As an advantage of this method, the focus position can be changed dynamically. However, there is a change in the angle of view due to a large change.
Second, as shown in FIG. 10C, the focus position is adjusted by controlling the aberration without changing the focal length (curvature). The merit of this method is that there is little fluctuation in the angle of view. However, since a large change is difficult, the focus adjustment width is narrow and the photographing distance does not increase as compared with FIG.

  FIG. 11A is a diagram for explaining principal point position fluctuations depending on the arrangement position of the dual-frequency drive type liquid crystal element (lens).

  Next, the principal point position fluctuation due to the arrangement position of the dual-frequency drive type liquid crystal element (lens) will be described with reference to FIGS. 11 (A) and 11 (B).

FIG. 11A is a diagram arranged so as to be adjacent to a surface where the power of the lens is strong in order to increase the fluctuation of the principal point position.
In this case, a large principal point position variation effect due to power variation of the dual frequency drive type liquid crystal element (lens) 212 can be expected. On the other hand, since the sensitivity is strong and the aberration is large, the performance is remarkably deteriorated unless sufficient aberration correction is performed.

  FIG. 11B is a diagram arranged so as to be adjacent to the surface where the lens power is weak. In this case, the sensitivity is weak and the occurrence of aberrations due to fluctuations in the dual-frequency drive type liquid crystal element (lens) 212 can be suppressed. However, since the change in the principal point position is also reduced, the focus adjustment amount is reduced as compared with FIG.

  In the present embodiment, the image processing apparatus 240 performs composition processing and correction processing of continuously captured images.

The image sensor 220 forms an image captured by the optical system 210, and outputs the primary image information of the image formation as the primary image signal FIM of the electrical signal to the image processing device 240 via the analog front end unit 230. And a CMOS sensor.
In FIG. 3, the imaging element 220 is described as a CCD as an example.

The analog front end unit 230 includes a timing generator 231 and an analog / digital (A / D) converter 232.
The timing generator 231 generates the drive timing of the CCD of the image sensor 220, and the A / D converter 232 converts an analog signal input from the CCD into a digital signal and outputs it to the image processing device 240.

The image processing apparatus 240 constituting a part of the image signal processing unit inputs a digital signal of a captured image coming from the AFE 130 in the previous stage, and combines a plurality of images obtained by continuously capturing images to expand the depth of field. It has a function to generate an image.
The image processing unit 240 performs composition after normalizing the difference in the size of the subject image between the plurality of images caused by the change in the angle of view accompanying the change in the focal position by image processing.
The image processing device 240 performs synthesis by averaging the data of the plurality of images.
In addition, the image processing apparatus 240 performs synthesis by selecting an image having the highest contrast among a plurality of images for each part of the image.
In other words, the image processing apparatus 240 has a function of correcting the angle-of-view fluctuation that occurs when the far-side focus and the near-side focus are switched by the dual-frequency drive type liquid crystal element 212 by coordinate conversion or image cutout.
The processing of the image processing device 240 will be described in further detail later.

  The camera signal processing unit (DSP) 250 performs processing such as color interpolation, white balance, YCbCr conversion processing, compression, and filing, and stores in the memory 260 and displays images on the image monitoring device 270. The image monitoring device 270 can perform color output or monochrome output under the control of the camera signal processing unit 250.

The camera signal processing unit 250 performs switching support of the far side focus and the near side focus by the two-frequency drive type liquid crystal element 212 by the driving unit 300 in cooperation with the exposure control device 290 and the driving unit 300.
The camera signal processing unit 250 has a function of storing a plurality of images for one period in a memory, performing image processing, and outputting during two-frequency driving.

  The exposure control device 290 performs exposure control and has operation inputs from the operation unit 280, etc., and determines the operation of the entire system according to those inputs, and the AFE 230, the image processing device 240, the camera signal processing unit 250, and the like. To control the mediation of the entire system.

In response to an instruction from the camera signal processing unit 250, the driving unit 300 is a dual frequency driving type liquid crystal element 212.
Switching between the far side focus and the near side focus has a function of making the focal length of the liquid crystal lens variable and changing the focus position by changing the lens power.
Further, the driving unit 300 has a function of changing the wavefront aberration best position by modulating the wavefront, making the focal length of the dual-frequency driving type liquid crystal element unchanged according to an instruction from the camera signal processing unit 250.
That is, the driving unit 300 switches the far side focus and the near side focus by the two-frequency driving type liquid crystal element 212 in accordance with an instruction from the camera signal processing unit 250, the focal length of the liquid crystal lens remains unchanged, and the focus position is changed by wavefront modulation. It has a function to change.

  Hereinafter, specific processing of the imaging apparatus of the present embodiment will be described physically.

  First, processing when applied to a machine vision camera such as the information code reader (barcode reader) of FIG. 1 will be described.

  FIG. 12 is a diagram showing a processing flow when applied to a machine vision camera such as the information code reading device (bar code reader) of FIG. 1 using the dual-frequency drive type liquid crystal element (lens). .

First, under the control of the camera signal processing unit 150, the drive unit 300 sets the state of the dual frequency drive type liquid crystal element (lens) 212 to an initial state (ST1). In this initial state, an image such as an information code is acquired (ST2).
Here, decoding determination is performed (ST3). In step ST3, if decoding determination is possible (determination OK), the determination result is assumed to be correct (ST4), and the process is terminated.
In step ST3, when it is determined that the decoding determination is not good (NG), the drive unit 300 sets the state of the two-frequency drive type liquid crystal element (lens) 212 to a high frequency state under the control of the camera signal processing unit 150. (ST5). In this initial state, an image such as an information code is acquired (ST6).
Here, decoding determination is performed (ST7). In step ST7, when decoding determination is possible (determination OK), it is determined that the determination result is correct (ST4), and the process is terminated.
In step ST7, when it is determined that the decoding determination is not good (NG), under the control of the camera signal processing unit 150, the driving unit 300 changes the state of the two-frequency driving type liquid crystal element (lens) 212 to the low frequency state. (ST8). In this initial state, an image such as an information code is acquired (ST9).
Here, decoding determination is performed (ST10). In step ST10, when decoding determination is possible (determination OK), it is determined that the determination result is correct (ST4), and the process ends.
If it is determined in step ST7 that the decoding determination is not good (NG), it is determined that the determination result is not correct (ST11), and the process is terminated.

  Next, a process for extending the depth by capturing a plurality of images and combining them using a dual frequency drive type liquid crystal lens will be described.

  FIG. 13 is a diagram showing a flow of processing for expanding the depth by taking a plurality of images and combining them using a dual frequency drive type liquid crystal lens.

In starting the photographing by this method, under the control of the camera signal processing unit 250, the driving unit 300 sets the state of the two-frequency driving type liquid crystal element (lens) 212 to the initial state (ST21), and then the high frequency state and the low level are set. Driving is performed so that the frequency state is periodically repeated at high speed alternately (ST22).
In this state, image capture (ST23) and temporary storage in the memory are performed a predetermined number of times (in this flow, n times) (ST24).
When a predetermined number of images have been acquired, the driving of the dual-frequency drive type liquid crystal element (lens) 212 is stopped (ST25), and the stored images are synthesized (ST26), and an image with a deep depth of field is obtained. Is generated, and the series of processing ends.

  FIGS. 14A and 14B are diagrams showing the relationship between the switching of the two-frequency drive type liquid crystal element (lens) and the period of image acquisition when performing the processing of FIG.

FIG. 14A shows an example in which the frequency of image acquisition is very short with respect to the switching cycle of the state of the dual-frequency drive type liquid crystal element (lens).
At this time, images are continuously captured for at least a half cycle or more at the time of switching of the dual-frequency driving type liquid crystal element (lens) 212, and if possible, the image is switched to the timing at which the lens power is maximized and minimized by switching the liquid crystal lens. By synchronizing the timing of capture, the depth can be extended to the maximum.

FIG. 14B shows an example in which the image acquisition cycle is relatively long with respect to the switching cycle of the state of the dual-frequency drive type liquid crystal element (lens) 212.
As in the case of FIG. 14B, it is not possible to acquire the image by properly capturing the change in the state of the dual-frequency driving type liquid crystal element (lens) 212, but the image is acquired more than a certain number of times. It is possible to capture various states evenly.
Of course, more efficient depth extension can be realized by intentionally operating the timing as described above.

  Here, n = 5 (the case of capturing and combining five images has been described as an example, but it goes without saying that in either case of FIGS. 14A and 14B, a higher image quality can be expected with a larger number of captured images. Yes.

However, as the number of captured images increases, it takes time to capture, so the subject may move or camera shake may occur during that time. In addition, the time required for subsequent image composition increases.
Therefore, it is necessary to determine the number of images to be captured and combined in consideration of these conditions.

  FIG. 15 is a diagram illustrating a specific example of a method for correcting a change in the angle of view accompanying a change in focal length.

When the power of the dual-frequency driving type liquid crystal element (lens) 212 changes, the focal length of the lens system changes accordingly, and the angle of view changes.
This means that the image magnification of the subject changes slightly as shown in FIG. For image composition, it is necessary to normalize the image magnification to make the subject image the same size. Here, the concept of one method is shown.

(Calculate the magnification for the normalized target)
Select an image with the target image size to be normalized from multiple (frame) images taken by periodically switching and driving the dual-frequency drive type liquid crystal element (lens) 212 between a high frequency state and a low frequency state. To do.
In FIG. 15, it is assumed that an image indicated by a solid line is the target.

  The magnification of the image of another frame is calculated | required with respect to this image. Focusing on an arbitrary feature point of the image, detecting where the feature point is located in another frame, and obtaining the ratio of the distance from the center of the optical axis, the magnification can be obtained.

The change of each point due to the change of the focal length becomes equal magnification in the radiation direction at the center of the optical axis.
As for the feature points for calculating the magnification, the resolution of the calculation can be improved in the peripheral portion than in the vicinity of the optical axis.
In addition, reliability can be improved by using as many feature points as possible so as not to be in one place.

More specifically, referring to FIG. 15, an image indicated by a solid line is a target image to be normalized selected from a plurality of (frame) images taken while driving the dual-frequency drive type liquid crystal element (lens) 212. Is.
On the other hand, I1 indicated by a broken line is an overlapped image taken with a short focal length, that is, a high image magnification. Conversely, an image taken with a long focal length, that is, a low image magnification. I2 is indicated by a broken line.

  As shown in FIG. 15, the change in the position of the feature point at a distance S from the optical axis is V1 for I1 and V2 for I2. From these, each magnification is obtained as follows.

[Equation 1]
Magnification of I1 image with respect to normalized target image Is = (S + V1) / S
Magnification of I2 image with respect to normalized target image Is = (S + V2) / S

(Normalize each image with different image magnification)
In order to perform image synthesis, image magnification is normalized using the obtained magnification.
Normalization can be carried out by performing coordinate transformation on the entire image with the optical axis coordinate as the origin and multiplying by the inverse of the magnification. That is, assuming that the optical axis coordinates are (X0, Y0) and the normalized word coordinates are (Xs, Ys), the following is obtained.

[Equation 2]
For I1 image Xs = X0 + (X−X0) * S / (S + V1)
Ys = X0 + (Y−Y0) * S / (S + V1)

[Equation 3]
For I2 image Xs = X0 + (X−X0) * S / (S−V2)
Ys = X0 + (Y−Y0) * S / (S−V2)

Here, the method of actual measurement and normalization of magnification was shown by comparing acquired images on the assumption that the amount of change in the angle of view and the image capture timing with respect to the change of the dual-frequency drive type liquid crystal element (lens) 212 cannot be grasped. The present invention can employ methods other than this normalization method.
If the amount of change in the angle of view corresponding to the transition state of the change in the dual-frequency drive type liquid crystal element (lens) 212 is accurately known and the image capture timing corresponding thereto can be grasped, the procedure for calculating the magnification is as follows. The size of the image can be normalized by adjusting the magnification of the acquired image in accordance with the reciprocal of each angle of view variation (so-called digital zoom processing).
In either case, it is necessary to know the exact optical axis coordinates individually.

  As described above, the imaging apparatus 200 according to the present embodiment applies the dual-frequency drive type liquid crystal element (liquid crystal lens) to the optical system 210 to change the focal position, and the dual-frequency drive type liquid crystal element (lens). The image processing device 240 has a function of pseudo-expanding the depth of field by driving and instantaneously switching between the far side focus and the near side focus, and the image processing device 240 combines a plurality of images obtained by continuous imaging. It includes a function to generate an image with an extended depth of field and normalizes the difference in the size of the subject image between multiple images caused by the change in the angle of view accompanying the change in the focal position, and then combines this composition. The following effects can be obtained by averaging the data of a plurality of images, or selecting and combining the images having the highest contrast between the plurality of images for each part of the image.

By using a dual-frequency drive type liquid crystal element (lens) that can respond to the speed of light, the depth of the image can be artificially expanded by adjusting the size and synthesizing the image obtained by changing the focal position. As a result, the computation of the restoration process is unnecessary and a large depth range can be realized.
Further, there is an advantage that the optical system can be simplified, the cost can be reduced, and a good restored image can be obtained.

  The imaging apparatus according to the present embodiment has been described by way of example when applied to an information code reader. However, the imaging apparatus includes a digital still camera, a mobile phone camera, a mobile information terminal camera, an image inspection device, The present invention can also be applied as an imaging device for electronic devices such as industrial cameras for automatic control.

1 is an external view showing an example of an information code reading device according to an embodiment of the present invention. It is a figure which shows the example of an information code. It is a block which shows the structural example of the imaging device applied to the information code reader of FIG. It is a figure which shows the 1st structural example of the optical system using a dual frequency drive type liquid crystal element. It is a figure which shows the 2nd structural example of the optical system using a dual frequency drive type liquid crystal element. The time change of the power of a dual frequency drive type liquid crystal element (lens) is shown. It is a figure for demonstrating the view angle fluctuation | variation accompanying a focal distance change. FIGS. 8A and 8B are diagrams for explaining the angle of view variation in more detail. FIG. 8A shows a state in which a person at a close distance is in focus, and FIG. 8B shows a state in which a tree at a medium distance is in focus. FIG. 8C is a diagram showing a state in which a distant mountain is in focus. It is a figure which shows the state which correct | amended the view angle fluctuation | variation accompanying a focal distance change. It is a figure for demonstrating the focus adjustment by a dual frequency drive type liquid crystal element (lens). It is a figure for demonstrating the principal point position fluctuation | variation by the arrangement position of a dual frequency drive type liquid crystal element (lens). It is a figure which shows the flow of a process in the case of applying to a camera for machine vision, such as the information code reader (barcode reader) of FIG. 1, using a dual frequency drive type liquid crystal element (lens). It is a figure which shows the flow of the process which expands a depth by image | photographing several images using a dual frequency drive type liquid crystal lens, and synthesize | combining them. It is a figure which shows the relationship between the frequency of a liquid crystal element (lens) switching of a dual frequency drive at the time of performing the process of FIG. It is a figure which shows the specific example of the correction method of the view angle fluctuation | variation accompanying the change of a focal distance. It is a figure which shows typically the structure and light beam state of a general imaging lens apparatus. It is a figure which shows the spot image in the light-receiving surface of the image pick-up element of the image pick-up lens apparatus of FIG. 16, Comprising: (A) is a case where a focus shifts 0.2 mm (Defocus = 0.2 mm), (B) is a focus point. In the case (Best focus), (C) is a diagram showing each spot image when the focus is shifted by −0.2 mm (Defocus = −0.2 mm).

Explanation of symbols

  DESCRIPTION OF SYMBOLS 100 ... Information code reader, 121 ... Information code, 200 ... Imaging device, 210, 210A, 210B ... Optical system, 220 ... Imaging element, 230 ... Analog front end part ( AFE), 240 ... Image processing device, 250 ... Camera signal processing unit, 280 ... Operation unit, 290 ... Exposure control device, 300 ... Drive unit, 211 ... First lens, 212... Two-frequency driving type liquid crystal element (lens), 213... Second lens, 214... Third lens, 215. lens.

Claims (12)

An imaging optical system;
A dual frequency drive type liquid crystal element disposed at any position on the optical axis of the imaging optical system;
An image sensor that captures a subject image via the imaging optical system;
A drive section of the dual frequency drive type liquid crystal element,
The drive unit is
An imaging apparatus that changes the state by driving the dual-frequency driving type liquid crystal element during imaging. The drive unit is
The imaging apparatus according to claim 1, wherein the focal position is changed by changing a lens power while changing a focal length of the dual-frequency driving type liquid crystal element. The drive unit is
The imaging apparatus according to claim 1, wherein a focal length of the dual-frequency driving type liquid crystal element is unchanged, and a wavefront aberration best position is changed by wavefront modulation. An image signal processing unit for processing an image signal output from the image sensor;
The image signal processor is
The imaging device according to any one of claims 1 to 3, wherein a plurality of images obtained by continuous imaging are combined by the image processing unit to generate an image with an extended depth of field. The image signal processor is
The imaging apparatus according to claim 4, wherein the composition is performed after normalizing a difference in size of a subject image between the plurality of images caused by a change in angle of view accompanying the change in the focal length by image processing. The image signal processor is
The imaging apparatus according to claim 4, wherein the synthesis is performed by averaging the data of the plurality of images. The image signal processor is
The imaging apparatus according to claim 4, wherein the composition is performed by selecting an image having the highest contrast among the plurality of images for each part of the image. The imaging optical system has a diaphragm,
The imaging device according to any one of claims 1 to 7, wherein the dual-frequency driving liquid crystal element is disposed at a location adjacent to the diaphragm. The imaging device according to any one of claims 1 to 7, wherein the dual-frequency driving liquid crystal element is disposed at a location adjacent to a surface having a higher power than the power of the entire imaging optical system. The imaging device according to any one of claims 1 to 9, wherein the dual-frequency driving liquid crystal element has an infrared cut filter function. Shoot continuously while changing the focal position through the dual-frequency drive type liquid crystal element in the imaging optical system,
An image generation method for generating an image with an extended depth of field by combining a plurality of images obtained by photographing. An electronic device having an imaging device,
The imaging device
An imaging optical system;
A dual frequency drive type liquid crystal element disposed at any position on the optical axis of the imaging optical system;
An image sensor that captures a subject image via the imaging optical system;
The dual-frequency drive type liquid crystal element, and a drive unit,
The drive unit is
An electronic device that changes the state by driving the dual-frequency drive type liquid crystal element during photographing.

Images