JP2021067911A - Imaging device - Google Patents

Abstract

To provide a structure with which it is possible to achieve an auto-focus function without utilizing the event data outputted from an imaging element.SOLUTION: An imaging element 21 is adopted that outputs event data including two-dimensional point data with which the position of a pixel is specified in correspondence to a pixel the luminance change of which increased to or above a prescribed threshold when light is received via a light-receiving lens 22. Meanwhile, a portion of light heading from the light-receiving lens 22 toward the imaging element 21 is spectrally separated by a half mirror 24, and a phase difference AF sensor 25 having received this spectrally separated light detects, by a phase difference AF scheme, the amount of shift of the light-receiving lens 22 from an in-focus position when said light is received. The drive of an adjustment mechanism 23 is controlled by a control unit 11 on the basis of this detected amount of shift, so that the focus position of the light-receiving lens 22 is adjusted toward the in-focus position.SELECTED DRAWING: Figure 1

Description

The present invention relates to an imaging device.

In recent years, an event camera disclosed in Patent Document 1 below is known as a technique for generating an image of a measurement object at a higher speed. This event camera is a brightness value difference output camera developed by taking inspiration from the retinal structure of living organisms, so that it senses the change in brightness for each pixel and outputs its coordinates, time, and polarity of the change in brightness. It is configured. With such a configuration, the event camera has a feature that it does not output pixel information that does not change in brightness, that is, redundant data like a conventional camera, so that it is possible to reduce the amount of data communication and the weight of image processing. By doing so, it is possible to generate an image of the measurement object at a higher speed.

U.S. Patent Application Publication No. 2016/0227135

By the way, in the image data acquired by a normal camera, each pixel always has some kind of luminance information, and an autofocus function using the luminance information is installed in many cameras as a standard function. On the other hand, in the event camera, even if the event data corresponding to the change in the brightness can be acquired, the brightness information itself cannot be acquired, so that the autofocus function using the brightness information cannot be adopted. For this reason, even with an event camera, if the object to be measured is blurred because it is out of focus, the light of the object to be measured is dispersed and the event data cannot be obtained accurately. is there.

The present invention has been made to solve the above-mentioned problems, and an object of the present invention is to provide a configuration capable of realizing an autofocus function without using event data output from an image sensor. It is in.

In order to achieve the above object, the imaging device (10) according to the invention according to claim 1 of the claims is
An image sensor that outputs event data including two-dimensional point data in which the position of the pixel is specified corresponding to a pixel whose brightness change is equal to or greater than a predetermined threshold value when light is received through the light receiving lens (22). 21) and
An adjustment mechanism (23) for adjusting the focal position of the light receiving lens, and
A control unit (11) that drives and controls the adjustment mechanism, and
A shift amount detection unit (25) that detects the amount of deviation from the focusing position of the light receiving lens by the phase difference AF method when light is received through the light receiving lens.
A spectroscopic unit (24) that disperses a part of the light directed from the light receiving lens toward the image sensor and causes the deviation amount detection unit to receive light.
With
The control unit is characterized in that the focus position is adjusted toward the in-focus position by driving and controlling the adjustment mechanism based on the deviation amount detected by the deviation amount detection unit.
The reference numerals in the parentheses indicate the correspondence with the specific means described in the embodiments described later.

According to the first aspect of the present invention, event data including two-dimensional point data in which the position of the pixel is specified corresponding to the pixel whose brightness change becomes equal to or higher than a predetermined threshold value when the light is received through the light receiving lens is output. An image sensor is used. Then, a part of the light directed from the light receiving lens to the image sensor is dispersed by the spectroscopic unit, and the deviation amount detecting unit that receives the dispersed light receives the focused position of the light receiving lens by the phase difference AF method. The amount of deviation from is detected. The adjustment mechanism is driven and controlled by the control unit based on the detected deviation amount, so that the focal position is adjusted toward the in-focus position.

In this way, the amount of deviation from the focusing position of the light receiving lens can be detected and the focal position can be adjusted by using the light dispersed by the spectroscopic unit, so that the event data output from the image sensor is not used. The autofocus function can be realized.

In the invention of claim 2, since the predetermined threshold value is set according to the light amount attenuation ratio by the spectroscopic unit, the light amount received by the image sensor by the spectroscopic unit is reduced, so that the output frequency of event data is reduced. It can be suppressed.

In the invention of claim 3, the predetermined threshold value is changed by the threshold value changing unit according to the deviation amount detected by the deviation amount detecting unit. If the object to be measured is moving, it may become difficult to focus and the output frequency of event data may continue to decrease. Therefore, by lowering the predetermined threshold value when the amount of deviation becomes large, it is possible to suppress a decrease in the output frequency of event data even if the out-of-focus state continues. On the other hand, when the focus is on, the output of excessive event data can be suppressed by raising the predetermined threshold value.

It is a block diagram which shows the schematic structure of the image pickup apparatus which concerns on 1st Embodiment. It is explanatory drawing which compares the luminance change in the state which is separated by a half mirror, and the luminance change in the state which is not spectroscopic. FIG. 3 (A) is an explanatory diagram illustrating an object to be measured, and FIG. 3 (B) shows a state in which the light from the object to be measured of FIG. 3 (A) is received and is in focus. It is explanatory drawing explaining the position change of the measured luminance and the position change of the luminance measured in the state of being out of focus. FIG. 4A is an explanatory diagram illustrating a change in the position of the brightness that changes when the measurement object shown in FIG. 3A moves to the right in the drawing while the object is in focus. FIG. 4B is an explanatory diagram illustrating a change in the position of the brightness that changes when the measurement object shown in FIG. 3A moves to the right in the drawing in a state of being out of focus, and FIG. 4C. ) Is an explanatory diagram illustrating the range in which the event data before the threshold value change is output in the light receiving state of FIG. 4 (B), and FIG. 4 (D) shows the event data after the threshold value change in the light receiving state of FIG. 4 (B). It is explanatory drawing which illustrates the range which the event data of is output.

[First Embodiment]
Hereinafter, the first embodiment embodying the image pickup apparatus of the present invention will be described with reference to the drawings.
The imaging device 10 according to the present embodiment is a device that functions as a so-called event camera. The image pickup device 10 outputs event data so as to include two-dimensional point data in which the position of the pixel is specified corresponding to the pixel whose brightness has changed, time, and the polarity of the brightness change, and within a certain period of time. By plotting the two-dimensional point data of the plurality of output event data as points on a predetermined plane, image data obtained by capturing an image of the measurement target is generated.

As shown in FIG. 1, in addition to a control unit 11 including a CPU and the like and a storage unit 12 including a semiconductor memory and the like, the image pickup apparatus 10 responds to an input operation and a display unit 13 whose display contents are controlled by the control unit 11. It includes an operation unit 14 that outputs the operation signal to the control unit 11, a communication unit 15 for communicating with an external device, and the like.

Further, the image pickup device 10 includes an image pickup element 21, a light receiving lens 22, an adjustment mechanism 23, a half mirror 24, a phase difference AF sensor 25, and the like as an image pickup unit. The image sensor 21 controls event data including two-dimensional point data in which the position of the pixel is specified corresponding to the pixel whose brightness change is equal to or greater than a predetermined threshold value when the light is received through the light receiving lens 22. It is configured to output to 11. That is, the image pickup element 21 outputs event data (two-dimensional point data, time, polarity of brightness change) corresponding to the pixel whose brightness change is equal to or more than a predetermined threshold value to the control unit 11, and refers to the pixel having no brightness change. It works so that no data is output. The adjustment mechanism 23 is a known mechanism for adjusting the focal position of the light receiving lens 22, and is driven and controlled by the control unit 11 to move the light receiving lens 22 to one side (adjustment direction) along the optical axis or another. By moving it to the side, the focal position of the light receiving lens 22 is adjusted.

The half mirror 24 is configured as a spectroscopic unit for splitting a part of the light directed from the light receiving lens 22 toward the image sensor 21 according to a predetermined light amount attenuation ratio and causing the phase difference AF sensor 25 to receive the light. Therefore, in the present embodiment, the predetermined threshold value in the image sensor 21 is set according to the predetermined light amount attenuation ratio by the half mirror 24.

Hereinafter, the reason for setting the predetermined threshold value in the image pickup device 21 according to the predetermined light amount attenuation ratio by the half mirror 24 will be described with reference to FIG. Note that FIG. 2 is an explanatory diagram for comparing the brightness change in the state of being separated by the half mirror 24 and the brightness change in the state of not being separated.

The image sensor 21 that outputs event data corresponding to pixels whose brightness changes are equal to or greater than a predetermined threshold is not affected by sensor noise even if the amount of received light is reduced by the half mirror 24. However, since the light receiving amount itself is reduced, the brightness change width is reduced, so that the output frequency of event data may be reduced. For example, as shown in FIG. 2, when a change in brightness in a non-spectroscopic state is detected as ΔL1a, the amount of light received is reduced according to the predetermined light amount attenuation ratio by spectroscopy, and the change in brightness is ΔL1b. It is assumed that it is detected as. In such a case, when the brightness change ΔL1a becomes large with respect to the predetermined threshold value, while the brightness change ΔL1b becomes small, the event data is not output because the spectrum is performed.

Therefore, in the present embodiment, the predetermined threshold value in the image sensor 21 is set according to the predetermined light amount attenuation ratio by the half mirror 24. Specifically, the predetermined threshold value is set by multiplying the threshold value in the case of no spectroscopy by ΔL1b / ΔL1a to reduce the threshold value. As a result, it is possible to suppress a decrease in the output frequency of event data due to spectroscopy by the half mirror 24.

The phase difference AF sensor 25 is a known sensor that detects the amount of deviation of the light receiving lens 22 from the in-focus position by the phase difference AF method when light is received through the light receiving lens 22. The phase-difference AF sensor 25 is configured to output a signal corresponding to the above-mentioned deviation amount to the control unit 11 when the light dispersed by the half mirror 24 is received. The phase difference AF sensor 25 can correspond to an example of a "shift amount detection unit" that detects the deviation amount of the light receiving lens 22 from the focusing position.

In the image pickup apparatus 10 configured in this way, in the focus position adjustment process performed by the control unit 11, the amount of deviation of the light receiving lens 22 detected by the phase difference AF sensor 25 from the in-focus position is reduced. By driving and controlling the adjusting mechanism 23, the focal position of the light receiving lens 22 is adjusted toward the focusing position.

As described above, in the image pickup device 10 according to the present embodiment, the position of the pixel is specified corresponding to the pixel whose brightness change is equal to or more than a predetermined threshold value when the light is received through the light receiving lens 22. An image sensor 21 that outputs event data including dimension point data is adopted. Then, a part of the light directed from the light receiving lens 22 to the image sensor 21 is dispersed by the half mirror 24, and the phase difference AF sensor 25 that receives the dispersed light receives the separated light by the phase difference AF method. The amount of deviation from the focusing position of 22 is detected. The adjustment mechanism 23 is driven and controlled by the control unit 11 based on the detected deviation amount, so that the focal position of the light receiving lens 22 is adjusted toward the in-focus position.

In this way, the amount of deviation from the focusing position of the light receiving lens 22 can be detected and the focal position can be adjusted by using the light dispersed by the half mirror 24, so that the event data output from the image sensor 21 can be obtained. The autofocus function can be realized without using it.

In particular, since the predetermined threshold value is set according to the light amount attenuation ratio by the half mirror 24, the amount of light received by the image sensor 21 by the half mirror 24 is reduced, so that the output frequency of event data is reduced. It can be suppressed.

[Second Embodiment]
Next, the image pickup apparatus according to the second embodiment will be described with reference to the drawings.
The second embodiment is mainly different from the first embodiment in that the predetermined threshold value is changed according to the amount of deviation detected by using the phase difference AF sensor 25. Therefore, the same components as those of the first embodiment are designated by the same reference numerals, and the description thereof will be omitted. Note that FIG. 3A is an explanatory diagram illustrating an object to be measured, and FIG. 3B is in focus when light from the object to be measured in FIG. 3A is received. It is explanatory drawing explaining the position change of the luminance measured in the state and the position change of the luminance measured in the state of being out of focus. FIG. 4A is an explanatory diagram illustrating a change in the position of the brightness that changes when the measurement object shown in FIG. 3A moves to the right in the drawing while the object is in focus. FIG. 4B is an explanatory diagram illustrating a change in the position of the brightness that changes when the measurement object shown in FIG. 3A moves to the right in the drawing in a state of being out of focus, and FIG. 4C. ) Is an explanatory view illustrating the range in which the event data before the threshold value change is output in the light receiving state of FIG. 4 (B), and FIG. 4 (D) shows the event data after the threshold value change in the light receiving state of FIG. 4 (B). It is explanatory drawing which illustrates the range which the event data of is output.

In the configuration that employs the image sensor 21 described above, if the object to be measured is moving, it may be difficult to focus and the output frequency of event data may continue to decrease. For example, as shown in FIG. 3A, it is assumed that a surface in which black and white changes into a wave shape along one direction is used as a measurement target and light from that surface is received. When such a measurement object is imaged in a stationary state, when light is received in a state of being in focus (a state in which the focal position coincides with the in-focus position), the brightness L2 shown in FIG. 3 (B) is obtained. , The luminance value changes so that the luminance band (see the reference numeral ΔL2m in FIG. 3B) becomes wider. On the other hand, if the light is received in a state of being out of focus (a state in which the focal position is out of focus), the blur becomes large. The luminance value changes so that the band (see the reference numeral ΔL3m in FIG. 3B) becomes narrower.

Therefore, when the measurement object shown in FIG. 3 (A) moves to the right in the drawing, the brightness change becomes relatively large and the focus shifts as can be seen from FIG. 4 (A) when the object is in focus. In this state, as can be seen from FIG. 4B, the change in brightness is relatively small. For example, in the vicinity of the minimum brightness value, the brightness change in the out-of-focus state (see the symbol ΔL3 in FIG. 4B) is the brightness change in the in-focus state (reference numeral ΔL2 in FIG. 4A). See).

In this way, when the object to be measured moves, the change in brightness becomes smaller in the out-of-focus state than in the in-focus state, so that the output frequency of the event data decreases. For example, in the light receiving state of FIG. 4 (B), the change in luminance is particularly small between the vicinity of the maximum luminance value and the vicinity of the minimum luminance value. Therefore, before the threshold value is changed, the event data output range Sa of FIG. 4 (C) is used. As shown, the event data is not output near the maximum luminance value and the minimum luminance value.

Therefore, in the present embodiment, the threshold value of the image sensor 21 is controlled by the control unit 11, and the predetermined threshold value, which is a reference when the image sensor 21 outputs event data, is detected by using the phase difference AF sensor 25. Change according to the amount of slippage. Specifically, for example, if the detected deviation amount is equal to or more than a preset predetermined amount, the image sensor 21 is threshold-controlled by the control unit 11 so as to lower the predetermined threshold value by a predetermined value. The control unit 11 that controls the threshold value for the image sensor 21 may correspond to an example of the “threshold value changing unit”.

As a result, for example, even in the light receiving state of FIG. 4 (B), after the threshold value is changed, as shown in the event data output range Sb of FIG. 4 (D), in FIG. 4 (C) before the threshold value is changed. The output range is wider than the event data output range Sa, and the output frequency of event data can be increased. That is, by lowering the predetermined threshold value when the amount of deviation becomes large, it is possible to suppress a decrease in the output frequency of event data even when the out-of-focus state continues.

On the other hand, when the detected deviation amount is less than or equal to a predetermined amount, that is, when the focus is substantially adjusted, the excessive output of event data can be suppressed by raising the predetermined threshold value.

The present invention is not limited to each of the above embodiments, and may be embodied as follows, for example.
(1) In the second embodiment, the predetermined threshold value, which is a reference when the image sensor 21 outputs event data, is changed so that the decrease amount increases as the detected deviation amount increases. May be good.

(2) The amount of deviation of the light receiving lens 22 from the in-focus position is not limited to being detected by the phase difference AF sensor 25, but may be detected by a deviation amount detection unit that employs another detection method.

10 ... Imaging device 11 ... Control unit (threshold change unit)
21 ... Image sensor 22 ... Light receiving lens 23 ... Adjustment mechanism 24 ... Half mirror (spectroscopic part)
25 ... Phase difference AF sensor (displacement amount detection unit)

Claims (3)

An image sensor that outputs event data including two-dimensional point data in which the position of the pixel is specified corresponding to a pixel whose brightness change is equal to or greater than a predetermined threshold value when light is received through the light receiving lens.
An adjustment mechanism for adjusting the focal position of the light receiving lens and
A control unit that drives and controls the adjustment mechanism,
A shift amount detection unit that detects the amount of deviation from the focusing position of the light receiving lens by the phase difference AF method when receiving light through the light receiving lens.
A spectroscopic unit that disperses a part of the light directed from the light receiving lens toward the image sensor and causes the deviation amount detection unit to receive light.
With
The control unit is an imaging device characterized in that the focus position is adjusted toward a focusing position by driving and controlling the adjustment mechanism based on the deviation amount detected by the deviation amount detecting unit. The imaging device according to claim 1, wherein the predetermined threshold value is set according to the light amount attenuation ratio by the spectroscopic unit. The imaging apparatus according to claim 1, further comprising a threshold value changing unit that changes the predetermined threshold value according to the deviation amount detected by the deviation amount detecting unit.

Images