JP6311875B2 - Intermittent tracking imaging device - Google Patents

Description

  The present invention relates to an intermittent tracking imaging apparatus, and more particularly to an intermittent tracking imaging apparatus that images a high-speed moving body or images from a high-speed moving body.

  Blur (or blur) that occurs when a moving object is photographed with a camera is called motion blur. Even if the subject to be photographed is not moving, motion blur similarly occurs when the camera itself moves. Blurring due to so-called camera shake can be considered as one of motion blurs.

  Various techniques for reducing motion blur have been studied. For example, in the field of digital cameras, a technique for preventing camera shake by driving a lens system based on information from a built-in vibration gyro sensor is known (for example, Patent Document 1).

JP 2002-214659 A

  One method for photographing an object that moves at high speed is a method that always tracks the direction of the line of sight of the camera in the direction of the moving object. According to this method, the motion blur of the moving object can be reduced. However, in this method, since it is necessary to always point the camera's line-of-sight direction toward the moving object, the camera's line-of-sight direction cannot be set to an arbitrary direction. That is, the viewing direction of the camera is constrained to the direction of the moving object.

  As a method of reducing motion blur of a high-speed moving object, there is a method of shortening the time for opening the shutter of the camera. However, if the shutter opening time is shortened, the amount of received light decreases, so that it becomes difficult to obtain an image with sufficient brightness or a wide dynamic range for visual recognition.

  On the other hand, a camera shake correction technique such as a digital camera corrects the movement of the camera itself using a built-in vibration gyro sensor, and therefore cannot eliminate motion blur caused by the moving of the object to be photographed at high speed.

  The present invention has been made in consideration of such circumstances, even when shooting an object moving at high speed, or even when shooting an object from the apparatus moving at high speed. It is an object of the present invention to provide an intermittent tracking imaging device that can perform imaging with high accuracy and high sensitivity by reducing motion blur.

In order to solve the above problems, an intermittent tracking imaging apparatus according to the present invention includes an image sensor that captures an imaging target at a predetermined frame rate, and a changing unit that changes a relative positional relationship between the imaging object and the image sensor. When the shutter in the frame is opened, the change in the relative positional relationship caused by the movement of the shooting object or the movement of the apparatus is suppressed by tracking the shooting object, and when the shutter in the frame is closed. A control unit that controls the changing means for each frame so as to return the relative positional relationship to a predetermined home position, and the control unit tracks the imaging target when the shutter is opened. Frame processing and a second position for maintaining the relative positional relationship at the home position even when the shutter is opened. A frame processing is performed by time division, characterized in that.

  According to the intermittent tracking imaging apparatus of the present invention, motion blur is reduced even when shooting an object moving at high speed or when shooting an object from the apparatus moving at high speed. As a result, it is possible to photograph the object to be photographed with high accuracy and high sensitivity.

The figure which shows the application example of this invention. 1 is a block diagram illustrating a configuration example of an intermittent tracking imaging device according to a first embodiment. The figure which shows typically the generation | occurrence | production state of the motion blur in the conventional imaging device for the comparison with the intermittent tracking imaging device which concerns on embodiment. The figure which shows typically that a motion blur can be suppressed with respect to a high-speed movement target with the intermittent tracking imaging device which concerns on embodiment. The block diagram which shows the structural example of the intermittent tracking imaging device of a 1st modification. The block diagram which shows the structural example of the intermittent tracking imaging device of a 2nd modification. The figure which shows the other example of a change means. The figure which shows the operation | movement concept of the intermittent tracking imaging device of 2nd Embodiment. The block diagram which shows the structural example of the intermittent tracking imaging device of 3rd Embodiment. The figure which shows the operation | movement concept of the intermittent tracking imaging device of 3rd Embodiment. The figure which shows the structure of the prototype used for verification experiment. The 1st figure which shows the verification result of a verification experiment. The 2nd figure which shows the verification result of verification experiment.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. FIG. 1 is a diagram for explaining an application example of the intermittent tracking imaging apparatus 1 according to the embodiment of the present invention.

  FIG. 1A shows an example in which the intermittent tracking photographing apparatus 1 is applied to a scene where a photographing object close to a tunnel inner wall or the like is photographed from a high-speed moving body (such as a Shinkansen) that moves at a speed of 300 km / h. . The speed of 300 km / h corresponds to a speed of 83 m / s or 83 mm / ms. Even if the shutter opening time is assumed to be 0.1 ms, the amount of movement during this time is much larger than the pixel pitch, resulting in a large motion blur. Therefore, it is practically difficult to inspect cracks and the like on the inner wall of the tunnel using an image photographed by a conventional photographing apparatus. For this reason, for example, it is necessary to inspect roads, railway tunnels, etc. while temporarily blocking traffic at night, etc., and slowly moving the vehicle or repeatedly moving and stopping. Will be required.

  In such a scene, if the intermittent tracking imaging device 1 is applied, an image without motion blur can be obtained even when mounted on a vehicle moving at high speed. Therefore, it is possible to inspect the infrastructure of roads, railway tunnels, and surrounding facilities without closing traffic.

  FIG. 1B shows an example in which the intermittent tracking imaging device 1 is applied to a scene where, for example, a cell flowing in a microchannel having a channel diameter of 50 to 80 μm is imaged with a microscope-type imaging device. In recent years, in the field of regenerative medicine, the importance of techniques for examining and classifying cells is increasing. In such an inspection, there is a need to recognize the shape of each cell from an image obtained by photographing cells flowing in the microchannel. Even if the flow rate of cells flowing through the microchannel is 1 m / s, for example, if the magnification is enlarged with a microscope to the extent that the shape of the cells can be recognized, the movement speed with respect to the pixel pitch is large, and motion blur occurs in conventional imaging devices. It is difficult to clearly recognize the cell shape. If the intermittent tracking imaging device 1 of the embodiment is applied to such a scene, the shape of the cell or the like can be imaged without motion blur while the cell is flowing in the microchannel. As a result, for example, the quality of a cell can be determined from a large number of cells in a short time, and cell inspection can be performed with high throughput.

  The above is merely an example, and when the subject moving at high speed is photographed or when the present apparatus is mounted on a high-speed moving body, the intermittent tracking photographing apparatus 1 according to the embodiment of the present invention is applied, A clear image can be obtained without motion blur. In addition, by applying this apparatus to a consumer video camera or the like, it is possible to prevent camera shake and to capture golf swings and high-speed moving vehicles with desired brightness and without motion blur.

(1) First Embodiment FIG. 2 is a block diagram illustrating a configuration example of an intermittent tracking imaging device 1 according to the first embodiment. FIG. 2 shows a state where the high-speed movement target (photographing target) indicated by a black circle on the upper left of the figure is photographed by the intermittent tracking photographing apparatus 1. On the contrary, even when a fixed target (photographing target) is photographed by the intermittent tracking photographing apparatus 1 mounted on a high-speed moving body, the configuration of the intermittent tracking photographing apparatus 1 is shown in the block diagram of FIG. It is the same as shown. Further, when both the imaging target and the intermittent tracking imaging device 1 move, the configuration is the same as the block diagram of FIG.

  The intermittent tracking imaging apparatus 1 includes an image sensor 10, an actuator 20 (changing unit 20), an image generation unit 30, a speed calculation unit 40, and a control unit 50. The control unit 50 includes a position control unit (1) 51, a position control unit (2) 52, a changeover switch 53, and a shutter control unit 54.

  The image sensor 10 is, for example, a CMOS image sensor or a CCD image sensor, and a plurality of pixels are arranged in a surface array. The frame rate of the image sensor 10 is not particularly limited, but a higher-speed image sensor 10 is preferable from the viewpoint of photographing a high-speed moving body with high time resolution. In FIG. 2, illustration of an optical system such as a lens installed on the front side (target side) of the image sensor 10 is omitted.

  A shutter control signal is input from the shutter control unit 54 to the image sensor 10. The shutter control signal is a timing signal for controlling the frame rate and a timing signal for determining a shutter opening time and a shutter closing time in the frame. For example, the shutter (electronic shutter) of the image sensor 10 is opened when the frame control signal is on, and the shutter (electronic shutter) of the image sensor 10 is closed when the frame control signal is off.

  The changing unit 20 changes the relative positional relationship between the subject to be imaged and the image sensor 10. In the first embodiment shown in FIG. 2, the changing means 20 is an actuator 10 that changes the position of the image sensor 10, and is configured as a piezo actuator, for example. A control signal (control voltage) for the actuator 10 is supplied from the control unit 50. Further, a signal (image sensor position) indicating the current position of the image sensor 10 is output from the actuator 20 to the control unit 50.

  Based on the pixel signal output from the image sensor 10, the image generation unit 30 generates an image obtained by shooting the shooting target for each frame.

  The speed calculation unit 40 calculates the relative motion speed between the imaging target and the image sensor 10 from the change in the position of the imaging target captured by the image sensor 10 using the image for each frame. The specific operation of the speed calculation unit 40 will be described later.

When the shutter in the frame is opened, the control unit 50 causes the subject to be tracked to suppress a change in the relative positional relationship caused by the movement of the subject to be photographed or the movement of the apparatus, and at the time of closing the shutter in the frame, The actuator 20 (changing means 20) is controlled for each frame so that the relative positional relationship is returned to the predetermined home position.

  For example, the control unit 50 has a first feedback control loop that controls the position of the actuator 20 when the shutter is opened, and a second feedback control loop that controls the position of the actuator 20 when the shutter is closed. The first feedback control loop includes a position control unit (1) 51, and the second feedback control loop includes a position control unit (2) 52.

  In the first feedback control loop, the position of the actuator 20 is controlled so that the relative motion speed between the imaging target calculated by the speed calculation unit 40 and the image sensor 10 becomes zero speed (target value). Specifically, the selector switch 53 is switched to the position control unit (1) 51 side when the shutter is opened, and the actuator drive signal output from the position control unit (1) 51 is supplied to the actuator 20. With the first feedback control, the image sensor 10 tracks the object to be imaged.

  On the other hand, in the second feedback control loop, the actuator 20 is controlled so that the position of the image sensor 10 returns to a predetermined home position (initial position) when the shutter is closed. For this purpose, when the shutter is closed, the changeover switch 53 is switched to the position control unit (2) 52 side, and the actuator drive signal output from the position control unit (2) 52 is supplied to the actuator 20.

  As described above, in the intermittent tracking imaging apparatus 1 of the embodiment, the imaging target is tracked when the shutter is opened, and the image sensor is returned to the home position without tracking when the shutter is closed. In other words, tracking is performed intermittently. In this respect, it is greatly different from the conventional method in which the direction of the image sensor (or the direction of the photographing camera) is always directed to the moving target and is continuously tracked.

  When the relative motion between the object to be imaged and the image sensor 10 is assumed to be a two-dimensional motion (a two-dimensional motion in a plane orthogonal to the line-of-sight direction (for example, in the XY plane)), the actuator 20 The image sensor 10 is configured to be able to move two-dimensionally in the XY plane. On the other hand, when the relative motion between the subject to be imaged and the image sensor 10 is assumed to be a one-dimensional motion (for example, as shown in FIG. 1B, a cell flowing in one direction in the microchannel is photographed. In other cases, the actuator 20 may be configured such that the image sensor 10 can be moved only in one relevant direction.

  Further, instead of moving the image sensor 10 in parallel in a plane orthogonal to the line-of-sight direction, the image sensor 10 may be rotationally moved so that the line-of-sight direction of the image sensor 10 with respect to the shooting target is tracked by the shooting target. .

  FIG. 3 is a diagram schematically showing the occurrence of motion blur in a conventional imaging apparatus for comparison with the intermittent tracking imaging apparatus 1 according to the embodiment of the present invention.

  The upper part of FIG. 3 shows the position of the image sensor corresponding to each of the start time and end time of the shutter opening period with respect to three frames of frame #N, frame # (N + 1), and frame # (N + 2). ing. Each image sensor is represented by a top view of the upper side of the two-dimensional image sensor. The lower side of each image sensor corresponds to the left side (L) toward the target, and the upper side of the figure is the target. It corresponds to the right side (R). Note that a plurality of black circles arranged vertically on the left side of the upper stage in FIG. 3 indicate that one high-speed movement target moves from left to right.

  The diagram shown in the middle of FIG. 3 shows a graph in which the horizontal axis represents time and the vertical axis represents the position of the image sensor for three frames, frame #N, frame # (N + 1), and frame # (N + 2). It is. In the conventional photographing apparatus, since the position of the image sensor is fixed, the position of the image sensor is at the home position at any time. Note that the frame time is constant in any frame. Further, the ratio between the shutter opening time and the shutter closing time in the frame is also constant in every frame.

  The lower part of FIG. 3 is a diagram schematically showing an image obtained in each of frame #N, frame # (N + 1), and frame # (N + 2).

  As can be seen from the upper diagram in FIG. 3, the high-speed movement target naturally moves during the shutter opening time in the frame. For example, the frame #N moves from the position P1 to P2 between the start time and end time of the shutter opening period. Then, when this moving distance extends over a plurality of pixels when converted into pixels of the image sensor, motion blur occurs.

  As shown in the lower part of FIG. 3, motion blur occurs in any frame of frame #N, frame # (N + 1), and frame # (N + 2). As a result, the image obtained in each frame is a blurred image in which the target shape, which was originally a circle (black circle), extends in the moving direction (in the present example, extends from left to right).

  On the other hand, FIG. 4 is a diagram schematically showing that motion blur can be suppressed even for the same high-speed moving target by the intermittent tracking imaging apparatus 1 according to the embodiment of the present invention.

  4 corresponds to the start time and end time of the shutter release period for the three frames of frame #N, frame # (N + 1), and frame # (N + 2), similarly to the upper portion of FIG. The position of the image sensor 10 is shown.

  4 is a graph in which time is plotted on the horizontal axis and the position of the image sensor is plotted on the vertical axis in the same manner as in the middle section of FIG. 3, with the frame #N, frame # (N + 1), and frame # (N + 2). This is shown for three frames. As described above, in the intermittent tracking imaging device 1 according to the embodiment of the present invention, when the shutter is opened, the relative motion speed between the imaging target and the image sensor 10 is zero ( The position of the actuator 20 is controlled so as to be a target value. Therefore, when the target is moving at a constant speed, the actuator 20 is controlled to move the image sensor 10 at a constant speed. As a result, when the shutter is opened, the position of the image sensor 10 changes linearly and tracks the movement target.

  On the other hand, when the shutter is closed, the second feedback control loop works so that the position of the image sensor 10 returns to the home position. As a result, when the shutter is closed, the position of the image sensor 10 moves toward the home position, and when the shutter reaches the home position, the position is maintained. When the next shutter is opened, the position of the image sensor 10 is changed according to the movement of the target by the first feedback control loop, and the moving target is tracked. As a result, when the target moves at a constant speed, the position of the image sensor 10 exhibits a sawtooth movement as shown in the middle stage of FIG.

  Even if the high-speed movement target moves from the position P1 to P2 between the start time and the end time of the shutter release period of the frame #N by the above operation, for example, the position of the image sensor 10 follows this movement. Also move. As a result, on the image sensor 10, the same pixel always keeps capturing the target from the start to the end of the shutter opening period, and motion blur does not occur. Since the above tracking operation is performed during the shutter opening period of each frame, a clear image without motion blur can be obtained in any frame. For example, when shooting the same high speed moving target (circular target indicated by a black circle) as in FIG. 3, an image without motion blur is obtained in each frame as shown in the lower part of FIG. As a result, the target shape can be accurately recognized in each frame image. Further, by continuously reproducing each frame image, it is possible to provide a moving image in which the target shape is accurately maintained.

  In the example of the image shown in the lower part of FIG. 4, for convenience of explanation, the difference in the position of the target (black circle) between adjacent frames is increased, but this is a problem of time resolution, and the frame rate is By increasing the time resolution, the time resolution can be increased. That is, by operating the intermittent tracking imaging device 1 of the embodiment at a high frame rate, it is possible to obtain a moving image with smooth motion and no motion blur in each frame.

  Here, an example of the speed calculation method performed by the speed calculation unit 40 will be described.

Now, an input image input from the image generation unit 30 to the speed calculation unit 40 is denoted as I (X, Y, t). Here, X and Y are pixel positions, and t is time. This input image is binarized with a threshold value θ. If the binarized image is B (X, Y, t), the binarized image is
It is represented by

Next, 0th-order and first-order moment characteristics of the binarized image B (X, Y, t) are calculated. The zero-order and first-order moment features M _mn (t) are expressed by the following (formula 2).

Next, based on the moment feature of (Expression 2), the gravity center position c (t) is calculated by the following (Expression 3).

Next, based on the center-of-gravity position c (t) at time t and the center-of-gravity position c (t−τ) at time (t−τ) one frame before, the relative relationship between the object to be imaged and the image sensor 10 is determined. The motion speed v (t) is calculated by the following (Equation 4).

  The motion speed v (t) calculated by the speed calculation unit 40 as described above is provided to the first feedback control loop of the control unit 50 for each frame. The method for calculating the exercise speed is merely an example, and other known calculation methods may be used.

(2) Modified Example of First Embodiment FIG. 5 is a block diagram illustrating a configuration example of an intermittent tracking imaging device 1b according to a first modified example of the first embodiment. The difference from the first embodiment (FIG. 2) is that an edge strength calculation unit 60 is provided instead of the speed calculation unit 40. The edge strength calculation unit 60 calculates the edge strength of the shooting target from the image of the shooting target generated by the image generation unit 30. When the motion blur is large, the shape of the object to be photographed is blurred, so that the edge strength becomes small. On the contrary, when the motion blur is small, the shape of the object to be photographed becomes clear and the edge strength increases. Therefore, in the first modification of the first embodiment, the position of the image sensor 10 is controlled by the actuator 20 so that the edge strength is maximized, that is, the motion blur is reduced. For this purpose, the calculated edge strength is provided to the first feedback control loop of the control unit 150 and compared with a predetermined maximum value of edge strength. Then, the actuator 20 is driven to control the position of the image sensor 10 when the shutter is opened so that the calculated edge strength becomes the maximum edge strength value. The control of the second feedback control loop is the same as that in the first embodiment.

  The first embodiment using the motion speed may be combined with the first modification using the edge strength. In this case, the first feedback control loop is controlled using both parameters of the motion speed calculated by the speed calculation unit 40 and the edge strength calculated by the edge strength calculation unit 60.

  FIG. 6 is a block diagram illustrating a configuration example of the intermittent tracking imaging device 1c according to the second modification of the first embodiment. The difference from the first embodiment (FIG. 2) is that an input unit 70 is provided instead of the speed calculation unit 40. In the second modification, the motion speed used in the first feedback control loop is input from the outside via the input unit 70 instead of from the image to be captured. When the intermittent tracking imaging device 1c is mounted on a high-speed moving body such as a vehicle and the intermittent tracking imaging device 1c captures a fixed imaging object around the vehicle, the relative imaging target and the image sensor 10 are relative to each other. The movement speed can be obtained from the movement speed of the mounted vehicle. In the second modified example, the process for obtaining the motion speed from the image is not required, so that the process as a whole is simplified.

  Up to this point, the actuator 20 that moves the image sensor 10 itself in parallel or rotationally has been described as an example of changing means for changing the relative positional relationship between the object to be imaged and the image sensor 10 (FIG. 7A). . However, the changing means for changing the relative positional relationship is not limited to the actuator 20 that moves the image sensor 10 itself. For example, an optical path changing unit that changes an optical path position or an optical path angle of light incident on the image sensor 10 may be used. For example, as shown in FIG. 7B, a galvano mirror 20b as a changing means is installed between the object to be imaged and the image sensor 10, and the rotation angle of the galvano mirror 20b is controlled to enter the image sensor 10. The relative positional relationship between the object to be imaged and the image sensor 10 may be changed by changing the optical path position or the optical path angle of the light.

  Further, for example, when the intermittent tracking imaging device 1 is configured as a microscope type imaging device, the position of the objective lens is moved by the actuator 20c as shown in FIG. The optical path position or the optical path angle of the light incident on may be changed.

(3) Second Embodiment In the intermittent tracking photographing apparatus 1 according to the first embodiment described above, as shown in the middle part of FIG. 4, the process of tracking the photographing object when the shutters of all the frames are opened. It is carried out. When the object to be imaged is an isolated high-speed moving target moving in the background, motion blur of the high-speed moving target can be suppressed by the intermittent tracking imaging device 1 according to the first embodiment. However, in this case, motion blur occurs in the background image, and an image in which a fast moving target without motion blur is captured in the blurred background image is obtained. Depending on the purpose of shooting, such an image may be preferable, but on the other hand, an image without motion blur may be required for both the background and the high-speed moving target. The intermittent tracking imaging device 1d according to the second embodiment meets such a demand.

FIG. 8 shows an operation concept of the intermittent tracking imaging apparatus 1d according to the second embodiment. Specifically, in the intermittent tracking imaging device 1d, the first frame processing for tracking the imaging target when the shutter is opened, and the second in which tracking is not performed while the relative positional relationship is maintained at the home position even when the shutter is opened. The frame processing is performed in a time-sharing manner. For example, as shown in FIG. 8, the shooting target is tracked in even frames (frame #N, frame # (N + 2), frame # (N + 4), etc.), and odd frames (frame # (N + 1), frame # (N + 3), etc. ), Frame # (N + 5), etc.) is not tracked. If only even-frame images are extracted, a group of frame images without motion blur can be obtained for the movement target. Further, if only the odd-numbered frame images are extracted, a frame image group having no motion blur with respect to the background can be obtained.

  Furthermore, only the area of the moving target is extracted from the image group of the even frame, and the extracted moving target is combined with the image group of the odd frame to generate an image without motion blur for both the moving target and the background. It is also possible to do.

  Note that the intermittent tracking imaging device 1d according to the second embodiment is different in the processing timing of the control unit 50 from the first embodiment, but the configuration itself is the same as that of the first embodiment (FIG. 2). Therefore, illustration of the block diagram is omitted.

(4) Third Embodiment FIG. 9 is a block diagram illustrating a configuration example of an intermittent tracking imaging device 1e according to a third embodiment. The intermittent tracking imaging device 1e according to the third embodiment further includes a luminance calculation unit 80 and an exposure control unit 90 in addition to the configuration of the first embodiment. The luminance calculation unit 80 calculates the luminance of the image to be imaged taken by the image sensor 10. The exposure control unit 90 controls exposure by changing the ratio between the shutter opening time and the shutter closing time within the frame time for each frame according to the calculated luminance.

  The luminance calculation method itself is not particularly limited. For example, the luminance calculation unit 80 calculates a luminance histogram for each frame from the frame image to be imaged. The exposure control unit 90 compares the reference luminance histogram with the calculated luminance histogram for each frame, and calculates an index representing the magnitude of the difference between the two. Then, the ratio between the shutter opening time and the shutter closing time within the frame time is determined based on the calculated index so that the calculated luminance histogram for each frame approaches the reference luminance histogram. Then, from this ratio and the frame time, the shutter opening time and the shutter closing time are determined for each frame and output to the shutter control unit 54.

  FIG. 10 shows an operation concept of the intermittent tracking imaging apparatus 1e according to the third embodiment. In the exposure control performed in the third embodiment, the exposure time is controlled by changing the ratio between the shutter opening time and the shutter closing time within the frame time while keeping the frame time itself constant. For example, when the subject is suddenly brightened, the shutter opening time is shortened and the shutter closing time is lengthened, as in frame # (N + 1) or frame # (N + 2) in FIG. On the other hand, when the subject is suddenly darkened, the shutter opening time is lengthened and the shutter closing time is shortened as in frame # (N + 3) to frame # (N + 5) in FIG.

  In the intermittent tracking imaging device 1e according to the third embodiment, the exposure control described above and the tracking for the imaging target already described are simultaneously performed. As a result, for example, for an object to be photographed that is bright and has a slow moving speed, the exposure time (shutter opening time) is shortened as shown in frame #N to frame # (N + 2) in FIG. The amount of movement of the image sensor is small. On the other hand, for example, for a shooting object that is dark and has a high moving speed, the exposure time (shutter opening time) becomes longer and the tracking speed is increased as in frame # (N + 3) to frame # (N + 5) in FIG. (Or the movement amount of the image sensor) increases.

(5) Verification Experiment In order to confirm the effect of the intermittent tracking imaging apparatus 1 of the embodiment described above, a prototype was produced and a verification experiment was performed. FIG. 11A is a diagram showing the configuration of the prototype 100.

  The prototype 100 simulates the microscopic intermittent tracking imaging device 1. The prototype 100 includes a microscope 101, a high-speed camera 102, a control PC 103, a DA board 104, a piezo driver 105, a piezo stage 106, and a linear slider 107.

  As a subject to be photographed, a slide glass A having a fine black circle having a diameter of 50 micrometers shown in FIG. 11B drawn thereon, and a fine cat having a length of 100 micrometers and a width of 200 micrometers shown in FIG. Two types of slide glass B on which the above illustration was drawn were used.

  In the configuration of the microscope-type imaging apparatus shown in FIG. 7C, the position of the objective lens is changed by an actuator. However, in the verification experiment, the objective lens of the microscope 101 is fixed and the slide glass A that is an imaging target is used. , B position is feedback controlled by the piezo stage 106. Then, the piezo stage 106 is fixed on the linear slider 107, and the linear slider 107 is moved in one direction at a constant speed by a driving device (not shown) and the linear slider 107 is moved manually in two cases. A verification experiment was conducted.

  The high-speed camera 102 is equipped with an image sensor having 512 × 512 pixels and a pixel pitch of 10 × 10 μm. The objective lens of the microscope 101 used in the verification experiment has a magnification of 10 times. In this case, 1 μm of the pattern on the slide glasses A and B to be photographed corresponds to the size of one pixel of the image sensor.

  The frame rate of the high-speed camera 102 can be set up to a maximum of 2000 fps, but in the verification experiment, the frame rate was set to 125 fps, that is, the frame time was set to 8 ms. Of the frame time of 8 ms, the shutter opening time is set to 4 ms, and the shutter closing time is set to 4 ms.

  For the image for each frame output from the high-speed camera 102, the calculation shown in (Expression 1) to (Expression 4) was performed by the control PC to calculate the motion speed v (t) of the imaging target. Further, a movement amount (control amount) relative to the piezo stage 106 corresponding to the output of the position control unit (1) 51 in FIG. 2 was calculated from the motion speed v (t). The calculated movement amount was converted into an analog amount by the DA board 104 and applied to the piezo stage 106 via the piezo driver 105.

  FIG. 12 is a diagram illustrating an example of a result of a verification experiment in which a minute black circle (slide glass A) is an imaging target. FIG. 12A shows the object to be photographed again and is the same as FIG. 11B.

  12 (b) and 12 (c), a piezo stage 106 for controlling the position of the slide glass A (photographing target) is fixed to a linear slider 107, and the linear slider 107 is moved at a constant speed. It is the image which image | photographed the slide glass A (photographing object). The moving direction is from left to right in FIGS. 12 (b) and 12 (c). Further, the experiment was conducted by setting the moving speed to 0 mm / s, 1 mm / s,..., 14 mm / s and different speeds by 1 mm / s. FIGS. 12B and 12C show images obtained in an experiment set at a speed of 12 mm / s.

  FIG. 12B is an image taken by performing intermittent tracking according to the present invention. FIG. 12 (c) is an image taken without intermittent tracking for comparison with the present invention. As can be seen from FIGS. 12B and 12C, the image shot without intermittent tracking (FIG. 12C) showed a large motion blur, whereas the intermittent tracking of the present invention was performed. In the image photographed in FIG. 12 (b), motion blur was greatly reduced.

  The motion blur coefficient (motion blur (pixel)) was calculated as an index that quantitatively shows motion blur. The motion blur coefficient is based on the 0, 1 and 2nd moment characteristics of the binarized image area including the minute black circles. The motion blur coefficient was defined as the length of the long axis and the short axis when the small black circle of the captured image was approximated to an ellipse, and the difference between the long axis and the short axis expressed in units of pixels. In the absence of any motion blur coefficient is zero.

  The motion blur coefficients calculated next to FIGS. 12B and 12C are shown. When the intermittent tracking was not performed, the motion blur coefficient was as large as 22 pixels, but when the intermittent tracking was performed, the motion blur coefficient was reduced to 5 pixels.

  FIGS. 12D to 12F are graphs showing the results of a verification experiment performed by manually moving the linear slider 107 in order to move the linear slider 107 at an irregular speed. 12D shows the estimated speed, FIG. 12E shows the position control signal for the piezo stage 106, and FIG. 12F shows the motion blur coefficient calculated for evaluation. The horizontal axis is time.

  As shown in FIG. 12 (f), when intermittent tracking was performed, the motion blur coefficient was significantly reduced as compared with the case where intermittent tracking was not performed. From this result, when intermittent tracking is performed, the motion blur coefficient is small even when the speed of the shooting target changes, and even if the movement speed of the shooting target is unknown and the movement is irregular, the motion blur is It was found that the coefficient can be suppressed.

  FIG. 13 is a diagram illustrating an example of a result of a verification experiment using a cat illustration (slide glass B) as a subject of photographing. FIG. 13 (a) shows the object to be photographed again and is the same as FIG. 11 (c). In this verification experiment, the linear slider 107 to which the piezo stage 106 was fixed was manually moved in the same manner as when the results shown in FIGS. 12D to 12F were obtained. In addition, frames that perform intermittent tracking and frames that do not perform intermittent tracking were alternately performed. In either case, the frame time itself is 8 ms.

  FIG. 13B shows an arrangement of frame images subjected to intermittent tracking in time series, and FIG. 13C shows an enlarged image of t = 0032s. On the other hand, 13 (d) shows a frame image in which the intermittent tracking is not performed in time series, and FIG. 13 (e) is an enlarged image of t = 0040s.

  As is clear from FIGS. 13B to 13E, a large motion blur occurs when intermittent tracking is not performed, whereas a motion occurs when intermittent tracking is performed. Blur was suppressed.

  Thus, it has been verified that the motion blur is significantly suppressed by the intermittent tracking according to the present invention also by the above-described verification experiment.

  As described above, according to the intermittent tracking imaging apparatus according to the present invention, even when shooting an object moving at high speed, it is also when shooting an object from the apparatus moving at high speed. However, it is possible to shoot with high accuracy and high sensitivity by reducing motion blur.

  Note that the present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, the constituent elements over different embodiments may be appropriately combined.

1 intermittent tracking imaging device 10 image sensor 20 actuator (changing means)
30 Image generation unit 40 Speed calculation unit 50 Control unit 60 Edge strength calculation unit 70 Input unit 80 Luminance calculation unit 90 Exposure control unit

Claims (6)

An image sensor that shoots a subject at a predetermined frame rate;
Changing means for changing a relative positional relationship between the photographing object and the image sensor;
When the shutter in the frame is opened, tracking of the object to be photographed suppresses the change in the relative positional relationship caused by the movement of the object to be photographed or the movement of the apparatus, and when the shutter in the frame is closed, A control unit for controlling the changing means for each frame so as to return the relative positional relationship to a predetermined home position;
Equipped with a,
The control unit time-divides first frame processing for tracking the object to be photographed when the shutter is opened and second frame processing for maintaining the relative positional relationship at the home position even when the shutter is opened. Do in,
Intermittent tracking imaging apparatus characterized by. The changing means is an actuator that changes the position or angle of the image sensor, or an optical path changing means that changes the optical path position or optical path angle of light incident on the image sensor.
The intermittent tracking imaging apparatus according to claim 1. A speed calculation unit that calculates a relative motion speed between the imaging target and the apparatus from a change in the position of the imaging target captured by the image sensor;
The control unit tracks the imaging target based on the calculated motion speed.
The intermittent tracking photographing apparatus according to claim 1 or 2, An edge strength calculation unit that calculates an edge strength of the shooting target from the image of the shooting target shot by the image sensor;
The control unit tracks the imaging target based on the calculated edge strength.
The intermittent tracking photographing apparatus according to claim 1 or 2, An input unit for inputting a moving speed of the imaging target or the apparatus;
The intermittent tracking imaging apparatus according to claim 1, wherein the control unit tracks the imaging target based on the input moving speed. A luminance calculation unit that calculates the luminance of the imaging target imaged by the image sensor;
An exposure control unit that performs exposure control by changing a ratio between the shutter opening time and the shutter closing time within a frame time for each frame according to the luminance;
The intermittent tracking imaging apparatus according to claim 1 , further comprising: