JP6248491B2 - Imaging apparatus control apparatus, imaging apparatus, imaging method, imaging program, storage medium, and microscope apparatus - Google Patents

Description

  The present invention relates to an imaging apparatus control apparatus, an imaging apparatus, an imaging method, an imaging program, a storage medium, and a microscope apparatus.

  In recent imaging apparatuses, a CMOS type image sensor that consumes less power and can be driven at high speed by a rolling shutter system is often used. This image sensor has a large number of pixels (photoelectric conversion elements) arranged two-dimensionally and uses a MOS transistor circuit as a signal readout circuit. The rolling shutter method is a driving method in which signals are read out by sequentially shifting the time for each pixel row arranged in the horizontal direction of the image sensor (see Patent Document 1).

US Pat. No. 7,667,740

  By the way, when the rolling shutter system is used, the timing of exposure (charge accumulation) differs from pixel row to pixel row from the first pixel row to the last pixel row. Therefore, when a moving subject or a changing subject is imaged, the image is distorted. Therefore, when imaging is performed while illuminating a subject, it has been proposed to irradiate illumination light only during a period in which the first pixel row and the last pixel row overlap to eliminate image distortion.

  However, if the exposure time of each pixel row is short, there is no overlapping period, so it is unavoidable to irradiate illumination light over the entire period from the first pixel row to the last pixel row. For example, when using a microscope with a rolling shutter system connected to a microscope, the position of the object to be observed is adjusted by moving it horizontally and vertically with respect to the optical system. At this time, the observation position is adjusted while viewing the live image taken through the camera. Therefore, fine adjustment is difficult at a low frame rate, and it is necessary to take an image at a high frame rate. The frame rate varies depending on the length of the exposure time, and in order to obtain a high frame rate, the exposure time must be shortened. Therefore, the user switches and uses two different illumination methods depending on whether there is an overlap period. Further, when observing a biological specimen, it is expected that the biological specimen will deteriorate if the exposure time is long when the preliminary observation for searching the observation position is performed before the main observation for performing the detailed observation. Therefore, it is preferable that the exposure time is short at the time of preliminary observation. Thus, the illumination method is different between the case where the exposure time is short and there is no overlap period, and the case where the exposure time is relatively long and there is an overlap period. Therefore, when the user adjusts the brightness of the image at the time of imaging, since it gradually becomes dark when the illumination light irradiation period switches from the full period to the overlap period when the adjustment is gradually increased from a short exposure time, There is a problem that the change becomes discontinuous and it is difficult to adjust.

According to the first aspect of the present invention, rolling for controlling the charge accumulation period t for each pixel row while sequentially shifting the time from the first pixel row to the last pixel row of the image sensor in which a plurality of pixels are two-dimensionally arranged. A shutter control unit, a first illumination control unit that irradiates the observation specimen with illumination light from the illumination unit over the entire period from the start of charge accumulation in the first pixel row to the end of charge accumulation in the last pixel row, and the charge in the first pixel row The second illumination control unit that irradiates the observation specimen with illumination light from the illumination unit only during the overlap period in which the accumulation period and the charge accumulation period of the last pixel row overlap, and the control by the second illumination control unit from the control by the first illumination control unit when I switched to, out of the image sensor based on the illumination of the illumination light by the brightness and the second illumination control section of the image output from the image sensor based on the illumination of the illumination light from the first illumination control unit As the difference between the brightness of the image decreases, the control unit of the imaging apparatus and a brightness correction unit for correcting the brightness of the image output from the image sensor is provided.

According to the 2nd mode of the present invention, an imaging device provided with the control device of the 1st mode mentioned above is provided.

According to the third aspect of the present invention, rolling that controls the charge accumulation period t for each pixel row while sequentially shifting the time from the first pixel row to the last pixel row of the image sensor in which a plurality of pixels are two-dimensionally arranged. An imaging method of an imaging apparatus including a shutter control unit , irradiating an observation sample with illumination light from an illumination unit over the entire period from the start of charge accumulation in the first pixel row to the end of charge accumulation in the last pixel row; Illuminating the observation specimen with illumination light from the illumination unit only during the overlap period in which the charge accumulation period of the first pixel row and the charge accumulation period of the last pixel row overlap, and illumination light from the illumination light irradiation over the entire period only during the overlap period when switched to the radiation, or the image sensor based on the illumination of the illumination light by brightness and the overlap period of the image output from the image sensor based on the illumination of the illumination light over the entire period As the difference between the brightness of the outputted image is reduced comprising: a correcting the brightness of the image output from the image sensor.

According to the fourth aspect of the present invention, rolling for controlling the charge accumulation period t for each pixel row while sequentially shifting the time from the first pixel row to the last pixel row of the image sensor in which a plurality of pixels are two-dimensionally arranged. A first illumination process for causing the control device of the imaging device including the shutter control unit to irradiate the observation specimen with illumination light from the illumination unit over the entire period from the start of charge accumulation in the first pixel row to the end of charge accumulation in the last pixel row; The second illumination process for irradiating the observation specimen with illumination light from the illumination unit only during the overlap period in which the charge accumulation period of the first pixel row and the charge accumulation period of the last pixel row overlap, and from the first illumination process to the second illumination process when switched, output from the image sensor based and brightness of the image output from the image sensor based on the illumination of the illumination light to the first illumination process, the irradiation of the illumination light from the second illumination process As the difference between the brightness of the image decreases, imaging program for executing a brightness correction process for correcting the brightness of the image output from the image sensor is provided.

  According to the fourth aspect of the present invention, rolling for controlling the charge accumulation period t for each pixel row while sequentially shifting the time from the first pixel row to the last pixel row of the image sensor in which a plurality of pixels are two-dimensionally arranged. A first illumination process for causing the control device of the imaging device including the shutter drive unit to irradiate the observation specimen with illumination light from the illumination unit over the entire period from the start of charge accumulation in the first pixel row to the end of charge accumulation in the last pixel row; The second illumination process for irradiating the observation specimen with illumination light from the illumination unit only during the overlap period in which the charge accumulation period of the first pixel row and the charge accumulation period of the last pixel row overlap, and from the first illumination process to the second illumination process An imaging program is provided that executes brightness correction processing for correcting the brightness of an image output from the image sensor when switching.

  According to the fifth aspect of the present invention, there is provided a computer-readable storage medium in which the imaging program of the fourth aspect described above is recorded.

  According to the sixth aspect of the present invention, there is provided a microscope apparatus equipped with the control device for the imaging apparatus according to the first aspect.

  According to the aspect of the present invention, even when the exposure time of the image sensor is changed, it is possible to acquire an image without feeling uncomfortable with the change in brightness.

It is a functional block block diagram which shows an example of embodiment of the imaging device containing a control apparatus. It is a functional block diagram explaining a brightness correction part. It is a figure explaining the rolling shutter system of an image sensor. It is a figure explaining simultaneous exposure mode. (A) is a figure explaining the all-line exposure mode when the period t and the time difference a are equal, (b) is a figure explaining the all-line exposure mode when the period t <time difference a. It is a figure explaining the outline | summary of the correction process by a brightness correction part. It is a figure explaining the correction process A by a brightness correction part. It is a figure explaining the correction process B by the brightness correction part. It is a figure explaining the correction process C by the brightness correction part. It is a figure showing the characteristic of correction processing A-C. It is a flowchart which shows the procedure which performs an example of the imaging method which concerns on embodiment. It is a flowchart which shows the process sequence corresponding to all the line exposure mode shown in FIG. It is a flowchart which shows the process sequence corresponding to the simultaneous exposure mode shown in FIG. It is a figure which shows the other example of the timing which switches all line exposure mode and simultaneous exposure mode. It is a functional block block diagram which shows an example of the microscope apparatus which concerns on embodiment.

  Hereinafter, although an embodiment is described, referring to drawings, it is not limited to this embodiment. Further, in the drawings, in order to describe the embodiment, the scale is appropriately changed and expressed by partially enlarging or emphasizing the description. Further, in FIG. 15, directions in the drawing may be described using an XYZ coordinate system.

<Imaging device>
FIG. 1 is a functional block configuration diagram illustrating an example of an imaging apparatus including a control device according to the embodiment. As illustrated in FIG. 1, the imaging device 1 includes an imaging unit 10 and a control device 30. The imaging unit 10 includes a CMOS type image sensor 12, an AFE (analog front end) circuit 13, a TG (timing generator) 17, a photographing lens 19, and a lens driving unit 21. As the image sensor 12, for example, a CMOS type image sensor using a CMOS transistor as a signal readout circuit is used. As the CMOS transistor, an nMOS transistor or a pMOS transistor is used. The image sensor 12 is formed by arranging a large number of pixels (photoelectric conversion elements) in a two-dimensional lattice pattern on a semiconductor substrate.

  The pixels arranged in the horizontal direction (horizontal direction) or the vertical direction (vertical direction) of the image sensor 12 are driven in units of pixel rows or pixel columns by a timing pulse signal from a TG 17 described later. That is, from the start of charge accumulation in each pixel row (in the following, the terms exposure start and electronic shutter open are used as appropriate) to the end of charge accumulation (in the following, the terms exposure end and electronic shutter close are used as appropriate). The timing of opening / closing the electronic shutter up to the above is controlled. A pixel row is expressed as a line.

  The AFE circuit 13 is a circuit that performs analog signal processing on the output from the image sensor 12. The AFE 13 performs correlated double sampling processing, image signal gain adjustment processing, and the like, and performs A / D conversion to convert the processed analog signal into a digital signal. The digital captured image signal output from the AFE circuit 13 is input to an image input unit 39 of the control device 30 described later.

  The TG 17 generates a pulse signal for driving the image sensor 12 according to an instruction from the CPU 31 of the control device 30 described later. The CPU 31 and the TG 17 form a rolling shutter drive unit 15 that drives the image sensor 12 by a rolling shutter system.

  The imaging lens 19 captures an image of the observation specimen P that is a subject and forms it on the imaging surface of the image sensor 12, and has a plurality of lenses in the lens barrel. A plurality of lens groups including a zoom lens that adjusts the imaging magnification of the observation specimen P and a focus lens that focuses the observation specimen P are disposed in the lens barrel of the imaging lens 19. FIG. 1 shows a lens barrel that houses a lens group. Although the observation specimen P illustrates a still life, it may be an animal or a fish that moves in addition to the still life. For example, at the time of photographing in a studio, these observation specimens P (arbitrary objects such as people, animals, fish) are illuminated by the illumination unit 23 and the reflected light or the like is captured by the image sensor 12 via the photographing lens 19.

  The lens drive unit 21 includes a motor (not shown) that moves the zoom lens and the focus lens within the lens barrel of the imaging lens 19. The lens driving unit 21 moves one or more focus lenses in the optical axis direction in response to an instruction from an AF adjustment unit 35 of the control device 30 described later.

  As shown in FIG. 1, the control device 30 includes a CPU (central processing unit) 31, a main memory (work memory) 33, an AF (automatic focus) adjustment unit 35, a storage device control unit 37, and an image input. A unit 39, a correction unit 41, and a display control unit 50. The CPU 31 and the like are connected to each other by a bus 47.

  The CPU 31 reads an imaging program stored in the main memory 33 or a hard disk (not shown) and performs various processes using the main memory 33. The CPU 31 outputs a control instruction to each unit such as the AF adjustment unit 35 as appropriate after performing various processes. If the imaging program is stored in a storage medium such as an optical disk, CD-ROM, USB memory, or SD card, it is read via an input / output (IO) device (not shown) and the imaging program is executed.

  As the main memory 33, for example, a nonvolatile semiconductor memory or the like is used. The main memory 33 stores image data captured by the image sensor 12 and the like. Further, the main memory 33 temporarily stores, for example, image data in a pre-process and post-process of image processing executed by the CPU 31.

  The AF adjustment unit 35 outputs a focusing instruction to the lens driving unit 21 based on an instruction from the CPU 31. Receiving the focus instruction, the lens driving unit 21 moves the focus lens of the photographing lens 19 to adjust its position, and aligns the observation sample P with the focal position of the photographing lens 19. Note that the focusing is not limited to being performed automatically by the AF adjustment unit 35, and the user may manually operate the focus lens of the photographing lens 19. The imaging magnification of the observation specimen P by the photographic lens 19 is adjusted by operating a zoom button (not shown) or manually by operating the zoom lens of the photographic lens 19.

  The storage device control unit 37 is connected to an external storage device 49 such as a hard disk. When the storage instruction is input from the user, the storage device control unit 37 stores the captured image of the observation specimen P in the external storage device 49 based on the storage instruction from the CPU 31.

  The image input unit 39 takes in the image signal output from the AFE circuit 13. The captured image signal is processed by the CPU 31 based on a predetermined processing program. The CPU 31 performs, for example, color interpolation processing, gradation conversion processing, contour enhancement processing, white balance adjustment, and the like on the digital image signal for one frame. Note that the CPU 31 determines a high-frequency component, contrast, and the like included in the image based on the processed image data, and outputs a control instruction to the AF adjustment unit 35. As described above, the AF adjustment unit 35 aligns the observation sample P with the focal position of the photographing lens 19 based on this control instruction.

  The correction unit 41 includes a first illumination control unit 42, a second illumination control unit 43, and a brightness correction control unit 44. The correction unit 41 is realized by the CPU 31 executing an imaging program. However, the correction unit 41 may be realized as hardware such as an electronic circuit instead of being realized by executing software. The first illumination control unit 42 controls to irradiate the observation specimen P with illumination light from the illumination unit 23 over the entire period from the start of charge accumulation in the first pixel row to the end of charge accumulation in the last pixel row (hereinafter, all line exposure). Mode). The second illumination control unit 43 controls to irradiate the observation specimen P with illumination light from the illumination unit 23 during an overlap period in which the charge accumulation period of the first pixel row and the charge accumulation period of the last pixel row overlap (hereinafter, simultaneous exposure mode). ).

  FIG. 2 is a functional block diagram illustrating the brightness correction unit 44. As shown in FIG. 2, the brightness correction unit 44 includes an irradiation time adjustment unit 44a, a gain adjustment unit 44b, and a light amount adjustment unit 44c. In the present embodiment, the brightness correction unit 44 is controlled to be executed in the simultaneous exposure mode. That is, the CPU 31 causes the correction unit 41 to execute any one of the irradiation time adjustment unit 44a and the like of the brightness correction unit 44 when the second illumination control unit 43 performs control.

  The irradiation time adjustment unit 44a adjusts the irradiation time of the illumination light from the illumination unit 23 and adjusts the output to the TG 17 so as to adjust the overlap period in the simultaneous exposure mode in synchronization with the irradiation time. The gain adjustment unit 44 b performs gain control of the AFE circuit 13 on the image signal output from the image sensor 12. The light amount adjustment unit 44 c adjusts the light amount (intensity) of illumination light from the illumination unit 23.

  The correction unit 41 includes a switching unit (not shown) for switching between the irradiation time adjustment unit 44a, the gain adjustment unit 44b, and the light amount adjustment unit 44c of the brightness correction unit 44. The CPU 31 instructs the switching unit of the correction unit 41 to switch one of the irradiation time adjustment unit 44 a and the like of the brightness correction unit 44. The switching unit switches and executes any one of the irradiation time adjustment unit 44a and the like based on this instruction. Note that the switching unit may be instructed manually by the user in addition to the case where it is automatically performed in accordance with the shooting mode (for example, shooting for storage or shooting for live observation). Good.

  Returning to FIG. 1, the display control unit 50 is connected to a monitor 51 such as a liquid crystal display device. The display control unit 50 displays an image (still image or moving image) of the observation sample P captured from the image sensor 12 on the monitor 51 in accordance with a display instruction from the CPU 31. The user can perform various operations while viewing this image. The display control unit 50 may include, for example, a DSP (digital signal processor) that performs dedicated image processing separately from the CPU 31. This DSP is connected to the bus 47. By using a DSP, the burden on the CPU 31 can be reduced.

  The input device 45 is connected to the bus 47 of the control device 30 and receives various input instructions from the user. Examples of the input device 45 include a keyboard, a pointing device such as a mouse, a touch panel, and a joystick. The touch panel is formed on the monitor 51, and performs input by touching the displayed screen. Further, as the input device 45, a user's instruction may be input by a microphone, and various instructions may be input by causing the voice data to be recognized.

  As instructions from the user, there are t input instructions, application input instructions, image storage instructions, and the like. The t input instruction sets a period t from exposure start (electronic shutter open) to exposure end (electronic shutter close). The application input instruction sets various shooting modes such as still image shooting, moving image shooting, live observation for imaging magnification and focus adjustment, and the like. The image storage instruction causes the external storage device 49 to store the image of the observation specimen P captured by the imaging unit 10.

  The illumination unit 23 is connected to a power source (not shown), and irradiates illumination light toward the observation specimen P according to an illumination instruction from the correction unit 41. As the illumination unit 23, various light sources may be used in addition to illumination using an LED (Light Emitting Diode) light source or a laser light source. According to the LED light source and the laser light source, it is possible to finely control the irradiation time of the illumination light and the intensity of the illumination light. When other light sources are used, the irradiation time and intensity may be controlled using a light valve such as a liquid crystal device.

  In FIG. 1, the imaging unit 10 and the control device 30 are illustrated as separate devices, but a configuration of a digital camera in which these are integrated may be employed. In this case, the monitor 51 corresponds to a rear surface of the camera body or a liquid crystal display unit having a vari-angle configuration. Moreover, although the illumination part 23 is represented as a separate body from the imaging part 10 etc. in FIG. 1, it may be attached to or built in the camera body. The monitor 51 is not limited to being connected to the display control unit 50 by wire, and may be connected wirelessly.

  FIG. 3 is a diagram for explaining the rolling shutter system of the image sensor 12. As shown in FIG. 3, the image sensor 12 has pixels arranged in a two-dimensional grid, and is driven and controlled in order for each pixel row in the horizontal direction (lateral direction). In each pixel row, an exposure time (charge accumulation time) from the opening of the electronic shutter to the closing of the electronic shutter is set to a period t. First, exposure is started simultaneously for each pixel in the first pixel row (first pixel row). Next, exposure is started at the same time for each pixel in the second pixel row with a slight delay from the start of exposure in the first pixel row. Thereafter, exposure is started while shifting the time for each pixel row in order.

  Each pixel in the pixel row is exposed for a required time in the period t, and the detection signal is read out of the image sensor 12 by the signal readout circuit in order from the pixel row after the exposure is completed. FIG. 3 shows the time lag from the start of exposure (electronic shutter open, charge accumulation start) to the end of exposure (electronic shutter close, charge accumulation end) for each pixel row.

  If the deviation of the exposure start timing between the pixel rows is, for example, 10 μsec and the number of pixel rows is 1000, a time difference a of 10 msec occurs between the first pixel row and the last pixel row. As a result, when an object that moves at high speed is imaged with a rolling shutter, simultaneity cannot be secured between the captured image of the first pixel row and the captured image of the last pixel row, and the image is distorted in an oblique direction. Therefore, as shown in FIG. 3, the illuminating light is irradiated to the observation specimen P only during the overlapping period in which all the pixel rows are simultaneously open with the electronic shutter, thereby ensuring the simultaneity of the captured images for all the pixel rows. It becomes possible. Note that the overlap period in which all the pixel rows are simultaneously opened with the electronic shutter is a period t-time difference a.

  In this specification, the first pixel row and the last pixel row refer to the first pixel row and the last pixel row on the entire light receiving surface when the entire light receiving surface (effective pixel region) of the image sensor 12 is used as an imaging region. is there. However, when only a part of the light receiving surface (for example, the central region) is used as the imaging region instead of the entire light receiving surface of the image sensor 12, the first pixel row and the last pixel row in a part of the light receiving surface are indicated. It is. Similarly, the above-described time difference “a” is different between the time of image capturing using the entire light receiving surface of the image sensor 12 and the time of image capturing using only a part.

  The time difference a is determined depending on the type of the image sensor 12. For example, the time difference a is determined by the difference in the number of pixel rows of the image sensor and the exposure start time difference between the first pixel row and the second pixel row that is the next row of the first pixel row. is there. Therefore, when two image sensors are compared, if the difference in exposure start time between the two is the same, the time difference a is shorter in an image sensor with fewer pixel rows. In this embodiment, the time difference a is stored in advance in the storage unit, or means such as inputting the time difference a determined by the image sensor used by the user is used.

  FIG. 4 shows a state in which the simultaneity of captured images is ensured for all the pixel rows by irradiating illumination light during the above overlapping period. That is, a state is shown in which the observation specimen P is imaged in the simultaneous exposure mode in which illumination light is irradiated during the overlap period. The simultaneous exposure mode (second illumination process) is controlled by the second illumination control unit 43 of the correction unit 41. In this simultaneous exposure mode, the observation specimen P is imaged simultaneously from the first pixel row to the last pixel row, and the simultaneity of the captured images is ensured.

  However, the image can be captured in the simultaneous exposure mode when the period t from the electronic shutter opening to the electronic shutter closing in each pixel row is longer than the time difference a between the first pixel row and the last pixel row, as shown in FIG. Limited to. That is, when the period t ≦ time difference a, there is no period in which the exposure time overlaps at the same time in all pixel rows.

  FIG. 5A shows a case where the period t and the time difference a are t = a. As shown in FIG. 5A, since there is no overlapping period of the pixel rows, the observation specimen P is irradiated with illumination light from the timing of opening the electronic shutter of the first pixel row to the timing of closing the electronic shutter of the last pixel row. I'm shooting. That is, the observation specimen P is imaged in the all-line exposure mode controlled by the first illumination control unit 42 of the correction unit 41 (first illumination process). As a result, each pixel row from the first pixel row to the last pixel row is irradiated with illumination light over a period t, so that there is no difference in charge accumulation time in each pixel row.

  FIG. 5B shows a case where the period t <time difference a. As shown in FIG. 5B, as in FIG. 5A, since there is no overlapping period of pixel rows, the observation specimen P is imaged in the all-line exposure mode controlled by the first illumination control unit.

  When imaging the observation specimen P, the user performs a procedure of setting an optimal exposure time while changing the exposure time. In this case, for example, if the exposure time is set to be short as shown in FIG. 5B, the observation specimen P is illuminated in the all-line exposure mode. An image is acquired. However, when the exposure time is further increased, the simultaneous exposure mode is entered when the period t becomes longer than the time difference a, that is, when it exceeds the state shown in FIG. The mode switches to the illumination light only.

  FIG. 6 shows a change in image brightness when the period t from the opening of the electronic shutter to the closing of the electronic shutter, that is, the exposure time is changed. In FIG. 6, the horizontal axis represents the period t, and the vertical axis represents the brightness of the image. As shown in FIG. 6, when the period t is lengthened sequentially from the state where the period t is 0, the brightness of the image sequentially increases in the all-line exposure mode. However, since the all-line exposure mode is switched to the simultaneous exposure mode when the period t exceeds the time difference a, the image brightness once becomes 0, and when the exposure time is further increased, the image brightness increases from 0 again. .

  In the all-line exposure mode, all the pixel rows receive illumination light in the period t, whereas in the simultaneous exposure mode, the illumination light is received in a part of the period t (period ta). ing. In other words, in the all-line exposure mode and the simultaneous exposure mode, the illumination light irradiation time differs with respect to the period t which is the exposure time. As a result, discontinuity (step difference at t = a) occurs in the image brightness.

  As described above, even when the exposure time is gradually increased, the image is once darkened, and then the image is successively brightened again. By the way, the user places the observation specimen P in the field of view of the photographing lens 19 and inputs an instruction to gradually increase the exposure time (period t) from 0, or conversely, gradually decrease from the maximum exposure amount. Thus, it is usual to search for the optimum exposure time while viewing the monitor 51 so that the brightness of the image to be captured is the optimum brightness. At this time, if the brightness of the image does not change seamlessly when adjusting the exposure amount, it causes not only a sense of incongruity but also a difficulty in adjustment. In addition, as shown in FIG. 6, since there are two modes of an all-line exposure mode and a simultaneous exposure mode with the same exposure amount (brightness of the image), when a certain exposure amount is set, the control device There is a possibility that 30 cannot determine which mode should be selected.

  In the present embodiment, as shown in FIG. 6, when the image is captured in the simultaneous exposure mode, the brightness correction unit 44 corrects the discontinuity in the brightness of the image so that the brightness of the image becomes brighter. To do. In FIG. 6, the change in the brightness of the image is corrected so as to have the same inclination in the all-line exposure mode and the simultaneous exposure mode. However, the present invention is not limited to this. The brightness of the image in the simultaneous exposure mode may be corrected. The brightness of the corrected image in the simultaneous exposure mode is not limited to having a linear inclination, and may be a curve that changes. In addition, when the period t exceeds the time difference a, the entire line exposure mode is switched to the simultaneous exposure mode. The brightness of the image in the simultaneous exposure mode immediately after switching is the image acquired in the all line exposure mode immediately before switching. It is not indispensable to make it completely coincide with the brightness, and it is sufficient that the obtained image is approximated so as not to feel uncomfortable when it is displayed on the monitor. The degree of approximation may be changed depending on the image display performance and setting difference of the monitor.

  Three examples of correction processing A, correction processing B, and correction processing C will be described as examples of correction processing performed by the brightness correction unit 44. However, the correction processing performed by the brightness correction unit 44 is not limited to these three. Switching between the correction processes A to C is performed by the switching unit of the correction unit 41 as described above.

[Correction A]
FIG. 7 is a diagram for explaining the correction process A by the irradiation time adjustment unit 44 a of the brightness correction unit 44. As described with reference to FIG. 6, the image brightness becomes discontinuous between the all-line exposure mode and the simultaneous exposure mode in the simultaneous exposure mode, and the actual exposure period is shorter than the period t by ta. Explained that this is the reason. Therefore, in the correction process A, when the user instructs a period t in which t> a, the irradiation time adjusting unit 44a extends the period ta and the illumination light illuminates the observation specimen P. In addition, control is performed so that each pixel accumulates electric charge during the extended period. In the correction process A, the intensity of irradiation light is the same as that in the all-line exposure mode, and the illumination time (irradiation time) is extended.

  When the correction processing A is selected, the irradiation time adjustment unit 44a extends the period t, and as shown in FIG. 7, illumination is performed over a period from the exposure start timing of the last pixel row to the exposure end timing of the first pixel row. The observation specimen P is irradiated with light. In FIG. 7, a period a similar to the time difference a is adopted as the extension time. By extending the time difference a, each pixel row accumulates charge in a period of t + a. Since the illumination light is irradiated in the period t of the period, the interval between the all-line exposure mode and the simultaneous exposure mode is obtained. This eliminates the discontinuity in image brightness. That is, the actual exposure period in the simultaneous exposure mode is (t + a) −a = t, and there is no step (discontinuity) in image brightness at t = a.

  In the correction process A, the correction unit 44 having the irradiation time adjustment unit 44a outputs a control command to the TG 17 of the imaging unit 10, and adds the period a to the period t instructed by the CPU 31 to thereby charge the pixel rows. Extend the accumulation time. At the same time, the correction unit 44 controls the illumination unit 23 to irradiate illumination light by adding the period a to the period t. Note that the CPU 31 may incorporate the processing function of the correction unit 44 and directly instruct the TG 17 of the period of t + a.

  The period extended by the irradiation time adjusting unit 44a is not limited to a, and may be set shorter or longer than the time difference a. However, if the extended period is shorter or longer than the time difference a, a slight level difference is caused in the image brightness with respect to the all-line exposure mode, but the size of the level difference is smaller than in the case where the correction is not performed, Reduces the discomfort and difficulty of adjustment for the user.

[Correction process B]
FIG. 8 is a diagram for explaining the correction process B by the gain adjustment unit 44b of the brightness correction unit 44. The discontinuity of the image brightness at t = a shown in FIG. 6 is that the actual illumination light irradiation time is shortened to ta with respect to the period t, and the signal amount of the captured image is the signal in the original period t. This is due to a drop from the amount. Therefore, in the correction process B, control is performed so that the gain adjustment unit 44b multiplies the signal amount based on a predetermined gain with respect to the signal amount of the captured image obtained in the period t. In the correction process B, the intensity of irradiation light is the same as that in the all-line exposure mode, and the charge accumulation period t remains the normal period t in the simultaneous exposure mode.

  When the correction process B is selected, the gain adjustment unit 44b, as shown in FIG. 8, AFE so as to multiply the signal amount output from each pixel row by a gain of t / (ta). An instruction is output to the circuit 13. The gain of t / (ta) is a value that increases the amount of decrease in the illumination light irradiation time by the time difference a in the period t. The signal amount is amplified by this gain, and discontinuity in image brightness is eliminated between the all-line exposure mode and the simultaneous exposure mode.

  However, it is not limited to using the time difference a as t / (ta) as the gain, and any numerical value can be used. However, when the gain amount is smaller or larger than t / (ta), a slight level difference is caused in the image brightness with respect to the all-line exposure mode, but the size of the level difference is not compared with the case where the level difference is not corrected. This reduces the uncomfortable feeling and difficulty of adjustment for the user. In the above description, the AFE circuit 13 multiplies the gain of the analog captured image signal. However, the present invention is not limited to this, and the digital captured image signal captured by the control device 30 may be multiplied by the gain.

[Correction process C]
FIG. 9 is a diagram for explaining the correction process C by the light amount adjustment unit 44 c of the brightness correction unit 44. As described above, the discontinuity of the image brightness at t = a shown in FIG. 6 is caused by a decrease in the exposure amount in the original period t. Therefore, in the correction process C, the light amount adjusting unit 44c controls to increase the intensity of the illumination light from the irradiation unit 23 at a predetermined rate and increase the brightness of the observation specimen P. In the correction process C, the charge accumulation period t in each pixel row remains the normal period t in the simultaneous exposure mode.

  When the correction process C is selected, the light amount adjustment unit 44c instructs the illumination unit 23 to increase the intensity of the illumination light to be irradiated to t / (ta) times as shown in FIG. Is output. t / (ta) times is a value that increases the amount of decrease in the irradiation time of the illumination light by the time difference a in the period t. Due to the increase in the intensity of the illumination light, the brightness of the image of the observation specimen P captured by each pixel is t / (ta) times, and the image brightness is discontinuous between the all-line exposure mode and the simultaneous exposure mode. Will be eliminated.

  However, it is not limited to using the time difference a such as t / (ta) times as a ratio by the light amount adjusting unit 44, and any numerical value can be used. However, when the ratio is smaller or larger than t / (ta) times, a slight level difference is caused in the image brightness with the all-line exposure mode, but the size of the level difference is not corrected. This reduces the uncomfortable feeling and difficulty of adjustment for the user.

  For example, in the case of an LED light source or a laser light source, the intensity of the illumination light emitted from the illumination unit 23 is changed by changing a voltage to be applied. Moreover, the intensity | strength of illumination light may be adjusted by using the filter which can change the transmittance | permeability of illumination light, and changing the transmittance | permeability of this filter suitably. Further, the intensity of the illumination light that illuminates the observation specimen P may be adjusted by arranging a plurality of light sources of the illumination unit 23 and changing the number of light sources that irradiate the illumination light.

  FIG. 10 is a diagram showing the characteristics of the correction processes A to C. FIG. Each of the correction processes A, B, and C described above has characteristics. Based on the characteristics of these correction processes A, B, and C, a correction process corresponding to the application of the imaging apparatus 1 may be selected. As feature points, as shown in FIG. 10, the frame rate, noise, the influence on the observation specimen P, the necessity of calibration on the light source side, and further examples of applications of each correction process are described. .

  Regarding the frame rate, the correction processing A extends the charge accumulation period t in each pixel row to t + a, and extends the illumination light irradiation time to increase the actual exposure time. Accordingly, the frame rate of imaging by the image sensor 12 is lowered. On the other hand, the correction process B and the correction process C use gain amplification and increase in intensity of illumination light, and therefore do not decrease the frame rate.

  Thereby, it can be said that the correction process A is unsuitable when acquiring a moving image. Further, in imaging, there is an imaging mode for live observation in which the observation specimen P is displayed on the monitor 51 while imaging. In the live observation, for example, the direction of the photographing lens 19 and the position of the observation specimen P are adjusted so that the moving observation specimen P or the changing observation specimen P is within the field of view of the photographing lens 19 while watching the live video on the monitor 51. It is also used when focusing on the observation specimen P. If the frame rate is low during this live observation, the image becomes rough, which is disadvantageous when various adjustments are made.

  Therefore, it can be said that the correction processing A is not suitable for live observation because the frame rate is low. On the other hand, in the correction process B and the correction process C, the frame rate does not decrease, so that the moving image does not become rough, and there is no disadvantage with respect to the frame rate at the time of moving image acquisition or live observation.

  Regarding the noise, since the correction process A extends the exposure time (period t), the ratio of the noise N included in the image signal to the signal amount S does not increase, and there is no deterioration in S / N. Further, since the correction process C also increases the intensity of the irradiation light from the irradiation unit 23, similarly, the ratio of the noise N included in the image signal to the signal amount S does not increase, and the S / N does not deteriorate. On the other hand, since the correction processing B multiplies the image signal by gain, noise included in the image signal is also amplified, and the noise component of the gain amplifier is superimposed to reduce the S / N. The image contains a lot.

  Therefore, the correction process A and the correction process C do not become disadvantageous with respect to noise when a high-quality image is stored. On the other hand, the correction process B is disadvantageous when storing a high-quality still image or moving image because the image includes a lot of noise.

  Regarding the influence on the observation sample P, the correction process A extends the irradiation time of the illumination light, so that the time for irradiating the observation sample P with the illumination light becomes long and the adverse effect and damage to the observation sample P are increased. There is. In particular, when the observation specimen P is a living body, it is also necessary to reduce damage to the observation specimen P during live observation before imaging. In addition, since the correction process C irradiates the observation specimen P with the intensity of illumination light being increased, there are cases where the observation specimen P is similarly adversely affected or damaged. On the other hand, the correction process B multiplies the imaging signal output from the image sensor 12 by gain, and therefore does not change the illumination light applied to the observation specimen P. Will not increase.

  Therefore, since the correction process A and the correction process C are applied in a state where the illumination light applied to the observation specimen P is increased from normal (time extension or intensity increase), adverse effects and damages to the observation specimen P are increased. In some cases, it is disadvantageous for live observation in imaging. On the other hand, the correction process B does not increase the adverse effect or damage to the observation specimen P, and thus does not become disadvantageous in the imaging mode in live observation.

  Regarding the calibration on the light source side, the correction process A and the correction process B do not change the intensity of the illumination light from the illumination unit 23, and therefore it is not necessary to perform calibration on the light source side. On the other hand, since the correction process C multiplies the intensity of the irradiation light by t / (ta), the light emission intensity of the illumination unit 23 is calibrated so that the light emission intensity can be corrected with high accuracy. It is essential. The light source side calibration may be performed periodically thereafter, for example, when the product is shipped at the time of shipment.

  Therefore, the correction process A and the correction process B do not require calibration on the light source side, do not increase the cost due to calibration on the light source side, and are not disadvantageous in a frequently used mode such as live observation. . On the other hand, the correction process C is disadvantageous in terms of cost because calibration on the light source side is essential.

  When the characteristics of the correction processes A, B, and C as described above are combined, as shown in the application of FIG. 6, the correction process A is advantageous in an application for storing a high-quality still image. Is advantageous for live observation in which the imaging lens 19 is focused on the observation specimen P while observing the live video displayed on the monitor 51, and the correction process C is performed by performing calibration on the light source side. When the illumination part 23 can be prepared, it is advantageous for the use which preserve | saves a moving image.

  The correction processes A, B, and C described above are not limited to being performed independently, and may be used in combination. For example, the correction process A and the correction process B or the correction process C may be combined, the correction process B and the correction process C may be combined, or the correction processes A to C may be combined. In this case, the features of the correction processes A to C shown in FIG. 10 are combined as appropriate.

  Further, the correction process provided in the brightness correction unit 44 is not limited to the correction processes A to C described above, and other correction processes are used as long as the brightness of the image is increased in the simultaneous exposure mode. May be. In addition, the brightness correction unit 44 is not limited to executing the above-described three correction processes A to C as the correction process, and one or two of the three correction processes A to C are performed. You can do it.

<Imaging method>
11 to 13 are flowcharts illustrating an example of an imaging method according to the embodiment. FIGS. 11 to 13 show the processing procedure of the imaging program that switches the correction processes A to C and corrects the brightness of the image in the simultaneous exposure mode. The imaging program is executed by the CPU 31 shown in FIG. 1 by reading it from the built-in hard disk or CD ROM into the main memory 33.

  As shown in FIG. 11, first, it is determined whether or not an instruction to end this imaging program is input from the user (step S01). If there is an end instruction from the user (step S01: YES), the CPU 31 ends the imaging program. When there is no end instruction from the user (step S01: NO), the CPU 31 takes in various inputs (period t input instruction, application input instruction) input from the input device 45 by the user (step S02). Subsequently, the CPU 31 compares the period t input from the user with a time difference a between the charge accumulation start time of the first pixel row and the charge accumulation start time of the last pixel row, and whether t> a is satisfied. Is determined (step S03).

  When the determination result is t ≦ a (step S03: NO), the CPU 31 sets the photographing mode to the all-line exposure mode (step S04). Subsequently, after performing the process corresponding to the all-line exposure mode (step S05), the process returns to step S01. When the determination result is t> a (step S03: YES), the CPU 31 sets the photographing mode to the simultaneous exposure mode (step S06). Subsequently, after performing the process corresponding to the simultaneous exposure mode (step S07), the process returns to step S01.

  FIG. 12 is a flowchart showing a detailed processing procedure for processing corresponding to the all-line exposure mode (see FIG. 5) in step S05 shown in FIG. As shown in FIG. 12, first, the CPU 31 operates the photographing lens 19 according to an instruction from the AF adjustment unit 35 to perform focus adjustment on the observation sample P (step S51). Subsequently, based on an instruction from the first illumination control unit 42, the CPU 31 controls the charge accumulation period t in each pixel row to a value input by the user (step S52).

  Subsequently, the CPU 31 (rolling shutter drive unit 15) drives the image sensor 12 based on the period t set in step S52, and displays an image captured by the image sensor 12 on the monitor 51 (step S53). Even when the application is input from the user in step S02 described above, the period t and the intensity of illumination light from the illumination unit 23 are not changed in step S53.

  Subsequently, the CPU 31 determines whether or not the user has issued an image saving instruction from the input device 45 (step S54). If there is an image storage instruction (step S54: YES), the CPU 31 stores the captured image in the external storage device 49 (step S55), and returns to step S01 in FIG. Also, when there is no save instruction (step S54: NO), the process returns to step S01 in FIG. In step S54, an instruction to save the image may be performed using a shutter button or the like as the input device 45.

  Thus, when the value of the period t input from the user is equal to or less than the time difference a, the operation from step S01 to step S54 is repeated each time the value of the period t is changed from the user. Thus, the image whose focus is adjusted and whose brightness is adjusted is displayed on the monitor 51.

  FIG. 13 is a flowchart showing a detailed processing procedure for processing corresponding to the simultaneous exposure mode (see FIG. 4) in step S07 shown in FIG. When the period t input by the user becomes t> a, the processing procedure shown in FIG. 11 proceeds to step S06 and step S07, and enters the simultaneous exposure mode processing shown in FIG. As shown in FIG. 13, first, the CPU 31 determines whether or not the application input from the user by the input device 45 is an application for image storage (step S71). That is, the CPU 31 acquires a high-quality still image and determines whether or not the captured image is stored in the external storage device 49.

  When the application is not an image storage application (step S71: NO), the CPU 31 determines that the input application is to put the observation specimen P in the field of view of the photographing lens 19, adjust the focus position of the observation specimen P, It is determined whether or not the purpose is to adjust the period t, such as adjusting brightness, that is, whether or not the input application is live observation for adjustment (step S72).

  When the application is live observation (step S72: YES), the CPU 31 selects the correction process B (see FIG. 8) as the brightness correction process (step S73). Subsequently, the CPU 31 operates the photographing lens 19 according to an instruction from the AF adjustment unit 35 to perform focus adjustment on the observation specimen P (step S74). Subsequently, the CPU 31 outputs an imaging signal from the image sensor 12 based on an instruction from the second illumination control unit 43, and uses the imaging signal based on an instruction from the gain adjustment unit 44b of the brightness correction unit 44. The gain correction of t / (ta) is performed according to the period t input by the person (step S75).

  Subsequently, the CPU 31 displays the image after gain correction on the monitor 51 (step S76), and returns to step S01 in FIG. When the period t is longer than the time difference a, the above-described operations from step S72 to step S76 are repeated each time the user changes the period t, unless the use input by the user is image storage.

  If the application is not live observation (step S72: NO), the CPU 31 determines that the application has not been input by the user, and displays a warning message for prompting input of the application on the monitor 51 (step S77). Return to step S01.

  When the application is image storage (step S71: YES), the CPU 31 determines whether the storage mode input by the user is a moving image or a still image (step S81). Note that when the user gives an image saving instruction using the input device 45, an instruction to record one or both of a still image and a moving image can be given. When the storage form is not a moving image (step S81: NO), the CPU 31 selects the correction process A (see FIG. 7) as the brightness correction process (step S82).

  Subsequently, the CPU 31 controls the TG 17 to extend the period t to t + a, and sets the illumination time of the illumination unit 23 in the normal exposure mode based on an instruction from the irradiation time adjustment unit 44a of the brightness correction unit 44. Is extended from the illumination time by time a (step S83). In this control, the CPU 31 outputs an image signal from the image sensor 12, and the image is displayed on the monitor 51 (step S84). Subsequently, the CPU 31 determines whether or not the user has input image saving using the input device 45 (step S85). In step S85, an instruction to save the image may be performed using a shutter button or the like as the input device 45.

  If there is an input for image storage (step S85: YES), the CPU 31 stores the captured still image (image) in the external storage device 49 (step S86) and returns to step S01 in FIG. If there is no image storage input (step S85: NO), the process returns to step S01 in FIG. When the still image is stored in step S86, the period t, the focus, the imaging magnification, and the like adjusted in the live observation from step S72 to step S74 may be used as they are.

  When the storage form is a moving image (step S81: YES), the CPU 31 selects the correction process C (see FIG. 9) as the brightness correction process (step S91). Subsequently, the CPU 31 controls the illumination unit 23 to set the intensity of illumination light to t / (ta) times (step S92). In this control, the CPU 31 outputs an image signal from the image sensor 12 to generate a moving image, and the moving image is displayed on the monitor 51 (step S84). Step S84 and subsequent steps are as described above. Note that a recording start button or the like may be used instead of the still image shutter button as the input device 45 for storing a moving image.

  Further, in step S54 shown in FIG. 12 and step S85 shown in FIG. 13, it has been described that the shutter button or the like is used as an image storage input. However, the image captured by pressing the shutter button is temporarily displayed on the monitor 51 and used. An image may be stored in the external storage device 49 by performing a predetermined input operation after the person sees and confirms the monitor 51.

  In the flowcharts shown in FIGS. 11 to 13, the procedure for switching the three correction processes A to C has been described. However, the procedure is not limited to this procedure. A part or all of the switching procedure of the correction processes A to C may be performed manually. Further, the present invention is not limited to the use of three correction processes. For example, the processes of steps S81 to S92 relating to the correction process C are omitted, and the two correction processes A and B are switched. But you can.

  In the above-described embodiment, the mode in which the user appropriately inputs the period t using the input device 45 has been described. However, a mode in which the period t is automatically increased or decreased may be provided. In this case, when the user approaches the brightness of the desired image while viewing the monitor 51, the user may stop the automatic increase or automatic decrease according to the user's instruction. Further, after the automatic increase or the like is stopped, the user may be able to operate the input device 45 so that the period t can be finely adjusted.

  Further, in the above-described embodiment, as shown in FIG. 6, using the time difference a between the start of charge accumulation in the first pixel row and the start of charge accumulation in the last pixel row, the all-line exposure mode at the timing of the period t = a. And the simultaneous exposure mode are switched. However, this switching timing is not limited to the period t = a, and can be arbitrarily set within the range of t> a. That is, the all-line exposure mode may be executed when the period t is t ≦ na (n ≧ 1), and the simultaneous exposure mode may be executed when the period t is t> na (n ≧ 1).

  In the above-described embodiment, in the all-line exposure mode and the simultaneous exposure mode, it is necessary to control lighting and extinguishing of the illumination unit 23 in a time shorter than the time difference a by the rolling shutter drive. This is easily realized by using an LED light source or a laser light source, but when using another light source, there is a possibility that this response time cannot be followed. In such a case, as described above, by setting the switching timing between the all-line exposure mode and the simultaneous exposure mode to t> a, the lighting and extinguishing time becomes longer in the simultaneous exposure mode, and the followability of lighting and extinguishing Can be improved.

  FIG. 14 is a diagram illustrating another example of timing for switching between the all-line exposure mode and the simultaneous exposure mode. As shown in FIG. 14, the all-line exposure mode and the simultaneous exposure mode are switched at the timing of t = 2a. In the case shown in FIG. 14, even when overlapping periods occur in the first pixel row and the last pixel row, the all-line exposure mode is maintained until the period t reaches 2a. Thereby, in the simultaneous exposure mode, the shortest period of illumination light is a, and it is not necessary to turn on and off in a time shorter than a. Accordingly, the followability of the illumination unit 23 in the simultaneous exposure mode can be improved.

  The timing for switching between the all-line exposure mode and the simultaneous exposure mode is not limited to the period t = 2a, and can be arbitrarily set such as a period t = 1.5a or a period t = 3a. Further, the user may be allowed to input n at the switching timing t = na to the input device 45.

  In the present embodiment, the user appropriately inputs the application (shooting mode) using the input device 45, but the present invention is not limited to this. For example, when the imaging apparatus 1 is initially turned on or when a use is not input from the user, it may be set to be in a live observation shooting mode by default. Even when an image saving instruction is input as an application, for example, when the user frequently performs imaging magnification or focus adjustment, or when the period t is changed, the CPU 31 displays this live. You may make it discriminate | determine from observation and switch a use to live observation automatically.

  As described above, according to the imaging device 1 or the imaging method of the present embodiment, the brightness of the image output from the image sensor is increased by the brightness correction unit. Even when the image of the observation specimen P is acquired while changing the brightness, there is no step in the brightness change of the image, and the brightness of the image can be changed seamlessly. Thus, the user can easily adjust the brightness without feeling uncomfortable in adjusting the brightness of the image.

  In addition, since the brightness correction unit 44 includes an irradiation time adjustment unit 44a that extends the period t and extends the irradiation time of the illumination light, a high-quality still image can be taken while increasing the brightness of the image. It becomes. In addition, since the brightness correction unit 44 includes the gain adjustment unit 44b that amplifies the output signal of the image sensor 12, the brightness of the image can be increased while reducing damage to the observation specimen P. In addition, since the brightness correction unit 44 includes a light amount adjustment unit 44c that adjusts the amount of illumination light, the brightness of the image can be increased without reducing the frame rate.

  In addition, the brightness correction unit 44 includes different correction processes A to C, and includes a switching unit that switches the correction process according to the purpose of imaging by the image sensor 12, so that an appropriate correction process according to the application is performed. Can be easily and reliably performed. Moreover, since the use of imaging includes the use of storing an image and the use of observing the observation specimen P, it is possible to appropriately perform correction processing according to the use of storage of the image and observation of the observation specimen P. it can.

  Further, the first illumination control unit irradiates illumination light over the entire period when the period t ≦ a, and the second illumination control unit irradiates illumination light only during the overlapping period when the period t> a, The all-line exposure mode and the simultaneous exposure mode can be surely executed. Further, the first illumination control unit irradiates illumination light over the entire period when the period t ≦ 2a, and the second illumination control unit irradiates illumination light only during the overlap period when the period t> 2a. In the simultaneous exposure mode, lighting and extinguishing of the illumination unit can be avoided in a short time, and the followability of illumination light can be improved. Since the illumination unit controlled by the first illumination control unit, the second illumination control unit, and the brightness correction unit is included, the illumination light can be easily controlled without being connected to an external illumination device or the like. it can.

  In addition, even when the control device 30 performs imaging in the continuous AE mode in which the period t is continuously changed so that the exposure (exposure amount) is adjusted with respect to the moving observation specimen P or the changing observation specimen P, the above-described operation is performed. Image brightness correction processing may be applied.

  Also, when performing high dynamic range (HDR) shooting, the above-described image brightness correction processing may be applied. High dynamic range imaging is a method in which images of a plurality of observation specimens P are acquired at different periods t, and a single image is created by synthesizing images of portions that are not crushed or blown out in each image. Yes, as described above, an appropriate correction process may be performed according to the period t.

<Microscope device>
FIG. 15 is a functional block configuration diagram illustrating an example of a microscope apparatus according to the embodiment. In the following description, the same or equivalent components as those of the above-described embodiment are denoted by the same reference numerals, and description thereof is omitted or simplified. A microscope apparatus MS illustrated in FIG. 15 includes a microscope main body 60 including an imaging unit 10a and a control apparatus 30a. That is, the microscope apparatus MS is equipped with an imaging apparatus including the imaging unit 10a and the control apparatus 30a. In addition, although the structure of the fluorescence microscope is employ | adopted for the microscope main body 60, it is not limited to this.

  The microscope main body 60 includes an XY moving stage 61, an X-axis drive motor 63, a Y-axis drive motor 65, a normal light source (illumination unit) 67 for transmitted illumination, a collimator lens 69, and a condenser lens 71. . The XY moving stage 61 is installed horizontally and is movable on the XY plane. At the center of the XY moving stage 61, an optical path hole 61a penetrating vertically is formed. A plate 73 on which a subject (sample) P is placed is fixed to the upper portion of the optical path hole 61a. The plate 73 may be a culture container that contains cells to be observed together with a culture medium, or may be a mineral sample other than a living body.

  The X-axis drive motor 63 moves the XY movement stage 61 in the X direction according to an instruction from an XY position adjustment unit 55 of the control device 30a described later. Similarly, the Y-axis drive motor 65 moves the XY movement stage 61 in the Y direction according to an instruction from the XY position adjustment unit 55. Note that the XY moving stage 61 may be moved in the Z direction by a motor (not shown) or the like. As the normal light source 67, for example, a white light source is used. The illumination light emitted from the normal light source 67 illuminates the observation specimen P on the plate 73 via the collimator lens 69 and the condenser lens 71. The illumination time and intensity of illumination light from the normal light source 67 are controlled by an instruction from the control device 30a.

  The microscope main body 60 includes an objective optical system 80 and an imaging unit 10a. The objective optical system 80 and the imaging unit 11 are arranged in series along the optical axis AX of illumination light from the normal light source 67 below the XY moving stage 61.

  The objective optical system 80 includes a first objective lens 81, a focus adjustment unit 83, an excitation light source 85, a fluorescence filter 87, a dimming filter 89, a second objective lens 91, and an optical path switch 93. Prepare. The first objective lens 81 and the focus adjustment unit 83 correspond to the photographing lens 19 and the lens driving unit 21 of the imaging unit 10 illustrated in FIG. The first objective lens 81 captures an image of the observation specimen P, and has a plurality of lenses in the lens barrel. In the lens barrel of the first objective lens 81, a plurality of lens groups including a zoom lens for adjusting the imaging magnification of the observation specimen P and a focus lens for focusing on the observation specimen P are arranged.

  The focus adjustment unit 83 includes a motor (not shown) that moves the zoom lens and the focus lens within the lens barrel of the first objective lens 81. The focus adjustment unit 83 moves one or a plurality of focus lenses in the optical axis direction in response to an instruction from the AF adjustment unit 35 of the control device 30a. Thereby, the focus of the first objective lens 81 is adjusted to the observation specimen P. Therefore, the observation specimen P is illuminated by the illumination light emitted from the normal light source 67, and the first objective lens 81 captures an image of the observation specimen P.

  The excitation light source (illumination unit) 85 emits excitation light set to a predetermined wavelength in order to excite fluorescence by a reagent or the like applied to the observation specimen P. The irradiation time of excitation light and the intensity of excitation light are controlled by an instruction from the control device 30a. Excitation light emitted from the excitation light source 85 enters the fluorescent filter 87 via the dimming filter 89. The excitation light incident on the fluorescent filter 87 is reflected by the wavelength selection mirror 87 a and is irradiated on the observation specimen P through the first objective lens 81.

  Note that the fluorescence excited in the observation specimen P by the excitation light is incident on the fluorescence filter 87 via the first objective lens 81. The fluorescence that has entered the fluorescence filter 87 passes through the wavelength selection mirror 87 a, enters the imaging unit 10 a via the second objective lens 91, and forms an image on the light receiving surface of the image sensor 12. Further, the image of the observation specimen P obtained by the above-described normal light source 67 also passes through the wavelength selection mirror 87a through the first objective lens 81, and enters the imaging unit 10a through the second objective lens 91. An image is formed on the light receiving surface of the sensor 12.

  The optical path switch 93 is disposed between the second objective lens 91 and the imaging unit 10a. The optical path switch 93 is provided with a total reflection mirror 93a having an inclination of 45 degrees with respect to the optical axis AX. The optical path switching unit 93 includes a motor 95 for inserting or retracting the total reflection mirror 93a with respect to the optical axis AX. When the total reflection mirror 93a is inserted into the optical axis AX, the light that has passed through the second objective lens 91 is reflected by the total reflection mirror 93a and emitted in the direction of an eyepiece (not shown).

  When the total reflection mirror 95a is retracted from the position of the optical axis AX, the light that has passed through the second objective lens 91 is incident on the imaging unit 10a. The motor 95 drives the optical path switch 93 according to an instruction from an optical path switching output unit 57 of the control device 30a described later, and inserts or retracts the total reflection mirror 93a with respect to the optical axis AX.

  When the total reflection mirror 93a is retracted, the imaging unit 10a takes in the light that has passed through the second objective lens 91 and images it with the image sensor 12. The image signal output from the imaging unit 10a is captured by the image input unit 39 of the control device 30a in the same manner as the imaging device 1 shown in FIG. The imaging unit 10a is the same except for the imaging unit 10 and the photographing lens 19 and the lens driving unit 21 shown in FIG.

  The control device 30a includes an XY position adjustment unit 55 and an optical path switching output unit 57 in addition to the configuration of the control device 30 shown in FIG. Both the XY position adjustment unit 55 and the optical path switching output unit 57 are connected to the bus 47. Except for the XY position adjustment unit 55 and the optical path switching output unit 57, the configuration is the same as that of the control device 30 shown in FIG.

  The XY position adjustment unit 55 outputs a control signal based on an instruction input from the user to the X-axis drive motor 63 and the Y-axis drive motor 65, and the observation target portion that the user wants to see in the sample 73 is the optical axis AX. The XY moving stage 61 is moved so as to come up. In addition to the period t input instruction, the application input instruction, and the image storage instruction as shown in FIG. 1, the user uses the input device 45 to issue an optical path switching instruction, an XY input instruction, and a light source switching instruction (not shown). input.

  The optical path switching instruction selects whether the total reflection mirror 93a of the optical path switch 93 is inserted into or retracted from the optical axis AX. The XY input instruction instructs the position in the X direction and the position in the Y direction of the XY moving stage 61. The light source switching instruction selects which of the normal light source 67 and the excitation light source 85 is used to illuminate the observation specimen P. In the XY input instruction, for example, the position in the X direction and the position in the Y direction may be assigned to the left and right and upper and lower cursor keys of the keyboard (input device 45).

  The XY moving stage 61 has an encoder (not shown) that detects a position in the X direction and a position in the Y direction. The detection signal from this encoder is taken into the control device 30a, and the position of the XY moving stage 61 in the X and Y directions is feedback-controlled with high accuracy. Further, the position (coordinate values) in the X and Y directions output from the encoder may be displayed on a part of the monitor 51.

  The optical path switching output unit 57 outputs a drive command to the optical path switching motor 95 based on an instruction input from the user by the input device 45, and inserts or retracts the total reflection mirror 93a with respect to the optical axis AX.

  In the microscope apparatus MS, the control device 30a includes a first illumination control unit 42, a second illumination control unit 43, and a brightness correction unit 44 in the correction unit 41 as in the imaging device 1 described above. . Therefore, regardless of whether the normal light source 67 or the excitation light source 85 is used, the above description using FIGS. 3 to 14 is also applied to the microscope apparatus MS as it is. Hereinafter, in the microscope apparatus MS, the same contents as those of the imaging apparatus 1 described above will be omitted or simplified.

  The flowcharts shown in FIGS. 11 to 13 are also applied to the microscope apparatus MS. Therefore, the user inputs live observation as an application from the input device 45 at the time of imaging, moves the XY moving stage 61 while looking at the monitor 51, searches for the observation target position in the observation specimen P, and performs imaging. Perform magnification and focus adjustment. The AF adjustment unit 35 of the control device 30a controls the focus adjustment unit 83 of the objective optical system 80 to perform magnification and focus adjustment of the observation specimen P using the first objective lens 81.

  In this live observation, the correction process B (see FIG. 8) is applied as described above. Further, the user adjusts the period t while watching the monitor 51 to adjust the brightness of the image to be optimum.

  Further, since the brightness of the image is increased by the correction process B during the live observation, the point that the damage to the observation specimen P due to the increase in brightness is not increased even if the observation specimen P is a living body or the like. It is as follows. The user selects either the normal light source 67 or the excitation light source 85 by inputting the selection of the light source to the input device 45 as the illumination light of the observation specimen P. Any light source can be used for input such as live observation and image storage. Further, the user can confirm the image of the observation specimen P directly with the eyepiece instead of displaying on the monitor 51 by inputting the optical path selection to the input device 45.

  The user inputs an image storage instruction to the input device 45 to store the image of the observation specimen P in the external storage device 49 or the like in a state set by live observation. When a still image is captured, the image brightness correction process is performed by the correction process A (see FIG. 7). When a moving image is captured, the image is corrected by the correction process C (see FIG. 9). The brightness correction is performed as described above.

  As described above, according to the microscope apparatus MS of the present embodiment, the brightness of the image output from the image sensor is increased by the brightness correction unit in the same manner as the imaging apparatus 1 described above. The brightness of the image can be changed seamlessly, and the brightness of the image can be easily adjusted. In addition, since an appropriate correction process is performed according to the application, even if the observation specimen P is a living body or the like, a highly accurate image can be acquired while reducing damage to the observation specimen P.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the content of above-described embodiment. Changes or improvements can be added to the above-described embodiment without departing from the spirit of the present invention. In addition, one or more of the requirements described in the above embodiments may be omitted. Such modifications, improvements, and omitted forms are also included in the technical scope of the present invention. In addition, the configurations of the above-described embodiments can be applied in appropriate combinations.

  Moreover, you may implement | achieve the one part structure of the imaging device 1 and microscope apparatus MS with a computer. For example, the control devices 30 and 30a may be realized by a computer. In this case, according to the imaging program stored in the storage unit, the computer determines that the period t is t ≦ na with respect to the time difference a between the start of charge accumulation in the first pixel row and the start of charge accumulation in the last pixel row of the image sensor 12. In the case of (n ≧ 1), the first illumination process for irradiating illumination light from the illumination unit over the entire period from the start of charge accumulation in the first pixel row to the end of charge accumulation in the last pixel row, and the period t is t> na In the case of (n ≧ 1), in the second illumination process in which the illumination light is irradiated from the illumination unit only during the overlap period in which the charge accumulation period of the first pixel row and the charge accumulation period of the last pixel row overlap, And brightness correction processing for increasing the brightness of the image output from the image sensor 12.

  The imaging program may be provided by being stored in a computer-readable storage medium such as an optical disc, a CD-ROM, a USB memory, or an SD card.

MS ... microscope device, P ... observation specimen, 1 ... imaging device, 10, 10a ... imaging unit, 12 ... image sensor, 15 ... rolling shutter drive unit, 23 ... illumination unit, 30, 30a ... control device, 41 ... correction unit (Switching unit), 42 ... first illumination control unit, 43 ... second illumination control unit, 44 ... brightness correction unit, 44a ... irradiation time adjustment unit, 44b ... gain adjustment unit, 44c ... light intensity adjustment unit, 60 ... Microscope body

Claims (16)

A rolling shutter controller that controls the charge accumulation period t for each pixel row while sequentially shifting the time from the first pixel row to the last pixel row of the image sensor in which a plurality of pixels are two-dimensionally arranged;
A first illumination control unit that irradiates the observation specimen with illumination light from the illumination unit over the entire period from the start of charge accumulation of the first pixel row to the end of charge accumulation of the last pixel row;
A second illumination control unit that irradiates the observation specimen with illumination light from an illumination unit only during an overlapping period in which the charge accumulation period of the first pixel row and the charge accumulation period of the last pixel row overlap;
When switching from the control by the first illumination control unit to the control by the second illumination control unit , the brightness of the image output from the image sensor based on the illumination light irradiation by the first illumination control unit, and the second A brightness correction unit that corrects the brightness of the image output from the image sensor so as to reduce the difference between the brightness of the image output from the image sensor based on the illumination light irradiation by the illumination control unit ;
A control apparatus for an image pickup apparatus. When the period t is t ≦ na (n ≧ 1) with respect to the time difference a between the charge accumulation start time of the first pixel row and the charge accumulation start time of the last pixel row, The illumination unit emits illumination light over the entire period from the start of charge accumulation in the first pixel row to the end of charge accumulation in the last pixel row, and the second illumination control unit has the period t of t> na (n ≧ 1). ), The illumination device emits illumination light only during an overlapping period in which the charge accumulation period of the first pixel row overlaps the charge accumulation period of the last pixel row. The brightness correction unit is configured to extend the period t by the rolling shutter control unit, the control unit of the imaging apparatus according to claim 1 or claim 2, wherein including the irradiation time adjustment unit to extend the irradiation time of the illumination light . 4. The control device for an imaging apparatus according to claim 1 , wherein the brightness correction unit includes a gain adjustment unit that amplifies an output signal of the image sensor. 5.   The control device for an imaging apparatus according to claim 1, wherein the brightness correction unit includes a light amount adjustment unit that adjusts a light amount of the illumination light.   6. The imaging apparatus according to claim 1, wherein the brightness correction unit includes a different correction process and a switching unit that switches the correction process according to a purpose of imaging by the image sensor. Control device.   The imaging apparatus control apparatus according to claim 6, wherein the imaging application includes an application for storing an image and an application for observing a subject. The first illumination control unit irradiates the illumination light over the entire period when the period t is t ≦ a,
The control device for an imaging apparatus according to claim 2, wherein the second illumination control unit irradiates the illumination light only during the overlap period when the period t is t> a. The first illumination control unit irradiates the illumination light over the entire period when the period t is t ≦ 2a,
The control device for an imaging apparatus according to claim 2, wherein the second illumination control unit irradiates the illumination light only during the overlap period when the period t is t> 2a. An imaging device comprising the control device according to any one of claims 1 to 9. An imaging method of an imaging apparatus including a rolling shutter control unit that controls a charge accumulation period t for each pixel row while sequentially shifting time from the first pixel row to the last pixel row of an image sensor in which a plurality of pixels are two-dimensionally arranged. There,
Irradiating the observation specimen with illumination light from the illumination unit over the entire period from the start of charge accumulation of the first pixel row to the end of charge accumulation of the last pixel row;
Irradiating the observation specimen with illumination light from the illuminator only during an overlap period in which the charge accumulation period of the first pixel row and the charge accumulation period of the last pixel row overlap;
When the illumination light irradiation over the entire period is switched to illumination light irradiation for the overlap period, the brightness of the image output from the image sensor based on the illumination light irradiation over the entire period, and the overlap period only. And correcting the brightness of the image output from the image sensor so that a difference between the brightness of the image output from the image sensor and the brightness of the image output from the image sensor is reduced .   When the period t is t ≦ na (n ≧ 1) with respect to the time difference a between the charge accumulation start of the first pixel row and the charge accumulation start of the last pixel row, the charge accumulation start of the first pixel row Until the end of the charge accumulation of the last pixel row,
  12. When the period t is t> na (n ≧ 1), the illumination unit emits illumination light only during an overlapping period in which the charge accumulation period of the first pixel row overlaps the charge accumulation period of the last pixel row. The imaging method described. To a control device of an imaging apparatus including a rolling shutter control unit that controls a charge accumulation period t for each pixel row while sequentially shifting the time from the first pixel row to the last pixel row of an image sensor in which a plurality of pixels are two-dimensionally arranged ,
A first illumination process for irradiating the observation specimen with illumination light from the illumination unit over the entire period from the start of charge accumulation in the first pixel row to the end of charge accumulation in the last pixel row;
A second illumination process for irradiating the observation specimen with illumination light from an illumination unit only during an overlap period in which the charge accumulation period of the first pixel row and the charge accumulation period of the last pixel row overlap;
When switching from the first illumination process to the second illumination process, the brightness of the image output from the image sensor based on the illumination light irradiation to the first illumination process and the illumination light irradiation by the second illumination process And a brightness correction process for correcting the brightness of the image output from the image sensor so that a difference from the brightness of the image output from the image sensor is reduced .   The first illumination processing is performed when the period t is t ≦ na (n ≧ 1) with respect to a time difference a between the charge accumulation start time of the first pixel row and the charge accumulation start time of the last pixel row. Illumination light is irradiated from the illumination unit over the entire period from the start of charge accumulation in the pixel row to the end of charge accumulation in the last pixel row
  In the second illumination process, when the period t is t> na (n ≧ 1), the illumination unit illuminates only during the overlap period in which the charge accumulation period of the first pixel row overlaps the charge accumulation period of the last pixel row. The imaging program of Claim 13 which irradiates light. A storage medium in which the imaging program according to claim 13 or 14 is recorded and readable by a computer. A microscope apparatus equipped with the control device for an imaging apparatus according to any one of claims 1 to 9.

Images