JP2018182741A - Imaging apparatus and imaging program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it possible to improve a user's convenience.SOLUTION: An imaging apparatus according to the embodiment includes an imaging unit, picture generation unit, calculation unit, and synthesis unit. The imaging unit receives light of a first type to generate a first electric signal and receives light of a second type to generate a second electric signal. The picture generation unit successively generates a first picture on the basis of the first electric signal and successively generates a second picture on the basis of the second electric signal. The calculation unit calculates a first control parameter value for making the value of brightness of the first picture agree with or approximate to a predetermined value and, on the basis of the first control parameter value and a predetermined ratio, calculates a second control parameter value on brightness of the second picture. The synthesis unit synthesizes the first picture whose brightness has been adjusted using the first control parameter with the second picture whose brightness has been adjusted using the second control parameter to generate a synthesized picture.SELECTED DRAWING: Figure 1

Description

  Embodiments of the present invention relate to an imaging device and an imaging program.

  When observing (normally observing) a subject using a white light source that generates visible light, the sensitivity (exposure time, amplification amount, etc.) is automatically controlled according to the brightness of the background (incident light level from the white light source). There is a process to keep the brightness constant. In addition, in the case of observing (fluorescence observation) fluorescence generated by irradiation of X-rays, ultraviolet light, infrared light or visible light as excitation light, there is automatic control of the same sensitivity.

  On the other hand, in the medical field, for example, an image (a fluorescence image) obtained by fluorescence observation may be superimposed on an image (a background image) obtained by ordinary observation using fluorescence observation as described above. In this case, the sensitivity of the background image is automatically controlled, but the sensitivity of the fluorescence image can not be specified automatically because the position and the amount of the displayed fluorescence part can not be specified, and the user needs to set it manually.

However, in the medical field of observation, it is general that the irradiation conditions change, for example, the amount of light changes as the distance between the light source and the subject or the irradiation angle to the subject changes after the user sets the sensitivity of the fluorescence image. It is.
When the irradiation conditions change, the sensitivity of the background image is kept constant by the automatic control, but the sensitivity of the fluorescence image is not automatically controlled. Therefore, it is necessary for the user to set the sensitivity of the fluorescence image in comparison with the brightness of the background image every time the irradiation condition changes, which is very time-consuming and troublesome.

Patent No. 5265055 specification Patent No. 4761899

  An object of the present embodiment is to improve the convenience of the user.

  The imaging device according to the present embodiment includes an imaging unit, an image generation unit, a calculation unit, and a combining unit. The imaging unit generates a first electrical signal by receiving the first type of light, and generates a second electrical signal by receiving the second type of light. The image generation unit sequentially generates a first image based on the first electrical signal, and sequentially generates a second image based on the second electrical signal. The calculation unit calculates a first control parameter value that causes the brightness of the first image to match or approximate to a predetermined value, and the brightness of the second image is calculated based on the first control parameter value and a predetermined ratio. Calculating a second control parameter value related to A combining unit combines the first image whose brightness is adjusted using the first control parameter value and the second image whose brightness is adjusted using the second control parameter value, and combines them. Generate an image.

1 is a block diagram showing an imaging device according to a first embodiment. 6 is a flowchart showing sensitivity control processing of the imaging device according to the first embodiment. FIG. 2 is a diagram showing an imaging sequence according to the first embodiment. FIG. 6 is a view showing another example of the imaging sequence according to the first embodiment. FIG. 7 is a view showing a display example of a composite image on which sensitivity control processing has been performed. FIG. 7 is a block diagram showing an imaging device according to a second embodiment.

  Hereinafter, an imaging device and an imaging program according to the present embodiment will be described with reference to the drawings. In the following embodiments, portions given the same reference numerals perform similar operations, and redundant description will be appropriately omitted.

An imaging apparatus according to the first embodiment will be described with reference to the block diagram of FIG.
The imaging device 1 according to the first embodiment includes an imaging unit 11, an image processing unit 13, an image generation circuit 15, an image combining circuit 17, a white light source 20, and an IR light source 22.
The imaging unit 11 includes a prism 111, shutters 112, 113, 114, and 115, and image sensors 116, 117, 118, and 119.

  The image processing unit 13 includes gain adjustment circuits 131, 132 and 133, a master gain adjustment circuit 134, an evaluation value calculation circuit 135, a sensitivity calculation circuit 136, a background sensitivity control circuit 137, an addition circuit 138, and a fluorescence sensitivity. And a control circuit 139.

  The white light source 20 outputs white light which is visible light in which light beams of respective wavelengths are mixed. The output white light is emitted to the subject P. The white light emitted to the subject P is reflected to generate reflected light.

  The IR light source 22 outputs near-infrared excitation light (hereinafter referred to as IR (infrared rays) light). The output IR light is emitted to the subject P. Here, assuming that a fluorescent substance excited by IR light is present in the subject P, fluorescence is generated from the fluorescent substance.

The prism 111 here is a three-plate type prism, and splits the reflected light and fluorescence from the subject P into three primary colors of R (red) light, G (green) light and B (blue) light. In the case of a three-plate type prism, the fluorescence is split into the same path as the R light, so in practice the shutter 112 and the image sensor 116 are commonly used, and the functions for R light and fluorescence are time-shared It is distributed. However, for convenience of explanation, it is illustrated that the R light and the fluorescence are virtually processed in the separated path.
That is, as shown in FIG. 1, while white light is incident on the prism 111, R light is transmitted through the shutter 112 and received by the image sensor 116, and fluorescence is incident on the prism 111, as shown in FIG. Paths are virtually divided and described so that fluorescence passes through the shutter 115 and is received by the image sensor 119. Note that an example of time division control will be described later with reference to FIG.

  The shutter 112 adjusts the amount of incident light (hereinafter referred to as exposure time) when the R light separated by the prism 111 is incident on the image sensor 116 in the subsequent stage.

  The shutter 113 adjusts the exposure time when the G light separated by the prism 111 is incident on the image sensor 117 at the subsequent stage.

  The shutter 114 adjusts the exposure time when the B light separated by the prism 111 is incident on the image sensor 118 at the subsequent stage.

  The shutter 115 adjusts the exposure time when the IR light separated by the prism 111 is incident on the image sensor 119 in the subsequent stage.

  Note that as a specific operation of each shutter, for example, when the configuration of each shutter is a so-called mechanical shutter, the amount of light incident on the image sensor is adjusted by physically opening and closing. That is, the “exposure time” in the case of using the mechanical shutter is the time during which the image sensor receives light.

  On the other hand, when the configuration of each shutter is a so-called electronic shutter, the amount of incident light to the image sensor is adjusted by adjusting the charge amount of the charge from the empty state of the capacitor. That is, the "exposure time" in the case of the electronic shutter is the time for recharging after discarding the charged charge once.

  The image sensor photoelectrically converts the R light to generate an R imaging signal which is an electrical signal.

  The image sensor 117 photoelectrically converts the G light to generate a G imaging signal which is an electric signal.

  The image sensor 118 photoelectrically converts the B light to generate a B imaging signal which is an electrical signal. Hereinafter, the R imaging signal, the G imaging signal, and the B imaging signal will be collectively referred to as an RGB imaging signal which is an electrical signal obtained by receiving white light.

  The image sensor 119 photoelectrically converts the IR light to generate an IR imaging signal which is an electrical signal.

  As each image sensor mentioned above, although the case where CCD (Charge-Coupled Device) is used is assumed, for example, you may use other imaging devices, such as CMOS (Complementary Metal-Oxide Semiconductor).

  Although not shown, the imaging unit 11 is provided with an excitation light cut filter for blocking excitation light. Therefore, the IR light is not incident on the image sensor 119, and only the fluorescence component is incident.

  The gain adjustment circuit 131 receives the R imaging signal from the image sensor 116 and adjusts the gain of the R imaging signal.

  The gain adjustment circuit 132 receives the B imaging signal from the image sensor 118 and adjusts the gain of the B imaging signal.

  The gain adjustment by the gain adjustment circuit 131 and the gain adjustment circuit 132 is a so-called white balance adjustment, and the gain is adjusted by adjusting the amplitudes of the R imaging signal and the B imaging signal as necessary by general processing. Should be adjusted.

  The gain adjustment circuit 133 receives an IR imaging signal from the image sensor 119 and a control signal related to the sensitivity of the fluorescence image generated based on the IR imaging signal (hereinafter referred to as fluorescence sensitivity) from the addition circuit 138 described later. The gain adjustment circuit 133 adjusts the gain of the IR imaging signal based on the control signal.

  The master gain adjustment circuit 134 receives an R image pickup signal from the gain adjustment circuit 131, a G image pickup signal from the image sensor 117, and a B image pickup signal from the gain adjustment circuit 132, respectively, and generates R image pickup signal, G image pickup signal and B image pickup signal. Adjust the overall gain.

  The evaluation value calculation circuit 135 receives the R image pickup signal, the G image pickup signal and the B image pickup signal whose gains are adjusted from the master gain adjustment circuit 134, and the background which is a color image based on the R image pickup signal, the G image pickup signal and the B image pickup signal Calculate the evaluation value for the brightness of the image. The evaluation value is, for example, a luminance value of a pixel.

  The sensitivity calculation circuit 136 receives the evaluation value from the evaluation value calculation circuit 135 and compares the evaluation value with a preset target value to set the setting value (control parameter) of the background image sensitivity (hereinafter referred to as background sensitivity). Calculate the value). For example, if the evaluation value is larger than the target value of brightness, the background sensitivity is set such that the brightness of the image becomes dark, and if the evaluation value is smaller than the target value, the brightness of the image becomes brighter Set the background sensitivity.

  The background sensitivity control circuit 137 receives the background sensitivity set from the sensitivity calculation circuit 136, determines which one of gain, exposure time and light quantity is to be used to control the background sensitivity, and a control signal for the determined control means. Generate

  The adder circuit 138 receives the background sensitivity set from the sensitivity calculation circuit 136 and the sensitivity ratio set by the user from the outside, respectively, and uses the background sensitivity and the sensitivity ratio to set the sensitivity of the fluorescence image based on the IR imaging signal ( Hereinafter, the set value (control parameter value) of the fluorescence sensitivity is calculated.

  The fluorescence sensitivity control circuit 139 receives the setting value of the fluorescence sensitivity calculated from the addition circuit 138, determines which one of gain, exposure time and light intensity to use to control the fluorescence sensitivity, and controls the determined control means Generate a signal.

  The image generation circuit 15 sequentially receives an R imaging signal, a G imaging signal and a B imaging signal from the master gain adjustment circuit 134 and an IR imaging signal from the gain adjustment circuit 133 each time each signal is generated. The image generation circuit 15 sequentially generates a background image based on the R imaging signal, the G imaging signal, and the B imaging signal, and sequentially generates a fluorescence image based on the IR imaging signal.

  The image combining circuit 17 combines the background image and the fluorescence image by α blending or the like to generate a combined image. As a result, an image in which the fluorescent component is superimposed as a marker on the background image is obtained and displayed on a display device (not shown) such as a monitor.

Next, sensitivity control processing of the imaging device 1 according to the first embodiment will be described with reference to the flowchart in FIG.
The operation of the imaging device 1 assumes that each imaging signal is sequentially acquired at a predetermined sampling timing so that a composite image based on each imaging signal can be displayed as a moving image of, for example, 60 fps. In addition, you may acquire a still image instead of a moving image.

  In step S201, a sensitivity ratio is set. As the sensitivity ratio, the image processing unit 13 may acquire a preset default value. Alternatively, a composite image is generated using a default value as a sensitivity ratio, and the composite image displayed on the display device is adjusted using a dial, a mouse, or the like so that the user can easily see the fluorescent portion against the background. Do. Thereafter, the ratio of the background sensitivity to the fluorescence sensitivity in a specific time phase in which an image obtained by the user pressing the confirmation button may be set as the sensitivity ratio.

  In step S202, a photometric area is set. As the photometric area, the image processing unit 13 may acquire the area manually set by the user. Alternatively, the image processing unit 13 in which the shape and size of the area for the image are preset may be used as the photometric area.

  In step S203, the imaging unit 11 acquires an R imaging signal, a G imaging signal, a B imaging signal, and an IR imaging signal.

  In step S204, the evaluation value calculation circuit 135 calculates an evaluation value of the background image. The range for evaluating the brightness of the background image may be the entire background image, or may be the photometric area set in step S202. Further, as the brightness of the background image, an average of luminance values for each pixel may be used, or an integral of maximum values of RGB pixel values may be used.

  In step S205, the sensitivity calculation circuit 136 calculates background sensitivity. The background sensitivity calculated here can be expressed by “exposure time × master gain value”.

  In step S206, the background sensitivity control circuit 137 determines control means for the background sensitivity based on the obtained background sensitivity. The background sensitivity is adjusted according to the determined control means.

  For example, when controlling the gain, the background sensitivity control circuit 137 sends a control signal to the master gain adjustment circuit 134 to obtain a gain that matches or approximates the setting value Sw of the background sensitivity. The master gain adjustment circuit 134 controls the overall gains of the R imaging signal, the G imaging signal, and the B imaging signal based on the control signal.

  When controlling the exposure time, the background sensitivity control circuit 137 sends, to the imaging unit 11, a control signal for setting the exposure time to coincide with or approximate to the setting value Sw of the background sensitivity. The shutters 112, 113, and 114 of the imaging unit 11 control the exposure time according to the control signal. More specifically, in the case of a mechanical shutter, the open / close time is controlled, and in the case of an electronic shutter, the amount of charge to be charged is controlled.

  When controlling the light amount, the background sensitivity control circuit 137 sends, to the white light source 20, a control signal for setting the light amount to match or approximate to the setting value Sw of the background sensitivity. The white light source 20 controls the output of the light amount according to the control signal.

  In addition, when background sensitivity is adjusted using one control means, the case where the background sensitivity can not be adjusted any more by the control means can not be assumed because the background sensitivity can not be made close to the set value Sw. For example, when controlling the gain to adjust the background sensitivity, it is also assumed that the gain can not be further increased. In such a case, the background sensitivity may be adjusted in combination with other adjustment methods such as controlling the shutters 112, 113, and 114 of the imaging unit 11 to extend the exposure time.

  When the background sensitivity is increased (the background image is made bright), the shutters 112, 113, and 114 are controlled to increase the exposure time at first, and when the luminance is still insufficient, the master gain adjustment circuit 134 Although it is desirable to control to increase the gain, it may be controlled from the gain first.

  In step S207, the adding circuit 138 calculates the setting value of the fluorescence sensitivity in accordance with the setting value Sw of the background sensitivity calculated in step S206 and the sensitivity ratio set in step S201. Assuming that the set value Sir of the fluorescence sensitivity is the sensitivity ratio Ro, then Sir = Ro × Sw may be calculated.

  In step S208, the fluorescence sensitivity control circuit 139 determines control means of the fluorescence sensitivity based on the set value Sir of the fluorescence sensitivity. The fluorescence sensitivity is adjusted in accordance with the determined control means. The fluorescence sensitivity can be controlled by “exposure time × master gain value” as in the background sensitivity. Therefore, for example, in the case of controlling the gain, the fluorescence sensitivity control circuit 139 sends, to the gain adjustment circuit 133, a control signal for obtaining a gain that matches or approximates the set value Sir of the fluorescence sensitivity. A gain adjustment circuit controls the gain of the IR imaging signal based on the control signal.

  When controlling the exposure time, the fluorescence sensitivity control circuit 139 sends, to the imaging unit 11, a control signal for setting the exposure time to coincide with or approximate the set value of the fluorescence sensitivity. The shutter 115 of the imaging unit 11 controls the exposure time in accordance with the control signal.

  When controlling the light amount, the fluorescence sensitivity control circuit 139 sends, to the IR light source 22, a control signal for setting the light amount to match or approximate the set value of the fluorescence sensitivity. The IR light source 22 controls the output of the light amount according to the control signal.

  In step S209, the image combining circuit 17 generates a combined image of the sensitivity-adjusted background image and the fluorescence image. It is assumed that the composite image is displayed on the display device so that the user can confirm it.

  After the fluorescence sensitivity is adjusted in step S209, the process returns to step S203, acquires each next imaging signal (RGB imaging signal, IR imaging signal), and repeats the processing of each step.

  For example, a process of determining whether the brightness of the background image has changed may be provided by the background sensitivity control circuit 137, and the processes of steps S204 to S209 may be executed only when the brightness changes. That is, when the luminance does not change, power consumption can be reduced by not performing the processing from step S204 to step S209 relating to sensitivity adjustment. The process of determining whether the luminance has changed may be performed, for example, after step S203.

  Also, the sensitivity ratio may be set after the photometric area is determined. For example, the image processing unit 13 stores the statistical correspondence between the subject (part of the subject), the photometric area, and the sensitivity ratio, and the sensitivity ratio is obtained from the correspondence according to the photometric area set in step S202. May be determined.

  The sensitivity ratio may be determined automatically as well as manually set by the user. For example, when the fluorescence image is spread over the entire screen, or when the fluorescence image is not displayed on the screen, it may be clear that the sensitivity setting of the fluorescence image is not appropriate. In such a case, the sensitivity ratio may be adjusted automatically so that the fluorescence image falls within an appropriate range.

  Specifically, when the fluorescence image is spread over the entire screen, it indicates that the ratio of the fluorescence sensitivity to the background sensitivity is too high. Therefore, the fluorescence sensitivity control circuit 139 may lower the fluorescence sensitivity until the condition of the upper limit of the allowable range is satisfied. As the condition of the allowable range, for example, it may be set whether the size of the region of the fluorescence image displayed on the screen is equal to or less than the threshold or the peak level of the fluorescence luminance value is equal to or less than the threshold.

On the other hand, when the fluorescence image is not displayed on the screen, it indicates that the ratio of the fluorescence sensitivity to the background sensitivity is too low. Therefore, the fluorescence sensitivity control circuit 139 may increase the fluorescence sensitivity until the condition of the lower limit of the allowable range is satisfied. As the condition of the allowable range, it may be set whether the area of the fluorescence image displayed on the screen has a size equal to or larger than the threshold or the peak level of the luminance value of the fluorescence image is equal to or larger than the threshold. The condition of the allowable range is not limited to this, and any adjustment regarding the fluorescence sensitivity may be used.
As described above, the sensitivity ratio may be adjusted more flexibly by giving a certain range to the sensitivity ratio according to the fluorescence sensitivity in the allowable range.

Next, an imaging sequence in the imaging unit 11 will be described with reference to FIGS. 3 and 4.
FIG. 3 shows, from top to bottom, (a) video output, (b) exposure time of each image sensor, (c) output time of white light (RGB) from white light source 20, and (d) IR light source 22 shows a timing chart regarding the output time of IR light from 22.

  As described above, since fluorescence due to R light and IR light is split in the same direction in the prism 111, the irradiation time of the white light source 20 and the irradiation time of the IR light source 22 are switched by time division and controlled. Light and fluorescence are taken into the image sensor respectively.

  Specifically, in the period 301, the white light source 20 is on (outputs white light) and the IR light source 22 is off (does not output IR light), so only white light is incident on the prism 111. . Therefore, the R light, the G light and the B light are received by the image sensors 116, 117 and 118, respectively.

  After the end of the period 301, in the next period 302, the white light source 20 is turned off and the IR light source 22 is turned on, so that only fluorescence is incident on the prism 111. Therefore, after the end of the period 301, the fluorescence is received by the image sensor 119.

  By repeating the above processing, it is possible to obtain an R imaging signal, a G imaging signal, a B imaging signal, and an IR imaging signal by using the prism 111 that splits the light into R light, G light and B light.

Next, another example of the imaging sequence in the imaging unit 11 is shown in FIG.
FIG. 4 shows, from top to bottom, (a) image output, (b) exposure time of each image sensor, (c) output time of GB light from white light source 20, and (d) from IR light source 22. The timing chart regarding the output time of IR light is shown.

  In the example of FIG. 4, the light output from the white light source 20 is not RGB light, but GB light in which the R light component is attenuated by a color filter or the like. That is, as the background image, a monochrome image is generated based on the G imaging signal and the B imaging signal obtained from the GB light, not the color image.

  Specifically, the white light source 20 and the IR light source 22 are always on, that is, the white light source 20 emits GB light and the IR light source 22 simultaneously emits IR light. At the timing of the period 401, the image sensors 117 and 118 can receive G light and B light, respectively, and the image sensor 119 can receive fluorescence.

  Since this is a monochrome image, IR light can be received at the timing of receiving R light, and time division processing of R light and IR light as shown in FIG. 3 may not be performed. Therefore, the decrease in sensitivity due to the decrease in exposure time can be suppressed.

Next, a display example of the composite image subjected to the sensitivity control process according to the first embodiment will be described with reference to FIG.
The left view of FIG. 5 is an example of a composite image in which a fluorescence image 502 by a fluorescence marker is superimposed on a background image 501 of an object captured by an endoscope, for example. Here, it is assumed that the reference sensitivity ratio is set in the photometric area 503.

The central diagram of FIG. 5 shows the case where the light source (white light source 20 and IR light source 22) or the imaging unit 11 approaches the subject.
When the light source or the imaging unit 11 approaches the subject, the amount of light incident on the imaging unit 11 increases, so the brightness of both the background image and the fluorescence image increases. Here, when the background sensitivity is automatically adjusted, the fluorescence sensitivity is set based on the background sensitivity and the sensitivity ratio. Therefore, for example, adjustment such as darkening the fluorescence image is performed so as to keep the sensitivity ratio constant. It can be done automatically.

On the other hand, the right view of FIG. 5 shows the case where the light source or the imaging unit 11 is moved away from the subject.
When the light source or the imaging unit 11 moves away from the subject, the amount of light incident on the imaging unit 11 decreases, so the brightness of both the background image and the fluorescence image decreases. Even in this case, the fluorescence sensitivity is set based on the background sensitivity and the sensitivity ratio as in the case where the light source or the imaging unit 11 approaches the subject, so the fluorescence image is brightened to keep the sensitivity ratio constant. Adjustments can be made automatically.

  When the light amount of the white light source 20 can not be controlled, the background sensitivity control circuit 137 may adjust the background sensitivity by at least one of the exposure time and the gain. Similarly, when the fluorescence sensitivity control circuit 139 can not control the light amount of the IR light source 22, the fluorescence sensitivity may be adjusted by at least one means of exposure time and gain.

According to the first embodiment described above, since the fluorescence sensitivity is set based on the background sensitivity and the sensitivity ratio, the distance between the subject and the imaging device or the light source changes to change the imaging condition. Even when the brightness of the image is automatically adjusted, the brightness of the fluorescence image is also automatically controlled so as to keep the sensitivity ratio between the background image and the fluorescence image constant.
By this, even when the distance between the subject and the imaging device or the light source changes, it is not necessary to set the brightness with the fluorescence image each time, and the sensitivity ratio and the visibility desired by the user can be maintained for the composite image. , The convenience of the user is improved.

Second Embodiment
In the first embodiment, it is assumed that the amount of light output from the light source can also be controlled. However, when the light amount can not be controlled, when the light amount of at least one of the white light source 20 and the IR light source 22 is changed, a deviation from the assumed sensitivity ratio may occur according to the control of the sensitivity ratio.

  For example, when the amount of light output from the white light source 20 is doubled, in the sensitivity adjustment of the background image, control is performed to halve the gain. At this time, when control is performed to make the sensitivity ratio constant, the gain of the fluorescence image is reduced to 1⁄2 although the light amount from the IR light source 22 is not changed. In this case, the sensitivity of only the fluorescence image is lowered.

  Therefore, in the second embodiment, the desired sensitivity ratio can be kept constant by referring to the intensity ratio of the white light source 20 and the IR light source 22 when the sensitivity ratio is set in addition to the sensitivity ratio.

An imaging device 1 according to the second embodiment will be described with reference to the block diagram of FIG.
In addition to the configuration of the first embodiment, an intensity ratio acquisition circuit 140 is included.

  The intensity ratio acquisition circuit 140 acquires the intensity ratio when the sensitivity ratio is set. The intensity ratio is a ratio of the intensity (light amount) of white light to the intensity (light amount) of IR light. The intensity ratio is acquired, for example, from the white light source 20 and the IR light source 22 at the timing when the user sets the sensitivity ratio described above in the first embodiment. Note that the intensity ratio set by the user may be acquired.

  The adder circuit 138 acquires the sensitivity ratio from the outside and the intensity ratio from the intensity ratio acquisition circuit 140, and performs processing to keep the sensitivity ratio constant with reference to the intensity ratio. Specifically, the intensity ratio Rlo can be expressed by “intensity of white light / intensity of IR light”. Also, assuming that the current intensity ratio is Rlt, the set value Sir of the fluorescence sensitivity can be expressed as “Sir = Sw × Ro × Rlt / Rlo”.

  Therefore, the adder circuit 138 can calculate the set value Sir of the fluorescence sensitivity by adding the information of the intensity ratio when calculating the sensitivity ratio.

  The fluorescence sensitivity control circuit 139 may determine the control means of the fluorescence sensitivity based on the setting value of the fluorescence sensitivity from the adding circuit 138 as in the first embodiment, and adjust the fluorescence sensitivity.

  According to the second embodiment described above, even when the intensity ratio between the white light source and the IR light source changes when the light quantity of the light source can not be controlled, the fluorescence sensitivity is calculated in consideration of the intensity ratio. The ratio of the background image and the fluorescence image can be kept constant, and the convenience of the user can be further improved.

  In the embodiment described above, since the three-plate type prism is used, by controlling the white light source 20 and the IR light source 22 by time division, four R light (IR light), G light and B light are obtained. An imaging signal is obtained. However, with a four-plate prism that can split light into four lights (channels) of R light, G light, B light, and IR light, each channel can be read directly, so even without controlling the light source by time division Good.

Also, R light, G light, and B light may be acquired using a color filter instead of the prism 111.
The imaging unit 11 may be configured to include the gain adjustment circuits 131 and 132. Furthermore, the master gain adjustment circuit 134 may be disposed in both the imaging unit 11 and the image processing unit 13 so that the gain can be adjusted by each. As a result, the image processor 13 may adjust the gain by the amount that is insufficient for the gain adjustment of the imaging unit 11.

  Although near-infrared rays are assumed in the present embodiment, the same processing can be performed even when a plurality of lights of other wavelength bands such as far-infrared rays and ultraviolet rays are used.

In the case of processing a plurality of fluorescence (for example, red and blue) in a plurality of wavelength bands, the shutter 112 and the image sensor 116 are used in common with R light for red fluorescence, and R light and red light in time division. The function for the fluorescence of For blue fluorescence, the shutter 114 and the image sensor 118 are used in common with B light, and the functions for B light and blue fluorescence are distributed in time division.
When fluorescence of a plurality of wavelength bands is used, after determining the first sensitivity ratio of the fluorescence of the first wavelength band to the background image, the second fluorescence of the second wavelength band and the background image is determined. In determining the sensitivity ratio, the second sensitivity ratio may be determined based on the first sensitivity ratio.

  Furthermore, each of the circuits described above may be an application specific integrated circuit (ASIC) incorporating a dedicated hardware circuit, a field programmable gate array (FPGA), and other composite circuits. It may be realized by a programmable logic device (Complex Programmable Logic Device: CPLD) or a Simple Programmable Logic Device (SPLD).

  While certain embodiments of the present invention have been described, these embodiments have been presented by way of example only, and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and the gist of the invention, and are included in the invention described in the claims and the equivalent scope thereof.

  DESCRIPTION OF SYMBOLS 1 ... Imaging device, 11 ... Imaging part, 13 ... Image processing part, 15 ... Image generation circuit, 17 ... Image synthetic circuit, 20 ... White light source, 22 ... IR Light source 111: prism 112, 113, 114, 115: shutter 116, 117, 118, 119: image sensor 131, 132, 133: gain adjustment circuit 134: master Gain adjustment circuit 135: evaluation value calculation circuit 136: sensitivity calculation circuit 137: background sensitivity control circuit 138: addition circuit 139: fluorescence sensitivity control circuit 140: 140 Intensity ratio acquisition circuit 301, 302, 401 ... period, 501 ... background image, 502 ... fluorescence image, 503 ... photometry area.

Claims (8)

An imaging unit that generates a first electrical signal by receiving a first type of light and generates a second electrical signal by receiving a second type of light;
An image generation unit that sequentially generates a first image based on the first electrical signal and sequentially generates a second image based on the second electrical signal;
A first control parameter value is calculated that causes the brightness of the first image to match or approximate to a predetermined value, and a second control is performed on the brightness of the second image based on the first control parameter value and a predetermined ratio. A calculation unit that calculates control parameter values;
Combining the first image whose brightness is adjusted using the first control parameter value and the second image whose brightness is adjusted using the second control parameter value to generate a composite image A synthesis unit,
An imaging device provided with   The imaging device according to claim 1, wherein the predetermined ratio is a ratio of the first control parameter value and the second control parameter value in a specific time phase. The first control parameter value includes at least one of an intensity of the first type of light, an exposure time of the first type of light, and a gain value applied to the first electrical signal.
The second control parameter value includes at least one of an intensity of the second type of light, an exposure time of the second type of light, and a gain value applied to the second electrical signal. Or the imaging device as described in 2. An acquisition unit for acquiring an intensity ratio of the first type of light and the second type of light when the predetermined ratio is determined;
The imaging device according to any one of claims 1 to 3, wherein the calculation unit calculates the second control parameter value based on the predetermined ratio and the intensity ratio.   The imaging device according to any one of claims 1 to 4, wherein the first type of light is visible light and the second type of light is fluorescence.   The imaging device according to claim 1, further comprising a control circuit that controls the predetermined ratio such that a condition regarding the adjusted second image is within an allowable range when determining the predetermined ratio. The image sensor receives a plurality of second types of light,
Using a first ratio of the first type of light and one of the plurality of second types of light, the first type of light and the other of the plurality of second types of light The imaging device according to claim 1, wherein respective ratios for and are determined. On the computer
A first image is sequentially generated based on a first electrical signal obtained by receiving the first type of light, and a second image is generated based on a second electrical signal obtained by receiving the second type of light. A generation function that sequentially generates an image,
A first control parameter value is calculated that causes the brightness of the first image to match or approximate to a predetermined value, and a second control is performed on the brightness of the second image based on the first control parameter value and a predetermined ratio. A calculation function for calculating control parameter values;
Combining the first image whose brightness is adjusted using the first control parameter value and the second image whose brightness is adjusted using the second control parameter value to generate a composite image With the generation function,
An imaging program for realizing