JP6090217B2 - IMAGING SYSTEM AND IMAGING SYSTEM CONTROL METHOD - Google Patents

Description

  The present invention relates to an imaging system and an imaging system control method.

  Conventionally, in order to image a subject in an environment where there is almost no visible light such as at night, a method of irradiating the subject with infrared light using an infrared projector and imaging infrared light reflected from the subject has been used. This method is an effective imaging method when a light that emits visible light cannot be used.

  However, an image obtained by imaging a subject by this imaging method is a monochrome image. In monochrome images, it may be difficult to identify an object. If a color image can be captured even in an environment where there is no visible light, the object identification can be improved. For example, in a surveillance camera, it is desired to capture a color image even in an environment without visible light in order to improve the object identification.

  Patent Document 1 describes an imaging apparatus that can capture a color image even in an environment without visible light. In the imaging device described in Patent Document 1, an infrared projector is also used. If the technique described in Patent Document 1 is installed in the surveillance camera, it becomes possible to convert the subject into a color image and improve the object identification.

JP 2011-50049 A

  The imaging device described in Patent Document 1 images a subject by sequentially projecting infrared light of three wavelengths by an infrared projector.

  In a monitoring system (imaging system), a subject to be monitored may be imaged using a plurality of monitoring cameras. It is conceivable to configure a monitoring system using the imaging device described in Patent Document 1 for each of a plurality of monitoring cameras.

  When a plurality of imaging devices individually project infrared light, infrared light having different wavelengths are mixed, and a color image capable of preferable color reproduction cannot be captured.

  An object of the present invention is to provide an imaging system and an imaging system control method capable of imaging a preferable image even when there are a plurality of imaging apparatuses that project infrared light and image a subject.

  The present invention includes a plurality of imaging devices and a synchronization signal supply device that supplies a synchronization signal to each of the plurality of imaging devices, and the plurality of imaging devices are The infrared signal is selected at a timing based on the synchronization signal received by the synchronization signal reception unit, and a synchronization signal reception unit that receives the synchronization signal and an infrared projector that can project a plurality of infrared lights. There is provided an imaging system comprising: a projection control unit that controls to project light automatically; and an imaging unit that images a subject in a state where infrared light is projected by the infrared projector.

  In addition, in order to solve the above-described problems of the related art, the present invention supplies a synchronization signal to each of a plurality of imaging devices from a synchronization signal supply device, and each of the plurality of imaging devices receives the synchronization signal. Then, an infrared projector capable of projecting a plurality of infrared lights is controlled to selectively emit infrared light at a timing based on the synchronization signal, and the infrared light is projected by the infrared projector. An imaging system control method is provided, in which an object is imaged by an imaging unit in a state where the imaging system is in the state of being captured.

  According to the imaging system and the control method of the imaging system of the present invention, it is possible to capture a preferable image even when there are a plurality of imaging devices that project infrared light and image a subject.

1 is a block diagram illustrating an overall configuration of an imaging apparatus according to a first embodiment. It is a figure which shows an example of the arrangement | sequence of the filter element in the color filter used for the imaging device of each embodiment. It is a characteristic view which shows the spectral sensitivity characteristic of the wavelength of three primary color lights and the relative sensitivity in the imaging part which comprises the imaging device of each embodiment. It is a characteristic view showing the relationship between the wavelength and the relative detection rate when the reflectance of the three primary colors from a predetermined substance is multiplied by the light receiving sensitivity of silicon. It is a block diagram which shows the specific structural example of the front signal process part 52 in FIG. It is a figure which shows schematically the relationship between exposure and the flame | frame of a video signal when the imaging device of each embodiment is operate | moving in normal mode. It is a figure for demonstrating a demosaic process when the imaging device of each embodiment is operate | moving in normal mode. It is a figure which shows roughly the relationship between exposure and the flame | frame of a video signal when the imaging device of each embodiment is operate | moving in intermediate | middle mode and night vision mode. It is a figure for demonstrating the pre-signal process when the imaging device of each embodiment is operate | moving in 1st intermediate mode. It is a figure for demonstrating a demosaic process when the imaging device of each embodiment is operate | moving in 1st intermediate | middle mode. It is a figure for demonstrating the pre-signal process when the imaging device of each embodiment is operate | moving in 2nd intermediate mode. It is a figure for demonstrating the demosaic process when the imaging device of each embodiment is operate | moving in 2nd intermediate mode. It is a figure for demonstrating the addition process of the surrounding pixel when the imaging device of each embodiment is operate | moving in night vision mode. It is a figure which shows the flame | frame in which the addition process of the surrounding pixel was performed. It is a figure for demonstrating the pre-signal process when the imaging device of each embodiment is operate | moving in 1st night-vision mode. It is a figure for demonstrating the demosaic process when the imaging device of each embodiment is operate | moving in 1st night-vision mode. It is a figure for demonstrating the front signal process when the imaging device of each embodiment is operate | moving in 2nd night-vision mode. It is a figure for demonstrating the demosaic process when the imaging device of each embodiment is operate | moving in 2nd night-vision mode. It is a figure for demonstrating the example of the mode switching in the imaging device of each embodiment. It is a figure which shows the state of each part when the imaging device of each embodiment is set to each mode. It is a partial block diagram which shows the 1st modification of the imaging device of each embodiment. It is a partial block diagram which shows the 2nd modification of the imaging device of each embodiment. It is a partial block diagram which shows the 3rd modification of the imaging device of each embodiment. It is a flowchart which shows the video signal processing method performed with the imaging device of each embodiment. It is a flowchart which shows the specific process of the normal mode shown to step S3 in FIG. It is a flowchart which shows the specific process of the intermediate | middle mode of step S4 in FIG. It is a flowchart which shows the specific process of night vision mode of step S5 in FIG. It is a flowchart which shows the process which a video signal processing program memorize | stored in the imaging device of each embodiment performs a computer. It is a block diagram showing an imaging system constituted by a plurality of imaging devices. It is a figure for demonstrating a problem when the light projection timing of the infrared light by a some imaging device has shifted | deviated slightly. It is a figure for demonstrating a problem when the light projection timing of the infrared rays by a several imaging device has shifted | deviated largely. It is a schematic block diagram which shows the state which made one side of the two imaging devices of 1st Embodiment the master, and made the other the slave. It is a figure for demonstrating the example of the synchronizing signal used with the imaging device of 1st-3rd embodiment. It is a figure for demonstrating the 1st example of operation | movement of a master and a slave when the imaging device of 1st-3rd embodiment is set to timing adjustment mode. FIG. 6 is a characteristic diagram illustrating how the luminance levels of R, G, and B change with respect to a shift in infrared light projection timing in the first example of the master and slave operations. It is a figure for demonstrating the 2nd example of operation | movement of a master and a slave when the imaging device of 1st-3rd embodiment is set to timing adjustment mode. In the 2nd example of operation of a master and a slave, it is a characteristic figure showing how a luminance level of R, G, and B changes with a shift of projection timing of infrared light. It is a block diagram which shows the whole structure of the imaging device of 2nd Embodiment. It is a schematic block diagram which shows the state which made one side of the two imaging devices of 2nd Embodiment the master and the other was a slave. It is a block diagram which shows the schematic structure of the imaging device of 3rd Embodiment. It is a block diagram which shows the whole structure of the imaging device of 4th Embodiment. It is a schematic block diagram which shows the state which set one side of the two imaging devices of 4th Embodiment as the master and the other as the slave. It is a figure for demonstrating operation | movement of a master and a slave when the imaging device of 4th-6th embodiment is set to timing adjustment mode. FIG. 11 is a characteristic diagram illustrating how the luminance levels of R, G, and B change with respect to a shift in infrared light projection timing when the imaging devices of the fourth to sixth embodiments are set to the timing adjustment mode. It is a flowchart which shows the adjustment method of the light projection timing of the infrared light performed when the imaging device of 4th-6th embodiment is set to timing adjustment mode. It is a block diagram which shows the schematic structure of the imaging device of 5th Embodiment. It is a block diagram which shows the schematic structure of the imaging device of 6th Embodiment. It is a block diagram which shows the whole structure of the imaging device used with the imaging system of 8th Embodiment. It is a block diagram which shows the schematic structure of the imaging system of 8th Embodiment. It is a figure which shows the relationship between the synchronizing signal and video signal in the imaging system of 8th Embodiment. It is a figure which shows the relationship between the synchronizing signal in the imaging system of 8th Embodiment, infrared light projection, and a video signal.

  Hereinafter, an imaging apparatus (imaging system) and an imaging apparatus (imaging system) control method according to each embodiment will be described with reference to the accompanying drawings.

<Configuration of Imaging Device of First Embodiment>
First, the overall configuration of the imaging apparatus 101 according to the first embodiment will be described with reference to FIG. The imaging apparatus 101 according to the first embodiment illustrated in FIG. 1 includes a normal mode suitable for an environment where there is sufficient visible light such as daytime, and a night vision mode suitable for an environment where there is almost no visible light such as nighttime. The imaging device can capture images in three modes, an intermediate mode suitable for an environment where slight visible light exists.

  Both the night vision mode and the intermediate mode are infrared light projection modes in which imaging is performed while projecting infrared light in an environment with little visible light. The infrared light projection mode may be only the night vision mode. In this embodiment, as a preferable configuration, an imaging device capable of imaging in three modes including the intermediate mode is taken as an example.

  In FIG. 1, the light indicated by the alternate long and short dash line reflected from the subject is collected by the optical lens 1. Here, the optical lens 1 has visible light in an environment in which visible light is sufficiently present, and infrared light in which an object reflects infrared light emitted from an infrared projector 9 described below in an environment with little visible light. Is incident.

  Under an environment where there is a slight amount of visible light, the optical lens 1 is incident with light in which visible light and infrared light emitted from the infrared projector 9 are reflected by the subject.

  In FIG. 1, only one optical lens 1 is shown for simplification, but actually, the imaging apparatus 101 includes a plurality of optical lenses.

  An optical filter 2 is provided between the optical lens 1 and the imaging unit 3. The optical filter 2 has two parts, an infrared cut filter 21 and a dummy glass 22. The optical filter 2 includes a state in which an infrared cut filter 21 is inserted between the optical lens 1 and the imaging unit 3 and a state in which a dummy glass 22 is inserted between the optical lens 1 and the imaging unit 3 by the driving unit 8. It is driven to either state.

  The imaging unit 3 includes an imaging element 31 in which a plurality of light receiving elements (pixels) are arranged in the horizontal direction and the vertical direction, and any one of red (R), green (G), and blue (B) corresponding to each light receiving element. And a color filter 32 in which filter elements of these colors are arranged. The image sensor 31 may be a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor).

  As an example, as shown in FIG. 2, the color filter 32 has R, G, and B filter elements arranged in an array called a Bayer array. The Bayer array is an example of a predetermined array of R, G, and B filter elements. In FIG. 2, the G filter element sandwiched between the R filter elements in each row is Gr, and the G filter element sandwiched between the B filter elements is Gb.

  In the Bayer array, horizontal rows in which R filter elements and Gr filter elements are alternately arranged and horizontal rows in which B filter elements and Gb filter elements are alternately arranged are vertically arranged. They are arranged alternately in the direction.

  FIG. 3 shows spectral sensitivity characteristics of the wavelengths of R light, G light, and B light and relative sensitivity in the imaging unit 3. The relative sensitivity is normalized to 1 at the maximum value. When the imaging apparatus 101 is operated in the normal mode, it is necessary to cut infrared light having a wavelength of 700 nm or more in order to capture a good color image by visible light.

  Therefore, the drive unit 8 drives the optical filter 2 so that the infrared cut filter 21 is inserted between the optical lens 1 and the imaging unit 3 based on control by the control unit 7.

  As can be seen from FIG. 3, the imaging unit 3 has sensitivity even in an infrared light region having a wavelength of 700 nm or more. Therefore, when the imaging apparatus 101 is operated in the intermediate mode or the night vision mode, the driving unit 8 removes the infrared cut filter 21 between the optical lens 1 and the imaging unit 3 based on the control by the control unit 7 and performs a dummy. The optical filter 2 is driven so that the glass 22 is inserted.

  In a state where the dummy glass 22 is inserted between the optical lens 1 and the imaging unit 3, infrared light having a wavelength of 700 nm or more is not cut. Therefore, the imaging apparatus 101 can obtain R, G, and B color information by using the sensitivity of the portion surrounded by the dashed ellipse in FIG. The reason why the dummy glass 22 is inserted is to make the optical path length the same as the optical path length when the infrared cut filter 21 is inserted.

  The infrared projector 9 includes projectors 91, 92, and 93 that project infrared light having wavelengths IR1, IR2, and IR3, respectively. In the intermediate mode or the night vision mode, the light projecting control unit 71 in the control unit 7 performs control so as to selectively project infrared light with wavelengths IR1 to IR3 from the light projecting units 91 to 93 in a time division manner. .

  Incidentally, a silicon wafer is used for the image sensor 31. FIG. 4 shows the relationship between the wavelength and the relative detection rate when the reflectance at each wavelength is multiplied by the light receiving sensitivity of silicon when a material exhibiting each color of R, G, and B is irradiated with white light. Yes. Also in FIG. 4, the maximum value of the relative detection rate is normalized to 1.

  As shown in FIG. 4, in the infrared light region, for example, the reflected light at a wavelength of 780 nm is highly correlated with the reflected light of a material exhibiting an R color, and the reflected light at a wavelength of 870 nm is a reflection of a material exhibiting a B color. Correlation with light is high, and reflected light at a wavelength of 940 nm is highly correlated with reflected light of a material exhibiting G color.

  Therefore, in this embodiment, the wavelengths IR1, IR2, and IR3 of the infrared light projected by the light projecting units 91, 92, and 93 are set to 780 nm, 940 nm, and 870 nm. These wavelengths are examples of the wavelengths IR1 to IR3, and may be other than 780 nm, 940 nm, and 870 nm.

  The light projecting unit 91 irradiates the subject with infrared light having the wavelength IR1, and assigns an image signal obtained by imaging the light reflected from the subject to the R signal. The light projecting unit 93 irradiates the subject with infrared light having a wavelength IR2, and assigns a video signal obtained by imaging the light reflected from the subject to the G signal. The light projecting unit 92 irradiates the subject with infrared light having a wavelength IR3, and assigns a video signal obtained by imaging the light reflected from the subject to the B signal.

  In this way, in principle, even in the intermediate mode or the night vision mode, it is possible to reproduce the same color as when the subject is imaged in an environment where visible light exists in the normal mode.

  Although the color image is different from the actual color of the subject, a wavelength IR1 of 780 nm may be assigned to the R light, a wavelength IR3 of 870 nm may be assigned to the G light, and a wavelength IR2 of 940 nm may be assigned to the B light. The wavelengths IR1, IR2, and IR3 can be arbitrarily assigned to the R light, G light, and B light.

  In the present embodiment, the wavelengths IR1, IR2, and IR3 that best reproduce the color of the subject are assigned to R light, G light, and B light, respectively.

  The control unit 7 controls imaging in the imaging unit 3 and each unit in the video processing unit 5. An imaging signal imaged by the imaging unit 3 is A / D converted by the A / D converter 4 and input to the video processing unit 5. The imaging unit 3 and the A / D converter 4 may be integrated. The video processing unit 5 and the control unit 7 may be integrated.

  The control unit 7 includes a mode switching unit 72 that switches between the normal mode, the intermediate mode, and the night vision mode. The mode switching unit 72 appropriately switches the operation in the video processing unit 5 as described later in correspondence with the normal mode, the intermediate mode, and the night vision mode. The mode switching unit 72 can be switched to a timing adjustment mode for adjusting the timing of projecting infrared light with wavelengths IR1, IR2, and IR3 by the infrared projector 9.

  Further, when the imaging unit is configured to include one imaging device 101 and another imaging device 101, the control unit 7 sets the imaging device 101 as a master imaging device or a slave imaging device. A master / slave setting unit 74 is provided.

  The master / slave setting unit 74 may select a master and a slave by a mechanical switch and hold a setting state of the selected master or slave. The master / slave setting unit 74 may select a master and a slave from a menu and hold the setting state of the selected master or slave.

  Further, the control unit 7 synchronizes the timing (phase) at which the infrared projector 9 of the imaging device 101 and the infrared projector 9 included in the other imaging device 101 project infrared light with wavelengths IR1, IR2, and IR3. The configuration is provided. Specifically, the control unit 7 includes a luminance level determination unit 75, a synchronization signal transmission unit 76t, and a synchronization signal reception unit 76r.

  When the imaging apparatus 101 is set as a master imaging apparatus, the synchronization signal transmission unit 76t transmits a synchronization signal Ssync based on the reference clock to another imaging apparatus 101. The synchronization signal Ssync is a signal based on the timing at which infrared light is projected in the imaging apparatus 101, for example. At this time, the synchronization signal receiving unit 76r does not operate.

  When the imaging apparatus 101 is set as a slave imaging apparatus, the synchronization signal receiving unit 76r receives the synchronization signal Ssync transmitted from the other imaging apparatus 101 that is the master. At this time, the synchronization signal transmission unit 76t does not operate. Further, the synchronization signal Ssync may not be received directly from the master imaging device, and may be received via another device.

  Specific operations of the luminance level determination unit 75, the synchronization signal transmission unit 76t, and the synchronization signal reception unit 76r will be described later.

  The video processing unit 5 includes switches 51 and 53, a previous signal processing unit 52, and a demosaic processing unit 54. The switches 51 and 53 may be physical switches, or may be conceptual switches for switching between operation and non-operation of the previous signal processing unit 52. A video signal is input from the video processing unit 5 to the control unit 7 in order to detect the brightness of the video being captured.

  As illustrated in FIG. 5, the previous signal processing unit 52 includes a surrounding pixel addition unit 521, a same-position pixel addition unit 522, and a synthesis unit 523.

  The video processing unit 5 generates R, G, and B primary color data and supplies them to the video output unit 6. The video output unit 6 outputs the three primary color data in a predetermined format to a display unit not shown.

  The video output unit 6 may output the R, G, and B signals as they are, or may convert the R, G, and B signals into luminance signals and color signals (or color difference signals) and output them. The video output unit 6 may output a composite video signal. The video output unit 6 may output a digital video signal or may output a video signal converted into an analog signal by a D / A converter.

  Hereinafter, specific operations in the normal mode, the intermediate mode, and the night vision mode will be described.

<Normal mode>
In the normal mode, the control unit 7 causes the driving unit 8 to insert the infrared cut filter 21 between the optical lens 1 and the imaging unit 3. The light projection control unit 71 turns off infrared light projection by the infrared projector 9.

  The image signal picked up by the image pickup unit 3 is converted into video data which is a digital signal by the A / D converter 4 and input to the video processing unit 5. In the normal mode, the mode switching unit 72 controls the switches 51 and 53 to be connected to the terminal Tb.

  FIG. 6A shows exposures Ex1, Ex2, Ex3,... Actually, the exposure time varies depending on conditions such as the shutter speed. Here, the exposures Ex1, Ex2, and Ex3 indicate the maximum exposure time.

  FIG. 6B shows the timing at which each video signal frame is obtained. Based on an exposure (not shown) before the exposure Ex1, a frame F0 of the video signal is obtained after a predetermined time. Based on the exposure Ex1, a frame F1 of the video signal is obtained after a predetermined time. Based on the exposure Ex2, a frame F2 of the video signal is obtained after a predetermined time. The same applies to exposure Ex3 and thereafter. The frame frequency of the video signal is set to 30 frames / second, for example.

  The frame frequency of the video signal may be set as appropriate, such as 30 frames / second or 60 frames / second for the NTSC system and 25 frames / second or 50 frames / second for the PAL system. The frame frequency of the video signal may be 24 frames / second used in movies.

  The video data of each frame output from the A / D converter 4 is input to the demosaic processing unit 54 via the switches 51 and 53. The demosaic processing unit 54 performs demosaic processing on the input video data of each frame. The video processing unit 5 performs various video processing such as white balance correction and gain correction in addition to demosaic processing, and outputs R, G, and B primary color data.

  The demosaic processing in the demosaic processing unit 54 will be described with reference to FIG. 7A shows an arbitrary frame Fm of video data. The frame Fm is a frame composed of pixels in the effective video period. The number of pixels of the video data is, for example, horizontal 640 pixels and vertical 480 pixels in the VGA standard. Here, for simplification, the number of pixels of the frame Fm is significantly reduced, and the frame Fm is conceptually illustrated.

  The video data generated using the Bayer array imaging unit 3 is data in which R, G, and B pixel data are mixed in the frame Fm. The demosaic processing unit 54 generates R interpolation pixel data Ri obtained by calculating R pixel data at a pixel position where no R pixel data is present using surrounding R pixel data. The demosaic processing unit 54 generates an R frame FmR in which all the pixels in one frame shown in FIG. 7B are made up of R pixel data.

  The demosaic processing unit 54 generates G interpolation pixel data Gi obtained by calculating G pixel data at a pixel position where no G pixel data exists using surrounding G pixel data. The demosaic processing unit 54 generates a G frame FmG in which all the pixels of one frame shown in FIG.

  The demosaic processing unit 54 generates B interpolated pixel data Bi obtained by calculating B pixel data at a pixel position where no B pixel data exists using surrounding B pixel data. The demosaic processing unit 54 generates a B frame FmB in which all the pixels of one frame shown in FIG.

  The demosaic processing unit 54 may use at least R pixel data when interpolating R pixel data, may use at least G pixel data when interpolating G pixel data, and B pixel data. When interpolation is performed, at least B pixel data may be used. In order to improve interpolation accuracy, the demosaic processing unit 54 may use pixel data of another color different from the color of the interpolation pixel data to be generated when interpolating R, G, and B pixel data. Good.

  Since there are pixels outside the effective video period in the imaging unit 3, R, G, and B pixel data can be interpolated even in pixels located at the upper, lower, left, and right ends of the frame Fm.

  The R frame FmR, G frame FmG, and B frame FmB generated by the demosaic processing unit 54 are output as R, G, and B primary color data. In FIG. 7, the R, G, and B pixel data has been described in units of frames in order to facilitate understanding, but actually, the R, G, and B pixel data are sequentially output for each pixel.

<Intermediate mode: first intermediate mode>
In the intermediate mode (first intermediate mode and second intermediate mode described later), the control unit 7 causes the driving unit 8 to insert the dummy glass 22 between the optical lens 1 and the imaging unit 3. The light projection controller 71 turns on the infrared light projection by the infrared light projector 9. The mode switching unit 72 controls the switches 51 and 53 to be connected to the terminal Ta.

  FIG. 8A shows a state of infrared light projection by the infrared projector 9. The control unit 7 divides one frame period of the normal mode into 1/3, and controls so as to project infrared light in the order of the light projecting units 91, 92, 93, for example.

  In the example shown in FIG. 8A, infrared light having a wavelength IR1 (780 nm) is irradiated onto the subject in the first ３ period of one frame. In the next 1/3 period of one frame, the subject is irradiated with infrared light having a wavelength IR2 (940 nm). In the last 1/3 period of one frame, the subject is irradiated with infrared light having a wavelength IR3 (870 nm). The order of projecting infrared light with wavelengths IR1 to IR3 is arbitrary. However, the order of wavelengths IR1, IR2, IR3 is the best.

  As shown in FIG. 8B, at the timing when infrared light having the wavelength IR1 is projected, the imaging unit 3 performs exposure Ex1R having a high correlation with the R light. At the timing when the infrared light having the wavelength IR2 is projected, the imaging unit 3 performs the exposure Ex1G having a high correlation with the G light. At the timing when the infrared light having the wavelength IR3 is projected, the imaging unit 3 performs exposure Ex1B having a high correlation with the B light.

  However, in the intermediate mode, since imaging is performed in an environment where there is a small amount of visible light, visible light and infrared light projected from the infrared projector 9 are mixed. Therefore, in the intermediate mode, the exposures Ex1R, Ex1G, Ex1B, Ex2R, Ex2G, Ex2B... Are exposures that combine exposure with visible light and exposure with infrared light.

  As shown in FIG. 8C, based on the exposures Ex1R, Ex1G, and Ex1B, a frame F1IR1 corresponding to the exposure Ex1R, a frame F1IR3 corresponding to the exposure Ex1G, and a frame F1IR2 corresponding to the exposure Ex1B are obtained after a predetermined time. It is done.

  Further, based on the exposures Ex2R, Ex2G, and Ex2B, a frame F2IR1 corresponding to the exposure Ex2R, a frame F2IR3 corresponding to the exposure Ex2G, and a frame F2IR2 corresponding to the exposure Ex2B are obtained after a predetermined time. The same applies to exposures Ex3R, Ex3G, and Ex3B.

  The frame frequency of the imaging signal in FIG. 8C is 90 frames / second. In the intermediate mode, since one frame of the video signal in the normal mode is time-divided and infrared light of wavelengths IR1 to IR3 is projected, in order to output a video signal in the same format as in the normal mode, ( The frame frequency of the imaging signal in c) is three times the frame frequency in the normal mode.

  As will be described later, one frame of a video signal having a frame frequency of 30 frames / second shown in FIG. 8D is generated based on the imaging signal of 3 frames shown in FIG. For example, a frame F1IR is generated based on the frames F1IR1, F1IR2, and F1IR3, and a frame F2IR is generated based on the frames F2IR1, F2IR2, and F2IR3.

  The operation in the intermediate mode for generating the video signal of each frame in FIG. 8D based on the three frames of the imaging signal in FIG. 8C will be specifically described.

  Video data of each frame corresponding to the imaging signal shown in FIG. 8C output from the A / D converter 4 is input to the previous signal processing unit 52 via the switch 51.

  The previous signal processing in the previous signal processing unit 52 will be described with reference to FIG. FIG. 9A shows an arbitrary frame FmIR1 of the video data generated at the timing when the infrared light having the wavelength IR1 is projected. The R, B, Gr, and Gb pixel data in the frame FmIR1 is attached with a subscript 1 indicating that it is generated in a state where infrared light having the wavelength IR1 is projected.

  FIG. 9B shows an arbitrary frame FmIR2 of video data generated at the timing when infrared light having a wavelength IR2 is projected. The R, B, Gr, and Gb pixel data in the frame FmIR2 is attached with a subscript 2 indicating that it is generated in a state where infrared light having a wavelength IR2 is projected.

  FIG. 9C shows an arbitrary frame FmIR3 of the video data generated at the timing when the infrared light having the wavelength IR3 is projected. The R, B, Gr, and Gb pixel data in the frame FmIR3 has a subscript 3 indicating that it is generated in a state where infrared light having the wavelength IR3 is projected.

  The frame FmIR1 shown in FIG. 9A is video data generated in a state where infrared light having a wavelength IR1 having a high correlation with the R light is projected. Therefore, the R pixel data is projected. The pixel data corresponding to the infrared light and the pixel data B and G are pixel data not corresponding to the projected infrared light. The hatching attached to the B, Gr, and Gb pixel data means that the pixel data does not correspond to the projected infrared light.

  The frame FmIR2 shown in FIG. 9B is video data generated in a state where infrared light having a wavelength IR2 having a high correlation with the G light is projected, so that the G pixel data is projected. The pixel data corresponding to the infrared light, and the R and B pixel data are pixel data not corresponding to the projected infrared light. The hatching attached to the R and B pixel data means that the pixel data does not correspond to the projected infrared light.

  The frame FmIR3 shown in FIG. 9C is video data generated in a state where infrared light having a wavelength IR3 having a high correlation with the B light is projected, so that the B pixel data is projected. The pixel data corresponding to the infrared light, and the R and G pixel data are pixel data not corresponding to the projected infrared light. The hatching attached to the R, Gr, and Gb pixel data means pixel data that does not correspond to the projected infrared light.

  The same-position pixel addition unit 522 in the previous signal processing unit 52 individually adds R, Gr, Gb, and B pixel data at the same pixel position according to the following formulas (1) to (3), and adds pixels Data R123, Gr123, Gb123, and B123 are generated. In the intermediate mode, the surrounding pixel addition unit 521 in the previous signal processing unit 52 does not operate.

R123 = ka × R1 + kb × R2 + kc × R3 (1)
G123 = kd × G1 + ke × G2 + kf × G3 (2)
B123 = kg × B1 + kh × B2 + ki × B3 (3)

  In the equations (1) to (3), R1, G1, and B1 are R, G, and B pixel data in the frame FmIR1, R2, G2, and B2 are R, G, and B pixel data in the frame FmIR2, R3, G3, B3 is R, G, B pixel data in the frame FmIR3. ka to ki are predetermined coefficients. G123 in Formula (2) is Gr123 or Gb123.

  At this time, the same-position pixel adding unit 522 adds R, Gr, Gb, and B pixel data at the same pixel position with hatching to the respective pixel data with R, Gr, Gb, and B that are not hatched. Is added.

  That is, the same-position pixel adding unit 522 adds the R pixel data at the same pixel position in the frames FmIR2 and FmIR3 to the R pixel data in the frame FmIR1 based on the equation (1), and adds the added pixel data R123. Generate. In other words, the addition pixel data R123 for red is generated using only the pixel data of the region corresponding to the red color filter in the light receiving element.

  The same-position pixel addition unit 522 adds the pixel data of Gr and Gb at the same pixel position in the frames FmIR1 and FmIR3 to the pixel data of Gr and Gb in the frame FmIR2 based on the equation (2), and adds pixel data Generate G123. That is, the green summed pixel data G123 is generated using only the pixel data of the region corresponding to the green color filter in the light receiving element.

  The same-position pixel addition unit 522 adds the B pixel data at the same pixel position in the frames FmIR1 and FmIR2 to the B pixel data in the frame FmIR3 based on the equation (3) to generate the added pixel data B123. . That is, the blue addition pixel data B123 is generated using only the pixel data of the region corresponding to the blue color filter in the light receiving element.

  The combining unit 523 in the previous signal processing unit 52 generates a frame FmIR123 of the combined video signal shown in (d) of FIG. 9 based on the added pixel data R123, Gr123, Gb123, and B123 generated at each pixel position. To do.

  Specifically, the synthesis unit 523 selects and synthesizes the addition pixel data R123 in the frame FmIR1, the addition pixel data Gr123 and Gb123 in the frame FmIR2, and the addition pixel data B123 in the frame FmIR3. Thereby, the synthesis unit 523 generates a frame FmIR123 of the synthesized video signal.

  In this way, the synthesis unit 523 generates a frame FmIR123 in which the addition pixel data R123, Gr123, Gb123, and B123 are arranged so that the arrangement is the same as the arrangement of the filter elements in the color filter 32.

  In the first intermediate mode, video data of the frame FmIR123 is generated using pixel data that is not hatched and pixel data that is hatched.

  The reason why the pixel data at the same pixel position is added by the same position pixel addition unit 522 is as follows. Since the imaging is performed in an environment where visible light is present in the intermediate mode, the hatched pixel data includes respective color components based on exposure with visible light. Therefore, the sensitivity of each color can be increased by adding pixel data at the same pixel position.

  If the visible light and infrared light are mixed and there is a relatively large amount of visible light, exposure with visible light becomes dominant. In this case, the video data of the frame FmIR123 mainly includes components based on the video signal exposed with visible light. If visible light and infrared light are mixed and there is a relatively large amount of infrared light, exposure with infrared light becomes dominant. In this case, the video data of the frame FmIR123 mainly includes components based on the video signal exposed with infrared light.

  When visible light is relatively small, the relationship between the coefficients ka, kb, and kc in equation (1) is ka> kb, kc, and the relationship between the coefficients kd, ke, and kf in equation (2). , Kf> kd, ke, and in equation (3), the magnitude relationship between the coefficients kg, kh, ki is preferably kh> kg, ki. This is because the wavelength IR1 has a high correlation with the R light, the wavelength IR2 has a high correlation with the G light, and the wavelength IR3 has a high correlation with the B light.

  In this way, the R pixel data is mainly the R pixel data in the frame FmIR1, the G pixel data is the G pixel data in the frame FmIR2, and the B pixel data is mainly the B pixel data in the frame FmIR3. it can.

  The video data of the frame FmIR123 output from the previous signal processing unit 52 is input to the demosaic processing unit 54 via the switch 53. The demosaic processing unit 54 performs demosaic processing on the video data of the input frame FmIR123, as in the normal mode. The video processing unit 5 performs various video processing such as white balance correction and gain correction in addition to demosaic processing, and outputs R, G, and B primary color data.

  The demosaic process in the demosaic processing unit 54 will be described with reference to FIG. FIG. 10A shows a frame FmIR123. The demosaic processing unit 54 calculates the R pixel data at the pixel position where the R pixel data does not exist using the surrounding R pixel data, and generates the R interpolation pixel data R123i. The demosaic processing unit 54 generates an R frame FmIR123R in which all the pixels in one frame shown in FIG.

  The demosaic processing unit 54 calculates the G pixel data at the pixel position where the G pixel data does not exist using the surrounding G pixel data, and generates the G interpolation pixel data G123i. The demosaic processing unit 54 generates a G frame FmIR123G in which all the pixels of one frame shown in FIG.

  The demosaic processing unit 54 calculates the B pixel data at the pixel position where the B pixel data does not exist using the surrounding B pixel data, and generates the B interpolation pixel data B123i. The demosaic processing unit 54 generates a B frame FmIR123B in which all the pixels in one frame shown in FIG.

  As can be understood by comparing the operation of the demosaic processing unit 54 shown in FIG. 7 in the normal mode with the operation of the demosaic processing unit 54 shown in FIG. 10 in the intermediate mode, they are substantially the same. The operation of the demosaic processing unit 54 does not change regardless of whether it is the normal mode or the intermediate mode.

  In the normal mode, the previous signal processing unit 52 is disabled, and in the intermediate mode, the previous signal processing unit 52 may be operated except for the surrounding pixel adding unit 521. In the normal mode and the intermediate mode, the demosaic processing unit 54 in the video processing unit 5 and the signal processing unit such as white balance correction and gain correction can be shared.

<Intermediate mode: second intermediate mode>
The operation in the second intermediate mode will be described with reference to FIGS. In the operation in the second intermediate mode, the description of the same part as the operation in the first intermediate mode is omitted. The frames FmIR1, FmIR2, and FmIR3 in (a) to (c) of FIG. 11 are the same as the frames FmIR1, FmIR2, and FmIR3 in (a) to (c) of FIG.

  The combining unit 523 selects and combines R1 which is R pixel data in the frame FmIR1, Gr2 and Gb2 which are G pixel data in the frame FmIR2, and B3 which is B pixel data in the frame FmIR3. As a result, the synthesis unit 523 generates a frame FmIR123 'of the synthesized video signal shown in FIG.

  That is, the frame FmIR123 'is video data obtained by collecting R, Gr, Gb, and B pixel data not hatched in the frames FmIR1, FmIR3, and FmIR2 into one frame.

  That is, in the frame FmIR123 ′, the pixel data for red using only the pixel data in the region corresponding to the red color filter in the state where the infrared light having the wavelength IR1 is projected, and the infrared light having the wavelength IR2 are projected. Only the pixel data for the green color using only the pixel data of the region corresponding to the green color filter in the selected state, and the pixel data of the region corresponding to the blue color filter in the state of projecting the infrared light having the wavelength IR3 are used. The pixel data is for blue.

  As described above, the synthesis unit 523 generates a frame FmIR123 'in which the pixel data R1, Gr2, Gb2, and B3 are arranged so that the arrangement is the same as the arrangement of the filter elements in the color filter 32.

  In the second intermediate mode, the same-position pixel adding unit 522 sets the coefficient ka in Equation (1) to 1, the coefficients kb and kc to 0, the coefficient ke in Equation (2) to 1, the coefficients kd and kf to 0, In equation (3), the coefficient ki is 1 and the coefficients kg and kh are 0.

  As a result, the R pixel data in the frame FmIR1, the Gr and Gb pixel data in the frame FmIR2, and the B pixel data in the frame FmIR3 have the same values.

  Therefore, similarly to the operation in the first intermediate mode, the combining unit 523 selects the R pixel data in the frame FmIR1, the Gr and Gb pixel data in the frame FmIR2, and the B pixel data in the frame FmIR3. FmIR123 'can be generated.

  In the second intermediate mode, the front signal processing unit 52 generates pixel data (hatched) with infrared light projected to generate pixel data having the same color as the pixel data. Video data of the frame FmIR123 ′ is generated using only (non-pixel data).

  According to the second intermediate mode, the sensitivity and color reproducibility are lower than those in the first intermediate mode, but the arithmetic processing can be simplified and the frame memory can be reduced.

  The demosaic process in the demosaic processing unit 54 will be described with reference to FIG. FIG. 12A shows the frame FmIR123 '. The demosaic processing unit 54 calculates the R pixel data at the pixel position where the R pixel data does not exist using the surrounding R pixel data, and generates the R interpolation pixel data R1i. The demosaic processing unit 54 generates an R frame FmIR123′R in which all the pixels in one frame shown in FIG. 12B are made up of R pixel data.

  The demosaic processing unit 54 calculates G pixel data at a pixel position where no G pixel data exists using the surrounding G pixel data, and generates G interpolation pixel data G2i. The demosaic processing unit 54 generates a G frame FmIR123′G in which all the pixels in one frame shown in FIG.

  The demosaic processing unit 54 calculates the B pixel data at the pixel position where the B pixel data does not exist using the surrounding B pixel data, and generates the B interpolation pixel data B3i. The demosaic processing unit 54 generates a B frame FmIR123′B in which all the pixels of one frame shown in FIG.

  As described above, in the intermediate mode, the pixel data for red is generated from the pixel data obtained from the region corresponding to the red color filter in the light receiving element, and obtained from the region corresponding to the green color filter in the light receiving element. Green pixel data is generated from the pixel data, and blue pixel data is generated from the pixel data obtained from the region corresponding to the blue color filter in the light receiving element.

<Night Vision Mode: First Night Vision Mode>
In the night-vision mode (the first night-vision mode and the second night-vision mode described later), as in the intermediate mode, the control unit 7 causes the drive unit 8 to place the dummy glass 22 between the optical lens 1 and the imaging unit 3. Insert it. The light projection controller 71 turns on the infrared light projection by the infrared light projector 9. The mode switching unit 72 controls the switches 51 and 53 to be connected to the terminal Ta.

  The schematic operation in the night vision mode is the same as that in FIG. However, in the night vision mode, since the imaging is performed in an environment where there is almost no visible light, the exposures Ex1R, Ex1G, Ex1B, Ex2R, Ex2G, Ex2B... In FIG. Is assumed.

  In an environment where there is almost no visible light and only infrared light, there is no difference in the characteristics of the filter elements in the color filter 32, so the imaging unit 3 can be regarded as a monochromatic imaging device. .

  Therefore, in the night vision mode, the surrounding pixel addition unit 521 in the previous signal processing unit 52 adds pixel data located in the periphery to each pixel data in order to improve the sensitivity of infrared light.

  Specifically, as illustrated in FIG. 13A, when the R pixel is the target pixel, the surrounding pixel addition unit 521 includes the G and B positioned around the R pixel data of the target pixel. The pixel data of 8 pixels are added.

  In other words, in the intermediate mode, red pixel data was generated from the pixel data obtained from the region corresponding to the red color filter in the light receiving element, but in the night vision mode, a wider area than in the intermediate mode. Pixel data for red is generated from the pixel data obtained from the above. In the example of FIG. 13, pixel data obtained from an area corresponding to nine pixels including the target pixel is used for each color.

  As illustrated in FIG. 13B, when the G pixel is the target pixel, the surrounding pixel adding unit 521 includes the eight G and B pixels positioned around the G pixel data of the target pixel. Add data. The G pixel in FIG. 13B is a Gr or Gb pixel.

  In other words, in the intermediate mode, the pixel data for green was generated from the pixel data obtained from the area corresponding to the green color filter in the light receiving element, but in the night vision mode, a wider area than in the intermediate mode. The pixel data for green is generated from the pixel data obtained from the above.

  As illustrated in FIG. 13C, when the B pixel is the target pixel, the surrounding pixel adding unit 521 includes the R and G pixels located in the periphery with respect to the B pixel data of the target pixel. Add data.

  In other words, in the intermediate mode, blue pixel data was generated from the pixel data obtained from the region corresponding to the blue color filter in the light receiving element, but in the night vision mode, a wider area than in the intermediate mode. The pixel data for blue is generated from the pixel data obtained from the above.

  The surrounding pixel adding unit 521 may simply add 9 pixels of the pixel data of the target pixel and the surrounding 8 pixel pixel data, or may apply a predetermined weight to the surrounding 8 pixel pixel data. May be added to the pixel data of the pixel of interest.

  The weighting for the pixel data of the surrounding 8 pixels may be partially 0. That is, the addition of the pixel data of the surrounding pixels to the pixel data of the target pixel in the surrounding pixel addition unit 521 is not limited to adding all the pixel data of 8 pixels.

  For example, the surrounding pixel addition unit 521 does not need to add pixel data of pixels positioned in the diagonal direction and add pixel data of pixels positioned in the vertical and horizontal directions to the pixel data of the target pixel.

  By the way, there is an image pickup device that can read out a plurality of pixels called binning as a single pixel. When an image sensor having a binning function is used as the image sensor 31, an addition process by an image sensor having a binning function may be performed instead of the addition process by the surrounding pixel addition unit 521. Binning by the image sensor is substantially equivalent to addition processing by the surrounding pixel addition unit 521.

  The frames FmIR1, FmIR3, and FmIR2 in (a) to (c) of FIG. 14 are the same as the frames FmIR1, FmIR3, and FmIR2 in (a) to (c) of FIG. In (d) to (f) of FIG. 14, R1ad, Gr1ad, Gb1ad, B1ad, R2ad, Gr2ad, Gb2ad, B2ad, R3ad, Gr3ad, Gb3ad, and B3ad are converted into R, Gr, Gb, and B pixel data, respectively. On the other hand, it is added pixel data obtained by adding pixel data of surrounding eight pixels.

  The surrounding pixel addition unit 521 performs addition processing shown in FIG. 13 on the pixel data of the frames FmIR1, FmIR3, and FmIR2, thereby performing frames FmIR1ad, FmIR2ad, and FmIR3ad shown in (d) to (f) of FIG. Is generated.

  The frames FmIR1ad, FmIR2ad, and FmIR3ad in (a) to (c) of FIG. 15 are the same as the frames FmIR1ad, FmIR2ad, and FmIR3ad in (d) to (f) of FIG.

  Similarly to the first intermediate mode, the same-position pixel addition unit 522 adds the pixel data of R2ad and R3ad at the same pixel position in the frames FmIR2ad and FmIR3ad to the pixel data of R1ad in the frame FmIR1ad based on Expression (1). Then, the addition pixel data R123ad is generated.

  The same-position pixel addition unit 522 adds the pixel data of Gr1ad, Gb1ad, Gr3ad, and Gb3ad at the same pixel position in the frames FmIR1ad and FmIR3ad to the pixel data of Gr2ad and Gb2ad in the frame FmIR2ad based on Expression (2). The addition pixel data Gr123ad and Gb123ad are generated.

  Based on Expression (3), the same-position pixel addition unit 522 adds the pixel data of B1ad and B2ad at the same pixel position in the frames FmIR1ad and FmIR2ad to the pixel data of B3ad in the frame FmIR3ad to obtain the added pixel data B123ad. Generate.

  Similar to the first intermediate mode, the combining unit 523 selects and combines the addition pixel data R123ad in the frame FmIR1ad, the addition pixel data Gr123ad and Gb123ad in the frame FmIR2ad, and the addition pixel data B123ad in the frame FmIR3ad. As a result, the synthesis unit 523 generates a frame FmIR123ad of the synthesized video signal shown in FIG.

  The synthesizing unit 523 generates a frame FmIR123ad in which the addition pixel data R123ad, Gr123ad, Gb123ad, and B123ad are arranged so that the arrangement is the same as the arrangement of the filter elements in the color filter 32.

  FIG. 16A shows a frame FmIR123ad. The demosaic processing unit 54 calculates the R pixel data at the pixel position where the R pixel data does not exist using the surrounding R pixel data, and generates the R interpolation pixel data R123adi. The demosaic processing unit 54 generates an R frame FmIR123adR in which all pixels in one frame shown in FIG. 16B are made up of R pixel data.

  The demosaic processing unit 54 calculates the G pixel data at the pixel position where the G pixel data does not exist using the surrounding G pixel data, and generates the G interpolation pixel data G123adi. The demosaic processing unit 54 generates a G frame FmIR123adG in which all the pixels in one frame shown in FIG.

  The demosaic processing unit 54 calculates the B pixel data at the pixel position where the B pixel data does not exist using the surrounding B pixel data, and generates the B interpolation pixel data B123adi. The demosaic processing unit 54 generates a B frame FmIR123adB in which all the pixels in one frame shown in FIG.

  The first intermediate mode and the first night-vision mode are different in that the former does not operate the surrounding pixel adding unit 521 while the latter operates the surrounding pixel adding unit 521. The mode switching unit 72 may operate the surrounding pixel addition unit 521 in the night vision mode.

  The operation of the demosaic processing unit 54 in the night vision mode is substantially the same as the operation of the demosaic processing unit 54 in the normal mode and the intermediate mode. In the normal mode, the intermediate mode, and the night vision mode, the demosaic processing unit 54 in the video processing unit 5 and a signal processing unit such as white balance correction and gain correction can be shared.

<Night Vision Mode: Second Night Vision Mode>
The operation in the second night-vision mode will be described with reference to FIGS. In the operation in the second night vision mode, the description of the same part as the operation in the first night vision mode is omitted. The frames FmIR1ad, FmIR2ad, and FmIR3ad in (a) to (c) of FIG. 17 are the same as the FmIR1ad, FmIR2ad, and FmIR3ad in (a) to (c) of FIG.

  The combining unit 523 selects and combines R1ad that is R pixel data in the frame FmIR1ad, Gr2ad and Gb2ad that are G pixel data in the frame FmIR2, and B3ad that is B pixel data in the frame FmIR3. Thereby, the synthesis unit 523 generates a frame FmIR123′ad of the synthesized video signal shown in FIG.

  The synthesizing unit 523 generates a frame FmIR123′ad in which the addition pixel data R1ad, Gr2ad, Gb2ad, and B3ad are arranged so that the arrangement is the same as the arrangement of the filter elements in the color filter 32.

  As described with reference to FIG. 13, the red addition pixel data R1ad in the frame FmIR123′ad is obtained from a wider area than the area used for generating the red pixel data in the intermediate mode. It is generated from the pixel data.

  In addition, the additional pixel data Gr2ad for green in the frame FmIR123'ad is generated from pixel data obtained from an area larger than the area used for generating the pixel data for green in the intermediate mode. ing.

  Furthermore, the additional pixel data B3ad for blue in the frame FmIR123'ad is generated from pixel data obtained from an area larger than the area used for generating the pixel data for blue in the intermediate mode. ing.

  In the second night-vision mode, as in the second intermediate mode, the same-position pixel addition unit 522 sets the coefficient ka in equation (1) to 1, the coefficients kb and kc to 0, and the coefficient ke in equation (2) to 1. The coefficients kd and kf are set to 0, the coefficient ki in the equation (3) is set to 1, and the coefficients kg and kh are set to 0.

  Thereby, the pixel data of R1ad in the frame FmIR1ad, the pixel data of Gr2ad and Gb2ad in the frame FmIR2ad, and the pixel data of B3ad in the frame FmIR3ad have the same values.

  Therefore, as in the operation in the first night vision mode, the combining unit 523 selects the pixel data of R1ad in the frame FmIR1ad, the pixel data of Gr2ad and Gb2ad in the frame FmIR2ad, and the pixel data of B3ad in the frame FmIR3ad. A frame FmIR123'ad can be generated.

  The demosaic process in the demosaic processing unit 54 will be described with reference to FIG. FIG. 18A shows a frame FmIR123′ad. The demosaic processing unit 54 calculates the R pixel data at the pixel position where the R pixel data does not exist using the surrounding R1ad pixel data, and generates the R interpolation pixel data R1adi. The demosaic processing unit 54 generates an R frame FmIR123′adR in which all the pixels of one frame shown in FIG. 18B are made up of R pixel data.

  The demosaic processing unit 54 calculates the G pixel data at the pixel position where the G pixel data does not exist using the surrounding Gr2ad and Gb2ad pixel data, and generates the G interpolation pixel data G2adi. The demosaic processing unit 54 performs interpolation to generate a G frame FmIR123′adG in which all the pixels of one frame shown in FIG.

  The demosaic processing unit 54 generates B interpolation pixel data B3adi obtained by calculating B pixel data at a pixel position where no B pixel data exists using surrounding B3ad pixel data. The demosaic processing unit 54 generates a B frame FmIR123′adB in which all the pixels in one frame shown in FIG.

  The second intermediate mode and the second night-vision mode differ in that the former does not operate the surrounding pixel adding unit 521 while the latter operates the surrounding pixel adding unit 521.

  Further, in the intermediate mode, pixel data for each color is generated from each pixel data obtained from the region corresponding to each color in the light receiving element, but in the night vision mode, the surrounding pixels are added. It can also be said that pixel data for each color is generated from pixel data obtained from a region wider than each region for generating pixel data for each color.

<Example of mode switching>
An example of mode switching by the mode switching unit 72 will be described with reference to FIG. FIG. 19A schematically shows, as an example, how the brightness of the surrounding environment changes as time passes from the daytime period to the nighttime period.

  As shown in FIG. 19 (a), the brightness decreases as time elapses from daytime to evening, and it becomes almost dark after time t3. The brightness shown in FIG. 19A substantially indicates the amount of visible light, and there is almost no visible light after time t3.

  The control unit 7 can determine the brightness of the surrounding environment based on the luminance level of the video signal (video data) input from the video processing unit 5. As shown in FIG. 19B, the mode switching unit 72 sets the normal mode when the brightness is equal to or higher than a predetermined threshold Th1 (first threshold), and sets the predetermined threshold Th2 (less than the threshold Th1). When it is equal to or greater than the second threshold), the intermediate mode is selected.

  The imaging apparatus 101 according to the present embodiment switches the mode to the normal mode from time t1 when the brightness reaches the threshold value Th1, to the intermediate mode from time t1 to time t2 when the brightness reaches the threshold value Th2, and to the night vision mode after time t2. Switch automatically. In FIG. 19B, the intermediate mode may be either the first intermediate mode or the second intermediate mode, and the night vision mode may be either the first night vision mode or the second night vision mode.

  In FIG. 19A, the brightness immediately before time t3 when almost no visible light is present is set as the threshold value Th2, but the brightness at time t3 may be set as the threshold value Th2.

  As shown in FIG. 19 (c), the mode switching unit 72 has a period of time t1 in which the visible light is relatively large in the intermediate mode period, the first intermediate mode, and a period of time t2 in which the visible light is relatively small. The period may be set to the second intermediate mode. In FIG. 19C, the night-vision mode may be either the first night-vision mode or the second night-vision mode.

  In the imaging apparatus 101 of the present embodiment, the light projection control unit 71 controls on / off of the infrared projector 9, and the mode switching unit 72 switches the operation / non-operation of each unit in the video processing unit 5. A mode can be realized.

  As shown in FIG. 20, in the normal mode, the infrared projector 9 is off, the surrounding pixel adding unit 521, the same position pixel adding unit 522 and the synthesizing unit 523 are not operating, and the demosaic processing unit 54 is operating.

  In the first intermediate mode, the infrared projector 9 is on, the surrounding pixel adding unit 521 is not operating, and the same-position pixel adding unit 522, the combining unit 523, and the demosaic processing unit 54 are operating. In the second intermediate mode, the infrared projector 9 is on, the surrounding pixel adding unit 521 and the same-position pixel adding unit 522 are not operating, and the synthesizing unit 523 and the demosaic processing unit 54 are operating.

  The operation and non-operation in the same position pixel addition unit 522 can be easily switched by appropriately setting the values of the coefficients ka to ki in the expressions (1) to (3) as described above.

  In the first night-vision mode, the infrared projector 9 is on, and the surrounding pixel addition unit 521, the same-position pixel addition unit 522, the synthesis unit 523, and the demosaic processing unit 54 are all in an operating state. In the second night-vision mode, the infrared projector 9 is on, the same-position pixel adding unit 522 is not operating, and the surrounding pixel adding unit 521, the combining unit 523, and the demosaic processing unit 54 are operating.

  By the way, the surrounding pixel adding unit 521 sets the coefficient multiplied by the surrounding pixel data to a coefficient exceeding 0 (for example, 1) in the calculation formula for adding the surrounding pixel data to the pixel data of the target pixel. The surrounding pixel addition processing can be put into an operation state.

  Further, the surrounding pixel adding unit 521 can set the surrounding pixel addition processing to the non-operating state by setting the coefficient by which the surrounding pixel data is multiplied to 0 in the calculation formula.

  The operation and non-operation of the surrounding pixel addition unit 521 can be easily switched by appropriately setting the coefficient value.

<First Modification of Imaging Device of First Embodiment>
The method by which the control unit 7 detects the brightness of the surrounding environment is not limited to the method based on the luminance level of the video signal.

  As shown in FIG. 21, the brightness sensor 11 may detect the brightness of the surrounding environment. In FIG. 21, the brightness of the surrounding environment may be determined based on both the luminance level of the video signal and the brightness detected by the brightness sensor 11.

<Second Modification of Imaging Device of First Embodiment>
The control unit 7 does not directly detect the brightness of the surrounding environment, but roughly assumes the brightness of the surrounding environment based on the time (date) and time (time zone) in one year, and the mode switching unit 72 may be switched to each mode.

  As shown in FIG. 22, in the mode setting table 12, any one of the normal mode, the intermediate mode, and the night vision mode is set corresponding to the combination of the date and the time zone. A clock 73 in the control unit 7 manages the date and time. The control unit 7 refers to the date and time indicated by the clock 73 and reads the mode set from the mode setting table 12.

  The light projection control unit 71 and the mode switching unit 72 control the imaging device so that the mode read from the mode setting table 12 is set.

<Third Modification of Imaging Device of First Embodiment>
As illustrated in FIG. 23, the imaging device may be controlled so that the user manually selects a mode using the operation unit 13 and the projection control unit 71 and the mode switching unit 72 are in the selected mode. The operation unit 13 may be an operation button provided on the housing of the imaging apparatus or a remote controller.

<Video signal processing method>
A video signal processing method executed by the imaging apparatus 101 shown in FIG. 1 will be described again with reference to FIG.

  In FIG. 24, when the imaging apparatus starts operation, the control unit 7 determines whether or not the brightness of the surrounding environment is equal to or greater than the threshold value Th1 in step S1. If it is equal to or greater than the threshold Th1 (YES), the control unit 7 causes the process in the normal mode to be executed in step S3. If it is not equal to or greater than the threshold value Th1 (NO), the control unit 7 determines whether or not the brightness of the surrounding environment is equal to or greater than the threshold value Th2 in step S2.

  If it is equal to or greater than the threshold value Th2 (YES), the control unit 7 causes the process in the intermediate mode to be executed in step S4. If it is not greater than or equal to the threshold Th2 (NO), the control unit 7 causes the process in the night-vision mode to be executed in step S5.

  Control part 7 returns processing to Step S1 after Steps S3-S5, and repeats after Step S1.

  FIG. 25 shows specific processing in the normal mode in step S3. In FIG. 25, the control part 7 (light projection control part 71) turns off the infrared light projector 9 in step S31. In step S32, the control unit 7 inserts the infrared cut filter 21. In step S33, the control unit 7 (mode switching unit 72) connects the switches 51 and 53 to the terminal Tb. The order of steps S31 to S33 is arbitrary and may be simultaneous.

  In step S34, the control unit 7 causes the imaging unit 3 to image the subject. In step S <b> 35, the control unit 7 controls the video processing unit 5 so that the demosaic processing unit 54 performs demosaic processing on the frames constituting the video signal generated by the imaging unit 3 imaging the subject.

  FIG. 26 shows specific processing in the intermediate mode in step S4. 26, in step S41, the control unit 7 (light projection control unit 71) turns on the infrared projector 9 so that infrared light with wavelengths IR1 to IR3 is projected in a time-sharing manner from the light projecting units 91 to 93. To.

  The control part 7 inserts the dummy glass 22 in step S42. In step S43, the control unit 7 (mode switching unit 72) connects the switches 51 and 53 to the terminal Ta. The order of steps S41 to S43 is arbitrary and may be simultaneous.

  In step S44, the control unit 7 causes the imaging unit 3 to image the subject. The imaging unit 3 projects infrared light with a wavelength IR1 associated with R, infrared light with a wavelength IR2 associated with G, and infrared light with a wavelength IR3 associated with B, respectively. Take a picture of the subject in the

In step S45, the control unit 7 (mode switching unit 72) controls the pre-signal processing unit 52 so that the surrounding pixel addition unit 521 is deactivated and the synthesis unit 523 is operated to generate a synthesized video signal.

  The frames constituting the video signal generated when the imaging unit 3 images the subject in the state where infrared light of wavelengths IR1, IR2, and IR3 are respectively projected are the first frame, the second frame, 3 frames are assumed.

  The synthesizing unit 523 converts the three primary color pixel data based on the R pixel data in the first frame, the G pixel data in the second frame, and the B pixel data in the third frame into color The filters 32 are arranged so as to have the same arrangement as the filter elements. The synthesizer 523 thus generates a synthesized video signal by synthesizing the first to third frames into one frame.

  In step S46, the control unit 7 controls the video processing unit 5 so that the demosaic processing unit 54 performs demosaic processing on the frame of the composite video signal.

  The demosaic processing unit 54 performs demosaic processing for generating an R frame, a G frame, and a B frame based on the frame of the synthesized video signal, and sequentially generates the demosaiced three primary color frames. .

  The demosaic processing unit 54 can generate an R frame by interpolating the R pixel data at a pixel position where the R pixel data does not exist. The demosaic processing unit 54 can generate the G frame by interpolating the G pixel data at the pixel position where the G pixel data does not exist. The demosaic processing unit 54 can generate the B frame by interpolating the B pixel data at the pixel position where the B pixel data does not exist.

  In the case of the first intermediate mode, the same position pixel addition unit 522 is operated in step S45, and in the case of the second intermediate mode, the same position pixel addition unit 522 is not operated in step S45. do it.

  FIG. 27 shows specific processing in the night-vision mode in step S5. In FIG. 27, in step S51, the control unit 7 (light projection control unit 71) turns on the infrared projector 9 so that infrared light with wavelengths IR1 to IR3 is projected in a time-sharing manner from the light projecting units 91 to 93. To.

  The control part 7 inserts the dummy glass 22 in step S52. In step S53, the control unit 7 (mode switching unit 72) connects the switches 51 and 53 to the terminal Ta. The order of steps S51 to S53 is arbitrary and may be simultaneous.

  In step S54, the control unit 7 causes the imaging unit 3 to image the subject. In step S55, the control unit 7 (mode switching unit 72) controls the previous signal processing unit 52 to operate the surrounding pixel addition unit 521 and the synthesis unit 523 to generate a synthesized video signal.

  In step S56, the control unit 7 controls the video processing unit 5 so that the demosaic processing unit 54 performs demosaic processing on the frame of the composite video signal.

  In the case of the first night vision mode, the same position pixel addition unit 522 is operated in step S55, and in the case of the second night vision mode, the same position pixel addition unit 522 is disabled in step S55. It may be an operation.

<Video signal processing program>
In FIG. 1, the control unit 7 or an integrated part of the video processing unit 5 and the control unit 7 is configured by a computer (microcomputer), and the video signal processing program (computer program) is executed by the computer. It is also possible to realize the same operation as that of the imaging apparatus 101 of this embodiment.

  With reference to FIG. 28, an example of a procedure executed by the computer when the control in the intermediate mode, which is step S4 in FIG. 24, is configured by the video signal processing program will be described. FIG. 28 shows processing that the video signal processing program causes the computer to execute.

  In FIG. 28, in step S401, the video signal processing program controls the infrared projector 9 so as to project infrared light of wavelengths IR1, IR2, and IR3 associated with R, G, and B to the computer. To execute the process.

  The process shown in step S401 may be executed outside the video signal processing program. In FIG. 28, the process of inserting the dummy glass 22 is omitted. The process for inserting the dummy glass 22 may also be executed outside the video signal processing program.

  In step S402, the video signal processing program captures the first frame of the video signal generated by the imaging unit 3 imaging the subject in a state where infrared light having the wavelength IR1 is projected on the computer. A process of acquiring pixel data to be configured is executed.

  In step S403, the video signal processing program captures the second frame of the video signal generated when the imaging unit 3 images the subject in a state where infrared light having the wavelength IR2 is projected on the computer. A process of acquiring pixel data to be configured is executed.

  In step S404, the video signal processing program captures a third frame of the video signal generated when the imaging unit 3 images the subject in a state where infrared light having the wavelength IR3 is projected on the computer. A process of acquiring pixel data to be configured is executed. The order of steps S402 to S404 is arbitrary.

  In step S405, the video signal processing program causes the computer to arrange the R, G, and B pixel data so as to be in the same arrangement as the arrangement of the filter elements in the color filter 32, and synthesize the synthesized video into one frame. A process for generating a signal is executed.

  In the intermediate mode, the video signal processing program does not cause the computer to execute the step of adding surrounding pixels in step S405.

  In step S406, the video signal processing program causes the computer to perform a process of generating a R, G, B frame by performing a demosaic process on the frame of the composite video signal.

  Although illustration is omitted, when the control in the night vision mode, which is step S5 in FIG. 24, is configured by the video signal processing program, the processing of adding peripheral pixels is executed in the computer in step S405 in FIG. You can do it.

  The video signal processing program may be a computer program recorded on a computer-readable recording medium. The video signal processing program may be provided in a state where it is recorded on a recording medium, or may be provided via a network such as the Internet so that the computer can download the video signal processing program. The computer-readable recording medium may be any non-temporary arbitrary recording medium such as a CD-ROM or a DVD-ROM.

  In the imaging apparatus configured as illustrated in FIG. 1, for example, a plurality of units may be provided as necessary, and the intermediate mode and the night vision mode may be executed simultaneously. In that case, the video output unit 6 may output both the video signal generated in the intermediate mode and the video signal generated in the night vision mode.

  The mode switching unit 72 switches between a state in which the video output unit 6 outputs the video signal generated in the intermediate mode and a state in which the video output unit 6 outputs the video signal generated in the night vision mode. May be. At this time, as described above, switching may be performed according to the brightness of the surrounding environment, the time, or the like. The video processing unit 5 (video processing device) may be separated from other units.

  Further, there may be a case where the normal mode is switched to the night vision mode or the night vision mode is switched to the normal mode without using the intermediate mode.

  When the intermediate mode is not used, the normal mode and the night vision mode may be selected and used in an environment where the use of the intermediate mode is suitable. In this case, the color image signal is not satisfactory as compared with the case where the intermediate mode is used, but imaging is possible.

  However, even with an imaging device equipped with only the normal mode and the night vision mode, the subject is photographed with one imaging device in a situation where the ambient brightness changes, for example, when the subject is photographed all day with a surveillance camera. There is an effect that can be done.

  Furthermore, the normal mode may be switched to the intermediate mode and the intermediate mode may be switched to the normal mode without using the night vision mode. When the night vision mode is not always used, the night vision mode may not be installed.

  The night vision mode may not be used in places with electric lights. An imaging apparatus equipped with only the normal mode and the intermediate mode can be used when the night vision mode does not have to be used.

  When the night vision mode is not used, the intermediate mode may be used in an environment where the above-described night vision mode is suitable. In this case, the color image signal is not good as compared with the case of using the night vision mode, but imaging is possible.

  Even in the case of an imaging device equipped only with the normal mode and the intermediate mode, similarly, there is an effect that a subject can be photographed with one imaging device in a situation where ambient brightness changes.

<Infrared light projection timing synchronization method: first example>
Next, as shown in FIG. 29, for example, a case where an imaging system that images the subject SB1 with two imaging devices 101 is configured and the timing adjustment mode is set is considered. In the timing adjustment mode, as described later, infrared light is projected by the infrared projector 9 to synchronize the light projection timing of the infrared light. Therefore, it is preferable to set the timing adjustment mode at night when visible light is weak.

  One of the two imaging devices 101 shown in FIG. 29 is assumed to be its own imaging device 101, and the other is assumed to be another imaging device 101. The two imaging devices 101 are connected by wire or wireless.

  In FIG. 30, (a) shows infrared light projection by the infrared projector 9 of its own imaging device 101, and (c) shows infrared light projection by the infrared projector 9 of another imaging device 101. . Assume that the phases of infrared light projection by the infrared projectors 9 of the two imaging devices 101 are not synchronized and are shifted by a period shorter than the period of projecting infrared light of one wavelength. In FIG. 30, (b) shows an exposure period (hereinafter, also simply referred to as a frame) of a frame generated by imaging by its own imaging apparatus 101.

  As shown in FIG. 30B, since the timing of infrared light projection by the infrared projectors 9 of the two imaging devices 101 is shifted, the infrared light having two wavelengths in the hatched period. Will be mixed. Therefore, color reproducibility is impaired during the hatched period.

  In FIG. 31, (a) shows infrared light projection by the infrared projector 9 of its own imaging device 101, and (c) shows infrared light projection by the infrared projector 9 of another imaging device 101. . It is assumed that the infrared light projections by the infrared projectors 9 of the two imaging devices 101 are not synchronized and are shifted by a period during which infrared light of one wavelength is projected. In FIG. 31, (b) shows a frame generated by imaging by its own imaging device 101.

  As shown in FIG. 31 (b), since the timing of the infrared light projection by the infrared projectors 9 of the two imaging devices 101 is completely shifted, infrared light of two wavelengths is emitted over the entire period. It will be in a mixed state. Therefore, color reproducibility is impaired over the entire period.

  As described above, when the infrared projectors 9 of the two imaging devices 101 individually project infrared light, infrared light having different wavelengths are mixed, and a color image capable of correct color reproduction cannot be captured.

  A synchronization method for synchronizing the timings at which the infrared projectors 9 of the two imaging devices 101 project infrared light with wavelengths IR1, IR2, and IR3 will be described. In FIG. 29, two imaging devices 101 are used, but three or more imaging devices may be used.

  As shown in FIG. 32, it is assumed that the master / slave setting unit 74 sets its own imaging device 101 as the master imaging device 101M and the other imaging device 101 as the slave imaging device 101S. FIG. 32 schematically shows the configuration of the imaging apparatus 101 described in FIG.

  The synchronization signal transmission unit 76t of the imaging device 101M transmits a synchronization signal Ssync to the imaging device 101S. The synchronization signal receiving unit 76r of the imaging device 101S receives the synchronization signal Ssync. The control unit 7 of the imaging device 101S controls the timing of projecting infrared light with wavelengths IR1, IR2, and IR3 based on the received synchronization signal Ssync.

  The imaging device 101S generates a reference clock with reference to the received synchronization signal Ssync using a PLL circuit (not shown) or the like. The imaging device 101S operates each unit based on this reference clock.

  As an example, the synchronization signal Ssync is a pulse indicating the head of one frame period for projecting infrared light of wavelengths IR1, IR2, and IR3 shown in FIG. 33B, as shown in FIG. Good. That is, the synchronization signal Ssync may be a pulse having a frame period. Here, the frame is a frame of the video signal output from the video output unit 6.

  The synchronization signal Ssync may be a pulse indicating the beginning of a period during which infrared light of wavelengths IR1, IR2, and IR3 is projected. A pulse indicating the head of a period during which infrared light of any one of the wavelengths IR1, IR2, and IR3 is projected may be used as the synchronization signal Ssync. The synchronization signal Ssync only needs to be synchronized with a predetermined period (one frame period) in which infrared light with wavelengths IR1, IR2, and IR3 is projected.

  The synchronization signal Ssync may be synchronized with one or two of the divided periods obtained by dividing the predetermined period into three. Further, a plurality of types of synchronization signals Ssync may be used, and different synchronization signals Ssync may be used in each of the divided periods obtained by dividing the predetermined period into three.

  If the synchronization signal Ssync is supplied from the imaging device 101M to the imaging device 101S, in principle, the infrared light projections by the respective infrared projectors 9 are synchronized, and infrared light with wavelengths IR1, IR2, and IR3 is generated. The timing to project can be matched.

  However, even when the imaging device 101M and the imaging device 101S are connected by wire, the wavelengths IR1 and IR2 are caused by the delay caused by the wiring length and the difference between the internal delay times of the two imaging devices 101. , IR3 infrared light projection timing may vary.

  Therefore, when it is necessary to further improve the accuracy, the timing of the infrared light by the infrared projector 9 of the imaging device 101S is matched with the timing of the infrared light by the infrared projector 9 of the imaging device 101M as follows. The infrared light projections by the two infrared projectors 9 are synchronized.

  When the imaging apparatus 101M is set to the timing adjustment mode, as shown in FIG. 34 (a), the light projection control unit 71 is one infrared light among the infrared lights having wavelengths IR1, IR2, and IR3. Infrared projector 9 is controlled so that only light is projected within one divided period in the three divided periods, and the remaining two divided periods are set to no light projection. One infrared light may be any of infrared light having wavelengths IR1, IR2, and IR3. In the present embodiment, only infrared light having a wavelength IR1 is projected.

  When the imaging apparatus 101S is set to the timing adjustment mode, the light projection control unit 71 controls the infrared projector 9 so that infrared light is not projected in all periods. The imaging device 101S captures the subject SB1 and generates a frame of the video signal in a state where only one infrared light is projected by the infrared projector 9 of the imaging device 101M as shown in FIG. To do.

  When the imaging devices 101M and 101S are set to the timing adjustment mode, the imaging device 101S is effectively irradiated with infrared light projected from the infrared projector 9 of the imaging device 101M toward the imaging device 101S. You may do it.

  FIG. 34B shows a state where the phase of the infrared light projection timing shown in FIG. 34A matches the phase of the frame generated by the imaging by the imaging apparatus 101S. The frame FIR1 is an R frame generated during a period in which infrared light having the wavelength IR1 is projected. The frames FIR2 and FIR3 are G and B frames that are generated in the non-projection period when infrared light with wavelengths IR2 and IR3 is projected.

  As shown in FIG. 34B, the R frame FIR1 is bright, and the G and B frames FIR2 and FIR3 are dark. Therefore, in the state of FIG. 34B, the luminance level determined by the luminance level determination unit 75 of the imaging apparatus 101S is increased only in the R luminance level in the frame FIR1, and the luminance in the G and B frames FIR2 and FIR3. The level is very low.

  FIG. 34 (c) shows a case where the phase is slightly delayed, and FIG. 34 (d) shows a case where the phase is slightly advanced.

  When the phase is delayed as shown in FIG. 34 (c), infrared light projection with the wavelength IR1 is started within the imaging period for generating the B frame FIR3. It becomes a slightly darker and slightly darker state than the case of 34 (b). Therefore, the B brightness level in the B frame FIR3 determined by the brightness level determination unit 75 is slightly higher than in the case of FIG. 34B.

  Conversely, when the phase advances as shown in FIG. 34D, imaging for generating the G frame FIR2 is started before the projection of the infrared light with the wavelength IR1 is completed. The frame FIR2 is in a slightly brighter and slightly darker state than in the case of FIG. Therefore, the G luminance level in the G frame FIR2 determined by the luminance level determination unit 75 is slightly higher than that in the case of FIG. 34B.

  In FIGS. 34 (c) and 34 (d), the “R” brightness level in the R frame FIR1 is the same as that in FIG. 34 (b), as “slightly bright” is described in the R frame FIR1. Is slightly lower than

  Therefore, the luminance levels of R, G, and B are determined based on the timing at which the infrared projector 9 of the imaging device 101M projects infrared light having the wavelength IR1, and the R, G, and B frames FIR1, FIR2, and FIR3 in the imaging device 101S. It changes as shown in FIG. 35 according to the generation timing.

  As shown in FIG. 35, in an appropriate state in which the phase of the infrared light projected from the infrared projector 9 of the imaging device 101M matches the phase of the frame generated by imaging by the imaging device 101S, R The luminance levels of G and B are equivalently low values. Equivalent is a state that can be regarded as substantially the same including a predetermined error.

  When the phase of the frame generated by the imaging by the imaging device 101S is delayed with respect to the timing of the infrared light projected from the infrared projector 9 of the imaging device 101M, the luminance level of B becomes higher than the luminance level of G. . The luminance level of R is lower than the proper state.

  When the phase of the frame generated by the imaging by the imaging device 101S advances with respect to the timing of the infrared light projected from the infrared projector 9 of the imaging device 101M, the luminance level of G becomes higher than the luminance level of B. . The luminance level of R is lower than the proper state.

  Therefore, the luminance level determination unit 75 of the imaging apparatus 101S determines the magnitude relationship between the G luminance level and the B luminance level. The control unit 7 advances or delays the timing of operating the infrared projector 9 and the previous signal processing unit 52 based on the synchronization signal Ssync that operates the imaging device 101S, depending on the magnitude relationship.

  Specifically, if the luminance level of B is higher than the luminance level of G, the light projection control unit 71 sets the infrared light projector 9 to advance the timing of projecting infrared light of wavelengths IR1, IR2, and IR3. Control. At the same time, the control unit 7 controls the previous signal processing unit 52 to advance the timing of generating the R, G, and B frames FIR1, FIR2, and FIR3.

  If the luminance level of G is higher than the luminance level of B, the light projection control unit 71 controls the infrared light projector 9 so as to delay the timing of projecting infrared light of wavelengths IR1, IR2, and IR3. At the same time, the control unit 7 controls the previous signal processing unit 52 so as to delay the timing of generating the R, G, and B frames FIR1, FIR2, and FIR3.

  The control unit 7 (light projection control unit 71) controls the infrared projector 9 and the front signal processing unit 52 until the luminance levels of G and B become equal values. The control unit 7 holds the adjustment amount at the timing when the luminance levels of G and B become equal values.

  After canceling the timing adjustment mode, the imaging apparatus 101S controls each unit to shift the timing for operating the imaging apparatus 101S by the adjustment amount obtained and held in the timing adjustment mode.

  As described above, the imaging system configured as shown in FIG. 29 can image the subject SB1 in a state where the infrared light projection timings of the infrared projectors 9 of the imaging device 101M and the imaging device 101S are synchronized. it can. In an environment where the brightness does not change much, the brightness level of R may be adjusted to the maximum.

<Infrared light projection timing synchronization method: second example>
In the first example of the infrared light projection timing synchronization method, the infrared light projection timing of the infrared projector 9 of the imaging device 101S matches the infrared light projection timing of the infrared projector 9 of the imaging device 101M. However, it may be reversed.

  In the second example of the infrared light projection timing synchronization method, the infrared light projection timing of the infrared projector 9 of the imaging device 101M matches the infrared light projection timing of the infrared projector 9 of the imaging device 101S. Let In the description of the second example, description of common parts with the first example is omitted.

  When the imaging apparatus 101S is set to the timing adjustment mode, as shown in FIG. 36 (a), the light projection control unit 71 has infrared light with a wavelength IR1 out of infrared light with wavelengths IR1, IR2, and IR3. The infrared projector 9 is controlled so that only light is projected within one divided period in the three divided periods, and the remaining two divided periods are set to no light projection.

  When the imaging apparatus 101M is set to the timing adjustment mode, the light projection control unit 71 controls the infrared projector 9 so that infrared light is not projected in all periods. As shown in FIG. 36A, the imaging device 101M images the subject SB1 and projects a frame of a video signal in a state where only the infrared light having the wavelength IR1 is projected by the infrared projector 9 of the imaging device 101S. Generate.

  FIG. 36B shows a state in which the infrared light projection timing in FIG. 36A matches the frame generated by the imaging by the imaging device 101M. FIG. 36C shows a case where the phase is delayed, and FIG. 36D shows a case where the phase is advanced.

  In the second example, the luminance levels of R, G, and B are determined by the timing at which the infrared projector 9 of the imaging apparatus 101S projects infrared light having the wavelength IR1, and the R, G, and B frames FIR1 in the imaging apparatus 101M. , FIR2, and FIR3 are changed as shown in FIG.

  As shown in FIG. 37, when the phase of the frame generated by the imaging by the imaging device 101M is delayed with respect to the timing of the infrared light projected from the infrared projector 9 of the imaging device 101S, the luminance level of G becomes B Higher than the luminance level. The luminance level of R is lower than the proper state.

  When the phase of the frame generated by the imaging by the imaging device 101M advances with respect to the timing of the infrared light projected from the infrared projector 9 of the imaging device 101S, the luminance level of B becomes higher than the luminance level of G. . The luminance level of R is lower than the proper state.

  The control unit 7 of the imaging device 101M controls the infrared projector 9 and the pre-signal processing unit 52 until the G and B luminance levels determined by the luminance level determination unit 75 become equal values. The control unit 7 holds the adjustment amount at the timing when the luminance levels of G and B become equal values.

  After canceling the timing adjustment mode, the imaging apparatus 101M controls each unit to shift the timing at which the imaging apparatus 101M is operated by the adjustment amount obtained and held in the timing adjustment mode.

  As described above, the second example in which the infrared light projection timing of the infrared projector 9 of the imaging device 101M is matched with the infrared light projection timing of the infrared projector 9 of the imaging device 101S is similar to that of the imaging device 101M. It is possible to synchronize the timing of projecting infrared light by the respective infrared projectors 9 with the imaging device 101S. Note that the second example can be used when there are only two imaging devices 101.

  In the first and second examples described above, in the timing adjustment mode, the imaging device 101M (or 101S) projects only infrared light having the wavelength IR1, and the imaging device 101S (or 101M) emits infrared light. In this state, the subject SB1 is imaged to generate a video signal. In an environment where the brightness does not change much, the brightness level of R may be adjusted to the maximum.

  Alternatively, you can do the following: The imaging device 101M (or 101S), which is the reference side of the infrared light projection timing, projects infrared light with wavelengths IR1, IR2, and IR3 as usual. The imaging device 101S (or 101M) on the side that adjusts the timing of projecting infrared light generates no image light and generates a video signal with the infrared cut filter 21 inserted.

  When the timing adjustment mode is set at night or the like when the visible light is weak, the dummy glass 22 is inserted. If an infrared cut filter 21 is inserted instead of the dummy glass 22 to generate a video signal, the imaging device 101M (or 101S) on the reference side may project all infrared light.

  For example, the light projecting unit 91 is a light emitting diode (LED) that emits infrared light of 780 nm. The wavelength band of light from the LED partially depends on the visible light band. Therefore, if imaging is performed with the infrared cut filter 21 inserted, a state equivalent to that in which only infrared light having the wavelength IR1 is projected can be obtained. Therefore, the projection timing can be adjusted in the same manner as when only infrared light having the wavelength IR1 is projected. The wavelength IR1 is assumed to be short enough not to be cut by the infrared cut filter 21.

<Configuration and Operation of Imaging Device of Second Embodiment>
In the imaging device 102 of the second embodiment shown in FIG. 38, the same parts as those of the imaging device 101 of the first embodiment shown in FIG. As shown in FIG. 29, an imaging system can be configured by a plurality of imaging devices 102.

  The imaging device 102 includes an infrared projector 9 </ b> B instead of the infrared projector 9. The infrared projector 9 </ b> B generates a synchronization signal Ssync indicating the timing of projecting infrared light with wavelengths IR <b> 1, IR <b> 2, IR <b> 3 and supplies the synchronization signal Ssync to the control unit 7. The infrared projector 9B includes a synchronization signal transmission unit 9Bt that transmits the synchronization signal Ssync and a synchronization signal reception unit 9Br that receives the synchronization signal Ssync.

  The control unit 7 in the imaging device 102 has a configuration in which the synchronization signal transmitting unit 76t and the synchronization signal receiving unit 76r in the control unit 7 of the imaging device 101 are omitted.

  As shown in FIG. 39, it is assumed that the master / slave setting unit 74 sets its own imaging device 102 as a master imaging device 102M and another imaging device 102 as a slave imaging device 102S. FIG. 39 schematically illustrates the configuration of the imaging apparatus 102 described with reference to FIG.

  The synchronization signal transmission unit 9Bt of the imaging device 102M transmits the synchronization signal Ssync to the imaging device 102S. The synchronization signal receiver 9Br of the imaging device 102S receives the synchronization signal Ssync. As described with reference to FIG. 33A, the synchronization signal Ssync may be a pulse having a frame period indicating the head of one frame period during which infrared light with wavelengths IR1, IR2, and IR3 is projected.

  The imaging device 102S does not generate the synchronization signal Ssync by its own infrared projector 9B. The infrared projector 9B of the imaging device 102S supplies the received synchronization signal Ssync to the control unit 7. The infrared projector 9B of the imaging device 102S projects infrared light with wavelengths IR1, IR2, and IR3 at a timing based on the received synchronization signal Ssync.

  When there is a difference in the timing of projecting infrared light with wavelengths IR1, IR2, and IR3 between the imaging device 102M and the imaging device 102S, the projection timing may be adjusted in the same manner as in FIGS. .

  That is, the infrared light projection timing by the infrared projector 9B of the imaging device 102M and the infrared light projection timing by the infrared projector 9B of the imaging device 102S are matched, and the infrared light by both infrared projectors 9B is matched. Synchronize the light emission.

  The first to third modifications shown in FIGS. 21 to 23 can also be applied to the imaging device 102 of the second embodiment. The same applies to imaging devices 103 to 106 according to third to sixth embodiments described later.

<Configuration and Operation of Imaging Device of Third Embodiment>
In the imaging device 103 of the third embodiment shown in FIG. 40, the same parts as those of the imaging device 101 of the first embodiment shown in FIG. FIG. 40 schematically illustrates the configuration of the imaging device 3.

  As shown in FIG. 29, an imaging system can be configured by a plurality of imaging devices 103. In FIG. 40, the own imaging device 103 is set as the master imaging device 103M, and the other imaging device 103 is set as the slave imaging device 103S.

  The infrared projector 9C of the master imaging device 103M has a synchronization signal transmitter 9Ct. The infrared projector 9C projects infrared light having wavelengths IR1, IR2, and IR3 based on the control by the control unit 7. The synchronization signal transmission unit 9Ct transmits a synchronization signal Ssync indicating the timing of projecting infrared light to the slave imaging device 103S.

  The infrared projector 9C of the imaging device 103S includes a synchronization signal receiver 9Cr. The synchronization signal Ssync received by the imaging device 103S is supplied to the control unit 7. The infrared projector 9C of the imaging device 103S projects infrared light with wavelengths IR1, IR2, and IR3 at a timing based on the received synchronization signal Ssync. The imaging device 103S may have the same configuration as the imaging device 102S in FIG.

  Also in the third embodiment, when there is a difference in the timing of projecting infrared light with wavelengths IR1, IR2, and IR3 between the imaging device 103M and the imaging device 103S, the timings are matched and the infrared projectors 9C of both are used. Synchronize the infrared light projection.

<Configuration and Operation of Imaging Device of Fourth Embodiment>
In the imaging device 104 of the fourth embodiment shown in FIG. 41, the same parts as those of the imaging device 101 of the first embodiment shown in FIG. As shown in FIG. 29, an imaging system can be configured by a plurality of imaging devices 104.

  The control unit 7 in the imaging device 104 includes a reference clock transmission unit 77t and a reference clock reception unit 77r instead of the synchronization signal transmission unit 76t and the synchronization signal reception unit 76r in the control unit 7 of the imaging device 101.

  When the imaging device 104 is set as a master imaging device, the reference clock transmission unit 77t transmits the reference clock Sclk to the other imaging device 104. At this time, the reference clock receiver 77r does not operate.

  When the image capturing apparatus 104 is set as a slave image capturing apparatus, the reference clock receiving unit 77r receives the reference clock Sclk transmitted from another master image capturing apparatus 104. At this time, the reference clock transmission unit 77t does not operate.

  As shown in FIG. 42, it is assumed that the master / slave setting unit 74 sets its own imaging device 104 as a master imaging device 104M and another imaging device 104 as a slave imaging device 104S. FIG. 42 schematically illustrates the configuration of the imaging device 104 described in FIG.

  The reference clock transmission unit 77t of the imaging device 104M transmits the reference clock Sclk to the imaging device 104S. The reference clock receiver 77r of the imaging device 104S receives the reference clock Sclk. The imaging device 104M and the imaging device 104S operate in synchronization with a common reference clock Sclk.

  The reference clock Sclk is used in MPEG2, for example, a 14.32 MHz clock that is four times the color subcarrier frequency (3.58 MHz) of the video signal, a reference signal used in a broadcast studio called GenLock (Generator Lock), etc. A 27 MHz clock can be used. Of course, the reference clock Sclk is not limited to these clocks.

  By operating the imaging device 104M and the imaging device 104S with a common reference clock Sclk, the imaging device 104M and the imaging device 104S can be synchronized in frequency. However, the timings at which the infrared projectors 9 of the imaging devices 104M and 104S project infrared light with wavelengths IR1, IR2, and IR3 cannot be synchronized.

  Therefore, in the imaging device 104 of the fourth embodiment, when the imaging devices 104M and 104S are set to the timing adjustment mode, the infrared light projection timings by the infrared projectors 9 of the imaging devices 104M and 104S are matched. Adjust to synchronize with the state.

  43 shows the phase relationship between the timing of light projection by the infrared projector 9 of the imaging apparatus 104M and the frame generated by imaging by the imaging apparatus 104S, as in FIG. 43B shows a state in which the timing of infrared light projection in FIG. 43A coincides with the frame, and FIG. 43C shows a case where the phase is slightly delayed. Yes.

  In the imaging device 104 according to the fourth embodiment, the phase may be significantly advanced or significantly delayed as shown in FIG.

  In the imaging device 104 of the fourth embodiment, the luminance levels of R, G, and B are the timing at which the infrared projector 9 of the imaging device 104M projects infrared light having the wavelength IR1, and the imaging device 104S has the frames FIR1, FIR2. , And FIR3 are generated as shown in FIG.

  The characteristic shown in FIG. 35 corresponds to that shown only in the vicinity of the proper state of FIG. In the first to third embodiments, it is only necessary to eliminate a slight shift in light projection timing due to a delay due to the wiring length or a difference in delay time. In the fourth embodiment, since only frequency synchronization is obtained in the first place, as shown in FIG. 44, a considerably large amount of timing adjustment may be required.

  The timing adjustment process by the control unit 7 will be described with reference to the flowchart shown in FIG. In FIG. 45, the control unit 7 (luminance level determination unit 75) determines whether or not the G luminance level and the B luminance level are equal in step S101.

  44 is equivalent to the position of Ph0 in the proper state where the phases in FIG. 44 are in agreement, the position of Ph (-3) where the phase is delayed, and the position of Ph (+3) where the phase is advanced. That is either.

  If they are equal (YES), the control unit 7 determines whether or not the luminance level of R is the lowest in step S102. If it is not the lowest (NO), it is an appropriate state at the position of Ph0, so the processing is terminated.

  If it is the lowest (YES) in step S102, it means that the ½ cycle is shifted (delayed or advanced) at the position of Ph (-3) or Ph (+3). In step S103, the (light projection control unit 71) advances or delays the timing by one-half cycle.

  On the other hand, if they are not equal in step S101 (NO), the control unit 7 shifts the processing to step S104. The control unit 7 shifts the process to step S104 even after the process of step S103 is completed.

  In step S104, the controller 7 determines whether or not the G luminance level is greater than the B luminance level. If the luminance level of G is larger than the luminance level of B (YES), the control unit 7 delays the timing slightly and returns the process to step S101. If the luminance level of G is not greater than the luminance level of B (NO), the control unit 7 advances the timing a little and returns the process to step S101.

  By repeating the above steps S101 to S106, it is possible to synchronize the infrared light projecting timings of the infrared projectors 9 of the imaging devices 104M and 104S in a matched state.

  After canceling the timing adjustment mode, the imaging device 104S controls each unit to shift the timing for operating the imaging device 104S by the adjustment amount obtained and held in the timing adjustment mode.

  Note that a reference clock may be output from a device separate from the imaging device, and each imaging device may receive the reference clock. For example, a switcher or the like for switching and displaying images of a plurality of imaging devices may generate and output a reference clock. Further, the imaging device that outputs the reference clock and the imaging device that operates as a master in the timing adjustment mode may be different.

<Configuration and Operation of Imaging Device of Fifth Embodiment>
In the imaging device 105 of the fifth embodiment shown in FIG. 46, the same parts as those of the imaging device 101 of the first embodiment shown in FIG. As shown in FIG. 29, an imaging system can be configured by a plurality of imaging devices 105.

  The imaging device 105 includes a power supply circuit 15 that supplies necessary power to each unit of the imaging device 105 based on power from a commercial AC power source. The imaging device 105 generates a reference clock synchronized with the frequency of the AC voltage and operates the imaging device 105. Therefore, it is possible to achieve frequency synchronization between the master imaging device 105M and the slave imaging device 105S.

  The imaging device 105S adjusts the infrared light projection timing by the infrared projector 9 in the same manner as in the fourth embodiment. As a result, the imaging devices 105M and 105S can be synchronized in a state in which the timing of projecting infrared light by the respective infrared projectors 9 is matched.

<Configuration and Operation of Imaging Device of Sixth Embodiment>
In the imaging device 106 of the sixth embodiment shown in FIG. 47, the same parts as those of the imaging device 101 of the first embodiment shown in FIG. As shown in FIG. 29, an imaging system can be configured by a plurality of imaging devices 106.

  The imaging device 106 includes an antenna 16a that receives radio waves from a satellite 200 for Global Navigation Satellite System (GNSS), and a GNSS receiver 16 that receives a GNSS signal output from the antenna 16a. The GNSS signal is generally an RF signal. GNSS is GPS (Global Positioning System) as an example.

  The GNSS receiver 16 generates a clock signal based on time information included in the radio wave from the satellite 200. The time information transmitted from the satellite 200 is synchronized with the atomic clock built in the satellite 200 with high accuracy. The GNSS receiver 16 supplies a clock signal synchronized with the atomic clock with high accuracy to the controller 7. The control unit 7 uses this clock signal as a reference clock. The control unit 7 operates the imaging device 106 based on the reference clock.

  Therefore, frequency synchronization between the master imaging device 106M and the slave imaging device 106S can be achieved.

  The imaging device 106S adjusts the infrared light projection timing by the infrared projector 9 in the same manner as in the fourth embodiment. As a result, the imaging devices 106M and 106S can be synchronized in a state in which the timing of projecting infrared light by the respective infrared projectors 9 is matched.

<Configuration and Operation of Imaging Device of Seventh Embodiment>
In the imaging devices 104 to 106 according to the fourth to sixth embodiments, it is assumed that frequency synchronization is achieved by a common reference clock. In the seventh embodiment, a common reference clock is not used in a plurality of imaging devices. The imaging apparatus according to the seventh embodiment includes a reference clock generation source that individually generates a highly accurate reference clock.

  As an example, a reference clock generation source that generates a highly accurate reference clock is a crystal oscillator.

  Seventh Embodiment A plurality of imaging apparatuses periodically adjust the light projection timing so as to match the light projection timing with the same method as in the fourth embodiment. What is necessary is just to set the time interval which adjusts a light projection timing according to the tolerance | permissibility of the shift | offset | difference of the light projection timing of a some imaging device.

  When high color reproducibility is required, the tolerance of phase shift is reduced, so it is preferable to set the time interval as short as possible. When such a high color reproducibility is not required, the tolerance of phase shift increases, so the time interval may be set relatively long.

<Configuration and Operation of Imaging Device of Eighth Embodiment>
In the imaging device 108 of the eighth embodiment shown in FIG. 48, the same parts as those of the imaging device 101 of the first embodiment shown in FIG.

  The imaging device 108 of the eighth embodiment has a configuration in which the synchronization signal transmission unit 76t in the imaging device 101 of the first embodiment is omitted. The synchronization signal receiving unit 76r receives a synchronization signal Ssync supplied from an external synchronization signal supply device.

  FIG. 49 illustrates a configuration example of an imaging system including a plurality of imaging devices 108 and a synchronization signal supply device that supplies a synchronization signal Ssync to each of the plurality of imaging devices 108. In FIG. 49, as an example, the camera switch 180 is a synchronization signal supply device. A display unit 280 is connected to the camera switcher 180.

  Normally, when an imaging system is configured by a plurality of imaging devices (for example, surveillance cameras), in order to select one or more video signals output from the plurality of imaging devices and display them on the display unit 280, a so-called switcher is used. The camera switcher 180 called.

  Therefore, it is convenient to use the camera switch 180 as a synchronization signal supply device that supplies the synchronization signal Ssync to each of the plurality of imaging devices 108.

  As shown in FIG. 49, the camera switching unit 180 includes a video switching unit 181 and a synchronization signal generation unit 182 that generates a synchronization signal Ssync. The video signal Svideo output from each imaging device 108 is input to the video switching unit 181.

  As an example, each imaging device 108 outputs the video signal Svideo with a frame period and phase synchronized with the synchronization signal Ssync. In this case, for example, as in the present embodiment, when the imaging device 108 that performs imaging by sequentially emitting infrared light and the imaging device that performs normal imaging without using infrared light are mixed. However, you can switch between images smoothly and display multiple images simultaneously.

  The video switching unit 181 selects one or a plurality of video signals Svideo and outputs them to the video output terminal 183. A display unit 280 is connected to the video output terminal 183, and the video signal Svideo selected by the video switching unit 181 is displayed.

  For example, the video switching unit 181 may divide each frame of the four video signals Svideo into one frame so that the screen of the display unit 280 is divided into four and the video signal Svideo is displayed in each region. . The number of screen divisions of the display unit 280 is arbitrary.

  The camera switcher 180 has an individual video output terminal 184 for individually outputting the video signal Svideo output from each imaging device 108. The individual video output terminal 184 can be omitted.

  The synchronization signal Ssync generated by the synchronization signal generator 182 has a frame period as shown in FIG. Although the synchronization signal Ssync shown in FIG. 33A is a positive pulse, the synchronization signal Ssync shown in FIG. 50A is a negative pulse. Of course, the synchronization signal Ssync generated by the synchronization signal generator 182 may also be a positive pulse.

  As shown in FIG. 50B, the video signal Svideo output from the imaging device 108 is assumed to be an interlace method in which one frame is composed of two fields of an odd field and an even field.

  An example of the relationship between the infrared light projection timing and the generation timing of the video signal Svideo in each imaging apparatus 108 will be described with reference to FIG. The light projection control unit 71 of the imaging device 108 performs control so that infrared light is selectively projected at a timing based on the synchronization signal Ssync.

  As shown in FIGS. 51B and 51C, the infrared projector 9 sequentially projects infrared light with wavelengths IR1 to IR3, the imaging unit 3 images the subject, and then the imaging device 108 receives the video signal. It takes about 1.5 frames to output Svideo. The frame of the video signal Svideo obtained based on the set IRset1 of the wavelengths IR1 to IR3 shown in (b) of FIG. 51 is a frame F11 shown in (c) of FIG.

  Therefore, as shown in FIGS. 51A and 51B, when the light projection control unit 71 of each imaging apparatus 108 receives the synchronization signal Ssync, after the time t11 of about 0.5 frame has elapsed, the wavelength IR1 It is better to project the infrared light.

  If the time t11 is longer than 0.5 frame, the frame F11 of the video signal Svideo cannot be obtained based on the set IRset1 of the wavelengths IR1 to IR3, and must be the frame F12 of the next timing. For this purpose, the capacity of the buffer memory for delay increases.

  When the time t11 is shorter than 0.5 frame, the frame F11 of the video signal Svideo can be obtained based on the set IRset1 of the wavelengths IR1 to IR3. However, in this case, since the amount of delay due to shortening the time t11 increases, the capacity of the buffer memory increases.

  Therefore, it is preferable to set the time t11 in accordance with the time required to generate the frame of the video signal Svideo based on each set of wavelengths IR1 to IR3.

  In FIG. 51, the time t11 between the synchronization signal Ssync and the infrared light projection timing of the wavelength IR1 is set, but the time between the synchronization signal Ssync and the infrared light projection timing of the wavelength IR2 or IR3 is set. May be.

  The frame of the video signal Svideo generated based on the set IRset1 of the wavelengths IR1 to IR3 may be intentionally the frame F12 instead of the frame F11 for a predetermined reason.

  By the way, as the plurality of imaging devices 108 in FIG. 49, the imaging device 108 that generates the frame F11 based on the set IRset1 of the wavelengths IR1 to IR3 and the imaging device 108 that generates the frame F12 may be mixed.

  In the imaging system shown in FIG. 49, a common synchronization signal Ssync is supplied to each of the plurality of imaging devices 108. Therefore, in a manner similar to the imaging devices 101 to 103 of the first to third embodiments, the plurality of imaging devices 108 in principle synchronize the infrared light projections by the respective infrared projectors 9 to obtain the wavelength IR1. , IR2 and IR3 can be made to coincide with the timing of projecting infrared light.

  When there is a deviation in the timing of projecting infrared light of wavelengths IR1, IR2, and IR3, the control unit 7 operates the infrared projector 9 and the pre-signal processing unit 52 as described with reference to FIG. You can advance or delay the timing. It should be noted that since there is little influence on the video quality if there is a slight timing shift, there is no need for adjustment.

<Summary of Imaging Apparatus of First to Third Embodiments>
In the first to third embodiments, the master imaging devices 101M to 103M are configured as follows.

  The light projection control unit 71 controls the length and order of each divided period in which infrared light having wavelengths IR1 to IR3 is selectively projected in three divided periods obtained by dividing a predetermined period into three. The projector 9 is controlled.

  The imaging unit 3 images the subject in a state where infrared light with wavelengths IR1 to IR3 is selectively projected. The synchronization signal transmission units 76t, 9Bt, and 9Ct transmit a synchronization signal to the outside of the apparatus, specifically, to other imaging apparatuses (slave imaging apparatuses 101S to 103S) other than the master imaging apparatuses 101M to 103M. The synchronization signal indicates the timing at which the light projection control unit 71 included in the slave-side imaging device projects infrared light by the infrared projector 9, and the master light projection control unit 71 projects infrared light by the infrared projector 9. It is used to synchronize with the timing.

  When the master imaging devices 101M to 103M are set to the timing adjustment mode for adjusting the timing at which the slave-side light projection control unit 71 projects infrared light by the slave-side infrared projector 9, The control unit 71 controls the infrared projector 9 as follows. The light projection control unit 71 projects only one infrared light of the infrared light with wavelengths IR1 to IR3 within one divided period in the three divided periods, and does not project the remaining two divided periods. And

  When the light projecting control unit 71 and the infrared projector 9 in the master imaging devices 101M, 102M, and 103M are the first light projecting control unit and the first infrared projector, the light projecting control unit 71 in the slave image capturing devices 101S to 103S, The infrared projector 9 is a second light projection control unit and a second infrared projector.

  In the first to third embodiments, the slave imaging devices 101S to 103S are configured as follows.

  The light projection control unit 71 controls the length and order of each divided period in which infrared light having wavelengths IR1 to IR3 is selectively projected in three divided periods obtained by dividing a predetermined period into three. The projector 9 is controlled.

  The imaging unit 3 images the subject in a state where infrared light with wavelengths IR1 to IR3 is selectively projected. The video processing unit 5 generates R, G, and B video signals based on the imaging signal captured by the imaging unit 3.

  The synchronization signal receivers 76r, 9Br, and 9Cr receive synchronization signals transmitted by other imaging devices (master imaging devices 101M to 103M) other than the slave imaging devices 101S to 103S. The synchronization signal is synchronized with the timing at which the slave-side projection control unit 71 projects infrared light by the infrared projector 9 and the timing at which the master-side projection control unit 71 projects infrared light by the infrared projector 9. Used to make

  The slave imaging devices 101S to 103S may include a luminance level determination unit 75 that determines the luminance level of the R, G, and B video signals generated by the video processing unit 5. The slave imaging devices 101S to 103S operate as follows when the projection control unit 71 is set to the timing adjustment mode for adjusting the timing at which the infrared projector 9 projects infrared light. Good.

  The light projection control unit 71 controls the infrared projector 9 so that infrared light is not projected. In the imaging unit 3, the infrared projector 9 on the master side projects only one infrared light of the infrared light with wavelengths IR1 to IR3 within one divided period in the three divided periods, and the remaining two divided parts. The subject is imaged in a state where the period is not projected.

  Alternatively, in the imaging unit 3, the infrared projector 9 on the master side projects infrared light of all wavelengths, and substantially only one wavelength of infrared light is projected through the infrared cut filter 21. The subject is imaged as if The infrared cut filter 21 is the infrared cut filter 21 provided in the slave imaging devices 101S to 103S.

  The light projection control unit adjusts the timing of projecting the infrared light based on the luminance levels of the R, G, and B video signals determined by the luminance level determination unit 75.

  Assuming that the light projection control unit 71 and the infrared projector 9 in the slave imaging devices 101S to 103S are the first light projection control unit and the first infrared projector, the light projection control unit 71 and the infrared projector in the master imaging devices 101M to 103M. Reference numeral 9 denotes a second light projection control unit and a second infrared projector.

<Summary of Slave Imaging Device of First to Seventh Embodiments>
The slave imaging devices 101S to 106S of the first to sixth embodiments and the slave imaging device of the seventh embodiment include a light projection control unit 71 (first light projection control unit), an imaging unit 3, and video processing. Unit 5 and a luminance level determination unit 75. The slave imaging device operates as follows when it is set to the timing adjustment mode. The light projection control unit 71 controls the infrared light projector 9 (first infrared light projector) so that infrared light with wavelengths IR1 to IR3 is not projected.

  In the imaging unit 3, the infrared projector 9 controlled by the projection control unit 71 provided in another imaging apparatus only divides one infrared light of the infrared light of wavelengths IR1 to IR3 into three divided periods. The subject is imaged in a state where light is projected within the period and the remaining two divided periods are not projected.

  The other imaging devices are the master imaging devices 101M to 106M and the master imaging device of the seventh embodiment. The light projection control unit 71 in the master imaging device is a second light projection control unit, and the infrared projector 9 in the master imaging device is a second infrared projector.

  Alternatively, in the imaging unit 3, the second infrared projector projects infrared light with wavelengths IR1 to IR3 and images the subject through the infrared cut filter.

  The light projection control unit 71 adjusts the timing of projecting infrared light with wavelengths IR1 to IR3 based on the luminance levels of the R, G, and B video signals determined by the luminance level determination unit 75.

  The first light projection control unit may adjust the timing of projecting the infrared light so that the brightness level determination unit 75 determines as follows. The luminance levels of the video signals of two colors other than the color associated with R projected by the second infrared projector are the same first level. In addition, the luminance level of the video signal of the color corresponding to R projected by the second infrared projector is a second level higher than the first level.

<Summary of Control Method of Imaging Device of First to Seventh Embodiments>
An imaging device control method used in an imaging system including a plurality of imaging devices is as follows. Each of the plurality of imaging devices synchronizes with each other the timings of selectively projecting infrared light of wavelengths IR1 to IR3 in three divided periods obtained by dividing the predetermined period into three by the individual infrared projectors 9. The subject is imaged by each of the plurality of imaging devices in a state where infrared light with wavelengths IR1 to IR3 is selectively projected.

<Summary of Control Method of Imaging Device of Fourth to Sixth Embodiments>
An imaging device control method used in an imaging system including a plurality of imaging devices is as follows. A plurality of imaging devices are operated with a common reference clock. One imaging device among the plurality of imaging devices is a master imaging device, and a device other than the master imaging device is a slave imaging device.

  A first light projection control unit included in the master imaging device controls the first infrared light projector, and only one infrared light of wavelengths IR1 to IR3 is divided into three for a predetermined period. The light is projected within one divided period in the three divided periods, and the remaining two divided periods are set to no light projection. Alternatively, infrared light having wavelengths IR1 to IR3 is projected in three divided periods.

  A second light projection control unit included in the slave imaging device controls the second infrared light projector so that infrared light with wavelengths IR1 to IR3 is not projected.

  The image capturing unit 3 included in the slave image capturing apparatus captures an image of a subject in a state where only one infrared light is being projected or infrared light having wavelengths IR1 to IR3 is passed through an infrared cut filter. A luminance level determination unit 75 included in the slave imaging device determines the luminance level of each of the R, G, and B video signals generated based on the imaging signal captured by the imaging unit 3.

  Based on the luminance level of the R, G, and B video signals determined by the luminance level determination unit 75, the second light projection control unit projects infrared light with wavelengths IR1 to IR3 by the second infrared projector. Adjust timing.

<Summary of Imaging System of Eighth Embodiment>
The imaging system includes a plurality of imaging devices 108 and a synchronization signal supply device that supplies a synchronization signal Ssync to each of the plurality of imaging devices.

  It is convenient to use the camera switch 180 as a synchronization signal supply device by providing a synchronization signal generator 182 in the camera switch 180 that switches the video signal Svideo output from each of the plurality of imaging devices 108.

  Each of the plurality of imaging devices 108 includes a synchronization signal receiving unit 76r that receives the synchronization signal Ssync, a light projection control unit 71, and the imaging unit 3. The light projecting control unit 71 selectively projects the infrared light at the timing based on the synchronization signal Ssync received by the synchronization signal receiving unit 76r from the infrared projector 9 capable of projecting a plurality of infrared lights. Control as follows. The imaging unit 3 images the subject in a state where infrared light is projected by the infrared projector 9.

  The present invention is not limited to the embodiment described above, and various modifications can be made without departing from the scope of the present invention. For example, in the imaging device shown in FIGS. 1, 38, 40, 41, 46, 47, and 48, the infrared projector 9 may be detachable from the housing of the imaging device. The infrared projector 9 may be configured outside the imaging apparatus. The imaging device may have a configuration for controlling the infrared projector 9 when the infrared projector 9 is attached.

  In addition, in order to synchronize the timing of projecting infrared light in a plurality of imaging devices, the timing of emitting infrared light and the timing of turning off the light do not have to be completely matched. It is only necessary that the slave imaging device does not project infrared light other than the predetermined infrared light in the exposure section corresponding to the predetermined infrared light in the master imaging device.

  If the time has a little influence on shooting, the slave imaging device projects infrared light other than the predetermined infrared light in the exposure section corresponding to the predetermined infrared light in the master imaging device. You may allow that.

  For example, if the slave imaging device does not project infrared light other than the predetermined infrared light in at least half of the exposure interval corresponding to the predetermined infrared light in the master imaging device. Good.

  Furthermore, the present invention is not limited to the present embodiment described above, and can be applied to other imaging techniques using infrared light. The control unit 7 and the video processing unit 5 can be realized by using one or a plurality of hardware processors.

  In the above description, the definition of master and slave may be different in each embodiment.

DESCRIPTION OF SYMBOLS 3 Image pick-up part 9 Infrared projector 71 Light projection control part 76r Sync signal receiving part 108 Image pick-up device 180 Camera switcher (synchronization signal supply apparatus)

Claims (4)

A plurality of imaging devices, and a synchronization signal supply device that supplies a synchronization signal to each of the plurality of imaging devices,
Each of the plurality of imaging devices is
A synchronization signal receiving unit for receiving the synchronization signal;
A light projecting control unit configured to control an infrared light projector capable of projecting a plurality of infrared lights to selectively project infrared light at a timing based on the synchronization signal received by the synchronization signal receiving unit; ,
An imaging unit for imaging a subject in a state where infrared light is projected by the infrared projector;
An imaging system comprising:   The imaging system according to claim 1, wherein the synchronization signal supply device supplies a synchronization signal having a frame period of a video signal output from the imaging device to each of the plurality of imaging devices.   The imaging system according to claim 1, wherein the synchronization signal supply device is a camera switching unit that switches video signals output from the plurality of imaging devices. From the synchronization signal supply device, supply a synchronization signal to each of the plurality of imaging devices,
Each of the plurality of imaging devices is
Receiving the synchronization signal;
An infrared projector capable of projecting a plurality of infrared lights is controlled to selectively project infrared light at a timing based on the synchronization signal,
An imaging system control method, comprising: imaging an object with an imaging unit in a state where infrared light is projected by the infrared projector.

Images