Classifications

• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
  • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
  • G01J3/46—Measurement of colour; Colour measuring devices, e.g. colorimeters
  • G01J3/50—Measurement of colour; Colour measuring devices, e.g. colorimeters using electric radiation detectors
  • G01J3/508—Measurement of colour; Colour measuring devices, e.g. colorimeters using electric radiation detectors measuring the colour of teeth
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
  • G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
  • G01J3/46—Measurement of colour; Colour measuring devices, e.g. colorimeters
  • G01J3/52—Measurement of colour; Colour measuring devices, e.g. colorimeters using colour charts
  • G01J3/524—Calibration of colorimeters
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
  • H04N23/125—Colour sequential image capture, e.g. using a colour wheel
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/80—Camera processing pipelines; Components thereof
  • H04N23/84—Camera processing pipelines; Components thereof for processing colour signals
  • H04N23/88—Camera processing pipelines; Components thereof for processing colour signals for colour balance, e.g. white-balance circuits or colour temperature control
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
  • H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
  • H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics

Definitions

• the present invention relates to an imaging system that performs color correction of an input image using spectral spectrum information of a subject.
• color management is performed in many fields such as the industrial field, the food field, and the medical field.
• color management of the colors of manufactured products is performed, and colorimeters such as spectrometers and colorimeters are used to check whether or not the products are finished to the specified color. It's being used.
• color management of skin color is performed. Digital cameras are often used to record changes in skin color.
• a digital camera has an advantage that an image of an affected part can be easily acquired and an image can be confirmed immediately after imaging, but the accuracy of color correction is low.
• the accuracy of color correction in digital cameras is degraded for various reasons. In particular, the drop in white balance detection accuracy has a significant effect on color correction accuracy.
• Document 1 Japanese Patent Laying-Open No. 2003-125422 proposes to improve the accuracy of white balance correction.
• the accuracy of white balance correction is improved by using information from a colorimetric sensor when photographing a digital camera. That is, Literature 1 installs a colorimetric sensor in almost the same direction as the area photographed by a digital camera, and corrects the digital camera signal value based on the obtained RGB value of the colorimetric sensor. is there. In this case, the RGB values of the image data captured by the digital camera are averaged over the entire screen for each RGB, and compared with the colorimetric sensor. Yes.
• the digital camera acquires an RGB value that complies with the CIE RGB color system, that is, an RGB value that matches the characteristics of the human eye. It is difficult to manufacture, and it is difficult to detect colors with the same accuracy as a spectrometer or a colorimeter.
• the present invention has been made in view of a powerful problem, and an object of the present invention is to provide a photographing system capable of performing highly accurate color correction on a photographed image.
• An imaging system is a imaging system for imaging an object, a color information detecting means for detecting color information of the object, and imaging a color image of the object. And a color correction means for correcting the color of a color image picked up by the color image pickup means from corresponding position information of the color information detection means and the color image pickup means. It is characterized by.
• the color image capturing means captures a color image of the subject, and the color information detecting means detects the color information of the subject.
• the position of the detected color information in the color image is determined from the corresponding position information between the color information detection means and the color image pickup means, and the corresponding position is corrected with high accuracy by color correcting the color image with the color information. Obtain a color image that has undergone color correction.
• FIG. 1 A block diagram showing an imaging system according to a first embodiment of the present invention.
• FIG. 2 is an explanatory diagram showing an appearance when the imaging system in FIG. 1 is applied to a digital camera.
• FIG. 3 is a block diagram showing a specific configuration of the digital camera 13 and the spectrometer 10 of FIG. 1.
• FIG. 4 is a block diagram showing a specific configuration of a color correction unit 3 in FIG.
• FIG. 7 is a block diagram showing another example of the color correction unit.
• FIG. 9 is a block diagram showing another example of the colorimeter.
• FIG. 10 is a flowchart for explaining calculation of position information.
• FIG. 1 A block diagram showing a second embodiment of the present invention.
• FIG. 12 is a block diagram showing a modification of the second embodiment.
• FIG. 14 is a block diagram showing a specific configuration of an image processing device 202 in FIG. 13.
• FIG. 15 is an explanatory view showing a modification of the third embodiment.
• ⁇ 16 ⁇ is an explanatory view showing a modification of the third embodiment.
• FIG. 19 is an explanatory diagram showing a configuration of a color separation filter 230.
• FIG. 21 is a block diagram showing an internal circuit configuration of a digital camera 229.
• FIG. 23 is a block diagram showing a specific configuration of a color correction unit 244 in FIG. 21.
• FIG. 25 is a block diagram showing a sixth embodiment of the present invention.
• FIG. 26 is an explanatory diagram showing a configuration of a spectral filter 54 in FIG. 25.
• FIG. 27 is an explanatory diagram for explaining the operation dial 8;
• FIG. 28 is an explanatory diagram showing an example in which the shooting direction and the angle of view of the digital camera 245 and the multi-band camera 50 are matched.
• FIG. 29 is an explanatory diagram for explaining camera shake correction.
• FIG. 30 is a block diagram showing a configuration of a displacement correcting unit.
• FIG. 31 is a block diagram showing a seventh embodiment of the present invention.
• FIG. 32 is a block diagram showing a specific circuit configuration of an image processing unit.
• FIG. 33 is a block diagram showing a specific configuration of a corresponding position calculation unit 107 in FIG. 32.
• FIG. 34 is an explanatory diagram showing an input image of the corresponding position calculation unit 107.
• FIG. 35 is an explanatory diagram for explaining each shooting mode.
• FIG. 36 is an explanatory diagram for explaining the operation of the embodiment.
• FIG. 37 is a block diagram showing another application example using an image processing unit 269 that uses a colorimetric image as an image processing unit.
• FIG. 38 is an explanatory view showing an eighth embodiment of the present invention.
• FIG. 1 is a block diagram illustrating an imaging system according to a first embodiment of the present invention.
• the photographing system has a color image capturing unit 1 and a color information detecting unit 2.
• the color image capturing unit 1 is configured by, for example, a digital camera or the like, captures an object (not shown), and outputs a color image such as an RGB primary color image to the color correcting unit 3.
• the color information detection unit 2 detects color information at a predetermined position of a part of the subject imaged by the color image imaging unit 1, and outputs the detected color information to the color correction unit 3. I have.
• the color correction unit 3 also receives corresponding position information indicating to which position in the color image the color information detected by the color information detection unit 2 corresponds from the color image pickup unit 1.
• the color correction unit 3 corrects information of a corresponding position in a color image based on the corresponding position information with color information, and outputs the corrected color image.
• FIG. 2 is an explanatory diagram showing the appearance when the imaging system of FIG. 1 is applied to a digital camera.
• a digital camera 13 that obtains RGB information as color image information is employed.
• a spectrometer (hereinafter also referred to as a colorimeter) 10 as a colorimeter for detecting a spectrum as color information is attached to the digital camera 13.
• Figure 2 shows only the main components of a digital camera. That is, the digital camera 13 includes a photographing lens 4, an RGB color image sensor 5, an image processing unit 6, an image display unit 7, and an operation dial 8.
• the housing of the digital camera 13 is provided with a connection section 9 for mounting the spectrometer 10, and the connection section 9 allows the spectrometer 10 to be mounted like a normal strobe.
• the spectrometer 10 includes a finder 11 and an angle sensor 12 as main components.
• FIG. 3 is a block diagram showing a specific configuration of the digital camera 13 and the spectrometer 10 of FIG.
• Reference numeral 14 denotes a subject to be photographed, and FIG. 3 shows an example in which the subject 14 is a human arm.
• Reference numeral 15 denotes a target part, for example, an affected part (inflamed part or the like) in the arm.
• the digital camera 13 forms a subject image on the RGB color image pickup device 5 by the taking lens 16.
• the signal processing unit 17 is an analog processing circuit that performs gain correction, offset correction, and the like.
• Reference numeral 18 denotes an AD conversion, and reference numeral 19 denotes an RGB image memory which is a storage unit of an RGB image.
• the colorimeter 10 includes a spectral detection unit 25 and a camera mounting unit 26, and the spectral detection unit 25 rotates up, down, left, and right with respect to the camera mounting unit 26.
• Reference numeral 22 denotes a photographing lens of the spectrometer 10.
• the luminous flux of the subject 14 is sent to the spectroscope 24 and the finder 11 via the half mirror 23.
• the angle sensor 12 detects the rotation angle of the spectral detection unit 25 and outputs angle information.
• the spectroscope 24 splits the incident light from the half mirror 23 and outputs spectrum information.
• the angle information from the angle sensor 12 and the spectrum information from the spectroscope 24 are respectively given to and stored in an angle data memory 28 or a colorimetric data memory 27 in the digital camera 13.
• the corresponding position detection unit 21 in the image processing unit 6 of the digital camera 13 is based on the angle information obtained from the angle sensor 12, the angle-of-view information of the taking lens 16, and the distance information to the subject. Calculates the position of the measured position on the subject of the colorimeter 10 in the captured RGB image.
• the corresponding position information is given to the color correction unit 3 as two-dimensional coordinate information Cx and Cy.
• Reference numeral 20 denotes an image storage unit
• reference numeral 7 denotes an image display unit, which stores and displays the RGB images corrected by the color correction unit 3, respectively.
• FIG. 4 is a block diagram showing a specific configuration of the color correction unit 3 in FIG.
• Reference numeral 29 denotes an image extraction unit that extracts an image based on the corresponding positional information Cx and Cy from the RGB image memory 19, 30 denotes a data averaging unit that calculates an average of the extracted data, and 31 denotes the averaged data Is a spectrum estimating unit for estimating the spectrum from, and 32 is a correction coefficient calculating unit for calculating a correction coefficient C ( ⁇ ).
• Reference numeral 33 denotes a subject spectrum estimating unit for estimating the spectrum of each position of the subject 14 based on the RGB signal stored in the RGB image memory 19, and 34 denotes a signal correcting unit.
• Reference numeral 35 denotes an RGB conversion unit for converting the spectrum signal color into RGB.
• FIG. 5 is an explanatory diagram showing a display example on the viewfinder 11.
• FIG. 6 is an explanatory diagram for explaining the relationship between the camera position and the subject.
• the digital camera 13 is set on a camera fixing device (not shown) such as a tripod.
• a camera fixing device such as a tripod.
• the patient sits on a chair or the like, and places the affected part (in this case, part of the arm) on a desk or the like so as to face the digital camera 13 in the imaging direction and fix it so as not to move.
• An operator such as a doctor or a nurse operates a zoom or a tripod handle (not shown) of the digital camera 13 to adjust the framing of the subject 14 to be photographed.
• a zoom or a tripod handle (not shown) of the digital camera 13 to adjust the framing of the subject 14 to be photographed.
• the affected part is not always located at the center of the screen of the digital camera 13.
• the framing is determined, the affected part is positioned at the center of the screen while looking at the finder 11 of the colorimeter 10 next.
• the viewfinder 11 looks as shown in FIG. 5, and the measurement direction of the colorimeter 10 can be directly opposed to the subject 14 by aligning the center circular part with the affected part. In this way, when the digital camera 13 and the colorimeter 10 are ready for shooting, shooting is performed, and the image data power of the shot subject image is stored in the GB image memory 19, The spectrum data is stored in the colorimetric data memory 27.
• FIG. 6 shows the relationship between the camera position at the time of shooting and the subject 14. That is, the angle of view oc of the digital camera 13, the distance information L to the subject 14 converted from the AF information color, the angles 0 and ⁇ of the colorimeter, and the base length B of the digital camera 13 and the colorimeter 10 From this, the position of the subject of interest (the image of the portion of interest 15) on the RGB image can be calculated.
• the corresponding position calculator 21 calculates the corresponding position as two-dimensional coordinate values Cx and Cy by calculation, and outputs it to the color corrector 3.
• the color correction unit 3 cuts out a rectangular area centered on the corresponding position of the subject image stored in the RGB image memory 19 based on the calculated two-dimensional coordinate values Cx and Cy.
• the size of the rectangular area is, for example, 16 ⁇ 16 pixels.
• the average value (Rave, Gave, Bave) of all pixels is obtained by the data averaging unit 30 from the image signal of this rectangular area.
• the spectrum estimating unit 31 estimates a spectrum signal S 1 ( ⁇ ) from the average value (Rave, Gave, Bave) by, for example, a method disclosed in Japanese Patent Application Laid-Open No. H11-085952.
• the correction coefficient calculation unit 32 calculates a correction coefficient C ( ⁇ ) by the following equation (1) using the spectrum information S2 ( ⁇ ) stored in the colorimetric data memory 27.
• image data is sequentially read from the RGB image memory 19 for each pixel, and is sequentially converted into a spectrum signal by the subject spectrum estimating unit 33.
• the signal correction unit 34 multiplies the correction coefficient C ( ⁇ ) calculated by the correction coefficient calculation unit 32 to correct the signal value.
• the corrected spectrum signal value is converted to an RGB value by the RGB conversion unit 35, and a corrected R'G'B 'signal as a corrected color image is output.
• the corrected R'G'B 'signal is sent to, for example, the image storage unit 20 and the image display unit 7.
• an RGB image obtained by imaging is corrected based on spectral data obtained by a separately provided colorimeter, so that extremely accurate color correction is performed. Is possible.
• the accuracy of the correction coefficient is extremely high because a predetermined area of the RGB image that accurately corresponds to the measurement position of the colorimeter is detected.
• the image data when converted into spectral data.
• an RGB correction coefficient calculation unit 35 is newly provided to reduce the amount of force calculation, and a correction coefficient corresponding to the spectrum is obtained as a coefficient corresponding to the RGB signal. You may make it multiply with RGB image data. In this case, the subject spectrum estimator and the RGB converter are not required, and the amount of calculation can be significantly reduced.
• the corresponding position of the colorimetric system may be superimposed and displayed by a mark such as a cross on the image display unit 7 of the digital camera!
• the mark position is displayed based on Cx and Cy obtained by the corresponding position calculation unit 21.
• the operator can move the colorimeter 10 up, down, left, and right while looking at the mark, and can take an image while adjusting the mark position to the affected part, which is the target part 15. Since the corresponding position can be confirmed on the screen, the corresponding position can be accurately aligned.
• the colorimeter 10 may be provided with a laser pointer 36 to determine the corresponding position.
• the colorimetric point of the colorimeter 10 is adjusted to the affected part (attention part 15) of the subject by the finder 11 in the same manner as described above.
• the digital camera 13 light is emitted from the laser pointer 36, and an image of the laser pointer 36 is photographed.
• an image not irradiated with the laser pointer 36 is captured.
• the corresponding position calculator 21 detects the difference between the image irradiated with the laser pointer 36 and the image not irradiated as shown in FIG. 10, and calculates the corresponding position using the point having the larger difference value.
• FIG. 11 is a block diagram showing a specific configuration of the digital camera 13 ′ and the spectrometer 10 according to the second embodiment of the present invention.
• the same components as those in FIG. 3 are denoted by the same reference numerals, and description thereof will be omitted.
• the colorimeter is driven up, down, left, right, up and down to detect its angles ⁇ and ⁇ to detect the position corresponding to the RGB image. It is characterized in that the photographing direction of the colorimeter is controlled at the point where the colorimeter is located.
• the present embodiment differs from the first embodiment and includes a corresponding angle calculation unit 40 and a rotary motor 41.
• the spectral detection unit 25 is configured to be able to move only up and down with respect to the camera mounting unit 26 '. Digital camera 13 'always keeps the attention 15 Try to capture at the center of the imaging range of the camera. Then, the corresponding angle calculation unit 40 calculates an angle at which the spectral detection unit 25 captures the target unit 15 from information on the distance to the object 14 and information on the angle of view of the camera, and controls the rotation motor 41 so that the angle is obtained. It is configured to be
• the digital camera 13 ′ is almost directly opposed to the subject 1, and the affected part (attention part 15) of the subject 14 is located at the center of the photographing screen. Adjust the camera position as follows. Thereafter, when the shirt button (not shown) is half-pressed, the AF operation is performed, and the distance to the subject 14 is measured. Based on this information, the corresponding angle calculation unit 40 calculates an angle ⁇ at which the photographing direction of the colorimeter 10 becomes the target part 15 (affected part) of the subject 14. Then, while using the information of the angle sensor 12, the spectral detection unit 25 is rotated by the rotation motor 41, and stopped at the position where the angle becomes the angle ⁇ .
• the direction of the colorimeter automatically changes according to the information on the photographing distance, it is extremely easy to perform photographing without the need for the photographer to adjust the position of the colorimeter. I can.
• the image display unit checks whether the force of the colorimeter has changed direction or not and shoots it, the relationship between the RGB image and the shooting range of the colorimeter can be accurately defined. .
• this confirmation is performed by displaying a specific mark on the image display unit.
• the confirmation may be made by sound, or a lamp such as an LED may be turned on.
• the rotation angle of the camera may be detected, and the colorimeter may be rotated right and left.
• FIG. 12 shows a modification of the present embodiment.
• the mirror 23 of the colorimeter 10 has a rotating structure and can move to the position indicated by the broken line. Mirror 23 moves to this broken line position
• the luminous flux from the white plate 200 enters the colorimeter 10, and the illumination spectrum around the digital camera 3 'can be detected.
• the illumination spectrum information detected in this way it is possible to detect accurate color information of the object 14.
• a method disclosed in Japanese Patent Application Laid-Open No. 11-595952 may be adopted.
• the spectral information of the target area 15 and the illumination vector of the surrounding area of the digital camera 3 are measured along with the information of the digital camera 13, and the accurate color of the target area 15 is estimated based on these information. Is done.
• FIG. 13 and FIG. 14 relate to a third embodiment of the present invention
• FIG. 13 is an explanatory diagram showing the appearance of the device.
• This embodiment shows an example in which a colorimeter and a digital camera can be configured separately by attaching them to a tripod or the like.
• reference numeral 201 denotes a tripod to which both the colorimeter 10 and the digital camera 13 ′ can be attached.
• the colorimeter 10 and the digital camera 13 ′ ′′ are connected to the image processing device 202, respectively.
• the image processing device 202 is a control device configured by a computer or the like.
• FIG. 14 is a block diagram showing a specific configuration of the image processing device 202 in FIG.
• reference numeral 204 denotes an external device controller, for example, a controller such as USB or RS-232C.
• a data input I / F 205 receives spectral information from the colorimeter 10, and receives RGB image data from the digital camera 13 '. The spectral information and the RGB image data taken into the data input I / F 205 are given to the colorimetric data memory 209 or the RGB image memory 210 and stored.
• the target position specifying unit 206 specifies whether the colorimetric point of the colorimeter 10 is located at the position of! / Or a shift on the RGB image.
• the color correction unit 212 performs color correction on the RGB image data based on the spectrum information.
• the color reproduction processing unit 207 further performs color correction on the RGB image color-corrected by the color correction unit 212 using the profile information of the image display unit 208.
• the image storage unit 213 stores the RGB image data corrected by the color correction unit 212 and the color reproduction unit 207.
• the CPU 211 controls the entire image processing device 202.
• the digital camera 13 ′′ under the control of the image processing device 202, the digital camera 13 ′′ is first photographed, and then the colorimeter 10 performs colorimetry.
• RGB image data from the digital camera 13 '''and spectral information from the colorimeter 10 They are stored in the RGB image memory 210 or the colorimetric data memory 209, respectively.
• the captured RGB image is displayed on the image display unit 208.
• the photographer specifies a colorimetric point on the colorimeter 10 using a screen position indicating device such as a mouse (not shown).
• Corresponding position information Cx and Cy based on angle information, angle-of-view information, distance information to the subject, and the like from the colorimeter 10 and the digital camera 13 ′′ are provided to the color correction unit 212 (not shown).
• the color correction unit 212 corrects the color of the RGB image data based on the output of the colorimeter 10 for an area based on the corresponding position information in the RGB image.
• both the digital camera 13 ′ ′ and the colorimeter 10 can be used as they are on the market, and the color correction with high accuracy can be easily performed with a simple configuration. Is possible.
• FIG. 16 shows a modification of the third embodiment.
• the hood 220 with illumination is attached to the digital camera 13 ′ ′′.
• the illuminated hood 220 has a built-in lighting device 221 and the colorimeter 10 can be fixedly mounted. Also, when taking a picture, the tip of the hood 220 is brought into contact with the subject 14 to be photographed, and the picture is taken by the colorimeter 10 at the point P at the photographing center of the digital camera 13 '' '. Color points are also set.
• the colorimeter 10 can easily measure the center position of the photographing screen of the digital camera 13 ′ simply by pressing the dedicated hood 220 against the subject 14. As a result, the corresponding position can always be fixedly set, and extremely simple and stable imaging can be performed.
• FIG. 17 is an explanatory diagram showing a fourth embodiment of the present invention.
• a digital camera and a colorimeter are not separately configured, and a photographing system including a colorimeter inside the digital camera.
• the stem is configured.
• a half mirror 215 and a spectral detection unit 216 are provided in digital camera 214 in the present embodiment.
• the half mirror 215 guides a part of the light flux of a subject (the center subject on the screen of the digital camera), not shown on the optical axis, to the spectral detection unit 216.
• the half mirror 215 rotates in the direction of the arrow in FIG. 17 when shooting with the RGB color image sensor 5, and guides the luminous flux of the subject to the RGB color image sensor 5.
• the spectral detection unit 216 is provided inside the camera, and is not separately configured as in the above-described embodiments, but has the same optical system and imaging system. It is extremely convenient because it uses image elements. In addition, by always setting the center position of the screen as the colorimetric point, stable colorimetry is possible without failing to detect the corresponding position.
• the spectral detection unit 216 detects only the point on the optical axis.
• a plurality of spectral detection units 216 corresponding to a plurality of focus detection positions are used. It is clear that these may be used by switching them according to the focus position.
• FIG. 18 to FIG. 23 relate to a fifth embodiment of the present invention
• FIG. 18 is an explanatory diagram showing the appearance of the device.
• This embodiment describes an example in which a multi-band camera is used instead of a colorimeter.
• FIG. 19 is an explanatory diagram showing the configuration of the color separation filter 230.
• the color separation filter 230 includes a filter turret 238 having a filter A, a filter B, and a filter C, and a filter holding unit 239. .
• the filter turret 238 is rotatably held in the filter holding portion 239.
• FIG. 19 (a) shows a filter turret 238 forming the color separation filter 230
• FIG. 19 (b) shows a filter holding section 239 forming the color separation filter 230.
• FIG. 20 is an explanatory diagram for explaining the characteristics of the photographing band.
• FIG. 20 (a) is a graph showing the spectral sensitivity characteristics of the RGB color imaging device 5, and FIGS. 20 (b) and 20 (c). 7 is a graph showing characteristics of filters A and B of the color separation filter 230, respectively.
• the spectral transmission characteristics of the filters A and B correspond to the peak positions of the respective spectral sensitivities of the RGB color image sensor 5 shown in FIG. 20 (a). It is possible to perform 6-band imaging by switching between Filter A and Filter B for imaging. In addition, the filter C is through, enabling normal RGB shooting. Note that, as the filters A and B, wavelength tunable filters that can be used only with interference filters can be used.
• the filter holding unit 239 has a filter rotating unit 234 so that the filter turret 238 can be directly rotated manually. Further, the filter holding section 239 can be directly mounted on the photographing lens 4 of the digital camera 229 by the lens mounting section 236. Note that a filter ID window 235 is provided in the filter holding unit 239. The filter ID window 235 allows the V and R filters to be disposed immediately before the photographing lens 4 to be filters of the type of displacement. The configuration is such that it can be checked visually.
• FIG. 21 is a block diagram showing an internal circuit configuration of the digital camera 229. Note that in FIG. 21, the same components as those in FIG. 3 are denoted by the same reference numerals, and description thereof will be omitted.
• reference numeral 52 denotes a multi-band image memory in which RGB images captured by the filters A and B are stored as a 6-band multi-band image.
• Reference numeral 240 denotes a switching unit that switches the storage destination of the multiband image and the RGB image, and performs switching according to the designated mode of the shooting mode switching unit 242.
• Reference numeral 241 denotes a corresponding position designating unit for designating a subject position in an RGB image and a subject position in a multi-band image.
• the multiband image has two types of R Since the image is composed of GB image colors, each of the RGB images is sequentially displayed on the image display unit 7 (total of 6 images), and the corresponding position with the RGB image is designated by the operation dial 8.
• FIG. 22 is an explanatory diagram for explaining the corresponding position.
• FIG. 22 (a) shows an RGB color image
• FIG. 22 (b) shows a multiband image.
• FIG. 23 is a block diagram showing a specific configuration of the color correction unit 244 in FIG.
• the color correction unit 244 differs from the above-described embodiment in that it has two signal extraction units 29 and 60, data averaging units 30 and 61, and spectrum estimation units 31 and 62. With this configuration, the color correction unit 244 detects a spectrum at a corresponding position of the multiband image and performs color correction.
• the “multi-band mode” is first specified at the time of shooting, and the filter A is set to perform shooting. Next, filter B is manually set and shooting is performed. The images captured by the filters A and B are stored in the multi-band image memory 52 (FIG. 23). Next, the “RGB mode” is designated, the filter C is selected and shooting is performed, and the captured image data is stored in the RGB image memory 19.
• the color correction unit 244 receives the multiband image from the multiband image memory 52, and the signal extraction unit 60, the data averaging unit 61, and the spectrum estimation unit 62 correspond to the corresponding positional information Cx2, Cy2. Acquires the spectrum information S2 () at the specified position.
• the color correction unit 244 receives the RGB color image from the RGB image memory 19, and the signal extraction unit 29, the data averaging unit 30, and the spectrum estimation unit 31 correspond to the corresponding position information Cxi and Cyl. To obtain the spectrum information of the specified position.
• the correction coefficient calculation unit 32 calculates a correction coefficient C ( ⁇ ) based on the above equation (1). Subsequent operations are the same as in the first embodiment.
• the color correction of the RGB image can be performed at extremely low cost by using the manually rotatable color separation filter.
• the camera since the corresponding position of the image captured by the color separation filter is manually specified using the operation dial, the camera may be moved when capturing the filters A, B, and C due to camera shake or the like. Even if it moves, the corresponding position can be specified without fail.
• a wavelength tunable filter 237 using liquid crystal or the like as shown in FIG. 24 may be used instead of the color separation filter. it is obvious.
• the operation of the color separation filter 230 is completely manual, and the communication with the digital camera 229 is not performed at all.
• filter rotation operation and filter ID detection may be performed.
• a photographing lens in which such a filter is formed may be used.
• FIGS. 25 to 30 relate to the sixth embodiment of the present invention, and FIG. 25 is a block diagram showing a specific configuration. This embodiment also shows an example in which a multi-band camera is used.
• FIG. 25 the same components as those in FIG. 21 are denoted by the same reference numerals, and description thereof will be omitted.
• the multi-band camera 50 is provided on the digital camera 245.
• the multi-band camera 50 includes a photographing lens 53, a spectral filter 54, a rotating motor 59, a monochrome sensor 55, a signal processing unit 57, and an AZD converter 58.
• FIG. 26 is an explanatory diagram showing the configuration of the spectral filter 54 in FIG.
• the spectral filter 54 includes a plurality of color filters 54a, 54b,... Having mutually different spectral transmission characteristics.
• the power used in the case of eight filters is not limited to eight.
• the photographing control unit 60 controls the focus and aperture of the photographing lens 16, the electronic shutter speed of the monochrome sensor 56, and the like.
• the digital camera 245 includes a multi-band image memory 52, a corresponding position specifying unit 241, and a position shift correcting unit 161 for correcting a position shift of the multi-band image based on camera shake information of the camera shake sensor 243. ing.
• the attention position designation unit 241 detects a corresponding position of the attention unit 15 (affected part) of the subject 14 from images captured by the digital camera 245 and the multi-band camera 50.
• the photographing lens 16 of the video camera is pointed at the subject 14 to be photographed, and the angle of view and the photographing position are determined by the operation dial 8 or the like.
• the photographer half-presses a shutter button (not shown)
• the AE / AF control operation of the digital camera 245 is started.
• Information based on this control is transmitted to the multi-band camera 50, and the shooting control unit 60 sets the focus position of the shooting lens 53 to the subject distance according to the AF information.
• AE information According to the information, the shutter speed of the monochrome sensor 56 and the aperture value of the photographing lens 53 are set.
• different color filters 54a, 54b,... Of the filter 54 are set to different shutter speed values so as to obtain proper exposure, so that an image with a good SN can be obtained.
• the displacement correcting unit 161 corrects the displacement of the multi-band image based on the camera shake information from the camera shake sensor 243, and causes the multi-band image memory 52 to sequentially store the images subjected to the displacement correction.
• the position of the affected part of the subject included in the captured RGB color image and the multi-band image is designated. That is, each image is displayed on the image display unit 7 and the designated cursor is displayed so as to be superimposed on the display image.
• FIG. 27 is an explanatory diagram for explaining the operation dial 8.
• the operation dial 8 includes up, down, left, and right arrow keys and a center determination key.
• the position of the affected part is determined according to the position on the image of the designated cursor at the timing when the enter key is operated.
• Such a specifying operation is performed on the RGB color image and the multi-band image, and the corresponding positions are obtained.
• the positions corresponding to the RGB color image are Cxi and Cyl
• the positions corresponding to the multiband image are Cyl and Cy2.
• the configuration of the color correction unit 244 is the same as that in FIG. 23, and performs color correction by the same operation as in the above embodiment.
• the RGB image is corrected based on the staple data calculated from the multi-band camera 50, extremely high-precision color correction is possible.
• the number of pixels is particularly unclear.
• the digital camera 245 is set to about 5 million pixels, and the multi-band camera 50 is set to about 400,000 pixels in consideration of sensitivity and the like.
• the multi-band power Although the camera 50 can obtain high-precision color information, it cannot obtain a sufficient resolution, but the digital camera 245 can obtain a high-resolution image. An image obtained by fusing the high-resolution image of the digital camera 245 with the high-precision color information of the multi-band camera 50 is obtained, and an extremely high-quality image can be obtained.
• the multi-band camera 50 of the present embodiment is a filter-rotation type surface-sequential type, a force that causes a shift in each spectral image due to camera shake or the like is caused by a camera shake in the digital camera 245 by a camera shake sensor or the like. Since the blur between the spectral images is corrected based on the information, the corresponding position can be obtained more accurately.
• a rotary filter type multi-band camera 50 as shown in FIG. 26 is used.
• the present invention is not limited to this, and a liquid crystal type wavelength tunable filter or the like may be used!
• a liquid crystal type wavelength tunable filter or the like may be used!
• the target position designating unit and the color correcting unit are provided in the digital camera, they may be performed using an arithmetic processing device other than the digital camera, for example, a personal computer.
• the multi-band camera 50 is configured to be directly connected to the digital camera 245.
• the multi-band camera 50 may exchange signals by radio or the like, which may be separately provided.
• optical path branching means 246 is provided so that the photographing direction and the angle of view of the digital camera 245 and the multi-band camera 50 are matched, designation of the corresponding position becomes extremely simple.
• the camera shake information information on a camera shake sensor may be used, and the amount of positional shift between the multiband images may be calculated and corrected.
• this may be performed using image information captured in an RGB color image.
• the timing of capturing each spectral image of the multi-band camera 50 and the digital camera By adjusting the shooting timing, the position shift is detected from two consecutive RGB images, and this information can be used to correct the spectral image to the ⁇ 1 image position.
• a position shift amount between the RGB images is detected by a correlation operation unit 247 as shown in FIG. 30, and based on this, a position shift correction unit 248 corrects the position of the multi-band image.
• the RGB image has a higher resolution.
• FIGS. 31 to 37 relate to the seventh embodiment of the present invention.
• FIG. 31 is a block diagram showing a specific configuration on the camera side
• FIG. 32 is a specific circuit configuration of the image processing unit.
• FIG. This embodiment is applied to the illumination type multi-band camera described in the specification of Japanese Patent Application No. 2002-218863 filed earlier by the applicant, and the photographing target is a tooth and a face including a tooth. It is suitable for the case.
• the imaging system includes a multi-band camera 69, a charging unit 72, and an image processing unit 68.
• the multi-band camera 69 further includes an illumination unit 70, an imaging unit 73, and a control unit 71.
• the lighting unit 70 shown by a thick line is detachably provided at the front end side of the multi-band camera 69, so that signals can be exchanged with the control unit 71 and power is supplied to the control unit 71 by the lighting unit contacts 77. I have. Although not shown, it may be fixed without attaching and detaching.
• the illumination unit 70 stores LED illumination units 70a and 70b, which are composed of a plurality of types of LED lights having different spectral characteristics of emitted light, an illumination optical system 74 for illuminating the objects with the LED illumination units, and information on the LEDs.
• the LED memory 75 includes a temperature sensor 76 for measuring a temperature near the LED.
• the LED lighting units 70a and 70b are configured by a total of 28 LEDs in which four of each of seven types of LEDs are arranged.
• the center wavelength of each LED is 450 nm, 465 nm, 505 nm, 525 nm, 575 nm, 605 nm, and 630 nm, respectively.
• the illumination optical system 74 is for irradiating the LED light to the object surface (the surface of the color chart 110 on the camera side in FIG. 31), and is configured to irradiate the LED light substantially uniformly.
• the imaging unit 73 includes a photographic lens 16, an RGB color imaging device 5, a signal processing unit 17 for performing analog processing for performing gain correction, offset correction, and the like, and an AD converter 18.
• the focus lever 79 is used to change the focus by using a manual control, and is also provided with a contact 80 for detecting the position of the focus lever 79.
• the camera control CPU 81 in the control unit 71 is a CPU for controlling the camera.
• the camera control CPU 81 controls the image pickup unit 73 while being connected to the oral bus 82 and the LCD controller 87. To output the captured color image signal to an external monitor Connected to the composite output terminal 85.
• the LED driver 83 is for controlling light emission of the LED lighting units 70a and 70b, and the data I / F 84 is an interface for receiving the contents of the LED memory 75 of the lighting unit 70 and the information of the temperature sensor 76. It is.
• the communication IZF controller 97 is a controller for controlling a communication IZF such as USB2, for example, and 98 is a communication IZF connection point for the connection.
• the lithium battery 99 is for supplying power to the entire multi-band camera 69, and is connected to a charging contact 100 that is a charging contact.
• the image memory 89 is for temporarily storing image data captured by the imaging unit 73.
• the LED lighting units 70a and 70b use seven types of LEDs, and the image memory 89 has a capacity capable of storing at least seven types of spectral images and one RGB color image.
• the LCD monitor 86 is a monitor for displaying an image being shot by the camera or an image already shot.
• the LCD monitor 86 is configured to display an image superimposed on the image pattern stored in the overlay memory 88 as necessary.
• the image pattern is, for example, a horizontal line that horizontally captures the entire tooth, or a cross line that crosses the horizontal line.
• the operation unit I / F 90 transmits and receives signals to and from operation units provided on the multi-band camera 69 and an output unit (not shown) for transmitting information.
• the operation buttons include a shooting mode switching switch 91 for switching between normal RGB shooting and multi-band shooting, a shutter button 92, and a change of image data displayed on the LCD monitor 86.
• the viewer control button 93 for operation is also equivalent.
• the power LED 94 functions as an output unit for transmitting information, and notifies a photographer of the state of the multi-band camera 69.
• a battery LED 95 for notifying a battery state, an alarm buzzer 96 for notifying a danger at the time of shooting, and the like are also provided on the rear side of the multi-band camera 69.
• Flashing green Preparation for shooting (initial warming, etc.)
• the charging unit 72 includes a color chart 110 for calibrating the multi-band camera 69 and a microswitch for confirming whether the multi-band camera 69 is correctly mounted on the charging unit 72. 111, a power switch 102 for turning on / off the power of the charging unit, a light that turns on in conjunction with ONZOFF of the power switch 102, a power lamp 103 that turns off the light, and a light that lights when the multi-band camera 69 is mounted in the normal position. It is constituted by the mounting lamp 104.
• the charging unit 72 is, for example, a desk-top type.
• the charging unit 100 of the multi-band camera 69 can be used. Can be supplied with power.
• the mounting lamp 104 lights in green when the charging unit 72 is mounted in a normal position of the multi-band camera 69, and flashes in red when the charging unit 72 is not mounted.
• the charging unit 72 is provided with a power supply connector 105 so that an AC adapter 106 is connected. Then, when the charge capacity of the lithium battery 99 is reduced and the yellow or red notch LED 95 is lit, the lithium battery 99 is charged when the multi-band camera 69 is placed on the charging unit 72. It is configured as follows.
• the image processing section 68 has a color correction section 250 having substantially the same configuration as the color correction section 244 in FIG.
• the image processing section 68 includes a corresponding position calculating section 107.
• the corresponding points are detected by manual operation, whereas in the present embodiment, the corresponding points are detected. Is configured to perform automatically.
• FIG. 33 is a block diagram showing a specific configuration of the corresponding position calculating section 107 in FIG.
• FIG. 34 is an explanatory diagram showing an input image of the corresponding position calculation unit 107.
• FIG. 34 (a) shows a multiband image
• FIG. 34 (b) shows an RGB image.
• the corresponding position calculation unit 107 includes a luminance conversion unit 108 for extracting a luminance signal from the multi-band image of FIG. 34, and a central tooth that extracts a tooth area substantially at the center of the screen.
• the image processing unit 68 includes a multi-band image memory 52, an RGB image memory 19, a color correction unit 250, and R ′ G from the color correction unit 250. It has a color reproduction processing unit 207 and an image storage unit 213 to which the 'B' image signal is given.
• the calibration section 253 of the color correction section 250 uses the color chart image stored in the color chart image memory 251 and the ⁇ current image stored in the dark current image memory 252 to calibrate the multiband image. Is being held.
• each shooting mode will be described with reference to FIG.
• This embodiment exemplifies white jung (bleaching) and denture construction in a dental office.
• full-jaw photography which is an image of the entire upper and lower teeth shown in (b)
• tooth photography which shows one or two teeth shown in Fig. 35 (c).
• Facial photography and full jaw photography are photography as RGB images
• tooth photography is photography as a multi-band image.
• the present embodiment corrects the color of an RGB image obtained by photographing a face or full jaw from a multiband image obtained by photographing a tooth.
• RGB shooting The photographer lifts the multiband camera 69 and removes it from the charging unit 72, and sets the shooting mode to “RGB mode”.
• the RGB color image sensor 5 sequentially shoots images, and the images are displayed on the LCD monitor 86. During this shooting, the LED lighting units 70a and 70b are turned off.
• the photographer (dentist or dental hygienist) adjusts the position of the subject (face or full chin) while watching the image on the LCD monitor 86, and then focuses using the focus lever 79.
• the camera control CPU 81 controls the electronic shutter speed of the RGB color image sensor 5 so as to obtain an appropriate exposure.
• the image photographed when the shirt button is pressed is stored in the image memory 89. At this time, additional information such as the RGB image mode is also stored.
• the photographer places multi-band camera 69 on charging unit 72. Then, the mounting lamp 104 is turned on, and the captured RGB image is transferred and stored in the RGB image memory 19 of the image processing unit 68.
• the photographer lifts the multiband camera 69, removes it from the charging unit 72, and sets the photographing mode to the “colorimetric mode”.
• the LED illuminating units 70a and 70b all of the seven types of LEDs are turned on, and the RGB color image sensor 5 sequentially captures images and displays the images on the LCD monitor 86.
• a contact cap 260 (see Fig. 36) is attached to the lighting unit 70, and the photographer (dentist or dental hygienist) adjusts the position to a specific tooth while watching the image on the LCD monitor 86, and focuses on the focus lever. 1 Use 79 to focus.
• the contact cap 260 is in contact with the tooth 261 to be photographed, and the position is fixed to some extent. Then, when desired alignment is performed, the photographer presses the shirt button to perform multi-band imaging.
• the LED lighting units 70a and 70b seven types of LEDs are sequentially turned on, and image data of a predetermined one of RGB images taken at each turn is stored in the image memory 89. In this case,
• the image of the color selected from the RGB image color corresponding to the center wavelength of the LED is stored in the image memory 89 as a multi-band image.
• the camera control CPU 81 controls the LED irradiation time, the irradiation intensity, the electronic shutter speed of the image pickup device, and the like so that the photographing of each wavelength is appropriately exposed at the time of photographing. If the temperature changes drastically during this shooting, an alarm buzzer sounds and a warning is issued.
• the contact cap is removed, and then, when the multi-band camera 69 is mounted on the charging unit 72, the mounting lamp 104 is turned on, and the calibration image is measured. At this time, the multiband camera 69 cannot be mounted on the charging unit 72 unless the contact cap is removed. That is, the color chart 110 is photographed by sequentially turning on the LED having the same wavelength as the LED used for photographing, and the photographed image is stored in the image memory 89 as a color chart image. Next, an image is taken in a state where the LED is not lit at all (under dark), and stored in the image memory 89 as a dark current image.
• the photographed multi-band image, color patch image, and dark current image are both transferred to the image processing unit 68, and the color patch image and the dark current image are respectively stored in the color patch image memory 251 or the dark current image memory 252.
• the subject image is stored in the multi-band image memory 52.
• the following calculation is performed to correct the dark current of the RGB color image sensor 5 and the deterioration of the light quantity and the wavelength shift of the LED illumination units 70a and 70b.
• the amount of light emitted from an LED changes with temperature. Therefore, calibrating according to the operating temperature is extremely effective for improving the accuracy.
• the operation after the calibration process is the same as in the above embodiment. Thus, highly accurate color correction can be performed on the RGB image.
• the multi-band camera 69 can be operated without a cable using a battery, and the usability has been significantly improved. Furthermore, since correction is performed using color patches, deterioration and fluctuation of LEDs and image sensors can be corrected, and extremely high-precision colorimetry can be realized.
• the color chart 110 is built in a cradle that also has a charging function, and the user does not need to perform complicated operations for calibration.
• the lighting of the mounting lamp can reduce operation mistakes in transferring image data, thereby enabling reliable data transfer.
• the state of charge can be always grasped by the battery lamp of the main body.
• a temperature sensor is provided to warn the user using an alarm buzzer when there is a temperature change during tooth photographing, or when there is a large temperature difference between the time of photographing the tooth and the time of the calibration. And stable shooting is possible.
• the image processing unit can be configured by a normal personal computer or the like, and in this case, it is apparent that, for example, the color correction unit may be realized by software.
• a shutter may be provided between the color chart and the lighting unit. If the multi-band camera is lifted, the shutter may be tightened, preventing outside light or dust from entering.
• FIG. 37 is a block diagram showing another application example using an image processing unit 269 using a colorimetric image as the image processing unit.
• the reference numeral 254 denotes the calibrated object image power, which is used to obtain the XYZ values of each image position.
• a chromaticity calculator 256 is a shade number calculator for calculating a shade number which is a number of the obtained XYZ value power crown color chart. The shade number calculator 256 compares the calculated XYZ values with the XYZ values of the shade guides of the respective companies stored in the shade number database 270 to determine the shade numbers.
• 255 is an RGB image operation unit for obtaining RGB image data
• 257 is its storage unit.
• Reference numeral 258 denotes a corrected image creating unit for correcting the color variation of the image display unit 7, and the color-corrected image is displayed on the image display unit 7.
• the color correction unit 272 configured as described above accurately determines the shade number of the tooth in the multi-band image power, and displays the correct color on the tooth on the image display unit 7.
• FIG. 38 is an explanatory diagram showing an eighth embodiment of the present invention.
• the color chart deteriorates due to various factors.
• the actual color chart has some variation from the beginning.
• For use in a dental clinic there is no problem if only a single system is used.For example, as shown in Fig. 36, if there are three sets of imaging systems, any multiband camera and charging unit can be used. I don't know if will be used in combination.
• a color chart characteristic memory storing the spectral reflectance of the color chart is provided in each charging unit, and further correction is performed using the spectral reflectance at the time of calibration. And then.
• the same imaging systems 264A to 264C as those in the seventh embodiment are provided.
• the shooting systems 264A to 264C include multi-band cameras 265A to 265C having the same configuration as the multi-band camera 69 and charging units 262A to 262C having the same configuration as the charging unit 72, respectively.
• the charging units 262A to 262C are provided with color chart characteristic memories 263A to 263C, respectively.
• the multi-band cameras 265A to 265C are connected to the microcomputer 266 that constitutes a common image processing unit for these three imaging systems 264A to 264C. Further, the microcomputer 266 is connected to a microcomputer (not shown) of the dental laboratory 268 via the Internet 267, and is also connected to a microcomputer (not shown) of the data management center 271 via the Internet 267. [0129] In the calibration process by the microcomputer 266,
• the following calculation is performed to correct the variation of each color chart. This allows camera compatibility between multiple systems. This correction is also effective when exchanging data between a dental clinic and a dental laboratory, for example.
• an identification code may be provided on a color chart, and the charging unit may automatically recognize the ID number.
• the means may be a barcode method, a wireless tag method, or the like.

Landscapes

• Physics & Mathematics (AREA)
• Spectroscopy & Molecular Physics (AREA)
• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• General Physics & Mathematics (AREA)
• Color Television Image Signal Generators (AREA)
• Spectrometry And Color Measurement (AREA)
• Image Analysis (AREA)
• Image Processing (AREA)
• Facsimile Image Signal Circuits (AREA)
• Color Image Communication Systems (AREA)

Priority Applications (4)

Applications Claiming Priority (2)

Related Child Applications (1)

Publications (1)

Family

ID=35451281

Family Applications (1)

Country Status (4)

Cited By (8)

Families Citing this family (55)

Citations (4)

Family Cites Families (13)

• 2004
  • 2004-05-26 JP JP2004156750A patent/JP4800591B2/ja not_active Expired - Fee Related
• 2005
  • 2005-05-26 DE DE112005001206T patent/DE112005001206T5/de not_active Withdrawn
  • 2005-05-26 WO PCT/JP2005/009643 patent/WO2005117452A1/ja not_active Ceased
• 2006
  • 2006-11-14 US US11/599,013 patent/US20070064119A1/en not_active Abandoned
• 2010
  • 2010-05-11 US US12/777,430 patent/US20100220211A1/en not_active Abandoned
• 2012
  • 2012-07-13 US US13/548,426 patent/US20120281109A1/en not_active Abandoned

Patent Citations (4)

Also Published As

Similar Documents

Legal Events