WO2006098360A1 - 画像処理装置および画像処理方法、画像処理システム、プログラム、並びに、記録媒体 - Google Patents
画像処理装置および画像処理方法、画像処理システム、プログラム、並びに、記録媒体 Download PDFInfo
- Publication number
- WO2006098360A1 WO2006098360A1 PCT/JP2006/305117 JP2006305117W WO2006098360A1 WO 2006098360 A1 WO2006098360 A1 WO 2006098360A1 JP 2006305117 W JP2006305117 W JP 2006305117W WO 2006098360 A1 WO2006098360 A1 WO 2006098360A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- luminance
- image
- range
- area
- gradation
- Prior art date
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 11
- 238000006243 chemical reaction Methods 0.000 claims description 575
- 238000003384 imaging method Methods 0.000 claims description 182
- 238000000034 method Methods 0.000 claims description 111
- 230000008569 process Effects 0.000 claims description 105
- 230000010365 information processing Effects 0.000 claims description 15
- 239000004065 semiconductor Substances 0.000 claims description 12
- 230000006835 compression Effects 0.000 abstract description 28
- 238000007906 compression Methods 0.000 abstract description 28
- 238000004364 calculation method Methods 0.000 description 165
- 238000010586 diagram Methods 0.000 description 103
- 238000004458 analytical method Methods 0.000 description 48
- 238000001514 detection method Methods 0.000 description 30
- 238000009826 distribution Methods 0.000 description 24
- 230000006854 communication Effects 0.000 description 18
- 238000004891 communication Methods 0.000 description 17
- 238000003860 storage Methods 0.000 description 17
- 230000005540 biological transmission Effects 0.000 description 16
- 230000035945 sensitivity Effects 0.000 description 12
- 230000008859 change Effects 0.000 description 8
- 238000000605 extraction Methods 0.000 description 8
- ORQBXQOJMQIAOY-UHFFFAOYSA-N nobelium Chemical compound [No] ORQBXQOJMQIAOY-UHFFFAOYSA-N 0.000 description 8
- GGCZERPQGJTIQP-UHFFFAOYSA-N sodium;9,10-dioxoanthracene-2-sulfonic acid Chemical compound [Na+].C1=CC=C2C(=O)C3=CC(S(=O)(=O)O)=CC=C3C(=O)C2=C1 GGCZERPQGJTIQP-UHFFFAOYSA-N 0.000 description 5
- 239000000284 extract Substances 0.000 description 4
- 238000005286 illumination Methods 0.000 description 4
- 230000000295 complement effect Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000002194 synthesizing effect Effects 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 230000004069 differentiation Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000002360 explosive Substances 0.000 description 2
- 230000033001 locomotion Effects 0.000 description 2
- 229910052751 metal Inorganic materials 0.000 description 2
- 239000002184 metal Substances 0.000 description 2
- 229910044991 metal oxide Inorganic materials 0.000 description 2
- 150000004706 metal oxides Chemical class 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000001454 recorded image Methods 0.000 description 2
- 229920006395 saturated elastomer Polymers 0.000 description 2
- 229910052709 silver Inorganic materials 0.000 description 2
- 239000004332 silver Substances 0.000 description 2
- -1 silver halide Chemical class 0.000 description 2
- 230000037303 wrinkles Effects 0.000 description 2
- 241000282412 Homo Species 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000000593 degrading effect Effects 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 239000004744 fabric Substances 0.000 description 1
- 230000005669 field effect Effects 0.000 description 1
- 230000007274 generation of a signal involved in cell-cell signaling Effects 0.000 description 1
- 230000003760 hair shine Effects 0.000 description 1
- 230000004297 night vision Effects 0.000 description 1
- 239000003973 paint Substances 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000010248 power generation Methods 0.000 description 1
- 238000003825 pressing Methods 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 238000009738 saturating Methods 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/14—Picture signal circuitry for video frequency region
- H04N5/20—Circuitry for controlling amplitude response
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration using histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
- G06T5/92—Dynamic range modification of images or parts thereof based on global image properties
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/71—Circuitry for evaluating the brightness variation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/741—Circuitry for compensating brightness variation in the scene by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/76—Circuitry for compensating brightness variation in the scene by influencing the image signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/78—Television signal recording using magnetic recording
- H04N5/782—Television signal recording using magnetic recording on tape
Definitions
- Image processing apparatus image processing method, image processing system, program, and recording medium
- the present invention relates to an image processing apparatus, an image processing method, an image processing system, a program, and a recording medium, and in particular, luminance compression so as to be easy to handle when various processes are performed using an image.
- the present invention relates to an image processing apparatus, an image processing method, an image processing system, a program, and a recording medium that can obtain an image having a wide dynamic range.
- Patent Document 1 Japanese Unexamined Patent Publication No. 2000-32303
- FIG. 1 is a block diagram showing a configuration of a conventional wide dynamic range camera that can obtain an image with a wide dynamic range by synthesizing images captured by a CCD at different shot times.
- the solid-state imaging device 11 captures a high-speed shotta image and a low-speed shotta image based on the control of the exposure control unit 21 of the control unit 14.
- the low-speed shot image is, for example, an image with a shot-ta speed of S1Z60
- the high-speed shot image is, for example, an image with a shot-ta speed of 1Z2000.
- the video signal obtained by the solid-state imaging device 11, that is, the low speed shot image Z, the high speed shot image is converted into an analog Z digital signal by the AZD converter 12, and the frame memory 31-1 or 31 of the digital processing unit 13 is converted. — Alternately written to 2.
- the signal read from the frame memory 31-1 or 31-2 is supplied to the synthesis processing circuit 32, synthesized there, and then outputted through the process circuit 33.
- the control unit 14 is composed of a CPU 22 and an exposure control unit 21.
- the CPU 22 performs calculations using the photometric data supplied from the digital processing unit 13, supplies the result to the exposure control unit 21 that controls the shutter speed and aperture of the solid-state imaging element 11, and based on the result.
- the digital processing unit 13 is controlled.
- a low-speed shotta captures a low brightness !, part (high brightness !, part is saturated) and captures a high-speed shotta.
- the high luminance part of the subject is captured (the low luminance part is dark and cannot be captured).
- the pixel value of the pixel in the bright area of the subject is limited to the maximum pixel value that can be output by the CCD image sensor.
- luminance clipping that limits the pixel value of pixels in the dark area of the subject to the minimum pixel value that can be output by the CCD image sensor, so that an image with a wide dynamic range can be obtained as described above. Requires complex image composition processing.
- the present invention has been made in view of such a situation, and it is possible to handle brightness when compressing various types of processing using an image without performing complicated processing. It is possible to obtain an image with a wide dynamic range.
- the image processing apparatus of the present invention includes an acquisition unit that acquires an image signal composed of pixel values that are substantially proportional to the logarithm of the incident light amount, and a total luminance range of the image signal acquired by the acquisition unit.
- a luminance range setting means for setting a predetermined luminance range; and the acquisition means so that more gradations are assigned to the predetermined luminance range set by the luminance range setting means than outside the luminance range.
- Conversion means for converting the luminance of the pixels included in the image signal.
- the acquisition unit, the luminance range setting unit, or the conversion unit is configured by dedicated hardware or realized by a computer or the like that has read a program.
- the computer is, for example, a CPU (Central Processing Unit), Consists of computing devices such as DSP (Digital Signal Processor).
- the image data obtained in this way when the image data obtained in this way is displayed, the image data can be recognized by the user, and the image data can be easily deleted.
- an image printing apparatus, an image recognition apparatus, an image recording apparatus In an external device that uses an image such as an image communication device, it can be used as wide dynamic range image data with compressed luminance.
- the luminance range setting means can set a plurality of the predetermined luminance ranges.
- a ZD conversion means for performing AZD conversion on the image signal in the entire luminance range of the image signal acquired by the acquisition means can be further provided.
- the image signal converted into the digital signal can be converted.
- the AZD conversion means is realized by a force constituted by dedicated hardware, a computer that reads a program, and the like, and the computer is constituted by an arithmetic device such as a CPU or DSP, for example.
- the conversion means acquisition is performed by the acquisition means so that the number of luminance gradations after conversion is less than the number of luminance gradations of the pixels included in the image signal converted into a digital signal by the AZD conversion means.
- the converted image signal can be converted.
- luminance is compressed means, for example, that the luminance is reduced by the step number power conversion of the luminance gradation that the entire captured image data has.
- the luminance gradation number of the image signal converted by the conversion means is converted into a gradation number that can be processed by a predetermined external device, and output for controlling the output of the converted image signal to the external device Control means can be further provided.
- the output control means is realized by a force constituted by dedicated hardware, a computer reading a program, and the like, and the computer is constituted by an arithmetic device such as a CPU or DSP, for example.
- the predetermined external device includes, for example, various information processing devices that execute processing using an image, such as an image printing device, an image recognition device, an image recording device, and an image communication device.
- the image signal converted by the conversion means is notably the luminance of the imaging target and the luminance of the image data. There is no need to correspond one-to-one with the data (the luminance of the object to be imaged and the luminance data of the image data correspond to the linear)! It is preferable to be used in an apparatus that executes image processing for executing wrinkle processing.
- the conversion means can further comprise AZD conversion means for AZD conversion of the image signal acquired by the acquisition means, and a determination means for determining the number of gradation steps in AZD conversion by the AZD conversion means.
- the determining means can determine the number of gradation steps in the AZD conversion so that more gradations are assigned to the predetermined luminance range set by the luminance range setting means than outside the luminance range.
- the AZD conversion means can perform AZD conversion of the image signal based on the number of gradation steps determined by the determination means.
- the AZD conversion means and the determination means are configured by dedicated hardware, and are realized by a computer or the like that has read a program.
- the computer is configured by an arithmetic device such as a CPU or DSP, for example.
- the ratio between the input analog signal level before AZD conversion and the digital signal level after conversion is constant.
- general image processing such as binarization and detection of a predetermined object becomes difficult.
- the number of key steps is relatively small, for example, there is no difference in image density when displayed or printed out.
- an image in the set luminance range has, for example, sufficient shading. Displayed or printed out with a difference.
- the threshold for binarization can be easily determined in various image processing, or based on the image. Predetermined target An object can be easily detected.
- the image signal can be captured by a logarithmic conversion type imaging device that outputs a pixel value substantially proportional to the logarithm of the incident light quantity by utilizing the sub-threshold characteristic of the semiconductor.
- the image pickup device is capable of picking up an image of a subject with a wider dynamic range than a human eye.
- the image processing method of the present invention is an image processing method of an image processing apparatus that processes a captured image signal that has a pixel value that is substantially proportional to the logarithm of the amount of incident light, and that acquires an image signal.
- a luminance range setting step for setting a predetermined luminance range out of the entire luminance range of the image signal acquired by the step and the processing of the acquisition step, and the predetermined luminance range set by the processing of the luminance range setting step.
- the image signal for example, can be taken by a logarithmic conversion type imaging device that outputs a pixel value that is substantially proportional to the logarithm of the incident light quantity by utilizing the sub-threshold characteristic of the semiconductor. .
- the image sensor is, for example, HDRC (High Dynamic Range CMOS (Complementary Metal
- Such an image sensor can capture a subject with a wider dynamic range than the human eye.
- the conversion step may convert the image signal converted into a digital signal by AZD conversion. it can.
- the conversion step it is necessary to reduce the number of luminance gradations after conversion from the number of luminance gradations of pixels included in the image signal converted into a digital signal by AZD conversion. Let's convert the image signal acquired by the acquisition step.
- the conversion step includes an AZD conversion step for AZD conversion of the image signal acquired in the acquisition step, and a number of gradation steps in the AZD conversion by the AZD conversion step.
- the number of gradation steps in AZD conversion can be determined, and in the AZD conversion step, the image signal can be AZD converted based on the number of gradation steps determined in the determination step. it can.
- the program of the present invention and the program recorded on the recording medium are programs for causing a computer to process a captured image signal having a pixel value substantially proportional to the logarithm of the incident light quantity.
- An acquisition control step for controlling the acquisition of the image signal, a luminance range setting step for setting a predetermined luminance range among all the luminance ranges of the image signal whose acquisition is controlled by the processing of the acquisition control step, and a luminance range setting The luminance of the pixels included in the image signal whose acquisition is controlled by the processing of the acquisition control step so that more gradations are assigned to the predetermined luminance range set by the processing of the step than outside the luminance range.
- the computer is caused to execute processing characterized by including a conversion step for converting.
- the image signal for example, can be taken by a logarithmic conversion type imaging device that outputs a pixel value that is substantially proportional to the logarithm of the incident light amount by utilizing the subthreshold characteristic of the semiconductor. .
- the image sensor is, for example, HDRC (High Dynamic Range CMOS (Complementary Metal
- Such an image pickup device can pick up an image of a subject with a wider dynamic range than human eyes.
- the conversion step may convert the image signal converted into a digital signal by AZD conversion. it can.
- the conversion step it is necessary to reduce the number of gradations of the luminance after conversion from the number of gradations of the luminance of the pixels included in the image signal converted into a digital signal by AZD conversion. Let's convert the image signal acquired by the acquisition step.
- the image signal acquired in the acquisition step is AZD converted.
- An AZD conversion step and a determination step for determining the number of gradation steps in the AZD conversion by the AZD conversion step can be further included.
- the determination step the predetermined range set by the luminance range setting step is included. It is possible to determine the number of gradation steps in the AZD conversion so that more gradations are assigned to the luminance range than outside the luminance range.
- the gradation step determined by the determination step is determined. Based on the number of images, the image signal can be AZD converted.
- a first image processing system of the present invention includes an image processing device that processes an image signal picked up by an image pickup device that outputs a pixel value that is substantially proportional to the logarithm of the amount of incident light, and a processing performed by the image processing device.
- An image processing system configured to include an information processing device that performs processing using the captured image signal, the image processing device acquiring an image signal captured by the image sensor and an acquisition unit Among the luminance ranges of the received image signal, the luminance range setting means for setting a plurality of luminance ranges, and the gradation range set by the luminance range setting means are assigned more gradations than outside the luminance range, and A conversion means for converting the image signal acquired by the acquisition means so that the number of luminance gradations is less than the number of luminance gradations of the pixels included in the image signal acquired by the acquisition means; Output that controls the output of the converted image signal to the information processing device while converting the number of luminance gradations of the image signal converted by the conversion means to the number of gradations that can be processed by the information processing device. And a control means.
- An imaging device that performs imaging using an imaging device that outputs a pixel value substantially proportional to the logarithm of the incident light quantity can be further provided.
- the image pickup device can be, for example, a logarithmic conversion type image pickup device that outputs a pixel value substantially proportional to the logarithm of the incident light amount by utilizing a sub-threshold characteristic of a semiconductor.
- a logarithmic conversion type image pickup device that outputs a pixel value substantially proportional to the logarithm of the incident light amount by utilizing a sub-threshold characteristic of a semiconductor.
- the information processing apparatus includes various information processing apparatuses that execute processing using an image, such as an image printing apparatus, an image recognition apparatus, an image recording apparatus, and an image communication apparatus.
- an image such as an image printing apparatus, an image recognition apparatus, an image recording apparatus, and an image communication apparatus.
- the luminance of the imaging target and the luminance data of the image data correspond one-to-one (the luminance of the imaging target and the luminance data of the image data correspond linearly). Yes) There is no need! ⁇ Perform image processing Information processing to perform image processing It is preferable to be used in a physical device.
- the second image processing system of the present invention includes an image processing device that processes a captured image signal, which has a pixel value substantially proportional to the logarithm of the incident light amount, and an image signal processed by the image processing device.
- the image processing apparatus includes an acquisition unit that acquires an image signal and a predetermined luminance range among the entire luminance range of the image signal acquired by the acquisition unit.
- Brightness range setting means for setting the image
- AZD conversion means for AZD conversion of the image signal acquired by the acquisition means
- determination means for determining the number of gradation steps in AZD conversion by the AZD conversion means
- conversion by the AZD conversion means Output control means for controlling the output of the received image signal to the information processing apparatus, and the determining means has a predetermined brightness range set by the brightness range setting means and a gradation greater than that outside the brightness range.
- the number of gradation steps in the AZD conversion is determined so as to be assigned, and the A / D conversion means performs A and D conversion on the image signal based on the number of gradation steps determined by the determination means. .
- the acquired image signal is captured by a logarithmic conversion type image sensor that outputs a pixel value substantially proportional to the logarithm of the incident light amount using, for example, the subthreshold characteristic of the semiconductor.
- the imaging device can be, for example, HDRC.
- the degree of the image signal captured by the image sensor that outputs a pixel value that is substantially proportional to the logarithm of the incident light quantity.
- a predetermined luminance range is set, and the luminance of the pixels included in the image signal is converted so that more gradations are assigned to this luminance range than outside the luminance range.
- a captured image signal can be processed, and in particular, an image with a wide dynamic range can be obtained with a small number of luminance gradation steps without performing complicated processing.
- FIG. 1 is a diagram illustrating the operating principle of a conventional wide dynamic range camera.
- FIG. 2 is a block diagram showing an embodiment of an image processing system of the present invention.
- FIG. 3 is a block diagram showing an embodiment of the imaging unit in FIG.
- FIG. 4 is a graph showing sensitivity characteristics of a logarithmic conversion type image sensor.
- FIG. 5 is a block diagram illustrating a first configuration example of an image generation unit.
- FIG. 6A is a diagram for explaining the conversion of the output level with respect to the input level.
- FIG. 6B is a diagram for explaining the conversion of the output level with respect to the input level.
- FIG. 6C is a diagram for explaining the conversion of the output level with respect to the input level.
- FIG. 7A is a diagram for explaining conversion of an output level with respect to an input level.
- FIG. 7B is a diagram for explaining the conversion of the output level with respect to the input level.
- FIG. 7C is a diagram for explaining the conversion of the output level with respect to the input level.
- FIG. 8A is a diagram for explaining an input signal, a signal after gradation conversion, and a signal to be displayed.
- FIG. 8B is a diagram for explaining an input signal, a signal after gradation conversion, and a signal to be displayed.
- FIG. 8C is a diagram for explaining an input signal, a signal after gradation conversion, and a signal to be displayed.
- FIG. 9 is a flowchart for explaining image display processing 1.
- FIG. 10 is a diagram for explaining a difference in display images.
- FIG. 11 is a diagram for explaining a difference in display images.
- FIG. 12 is a diagram for explaining a difference in display images.
- FIG. 13 is a diagram for explaining a display image.
- FIG. 14 is a block diagram illustrating a second configuration example of the image generation unit.
- FIG. 15A is a diagram for explaining the conversion of the output level with respect to the input level.
- FIG. 15B is a diagram for explaining the conversion of the output level with respect to the input level.
- FIG. 15C is a diagram for explaining the conversion of the output level with respect to the input level.
- FIG. 16A is a diagram for explaining an input signal, a signal after gradation conversion, and a signal to be displayed.
- FIG. 16B is a diagram for explaining an input signal, a signal after gradation conversion, and a signal to be displayed.
- FIG. 16C is a diagram for describing an input signal, a signal after gradation conversion, and a signal to be displayed.
- FIG. 17 is a flowchart for explaining image display processing 2;
- FIG. 18 is a diagram for explaining a display image.
- FIG. 19 is a diagram for explaining a display image.
- FIG. 20 is a block diagram illustrating a third configuration example of the image generation unit.
- FIG. 21 is a block diagram illustrating a fourth configuration example of the image generation unit.
- FIG. 22 is a diagram for explaining the cutout region.
- FIG. 23 is a flowchart for explaining image display processing 3;
- FIG. 24 is a block diagram illustrating a fifth configuration example of the image generation unit.
- FIG. 25 is a flowchart for explaining image display processing 4.
- FIG. 26 is a block diagram illustrating a sixth configuration example of the image generation unit.
- FIG. 27 is a block diagram illustrating a seventh configuration example of the image generation unit.
- FIG. 28 is a diagram for explaining histogram analysis and luminance range setting.
- FIG. 29 is a flowchart for explaining image display processing 5;
- FIG. 30 is a block diagram illustrating an eighth configuration example of an image generation unit.
- FIG. 31 is a flowchart for explaining image display processing 6;
- FIG. 32 is a block diagram illustrating a ninth configuration example of the image generation unit.
- FIG. 33 is a diagram for explaining histogram analysis and luminance range setting.
- FIG. 34 is a flowchart for explaining image display processing 7;
- FIG. 35 is a block diagram showing an image processing system according to an embodiment of the present invention.
- FIG. 36 is a block diagram showing an embodiment of the imaging unit in FIG. 35.
- FIG. 37 is a block diagram illustrating a first configuration example of a gradation assignment determination unit.
- FIG. 38A is a diagram for explaining gradation assignment for AZD conversion.
- FIG. 38B is a diagram for explaining gradation assignment for AZD conversion.
- FIG. 38C is a diagram for explaining gradation assignment for AZD conversion.
- FIG. 39A is a diagram for explaining gradation assignment for AZD conversion.
- FIG. 39B is a diagram for describing gradation assignment for AZD conversion.
- FIG. 39C is a diagram for describing gradation assignment for AZD conversion.
- FIG. 40A is a diagram for describing an input analog signal, a signal after AZD conversion, and a signal to be displayed.
- FIG. 40B is a diagram for describing an input analog signal, a signal after AZD conversion, and a signal to be displayed.
- FIG. 40C is a diagram for describing an input analog signal, a signal after AZD conversion, and a signal to be displayed.
- [41A] It is a diagram for explaining gradation assignment of AZD conversion.
- FIG. 41B is a diagram for describing gradation assignment for AZD conversion.
- ⁇ 41C It is a diagram for explaining gradation assignment of AZD conversion.
- FIG. 42A is a diagram for describing an input analog signal, a signal after AZD conversion, and a signal to be displayed.
- FIG. 42B is a diagram for describing an input analog signal, a signal after AZD conversion, and a signal to be displayed.
- FIG. 42C is a diagram for describing an input analog signal, a signal after AZD conversion, and a signal to be displayed.
- FIG. 43 is a flowchart for explaining image display processing 8;
- FIG. 44 is a block diagram illustrating a second configuration example of the gradation assignment determining unit.
- FIG. 45A is a diagram for explaining gradation assignment for AZD conversion.
- FIG. 45B is a diagram for explaining gradation assignment for AZD conversion.
- FIG. 45C is a diagram for explaining gradation assignment for AZD conversion.
- FIG. 46A is a diagram for explaining an input analog signal, a signal after AZD conversion, and a signal to be displayed.
- FIG. 46B is a diagram for explaining an input analog signal, a signal after AZD conversion, and a signal to be displayed.
- FIG. 46C is a diagram for describing an input analog signal, a signal after AZD conversion, and a signal to be displayed.
- FIG. 47 is a flowchart for explaining image display processing 9;
- FIG. 48 is a block diagram showing a third configuration example of the gradation assignment determining unit.
- FIG. 49 is a block diagram illustrating a fourth configuration example of the gradation assignment determining unit.
- FIG. 50 is a flowchart for explaining image display processing 10.
- FIG. 51 is a block diagram showing a fifth configuration example of the gradation assignment determining unit.
- FIG. 52 is a flowchart for explaining image display processing 11.
- FIG. 53 is a block diagram illustrating a sixth configuration example of the gradation assignment determining unit.
- FIG. 54 is a block diagram showing a seventh configuration example of the gradation assignment determining unit.
- FIG. 55 is a diagram for explaining histogram analysis and luminance range setting.
- FIG. 56 is a flowchart for explaining image display processing 12
- FIG. 57 is a block diagram illustrating an eighth configuration example of a gradation assignment determining unit.
- FIG. 58 is a flowchart for explaining image display processing 13;
- FIG. 59 is a block diagram illustrating a ninth configuration example of the gradation assignment determining unit.
- FIG. 60 is a diagram for explaining histogram analysis and luminance range setting.
- FIG. 61 is a flowchart for explaining image display processing 14;
- FIG. 62 is a block diagram illustrating a configuration example of a personal computer.
- FIG. 2 is a block diagram showing an embodiment of the image processing device 81.
- the imaging unit 91 images the subject based on the user's operation input supplied from the operation input unit 92 and supplies the obtained image signal to the image generation unit 93. Details of the imaging unit 91 will be described later with reference to FIG.
- the operation input unit 92 includes input devices such as buttons such as a relays button, operation keys, and a touch panel, for example, receives an operation input from the user, and supplies a command from the user to the imaging unit 91.
- the user's operation input is, for example, an imaging timing command (relays In addition to pressing a button, for example, it may be a command to start or end moving image capturing (in other words, continuous frame imaging), and further, the time of image capturing start or end It may be a setting.
- the operation input unit 92 receives an input of a predetermined set value used for processing of the image generation unit 93 by the user, the operation input unit 92 supplies the set value to the image generation unit 93.
- the image generation unit 93 executes processing for converting the image signal supplied from the imaging unit 91 into an image signal suitable for display and print output, and supplies the image signal to the display control unit 94 or the output control unit 95.
- the image generation unit 93 receives input of setting values necessary for processing from the operation input unit 92 as necessary, and based on this, the image signal supplied from the imaging unit 91 is displayed or printed. A process of converting to a suitable image signal is executed.
- the display control unit 94 performs processing for converting the processed image signal supplied from the image generation unit 93 into the resolution and the number of gradations of the display 82, and supplies the processed signal to the display 82. To do.
- the output control unit 95 performs a process of converting the processed image signal supplied from the image generation unit 93 into a resolution and the number of gradations that can be processed by the image using device 83, and the processed signal is Output to the image utilization device 83.
- the display 82 receives an input of a display image signal supplied from the display control unit 94, and displays an image (a still image or a moving image including a plurality of frames).
- the image utilization device 83 requests input of the image signal supplied from the output control unit 95, and executes predetermined processing.
- various information processing devices that perform processing using an image such as an image printing device, an image recognition device, an image recording device, and an image communication device may be used. it can.
- the image signal generated by the image generation unit 93 has a one-to-one correspondence between the luminance of the imaging target and the luminance data of the image data (the luminance of the imaging target, the luminance data of the image data, and the power S). It is preferable to use the image utilization apparatus 83 that executes processing that does not need to be performed in a linear manner.
- the brightness of the imaging target and the brightness data of the image data do not have to correspond one-to-one!
- the processing includes, for example, print output, recording processing, processing for recognizing a predetermined object in the image, image There are processing to detect the edge portion or straight line portion in the inside, binarization processing, or transmission processing to other devices that execute such processing.
- FIG. 3 is a block diagram illustrating a more detailed configuration example of the imaging unit 91 of the image processing apparatus 81 in FIG.
- the imaging unit 91 is configured to include a lens 101 and a logarithmic conversion type imaging device 102.
- the logarithmic conversion type imaging device 102 is a logarithmic conversion type imaging device such as HDRC (High Dynamic Range CMOS (Complementary Metal Oxide Semiconductor)), for example, and includes a light detection unit 111, a logarithmic conversion unit 112, an AZD conversion unit 113, And an imaging timing control unit 114.
- HDRC High Dynamic Range CMOS (Complementary Metal Oxide Semiconductor)
- the light that is also generated by the subject imaged by the imaging unit 91 enters the lens 101, and the light (not shown) of the light detection unit 111 of the logarithmic conversion type imaging device 102 An image is formed on the detection surface.
- the light detection unit 111 is configured by, for example, a light-receiving element that has power, such as a plurality of photodiodes.
- the light detection unit 111 converts the light of the subject imaged by the lens 101 into a charge corresponding to the brightness (illuminance) of the incident light, and accumulates the converted charge.
- the light detection unit 111 supplies the accumulated charge to the logarithmic conversion unit 112 in synchronization with the control signal supplied from the imaging timing control unit 114.
- the logarithmic conversion unit 112 is configured by, for example, a plurality of MOSFETs (Metal Oxide Semiconductor Field Effect Transistors).
- the logarithmic conversion unit 112 uses the sub-threshold characteristic of the MOSFET to convert the charge supplied from the light detection unit 111 to the logarithm of the number of charges (the intensity of current) for each pixel (the logarithm of the amount of light of the subject).
- An analog electrical signal converted to a voltage value approximately proportional to is generated.
- the logarithmic conversion unit 112 supplies the generated analog electrical signal to the AZD conversion unit 113.
- the AZD conversion unit 113 performs AZD conversion of an analog electrical signal into digital image data in synchronization with the control signal supplied from the imaging timing control unit 114. For example, when converted to 24-bit unsigned binary digital image data, the pixel value of the image data takes a value in the range from 0, the darkest to 224-1, which is the darkest. The AZD conversion unit 113 supplies the converted digital image data to the image processing device 92.
- the imaging unit 91 determines the brightness of the light of the subject incident on the light detection unit 111 (incident light amount). Digital image data having a pixel value power proportional to the logarithm of) is output.
- the details of the logarithmic conversion type image sensor are disclosed in, for example, JP-T-7-506932.
- FIG. 4 is a graph showing sensitivity characteristics of the logarithmic conversion type imaging device 102, a CCD (Charge Coupled Device) imaging device, a silver salt film, and the human eye.
- the horizontal axis in Fig. 4 shows the logarithmic value of the illuminance of incident light (unit: lux), and the vertical axis shows the sensitivity to the illuminance of incident light.
- Line 121 shows the sensitivity characteristic of the logarithmic conversion image sensor 102
- line 122 shows the sensitivity characteristic of the CCD image sensor
- line 123 shows the sensitivity characteristic of the silver salt film
- line 124 shows the sensitivity characteristic of the human eye.
- the logarithmic conversion type image pickup element 102 outputs image data composed of luminance values (or pixel values) substantially proportional to the logarithm of the incident light quantity. Therefore, even when the incident light quantity increases, The capacitance of elements such as photodiodes and MOSFETs constituting the logarithmic conversion type image sensor 102 is saturated, and the current flowing through each element and the applied voltage exceed the range in which output corresponding to the input of each element can be performed. There is nothing. Therefore, it is possible to obtain a luminance value (or a pixel value) according to the variation in the amount of incident light almost accurately within the range of luminance that can be imaged.
- the intensity of the incident light intensity of the subject force is wider than the human eye, for example, about 170 dB dynamic range from about 1 millilux to about 500 kilolux higher than the brightness of sunlight. It is possible to capture an image consisting of a luminance value (or pixel value) that accurately reflects.
- the dynamic range of the logarithmic conversion type image sensor 102 used in the imaging unit 91 is not limited to the above-mentioned 170 dB, and corresponds to the required dynamic range, such as about 100 dB or 200 dB, depending on the purpose of use. If you use something,
- the logarithmic conversion type image pickup element 102 outputs image data having a pixel value having a value approximately proportional to the logarithm of the incident light quantity, thereby forming the logarithmic conversion type image pickup element 102.
- the CCD image sensor, the silver salt film, and about 1 millilux force which is wider than the human eye, is higher than the brightness of sunlight.
- the subject can be imaged with a dynamic range of approximately 170 dB up to k lux.
- the imaging unit 91 using the logarithmic conversion type imaging device 102 does not generate luminance clipping in the luminance range that can be visually recognized by humans, and therefore adjusts the amount of incident light by adjusting the aperture, shirt speed, and the like. There is no need. That is, the imaging unit 91 can faithfully capture a detailed change in luminance of the subject without adjusting the amount of incident light.
- the imaging unit 91 when imaging the front of the vehicle from the inside of the vehicle in the daytime, the imaging unit 91 faithfully reproduces the state of the sun and the road ahead without adjusting the amount of incident light even if the sun is within the angle of view. It is possible to capture a reproducible image.
- the imaging unit 91 when imaging the front of the vehicle from inside the vehicle at night, the imaging unit 91 does not adjust the amount of incident light and adjusts itself from the light of the headlight of the oncoming vehicle, even if the headlight of the oncoming vehicle is also illuminated. It is possible to capture an image that faithfully reproduces even the part not illuminated by the car headlight.
- the logarithmic conversion is performed in comparison with the case where the sensitivity characteristic is not proportional to the logarithm of the illuminance of incident light due to factors such as the gamma characteristic.
- the sensitivity characteristic is almost proportional to the logarithm of the illuminance of incident light.
- the image pickup unit 91 using the logarithmic conversion type image pickup element 102 is not affected by the occurrence of luminance clipping, adjustment of the amount of incident light, and gamma characteristics.
- the pixel value fluctuates so as to reflect the change in luminance of the subject and the movement of the subject almost faithfully.
- the pixel value of the image data output from the imaging unit 91 is a value that is approximately proportional to the logarithm of the incident light quantity, so that regardless of the brightness (illuminance) of the light applied to the subject.
- the difference in the histogram width indicating the pixel value distribution of the first and second image data is about 100 times.
- the variation values of the pixel values of the image data obtained by imaging the subject are substantially the same. For example, if there are two areas with a luminance ratio of 100: 1 in the subject, the illuminance of the light radiated to the subject changes almost uniformly, and the subject's luminance changes by + 5%, which is about the same ratio. When this is done, the fluctuation values of the pixel values corresponding to the two areas are almost the same value (log 1.05).
- the difference between the fluctuation values of the pixel values corresponding to the two areas described above is about 100 times.
- the logarithmic conversion type image sensor 102 has the sensitivity of the human eye (characteristic shown by the line 124 in FIG. 4), the sensitivity of the CCD image sensor (in FIG. 4). Compared to the characteristics shown by line 122) and the sensitivity of the silver salt film (characteristic shown by line 123 in Fig. 4), the brightness range is very wide. For this reason, when all gradations of image data captured by the imaging unit 91 using the logarithmic conversion type image sensor 102 are converted into gradations corresponding to the display 82 and displayed, they are visible to the human eye. Compared to images and images taken using a conventional CCD image sensor or silver halide film, there is no difference in density, in other words, an image without sharpness.
- the force-captured image data is obtained by faithfully capturing both a very dark part and a very bright part.
- the display 82 is displayed with the gradation converted based on the number of gradations that can be displayed, both the captured and very bright parts and the very bright parts are referred to the displayed image by the user.
- the central portion of the entire gradation that can be identified by the user is displayed with a small difference in shading.
- the captured image data is converted to the number of gradations that can be processed by the output destination device, the printed image will be Similarly, there is no difference in shading compared to images that are visible to the human eye or images captured using a conventional CCD image sensor or silver halide film, in other words, an image that is not sharp. .
- the luminance range that can be obtained by converting the entire luminance range of an image captured by the imaging unit 91 using the logarithmic conversion type imaging element 102 into a gradation that can be displayed on the display 82 is not obtained.
- the highest and lowest luminance values of the pixels included in the obtained image are extracted.
- the image quality of the image displayed when the luminance range included in one image is relatively narrow is clearly improved.
- the user's ability to refer to the displayed image can be used for both the very dark and very bright images. Is indistinguishable, resulting in a display image with a small difference in shading in the central part of the overall gradation that can be identified by the user.
- the image generation unit 93 when the image data captured by the imaging unit 91 using the logarithmic conversion type image sensor 102 is supplied and displayed at a gradation corresponding to the display 82, or Image processing that can generate image data that can be displayed or printed out in a state that can be identified by the user even when output at a gradation corresponding to the external device of the output destination Is executed.
- FIG. 5 is a block diagram showing a configuration of an image generation unit 93-1 which is a first example of the configuration of the image generation unit 93 in FIG.
- the average luminance calculation unit 131 acquires the image signal supplied from the imaging unit 91, calculates the average luminance, and supplies the average luminance calculation result to the main region luminance range setting unit 132.
- the main area luminance range setting unit 132 sets the luminance range of the main area based on the average luminance of the image signal supplied from the average luminance calculation unit 131, and outputs the set luminance range of the main area as an output level.
- a conversion processing unit 135 and a second luminance area luminance average value calculating unit 133 are supplied.
- the main area luminance range setting unit 132 may use a predetermined luminance range centered on the average luminance of the image signal as the main area luminance range, for example, and the pixel that is the average luminance of the image signal. As a center, a pixel having a predetermined number of bits may be selected in order from a pixel having a luminance value close to the luminance, and the luminance range of the main region may be selected.
- the second luminance area luminance average value calculation unit 133 has a higher luminance range than the luminance range of the main region set by the main region luminance range setting unit 132 out of the image signal supplied from the imaging unit 91.
- the average brightness of the pixels is calculated, and the calculation result is supplied to the second brightness area brightness range setting unit 134.
- the second luminance area luminance range setting unit 134 is based on the average luminance of the pixels in the higher luminance range than the luminance range of the supplied main area.
- the brightness range of the second brightness area is set, and the set brightness range of the second brightness area is supplied to the output level conversion processing unit 135.
- the second luminance area luminance range setting unit 134 sets a predetermined luminance range centered on the average luminance of pixels in a higher luminance range than the luminance range of the main area to the luminance range of the second luminance area.
- a pixel having a predetermined number of bits may be selected in order from a pixel having a luminance value close to this luminance around a pixel having an average luminance of pixels in a luminance range higher than the luminance range of the main region.
- the luminance range of the second luminance region may be used.
- the output level conversion processing unit 135 acquires the image signal supplied from the imaging unit 91 and is supplied from the main region luminance range setting unit 132 and the second luminance region luminance average value calculation unit 133. The output level of the acquired image signal is converted based on the information on the luminance range of the main area and the luminance range of the second luminance area.
- the output level conversion processing unit 135 outputs an output level signal having a predetermined number of steps (in the case where the luminance is divided by a predetermined number of gradation steps) with respect to the level of the input luminance signal. (Brightness gradation level signal) is assigned, and the assigned output level signal is output.
- the output level conversion processing unit 187 assigns the luminance gradation level of the output level signal differently between the luminance range set as the main region and the second luminance region and the other ranges. Shall. That is, the output level conversion processing unit 187 assigns a larger number of output level steps to the luminance ranges set as the main region and the second luminance region, and the number of gradations of pixels in the corresponding luminance range increases. A conversion process like this is executed. In this way, the partial force in the luminance range corresponding to the main area and the second luminance area in the displayed or printed image can be recognized more by the user.
- the output level conversion processing unit 135 divides and assigns all the gradation step numbers to the input signals included in the luminance range of the main area and the luminance range of the second luminance area, for example. Range, that is, a luminance range lower than the main region, a luminance range between the main region and the second luminance region, and a range that is higher than the second luminance region. It is possible to do so without assigning the number of steps at the same level.
- the output of the pixel whose luminance input level is lower than the luminance range of the main area is output.
- the level is 0 (ie black).
- a pixel within the luminance range of the main area is assigned an output level of a predetermined gradation according to the input level.
- the pixels in the luminance range between the main area and the second luminance area are output at the maximum value of the output level assigned to the luminance range of the main area regardless of the input level.
- the pixels in the second luminance area have the same or substantially the same number of steps as the main area, and the overall output level is determined from the maximum output level assigned to the luminance range of the main area according to the input level.
- a gray level output level up to the maximum value of the bell is assigned.
- a pixel having a higher luminance than the second luminance area is assigned the maximum value of the output level assigned to the second luminance area, that is, the maximum value of the entire output level, regardless of the input level.
- the output level conversion processing unit 135, assigns a predetermined number of steps of the output level to the main area and the second luminance area, and between the main area and the second luminance area.
- the number of steps is less than the number of steps assigned to the main area or the second luminance area (in other words, the gradation width is narrower than the main area or the second luminance range). Can be assigned to the lower brightness range than the main area and the higher brightness range than the second brightness area without assigning the number of output level steps.
- the output of a pixel having a luminance lower than the luminance range of the main region is output.
- the value is 0.
- a pixel within the luminance range of the main area is assigned an output level of a predetermined gradation according to the input level.
- the output level corresponding to the input level is less than the number of steps assigned to the luminance range of the main area. Assigned.
- the pixels in the second luminance area have the same or approximately the same number of steps as the main area, and the maximum output level assigned to the luminance range between the main area and the second luminance area according to the input level.
- the power level is assigned to the output level up to the maximum gradation value. It is. Then, the maximum value of the output level assigned to the second luminance area, that is, the maximum output level is assigned to the pixels having higher luminance than the second luminance area, regardless of the input level.
- the output level conversion processing unit 13 sets a predetermined luminance range centered on the upper limit value of the luminance of the main area as a section (X, a predetermined value centered on the lower limit value of the luminance of the second luminance area).
- the luminance range of the output level is assumed to be section ⁇ , and a predetermined number of steps of the output level is assigned to pixels other than section ⁇ or section j8 in the main area or the second brightness area, and section ⁇ or section ⁇ is assigned. Assign a smaller number of steps than the number of steps assigned to sections other than section a or section ⁇ of the main area and the second brightness area, and lower brightness than the main area and higher brightness than the second brightness area.
- the input level is reduced with a smaller number of steps than the number of steps assigned to the interval a.
- the power that is assigned to the output level according to the level or the maximum value of the output level that is assigned to the section OL regardless of the input level is assigned.
- the pixels in section / 3 have a predetermined number greater than the maximum value of the output level in the luminance range lower than section ⁇ , with the number of steps smaller than the number of steps assigned to the part other than section a of the main area.
- Output level force of the same number of steps The number of steps is the same or approximately the same as interval a Assigned by Then, for pixels other than section ⁇ within the brightness range of the second brightness area, output of gradations up to the maximum output level of the entire maximum value assigned to section j8 according to the input level. A level is assigned.
- the pixels other than the section j8 within the brightness range of the second luminance area have the same or substantially the same number of steps as the portions other than the section a in the main area, that is, than the sections a and j8.
- Output levels with many steps are assigned.
- a pixel having a higher luminance than the second luminance region is assigned the maximum output level assigned to the second luminance region, that is, the maximum output level, regardless of the input level. .
- a predetermined luminance range centered on the upper limit of the luminance of the main area is section a
- a predetermined section on the brightness limit side of the second luminance area is section a
- the second luminance area The predetermined interval on the lower limit side in the luminance range is defined as the interval ⁇ , or the predetermined luminance range centered on the lower limit value of the main region is defined as the interval ⁇ .
- the upper luminance range in the luminance range of the main region A predetermined section having a luminance higher than the upper limit of the second luminance area may be defined as section oc
- a predetermined section having a luminance lower than the lower limit of the luminance range of the second luminance area may be defined as section ⁇ .
- an area that is assigned the same number of output level steps as the section oc and the section ⁇ is set on the lower limit side of the brightness of the main area and the upper limit side of the brightness of the second brightness area. Also good.
- all the pixels having luminance less than or equal to the luminance range of the main region are assumed to be output 0 (black), and the range of luminance exceeding the luminance range of the second luminance region. All pixels have the same output level (maximum output level) as the highest brightness in the brightness range of the second brightness area, but below the brightness range of the main area and in the brightness range of the second brightness area. In each of the above luminance ranges, a certain number of output level steps may be assigned.
- the output level conversion processing unit 135 can convert the output level with respect to the input level, for example, as shown in FIG.
- the output level of a predetermined gradation is assigned to the pixels within the main region luminance range according to the input level. Also the main area and the second brightness Pixels in the luminance range between regions are output at the maximum output level assigned to the main region, regardless of the input level.
- the output level of a predetermined gradation is the same as or substantially the same as that of the main area from the maximum value of the output level assigned to the main area according to the input level. Assigned by the number of steps. For pixels with higher luminance than the second luminance region, the main region and the second region from the maximum value of the output level assigned to the luminance range of the second luminance region to the maximum output level. The output level corresponding to the input level is assigned so that the number of steps is smaller than the number of steps assigned to the luminance area.
- the output level conversion processing unit 135 can convert the output level with respect to the input level, for example, as shown in FIG. 7B. That is, in a pixel whose luminance input level is lower than the luminance range of the main area, corresponding to the input level, from 0 (i.e., black) to an output level of a predetermined number of steps smaller than the number of steps assigned to the main area. Is assigned. Then, an output level of a predetermined gradation is assigned to the pixels within the main region luminance range according to the input level. In addition, for the pixels in the luminance range between the main area and the second luminance area, the output level corresponding to the input level is assigned with a smaller number of steps than the number of steps assigned to the main area.
- the maximum power of the output level assigned to the luminance range between the main region and the second luminance region is also applied to the pixels in the second luminance region.
- the number of steps is assigned with the same or approximately the same number as the main area. For pixels with higher brightness than the second brightness area, the steps assigned to the main area from the maximum output level assigned to the brightness range of the second brightness area to the maximum output level.
- the output level corresponding to the input level is assigned so that the number of steps is less than the number.
- the output level conversion processing unit 135 can convert the output level with respect to the input level, for example, as shown in FIG. 7C. That is, in a pixel whose luminance input level is lower than the luminance range of the main area, corresponding to the input level, from 0 (i.e., black) to an output level of a predetermined number of steps smaller than the number of steps assigned to the main area. Is assigned. Then, an output level of a predetermined gradation is assigned to the pixels within the main region luminance range and other than the section ⁇ according to the input level.
- the interval ⁇ The pixel has a predetermined number of steps that is less than the number of steps assigned to the part other than the section a in the main area and is larger than the maximum output level assigned to the part other than the section a in the main area. Output levels are assigned.
- the input level in the luminance range between the main area and the second luminance area, for pixels that do not fall within the interval a or the interval / 3, the input level has a smaller number of steps than the number of steps assigned to the interval a.
- the power that is assigned to the output level according to the value, or the maximum value of the output level that is assigned to the section OL regardless of the input level is assigned.
- the pixels in section / 3 have a predetermined number greater than the maximum value of the output level in the luminance range lower than section ⁇ , with the number of steps smaller than the number of steps assigned to the part other than section a in the main area.
- the output level of the gradation is assigned.
- the maximum output power of the gradation assigned to the section j8 also has an output level of a predetermined gradation according to the input level. It is assigned with the same or approximately the same number of steps as the luminance range other than section OC.
- the output level corresponding to the input level is assigned so that the number of steps is less than the number of steps assigned to this part.
- the predetermined luminance range centered on the upper limit value of the luminance of the main area is also represented by section a
- the predetermined luminance range centered on the lower limit value of the luminance of the second luminance area is represented by section ⁇ .
- a predetermined section on the upper limit side within the main area luminance range is defined as section (X)
- a predetermined section on the lower limit side within the luminance range of the second luminance area is defined as section ⁇
- the upper limit of the main area luminance range is set.
- the predetermined section with higher brightness than the lower limit of the brightness range of the second brightness area may be set as section ⁇ , and the predetermined section with lower brightness than the lower limit of the brightness range of the second brightness area may be set as section ⁇ . It is also possible to set an area to which the same number of output level steps as those in the section a and the section ⁇ are assigned to the upper side of the brightness and the second brightness area.
- the output level conversion processing unit 13 for example, in the luminance range other than the luminance region than the ratio of the output level to the input level in the main region and the second luminance region (straight line). Reduce the ratio of output level to input level
- the number of gradation steps assigned to each luminance region may be determined.
- the main area and the second luminance area set in the image generation unit 93-1 are set based on a captured image that is not within a predetermined luminance range.
- the image generated by the image generation unit 93-1 includes a luminance range that is most important for the user to recognize the image, such as a subject that occupies most of the screen, among the captured images, and the like. In the luminance range higher than the luminance range, it occupies most of the luminance range, so that many of the limited number of gradations are assigned to the luminance range.
- FIG. 8A shows the input signal when the gradation conversion described with reference to FIG. 6A is performed, the signal after the gradation conversion by the output level conversion processing unit 135, and the display 82.
- FIG. 6 is a diagram showing the luminance level of a signal output to the image utilization device 83 for processing such as printing, image recognition, recording, or image communication.
- the signals of the main luminance region and the second luminance region which are discrete luminance ranges, are applied to each luminance gradation width (the number of steps between the maximum value and the minimum value of the gradation in the region). ) Is sufficiently given and converted into continuous luminance gradations.
- gradation conversion (the number of gradations is compressed) is performed in accordance with the condition of signal output after conversion or output to the image utilization device 83. Specifically, for example, when the display 82 can display an image with 256 gradations, the display control unit 94 outputs a signal supplied from the image generation unit 93-1, that is, an output. The signal after gradation conversion by the level conversion processing unit 135 is converted into a signal of 256 gradations.
- the image utilization device 83 is an image recognition process, and the number of gradations used for the process in the image process executed to detect a specific object (for example, a person or a car) from the supplied image is determined.
- the output control unit 95 converts the signal supplied from the image generation unit 93-1, that is, the signal after gradation conversion by the output level conversion processing unit 135, into a signal of 256 gradations.
- FIG. 8B shows the input signal when the gradation conversion described with reference to FIG. 6B is performed, the signal after the gradation conversion by the output level conversion processing unit 135, and the display 82.
- FIG. 6 is a diagram showing a luminance level of a signal output to the image utilization device 83.
- the gradation between the main area and the second luminance area is provided with a smaller number of steps than the main area and the second luminance area.
- the converted signal has a smaller number of steps in the area between the main area and the second luminance area, so that the number of gradations of luminance is significantly compressed than the input signal, but the main area and the second luminance area are reduced.
- the signal in the second luminance region has a sufficient luminance gradation width. Then, the converted signal is subjected to gradation conversion (the number of gradations is compressed) according to the conditions of display or output to the image utilization device 83.
- FIG. 8C shows the input signal when the gradation conversion described with reference to FIG. 6C is performed, the signal after the gradation conversion by the output level conversion processing unit 135, and the display 82.
- FIG. 6 is a diagram showing a luminance level of a signal output to the image utilization device 83.
- the number of luminance gradation steps assigned in the main region, the second luminance region, and the region between them is changed gradually according to the above-mentioned interval (X and interval ⁇ ).
- X and interval ⁇ the above-mentioned interval
- the number of luminance gradations is greatly compressed compared to the input signal, and the converted signal is displayed or displayed on the image utilization device 83.
- gradation conversion (the number of gradations is compressed) is performed.
- step S1 the imaging unit 91 captures an image of a subject based on the user's operation input supplied from the operation input unit 92, logarithmically converted, and AZD converted captured image signal. — Supply to 1.
- the image generator 93-1 acquires the captured image signal.
- step S2 the average luminance calculation unit 131 of the image generation unit 93-1 calculates the average luminance of the entire captured image, and supplies the calculation result to the main region luminance range setting unit 132.
- step S3 the main area luminance range setting unit 132 sets the luminance range of the main area based on the average luminance of the entire image supplied from the average luminance calculation unit 131, and sets the set luminance range of the main area. Are supplied to the output level conversion processing unit 135 and the second luminance area luminance average value calculating unit 133.
- the main area luminance range setting unit 132 may use, for example, a predetermined luminance range centered on the average luminance of the image signal as the luminance range of the main area, and the pixel that is the average luminance of the image signal. As a center, a pixel having a predetermined number of bits may be selected in order from a pixel having a luminance value close to the luminance, and the luminance range of the main region may be selected.
- step S4 the second luminance area luminance average value calculation unit 133 uses the luminance range of the main area set by the main area luminance range setting unit 132 in the image signal supplied from the imaging unit 91. The average brightness of the bright area is obtained, and the result is supplied to the second brightness area brightness range setting unit 134.
- step S5 the second luminance area luminance range setting unit 134 is based on the average luminance of the area brighter than the luminance range of the main area supplied from the second luminance area luminance average value calculating unit 133. Then, the brightness range of the second brightness area is set, and the set brightness range of the second brightness area is supplied to the output level conversion processing unit 135.
- the second luminance area luminance range setting unit 134 uses a predetermined luminance range centered on the average luminance of pixels in a higher luminance range than the luminance range of the main area as the luminance range of the second luminance area.
- a pixel having a predetermined number of bits may be selected in order from a pixel having a luminance value close to this luminance around a pixel having an average luminance of pixels in a luminance range higher than the luminance range of the main region.
- the luminance range of the second luminance region may be used.
- step S 6 the output level conversion processing unit 135 is set by the luminance range of the main area set by the main area luminance range setting unit 1 32 and the second luminance area luminance average value calculation unit 133. Based on the luminance range of the second luminance region, for example, the conversion characteristics between the input level and the output level as described with reference to FIG. 6 or FIG. 7 are determined. [0128] In step S7, the output level conversion processing unit 135 converts the gradation of the captured image supplied from the imaging unit 91 based on the conversion characteristics determined in step S6, and supplies it to the output control unit 95. To do.
- step S8 the output control unit 95 converts the supplied image signal into a gradation suitable for processing that can be executed by the image utilization device 83, as described with reference to FIG.
- step S9 the output control unit 95 controls the output of the converted image data converted to the gradation suitable for the processing executable by the image utilization device 83 to the image utilization device 83. The process is terminated.
- step S1 to step S7 basically the same processing is executed, the image signal is supplied to the display control unit 94, converted to a gradation that can be processed in the display 82, and output, and the display is controlled. Is done.
- a wide dynamic range image picked up by the image pickup unit 91 using the logarithmic conversion type image pickup device 102 is processed using the image processing device 81 having the image generation unit 93-1, and then displayed and output. Under the above conditions, the difference in the display image will be described with reference to FIGS.
- FIG. 10 shows an example of the displayed image.
- a wide dynamics range image captured with the imaging unit 91 using the logarithmic conversion type image sensor 102 does not cause overexposure or underexposure, but the display device handles a very wide dynamics range. Because it converts to a possible gradation, an image with no difference in density (not sharp) is displayed.
- the number of gradations of an image that can be displayed by a general display device can be displayed faithfully. It's not so wide. Specifically, for example, in the case where the pixels displayed as different gradations in the image captured by the CCD exposed on the basis of the brightness of the road surface in FIG. 10, the same gradation or CCD It is displayed with a smaller number of tones than when using.
- FIG. 11 shows an example of a display image when an image captured by normal exposure control is displayed by a conventional CCD camera.
- the exposure of a CCD camera is set based on the brightness of the road surface in the tunnel, which occupies a large part of the image to be captured, as shown in FIG. Images outside the tunnel that are extremely brighter than the inside road will cause over-exposure. Note that when the exposure time is set very short, the part outside the tunnel does not cause overexposure as shown in Fig. 11, but the brightness is significantly lower than the part outside the tunnel. Blackening occurs in the part of the tunnel, making it difficult to identify the corresponding part of the display image.
- FIG. 12 shows an example of a display image when the mixed range image is processed using the image processing device 81 having the image generation unit 93-1.
- an image of the front side of a car traveling near the tunnel exit in the tunnel is captured, it is set as the luminance range of the luminance peripheral force main area of the road portion in front of the car.
- a portion outside the tunnel is set as the second luminance region as the main portion of the pixels having higher luminance than the main region.
- the display image when processed using the image processing device 81 having the image generation unit 93-1 is the main image of the captured image, compared to the case described with reference to FIG. 10 or FIG.
- the main part (here, the road part of the car) and the main part of the discrete brightness range (here, the part outside the tunnel) are displayed so that the user can easily recognize them.
- the image force obtained under the three conditions described above will be considered when supplied to the image utilization device 83 for processing.
- a 14-bit AZD-converted wide dynamics range image contains a very wide luminance range, so the processing performed by the conventional image utilization device 83 cannot sufficiently achieve the purpose.
- the processing performed by the conventional image utilization device 83 cannot sufficiently achieve the purpose.
- the Laplacian conversion process is performed on the wide dynamics range image data that has been AZD-converted at 14 bits, the differential process is performed in the same way as normal image data. It cannot be obtained.
- binary binary processing is performed on 14-bit AZD-converted wide dynamic range image data, the number of brightness candidates that can be used as thresholds increases so that the amount of processing increases explosively. Resulting in.
- an image of a wide dynamics range imaged using the imaging unit 91 using the logarithmic conversion type imaging device 102 was processed using the image processing device 81 having the image generation unit 93-1. Thereafter, by supplying the image to the image utilization apparatus 83, the image utilization apparatus 83 may be able to execute the conventional image processing process as it is.
- the data of the luminance area necessary for detecting the car existing outside the tunnel is supplied with a sufficient number of luminance gradation steps. Therefore, the object can be recognized with high accuracy.
- the image utilization device 82 When the image utilization device 82 is configured to execute processing such as transmission and recording of generated image data, the image utilization device 82 is imaged using the imaging unit 91 using the logarithmic conversion type imaging device 102. Although it is possible to transmit or record an image of a wide dynamics range as it is, an image of a wide dynamics range captured using the imaging unit 91 using the logarithmic conversion type image sensor 102 is very Due to the large amount of data, depending on the purpose of the recorded image data or transmitted image data, the limited recording capacity or traffic on the transmission / reception path may be wasted. Therefore, depending on the purpose of use of recorded image data or transmitted image data, it is possible to save recording capacity or traffic on the transmission / reception path by allocating the number of gradation steps mainly to the luminance area to be used. It is preferable to make it possible.
- the image generation unit 93-1 described above is performed on the luminance range of the main area and the luminance range of the second luminance area, which is the main part of the luminance range higher than the luminance range of the main area. In the above description, a large number of gradation steps are assigned. On the other hand, a case will be described where the set luminance range is not two of the luminance range of the main region and the luminance range of the second luminance region.
- FIG. 14 is a block diagram showing a configuration of an image generation unit 93-2 which is a second example of the configuration of the image generation unit 93 of FIG.
- the image generation unit 93-2 includes the luminance range of the second luminance area, which is the main part of the luminance range higher than the luminance range of the main area, and the luminance range of the main area.
- the image generation unit 93-2 in FIG. 14 has an average luminance calculation unit 131 and a main region luminance range setting unit 132 that are basically the same as the image generation unit 93-1 described with reference to FIG.
- the second luminance area luminance average value calculation unit 133 and the second luminance area luminance range setting unit 134 of the image generation unit 93-1 are omitted, and the high luminance area luminance average value calculation unit 161, the high luminance A region luminance range setting unit 162, a low luminance region luminance average value calculating unit 163, and a low luminance region luminance range setting unit 164 are newly provided.
- an output level conversion processing unit 165 is provided. Is provided.
- the high-luminance area luminance average value calculation unit 161 acquires the image signal supplied from the imaging unit 91, and among the acquired image signals, the main area luminance range setting unit 132 sets the main area. The average luminance of the pixels in the higher luminance range than the luminance range is calculated, and the calculation result is supplied to the high luminance region luminance range setting unit 162.
- the high brightness area brightness range setting unit 162 has higher brightness than the main area based on the average brightness of pixels in the brightness range higher than the brightness range of the main area supplied from the high brightness area brightness average value calculation unit 161.
- the luminance range of the second luminance area is set, and the set luminance range of the second luminance area is supplied to the output level conversion processing unit 165.
- the high luminance area luminance range setting unit 162 sets a predetermined luminance range centered on the average luminance of pixels in a luminance range higher than the luminance range of the main area as the luminance range of the second luminance area. It is also possible to select a pixel having a predetermined number of bits in order from the pixel having a luminance value close to this luminance around the pixel that is the average luminance of the pixels in the luminance range higher than the luminance range of the main area.
- the luminance range of the luminance region may be the same.
- the low-luminance area luminance average value calculation unit 163 acquires the image signal supplied from the imaging unit 91, and among the acquired image signals, the main area luminance range setting unit 132 sets the main area. The average luminance of the pixels in the luminance range lower than the luminance range is calculated, and the calculation result is supplied to the low luminance region luminance range setting unit 164.
- the low luminance area luminance range setting unit 164 has a luminance lower than that of the main area based on the average luminance of pixels in a lower luminance range than the luminance range of the main area supplied from the low luminance area luminance average value calculation unit 163.
- the luminance range of the third luminance area is set, and the set luminance range of the third luminance area is supplied to the output level conversion processing unit 165.
- the low luminance area luminance range setting unit 164 sets, for example, a predetermined luminance range centered on the average luminance of pixels in a luminance range lower than the luminance range of the main area as the luminance range of the third luminance area.
- the output level conversion processing unit 165 acquires the image signal supplied from the imaging unit 91, and obtains the main region luminance range setting unit 132, the high luminance region luminance range setting unit 162, and the low luminance region luminance. Based on the information on the luminance range of the main area, the luminance range of the second luminance area, and the luminance range of the third luminance area supplied from the range setting unit 164, the output level of the acquired image signal is converted. .
- the output level conversion processing unit 165 can convert the output level with respect to the input level, for example, as shown in FIG. 15A.
- the pixel whose luminance input level is lower than that in the third luminance region is 0 (that is, completely black) regardless of the input level.
- an output level of a predetermined gradation is assigned to pixels in the third luminance area, the main area, and the second luminance area according to the input level.
- the pixels in the luminance range between the third luminance region and the main luminance region and the luminance range between the main luminance region and the second luminance region are assigned immediately before that range regardless of the input level.
- Output at the specified output level is assigned to a pixel having a higher luminance than the second luminance area.
- the output level conversion processing unit 165 can convert the output level with respect to the input level, for example, as shown in FIG. 15B. That is, the output level of a pixel whose luminance input level is lower than that of the third luminance region is 0 (that is, black) regardless of the input level. Then, an output level of a predetermined gradation is assigned to the pixels in the third luminance region, the main region, and the second luminance region according to the input level. Also, the number of pixels in the luminance range between the third luminance area and the main area, and the luminance range between the main area and the second luminance area are less than the number of steps assigned to the main area! / ⁇ The output level corresponding to the input level is assigned by the number of steps. And the second brightness An output level that is the maximum value of the output level assigned to the second luminance area is assigned to a pixel having a higher luminance than the area.
- the output level conversion processing unit 165 can convert the output level with respect to the input level, for example, as shown in FIG. 15C. That is, the output level conversion processing unit 165, for example, has a predetermined luminance range centered on the upper limit value of the luminance of the third luminance region in the interval O and a predetermined luminance value centered on the lower limit value of the luminance of the main region. Assume that the range is section ⁇ , the predetermined brightness range centered on the upper limit value of luminance in the main area is section ⁇ , and the predetermined brightness range centered on the lower limit value of brightness in the second brightness area is section ⁇ .
- a pixel whose luminance input level is lower than that in the third luminance region is 0 (that is, black) regardless of the input level.
- pixels other than the section oc, within the main area brightness range, pixels other than the section ⁇ or section ⁇ , and within the second brightness area range, other than the section ⁇ An output level of a predetermined gradation is assigned to the pixel according to the input level.
- the pixels in the interval ⁇ , the interval / 3, the interval ⁇ , and the interval ⁇ are the main region luminance range, etc., and are smaller than the number of steps assigned to portions other than the intervals O to ⁇ .
- An output level having a predetermined number of steps is assigned. Then, an output level that is the maximum value of the output level assigned to the second luminance area is assigned to a pixel having a higher luminance than the second luminance area.
- a predetermined luminance range centered on the upper limit value of the luminance of the third luminance area is defined as section O
- a predetermined luminance range centered on the lower limit value of the luminance of the main area is defined as section.
- ⁇ a predetermined luminance range centered on the upper limit value of the luminance of the main area is defined as interval ⁇
- a predetermined luminance range centered on the lower limit value of the luminance of the second luminance area is defined as interval ⁇ .
- the predetermined brightness range on the upper limit side in the brightness area of the main area is section OC
- the predetermined brightness range on the lower limit side in the main area is section ⁇
- the predetermined brightness range on the upper limit side in the main area is section ⁇
- the second The predetermined luminance range on the lower limit side in the luminance area is defined as the interval ⁇
- the predetermined luminance range higher than the upper limit of the third luminance region is the interval OC
- the predetermined luminance lower than the lower limit of the main region The range is section ⁇
- the predetermined brightness range is higher than the upper limit of the main area, section ⁇
- a predetermined luminance range with lower luminance may be set as the section ⁇ .
- the interval value is set on the lower limit side of the luminance of the third luminance area and the upper limit side of the luminance of the second luminance area.
- an area to which the number of steps having the same output level as that in the section ⁇ is assigned may be set.
- the number of output level steps (the number of gradation steps) in the range of lower luminance than the third luminance region and higher luminance than the second luminance region.
- the output level conversion processing unit 165 has lower luminance than the third luminance region and higher luminance than the second luminance region, for example, as described with reference to FIG. Even in the range, you may assign fewer steps than the main area, the second luminance area, and the third luminance area.
- the output level conversion processing unit 165 determines the ratio of the output level to the input level (straight line) in the main area, the second luminance area, and the third luminance area.
- the number of gradation steps allocated to each luminance region may be determined so that the ratio of the output level to the input level in the luminance range other than the above is reduced.
- FIG. 16A shows the input signal when the gradation conversion described with reference to FIG. 15A is performed, the signal after the gradation conversion by the output level conversion processing unit 165, and the display 82, or FIG. 10 is a diagram illustrating the luminance level of a signal output to the image utilization device 83 for processing such as printing, image recognition, recording, or image communication.
- continuous luminance gradations are provided with sufficient luminance gradation widths of signals in the main luminance region, the second luminance region, and the third luminance region, which are discrete luminance ranges.
- FIG. 16B shows the input signal when the gradation conversion described with reference to FIG. 15B is performed, the signal after the gradation conversion by the output level conversion processing unit 165, and the display 82.
- FIG. 6 is a diagram showing a luminance level of a signal output to the image utilization device 83.
- the region between the third luminance region and the main region and between the main region and the second luminance region includes the main region, the second luminance region, and the third luminance. Only a gradation with a smaller number of steps than the area is given.
- the converted signal has a smaller number of gradations of luminance than the input signal because the number of steps in the region between the third luminance region and the main region and between the main region and the second luminance region is smaller.
- the luminance gradation widths of the signals in the main region, the second luminance region, and the third luminance region are sufficiently given.
- the converted signal is subjected to gradation conversion (the number of gradations is compressed) according to the conditions of display or output to the image utilization device 83.
- FIG. 16C shows the input signal when the gradation conversion described with reference to FIG. 15C is performed, the signal after the gradation conversion by the output level conversion processing unit 165, and the display 82.
- FIG. 16C is a diagram showing a luminance level of a signal output to the image utilization device 83.
- FIG. 16C the number of gradation steps assigned to luminance in the main area, the second luminance area, the third luminance area, and the area between them is represented by the above-described intervals (X to ⁇
- the interval ⁇ to interval ⁇ has a smaller number of steps than the portions other than the intervals oc to ⁇ of the main region, the second luminance region, and the third luminance region.
- the main area, the second luminance area, and the third luminance area are given sufficient intensity gradation width in the converted signal.
- the number of gradations of luminance is greatly compressed compared to the input signal, and the converted signal is gradation converted (the number of gradations is compressed) according to the conditions of display or output to the image utilization device 83. It is made to be!
- step S31 the imaging unit 91 captures an image of the subject based on the user's operation input supplied from the operation input unit 92, and the captured image signal that has been logarithmically converted and AZD converted is converted into an image generation unit 93. — Supply to 2.
- the image generation unit 93-2 acquires the captured image signal.
- step S32 the average luminance calculation unit 131 of the image generation unit 93-2 captures the image. The average luminance of the entire image is obtained, and the calculation result is supplied to the main region luminance range setting unit 132.
- step S33 the main region luminance range setting unit 132 sets the luminance range of the main region based on the average luminance of the entire image supplied from the average luminance calculation unit 131, and sets the luminance of the main region thus set.
- the range is supplied to the output level conversion processing unit 165, the high luminance area luminance average value calculation unit 161, and the low luminance region luminance average value calculation unit 163.
- the main area luminance range setting unit 132 may use, for example, a predetermined luminance range centered on the average luminance of the image signal as the luminance range of the main area, and the pixel that is the average luminance of the image signal. As a center, a pixel having a predetermined number of bits may be selected in order from a pixel having a luminance value close to the luminance, and the luminance range of the main region may be selected.
- step S34 the high luminance area luminance average value calculation unit 161 is an area brighter than the luminance range of the main area set by the main area luminance range setting unit 132 in the image signal supplied from the imaging unit 91. The average luminance is obtained, and the result is supplied to the high luminance region luminance range setting unit 162.
- step S35 the high brightness area brightness range setting unit 162 sets the second brightness based on the average brightness of the area brighter than the brightness range of the main area supplied from the high brightness area brightness average value calculation unit 161.
- the luminance range of the area is set, and the set luminance range of the second luminance area is supplied to the output level conversion processing unit 165.
- the high luminance area luminance range setting unit 162 uses, for example, a predetermined luminance range centered on the average luminance of pixels in a luminance range higher than the luminance range of the main area as the luminance range of the second luminance area. It is also possible to select a pixel having a predetermined number of bits in order from the pixel having a luminance value close to this luminance around the pixel that is the average luminance of the pixels in the luminance range higher than the luminance range of the main area.
- the luminance range of the luminance region may be the same.
- step S36 the low luminance area luminance average value calculation unit 163 is an area darker than the luminance range of the main area set by the main area luminance range setting unit 132 in the image signal supplied from the imaging unit 91. Is obtained, and the result is supplied to the low luminance area luminance range setting unit 164.
- step S37 the low luminance area luminance range setting unit 164 performs the low luminance area luminance average. Based on the average brightness of the darker area than the brightness range of the main area supplied from the value calculation unit 163, the brightness range of the 3rd brightness area is set, and the brightness range of the set 3rd brightness area is set as the output level. The data is supplied to the conversion processing unit 165.
- the low luminance area luminance range setting unit 164 sets, for example, a predetermined luminance range centered on the average luminance of pixels in a luminance range lower than the luminance range of the main area as the luminance range of the third luminance area. It is also possible to select pixels having a predetermined number of bits in order from the pixel having the luminance value close to this luminance around the pixel that is the average luminance of the pixels in the luminance range lower than the luminance range of the main region. It is good also as a local luminance range.
- step S38 the output level conversion processing unit 165 sets the luminance range of the main region set by the main region luminance range setting unit 132 and the second luminance region set by the high luminance region luminance range setting unit 162. Based on the luminance range and the luminance range of the third luminance region set by the low luminance region luminance range setting unit 164, for example, the conversion characteristics of the input level and the output level as described with reference to FIG. 15 are obtained. decide.
- step S39 the output level conversion processing unit 165 changes the gradation of the captured image supplied from the imaging unit 91 based on the conversion characteristics determined in step S39. Then, it is supplied to the output control unit 95.
- step S40 the output control unit 95 converts the supplied image signal into a gradation suitable for processing that can be executed by the image using apparatus 83, as described with reference to FIG.
- step S41 the output control unit 95 controls the output of the converted image data converted to the gradation suitable for the processing executable by the image utilization device 83 to the image utilization device 83. The process is terminated.
- step S31 to step S39 basically the same processing is executed, the image signal is supplied to the display control unit 94, converted into a gradation that can be processed in the display 82, and output, and the display is controlled.
- the high luminance with a wide luminance range, in particular, the main luminance range force is also discrete. Even if the image information required by the user exists in both the low-brightness area and the low-brightness area, such an image is displayed with a gradation corresponding to the display 82, or Generates image data that can be displayed or printed out in a state that can be identified by the user, or that can be easily processed by the image utilization device 83, when output in a gradation that corresponds to the processing executed by the 83. can do.
- the image displayed by the image processing device 81 including the image generation unit 92-2 in FIG. 14 is, for example, as shown in FIG.
- the angle of view includes a sky part that is much brighter than the brightness of the road surface, and a black that is much lower than the brightness of the road surface. Even if it is included, a large number of gradation steps are assigned near the brightness corresponding to each of the road surface, the sky, and a person wearing a black suit. Therefore, by using the image processing device 81 including the image generation unit 93-2 in FIG. 14, in the image display process, an image that is difficult to recognize due to white sky is displayed or captured. Regardless, black, a person wearing a suit cannot be identified by the user, and the image may be displayed in a black suit, or a person wearing a black suit cannot be recognized (or extracted) in the image recognition process. Can be prevented.
- the image displayed by the image processing device 81 including the image generation unit 92-2 of Fig. 14 is specifically, for example, a captured image as shown in Fig. 19.
- Most of the area is the dark road surface in the tunnel, and the angle of view includes the white wall inside the tunnel, which is slightly brighter than the brightness of the road surface, and the outside of the tunnel, which is very bright.
- the angle of view includes a black car in the tunnel, which is much lower than the brightness of the road surface in the tunnel, the road surface in the tunnel, the tunnel walls and the outside of the tunnel, and Thus, a large number of gradation steps are assigned near the brightness corresponding to each of the black cars in the tunnel!
- the image processing device 81 including the image generation unit 93-2 in FIG. 14 for example, in the image display processing, an image that is difficult to recognize due to whiteout portions outside the tunnel is displayed or captured. Despite the fact that the black inside the tunnel and the car cannot be identified by the user, it is prevented from being blacked out and displayed in the image recognition process, such as cars and guardrails outside the tunnel. , Alternatively, it is possible to prevent a black wagon in the tunnel from being extracted as a recognition target.
- the image generation unit 93 three or more areas are set so that the number of gradation steps assigned to the set area and the non-set area can be made different. May be.
- FIG. 20 is a block diagram showing a configuration of an image generation unit 93-3 that is a third example of the configuration of the image generation unit 93 of FIG.
- the image generation unit 93-3 sets a plurality of luminance areas and their luminance ranges, and more than the luminance ranges that are not set for the plurality of set luminance ranges. The number of gradation steps can be assigned.
- the image generation unit 93-3 in FIG. 20 has an average luminance calculation unit 131 and a main region luminance range setting unit 132 that are basically the same as the image generation unit 93-1 described with reference to FIG.
- the second luminance area luminance average value calculation unit 133 and the second luminance area luminance range setting unit 134 of the image generation unit 93-1 are omitted, and the second luminance area luminance average value calculation unit 181, Second luminance region luminance range setting unit 182, third luminance region luminance average value calculating unit 183, third luminance region luminance range setting unit 184, fourth luminance region luminance average value calculating unit 185, and fourth
- the brightness area brightness range setting unit 186 is newly provided, and an output level conversion processing unit 187 is provided instead of the output level conversion processing unit 135.
- the second luminance area luminance average value calculation unit 181 has a predetermined value other than the luminance range of the main area set by the main area luminance range setting unit 132 out of the image signal supplied from the imaging unit 91. Calculate the average luminance of the pixels that have the luminance included in the range (for example, the luminance range higher than the luminance range of the main area is further divided into two, and the highest luminance part, etc.) This is supplied to the area luminance range setting unit 182.
- the second luminance area luminance range setting unit 182 is based on the average luminance of pixels having luminance included in a predetermined range other than the supplied main area. Set the brightness range of the second brightness area, and set the brightness range of the set second brightness area. The range is supplied to the output level conversion processing unit 187.
- the second luminance area luminance range setting unit 182 may set a predetermined luminance range centered on the average luminance of pixels having a predetermined range of luminance as the luminance range of the second luminance area, for example.
- a pixel having a predetermined number of bits is selected in order from a pixel having a luminance value close to this luminance around a pixel having an average luminance of pixels having a luminance in a predetermined range as a luminance range of the second luminance area. Also good.
- the third luminance area luminance average value calculation unit 183 has a predetermined value other than the luminance range of the main area set by the main area luminance range setting unit 132 out of the image signal supplied from the imaging unit 91. Calculate the average luminance of the pixels that have the luminance included in the range (for example, the range of higher luminance than the luminance range of the main region is further divided into two, and the lower luminance portion of the two divided regions, etc.) , And supplied to the third luminance area luminance range setting unit 184.
- the third luminance area luminance range setting unit 184 is based on the average luminance of pixels having luminance included in a predetermined range other than the supplied main area.
- the brightness range of the third brightness area is set, and the set brightness range of the third brightness area is supplied to the output level conversion processing unit 187.
- the third luminance area luminance range setting unit 184 may use a predetermined luminance range centered on the average luminance of pixels having a predetermined range of luminance as the luminance range of the third luminance area.
- a pixel with a predetermined number of bits is selected in order from the pixel having the luminance value close to this luminance around the pixel having the average luminance of the pixels having the luminance in the predetermined range as the third luminance range. good.
- the fourth luminance area luminance average value calculation unit 185 has a predetermined value other than the luminance range of the main area set by the main area luminance range setting unit 132 out of the image signal supplied from the imaging unit 91. Calculate the average luminance of the pixels that have the luminance included in the range (for example, the luminance range lower than the luminance range of the main area is further divided into two, and the low luminance portion of the two divided areas, etc.). , And supplied to the fourth luminance area luminance range setting unit 186.
- the fourth luminance area luminance range setting unit 186 is based on the average luminance of pixels having luminance included in a predetermined range other than the supplied main area. Set the brightness range of the fourth brightness area, and set the brightness range of the set fourth brightness area. The range is supplied to the output level conversion processing unit 187.
- the fourth luminance area luminance range setting unit 186 may set a predetermined luminance range centered on the average luminance of pixels having a predetermined luminance range as the luminance range of the fourth luminance area.
- a pixel having a predetermined number of bits is selected in order from a pixel having a luminance value close to this luminance around a pixel that is an average luminance of pixels having a luminance in a predetermined range. good.
- the output level conversion processing unit 187 acquires the image signal supplied from the imaging unit 91, and obtains the main region luminance range setting unit 132, the second luminance region luminance range setting unit 182, and the third luminance region.
- Luminance range setting section 184 and fourth luminance area luminance range setting section 186, the main area luminance range, second luminance area luminance range, third luminance area luminance range and The output level of the acquired image signal is converted based on the information on the luminance range of 4 luminance areas.
- the output level conversion processing unit 187 outputs the output level signal of a predetermined number of steps to the level of the input luminance signal (each luminance when the luminance is divided by the predetermined number of gradation steps). Tone level signal) is assigned and the assigned output level signal is output.
- the output level conversion processing unit 187 assigns the luminance gradation level of the output level signal as the main area and other areas, for example, as described with reference to FIG. 6, FIG. 7, or FIG.
- the brightness range is different from the other range. That is, the output level conversion processing unit 187 assigns a larger number of output level steps to the luminance range set as the main region and other regions, and the number of gradations of pixels in the corresponding luminance range is set. The conversion process is performed so that the user can more easily recognize the corresponding luminance range of the displayed or printed image.
- FIG. 20 shows a second luminance area luminance average value calculation unit 181 and a second luminance area luminance range setting unit for setting the second to fourth luminance areas in addition to the main area. 182, a third luminance area luminance average value calculating unit 183, a third luminance area luminance range setting unit 184, a fourth luminance area luminance average value calculating unit 185, and a fourth luminance area luminance range setting unit 186.
- the image generation unit 93-3 can set more luminance regions.
- another luminance area luminance average value calculation unit and luminance region luminance range setting unit may be provided.
- the process executed by the image generation unit 93-3 in Fig. 20 is basically the same as the process of the image display process 2 described with reference to Fig. 17, and the number of set areas is increased. The explanation is omitted because it corresponds to the case.
- the image generation unit 93-1 to image generation unit 93-3 described above set the main region based on the average luminance value of the entire captured image.
- the main area may be set based on the average value of the luminance of the pixels included in a predetermined area in the captured image.
- FIG. 21 is a block diagram showing a configuration of an image generation unit 93-4, which is a fourth example of the configuration of the image generation unit 93 in FIG.
- the image generation unit 93-4 cuts out pixels included in a predetermined area from the captured image, and sets the main area based on the average luminance value of the cut out area. Speak.
- the image generation unit 93-4 in FIG. 21 is basically the same except that the main region cutout unit 201 and the main region luminance average value calculation unit 202 are provided in place of the average luminance calculation unit 131. Further, it has the same configuration as that of the image generation unit 93-1 in FIG.
- the main region cutout unit 201 acquires the image signal supplied from the imaging unit 91, cuts out a preset image region from the acquired image signal, and uses the cutout region pixel as a main portion. This is supplied to the area luminance average value calculation unit 202.
- the region of the image cut out by the main region cutout unit 201 seems to be constantly imaged on the road surface.
- the brightness of the road surface that occupies the main part of the displayed image can be made substantially constant by making the segmented area an area where the same image is supposed to be captured constantly. Therefore, it is possible to prevent the brightness of the display image felt by the driver from frequently flickering, or to perform processing without changing the parameters for object extraction.
- the main area luminance average value calculation unit 202 calculates the average luminance of the pixels in the clipped region supplied from the main region cutout unit 201, and sets the calculation result of the average luminance as the main region luminance range setting. Supply to part 132.
- the luminance range of the main region is set based on the average luminance of the clipped region calculated by the main region luminance average value calculation unit 202. Based on the luminance range of the main area, the luminance range of the second luminance area that is higher than the luminance range of the main area is set. Then, the output level conversion processing unit 135 assigns more output level steps to the luminance ranges set as the main area and the second luminance area, and the case described with reference to FIGS. Similarly, the number of gradations of the pixels in the corresponding luminance range increases, and the corresponding luminance range portion of the image displayed or printed out can be more easily recognized by the user or processed by the image utilization device 83. Conversion processing is performed so that image data suitable for the image can be generated
- step S71 the imaging unit 91 captures an image of the subject based on the user's operation input supplied from the operation input unit 92, and the captured image signal obtained by logarithmic conversion and AZD conversion is captured by the image generation unit 93. — Supply to 4. The image generation unit 93-4 acquires the captured image signal.
- step S72 the main region cutout unit 201 of the image generation unit 93-4 is preset in the image signal supplied from the imaging unit 91, for example, as described with reference to FIG.
- the image area to be beaten is cut out, and the pixels in the cut out area are supplied to the main area luminance average value calculation unit 202.
- step S73 the main area luminance average value calculation unit 202 obtains the average luminance of the pixels in the clipped region supplied from the main region cutout unit 201, and the calculation result is sent to the main region luminance range setting unit 132. Supply.
- step S74 the main region luminance range setting unit 132 sets the luminance range of the main region based on the average luminance of the pixels in the clipped region supplied from the main region luminance average value calculation unit 202. Then, the set luminance range of the main area is supplied to the second luminance area luminance average value calculation unit 133 and the output level conversion processing unit 135.
- steps S75 to S80 basically the same processing as in steps S4 to S9 in Fig. 9 is executed.
- the second luminance area luminance average value calculating unit 133 obtains the average luminance of the area brighter than the luminance range of the main area from the image signal supplied from the imaging unit 91, and obtains the second luminance value.
- the brightness area brightness range setting unit 134 sets the brightness range of the second brightness area based on the average brightness of the area brighter than the brightness range of the main area.
- the conversion characteristic of the output level is determined, the gradation of the captured image supplied from the imaging unit 91 is converted based on the conversion characteristic, and is supplied to the output control unit 95.
- the output control unit 95 converts the supplied image signal into a gradation suitable for the processing that can be performed by the image using device 83 as described with reference to FIG. 8, and the image using device of the converted image signal
- the output to 83 is controlled and the process is terminated.
- step S71 to step S78 basically the same processing is executed, the image signal is supplied to the display control unit 94, converted to a gradation that can be processed in the display 82, and output, and the display is controlled.
- a luminance range to which many gradations are assigned is set with reference to a predetermined area, it is possible to prevent the brightness of the display image from flickering frequently, especially when displaying a moving image, It is possible to process without changing the parameters for extraction of each time.
- the image generation unit 93-4 described above cuts out a predetermined area in the captured image, sets the luminance range of the main area based on the pixels of the cut out area, and sets the luminance range of the main area. It has been described that a large number of gradation steps are assigned to the brightness range of the second brightness area, which is the main part of the brightness range that is higher than the brightness range of the main area. On the other hand, the brightness range to be set will not be described in two cases, that is, the brightness range of the main area and the brightness range of the second brightness area.
- FIG. 24 is a block diagram showing a configuration of an image generation unit 93-5 which is a fifth example of the configuration of the image generation unit 93 of FIG.
- the image generator 93-5 cuts out a predetermined area in the captured image, sets the luminance range of the main area based on the pixels of the cut out area, and sets the main area luminance range in addition to the main area luminance range.
- the brightness range of the second brightness area which is the main part of the brightness range higher than the brightness range of the area
- the third brightness which is the main part of the brightness range lower than the brightness range of the main area.
- Three brightness areas are set with the area brightness range, and a large number of gradation steps can be assigned to the set three brightness areas. Note that portions corresponding to those in FIG. 14 or FIG. 21 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.
- the image generation unit 93-5 in FIG. 24 includes the main area cutout unit 201 and the main area luminance average value calculation unit 202 similar to those described with reference to FIG. Similar to the case described above, main area luminance range setting unit 132, high luminance area luminance average value calculating unit 161, high luminance area luminance range setting unit 162, low luminance area luminance average value calculating unit 163, low luminance area A luminance range setting unit 164 and an output level conversion processing unit 165 are provided.
- the image generation unit 93-5 in FIG. 24 cuts out a predetermined region in the captured image and cuts out the same region as the image generation unit 93-4 described with reference to FIG.
- the luminance range of the main region is set in addition to the luminance range of the main region in the same manner as the image generation unit 93-2 described with reference to FIG.
- the three brightness areas are set, and a large number of gradation steps can be assigned to the set three brightness areas!
- steps S101 to S103 basically the same processing as in steps S71 to S73 of FIG. 23 is executed.
- the imaging unit 91 captures an image of a subject based on the user's operation input supplied from the operation input unit 92, and performs logarithmic conversion and A / D conversion of the captured image signal as an image generation unit 935.
- the image generation unit 93-5 acquires the captured image signal.
- the main region cutout unit 201 of the image generation unit 93-5 cuts out and cuts out a preset image region from the image signal supplied from the imaging unit 91, for example, as described with reference to FIG.
- the pixels in the extracted area are supplied to the main area luminance average value calculation unit 202.
- the main area luminance average value calculation unit 202 obtains the average luminance of the pixels in the clipped region supplied from the main region cutout unit 201 and supplies the calculation result to the main region luminance range setting unit 132.
- step S104 the main region luminance range setting unit 132 sets the luminance range of the main region based on the average luminance of the pixels in the clipped region supplied from the main region luminance average value calculation unit 202. Then, the set luminance range of the main region is supplied to the output level conversion processing unit 165, the high luminance region luminance average value calculating unit 161, and the low luminance region luminance average value calculating unit 163.
- steps S105 to S112 basically the same processing as in steps S34 to S41 of Fig. 17 is executed.
- the high luminance area luminance average value calculating unit 161 obtains the average luminance of the area brighter than the luminance range of the main area in the image signal supplied from the imaging unit 91, and the high luminance area luminance range setting unit 162 sets the brightness range of the second brightness area based on the average brightness of the area brighter than the brightness range of the main area, and outputs the set brightness range of the second brightness area to the output level conversion processing unit 165. Supply.
- the low luminance area luminance average value calculation unit 163 obtains the average luminance of the darker area than the luminance range of the main area in the image signal supplied from the imaging unit 91, and sets the low luminance area luminance range.
- the unit 164 sets the luminance range of the third luminance region based on the average luminance of the region darker than the luminance range of the main region, and outputs the set luminance range of the third luminance region to the output level conversion processing unit 165. Supply.
- the output level conversion processing unit 165 sets the brightness range of the set main area, the second Based on the luminance range of the luminance region and the luminance range of the third luminance region, for example, the conversion characteristics between the input level and the output level as described with reference to FIG. 15 are determined and supplied from the imaging unit 91.
- the gradation of the captured image is converted based on the determined conversion characteristic and supplied to the output control unit 95.
- the output control unit 95 converts the supplied image signal into a gradation suitable for the processing that can be executed by the image using device 83 as described with reference to FIG.
- the output of the converted image signal to the image utilization device 83 is controlled, and the processing is terminated.
- step S101 to step S110 basically the same processing is executed, and the image signal is supplied to the display control unit 94, converted into a gradation that can be processed in the display 82, and output. Be controlled.
- the image generation unit 93 may cut out a predetermined area in the captured image and set three or more areas based on the pixels in the cut out area. Then, the image generation unit 93 can change the number of gradation steps assigned to the set area and the non-set area.
- FIG. 26 is a block diagram showing a configuration of an image generation unit 93-6 that is a sixth example of the configuration of the image generation unit 93 of FIG.
- the image generation unit 93-6 is a predetermined area in the captured image. Cut out the area, set the luminance range of the main area based on the pixels of the cut out area, set the corresponding luminance range of multiple luminance areas in addition to the luminance range of the main area, and set the multiple It is possible to assign more gradation steps to the brightness range than outside the set range!
- the image generation unit 93-6 in FIG. 26 is provided with a main region cutout unit 201 and a main region luminance average value calculation unit 202 similar to those described with reference to FIG.
- the main area luminance range setting unit 132, the second luminance area luminance average value calculating unit 181, the second luminance area luminance range setting unit 182, and the third luminance similar to the case described with reference to FIG.
- Area luminance average value calculating section 183, third luminance area luminance range setting section 184, fourth luminance area luminance average value calculating section 185, fourth luminance area luminance range setting section 186, and output level conversion processing section 187 is provided.
- the image generation unit 93-6 in FIG. 26 cuts out a predetermined region in the captured image in the same manner as the image generation unit 93-4 described with reference to FIG.
- the luminance range of the main area is set on the basis of the pixels, and in the same way as the image generation unit 93-3 described with reference to FIG.
- Each brightness range is set, and more gradation steps than the set range can be assigned to the set brightness ranges.
- FIG. 26 shows a second luminance area luminance average value calculating unit 181 and a second luminance area luminance range setting unit for setting the second to fourth luminance areas in addition to the main area. 182, a third luminance area luminance average value calculating unit 183, a third luminance area luminance range setting unit 184, a fourth luminance area luminance average value calculating unit 185, and a fourth luminance area luminance range setting unit 186.
- the image generation unit 93-3 is further provided with another luminance region luminance average value calculation unit and luminance region luminance range setting unit so that a large number of luminance regions can be set. May be.
- the process executed by the image generation unit 93-6 in Fig. 26 is basically the same as the process of the image display process 4 described with reference to Fig. 25, and the number of set areas is increased. Vs. case The description thereof will be omitted.
- the image generation unit 93-1 to the image generation unit 93-6 described above set the main region based on the entire captured image or the average luminance value of a predetermined portion. It was done. On the other hand, a histogram showing the distribution of the luminance values of the pixels included in the captured image is created and analyzed to set a plurality of luminance ranges. Thus, it may be possible to assign a larger number of gradation steps than outside the setting range.
- FIG. 27 is a block diagram showing a configuration of an image generation unit 93-7, which is a seventh example of the configuration of the image generation unit 93 in FIG.
- the image generation unit 93-7 analyzes a histogram of luminance values of each pixel of the captured image, and sets a plurality of luminance ranges based on the analysis result.
- the image generation unit 93-7 includes a histogram analysis unit 251, a threshold comparison processing unit 252, a multi-step luminance range setting unit 253, and an output level conversion processing unit 254.
- the histogram analysis unit 251 acquires the image signal supplied from the imaging unit 91, and generates a histogram indicating the distribution of luminance values of each pixel of the captured image based on the acquired image signal. The analysis result is supplied to the threshold value comparison processing unit 252.
- the threshold comparison processing unit 252 compares the number of pixels corresponding to each luminance value of the input signal with a predetermined threshold value. In other words, the threshold value comparison processing unit 252 extracts a luminance range of a captured image that has a certain number of pixels or more. The threshold value comparison processing unit 252 supplies information indicating the luminance value determined that the number of pixels is equal to or larger than the threshold value as a result of the comparison with the threshold value to the multi-level luminance range setting unit 253.
- the threshold value may be a value that is obtained experimentally and empirically and set in advance, or may be a value that can be appropriately set by a user. If the threshold is set too low, most of the information remains, so the resulting image is, for example, a wide dynamics range imaged using the logarithmic conversion image sensor 102 described with reference to FIG. This is an image with no difference in shading (no sharpness) like a display image when the image processing device 81 is not used to process the image. On the other hand, the threshold is set too high. In such a case, information may be missed, and only a part of the luminance range may be clearly displayed, resulting in an image.
- the multi-level luminance range setting unit 253 has a larger number of gradation steps than that outside the setting range based on the luminance value supplied from the threshold comparison processing unit 252 and determined that the number of pixels is equal to or greater than the threshold.
- a plurality of luminance ranges to be assigned are set, and the set luminance ranges are supplied to the output level conversion processing unit 254.
- the number of luminance ranges set by the multi-step luminance range setting unit 253 is a force determined by the comparison result supplied from the threshold comparison processing unit 252. For example, an upper limit of the number may be determined in advance.
- the output level conversion processing unit 254 acquires the image signal supplied from the imaging unit 91, and based on the information on the set luminance range supplied from the multiple-step luminance range setting unit 253, for example For example, in the same manner as described with reference to FIG. 6, FIG. 7, or FIG. 15, the number of steps of the luminance gradation level of the output level signal assigned to the set luminance region is otherwise set.
- the output level of the acquired image signal is converted so as to be larger than the number of steps of the luminance gradation level of the output level signal assigned to the luminance region.
- the histogram analysis unit 251 analyzes the histogram indicating the luminance value distribution of each pixel of the captured image. Then, the threshold value comparison processing unit 252 compares with the threshold value, and the luminance having the number of pixels equal to or larger than the threshold value is extracted in the same image (within one frame). Then, based on the extracted luminance, a plurality of luminance ranges are set in the multi-level luminance range setting unit 253, and the output level conversion processing unit 254 gives priority to the number of gradation steps in the range. Since the conversion characteristic of the output level with respect to the input level is required to be assigned, the luminance gradation width of each set luminance range is sufficiently given. Then, the converted signal is subjected to gradation conversion (the number of gradations is compressed) and output in the display control section 94 or the output control section 95 according to various processing conditions including display or printing. It is made like that.
- the image display processing 5 executed by the image processing apparatus 81 will be described.
- step S141 the imaging unit 91 operates the user's operation supplied from the operation input unit 92.
- the subject is imaged based on the input, and the captured image signal that has been logarithmically converted and AZD converted is supplied to the image generation unit 93-7.
- the image generation unit 93-7 acquires the captured image signal.
- step S142 the histogram analysis unit 251 of the image generation unit 93-7 generates a histogram indicating the luminance distribution of each pixel of the captured image based on the image signal supplied from the imaging unit 91. Generate and analyze, and supply the analysis result to the threshold comparison processing unit 252.
- the threshold value comparison processing unit 252 compares the number of pixels corresponding to the luminance value of each input signal with a predetermined threshold value based on the analysis result of the histogram supplied from the histogram analysis unit 251. .
- the threshold value comparison processing unit 252 supplies information indicating the luminance value determined that the number of pixels is equal to or greater than the threshold value as a result of the comparison with the threshold value to the multi-step luminance range setting unit 253.
- step S144 the multi-step luminance range setting unit 253 is also supplied with the threshold comparison processing unit 252. Based on the luminance value determined to be equal to or greater than the threshold value, more gradation steps than those outside the setting range are provided. Multiple luminance ranges to which numbers are assigned are set, and the set luminance ranges are supplied to the output level conversion processing unit 254.
- step S145 the output level conversion processing unit 254, based on the information on the set luminance range supplied from the multi-step luminance range setting unit 253, for example, FIG. 6, FIG. 7, or FIG. Basically, the conversion characteristics of the input level and output level are determined in the same way as described using 15.
- step S146 the output level conversion processing unit 254 determines, based on the conversion characteristics between the input level and the output level, the number of steps of the luminance gradation level of the output level signal assigned to the set plurality of luminance ranges. Output level assigned to other luminance ranges The output level of the image signal supplied from the imaging unit 91 is converted so as to be greater than the number of steps of the luminance gradation level of the signal, and supplied to the output control unit 95 To do.
- step S147 the output control unit 95 converts the supplied image signal into a gradation suitable for the processing executed in the image utilization device 83, as described with reference to FIG.
- step S148 the output control unit 95 performs processing that can be executed by the image utilization device 83.
- the output to the image utilization device 83 of the converted image data that has been converted to a gradation that conforms to the above is controlled, and the process ends.
- steps S141 to S146 basically the same processing is executed, and the image signal is supplied to the display control unit 94 and converted into gradations that can be processed by the display 82. Is output and its display is controlled.
- a luminance region to which a large number of gradation steps are assigned is set based on the captured image.
- a luminance region to which a large number of gradation steps are assigned may be determined in advance, or may be set by an operation input with user power.
- the luminance area of the pixel corresponding to the image information required by the user in the captured image for example, when the angle of view of the captured image is fixed or the subject is irradiated with constant illumination.
- a luminance region to which a large number of gradation steps are assigned can be determined in advance. As a result, the processing can be simplified and the cost of the apparatus can be reduced.
- FIG. 30 is a block diagram showing a configuration of an image generation unit 93-8, which is an eighth example of the configuration of the image generation unit 93 in FIG.
- the image generation unit 93-8 is used when a luminance region to which a large number of gradation steps are assigned is determined in advance.
- the first luminance area luminance range setting unit 281 receives the input of the setting value of the luminance range of the first luminance area from the operation input unit 92 or the first luminance area stored in a storage unit (not shown). The set value of the luminance region of 1 is acquired, and the set value of the first luminance range is supplied to the output level conversion processing unit 187.
- the second luminance area luminance range setting unit 282 receives the input of the setting value of the luminance range of the second luminance area from the operation input unit 92 or the second luminance area stored in a storage unit (not shown). The setting value of the second luminance area is acquired, and the setting value of the luminance range of the second luminance area is supplied to the output level conversion processing unit 187.
- the third luminance area luminance range setting unit 283 receives the input of the setting value of the luminance range of the third luminance area from the operation input unit 92, or is stored in a storage unit (not shown). The setting value of the luminance region of 3 is acquired, and the setting value of the luminance range of the third luminance region is supplied to the output level conversion processing unit 187.
- the output level conversion processing unit 187 performs basically the same processing as in the case of the image generation unit 93-3 in Fig. 20, and based on a plurality of set luminance ranges, the imaging unit 91 An image signal supplied with force is acquired and its output level is converted. That is, the output level conversion processing unit 187 is supplied from the first luminance region luminance range setting unit 281, the second luminance region luminance range setting unit 282, and the third luminance region luminance range setting unit 283. The output level of the image signal supplied from the imaging unit 91 is converted based on the information on the luminance range of the first luminance range, the luminance range of the second luminance region, and the luminance range of the third luminance region.
- FIG. 30 shows a first luminance area luminance range setting section 281, a second luminance area luminance range setting section 282, and the like for setting the luminance ranges of the first to third luminance areas.
- the power generation unit 93-8 in which the third luminance region luminance range setting unit 283 is illustrated, can further receive other luminance regions so that the luminance range of many luminance regions can be set.
- a brightness range setting section may be provided.
- the image generation process 93-8 of FIG. 30 is used for the image display process 6 executed by the image processing apparatus 81, V. explain.
- step S171 the operation input unit 92 receives input of set values of a plurality of luminance ranges for user power, and supplies the input set values to the image generation unit 93-8.
- step S172 the imaging unit 91 captures a subject based on the user's operation input supplied from the operation input unit 92, and performs logarithmic conversion and AZD conversion of the captured image signal. This is supplied to the image generator 93-8.
- the image generation unit 93-8 acquires the captured image signal.
- step S173 the first luminance area luminance range setting unit 281, the second luminance area luminance range setting unit 282, and the third luminance area luminance range setting unit 283 of the image generation unit 93-8 are Then, the setting values of the plurality of luminance ranges supplied from the operation input unit 92 are acquired and supplied to the output level conversion processing unit 187.
- the output level conversion processing unit 187 includes the first luminance area luminance range setting unit 281, the second luminance area luminance range setting unit 282, and the third luminance area luminance range setting unit 283.
- step S175 the output level conversion processing unit 187 converts the gradation of the captured image supplied from the imaging unit 91 based on the conversion characteristics determined in step S174, and supplies it to the output control unit 95. .
- step S176 the output control unit 95 converts the supplied image signal into a gradation suitable for the processing executed by the output control unit 95, for example, as described with reference to FIGS. To do.
- step S177 the output control unit 95 controls the output of the converted image data converted to the gradation suitable for the processing executable by the image utilization device 83 to the image utilization device 83, and Processing is terminated.
- steps S 171 to S 175 basically the same processing is executed, and the image signal is supplied to the display control unit 94 and converted into gradations that can be processed in the display 82. Is output and its display is controlled.
- the force described as receiving the setting value of the luminance range of each luminance region from the operation input unit 92 When stored in the storage unit, the stored brightness value is acquired. Needless to say, what you do.
- a histogram indicating the distribution of the luminance values of the pixels included in the captured image is analyzed in a predetermined luminance region, and a certain number of pixels within a predetermined luminance range are determined. It is also possible to extract the luminance that has and to determine the luminance area to which a large number of gradation steps are assigned based on the extraction result.
- FIG. 32 is a block diagram showing a configuration of an image generation unit 93-9 that is a ninth example of the configuration of the image generation unit 93 in FIG.
- the image generation unit 93-9 extracts luminance having a certain number of pixels or more in a predetermined luminance region by histogram analysis, and assigns a large number of gradation steps based on the extraction result. It is made to determine the luminance area to be controlled.
- the image generation unit 93-9 in FIG. 32 includes the first luminance region luminance range setting unit 281 and the second luminance region luminance range setting unit of the image generation unit 93-8 described with reference to FIG. 282, a third luminance area luminance range setting unit 283, and an output level conversion processing unit 187. Further, the image generation unit 93-9 includes the image generation unit described with reference to FIG. 27 in order to analyze the histogram and compare it with a predetermined threshold value based on the set value of the luminance range output from each.
- the histogram analysis units 251-1 to 2 51-3 and the threshold value comparison processing units 252-1 to 252 can execute the same processing as the histogram analysis unit 251 and the threshold value comparison processing unit 252 provided in 93-7. — 3 is provided.
- the histogram analysis units 251-1 to 2 51-3 analyze the histogram of the luminance value of each pixel of the captured image, and the threshold comparison processing units 252-1 to 252- In step 3, the luminance is compared with the threshold value, and the luminance having the number of pixels equal to or larger than the predetermined threshold value is extracted.
- the luminance is determined by the number of gradation steps. It is not set to the brightness range that is preferentially assigned.
- the output level conversion processing unit 187 assigns a priority to the number of gradation steps to the pixels in the range, and each luminance level in the set luminance range. It is designed to be converted into a continuous luminance gradation with sufficient adjustment range. Then, the converted signal is subjected to gradation conversion (the number of gradations is compressed) and displayed in the display control section 94 or the output control section 95 according to the display or print output conditions. It is output for various processes such as printing, image recognition, recording, or transmission.
- pixels of the image corresponding to the image information required by the user among the images to be captured such as when the angle of view of the image to be captured is fixed or the subject is irradiated with a certain amount of illumination.
- the luminance area of a pixel corresponding to the image information required by the user changes in several patterns over time, even in the same luminance area at daytime Ability to have a lot of information, with little information in the evening, or to have a lot of information in the evening or at night There may be little information in the morning or at noon .
- the image generation unit 93-9 in FIG. 32 is used.
- the image display process 7 executed by the image processing apparatus 81 will be described.
- step S201 the operation input unit 92 receives input of set values of a plurality of luminance ranges with user power, and supplies the input set values to the image generation unit 93-9.
- Image generator 9
- step 3-9 the captured image signal is acquired.
- step S202 the imaging unit 91 captures an image of the subject based on the user operation input supplied from the operation input unit 92, logarithmically converted, and AZD-converted captured image signal. — Supply to 9.
- step S203 the first luminance area luminance range setting unit 28 of the image generation unit 93-9.
- step S204 the histogram analysis units 251-1 to 251-2-3, based on the image signal supplied from the imaging unit 91, as described with reference to FIG. A histogram showing the distribution of luminance values in the third to third ranges is generated and analyzed, and the analysis result is supplied to the threshold value comparison processing units 252-1 to 252-2.
- the threshold comparison processing units 252-1 to 252-3 display the analysis results of the histograms in the first to third ranges supplied from the histogram analysis units 251-1 to 251-2. Based on this, the number of pixels corresponding to the luminance value of each input signal is compared with a predetermined threshold value.
- the threshold value comparison processing units 252-1 to 252-3 supply the output level conversion processing unit 187 with the luminance value determined as a result of comparison with the threshold value that the number of pixels is equal to or greater than the threshold value.
- step S206 the output level conversion processing unit 187 sets a luminance range to which a large number of gradation steps are allocated based on the luminance values supplied from the threshold comparison processing units 252-1 to 252-2. .
- step S207 the output level conversion processing unit 187 determines the conversion characteristics between the input level and the output level as described with reference to FIG. 6, FIG. 7, or FIG.
- step S208 the output level conversion processing unit 187 converts the gradation of the captured image supplied from the imaging unit 91 based on the conversion characteristics determined in step S74. To the output control unit 95. [0304] In step S209, the output control unit 95 converts the supplied image signal into a gradation suitable for the processing of the output control unit 95, for example, as described with reference to FIG.
- step S210 the output control unit 95 controls the output of the converted image data converted to the gradation suitable for the processing executable by the image utilization device 83 to the image utilization device 83, and Processing is terminated.
- step S201 to step S208 basically the same processing is executed, the image signal is supplied to the display control unit 94, converted into a gradation that can be processed in the display 82, and output, and the display is displayed. Be controlled.
- the image processing device 81 in which the image generation units 93-1 to 93-9 are used, the image is captured by the imaging unit 91 using the logarithmic conversion type imaging device 102. Even if the image data required by the user exists in a wide luminance range in the image data, such an image is displayed with a gradation corresponding to the display 82, or is output. Information required by the user when output in the gradation corresponding to the previous external device (for example, dark objects in the dark or bright parts existing in the same image as the dark one) Image) can be displayed in an identifiable state, or image data suitable for use in various processes such as printing, image recognition, storage, and transmission can be generated.
- Luminance compression is to reduce the number of gradations (number of gradation steps) of the luminance value of image data.
- the image conversion process to which the present invention is applied is executed, for example, the luminance region portion to be recognized by the user is displayed or printed out with a sufficient contrast, or various images are displayed.
- the processing it is possible to easily determine a threshold value of binary values and to easily detect a predetermined object based on an image.
- the image using device 83 is a device that executes an image using process such as an image printout process, an image recognition process, an image recording process, and an image communication process.
- the brightness-compressed wide dynamic range image data obtained as described above includes information necessary for recognizing the target content in the original wide dynamic range image, and The amount of data has been greatly reduced. Therefore, in a process that does not require luminance information that corresponds to the luminance of the object photographed for each pixel in a one-to-one relationship, the wide dynamic range image data with luminance compression obtained as described above is used. Can be used.
- 14bit AZD converted wide dynamic range image is very wide luminance band
- the conventional image processing apparatus cannot cope with it as it is because it includes a region. Specifically, for example, when a Laplacian transformation that performs differentiation processing is performed, only noise-like results can be obtained, or there are too many luminance candidates that can serve as a threshold value for binary key processing. Problems such as an explosive increase in volume occur.
- wide dynamic range images have the same purpose as conventional image processing because they are different in nature from images used in conventional image processing. However, if a wide dynamic range image is used, the processing process will require significant modifications.
- the wide dynamic range image data should be used. It is very large data that includes even no information, and it is inefficient to handle such data.
- luminance compression is performed even when the subject to be processed is dispersed in the deviated luminance region, and the subject is viewed. Even when viewed as a normal image, an image with no sense of incongruity can be obtained by observing with the human eye, as in the case of shooting a narrow luminance range with a conventional imaging device. It can be applied in the processing process.
- luminance compression is performed in a state where the content of the image is recognizable, so that it is possible to prevent recording or transmitting huge data including unnecessary information. .
- the compression rate of the image data depends on the slope of the corresponding curve of the output signal level relative to the input signal level assigned to the selected luminance region. Is determined. Therefore, the slope of the corresponding curve of the output signal level with respect to the input signal level, that is, the conversion characteristic between the input level and the output level, should be determined according to the number of gradation steps required by the image utilization device 83! If the slope of the conversion characteristic between the input level and the output level is moderated, the compression ratio can be set high. If the slope of the conversion characteristic is steep, the compression ratio can be set low.
- the image utilization device 83 captures an image of the front of a traveling car and takes the vehicle, for example.
- a device that uses an image close to a binarized image such as a detection device that executes processing for detecting a white line on the road, it is preferable to set the compression rate high.
- a certain level of gradation such as when recording an image, it is preferable to assign the necessary number of gradation steps.
- the image utilization device 83 captures an image of the front surface of a running car, and the state of the running road surface, specifically, the road surface is dry, wet,
- a road condition detection device that detects force / force in a frozen state or a snowy condition
- a wide luminance range is selected for the luminance region corresponding to the road surface, and the other luminance range is selected. If a relatively narrow luminance range is selected, the compression rate can be increased without reducing the amount of road surface information.
- the image utilization device 83 is a recording device for night vision (night-time forward image display device), for example, the gradation of the bright area should be less than the gradation of the dark area.
- the compression rate can be increased without degrading the image quality in the important luminance region.
- the image processing device 81 when the state of the imaging environment such as day and night can be determined using other means such as a sensor, the image processing device 81 has a luminance with a different pattern depending on the state of the target. It is desirable to set the range. For example, if a large amount of brightness is assigned to a relatively bright area during the day and an average is assigned to the low brightness, central brightness, and high brightness areas at night, the brightness compression rate can be increased.
- the slopes of the conversion characteristics of the input level and the output level in the main area luminance range and the second luminance range indicated by B and C are the same. This is, for example, shown in FIG. This is because the brightness distribution inside and outside the tunnel is matched with the tunnel image shown in Fig. 1.
- the image utilization device 83 that executes processing that is affected by the density distribution, in each of FIGS. 15A, B, and C. It is desirable that the slopes of the input level and output level conversion characteristics in the main luminance range, the second luminance range, and the third luminance range are the same.
- the inclination of the conversion characteristics of the input level and output level is changed as necessary for each region, The compression rate can be increased.
- the luminance compression rate can be reduced by setting the luminance range accordingly. Can be increased. For example, when shooting in front with an in-vehicle camera, the road surface appears in the lower center of the screen, so if the upper and lower luminance ranges are determined based on the luminance of the road surface, the required luminance range can be set accurately. Thus, the compression rate can be increased without losing the necessary luminance range.
- a wide dynamic range image that has been AZD converted with 14 bits includes a very wide luminance band, and thus cannot be directly supported by a conventional image processing apparatus.
- a conventional image processing apparatus when performing Laplacian transformation that performs differential processing, there are too many luminance candidates that can only be a noise-like result or can be a threshold value for binary processing. Problems such as an explosive increase in throughput will occur.
- wide dynamic range images have the same purpose as conventional image processing because their properties are different from those used in conventional image processing. However, if a wide dynamic range image is used, the processing process will require significant modifications.
- the wide dynamic range image data should be used. It is very large data that includes even no information, and it is inefficient to handle such data.
- the subject to be processed is dispersed in the separated luminance regions. Even in such cases, brightness compression of AZD-converted wide dynamic range images is performed, so conventional image recognition devices can be applied in the same processing process. In the same way as when photographing a narrow luminance range with this image sensor, it is possible to obtain a completely uncomfortable image by observing with the human eye.
- a predetermined luminance range is uniformly AZD-converted by a 12-bit AZD converter
- 4096-gradation image data is obtained.
- the luminance range that can be photographed is other than the luminance range that can be imaged, which is very narrow compared to the logarithmic conversion type image sensor used in the present invention. Since no gradation can be obtained, even if the luminance range is expressed with 4096 gradations, an image with no sense of incongruity can be obtained by observing with the human eye.
- the luminance range from darkness at night to direct sunlight can be imaged. Even if the range is expressed in 4096 gradations, the image becomes unnatural and uncomfortable.
- the logarithmic conversion type image sensor used in the present invention is configured to AZD convert a signal obtained by logarithmically converting the luminance level obtained by imaging.
- AZD a signal obtained by logarithmically converting the luminance level obtained by imaging.
- an image of a range illuminated by a single light source eg, the sun or a streetlight
- the color of the road surface is usually gray, so it is distributed in a relatively dark area within a narrow luminance range of about 1Z16.
- the 128 gradations are insufficient, so an unnatural part appears when displayed.
- paint such as white lines, stop lines, and pedestrian crossings drawn on the road surface by image processing
- a slight luminance difference is output as a large difference of 1Z128 due to quantization error. This increases the possibility of misrecognition.
- FIG. 11 is a block diagram showing a configuration of an image processing apparatus 381 that is set so that gradations are sufficiently allocated.
- FIG. 35 uses FIG. 2 except that an imaging unit 391 is provided instead of the imaging unit 91, and an image generation unit 392 is provided instead of the image generation unit 93. This has basically the same configuration as the image processing apparatus 81 described.
- the imaging unit 391 captures an image of the subject based on the user's operation input supplied from the operation input unit 92, and based on the captured image signal or based on the operation input by the user. Brightness The gradation assignment is determined for each degree range, AZD conversion is performed, and the obtained image signal is supplied to the image generation unit 392. Details of the imaging unit 391 will be described later with reference to FIG.
- the operation input unit 92 includes, for example, an input device such as a button such as a relays button, operation keys, and a touch panel.
- the operation input unit 92 receives an operation input from the user, and issues an instruction from the user to the imaging unit 391 and the image generation unit. Supply to 392. Further, when the operation input unit 92 receives an input of a predetermined setting value related to gradation allocation of the AZD conversion process executed by the imaging unit 391 by the user, the operation input unit 92 supplies the setting value to the imaging unit 391. .
- the image generation unit 392 executes processing for converting the image signal supplied from the imaging unit 391 into an image signal suitable for display or print output, and supplies the image signal to the display control unit 94 or the output control unit 95.
- the image generation unit 392 does not execute the brightness range setting process and the output level conversion process as described with reference to FIGS. Only the image signal generation process is executed.
- the image generation unit 392 when the image generation unit 392 receives a range setting of an image to be displayed or printed out from the operation input unit 92, for example, the image generation unit 392 performs image region clipping or overall contrast. When the operation input for adjusting the image is received, the overall contrast of the image is adjusted, but the process of changing the gradation assignment of the AZD conversion process executed in the imaging unit 391 is not performed.
- the display control unit 94 performs processing to convert the processed image signal supplied from the image generation unit 392 into the resolution and the number of gradations of the display 82, and supplies the processed signal to the display 82. To do.
- the output control unit 95 performs a process of converting the processed image signal supplied from the image generation unit 392 into a resolution and the number of gradations that can be processed by the image utilization device 83, and outputs the processed signal. And output to the image utilization device 83.
- the display 82 receives, for example, the display image signal supplied from the display control unit 94, and displays an image (a still image or a moving image including a plurality of frames).
- the image using device 83 requests input of the image signal supplied from the output control unit 95, and executes predetermined processing.
- the image utilization device 83 includes, for example, various types of information that execute processing using an image, such as an image printing device, an image recognition device, an image recording device, and an image communication device.
- the information processing device can be used.
- the image signal generated by the image generation unit 392 also has a one-to-one correspondence between the luminance of the imaging target and the luminance data of the image data (the luminance of the imaging target and the image data).
- the brightness data of the image data must correspond linearly) It is preferable to use the image utilization device 83 for executing the wrinkle processing.
- the brightness of the imaging target and the brightness data of the image data do not have to correspond one-to-one!
- the processing includes, for example, print output, recording processing, processing for recognizing a predetermined target in the image, There are processing to detect an edge portion or straight line portion in an image, binarization processing, or transmission processing to another device that executes such processing.
- FIG. 36 is a block diagram showing a more detailed configuration example of the imaging unit 391 of the image processing device 381 in FIG.
- the imaging unit 391 basically has the same configuration as the imaging unit 39 in FIG. 3 except that a logarithmic conversion type imaging device 401 is provided instead of the logarithmic conversion type imaging device 102. Yes.
- the logarithmic conversion type imaging device 401 basically has the gradation assignment determining unit 41 1 and a AZD conversion unit 412 instead of the AZD conversion unit 113. It has the same configuration as the logarithmic conversion type image sensor 102
- the logarithmic conversion type imaging device 401 is a logarithmic conversion type imaging device such as HDRC, for example, and includes a light detection unit 111, a logarithmic conversion unit 112, a gradation assignment determination unit 411, an AZD conversion unit 412, and an imaging unit.
- the timing control unit 114 is configured to be included.
- the light generated by the imaging unit 391 and also having a subject power (or light reflected by the subject) is incident on the lens 101 and is not shown in the light detection unit 111 of the logarithmic conversion type imaging device 401.
- An image is formed on the detection surface.
- the light detection unit 111 converts the light of the subject imaged by the lens 101 into a charge corresponding to the brightness (illuminance) of the incident light, and accumulates the converted charge.
- the light detection unit 111 supplies the accumulated charge to the logarithmic conversion unit 112 in synchronization with the control signal supplied from the imaging timing control unit 114.
- the logarithmic converter 112 uses the subthreshold characteristic of the MOSFET to charge the charge supplied from the light detector 111 for each pixel.
- the logarithmic conversion unit 112 supplies the generated analog electric signal to the AZD conversion unit 412 and also supplies it to the gradation assignment determination unit 411.
- the gradation assignment determination unit 411 analyzes the analog electrical signal supplied from the logarithmic conversion unit 112, and determines gradation assignment for AZD conversion executed in the AZD conversion unit 412.
- the gradation assignment determination unit 411 detects the main luminance range (luminance region) in the luminance distribution of the input image, so that the image in the luminance range can be sufficiently recognized. Assigns the number of gradation steps for / D conversion.
- the brightness range may be set to one or more than one, and the area between the multiple areas may not have a gradation step at all. In some cases, coarse and gradation can be obtained.
- the set luminance range may be automatically selected and set from the captured image, or may be set by user operation input.
- the 80 conversion unit 412 performs AZD conversion of the analog electrical signal into digital image data in synchronization with the control signal supplied from the imaging timing control unit 114. At this time, the AZD conversion unit 412 performs A according to the gradation allocation determined by the gradation allocation determination unit 411.
- the AZD conversion unit 412 supplies the converted digital image data to the image processing device 92.
- the imaging unit 391 is not proportional to the logarithm of the brightness (incident light amount) of the subject incident on the light detection unit 111, and the gradation assigned by the gradation assignment determination unit 411 is adjusted.
- Base! / ⁇ Outputs AZD-converted digital image data.
- FIG. 37 is a block diagram showing a configuration of a gradation assignment determining unit 411-1 which is a first example of a configuration of the gradation assignment determining unit 411 in FIG.
- the average luminance calculation unit 451 acquires the analog image signal supplied from the logarithmic conversion unit 112, calculates the average luminance, and outputs the average luminance calculation result to the main area luminance range setting unit. Supply to 452.
- the main area luminance range setting unit 452 sets the luminance range of the main area based on the average luminance of the image signal supplied from the average luminance calculation unit 451, and converts the set luminance range of the main area to the gradation. This is supplied to the allocation calculation unit 455 and the second luminance area luminance average value calculation unit 453.
- the main area luminance range setting unit 452 may use, for example, a predetermined luminance range centered on the average luminance of the image signal as the luminance range of the main area, and the pixel that is the average luminance of the image signal. As a center, a predetermined number of pixels may be selected in order from the one having a luminance value close to this luminance, and the luminance range of the main area may be selected.
- Second luminance area luminance average value calculation section 453 has a higher luminance than the luminance range of the main area set by main area luminance range setting section 452 among the image signals supplied from logarithmic conversion section 112. The average luminance of the pixels in the range is calculated, and the calculation result is supplied to the second luminance area luminance range setting unit 454.
- the second luminance area luminance range setting unit 454 is based on the average luminance of the pixels in the higher luminance range than the luminance range of the supplied main area.
- the luminance range of the second luminance area is set, and the set luminance range of the second luminance area is supplied to the gradation assignment calculating unit 455.
- the second luminance area luminance range setting unit 454 sets a predetermined luminance range centered on the average luminance of pixels in a luminance range higher than the luminance range of the main area as the luminance range of the second luminance area.
- a predetermined number of pixels may be selected in order from pixels having a luminance value close to this luminance, centering on a pixel having an average luminance of pixels in a luminance range higher than the luminance range of the main region.
- the luminance range of the luminance region may be the same.
- the gradation allocation calculation unit 455 acquires the image signal supplied from the logarithmic conversion unit 112, and is supplied with the main region luminance range setting unit 452 and the second luminance region luminance average value calculation unit 453. In addition, based on the information on the luminance range of the main area and the luminance range of the second luminance area, the number of gradation steps to be assigned to which luminance range in the AZD conversion by the AZD conversion unit 412 decide.
- the gradation allocation calculation unit 455 performs the following on the level of the input luminance signal: Allocation of the number of gradation steps for AZD conversion shall be different between the luminance range set as the main area and the second luminance area, and other ranges. That is, the gradation assignment calculation unit 455 determines the gradation assignment so that a larger number of gradation steps are assigned to the luminance ranges set as the main area and the second luminance area. In the converter 412, AZD conversion processing is executed so that the number of gradations of pixels in the corresponding luminance range is increased. By doing so, the portion of the luminance range corresponding to the main region and the second luminance region in the image displayed or printed out can be more easily recognized by the user.
- the gradation assignment calculation unit 455 divides and assigns all the gradation step numbers to the luminance range of the main area and the luminance range of the second luminance area, for example.
- the AZD conversion level is used for a range with a lower luminance than the main region, a luminance range between the main region and the second luminance region, and a range with a higher luminance than the second luminance region. It is possible to do so without assigning the number of key steps.
- the analog signal input to A / D conversion section 412 is converted into a digital signal output after AZD conversion.
- the output level of the pixel whose luminance input level is lower than the luminance range of the main area is 0 (that is, black).
- a predetermined number of gradation steps is assigned to the pixels in the luminance range of the main area according to the input level, and AZD conversion is executed. Pixels in the luminance range between the main area and the second luminance area are output as digital signals with the same gradation as the maximum output level assigned to the luminance range in the main area, regardless of the input level. Is done.
- the pixels in the second luminance area have the same or substantially the same number of steps as the main area, and the overall gradation from the maximum gradation assigned to the luminance range of the main area according to the input level.
- the number of gradation steps up to the maximum value is assigned, and AZD conversion is executed.
- Pixels that are brighter than the second luminance area have the maximum gradation value assigned to the second luminance area, that is, the maximum value of the entire gradation (maximum output level) regardless of the input level. ) Digital signal.
- the gradation allocation calculation unit 45 for example, for the main area and the second luminance area, Assign a predetermined number of steps of all gradation steps in AZD conversion
- the luminance range between the main area and the second luminance area is smaller than the number of steps assigned to the main area and the second luminance area (in other words, the main area and the second luminance area). Narrow gradation width than the brightness range of the main area, and assigning the number of steps in the AZD conversion to the brightness range lower than the main area and the brightness area higher than the second brightness area. Can be.
- the maximum number of gradations assigned to the luminance range between the main area and the second luminance area is the same or approximately the same number of steps as the main area, depending on the input level. Gradation steps between the gradations up to the maximum value of the entire value are assigned and AZD conversion is executed. Then, the pixels having higher luminance than the second luminance area are output as a digital signal having the maximum gradation level assigned to the second luminance area, that is, the maximum output level, regardless of the input level.
- the gradation allocation calculation unit 45 performs a predetermined luminance range centered on the upper limit value of the luminance of the main region in the section ex and a predetermined value centered on the lower limit value of the luminance of the second luminance region.
- the brightness range is set to section ⁇ , and a predetermined number of gradation steps among the number of gradation steps in AZD conversion is assigned to pixels other than section ⁇ or section ⁇ in the main area or the second luminance area.
- the number of steps is smaller than the number of steps assigned to the part other than section a in the main area, and is larger than the maximum gradation value assigned to the part other than section ⁇ in the main area.
- a ZD conversion is executed by assigning a predetermined gradation.
- the gradation step is performed with a smaller number of steps than the number of steps assigned to the interval (X). The same gradation as the maximum value assigned to section a is assigned regardless of the power to which the number is assigned or the input level.
- the pixels in section / 3 have a number of steps smaller than the number of steps assigned to the part other than section a in the main area, and are larger than the maximum gradation value in the luminance range lower than section ⁇ .
- the maximum value of the gradation assigned to the section / 3 also depends on the input level. Steps are assigned.
- pixels other than section ⁇ within the brightness range of the second brightness area have the same or substantially the same number of gradation steps as the section other than section OC in the main area, that is, section (X and section ⁇
- the number of gradation steps is assigned more than the number of gradation steps, and the pixels with higher luminance than the second luminance area, regardless of the input level, are the maximum gradation value assigned to the second luminance area, that is, Is output as a digital signal with the maximum output level.
- the predetermined interval on the upper limit side in the luminance range of the main region is the interval (X, second
- the predetermined interval on the lower limit side in the luminance range of the luminance region is defined as the interval ⁇
- the predetermined interval higher than the upper limit of the luminance range of the main region is defined as the interval (X, from the lower limit of the luminance range of the second luminance region.
- a predetermined low-luminance zone may be set as zone ⁇ , and the same as zone oc and zone ⁇ on the lower limit side of the main area and the upper limit side of the brightness of the second brightness area. It is also possible to set an area where the number of output level steps can be assigned.
- all the pixels having the luminance less than or equal to the luminance range of the main area are assumed to be output 0 (black), and the range having the luminance higher than the luminance range of the second luminance area. All pixels have the same output level (maximum output level) as the highest brightness in the brightness range of the second brightness area, but below the brightness range of the main area and in the brightness range of the second brightness area. Even if each of the above luminance ranges is set, a certain number of gradation steps may be assigned.
- the gradation assignment calculating unit 455. can determine assignment of the number of gradation steps for A / D conversion, for example, as shown in FIG. That is, in a pixel whose luminance input level is lower than the luminance range of the main area, a predetermined number of gradations corresponding to the input level from 0 (that is, black) to a number smaller than the number of steps assigned to the main area. The number of steps is assigned and AZD conversion is performed. Then, a predetermined gradation step number is assigned to the pixels within the main region luminance range according to the input level, and A / D conversion is executed.
- the pixels in the luminance range between the main area and the second luminance area are output as digital signals having the maximum gradation value assigned to the main area regardless of the input level.
- a predetermined number of gradation steps from the maximum value of the output level assigned to the main area according to the input level is the same or substantially the same as the main area.
- a / D conversion is performed.
- the main region and the second region from the maximum value of the gradation step assigned to the luminance range of the second luminance region to the maximum gradation are used.
- the number of gradation steps corresponding to the input level is assigned so that the number of steps is smaller than the number of steps assigned to the luminance area, and AZD conversion is executed.
- the gradation assignment calculating unit 455 can determine the assignment of the number of gradation steps for AZD conversion, for example, as shown in FIG. 39B. That is, in a pixel whose luminance input level is lower than the luminance range of the main area, a predetermined number of gradation steps is allocated corresponding to the input level, which is smaller than the number of steps allocated from 0 (that is, black) to the main area. And AZD conversion is executed.
- a predetermined number of gradation steps is assigned to the pixels in the main area luminance range according to the input level, and AZD conversion is performed.
- the number of gradation steps corresponding to the input level is assigned with a smaller number of steps than the number of steps assigned to the main area.
- AZD conversion is performed.
- the maximum intensity of the gradation assigned to the luminance range between the main area and the second luminance area is applied to the pixels in the second luminance area.
- the AZD conversion is executed by assigning the same number of steps as the main area.
- the number of steps assigned to the main area from the maximum gradation assigned to the luminance range of the second luminance area to the maximum gradation.
- the number of gradation steps corresponding to the input level is assigned so that the number of steps is smaller, and AZ D conversion is executed.
- the gradation assignment calculating unit 455 can determine assignment of the number of gradation steps for AZD conversion, for example, as shown in Fig. 56C. That is, in a pixel whose luminance input level is lower than the luminance range of the main area, a predetermined number of gradation steps is allocated corresponding to the input level, which is smaller than the number of steps allocated from 0 (that is, black) to the main area. And AZD conversion is executed. A predetermined gradation step number is assigned to pixels within the main region luminance range and other than the interval ⁇ in accordance with the input level, and AZD conversion is executed.
- the pixels in section a have a number of steps smaller than the number of steps assigned to the part other than section a in the main area, and are larger than the maximum gradation value assigned to the part other than section ⁇ in the main area. Tone steps are assigned and AZD conversion is executed. Also, in the luminance range between the main area and the second luminance area, in the pixels that do not fall within the interval ⁇ or j8, the number of steps assigned to the interval a is smaller than the number of steps assigned to the input level. Is the corresponding gradation assigned? Regardless of the input level, the interval (the maximum gradation value assigned to X is assigned and AZD conversion is executed.
- the pixels in section / 3 have a number of steps smaller than the number of steps assigned to the part other than section a of the main area, and are larger than the maximum gradation value in the luminance range lower than section ⁇ .
- a number of tones are assigned.
- pixels other than the section j8 within the brightness range of the second brightness area are compared with the brightness range other than the section ⁇ of the main area from the maximum gradation assigned to the section j8 according to the input level.
- the gradations with the same or approximately the same number of steps are assigned.
- the range other than the section a of the main area from the maximum gradation value assigned to the luminance range of the second luminance area to the maximum gradation value.
- the gradation corresponding to the input level is assigned so that the number of steps is less than the number of steps assigned to the part.
- a predetermined luminance range centered on the upper limit value of the luminance of the main area is also represented by section a
- a predetermined luminance range centered on the lower limit value of the luminance of the second luminance area is represented by section ⁇ .
- a predetermined section on the upper limit side in the main area luminance range is defined as section (X)
- a predetermined section on the lower limit side in the luminance range in the second luminance area is defined as section ⁇
- the upper limit of the main area luminance range is set.
- the predetermined section with higher brightness than the lower limit of the brightness range of the second brightness area may be set as section ⁇ , and the predetermined section with lower brightness than the lower limit of the brightness range of the second brightness area may be set as section ⁇ . It is also possible to set an area to which the same number of output level steps as those in the section a and the section ⁇ are assigned to the upper side of the brightness and the second brightness area.
- the gradation assignment calculating unit 45 determines the ratio of the number of gradation steps assigned to the input level in the main area and the second luminance area (straight line). It is also possible to determine the number of gradation steps assigned to each luminance region so that the ratio of the gradation step number assigned to the input level in the luminance range other than the luminance region is reduced.
- the main area and the second luminance area set in the gradation assignment determining unit 411-1 are set based on the captured image that is not within the predetermined luminance range. It is what is done. That is, the gradation allocation in the AZD conversion determined by the gradation allocation determination unit 411-1 is, for example, occupying most of the screen in the entire captured image. The number of gradations is limited in the luminance range that is most important for the user to recognize the image, such as the subject to be imaged, and the luminance range that occupies most of the luminance range that is higher than the luminance range. Made to allocate lots of!
- FIG. 40A shows the assignment of the gradation step number by the analog input signal and gradation assignment calculation unit 455 when the AZD conversion based on the gradation step assignment described with reference to FIG. 38A is performed.
- AZD conversion is performed with a large number of gradation steps assigned in the main area and the second brightness area, which are discrete brightness ranges. All pixels in the luminance range between the main area and the second luminance area are output at the maximum output level of the main area. Then, the power after which the signal after AZD conversion is subjected to image processing by the image generation unit 392 in accordance with the conditions of display or output to the image utilization device 83.
- the gradation No conversion is performed. That is, when the image generation unit 392 processes the signal supplied from the gradation assignment determination unit 411-1, that is, the signal after AZD conversion based on the gradation assignment by the gradation assignment calculation unit 445, Do not change the key assignment percentage.
- FIG. 40B shows an analog input signal and gradation step number assignment by the gradation assignment calculation unit 455 when the AZD conversion based on the gradation step assignment described with reference to FIG. 38B is performed.
- the digital signal after AZD conversion based on this is displayed on the display 82, or processing such as printing, image recognition, recording, or image communication is performed.
- the gradation between the main area and the second luminance area is provided with gradations having a smaller number of steps than the main area and the second luminance area.
- FIG. 40C shows the assignment of the number of gradation steps by the analog input signal and the gradation assignment calculation unit 455 when the AZD conversion based on the gradation step assignment described with reference to FIG. 38C is performed.
- the number of gradation steps assigned in the main region, the second luminance region, and the region between them is changed gradually by the above-described interval (X and interval ⁇ , that is, In the interval ⁇ and the interval ⁇ , gradations with a smaller number of steps are given than in the main area and the second luminance area, so that the number of gradation steps given to the entire image is small.
- a sufficient luminance gradation width is given to the main area and the second luminance area, and it is possible to reduce the luminance area expressed by only one gradation.
- second luminance area luminance average value calculating section 453 and second luminance area luminance range setting section 454 are each provided. What is necessary is just not to perform a process. In this way, a large number of gradation steps are assigned only to the luminance range of the main region.For example, as shown in FIG. 41, for the analog signal input to the AZD conversion unit 412, AZD Digital signals that are converted and output can be obtained.
- the A ZD conversion is performed on the analog signal input to the AZD conversion unit 412.
- a digital signal to be output can be obtained.
- the number of gradation steps less than the main area luminance range is assigned before and after the main area luminance range, As shown in FIG. 41B, it is possible to obtain a digital signal that is AZD converted and output from the analog signal input to the AZD conversion unit 412.
- the number of gradation steps is smaller than that in the main area luminance range.
- the number of gradation steps is smaller than section a and section ⁇ .
- An analog signal input to the AZD converter 412 can be obtained as a digital signal output after AZD conversion.
- FIG. 42 shows the input signal, the signal after AZD conversion based on the gradation step assignment by the gradation assignment calculation unit 455, and the signal when the gradation assignment described with reference to FIG. 41 is performed. It is a figure which shows the luminance level of the signal displayed.
- FIG. 42 (b) shows the assignment of the number of gradation steps by the analog input signal and the gradation assignment calculation unit 455 when the AZD conversion based on the gradation step assignment described with reference to FIG. 41A is performed.
- AZD-converted digital signal based on, and the luminance of the signal displayed on the display 82 or output to the image utilization device 83 for processing such as printing, image recognition, recording, or image communication It is a figure which shows a level.
- AZD conversion is performed with a large number of gradation steps assigned only to the main area, and all pixels in the luminance range below the main area are output at an output level of 0. All pixels in the luminance range are output at the maximum output level of the main area.
- FIG. 42 (b) shows an analog input signal and gradation step calculation unit 455 assigning gradation steps when the AZD conversion based on the gradation step assignment described with reference to FIG. 41B is performed.
- the image utilization device 83 for processing such as printing, image recognition, recording, or image communication It is a figure which shows a level.
- the areas other than the main area are given a gradation with fewer steps than the main area.
- the converted signal has a small number of gradation steps applied to the entire image, but the number of steps in the region other than the main region is small. Accordingly, a sufficient luminance gradation width is given to the signal in the main area.
- FIG. 42C shows the assignment of the number of gradation steps by the analog input signal and gradation assignment calculation unit 455 when the AZD conversion based on the gradation step assignment described with reference to FIG. 41C is performed.
- the number of assigned gradation steps in the main region and the other regions is changed gently according to the above-described interval (X and interval ⁇ , that is, in the interval (X and interval ⁇ Therefore, the number of gradation steps is larger than that of the main area, which is smaller than that of the main area, so that even if the number of gradation steps given to the entire image is small, the number of steps after conversion is small.
- a sufficient luminance gradation width is given to the main region, and the luminance region near the main region has a relatively larger number of gradation steps than the luminance region deviated from the main region. Can be assigned.
- step S301 the light detection unit 111 of the imaging unit 391 supplies the accumulated charge to the logarithmic conversion unit 112 in synchronization with the control signal supplied from the imaging timing control unit 114. That is, the captured image signal is acquired.
- the logarithmic converter 112 uses the sub-threshold characteristic of the MOSFET to generate an analog electrical signal by converting the charge supplied from the light detector 111 into a voltage value approximately proportional to the logarithm of the number of charges for each pixel. And supplied to the AZD conversion unit 412 and to the gradation assignment determination unit 411-1.
- step S302 the average luminance calculating unit 451 of the gradation assignment determining unit 41-1 calculates the average luminance of the entire captured image and supplies the calculation result to the main region luminance range setting unit 452.
- step S303 the main region luminance range setting unit 452 sets the luminance range of the main region based on the average luminance of the entire image supplied from the average luminance calculation unit 451, and sets the luminance of the set main region. Range, gradation assignment calculation unit 455, and second luminance area luminance This is supplied to the average value calculation unit 453.
- the main area luminance range setting unit 452 may use, for example, a predetermined luminance range centered on the average luminance of the image signal as the luminance range of the main area, and the pixel that is the average luminance of the image signal. As a center, a predetermined number of pixels may be selected in order from the one having a luminance value close to this luminance, and the luminance range of the main area may be selected.
- the second luminance area luminance average value calculation unit 453 includes the luminance range of the main area set by the main area luminance range setting unit 452 in the image signal supplied from the logarithmic conversion unit 112. The average brightness of the brighter area is obtained, and the result is supplied to the second brightness area brightness range setting unit 454.
- step S305 the second luminance area luminance range setting unit 454 is based on the average luminance of the area brighter than the luminance range of the main area supplied from the second luminance area luminance average value calculation unit 453. Then, the luminance range of the second luminance region is set, and the set luminance range of the second luminance region is supplied to the gradation allocation calculating unit 455.
- the second luminance area luminance range setting unit 454 uses a predetermined luminance range centered on the average luminance of the pixels in the luminance range higher than the luminance range of the main area as the luminance range of the second luminance area.
- a predetermined number of pixels may be selected in order from pixels having a luminance value close to this luminance, centering on a pixel having an average luminance of pixels in a luminance range higher than the luminance range of the main region.
- the luminance range of the luminance region may be the same.
- step S306 the gradation assignment calculation unit 455 is set by the main region luminance range set by the main region luminance range setting unit 452, and the second luminance region luminance average value calculation unit 453. Based on the luminance range of the second luminance area, for example, as described with reference to FIG. 38 or FIG. 39, gradation assignment of AZD conversion in each area is determined and supplied to the AZD conversion unit 412.
- step S307 the AZD conversion unit 412 converts the analog electrical signal supplied from the logarithmic conversion unit 112 into a digital signal based on the gradation assignment supplied from the gradation assignment calculation unit 455, and This is supplied to the image generation unit 392.
- step S308 the image generation unit 392 performs image processing on the supplied image signal (does not perform gradation conversion), and supplies it to the output control unit 95.
- step S309 the output control section 95 controls the output of the image data after the image processing to the image utilization device 83, and the processing is terminated.
- step S301 to step S307 basically the same processing is executed, the image signal is supplied to the display control unit 94, and the display on the display 82 is controlled.
- the limited number of gradation steps that the AZD conversion unit 412 has is effective for image data that has been imaged by the imaging unit 391 using the logarithmic conversion type imaging device 401 and has been AZD converted. Therefore, even if the captured image has a wide dynamic range, a portion of the luminance range corresponding to the main region and the second luminance region can be displayed or A printed output image can be obtained.
- the information power required by the user among the images captured by the wide dynamic range is easily recognized by the user.
- a sufficient number of gradation steps is assigned and AZD conversion is performed, and then display or printout is performed.
- a wide dynamic range image obtained by AZD conversion to which a large number of bits such as 16 bits are allocated includes a very wide luminance global area.
- Laplacian transformation processing that performs differentiation processing is performed in the same manner as in, noise results and cannot be obtained as much as an image with extremely low contrast.
- binary processing when binary processing is performed, the number of luminance candidates that can be used as threshold values increases so much that the amount of processing increases explosively.
- a / D converter 412 has a limited number of gradation steps (e.g., 12-bit brightness range that can be captured is very narrow compared to the logarithmic conversion type image sensor used in the present invention.
- the number of bits is effectively allocated to perform AZD conversion. If it is applied, the image generation unit 392 can execute the conventional image processing process as it is.
- the imaging unit 391 using the logarithmic conversion type imaging device 401 is used. Since the number of gradation steps is mainly assigned to the luminance area to be used in the image of the wide dynamic range captured in this way, it is necessary to save recording capacity or transmission / reception path traffic while transmitting necessary information. Can do.
- the luminance range of the main area and the second luminance area which is the main part of the luminance range higher than the luminance range of the main area, are selected. It has been described that AZD conversion is performed by assigning a large number of gradation steps to the luminance range (or only the luminance range in the main area). On the other hand, the case where the brightness range to be set is not two of the brightness range of the main area and the brightness range of the second brightness area will be described.
- FIG. 44 is a block diagram showing a configuration of a gradation assignment determining unit 411-2 which is a second example of a configuration of the gradation assignment determining unit 411 of FIG.
- the gradation assignment determination unit 411-2 includes the luminance range of the second luminance area, which is the main part of the luminance range higher than the luminance range of the main area, and the luminance of the main area.
- the lower brightness range Set three luminance areas with the main area of the third luminance area, and assign a large number of gradation steps to the set three luminance areas. Has been made.
- the gradation assignment determining unit 41-2 in FIG. 44 has an average luminance calculation unit 451 basically the same as the gradation assignment determining unit 411-1 described with reference to FIG.
- Area luminance range setting unit 452, second luminance area luminance average value calculating unit 453 and second luminance area luminance range setting unit 454 of gradation allocation determining unit 411-1 are omitted, and high luminance region luminance
- An average value calculation unit 461, a high luminance region luminance range setting unit 462, a low luminance region luminance average value calculation unit 463, and a low luminance region luminance range setting unit 464 are newly provided, replacing the gradation assignment calculation unit 455.
- a gradation allocation calculation unit 465 is provided.
- the high-luminance area luminance average value calculation unit 461 acquires the image signal supplied from the logarithmic conversion unit 112, and among the acquired image signals, the main area set by the main-area luminance range setting unit 452 The average luminance of the pixels in the higher luminance range than the luminance range is calculated, and the calculation result is supplied to the high luminance region luminance range setting unit 462.
- the brightness range setting unit 462 of the high brightness area has a brightness higher than that of the main area based on the average brightness of the pixels in the brightness range higher than the brightness range of the main area supplied from the high brightness area brightness average value calculation unit 461.
- the luminance range of the second luminance area is set, and the set luminance range of the second luminance area is supplied to the gradation allocation calculating unit 465.
- the high luminance area luminance range setting unit 462 uses a predetermined luminance range centered on the average luminance of pixels in a luminance range higher than the luminance range of the main area as the luminance range of the second luminance area. It is also possible to select a predetermined number of pixels in order from the one having a luminance value close to this luminance around the pixel that is the average luminance of the pixels in the luminance range higher than the luminance range of the main area, and then select the second luminance.
- the luminance range of the area may be used.
- the low-brightness area luminance average value calculation unit 463 acquires the image signal supplied from the logarithmic conversion unit 112, and among the acquired image signals, the main area set by the main-area luminance range setting unit 452 Calculate the average brightness of pixels in the brightness range lower than the brightness range of Is supplied to the low luminance area luminance range setting unit 464.
- the low luminance area luminance range setting unit 464 is lower than the main area based on the average luminance of the pixels in the luminance range lower than the main area luminance range supplied from the low luminance area luminance average value calculation unit 463.
- the luminance range of the third luminance area is set, and the set luminance range of the third luminance area is supplied to the gradation allocation calculating unit 465.
- the low luminance area luminance range setting unit 464 uses, for example, a predetermined luminance range centered on the average luminance of pixels in a luminance range lower than the luminance range of the main area as the luminance range of the third luminance area. It is also possible to select a predetermined number of pixels in order from the pixel having the luminance value close to this luminance around the pixel that is the average luminance of the pixels in the luminance range lower than the luminance range of the main region, and select the third region. It may be a luminance range.
- the gradation allocation calculation unit 465 acquires the image signal supplied from the logarithmic conversion unit 112, and obtains the main region luminance range setting unit 452, the high luminance region luminance range setting unit 462, and the low luminance region luminance. Based on the information on the luminance range of the main region, the luminance range of the second luminance region, and the luminance range of the third luminance region supplied from the range setting unit 464, the AZD conversion unit 41 2 performs the AZD conversion.
- ⁇ Determine the number of gradation steps to be assigned to the brightness range of deviation.
- the gradation assignment calculation unit 465 assigns the number of gradation steps for AZD conversion to the level of the input luminance signal, the main area, the second luminance area, and the first The brightness range set as the brightness range 3 and the other range are different. That is, the gradation assignment calculation unit 465 assigns gradation assignments so that a larger number of gradation steps are assigned to the luminance ranges set as the main area, the second luminance area, and the third luminance area. Therefore, in the AZD conversion unit 412, the AZD conversion process is performed so that the number of gradations of the pixels in the corresponding luminance range is increased. As a result, the main area, the second luminance area, and the portion of the luminance range corresponding to the third luminance area in the displayed or printed image can be more easily recognized by the user. It ’s like this.
- the gradation allocation calculation unit 465 shows the digital signal output by AZD conversion for the analog signal input to the AZD conversion unit 412 as shown in FIG. 45A.
- gradation can be assigned. That is, with respect to the analog signal input to the AZD conversion unit 412, in the digital signal output after AZD conversion, the output level of the pixel whose luminance input level is lower than the luminance range of the third luminance region is 0. (That is, black). Then, a predetermined number of gradation steps is assigned to the pixels in the third luminance area, the main area, and the second luminance area according to the input level, and AZ D conversion is executed. .
- the pixels in the luminance range between the third luminance region and the main region and the luminance range between the main region and the second luminance region are assigned immediately before that range regardless of the input level. It is output as a digital signal with the same gradation as the maximum output level. Pixels that are brighter than the second luminance area are digital values of the maximum gradation assigned to the luminance range of the second luminance area, that is, the maximum value (maximum output level) of the entire gradation. Output as a signal.
- the gradation allocation calculation unit 465 for example, the analog signal input to the AZD conversion unit 412 and the digital signal output after AZD conversion is as shown in FIG. 45B.
- the pixels in the third luminance area, the main area, and the second luminance area are assigned an output level of a predetermined gradation according to the input level, and AZD conversion is executed. .
- the number of pixels in the luminance range between the third luminance area and the main area and the luminance range between the main area and the second luminance area is smaller than the number of steps assigned to the main area.
- the output level corresponding to the input level is assigned by the number of steps, and AZD conversion is executed. Pixels that are brighter than the second luminance area, regardless of the input level, have the maximum gradation value assigned to the second luminance area, that is, the maximum gradation value (maximum output). Level) digital signal.
- the gradation allocation calculation unit 465 for example, the analog signal input to the AZD conversion unit 412 so that the digital signal output by AZD conversion is as shown in FIG. 45C.
- a predetermined luminance range centered on the upper limit value of the luminance of the third luminance area is set as the section ⁇
- the lower limit value of the luminance of the main area is set as the center.
- the predetermined luminance range is defined as section ⁇
- the predetermined luminance range centered on the upper limit value of the luminance in the main area is defined as section ⁇
- the predetermined luminance range centered on the lower limit value of the luminance in the second luminance area is defined as section ⁇ .
- the pixel whose luminance input level is lower than that in the third luminance region is the output level regardless of the input level. Is assumed to be 0 (ie, black).
- pixels other than section OC, within the main area luminance range, pixels other than section ⁇ or ⁇ , and within the second luminance area, section ⁇ Pixels other than are assigned an output level of a predetermined gradation according to the input level, and AZD conversion is executed.
- the number of steps is less than the number of steps assigned to the part other than section O to section ⁇ , such as the main area luminance range.
- AZD conversion is executed by assigning the output level of the predetermined gradation. Pixels that are brighter than the second luminance area have the maximum gradation value assigned to the second luminance area, that is, the maximum gradation value (maximum output) regardless of the input level. Level) digital signal.
- a predetermined luminance range centered on the upper limit value of the luminance of the third luminance area is defined as section O
- a predetermined luminance range centered on the lower limit value of the luminance of the main area is defined as section.
- ⁇ a predetermined luminance range centered on the upper limit value of the luminance of the main area is defined as interval ⁇
- a predetermined luminance range centered on the lower limit value of the luminance of the second luminance area is defined as interval ⁇ .
- the predetermined brightness range on the upper limit side in the brightness area of the main area is section OC
- the predetermined brightness range on the lower limit side in the main area is section ⁇
- the predetermined brightness range on the upper limit side in the main area is section ⁇
- the second The predetermined luminance range on the lower limit side in the luminance area is defined as the interval ⁇
- the predetermined luminance range higher than the upper limit of the third luminance region is the interval OC
- the predetermined luminance lower than the lower limit of the main region The range is section ⁇
- the predetermined brightness range is higher than the upper limit of the main area, section ⁇
- a predetermined luminance range with lower luminance may be set as the section ⁇ .
- an area is set so that the number of steps of the same output level as that of the section or section ⁇ is assigned to the lower limit side of the brightness of the third brightness area and the upper limit side of the brightness of the second brightness area.
- the luminance is lower than that of the third luminance region, and is lower than that of the second luminance region.
- the number of output level steps (the number of gradation steps) is not assigned to the high luminance range.
- the gradation allocation calculation unit 465 has a lower luminance than the third luminance region and a higher luminance than the second luminance region, for example, as described with reference to FIG. Even in the range, you may assign fewer steps than the main area, the second luminance area, and the third luminance area.
- the gradation allocation calculating unit 465 determines the luminance levels of the main area, the second luminance area, and the third luminance area from the ratio of the output level to the input level (straight line). It is also possible to determine the number of gradation steps assigned to each luminance area so that the ratio of the number of gradation steps assigned to the input level in the luminance range other than the area is reduced.
- FIG. 46A shows an analog input signal when the AZD conversion based on the gradation step assignment described with reference to FIG. 45A is performed. Indicates the luminance level of the signal after AZD conversion based on this and the signal displayed on the display 82 or output to the image using device 83 for processing such as printing, image recognition, recording, or image communication FIG.
- FIG. 46A in the main area, the second luminance area, and the third luminance area, which are discrete luminance ranges, many gradation step numbers are assigned and AZD conversion is executed.
- All pixels in the luminance range between the third luminance area and the main area are output at the maximum output level of the third luminance area, and all pixels in the luminance range between the main area and the second luminance area are output. Output at the maximum output level of the main area.
- FIG. 46B shows an analog input signal based on the gradation step assignment by the gradation assignment calculation unit 465 when the AZD conversion based on the gradation step assignment described with reference to FIG. 45B is performed.
- FIG. 6 is a diagram showing luminance levels of signals after AZD conversion and signals displayed on the display 82 or output to the image utilization device 83. As shown in Figure 46B, between the third luminance region and the main region, and between the main region and the second region. The area between the luminance areas is provided with gradations having a smaller number of steps than the main area, the second luminance area, and the third luminance area.
- the number of steps in the area between the third luminance area and the main area and between the main area and the second luminance area is small despite the small number of gradation steps given to the entire image. Minute, main area, second luminance area, and third luminance area signals are given sufficient luminance gradation widths, and AZD conversion is executed.
- FIG. 46C is a diagram illustrating the analog input signal and gradation step assignment by the gradation assignment calculation unit 465 when the AZD conversion based on the gradation step assignment described with reference to FIG. 45C is performed.
- FIG. 7 is a diagram showing the luminance level of the signal after AZD conversion based on the signal and the signal displayed on the display 82 or output to the image utilization device 83.
- the number of gradation steps assigned to the main area, the second luminance area, the third luminance area, and the area between them is gently changed according to the above-described interval oc to interval ⁇ .
- step S331 the light detection unit 111 of the imaging unit 391 supplies the accumulated charge to the logarithmic conversion unit 112 in synchronization with the control signal supplied from the imaging timing control unit 114. That is, the captured image signal is acquired.
- the logarithmic converter 112 uses the sub-threshold characteristic of the MOSFET to generate an analog electrical signal by converting the charge supplied from the light detector 111 into a voltage value approximately proportional to the logarithm of the number of charges for each pixel. And supplied to the AZD conversion unit 412 and to the gradation assignment determination unit 411-2.
- step S332 the average luminance calculating unit 451 of the gradation assignment determining unit 411-2 calculates the average luminance of the entire captured image, and the calculation result is used as the main region luminance range setting unit 45. Supply to 2.
- step S333 the main area luminance range setting unit 452 sets the luminance range of the main area based on the average luminance of the entire image supplied from the average luminance calculation unit 451, and sets the luminance of the set main area.
- the range is supplied to the gradation allocation calculating unit 465, the high luminance region luminance average value calculating unit 461, and the low luminance region luminance average value calculating unit 463.
- the main area luminance range setting unit 452 may use, for example, a predetermined luminance range centered on the average luminance of the image signal as the luminance range of the main area, and the pixel that is the average luminance of the image signal. As a center, a predetermined number of pixels may be selected in order from the one having a luminance value close to this luminance, and the luminance range of the main area may be selected.
- the high luminance area luminance average value calculation unit 461 includes the luminance range of the main area set by the main area luminance range setting unit 452 in the image signal supplied from the logarithmic conversion unit 112. The average brightness of the bright area is obtained, and the result is supplied to the high brightness area brightness range setting unit 462.
- step S335 the high luminance area luminance range setting unit 462 performs the second calculation based on the average luminance of the area brighter than the luminance range of the main area supplied from the high luminance area luminance average value calculation unit 461.
- the brightness range of the brightness area is set, and the set brightness range of the second brightness area is supplied to the gradation allocation calculating unit 465.
- the high luminance area luminance range setting unit 462 uses a predetermined luminance range centered on the average luminance of pixels in a higher luminance range than the luminance range of the main area as the luminance range of the second luminance area. It is also possible to select a predetermined number of pixels in order from the one having a luminance value close to this luminance around the pixel that is the average luminance of the pixels in the luminance range higher than the luminance range of the main area, and then select the second luminance.
- the luminance range of the area may be used.
- step S336 the low luminance area luminance average value calculation unit 463 determines that the image signal supplied from the logarithmic conversion unit 112 is larger than the luminance range of the main area set by the main area luminance range setting unit 452. The average brightness of the dark area is obtained, and the result is supplied to the low brightness area brightness range setting unit 464.
- step S337 the low luminance area luminance range setting unit 464 calculates the average luminance of the darker area than the luminance range of the main area supplied from the low luminance area luminance average value calculating unit 463. Based on this, the luminance range of the third luminance region is set, and the set luminance range of the third luminance region is supplied to the gradation allocation calculating unit 465.
- the low luminance area luminance range setting unit 464 uses a predetermined luminance range centered on the average luminance of pixels in a luminance range lower than the luminance range of the main area as the luminance range of the third luminance area. It is also possible to select a predetermined number of pixels in order from the pixel having the luminance value close to this luminance around the pixel that is the average luminance of the pixels in the luminance range lower than the luminance range of the main region, and select the third region. It may be a luminance range.
- step S3308 the gradation allocation calculating unit 465 determines the luminance range of the main region set by the main region luminance range setting unit 452, the second luminance region set by the high luminance region luminance range setting unit 462. Based on the luminance range of the third luminance region set by the luminance range setting unit 464 and the low luminance region luminance range setting unit 464, for example, as described with reference to FIG. The gradation assignment is determined and supplied to the AZD conversion unit 412.
- step S339 the AZD conversion unit 412 converts the analog electrical signal supplied from the logarithmic conversion unit 112 into a digital signal based on the gradation assignment supplied from the gradation assignment calculation unit 465, and This is supplied to the image generation unit 392.
- step S 340 the image generation unit 392 performs image processing on the supplied image signal (no gradation conversion is performed), and supplies the processed image signal to the output control unit 95.
- step S341 the output control unit 95 controls the output of the image data after the image processing to the image utilization device 83, and the process ends.
- steps S331 to S339 basically the same processing is executed, and an image signal is supplied to the display control unit 94 to control its display.
- the image data captured by the imaging unit 391 using the logarithmic conversion type image sensor 401 has a high luminance and a low luminance in which the main luminance range force is discrete, particularly in a wide luminance range. Even if the image information required by the user exists in any of the areas, the limited number of gradation steps of the AZD conversion unit 412 is effectively allocated. Therefore, even if the captured image has a wide dynamic range, the portion of the luminance range corresponding to the main region and the second luminance region can be recognized better by the user. Such a display or print output image can be obtained.
- the image displayed by the image processing device 381 including the gradation assignment determining unit 4112 in Fig. 44 is, for example, most of the captured images as described with reference to Fig. 18.
- the angle of view includes the sky, which is much higher than the brightness of the road surface, and the person wearing a black suit whose brightness is much lower than the brightness of the road. Even if it is included in the corner, a large number of gradation steps are assigned near the brightness corresponding to each of the road surface, the sky, and a person wearing a black suit. Therefore, by using the image processing device 381 including the gradation assignment determination unit 411-2 in FIG. 44, an image that is difficult to recognize due to white sky appearing in the image display process is displayed.
- the image displayed by the image processing device 381 including the gradation assignment determining unit 411-2 in Fig. 44 is specifically, for example, as described with reference to Fig. 19.
- Most of the captured image is a dark road surface inside the tunnel, and the angle of view includes the white wall inside the tunnel that is slightly brighter than the brightness of the road surface, and the part outside the tunnel that is very bright.
- the angle of view includes a black car in the tunnel that is much lower than the brightness of the road surface in the tunnel, the road surface in the tunnel, the walls of the tunnel and the tunnel Many gradation steps are assigned near the brightness corresponding to each of the black cars inside and outside the tunnel!
- the gradation assignment determining unit 411 may set three or more areas, and the number of gradation steps assigned to the set dark area is different from the set area. You can do it!
- FIG. 48 is a block diagram showing a configuration of a gradation assignment determining unit 411-3 that is a third example of the configuration of the gradation assignment determining unit 411 of FIG.
- the gradation assignment determination unit 411-3 sets a plurality of luminance areas and their luminance ranges in addition to the luminance range of the main area, and sets the plurality of luminance ranges more than the luminance ranges that are not set. It is made possible to assign a large number of gradation steps!
- the gradation assignment determination unit 411-3 in FIG. 48 has an average luminance calculation unit 451 that is basically the same as the gradation assignment determination unit 411-1 described with reference to FIG.
- the second luminance area luminance range setting unit 454 and the second luminance area luminance average value calculating unit 453 of the gradation assignment determining unit 411-1 are omitted, and the second luminance area luminance range setting unit 452 is included.
- the second luminance area luminance average value calculation unit 481 is a predetermined value other than the luminance range of the main area set by the main area luminance range setting unit 452 among the image signals supplied from the logarithmic conversion unit 112. (E.g., the range with higher luminance than the luminance range of the main region is further divided into two, and the average luminance of pixels having the luminance included in the highest luminance portion, etc.) is calculated, and the calculation result is Is supplied to the luminance range luminance range setting unit 482.
- the second luminance area luminance range setting unit 482 is based on the average luminance of pixels having luminance included in a predetermined range other than the supplied main area. In addition, the brightness range of the second brightness area is set, and the set brightness range of the second brightness area is supplied to the gradation allocation calculating unit 487. [0458]
- the second luminance area luminance range setting unit 482 may set, for example, a predetermined luminance range centered on the average luminance of pixels having a predetermined luminance range as the luminance range of the second luminance area. A predetermined number of pixels are selected in order from the one having a luminance value close to this luminance around the pixel having the average luminance of the pixels having the luminance in the predetermined range as the luminance range of the second luminance region. good.
- the third luminance area luminance average value calculating unit 483 is a predetermined value other than the luminance range of the main area set by the main area luminance range setting unit 452 among the image signals supplied from the logarithmic conversion unit 112. (E.g., the range of higher brightness than the brightness range of the main area is further divided into two, and the average brightness of the pixels that have the brightness included in the two-part divided, etc.) , And supplied to the third luminance area luminance range setting unit 484.
- the third luminance area luminance range setting section 484 is based on the average luminance of pixels having luminance included in a predetermined range other than the supplied main area.
- the brightness range of the third brightness area is set, and the set brightness range of the third brightness area is supplied to the gradation allocation calculating unit 487.
- the third luminance area luminance range setting unit 484 may use, for example, a predetermined luminance range centered on the average luminance of pixels having a predetermined luminance range as the luminance range of the third luminance area.
- the third area luminance range may be selected by selecting a predetermined number of pixels in order from the one having a luminance value close to this luminance around the pixel having the average luminance of the pixels having the luminance in the predetermined range. .
- the fourth luminance area luminance average value calculating unit 485 is a predetermined value other than the luminance range of the main area set by the main area luminance range setting unit 452 among the image signals supplied from the logarithmic conversion unit 112. (E.g., the range with lower brightness than the brightness range of the main area is further divided into two, and the average brightness of the pixels with the brightness included in the lower brightness range etc. is calculated. , And supplied to the fourth luminance area luminance range setting unit 486.
- the fourth luminance area luminance range setting unit 486 is based on the average luminance of pixels having luminance included in a predetermined range other than the supplied main area.
- the brightness range of the fourth brightness area is set, and the set brightness range of the fourth brightness area is supplied to the gradation allocation calculating unit 487.
- the fourth luminance area luminance range setting unit 486 may use a predetermined luminance range centered on the average luminance of pixels having a predetermined luminance range as the luminance range of the fourth luminance area.
- the fourth area luminance range may be selected by selecting a predetermined number of pixels in order from the one having a luminance value close to this luminance around the pixel having the average luminance of the pixels having the luminance in the predetermined range. .
- the gradation allocation calculation unit 487 acquires the image signal supplied from the logarithmic conversion unit 112, and obtains the main region luminance range setting unit 452, the second luminance region luminance range setting unit 482, and the third luminance region.
- the gradation assignment calculation unit 487 performs assignment of the number of gradation steps for AZD conversion, as in the case described with reference to Fig. 38, Fig. 39, or Fig. 45, for example.
- the brightness range set as the area is different from the other range.
- the gradation allocation calculation unit 487 determines the gradation allocation so that a larger number of gradation steps are allocated to the luminance range set as the main area and other areas.
- AZD conversion processing is executed so that the number of gradations of pixels in the corresponding luminance range is increased. This makes it possible for the user to better recognize the portion of the luminance range corresponding to the set luminance region in the displayed or printed image.
- FIG. 48 shows a second luminance area luminance average value calculation unit 481 and a second luminance area luminance range setting unit for setting the second to fourth luminance areas in addition to the main area. 482, a third luminance area luminance average value calculating unit 483, a third luminance area luminance range setting unit 484, a fourth luminance area luminance average value calculating unit 485, and a fourth luminance area luminance range setting unit 486.
- the gradation assignment determining unit 411-3 is further provided with another luminance region luminance average value calculating unit and luminance region luminance range setting unit so that a large number of luminance regions can be set. It ’s okay.
- the processing executed by the gradation assignment determining unit 411-3 in Fig. 48 is basically the same as the processing in the image display processing 9 described with reference to Fig. 47, and the number of areas to be set is determined. Since it corresponds to the case where it increases, the explanation is omitted.
- the gradation assignment determining unit 411-1 to the gradation assignment determining unit 411-3 described above are configured to set the main region based on the average luminance value of the entire captured image.
- the main area may be set based on the average value of the luminance of the pixels included in a predetermined area in the captured image.
- FIG. 49 is a block diagram showing a configuration of a gradation assignment determining unit 411-4 that is a fourth example of the configuration of the gradation assignment determining unit 411 in FIG.
- the gradation assignment determining unit 411-4 cuts out pixels included in a predetermined area from the captured image, and sets the main area based on the average value of the luminance of the cut out area. Be done!
- tone allocation determining section 411-4 in FIG. 49 is different from average brightness calculating section 451 except that main area cutout section 501 and main area brightness average value calculating section 502 are provided. Basically, it has the same configuration as the gradation assignment determining unit 411-1 in FIG.
- the main region cutout unit 501 acquires the image signal supplied from the logarithmic conversion unit 112, cuts out a preset image region from the acquired image signal, and sets the pixels in the cutout region as the main pixels. This is supplied to the area luminance average value calculation unit 502.
- the region of the image cut out by the main region cutout unit 501 is shown in FIG.
- the road surface is constantly imaged, for example, the region 221 at the center of the screen and slightly below the left.
- the segmented area an area where the same image is supposed to be captured constantly, the brightness of the road surface that occupies the main part of the displayed image can be made substantially constant. As a result, it is possible to prevent the display image felt by the driver from flickering frequently and to process the object extraction parameters without changing each time.
- the main area luminance average value calculation unit 502 calculates the average luminance of the pixels in the clipped area supplied from the main area cutout unit 501, and sets the average luminance calculation result as the main area luminance range setting. Part 452.
- the luminance range of the main region is calculated based on the average luminance of the clipped region calculated by the main region luminance average value calculation unit 502. Based on the brightness range of the main area, the brightness range of the second brightness area that is higher than the brightness range of the main area is set. Then, the gradation assignment calculation unit 455 assigns more gradation steps to the luminance ranges set as the main area and the second luminance area and executes AZD conversion. Alternatively, the portion of the corresponding luminance range of the image to be printed out can be recognized more by the user, or image data suitable for processing in the image utilization device 83 can be generated.
- the gradation assignment determining unit 411-4 of FIG. 49 is used for the image display processing 10 executed by the image processing device 381. I will explain.
- step S371 the light detection unit 111 of the imaging unit 391 supplies the accumulated charge to the logarithmic conversion unit 112 in synchronization with the control signal supplied from the imaging timing control unit 114. That is, the captured image signal is acquired.
- the logarithmic converter 112 uses the sub-threshold characteristic of the MOSFET to generate an analog electrical signal by converting the charge supplied from the light detector 111 into a voltage value approximately proportional to the logarithm of the number of charges for each pixel. And supplied to the AZD conversion unit 412 and to the gradation assignment determination unit 411-4.
- step S372 the main area cutout unit 501 of the gradation assignment determination unit 411-4 Is a predetermined image area cut out from the image signal supplied from the logarithmic conversion unit 112, for example, as described with reference to FIG. It supplies to the calculation part 502.
- step S373 the main area luminance average value calculation unit 502 obtains the average luminance of the pixels in the clipped area supplied from the main area cutout unit 501, and calculates the calculation result as the main area luminance range setting unit 452. To supply.
- step S374 the main area luminance range setting unit 452 sets the luminance range of the main area based on the average luminance of the pixels in the clipped area supplied from the main area luminance average value calculation unit 502. Then, the set luminance range of the main area is supplied to the second luminance area luminance average value calculation unit 453 and the gradation allocation calculation unit 455.
- steps S375 to S380 basically the same processing as in steps S304 to S309 in Fig. 43 is executed.
- the second luminance area luminance average value calculation unit 453 obtains the average luminance of the area brighter than the luminance range of the main area from the image signal supplied from the logarithmic conversion unit 112, and obtains the second luminance value.
- the brightness area brightness range setting unit 454 sets the brightness range of the second brightness area based on the average brightness of the area brighter than the brightness range of the main area.
- the gradation allocation calculating unit 455, based on the luminance range of the main region and the luminance range of the second luminance region, for example, as described with reference to FIG. 38 or FIG. Determines the AZD conversion gradation assignment in, and supplies it to the AZD conversion unit 412.
- a The ZD conversion unit 412 converts the analog electrical signal supplied from the logarithmic conversion unit 112 into a digital signal based on the gradation assignment supplied from the gradation assignment calculation unit 465, and supplies the digital signal to the image generation unit 392.
- the image generation unit 392 performs image processing on the supplied image signal (does not perform gradation conversion), and supplies it to the output control unit 95.
- the output control unit 95 controls the output of the image data after the image processing to the image utilization device 83, and the processing ends.
- the limited number of gradation steps that the AZD conversion unit 412 has for the wide dynamic range image data captured by the imaging unit 391 using the logarithmic conversion type imaging device 401 is reduced. Since the AZD conversion is performed with effective distribution, even if the captured image has a wide dynamic range, the portion of the luminance range corresponding to the main region and the second luminance region is more determined by the user. It is possible to obtain a display or printed output image that can be easily recognized. In particular, the brightness of the displayed image is often increased when moving images are displayed by cutting out an area where the same image is expected to be captured and performing processing based on the brightness of this area. It is possible to prevent flickering and to process the parameters for extracting the object without changing each time.
- the gradation assignment determining unit 411-4 described above cuts out a predetermined area in the captured image, sets the luminance range of the main area based on the pixels of the cut out area, and sets the main area luminance range.
- the brightness range of the second brightness area which is the main part of the brightness range of the brightness area of the main area and the brightness range of the main area.
- the brightness range to be set will not be described in two cases, that is, the brightness range of the main area and the brightness range of the second brightness area.
- FIG. 51 is a block diagram showing a configuration of a gradation assignment determining unit 411-5 that is a fifth example of the configuration of the gradation assignment determining unit 411 of FIG.
- the gradation assignment determination unit 411-5 cuts out a predetermined area in the captured image, sets the luminance range of the main area based on the pixels of the cut out area, and besides the luminance range of the main area, The brightness range of the second brightness area, which is the main part of the brightness range higher than the brightness range of the main area, and the third part, which is the main part of the brightness range lower than the brightness range of the main area. Three brightness areas are set with the brightness range of the brightness area, and a large number of gradation steps can be assigned to the set three brightness areas!
- the gradation assignment determining unit 411-5 in FIG. 51 performs the case described with reference to FIG. Similar to the case described with reference to FIG. 44, the main region luminance range setting unit 452 and the high luminance region luminance average value calculation are provided.
- a unit 461, a high luminance area luminance range setting unit 462, a low luminance area luminance average value calculating unit 463, a low luminance area luminance range setting unit 464, and a gradation allocation calculating unit 465 are provided.
- the gradation assignment determining unit 411-5 in FIG. 51 cuts out a predetermined region in the captured image in the same manner as the gradation assignment determination unit 411-4 described with reference to FIG.
- the luminance range of the main area is set based on the pixels of the selected area, and in addition to the luminance range of the main area, the luminance range of the main area is set in the same manner as the gradation assignment determination unit 411-2 described with reference to FIG.
- the luminance range of the second luminance area which is the main part of the higher luminance range than the luminance range
- the luminance of the third luminance area which is the main part of the lower luminance range of the main area.
- Three luminance areas with a range are set, and a large number of gradation steps are assigned to the set three luminance areas so that AZD conversion is executed.
- steps S401 to S403 basically the same processing as in steps S371 to S373 of Fig. 50 is executed.
- the light detection unit 111 of the imaging unit 391 is supplied from the imaging timing control unit 114.
- the accumulated charge is supplied to the logarithmic conversion unit 112 in synchronization with the control signal. That is, the captured image signal is acquired.
- the logarithmic converter 112 uses the sub-threshold characteristic of the MOSFET to convert an analog electric signal obtained by converting the charge supplied from the light detector 111 into a voltage value approximately proportional to the logarithm of the number of charges for each pixel. Is generated and supplied to the AZD conversion unit 412 and to the gradation assignment determination unit 411-4.
- the main area cutout unit 501 of the gradation assignment determination unit 411-4 cuts out a preset image area from the image signal supplied from the logarithmic conversion unit 112, for example, as described with reference to FIG. Then, the pixels in the extracted area are supplied to the main area luminance average value calculation unit 502.
- the main area luminance average value calculation unit 502 calculates the average luminance of the pixels in the clipped area supplied from the main area cutout unit 501 and supplies the calculation result to the main region luminance range setting unit 452.
- step S404 the main region luminance range setting unit 452 sets the luminance range of the main region based on the average luminance of the pixels in the clipped region supplied from the main region luminance average value calculation unit 502. Then, the set luminance range of the main area is supplied to the gradation allocation calculating unit 465, the high luminance area luminance average value calculating unit 461, and the low luminance area luminance average value calculating unit 463.
- steps S405 to S412 basically the same processing as in steps S334 to S341 in Fig. 47 is executed.
- the high brightness area brightness average value calculation unit 461 obtains the average brightness of the area brighter than the brightness range of the main area from the image signal supplied from the logarithmic conversion unit 112, and sets the high brightness area brightness range.
- Unit 462 sets the luminance range of the second luminance area based on the average luminance of the area brighter than the luminance range of the main area, and sets the luminance range of the second luminance area to gradation allocation calculation unit 465. Supply.
- the low-luminance area luminance average value calculation unit 463 obtains the average luminance of the area darker than the luminance range of the main area from the image signal supplied from the logarithmic conversion unit 112, and obtains the low-luminance area luminance range.
- the setting unit 464 sets the brightness range of the third brightness area based on the average brightness of the darker area than the brightness range of the main area, and calculates the assigned brightness range of the third brightness area by gradation. Supply to part 465.
- the gradation allocation calculation unit 465 sets the luminance range of the set main area, the second luminance Based on the luminance range of the region and the luminance range of the third luminance region, for example, as described with reference to FIG. 45, the gradation assignment of AZD conversion in each region is determined, and the A ZD conversion unit 412 To supply.
- the AZD conversion unit 412 converts the analog electrical signal supplied from the logarithmic conversion unit 112 into a digital signal based on the gradation assignment supplied from the gradation assignment calculation unit 465, and supplies the digital signal to the image generation unit 392.
- the image generation unit 392 performs image processing on the supplied image signal (does not perform gradation conversion! / ⁇ ), and supplies it to the output control unit 95.
- the output control unit 95 controls the output of the image data after the image processing to the image utilization device 83, and the processing is completed.
- step S401 to step S410 basically the same processing is executed, the image signal is supplied to the display control unit 94, converted into a gradation that can be processed in the display 82, and output, and the display is displayed. Be controlled.
- the limited number of gradation steps that the AZD conversion unit 412 has in the AZD converted image data captured by the imaging unit 391 using the logarithmic conversion type imaging device 401 Even if the captured image has a wide dynamic range, the partial power of the luminance range corresponding to the main area, the second luminance area, and the third luminance area is determined by the user. It is possible to obtain a display or printed output image that can be more easily recognized. In particular, when displaying a moving image, it is possible to prevent the brightness of the display image from flickering frequently and to perform processing without changing the parameters for extracting an object each time.
- the gradation assignment determination unit 411 cuts out a predetermined area in the captured image, and sets three or more areas based on the pixels in the cut out area. You can do it.
- the gradation assignment determining unit 411 can make the number of gradation steps assigned to the set area and the non-set area different.
- FIG. 53 is a block diagram showing a configuration of a gradation assignment determining unit 411-6 that is a sixth example of the configuration of the gradation assignment determining unit 411 of FIG.
- the gradation assignment determining unit 411-6 cuts out a predetermined area in the captured image, and mainly uses the pixels in the cut out area. Set the brightness range of the main area, set the corresponding brightness range of the multiple brightness areas in addition to the brightness range of the main area, and for the set multiple brightness ranges, more gradations than outside the set range You can assign a number of steps.
- tone allocation determining section 411-6 in FIG. 53 is provided with main area cutout section 501 and main area luminance average value calculating section 502 similar to those described with reference to FIG.
- main area luminance range setting unit 452, second luminance area luminance average value calculating unit 481, second luminance area luminance range setting unit 482, third luminance area luminance An average value calculating unit 483, a third luminance area luminance range setting unit 484, a fourth luminance area luminance average value calculating unit 485, a fourth luminance area luminance range setting unit 486, and a gradation allocation calculating unit 487 are provided. It has been.
- the gradation assignment determination unit 411-6 in Fig. 53 cuts out a predetermined region in the captured image in the same manner as the gradation assignment determination unit 411-4 described with reference to Fig. 49.
- the luminance range of the main area is set based on the pixels of the selected area, and in addition to the luminance range of the main area, in the same manner as the gradation assignment determination unit 413-1 described with reference to FIG.
- Each luminance range of a plurality of luminance areas is set, and a larger number of gradation steps can be assigned to the set plurality of luminance ranges than outside the setting range.
- FIG. 53 shows a second luminance area luminance average value calculation unit 481 and a second luminance area luminance range setting unit for setting the second to fourth luminance areas in addition to the main area. 482, a third luminance area luminance average value calculating unit 483, a third luminance area luminance range setting unit 484, a fourth luminance area luminance average value calculating unit 485, and a fourth luminance area luminance range setting unit 486.
- the gradation assignment determining unit 411-3 is further provided with another luminance region luminance average value calculating unit and luminance region luminance range setting unit so that a large number of luminance regions can be set. It ’s okay.
- the processing executed by the gradation assignment determining unit 411-6 in Fig. 53 is basically the same as the processing in the image display processing 11 described with reference to Fig. 52, and the number of areas to be set is determined. Increased Since this corresponds to the case, the description thereof is omitted.
- the gradation assignment determination unit 411-1 to gradation assignment determination unit 411-6 described above set the main region based on the entire captured image or the average luminance value of a predetermined portion. It was made to fix. On the other hand, a histogram showing the distribution of the luminance values of the pixels included in the captured image is created and analyzed to set multiple luminance ranges. It is also possible to assign a larger number of gradation steps than outside the setting range so that AZD conversion can be performed.
- FIG. 54 is a block diagram showing a configuration of a tone assignment determination unit 411-7, which is a seventh example of the configuration of the tone assignment determination unit 411 in FIG.
- the gradation assignment determining unit 411-7 analyzes the histogram of the luminance value of each pixel of the captured image, and sets a plurality of luminance ranges based on the analysis result.
- the gradation assignment determination unit 411-7 includes a histogram analysis unit 551, a threshold comparison processing unit 552, a multi-level luminance range setting unit 553, and a gradation assignment calculation unit 554.
- the histogram analysis unit 551 acquires the image signal supplied from the logarithmic conversion unit 112, and based on the acquired image signal, generates a histogram indicating the luminance value distribution of each pixel of the captured image. Generate and analyze, and supply the analysis result to the threshold comparison processing unit 552.
- the threshold comparison processing unit 552 compares the number of pixels corresponding to each luminance value of the input signal with a predetermined threshold value. In other words, the threshold comparison processing unit 552 extracts a luminance range of a captured image having a certain number of pixels or more.
- the threshold value comparison processing unit 552 supplies the multi-step luminance range setting unit 553 with information indicating the luminance value determined as a result of comparison with the threshold value that the number of pixels is equal to or greater than the threshold value.
- the threshold value may be a value that is obtained experimentally and empirically and is set in advance, or may be a value that can be appropriately set by a user. If the threshold is set too low, most of the information remains, so the resulting image is, for example, a wide dynamics range imaged using the logarithmic conversion image sensor 401 described with reference to FIG. As shown in the display image when the image is not processed using the image processing device 381, The result is an image with no difference (no sharpness). On the other hand, if the threshold value is set too high, information may be missed, and only a part of the luminance range may be displayed clearly!
- the multi-level luminance range setting unit 553 has a larger number of gradation steps than the setting range based on the luminance value supplied from the threshold comparison processing unit 552 and determined that the number of pixels is equal to or greater than the threshold.
- a plurality of luminance ranges to be assigned are set, and the set luminance ranges are supplied to the gradation assignment calculating unit 554.
- the number of luminance ranges set by the multi-level luminance range setting unit 553 is determined by the comparison result supplied from the threshold comparison processing unit 552. For example, an upper limit of the number may be determined in advance.
- the gradation allocation calculation unit 554 acquires the image signal supplied from the logarithmic conversion unit 112, and based on the set luminance range information supplied from the multi-step luminance range setting unit 553, AZD In the AZD conversion by the conversion unit 412, it is determined how many gradation steps are assigned to which luminance range. For example, the gradation assignment calculation unit 554 assigns the number of gradation steps for the AZD conversion of the set luminance area in the same manner as described with reference to FIG. 38, FIG. 39, or FIG. Is set to be larger than the number of gradation steps of AZD conversion assigned to other luminance regions, and the assignment of the gradation step number is determined.
- the histogram analysis unit 551 analyzes the luminance value (analog signal) of each pixel of the captured image.
- the histogram indicating the cloth is analyzed and compared with the threshold value in the threshold value comparison processing unit 552, and the luminance having the number of pixels equal to or larger than the threshold value is extracted in the same image (within one frame).
- a plurality of luminance ranges are set by the multi-level luminance range setting unit 553, and the gradation step number is preferentially allocated within the range by the gradation allocation calculating unit 554.
- the assignment of the number of gradation steps for AZD conversion is calculated, a sufficient luminance gradation width is provided for each set luminance range.
- the gradation assignment determining unit 411-7 of FIG. 54 is used for the image display processing 12 executed by the image processing device 381. Describe To do.
- step S441 the light detection unit 111 of the imaging unit 391 supplies the accumulated charge to the logarithmic conversion unit 112 in synchronization with the control signal supplied from the imaging timing control unit 114. That is, the captured image signal is acquired.
- the logarithmic converter 112 uses the sub-threshold characteristic of the MOSFET to generate an analog electrical signal by converting the charge supplied from the light detector 111 into a voltage value approximately proportional to the logarithm of the number of charges for each pixel. And supplied to the AZD conversion unit 412 and to the gradation assignment determination unit 411-7.
- step S442 the histogram analysis unit 551 of the gradation assignment determination unit 411-7, based on the image signal supplied from the logarithmic conversion unit 112, shows a histogram indicating the luminance distribution of each pixel of the captured image. Is generated and analyzed, and the analysis result is supplied to the threshold value comparison processing unit 552.
- step S443 the threshold comparison processing unit 552 compares the number of pixels corresponding to the luminance value of each input signal with a predetermined threshold based on the analysis result of the histogram supplied from the histogram analysis unit 551.
- the threshold value comparison processing unit 552 supplies information indicating the luminance value determined that the number of pixels is equal to or greater than the threshold value as a result of the comparison with the threshold value to the multi-step luminance range setting unit 553.
- step S444 the multi-step luminance range setting unit 553 is also supplied with the threshold comparison processing unit 552, and based on the luminance value determined to be greater than or equal to the threshold, more gradation steps than outside the setting range. Multiple brightness ranges to which numbers are assigned are set, and the set brightness ranges are supplied to the gradation assignment calculation unit 554.
- step S445 the gradation allocation calculation unit 554, for example, based on the information on the set luminance range supplied from the multi-step luminance range setting unit 553, for example, FIG. 38, FIG. 39, or In the same manner as described with reference to FIG. 45, the gradation assignment of AZD conversion in each area is determined and supplied to the AZD conversion unit 412.
- step S446 the AZD conversion unit 412 converts the analog electrical signal supplied from the logarithmic conversion unit 112 into a digital signal based on the gradation assignment supplied from the gradation assignment calculation unit 465, and This is supplied to the image generation unit 392.
- step S447 the image generation unit 392 performs image processing on the supplied image signal. Then, it is supplied to the output control unit 95 (no gradation conversion is performed).
- step S448 the output control unit 95 controls the output of the image data after the image processing to the image utilization device 83, and the process ends.
- steps S441 to S446 basically the same processing is executed, and an image signal is supplied to the display control unit 94 to control its display.
- a luminance region to which a large number of gradation steps are assigned is set based on the captured image.
- a luminance region to which a large number of gradation steps are assigned may be determined in advance, or may be set by an operation input from the user. For example, the brightness of the pixel corresponding to the image information required by the user in the captured image, such as when the angle of view of the captured image is fixed or the subject is illuminated with a certain amount of illumination.
- a luminance region to which a large number of gradation steps are assigned can be determined in advance. As a result, processing can be simplified and the cost of the apparatus can be reduced.
- FIG. 57 is a block diagram showing a configuration of a gradation assignment determining unit 411-8 that is an eighth example of the structure of the gradation assignment determining unit 411 in FIG.
- the gradation assignment determining unit 411-8 is used when a luminance region to which a large number of gradation steps are assigned is determined in advance.
- the first brightness area brightness range setting unit 581 receives the brightness of the first brightness area from the operation input unit 92. Force that receives the input of the set value of the degree range, or the first stored in the storage unit (not shown)
- the setting value of the first luminance area is acquired, and the setting value of the first luminance range is supplied to the gradation allocation calculating unit 487.
- the second luminance area luminance range setting unit 582 receives the input of the setting value of the luminance range of the second luminance area from the operation input unit 92, or is stored in a storage unit (not shown). The setting value of the second luminance area is acquired, and the setting value of the luminance range of the second luminance area is supplied to the gradation assignment calculating unit 487.
- the third luminance area luminance range setting unit 583 receives the input of the setting value of the luminance range of the third luminance area from the operation input unit 92, or is stored in a storage unit (not shown). The setting value of the luminance region of 3 is acquired, and the setting value of the luminance range of the third luminance region is supplied to the gradation assignment calculating unit 487.
- the gradation allocation calculation unit 487 performs basically the same processing as that in the gradation allocation determination unit 411-3 in Fig. 48, and based on a plurality of set luminance ranges, A In the AZD conversion by the / D conversion unit 412, it is determined how many gradation steps are assigned to which luminance range.
- the gradation allocation calculation unit 487 is supplied from the first luminance region luminance range setting unit 581, the second luminance region luminance range setting unit 582, and the third luminance region luminance range setting unit 583, Based on the information on the luminance range of 1, the luminance range of the second luminance region, and the luminance range of the third luminance region, the number of gradation steps of AZD conversion assigned to each luminance range is determined.
- FIG. 57 shows a first luminance area luminance range setting unit 581, a second luminance area luminance range setting unit 582, and the like for setting the luminance ranges of the first to third luminance areas.
- a third luminance area luminance range setting unit 583 is illustrated, the gradation allocation determining unit 411-8 can further receive the luminance range setting of many luminance areas. It may be possible to provide a luminance area luminance range setting section.
- the gradation assignment determining unit 411-8 of FIG. 57 is used for the image display processing 13 executed by the image processing device 381. I will explain.
- step S471 the operation input unit 92 sets the user power to a set value of a plurality of luminance ranges. And the input set value is supplied to the gradation assignment determining unit 4118.
- step S472 the light detection unit 111 of the imaging unit 391 supplies the accumulated charge to the logarithmic conversion unit 112 in synchronization with the control signal supplied from the imaging timing control unit 114. That is, the captured image signal is acquired.
- the logarithmic converter 112 uses the sub-threshold characteristic of the MOSFET to generate an analog electrical signal by converting the charge supplied from the light detector 111 into a voltage value approximately proportional to the logarithm of the number of charges for each pixel. And supplied to the AZD conversion unit 412 and to the gradation assignment determination unit 411-8.
- step S473 the first luminance region luminance range setting unit 581, the second luminance region luminance range setting unit 582, and the third luminance region luminance range setting unit of the gradation assignment determining unit 411-8 583 acquires the set values of the plurality of luminance ranges supplied from the operation input unit 92 and supplies them to the gradation assignment calculation unit 487.
- step S474 the gradation assignment calculation unit 487, the first luminance region luminance range setting unit 581, the second luminance region luminance range setting unit 582, and the third luminance region luminance range setting unit 583 Based on the set values of the plurality of luminance ranges supplied from, determine the AZD conversion gradation allocation in each area, for example, as described using FIG. 38, FIG. 39, or FIG. This is supplied to the conversion unit 412.
- step S475 the AZD conversion unit 412 converts the analog electric signal supplied from the logarithmic conversion unit 112 into a digital signal based on the gradation assignment supplied from the gradation assignment calculation unit 465, and This is supplied to the image generation unit 392.
- step S476 the image generation unit 392 performs image processing on the supplied image signal (does not perform gradation conversion), and supplies it to the output control unit 95.
- step S477 the output control unit 95 controls the output of the image data after the image processing to the image utilization device 83, and the processing is terminated.
- step S471 to step S475 the same processing is basically executed, and an image signal is supplied to the display control unit 94 to control its display.
- the setting value of the luminance range of each luminance area is input from the operation input unit 92.
- the force described as receiving the input The brightness range of each brightness area is not shown in advance. If it is stored in the storage unit, the setting value of the brightness area is acquired. Well, needless to say.
- the brightness area of the pixel corresponding to the image information required by the user is pre-existing in the wide dynamic range of the captured image data imaged using the logarithmic conversion type imaging device 401. If this is the case, it is possible to display the image information required by the user in a state that can be identified by the user by a simple process by determining a luminance area to which a large number of gradation steps are assigned. Or printing, image recognition, recording, or
- Image data suitable for executing various processes such as transmission can be generated, and the cost of the apparatus can be reduced.
- a histogram indicating the distribution of the luminance values of the pixels included in the captured image is analyzed in a predetermined luminance area, and a certain number of pixels within a predetermined luminance range are determined. It is also possible to extract the luminance that has and to determine the luminance area to which a large number of gradation steps are assigned based on the extraction result.
- FIG. 59 is a block diagram showing a configuration of a tone assignment determination unit 411-9, which is a ninth example of the configuration of the tone assignment determination unit 411 in FIG.
- the gradation assignment determination unit 411-9 extracts luminance having a certain number of pixels or more from a predetermined luminance region by histogram analysis, and assigns a large number of gradation steps based on the extraction result. It is designed to determine the luminance area to be controlled.
- the gradation assignment determining unit 411-9 in FIG. 59 has the first luminance area luminance range setting unit 581 and the second luminance assignment determining unit 411-8 described with reference to FIG.
- a luminance region luminance range setting unit 582, a third luminance region luminance range setting unit 583, and a gradation assignment calculation unit 487 are provided.
- the gradation assignment determination unit 411-9 has been described with reference to FIG. 54 in order to analyze the histogram and compare it with a predetermined threshold value based on the set value of the luminance range output from each.
- the gradation assignment determining unit 411-7 can be equipped with the same processing as the histogram analysis unit 551 and the threshold comparison processing unit 552.
- histogram analysis units 551-1 to 251-3 and threshold comparison processing units 552-1 to 252 3 are provided.
- the histogram analysis units 5 51-1 to 251 The luminance histogram of each pixel of the captured image is analyzed by 3 and compared with the threshold value in the threshold comparison processing units 552-1 to 252-3, and the luminance having the number of pixels equal to or greater than the predetermined threshold value is detected. Extracted.
- the luminance is determined by the number of gradation steps. It is not set to the brightness range that is preferentially assigned.
- the gradation assignment calculation unit 487 preferentially assigns the number of gradation steps to the pixels within the range, and AZD conversion is performed based on the assignment of the gradation steps. It is made to be executed.
- pixels of the image corresponding to the image information required by the user among the images to be captured such as when the angle of view of the image to be captured is fixed or the subject is irradiated with a certain amount of illumination.
- the luminance area of a pixel corresponding to the image information required by the user changes in several patterns over time, even in the same luminance area at daytime Ability to have a lot of information, with little information in the evening, or to have a lot of information in the evening or at night There may be little information in the morning or at noon .
- the gradation assignment determining unit 411-9 of FIG. 59 is used for the image display processing 14 executed by the image processing device 381. Describe To do.
- step S501 the operation input unit 92 receives input of set values of a plurality of luminance ranges with user power, and supplies the input set values to the gradation assignment determining unit 4119.
- step S502 the light detection unit 111 of the imaging unit 391 supplies the accumulated charge to the logarithmic conversion unit 112 in synchronization with the control signal supplied from the imaging timing control unit 114. That is, the captured image signal is acquired.
- the logarithmic converter 112 uses the sub-threshold characteristic of the MOSFET to generate an analog electrical signal by converting the charge supplied from the light detector 111 into a voltage value approximately proportional to the logarithm of the number of charges for each pixel. And supplied to the AZD conversion unit 412 and to the gradation assignment determination unit 411-9.
- step S503 the first luminance area luminance range setting section 581, the second luminance area luminance range setting section 582, and the third luminance area luminance range setting section of the gradation assignment determining section 411-9 583 acquires the set values of the plurality of luminance ranges supplied from the operation input unit 92 and supplies them to the histogram analysis units 551-1 to 251-3, respectively.
- step S504 the histogram analysis units 551-1 to 251-3, based on the image signal supplied from the logarithmic conversion unit 112, as described with reference to FIG. Histograms indicating the distribution of luminance values in the first to third ranges are generated and analyzed, and the analysis results are supplied to the threshold comparison processing units 552-1 to 252-2.
- the threshold comparison processing units 552-1 to 252-3 represent the analysis results of the histograms in the first to third ranges supplied from the histogram analysis units 551-1 to 251-3. Based on this, the number of pixels corresponding to the luminance value of each input signal is compared with a predetermined threshold value.
- the threshold value comparison processing units 552-1 to 252-3 supply the gradation assignment calculation unit 487 with the luminance value determined as a result of comparison with the threshold value that the number of pixels is equal to or greater than the threshold value.
- step S506 the gradation assignment calculating unit 487 selects a luminance range to which a large number of gradation steps are assigned based on the luminance values supplied from the threshold comparison processing units 552-1 to 252-3. Set.
- step S507 the gradation assignment calculating unit 487 determines gradation assignment for AZD conversion in each region, for example, as described with reference to FIG. 38, FIG. 39, or FIG.
- the AZD converter 412 is supplied.
- step S508 the AZD conversion unit 412 converts the analog electrical signal supplied from the logarithmic conversion unit 112 into a digital signal based on the gradation assignment supplied from the gradation assignment calculation unit 465, and This is supplied to the image generation unit 392.
- step S509 the image generation unit 392 performs image processing on the supplied image signal (does not perform gradation conversion), and supplies it to the output control unit 95.
- step S510 the output control unit 95 controls the output of the image data after the image processing to the image utilization device 83, and the process is terminated.
- steps S501 to S508 basically the same processing is executed, and the image signal is supplied to the display control unit 94, converted into gradations that can be processed in the display 82, and output. Be controlled.
- the user needs a wide luminance range in the image data captured by the logarithmic conversion type image sensor 401. Even when image information exists, the gradation in AZD conversion can be efficiently allocated by assigning the number of gradation steps in AZD conversion. Even if the number of steps is limited in advance, the information required by the user (for example, ⁇ , black in ⁇ , objects, ⁇ , brightness, part etc. present in the same image as each other) Appropriate image data can be generated by being displayed in an identifiable state or used for various processes such as printing, image recognition, storage, and transmission.
- Luminance compression is to reduce the number of gradations (the number of gradation steps) in the brightness value of image data.
- the image conversion processing to which the present invention is applied is executed, for example, the luminance region portion that should be recognized by the user is displayed or printed out with a sufficient gray level difference, or various types of In image processing, it is possible to easily determine a threshold value for binarization or to easily detect a predetermined object based on an image.
- the wide dynamic range image data should be used. It is very large data that includes even no information, and it is inefficient to handle such data.
- luminance compression is performed even when the subject to be processed is dispersed in the deviated luminance region, and the subject is viewed. Even when viewed as a normal image, it is observed with the human eye, just as when shooting a narrow luminance range with a conventional image sensor. As a result, an image with no sense of incongruity can be obtained.
- a conventional image recognition apparatus can be applied in the processing process as it is.
- luminance compression is performed in a state where the content of an image is recognizable, so that it is possible to prevent recording or transmitting huge data including unnecessary information. .
- the image processing device 381 shines in a different pattern depending on the state of the target. It is desirable to set the degree range. For example, if a large amount of brightness is assigned to a relatively bright area during the day and an average is assigned to the low brightness, central brightness, and high brightness areas at night, the brightness compression rate can be increased.
- the conventional still image Z moving image compression method such as JPEG and MPEG is used as it is. Can be applied. As a result, a very high compression rate can be obtained.
- the series of processes described above can also be executed by software.
- the software is a computer that is built in dedicated hardware for the program power that constitutes the software, or that can execute various functions by installing various programs, such as a general-purpose personal computer
- the recording medium power is also installed.
- all or part of the image processing device 81 described with reference to FIG. 2 may be a personal computer 301 as shown in FIG. Consists of.
- a CPU (Central Processing Unit) 311 is a program stored in a ROM (Read Only Memory) 312 or a program loaded from a storage unit 318 to a RAM (Random Access Memory) 313. Therefore, various processes are executed.
- the RAM 313 also appropriately stores data necessary for the CPU 311 to execute various processes.
- the CPU 311, ROM 312, and RAM 313 are connected to each other via a bus 314.
- An input / output interface 315 is also connected to the bus 314.
- the input / output interface 315 includes an input unit 316 including a keyboard and a mouse, a display An output unit 317 including a play and a speaker, a storage unit 318 including a hard disk, and a communication unit 319 including a modem and a terminal adapter are connected.
- the communication unit 319 performs communication processing via a network including the Internet.
- a drive 320 is also connected to the input / output interface 315 as necessary, and a magnetic disk 331, an optical disk 332, a magneto-optical disk 333, or a semiconductor memory 334, etc. are appropriately mounted to read the power.
- the recorded computer program is installed in the storage unit 318 as necessary.
- a program that configures the software is installed in a dedicated hardware, or various programs can be installed by installing various programs.
- a general-purpose personal computer or the like can be installed on a network or a recording medium.
- this recording medium is distributed to supply a program to the user separately from the main body of the apparatus, and includes a magnetic disk 331 (including a floppy disk) storing the program, Optical disk 332 (including compact disk-read only memory (CD-ROM), DVD (digital versatile disk)), magneto-optical disk 333 (including MD (mini-disk) (trademark)), or semiconductor memory 334 It is composed of a ROM 312 that stores a program and is stored in a storage unit 318, which is supplied to the user in a state of being pre-installed in the apparatus main body just by being constituted by a knock medium.
- a magnetic disk 331 including a floppy disk
- Optical disk 332 including compact disk-read only memory (CD-ROM), DVD (digital versatile disk)
- magneto-optical disk 333 including MD (mini-disk) (trademark)
- semiconductor memory 334 It is composed of a ROM 312 that stores a program and is stored in a storage unit 318, which
- the steps for describing the program recorded on the recording medium are not necessarily processed in time series in the order described, but are necessarily processed in time series. It includes processing executed in parallel or individually.
- the present invention is applicable even when the processing power executed by one device is realized by a plurality of devices.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
- Studio Devices (AREA)
- Picture Signal Circuits (AREA)
- Facsimile Image Signal Circuits (AREA)
Abstract
Description
Claims
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP06729138A EP1871093A4 (en) | 2005-03-15 | 2006-03-15 | IMAGE PROCESSOR, IMAGE PROCESSING METHOD AND SYSTEM, PROGRAM, AND RECORDING MEDIUM |
CN2006800086218A CN101142810B (zh) | 2005-03-15 | 2006-03-15 | 图像处理装置和图像处理方法、图像处理系统 |
US11/908,955 US20090051794A1 (en) | 2005-03-15 | 2006-03-15 | Image processing apparatus, image processing method, image processing system, program and recording medium |
JP2007508177A JP5182555B2 (ja) | 2005-03-15 | 2006-03-15 | 画像処理装置および画像処理方法、画像処理システム、プログラム、並びに、記録媒体 |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2005-072840 | 2005-03-15 | ||
JP2005072840 | 2005-03-15 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2006098360A1 true WO2006098360A1 (ja) | 2006-09-21 |
Family
ID=36991702
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2006/305117 WO2006098360A1 (ja) | 2005-03-15 | 2006-03-15 | 画像処理装置および画像処理方法、画像処理システム、プログラム、並びに、記録媒体 |
Country Status (5)
Country | Link |
---|---|
US (1) | US20090051794A1 (ja) |
EP (1) | EP1871093A4 (ja) |
JP (1) | JP5182555B2 (ja) |
CN (1) | CN101142810B (ja) |
WO (1) | WO2006098360A1 (ja) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2009177309A (ja) * | 2008-01-22 | 2009-08-06 | Fuji Heavy Ind Ltd | 撮像手段の調整装置および車外監視装置 |
WO2013153848A1 (ja) * | 2012-04-13 | 2013-10-17 | 株式会社デンソー | 画像処理装置及び方法 |
JP2014518030A (ja) * | 2011-04-28 | 2014-07-24 | コーニンクレッカ フィリップス エヌ ヴェ | Hdr画像を符号化及び復号化するための装置及び方法 |
WO2014181743A1 (ja) * | 2013-05-07 | 2014-11-13 | 株式会社デンソー | 画像処理装置及び画像処理方法 |
JP2018037061A (ja) * | 2016-09-01 | 2018-03-08 | 三星電子株式会社Samsung Electronics Co.,Ltd. | 自律走行車両のためのビジョンセンサの制御方法及び装置 |
Families Citing this family (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2008020544A1 (en) * | 2006-08-18 | 2008-02-21 | Nec Corporation | Vehicle detection device, vehicle detection method, and vehicle detection program |
JP4961982B2 (ja) * | 2006-12-07 | 2012-06-27 | ソニー株式会社 | 固体撮像装置、固体撮像装置の駆動方法および撮像装置 |
JP5439197B2 (ja) * | 2010-01-15 | 2014-03-12 | オリンパスイメージング株式会社 | 撮像装置、撮像方法、および、撮像プログラム |
CN101867721B (zh) * | 2010-04-15 | 2012-07-04 | 青岛海信网络科技股份有限公司 | 宽动态图像的实现方法、实现装置及成像设备 |
JP2012044382A (ja) * | 2010-08-18 | 2012-03-01 | Sony Corp | 映像データ処理装置、コントラスト補正方法 |
CN102497490B (zh) * | 2011-12-16 | 2014-08-13 | 上海富瀚微电子有限公司 | 实现图像高动态范围压缩的系统及其方法 |
CN103182985B (zh) * | 2011-12-27 | 2016-08-17 | 大陆汽车电子(长春)有限公司 | 利用图像处理装置的车辆安全装置和车辆安全控制方法 |
KR101889932B1 (ko) * | 2012-07-25 | 2018-09-28 | 삼성전자주식회사 | 촬영 장치 및 이에 적용되는 촬영 방법 |
JPWO2015049899A1 (ja) * | 2013-10-01 | 2017-03-09 | オリンパス株式会社 | 画像表示装置および画像表示方法 |
JP6414386B2 (ja) * | 2014-03-20 | 2018-10-31 | 株式会社島津製作所 | 画像処理装置および画像処理プログラム |
CN107211096B (zh) * | 2015-01-22 | 2020-03-03 | 三菱电机株式会社 | 摄像装置和方法以及记录介质 |
JP6390512B2 (ja) * | 2015-05-21 | 2018-09-19 | 株式会社デンソー | 車載カメラ装置 |
JP6700731B2 (ja) * | 2015-11-13 | 2020-05-27 | キヤノン株式会社 | 投影装置および投影システム |
JP6789877B2 (ja) * | 2017-04-28 | 2020-11-25 | キヤノン株式会社 | 情報処理装置、画像処理システム、制御システム及び情報処理方法 |
JP7069751B2 (ja) * | 2018-01-29 | 2022-05-18 | カシオ計算機株式会社 | 印刷装置 |
CN108805834A (zh) * | 2018-05-28 | 2018-11-13 | 电子科技大学 | 一种内窥镜自动调光的图像处理算法 |
US10991082B2 (en) * | 2019-08-22 | 2021-04-27 | Mikhail Dichenko | Determining an optimality coefficient of an image |
US11341621B2 (en) * | 2020-03-04 | 2022-05-24 | GM Global Technology Operations LLC | Enhanced imaging system for a motor vehicle |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH09224174A (ja) * | 1996-01-18 | 1997-08-26 | Samsung Electron Co Ltd | 積分ルックアップテーブルを用いる適応型ガンマ補正装置 |
JP2001245214A (ja) * | 2000-02-28 | 2001-09-07 | Minolta Co Ltd | 固体撮像装置 |
JP2002077619A (ja) * | 2000-09-01 | 2002-03-15 | Rohm Co Ltd | 画像処理装置 |
JP2002132243A (ja) * | 2000-10-27 | 2002-05-09 | Sony Corp | 画像処理装置および方法、並びに記録媒体 |
JP2002217733A (ja) * | 2001-01-24 | 2002-08-02 | Fujitsu Kiden Ltd | A/d変換器、映像表示装置及びa/d変換方法 |
JP2004221644A (ja) * | 2003-01-09 | 2004-08-05 | Sony Corp | 画像処理装置および方法、記録媒体、並びにプログラム |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5249241A (en) * | 1991-07-09 | 1993-09-28 | The United States Of America As Represented By The Secretary Of The Air Force | Real-time automated scene display for infrared cameras |
DE4209536C3 (de) * | 1992-03-24 | 2000-10-05 | Stuttgart Mikroelektronik | Bildzelle für einen Bildaufnehmer-Chip |
EP1152604A1 (en) * | 2000-04-24 | 2001-11-07 | Pulnix America, Inc. | Video glare reduction |
TW565684B (en) * | 2001-02-14 | 2003-12-11 | Honda Motor Co Ltd | Welding state monitoring device |
GB0110748D0 (en) * | 2001-05-02 | 2001-06-27 | Apical Ltd | Image enhancement methods and apparatus therefor |
-
2006
- 2006-03-15 WO PCT/JP2006/305117 patent/WO2006098360A1/ja active Application Filing
- 2006-03-15 CN CN2006800086218A patent/CN101142810B/zh not_active Expired - Fee Related
- 2006-03-15 EP EP06729138A patent/EP1871093A4/en not_active Withdrawn
- 2006-03-15 JP JP2007508177A patent/JP5182555B2/ja not_active Expired - Fee Related
- 2006-03-15 US US11/908,955 patent/US20090051794A1/en not_active Abandoned
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH09224174A (ja) * | 1996-01-18 | 1997-08-26 | Samsung Electron Co Ltd | 積分ルックアップテーブルを用いる適応型ガンマ補正装置 |
JP2001245214A (ja) * | 2000-02-28 | 2001-09-07 | Minolta Co Ltd | 固体撮像装置 |
JP2002077619A (ja) * | 2000-09-01 | 2002-03-15 | Rohm Co Ltd | 画像処理装置 |
JP2002132243A (ja) * | 2000-10-27 | 2002-05-09 | Sony Corp | 画像処理装置および方法、並びに記録媒体 |
JP2002217733A (ja) * | 2001-01-24 | 2002-08-02 | Fujitsu Kiden Ltd | A/d変換器、映像表示装置及びa/d変換方法 |
JP2004221644A (ja) * | 2003-01-09 | 2004-08-05 | Sony Corp | 画像処理装置および方法、記録媒体、並びにプログラム |
Non-Patent Citations (1)
Title |
---|
See also references of EP1871093A4 * |
Cited By (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2009177309A (ja) * | 2008-01-22 | 2009-08-06 | Fuji Heavy Ind Ltd | 撮像手段の調整装置および車外監視装置 |
JP2014518030A (ja) * | 2011-04-28 | 2014-07-24 | コーニンクレッカ フィリップス エヌ ヴェ | Hdr画像を符号化及び復号化するための装置及び方法 |
JP2014519730A (ja) * | 2011-04-28 | 2014-08-14 | コーニンクレッカ フィリップス エヌ ヴェ | Hdr画像を符号化及び復号するための装置及び方法 |
WO2013153848A1 (ja) * | 2012-04-13 | 2013-10-17 | 株式会社デンソー | 画像処理装置及び方法 |
JP2013222261A (ja) * | 2012-04-13 | 2013-10-28 | Denso Corp | 画像処理装置 |
US9396527B2 (en) | 2012-04-13 | 2016-07-19 | Denso Corporation | Image processing device and method |
WO2014181743A1 (ja) * | 2013-05-07 | 2014-11-13 | 株式会社デンソー | 画像処理装置及び画像処理方法 |
JP2014219790A (ja) * | 2013-05-07 | 2014-11-20 | 株式会社デンソー | 画像処理装置及び画像処理方法 |
US9800881B2 (en) | 2013-05-07 | 2017-10-24 | Denso Corporation | Image processing apparatus and image processing method |
JP2018037061A (ja) * | 2016-09-01 | 2018-03-08 | 三星電子株式会社Samsung Electronics Co.,Ltd. | 自律走行車両のためのビジョンセンサの制御方法及び装置 |
Also Published As
Publication number | Publication date |
---|---|
CN101142810B (zh) | 2010-09-01 |
US20090051794A1 (en) | 2009-02-26 |
CN101142810A (zh) | 2008-03-12 |
EP1871093A1 (en) | 2007-12-26 |
JP5182555B2 (ja) | 2013-04-17 |
EP1871093A4 (en) | 2009-09-02 |
JPWO2006098360A1 (ja) | 2008-08-28 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP5182555B2 (ja) | 画像処理装置および画像処理方法、画像処理システム、プログラム、並びに、記録媒体 | |
JP4835593B2 (ja) | 画像処理装置および画像処理方法、プログラム、並びに、記録媒体 | |
JP4985394B2 (ja) | 画像処理装置および方法、プログラム、並びに記録媒体 | |
JP4678487B2 (ja) | 画像処理システム、画像処理装置および方法、記録媒体、並びにプログラム | |
JP4766302B2 (ja) | 画像処理装置および方法、記録媒体、並びにプログラム | |
KR101353052B1 (ko) | 교통표지판 인식을 위한 차량용 영상인식시스템 | |
JP6740866B2 (ja) | 画像出力装置 | |
WO2006109398A1 (ja) | 画像処理装置および方法、プログラム、並びに記録媒体 | |
JP2006238412A (ja) | 輝度レベル変換装置、輝度レベル変換方法、固体撮像装置、輝度レベル変換プログラム、および記録媒体 | |
US10091422B2 (en) | Image processing device and recording medium | |
WO2007074659A1 (ja) | 映像信号処理方法および映像信号処理装置 | |
JP6531542B2 (ja) | 画像処理システム、画像処理装置、撮像装置、画像処理方法、プログラムおよび記録媒体 | |
JP2006237851A (ja) | 画像入力装置 | |
WO2020090176A1 (ja) | 画像処理装置および画像処理方法 | |
JP2016173777A (ja) | 画像処理装置 | |
CN112822413A (zh) | 拍摄预览方法、装置、终端和计算机可读存储介质 | |
JP2006259899A (ja) | 画像処理システム、画像処理装置および方法、記録媒体、並びにプログラム | |
JP4304610B2 (ja) | カメラ式車両感知器における画面の輝度調整方法及び装置 | |
CN101142811A (zh) | 图像处理装置和图像处理方法、程序以及记录介质 | |
JP2007249568A (ja) | 画像処理装置および方法、記録媒体、並びに、プログラム | |
JP7547823B2 (ja) | 撮像制御装置、撮像制御方法及びプログラム | |
JP2009044451A (ja) | 画像処理装置、車載用画像処理装置および車載用画像処理方法 | |
JP2023130999A (ja) | 撮像システム、撮像システム制御方法、及びプログラム | |
JPH11337998A (ja) | 画像処理装置 | |
JP2023154475A (ja) | 撮像装置およびその制御方法 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 200680008621.8 Country of ref document: CN |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
DPE1 | Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101) | ||
WWE | Wipo information: entry into national phase |
Ref document number: 11908955 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2006729138 Country of ref document: EP |
|
NENP | Non-entry into the national phase |
Ref country code: RU |
|
WWP | Wipo information: published in national office |
Ref document number: 2006729138 Country of ref document: EP |
|
ENP | Entry into the national phase |
Ref document number: 2007508177 Country of ref document: JP Kind code of ref document: A |