JP2012138723A - Imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus which recognizes and displays a subject intended by a user, while keeping the visibility of a displayed image in imaging using an imaging element having nonlinear characteristics.SOLUTION: The imaging apparatus comprises: an imaging element 101 which photoelectrically converts incident light and outputs the result as an image signal and has nonlinear input/output characteristics; characteristic change means 102 which converts the image signal to two or more image signals having different input/output characteristics and outputs the results; camera signal processing means 103 which performs signal processing needed as a camera device on the image signal; recognition means 105 which detects a signal meeting a specific condition with respect to the image signals recognition information and generates recognition information; information addition means 106 for adding the recognition information to the image signals; and output means 104 which performs signal processing needed to output the image signals.

Description

  The present invention relates to an imaging apparatus.

  As a background art of this technical field, there is JP 2009-17157 A (Patent Document 1). According to the publication, “the area setting unit 131 indicates that there are obstacle areas and lanes that are assumed to be present in the processed image based on the position of the object in front of the host vehicle detected by the radar device 112. Two divided areas of the assumed road surface area are set, and the image generation unit 132 adjusts the dynamic range and resolution of the front image captured by the camera 111 for each divided area based on the brightness in front of the vehicle. The lane detection unit 141 detects a lane based on the image in the road area of the processed image, and the vehicle detection unit 142 moves forward based on the image in the obstacle area of the processed image. The present invention can be applied to an in-vehicle image processing apparatus ”(see abstract).

JP 2009-17157 A

  In recent years, image pickup devices having nonlinear input / output characteristics have become widespread in order to suppress saturation of pixel signals. By using the image sensor, it has become possible to capture an image having a wider dynamic range than the human eye.

  By the way, in general image recognition processing, for example, the correlation between pixel signals of a reference image and a processed image is compared so that abnormality detection is performed by comparing a background image calculated by time averaging with an image to be recognized as a reference image. By doing so, the target information is recognized.

  However, since the image sensor realizes nonlinear input / output characteristics by logarithmic compression conversion or the like, the image signal has a non-constant gradation. For this reason, when conventional image recognition processing is performed on an image captured by an image sensor having nonlinear input / output characteristics, there is a region where the gradation is compressed depending on the signal level, so that the recognition accuracy decreases. Further, since the contrast is weak and the color is light in the area where the gradation is compressed, the visibility of the output image is also lowered.

  In Patent Document 1, “for each set region, the luminance resolution and the dynamic range are adjusted by extracting a predetermined range of bits from the bit string representing the luminance value of the image from which the processed image is generated. Is described.

  However, regarding the method of providing the recognized information to the user, there is a room for improvement because the method related to the driving support device is mentioned and the point of displaying as an image is not shown.

  Therefore, the present invention performs processing using an image signal having a dynamic range suitable for each of recognition and display, adds recognition information to the image, and generates an output image. Provided is an imaging device that displays a recognized information in an easy-to-understand manner.

  In order to solve the above object, the configuration described in the claims is adopted.

  ADVANTAGE OF THE INVENTION According to this invention, the imaging device which displays that it recognized for the user clearly can be provided.

  Problems, configurations, and effects other than those described above will be clarified by the following description of embodiments.

Basic configuration diagram of Embodiment 1 of the present invention The figure explaining the input-output characteristic of the image pick-up element 101 The figure explaining the dynamic range of the signal which the characteristic change means 102 outputs The figure explaining the dynamic range of the signal which the characteristic change means 102 outputs The figure explaining the dynamic range of the signal which the characteristic change means 102 outputs The figure explaining the dynamic range of the signal which the characteristic change means 102 outputs The figure explaining the recognition process result of the recognition means 105 The figure explaining the display image which the camera signal processing means 103 outputs The figure explaining the method of adding recognition information to the image for a display The figure explaining the method of adding recognition information to the image for a display

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  In the present embodiment, a description will be given using an example in which a backlight scene is captured by a surveillance camera that is an example of an imaging apparatus. This embodiment is particularly effective when shooting a backlight scene in a general use scene of a surveillance camera.

  Although the present embodiment will be described using an example of shooting with a monitoring camera, it can also be applied to a case where a recognition result is displayed in a specific scene other than the monitoring camera.

  Example 1 of the present invention will be described in detail. FIG. 1 is a basic configuration diagram in Embodiment 1 of the present invention.

  An image sensor 101 that photoelectrically converts incident light and outputs it as an image signal, and its input / output characteristics are nonlinear, and characteristic changing means 102 that converts the image signal into two or more image signals having different input / output characteristics and outputs the converted image signal. A camera signal processing unit 103 that performs signal processing necessary as a camera device on the image signal, a recognition unit 105 that detects a signal that satisfies a specific condition for the image signal, and generates recognition information; The information adding means 106 for adding to the image signal and the output means 104 for performing signal processing necessary for outputting the image signal are provided.

  A characteristic changing unit 102 captures an image signal having a wide dynamic range and a linear characteristic by a method described later for each processing such as subsequent display or recognition of the image signal having a wide dynamic range output by the image sensor 101 having nonlinear characteristics. Two image signals having different input / output characteristics of an image signal having uncompressed gradation having characteristics close to those obtained when the image is taken by the element are output. With this configuration, an image signal suitable for each subsequent process can be input. The recognition unit 105 generates recognition information by detecting an image signal in which a desired subject is captured with respect to an image signal having a wide dynamic range output from the previous characteristic changing unit 102. The camera signal processing means 103 performs noise removal, gamma correction, contour enhancement, white balance processing, filter on an image signal having characteristics close to those obtained when imaged with the linear characteristic imaging device output from the characteristic changing means 102. Predetermined image signal processing such as processing is performed to generate a display image. Further, the information adding means 106 adds the recognition information to the display image output by the camera signal processing means 103, and outputs the output image by performing output interface processing described later by the output means 104. By adopting such a configuration, a display image close to the impression of the appearance similar to the case of shooting with a camera having an image sensor with a linear characteristic in the past, and a subject in an area that is overexposed on the display image It is possible to provide the user with both recognition results.

  Details of the output means 104 will be described. The output interface process is a process of converting to a signal format of an output device such as a TV or a storage, for example, converting to a video output of NTSC or PAL, for example, converting to an HDMI signal, for example, network transmission Therefore, the signal is converted into a predetermined signal. Note that the output unit 104 may be configured to include predetermined image signal processing such as gain control for level adjustment, filter processing, and compression processing by encoding.

  FIG. 2 is a diagram illustrating the input / output characteristics of the image sensor 101 (FIG. 1). The horizontal axis indicates the luminance level of incident light, and the vertical axis indicates the output signal level of the image sensor. FIG. 2A shows a case where the input / output characteristics of the image sensor are linear characteristics, and FIG. 2B shows a case where the input / output characteristics of the image sensor are nonlinear. By providing the non-linear characteristic shown in FIG. 2B, the pixel saturation point is shifted to the high luminance side while maintaining the gradation of the low luminance region as compared with the case of the linear characteristic shown in FIG. Therefore, an image signal with a wide dynamic range can be obtained. In FIG. 2, the gradation of the high brightness area is compressed, but the area to be compressed is not limited to the high brightness area, and the same effect can be obtained even in other areas.

  FIG. 3 is a diagram for explaining a method for converting a signal output from the characteristic changing unit 102 (FIG. 1). FIG. 3A shows the input / output characteristics of the image sensor having non-linear characteristics. The horizontal axis represents the luminance level of the incident light, and the vertical axis represents the output signal level of the image sensor. The input signal level of 1) is shown. A range a indicates a range including all signals of the input signal, a range b indicates a range of a signal in which gradation is hardly compressed, and a range c is a level compared with an image sensor having a linear characteristic. The range of the signal whose key is compressed is shown. Further, the ranges b and c in FIG. 3B and the ranges a and c in FIG. 3C correspond to the ranges a, b, and c in FIG. 3A, respectively.

  FIG. 3B and FIG. 3C show an example of a signal conversion method of the characteristic changing unit 102, the horizontal axis shows the input signal level of the characteristic changing unit 102, and the vertical axis shows the characteristic changing unit. 102 shows the output signal level.

  As shown in FIG. 3B, since the gradation is compressed, the signal in the range c with poor visibility is discarded, and the gradation of the signal in the range b is expanded, thereby improving the visibility. The signal in the range c is not completely cut off, and the same effect can be obtained by changing the characteristics so that the gradation is further compressed.

  Since the signal level in the range c can be estimated from the input / output characteristics and signal level of the image sensor 101 (FIG. 1) when the linear image sensor is used, the signal approximates the linear characteristic as shown in FIG. By converting in this way, it is possible to suppress a decrease in recognition accuracy due to gradation compression.

  FIG. 3 (d) also shows an example of the signal conversion method of the characteristic changing unit 102. The range d is a 12-bit input signal, the range e is a 10-bit output signal, and the range f is a low-order 2-bit noise component. Signals not related to processing are shown. As shown in FIG. 3 (d), the range f signal is cut off from the input signal in the range d and the signal in the range e is output, so that the recognition accuracy is obtained even when the conversion shown in FIG. 3 (c) is performed. The processing amount can be reduced without incurring a decrease in.

  Since the image signal in the range a shown in FIG. 3A includes the range c in which the gradation is compressed, the visibility is poor, but it is suitable for recognition processing because of its wide dynamic range. Further, since the image signal in the range b has a narrow dynamic range but the gradation is hardly compressed, by converting the characteristics as shown in FIG. 3B, the image signal has good visibility and is suitable for display processing. It becomes.

  By performing the processing as described above, the characteristic changing unit 102 (FIG. 1) is suitable for the display processing as shown in FIG. 3B for the camera signal processing unit 103 (FIG. 1) for generating a display image. An image signal suitable for recognition processing as shown in FIGS. 3C and 3D is output to the recognition means 105 (FIG. 1) that outputs an image signal and performs recognition processing. It is possible to input an image signal having characteristics suitable for.

  FIG. 4 is a diagram for explaining the recognition processing result of the recognition means 105 (FIG. 1). FIG. 4 shows an image signal input to the recognizing means 105 when a backlit scene where a dark tunnel and a bright outside are mixed in the image is photographed. A region indicated by hatching indicates a low luminance region inside the tunnel, and a region A indicates a high luminance region outside the tunnel. The car 1 indicates a car that exists in a dark tunnel, and the car 2 indicates a car that exists in a bright outside. Since the image signal having a wide dynamic range includes both a low luminance region signal and a high luminance region signal, the recognition means 105 (FIG. 1) recognizes both the low luminance region vehicle 1 and the high luminance region vehicle 2. can do. However, when the user confirms the output image through the display device, since the dynamic range of the display device is fixed, all image signals having a wide dynamic range cannot be displayed. For this reason, since there is a region where the gradation is compressed as compared with a case where an image is taken with an imaging element having linear characteristics, visibility may be deteriorated. In order to avoid such an adverse effect, in this embodiment, the position coordinate of the subject recognized by the recognition unit 105 (FIG. 1) is output, and the display image is generated by the camera signal processing unit in another block.

  FIG. 5 is a diagram for explaining a display image output by the camera signal processing means 103 (FIG. 1). FIG. 5 shows a display image when the same backlight scene as in FIG. 4 is taken. A display image is generated from an image signal as shown by the range b in FIG. 3, and a high luminance area such as the area A is blown out, but the gradation is not compressed. Even when an image is confirmed, the visibility can be kept good.

  FIG. 6 is a diagram for explaining a method of adding recognition information to a display image. 6A and 6B show examples of output images when the same backlight scene as that in FIG. 4 is taken.

  FIG. 6A shows a detection frame superimposed on a position on the display image corresponding to the position coordinates of the car 1 and the car 2. In addition, masking by colors or characters may be displayed so as to indicate that the subject has been recognized. By performing such processing, the result of recognizing the subject can be easily shown to the user.

  FIG. 6B shows a specific pattern image superimposed on a position on the display image corresponding to the position coordinates of the car 1 and the car 2. The specific pattern image only needs to represent the characteristics of the recognized subject.For example, only the reference image of the subject used in the recognition processing such as the pattern matching method or the image area including the recognized subject is suitable for display. May be converted and displayed. By performing such processing, it is possible to specifically provide the user with features such as the size, color, and orientation of the recognized subject.

  With such a configuration, the visibility of the output image can be improved even in a scene in which the subject cannot be confirmed by the user's own eyes because the target subject exists in an overexposed image region such as the region A. It is possible to confirm by displaying the recognition result of the subject on the output image while maintaining the above.

  In addition, this invention is not limited to an above-described Example, Various modifications are included. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described. In addition, a part of the configuration of a certain embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of a certain embodiment. Further, it is possible to add, delete, and replace other configurations for a part of the configuration of each embodiment.

  In addition, each of the above-described configurations may be configured such that some or all of them are configured by hardware, or are implemented by executing a program by a processor. Further, the control lines and information lines indicate what is considered necessary for the explanation, and not all the control lines and information lines on the product are necessarily shown. Actually, it may be considered that almost all the components are connected to each other.

101 Imaging device
102 Characteristic change means
103 Camera signal processing means
104 Output means
105 Recognition means
106 Information addition means

Claims (3)

An image sensor that photoelectrically converts incident light from a subject and outputs it as an image signal, and its input / output characteristics are nonlinear;
Characteristic changing means for converting the image signal from the image sensor into two or more image signals having different input / output characteristics;
Camera signal processing means for generating a display image by performing predetermined image signal processing on the image signal from the characteristic changing means;
Recognizing means for generating recognition information by detecting an image signal including a desired subject in the image signal from the characteristic changing means;
Information adding means for adding the recognition information generated from the recognition means to the display image generated by the camera signal processing means;
An imaging apparatus comprising: 2. The imaging apparatus according to claim 1, wherein the information adding unit displays a detection frame, masking, characters, or a combination thereof in an area including a subject on a display image based on recognition information. Imaging device. The imaging apparatus according to claim 1, wherein the information adding unit displays an image representing a subject superimposed on a region including the subject on the display image based on the recognition information.

Images