JP2012222374A - On-vehicle camera system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve distortion of a captured image which occurs when a CMOS imaging element of a rolling shutter system is applied to an on-vehicle camera system.SOLUTION: The on-vehicle camera system is provided with means which acquires speed information of a vehicle from a control computer of the vehicle, predicts the distortion caused by a rolling shutter on the basis of the acquired speed information, and corrects the acquired image from the CMOS imaging element using the prediction result.

Description

  The present invention relates to an in-vehicle camera system.

  As background art of the present invention, for example, there is JP-A-2003-250081 (Patent Document 1). In this publication, “[Problem] To provide an in-vehicle camera capable of outputting a clear image signal to an in-vehicle monitor even at night when the illuminance of the subject is below the minimum illuminance. [Solution] In the in-vehicle camera of the present invention. The image signal is bright enough at night when the illuminance of the subject is below the minimum illuminance by having the function of increasing the charge accumulation time of the CCD device by reducing the number of frames to the number of frames that can ensure the minimum illuminance. It can also be output to an in-vehicle monitor, and by having the function of reducing and correcting the number of frames within a range where real-time performance can be ensured, a jerky video can be obtained ”(see summary). ).

JP 2003-250081 A

  For the purpose of a driver confirming safety in an automobile, a camera for photographing a surrounding situation using a solid-state image sensor is used. In a conventional in-vehicle camera, a CCD image pickup device having excellent sensitivity is generally used as a solid-state image pickup device as disclosed in Patent Document 1.

  However, in recent years, CMOS image sensor technology has improved, and demands for increasing the number of pixels in in-vehicle cameras have increased. Therefore, a CMOS image sensor suitable for structurally reading out signals from many pixels at high speed is provided. It has come to be applied to in-vehicle cameras.

  However, in many cases, a CMOS image sensor has a rolling shutter structure that gives priority to sensitivity, and if a rolling shutter type CMOS image sensor is applied to an in-vehicle camera that takes an image while moving at high speed, the captured image is distorted. is there. An object of the present invention is to improve the distortion of a captured image that occurs when a rolling shutter type CMOS image sensor is applied to an in-vehicle camera system.

  The above-described problems are solved or improved by the invention described in the claims. For example, there is provided means for acquiring vehicle speed information from a vehicle control computer, predicting distortion generated by the rolling shutter from the acquired speed information, and correcting the acquired image from the CMOS image sensor using the prediction result. .

  ADVANTAGE OF THE INVENTION According to this invention, in the vehicle-mounted camera system using a rolling shutter type CMOS image sensor, the distortion of the acquired image generated by the change of the imaging position by driving | running | working of a vehicle can be improved.

  Problems, configurations, and effects other than those described above will be clarified by the following description of embodiments.

It is an example (Example 1) of the system block diagram of a vehicle-mounted camera. It is a figure explaining the mode of correction of distortion generated in a photographed image. It is an example (Example 2) which applied the vehicle-mounted camera to the vehicle-mounted recognition system.

  FIG. 1 shows an example of a vehicle camera system configuration according to the present invention. In the figure, 101 is a lens, 102 is a CMOS image sensor, 103 is camera signal processing means, 104 is coordinate transformation processing means, 105 is a camera control microcomputer, 106 is a communication port, 107 is a video signal output terminal, and 108 is an automobile control microcomputer. 109 denotes a sensing means group, and 110 denotes a display processing means. Moreover, 111 enclosed with the broken line has shown the vehicle-mounted camera in this invention.

  A lens 101 forms an image of light coming from an object to be photographed on an imaging surface of a CMOS image sensor 102.

  The CMOS image sensor 102 has a photoelectric conversion element (hereinafter generally referred to as a pixel according to the expression) arranged on an array on the imaging surface, and converts the light imaged on the imaging surface into electrical energy. Convert to The electrical energy accumulated in the pixel is proportional to the exposure amount of the pixel, and the accumulated electrical energy is sequentially read out from the CMOS image sensor as an imaging signal and supplied to the camera signal processing means 103.

  The camera signal processing unit 103 performs a spatial filter process (generally two-dimensional) on the imaging signal and a gamma process that matches the characteristics of the display device, and includes a luminance signal and a color difference signal (when the color difference signal is a color camera). Generate a video signal.

  The coordinate conversion unit 104 corrects image distortion caused by the rolling shutter in the luminance signal Yin (x, y) and the color difference signal Cin (x, y) that are input video signals, and generates a new luminance signal Yout (x, y). ) And a color difference signal Cout (x, y) are generated and output as a video signal. The luminance signal Yout (x, y) and the color difference signal Cout (x, y) are specified for each coordinate (x, y) of the luminance signal Yout (x, y) and the color difference signal Cout (x, y). It is generated with reference to the values of the luminance signal Yin (x, y) and the color difference signal Cin (x, y) at the reference coordinates (fx [v], fy [v]). At this time, the value of the reference coordinate does not necessarily have to be an integer, and when it has a value less than a decimal, the luminance signal Yin (x, y) and the color difference signal Cin ( The luminance signal Yout (x, y) and the color difference signal Cout (x, y) are acquired by generating an interpolation signal from the value of x, y). Here, since the reference coordinates (fx [v], fy [v]) are generally expressed by a function having the moving speed of the camera as a variable, the reference coordinates (fx [v], fy [v]) take a method that changes according to the moving speed of the camera. And Here, the reference coordinates (fx [v], fy [v]) are ideally calculated in real time using the moving speed of the camera as a variable, but are calculated using a database calculated in advance. May be simplified.

  A camera control microcomputer 105 is a camera control unit that controls a CMOS image sensor 102, a camera signal processing unit 103, and a coordinate conversion unit 104. Specific control contents in the camera control microcomputer include Auto Exposure control, White Balance control, and reference coordinate control in the coordinate conversion means. Of these, Auto Exposure control acquires the luminance level of the video signal from the camera signal processing means, and adjusts the exposure time of the CMOS image sensor and the gain of the video signal. In addition, the white balance control acquires a white signal component from the camera signal processing means and adjusts the gain of the RGB component so that a white subject looks white. The reference coordinate control in the coordinate conversion means changes the reference coordinates according to the moving speed of the camera.

  The communication port 106 is a communication port for communicating with the automobile control microcomputer 108, and the camera control microcomputer acquires the moving speed information of the automobile from the automobile control microcomputer via the communication port.

  A video signal output terminal 107 is a terminal for outputting a video signal photographed by a vehicle-mounted camera 111. 110 display processing means are connected to the video signal output terminal, and images taken by the in-vehicle camera can be displayed on the display processing means.

  Further, the automobile control microcomputer 108 detects the operating state of each part of the automobile through 109 sensing means connected to the automobile control microcomputer, and controls the operation of the automobile based on the detected information.

  FIG. 2 is a diagram for explaining a state of correcting distortion generated in a photographed image, and particularly shows a case where the traveling direction of the automobile is photographed. In the figure, 201 is an angle of view at the start of photographing, 202 is an angle of view at the end of photographing, and 203 is an example of a distortion-corrected image.

  In a rolling shutter type CMOS image sensor, pixels are exposed in line sequential order. Therefore, when a camera using the image sensor is installed on a moving body and imaging is performed, the angle of view at the start and end of shooting is different. It will change. For example, a camera installed in the moving direction of the moving body has an angle of view as indicated by 201 in FIG. 2 at the initial stage of shooting a certain frame image, whereas at the end stage of the frame image, 2 changes to the angle of view as shown at 202 in FIG. Here, assuming that the frame image is taken in line sequential from the upper part of the image, when the photographed image is displayed on the screen of the display processing means as it is, the image is enlarged as it goes to the lower part of the screen (the part with the slower exposure timing). A video like that will be displayed.

  Therefore, in order to solve such a problem, the vehicle-mounted camera of the present invention converts the coordinates of the captured image using the coordinate conversion means 104 so that the image is reduced toward the lower part of the screen and outputs it as a video signal. . At this time, the frame image displayed on the display processing means is an image in which the distortion of the image due to the rolling shutter method is corrected, and can be as shown in 203 of FIG.

  Further, in the camera installed in the direction opposite to the traveling direction, the relationship between the angle of view at the start of shooting and the angle of view at the end of shooting in FIG. 2 is reversed, and the frame image shot by the camera goes to the bottom of the screen. Since the image is reduced further, the coordinate conversion unit is used to convert the coordinates of the captured image so that the image is enlarged toward the lower portion of the screen and output as a video signal.

  Furthermore, in a camera installed in a direction perpendicular to the traveling direction, the shooting range shifts in the traveling direction as it goes to the lower part of the screen.

  According to the above, it is shown that the use of the present invention can improve the distortion of the acquired image caused by the change of the imaging position due to the traveling of the vehicle in the in-vehicle camera system using the rolling shutter type CMOS image sensor. It is.

  FIG. 3 shows an example in which the in-vehicle camera of the present invention is applied to an in-vehicle recognition system. In the figure, reference numeral 301 denotes an image recognition engine, and other components denoted by the same reference numerals as those in FIG. 1 are the same as those in FIG.

  In a general recognition engine, a filtering process is performed on an input image to extract an image feature amount, and matching is performed with a database stored in the recognition engine to perform a recognition process. Therefore, when there is a geometric distortion in the input image, the matching process with the database cannot be performed smoothly, and the recognition process may take a long time or the recognition system may be degraded.

  According to the present invention, since geometric distortion is corrected by the camera, it is possible to improve the reduction in recognition processing speed and system.

DESCRIPTION OF SYMBOLS 101 Lens 102 CMOS image pick-up element 103 Camera signal processing means 104 Coordinate conversion processing means 105 Camera control microcomputer 106 Communication port 107 Image signal output terminal 108 Car control microcomputer 109 Sensing means group 110 Display processing means 111 Car-mounted camera 201 in this invention At the time of imaging | photography start Angle of view 202 Angle of view 203 at the end of shooting Example of distortion-corrected image 301 Image recognition engine

Claims (3)

A lens that forms an image of light coming from an object to be imaged, a CMOS image sensor that converts the imaged light into electrical energy, and an image signal output from the CMOS image sensor is processed to generate a video signal An in-vehicle camera configured to include a camera signal processing unit that performs a coordinate conversion processing unit that corrects distortion generated in the video signal,
An in-vehicle camera that changes a correction amount performed by the coordinate conversion unit in accordance with a moving speed of the in-vehicle camera. An in-vehicle camera system using the in-vehicle camera according to claim 1,
An in-vehicle camera system having display processing means for displaying a video signal photographed by the in-vehicle camera. An in-vehicle camera system using the in-vehicle camera according to claim 1,
A vehicle-mounted camera system having an image recognition engine that performs a recognition process by extracting a feature amount from a video signal photographed by the vehicle-mounted camera.

Images