JP2000186929A - On-vehicle and etc stereoscopic camera device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately recognize a marker, a signal and a vehicle by measuring the positions of the marker and the signal with a stereoscopic camera, and controlling the pan, tilt and zoom of a high-picture element camera with this information. SOLUTION: The positions of a marker and a signal are measured by a stereoscopic camera 2, and pan, tilt and zoom of a high-pixel camera are controlled, based on this information to recognize the states of the marker and the signal. Objects of recognition such as the marker, the signal and a vehicle are photographed nearly concurrently by the camera 2 via a lens 3. The photographed signal is amplified by an image amplifier 4, converted by an A/D converter 14 and processed by a digital signal processor 15. The processed image signal is converted into an analog image signal by an encoder and D/A converter 16, reconverted by an A/D converter 6 after an excess signal component is removed by an image filter 5, and is then fed to a CPU 7. At the CPU 7, positional accuracy is corrected, and the positions of the objects of recognition such as the marker, signal and vehicle are measured.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an in-vehicle and ETC stereo camera apparatus, and more particularly to a three-dimensional camera for measuring the position of an object to be recognized, such as a sign or a signal, using a stereo camera. The pan, tilt, and zoom are controlled to accurately image the recognition target.

[0002]

2. Description of the Related Art Generally, in a vehicle system using a stereo camera, a road shape and the presence and position (distance) of an obstacle are determined by two cameras (for example, CCD cameras) using the same parallax principle as the human eye. At the same time, it is widely known to those skilled in the art to recognize three-dimensionally.

[0003]

However, in the vehicle system using the above-mentioned conventional stereo camera, the left and right
The two cameras are arranged at a distance of several tens of centimeters with higher accuracy (six axes such as horizontal, vertical, rotation direction, tilt, etc.) than the pixel unit (several microns) of the CCD, and the optical system (lens etc.) Since a high-precision pixel-by-pixel unit is used, there is a drawback that it is not feasible in terms of mass productivity and cost.

Further, in a vehicle system using a conventional stereo camera, (1) there is a case where information of a broken image cannot be obtained due to high luminance such as reflection of the sun or reflection of a strobe light. (2) In many cases, a monochrome camera is used, and color information cannot be obtained. (3) It is difficult to obtain information at night because the system using the visible light range camera is exclusively used. (4) In imaging at a synchronization frequency of a video camera, an image flows on a fast-moving subject, and information cannot be obtained. Further, even when the still image information is acquired after resetting, when high-accuracy position information is required as in a stereo camera, a temporal displacement of a fast-moving subject cannot be suppressed. There was a problem.

[0005]

In order to solve the above-mentioned problems, a vehicle-mounted and ETC stereo camera device according to the present invention uses a stereo camera to determine the position of a recognition target such as a sign, a signal, and a vehicle. It is characterized in that measurement is performed two-dimensionally, and the pan, tilt, and zoom of the high-pixel camera are controlled based on the information to accurately image the recognition target.

[0006] Thus, it is possible to accurately recognize a sign, a signal, a vehicle, and the like.

[0007]

DESCRIPTION OF THE PREFERRED EMBODIMENTS According to the first aspect of the present invention, a position of a sign, a signal, or the like is measured using a stereo camera, and the pan, tilt, and zoom of a high pixel camera are controlled based on the information. This is a vehicle-mounted stereo camera device characterized by accurately recognizing the states of signs, signals, and the like with a high-pixel camera, and has the effect of enabling recognition of signs, signals, and the like with high accuracy.

[0008] The invention described in claim 2 of the present invention provides:
The position of a vehicle moving at high speed is measured using a stereo camera, and the pan, tilt,
This is an in-vehicle stereo camera device characterized by controlling the zoom and accurately recognizing a state of a vehicle or the like with a high-pixel camera, and has an effect that a vehicle or the like can be accurately recognized. .

Further, the invention according to claim 3 of the present invention provides:
3. The in-vehicle stereo camera device according to claim 1, wherein the stereo camera performs pre-processing before image processing, and corrects the installation position accuracy of the left and right cameras. There is an effect that it is possible to accurately recognize a sign, a signal, a vehicle, and the like.

[0010] The invention described in claim 4 of the present invention provides:
3. The on-vehicle vehicle according to claim 1, wherein even if the physical position information of the stereo camera is somewhat low, stereo processing can be performed by correcting position accuracy by image processing. This is a stereo camera device for use, and has the effect of being able to accurately recognize signs, signals, vehicles, and the like.

Further, the invention according to claim 5 of the present invention provides:
The position of a vehicle moving at high speed is measured using a stereo camera, and the pan, tilt,
This is an ETC stereo camera device characterized by controlling the zoom and accurately recognizing the state of the vehicle or the like with a high pixel camera, and has an effect that the vehicle or the like can be recognized accurately. .

[0012] The invention described in claim 6 of the present invention provides:
6. The ETC stereo camera device according to claim 5, wherein the stereo camera performs pre-processing before image processing, and corrects the installation position accuracy of the left and right cameras. This has the effect that recognition can be performed with high accuracy.

The invention according to claim 7 of the present invention provides:
6. The ETC stereo camera device according to claim 5, wherein even if the physical position information of the stereo camera is somewhat low, stereo processing can be performed by correcting position accuracy by image processing. This has the effect that recognition of a vehicle or the like can be performed with high accuracy.

Further, the invention according to claim 8 of the present invention provides:
8. A system according to claim 1, wherein a super dynamic range camera is used as said stereo camera so that information can be obtained without loss of image even in a high-luminance subject.
And a stereo camera device for in-vehicle and ETC use, which has the effect that information can be obtained without loss of image even with a high-luminance subject.

The invention according to claim 9 of the present invention provides:
The in-vehicle device according to any one of claims 1 to 7, wherein not only luminance information but also color information is obtained by using a color camera as the stereo camera.
This is a TC stereo camera device, and has an effect that not only luminance information but also color information can be obtained.

According to a tenth aspect of the present invention, night information can be obtained by using a far-infrared camera or a near-infrared camera as the stereo camera. Item 7 is a stereo camera device for in-vehicle use and ETC according to any one of Items 7, and has an effect that nighttime information can be obtained.

Further, according to the invention described in claim 11 of the present invention, by using a high-speed reset shutter camera as the stereo camera, stereo image processing can be performed even on an object moving at high speed.
A stereo camera device for in-vehicle use and ETC according to any one of claims 7 to 7, which has an effect that stereo image processing can be performed even on a subject moving at high speed.

An embodiment of a stereo camera device for on-vehicle and ETC according to the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of a vehicle-mounted and ETC stereo camera device of the present invention. The outline of the vehicle-mounted and ETC stereo camera device of the embodiment of the present invention will be described with reference to FIG. Before starting the explanation, the in-vehicle stereo camera device is for moving the camera and the subject is fixed, while the ETC (automatic toll collection system) stereo camera device is for the fixed camera and the moving subject It is refused in advance that the relationship between the camera and the subject is exactly opposite to each other.

In order for the principle of parallax similar to that of the human eye to operate, the left and right cameras, such as a high-speed reset shutter camera, a super dynamic range camera, etc. (the camera itself is constituted by a CCD camera, for example),
Signals, objects to be recognized such as vehicles, for example, road signs, signals, vehicle license plates, curve mirrors in mountainous areas,
The position (distance) of an obstacle or the like is simultaneously and three-dimensionally recognized.

That is, a stereo camera 2 composed of, for example, a high-speed reset shutter camera, a super dynamic range camera, or the like via the lens 3 (in the illustrated example, one of the left and right cameras is specified, and the other It should be noted that this also has a similar configuration). Recognized objects such as picked-up signs, signals, and vehicles are amplified by the video amplifier 4 and A / D converted by the A / D converter 14, and further processed by the DSP 15 such as super dynamic processing and correction processing. And
The processed digital video signal to be recognized is converted back to an analog video signal by the encoder / D / A converter 16, and the analog video signal is transferred to the video filter 5, and the video filter 5 After removing the signal component, the signal component is again A / D converted by the A / D converter 6 and passed to the control CPU 7.

In the CPU 7, signs, signals,
A stereo image processing unit that performs stereo image processing on a recognition target such as a vehicle to correct positional accuracy, and corrects the installation position accuracy of the left and right cameras with respect to a recognition target such as a sign, a signal, and a vehicle before the image processing. A correction processing unit is provided,
Signs, signals, which have been A / D converted by the A / D converter 6,
When an object to be recognized, such as a vehicle, is input to the CPU 7, first, the correction processing unit corrects the installation position accuracy, and then the sign, signal, vehicle, etc., which have been A / D converted by the A / D converter 6 by the stereo image processing unit. Measure the position of the recognition target.

Here, the contents of the stereo processing in the CPU 7 will be described with reference to FIGS. In FIG. 2, for stereo processing by the CPU 7, the reference camera Ref. Is imaged in advance by the left and right cameras R and L, and the points A to E on the reference chart obtained from the video data are obtained by the R camera and the L camera. , The coordinates of points A to E are obtained. That is, the point A: the central point point B~ point E: as a point near each coordinate _{R A (X, Y) ~R} E (X, Y) coordinates _{L A (X, Y) ~L} E (X, Y ) Is determined from the captured video signal.

Next, as shown in FIG.
(Or left camera L), for point A, R
_{A (X, Y) = f} (L A (X, Y)) and made to perform the conversion. Similar conversion is performed for the other points B to E.

After performing these pre-processes, for example, the stereo process for an arbitrary point X means, as shown in FIG. 4, a point B, a point C, a point D, This corresponds to obtaining a vector average value with the point E. To increase the accuracy, it is possible to cope with the problem by increasing the number of peripheral sample points.

Based on the result of the measurement, the information is sent to a high-precision camera (not shown), and the PAN (pan) is set so that the object to be recognized, such as a sign, a signal, or a vehicle, captured by the high-precision camera is at the center of the screen. / TILT (tilt) control of the camera, etc.
By controlling lenses such as M (zoom) / FOCUS (focus), recognition targets such as signs, signals, and vehicles can be recognized with high accuracy. After this, the recognition target such as a sign, a signal, a vehicle, etc. recognized with high accuracy is displayed on the monitor 8.

In the above description, the contents of processing such as super dynamic processing and correction processing by the DSP 15 are generally such that the amount of light drops in four directions, up, down, left and right, and resolution degradation occurs due to the performance of the lens used. This lens performance degradation is corrected by the DSP 15 by digital video signal processing. In addition, a reduction in the dynamic range of the image captured by the camera is compensated for by performing a super dynamic process.

In the above description, the contents stored in advance in the memory 9 and the contents recognized in the memory 9 in advance determine the contents of the objects to be recognized, such as signs, signals, vehicles, etc., recognized with high accuracy. CP for signals and results to be recognized such as vehicles
The contents are determined by comparing them in U7, and the determined result is subjected to character conversion by the character converter 10 or voice conversion by the voice converter 11 and mixing by the mixing circuit 12, and the character information is monitored by the monitor 8. Alternatively, the audio information can be notified by a speaker 13 attached to the monitor 8.

In the above, a high-speed reset shutter camera and a super dynamic range camera have been exemplified as examples of cameras used as stereo cameras. In addition, color cameras, far-infrared cameras, near-infrared cameras, An external camera or the like can be used as appropriate. If a high-speed reset shutter camera is used, for example, a vehicle moving at 180 km / h can be reset and image information can be obtained at a shutter speed of about 1 / 100,000 seconds. The information of the left and right cameras can be obtained with a positional deviation of about 0.5 mm from the subject. For a specific example of the configuration of the high-speed reset shutter camera itself used for ETC, see Japanese Patent Application No. 10-185744 filed by the present applicant.

If a super dynamic range camera (also known as a Super D camera) is used, high brightness due to the reflection of the sun or the reflection of a strobe, etc., can be used to obtain information even in a portion where an ordinary camera can lose the image. Obtainable.

When a color camera is used, not only luminance information but also color information can be naturally obtained. If a far-infrared camera is used, far-infrared (10 μm to 12 μm
Information can be obtained even for nighttime subjects. Furthermore, if a near-infrared camera is used, the near-infrared (7
Information can be obtained even for a subject having a size of 00 nm to 1000 nm at night.

[0032]

As is apparent from the above description, according to the present invention, preprocessing is performed before image processing, and the positional accuracy of the left and right cameras is corrected, and the positional accuracy is corrected by image processing. For this reason, even if the physical position information of the stereo camera is low to some extent, that is, a stereo camera device for in-vehicle and ETC that can perform stereo image processing even in a low position precision arrangement that allows a displacement of 2 to 3 pixels in CCD pixels. Is obtained.

If a super dynamic range camera is used as a stereo camera for in-vehicle and ETC,
There is an effect that an in-vehicle and an ETC stereo camera device that can obtain information without loss of image even with a high-luminance subject can be obtained.

Further, if a color camera is used as the on-vehicle and ETC stereo camera, there is an effect that a on-vehicle and ETC stereo camera device which can obtain not only luminance information but also color information can be obtained. Further, if a far-infrared camera or a near-infrared camera is used as the in-vehicle and ETC stereo camera, an in-vehicle and ETC stereo camera device that can obtain nighttime information can be obtained.

Also, if a high-speed reset shutter camera is used as a vehicle-mounted and ETC stereo camera, there is an effect that a vehicle-mounted and ETC stereo camera device capable of performing stereo image processing even on a subject moving at a high speed is obtained.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of an on-board and ETC stereo turtle device of the present invention;

FIG. 2 is a conceptual diagram for explaining stereo processing in a CPU 7,

FIG. 3 is a diagram showing a state of coordinate conversion in stereo R and L cameras;

FIG. 4 is a diagram illustrating an image of a stereo processing result at an arbitrary point X;

[Explanation of symbols]

 2 Camera 3 Lens 4 Video amplifier 5 Video filter 6 A / D converter 7 CPU 8 Monitor 9 Memory 10 Character converter 11 Voice converter 12 Mixing circuit 13 Speaker 14 A / D converter 15 DSP 16 Encoder and D / A conversion vessel

 ──────────────────────────────────────────────────続 き Continued on the front page (72) Mitsuyoshi Sugimoto 4-3-1 Tsunashima Higashi, Kohoku-ku, Yokohama, Kanagawa Prefecture Inside Matsushita Communication Industrial Co., Ltd. (72) Inventor Toshihiko Suda Tsunashima-Higashi, Kohoku-ku, Yokohama, Kanagawa 4-3-1, Matsushita Communication Industrial Co., Ltd. F-term (reference) 2F065 AA04 BB05 BB15 CC11 CC40 FF01 FF05 JJ03 JJ05 JJ26 QQ31

Claims (11)

[Claims] 1. The position of a sign, a signal, etc. is measured using a stereo camera, and the pan, tilt, zoom of the high pixel camera is controlled based on the information, and the state of the sign, signal, etc. is accurately detected by the high pixel camera. An in-vehicle stereo camera device characterized by recognition. 2. The position of a vehicle or the like moving at high speed is measured using a stereo camera, and the pan, tilt, and zoom of the high pixel camera are controlled based on the information, and the state of the vehicle or the like is accurately detected by the high pixel camera. An in-vehicle stereo camera device characterized by recognition. 3. The in-vehicle stereo camera according to claim 1, wherein the stereo camera performs pre-processing before image processing to correct the installation position accuracy of the left and right cameras. apparatus. 4. The stereo processing according to claim 1, wherein even if the physical position information of said stereo camera is low to some extent, stereo processing can be performed by correcting position accuracy by image processing. 3. The in-vehicle stereo camera device according to 2. 5. The position of a vehicle or the like moving at high speed is measured using a stereo camera, and the pan, tilt, and zoom of the high pixel camera are controlled based on the information, and the state of the vehicle or the like is accurately detected by the high pixel camera. A stereo camera device for ETC, characterized by recognition. 6. The ETC stereo camera apparatus according to claim 5, wherein the stereo camera performs pre-processing before image processing to correct the installation position accuracy of the left and right cameras. 7. The stereo processing according to claim 5, wherein even if physical information of the stereo camera is low to some extent, stereo processing can be performed by correcting position accuracy by image processing. Stereo camera device for ETC. 8. The in-vehicle and ETC system according to claim 1, wherein a super dynamic range camera is used as the stereo camera so that information can be obtained without loss of image even in a high-luminance subject. Stereo camera device. 9. The in-vehicle and ETC stereo camera device according to claim 1, wherein not only luminance information but also color information is obtained by using a color camera as said stereo camera. . 10. The in-vehicle and ETC device according to claim 1, wherein a far-infrared camera or a near-infrared camera is used as said stereo camera to obtain information at night. Stereo camera device. 11. The on-vehicle vehicle according to claim 1, wherein a high-speed reset shutter camera is used as the stereo camera so that stereo image processing can be performed even on a subject moving at high speed. Stereo camera device for ETC.