JP2004001658A - Optical axis deviation detector for in-vehicle camera - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately detect the optical axis deviation of an in-vehicle camera regardless of a place where a vehicle travels and in every system for picking up an image around the vehicle with the in-vehicle camera. <P>SOLUTION: The on-vehicle camera 2 is mounted to the vehicle so that a part of a vehicle body such as a rear bumper comes into a visual field. A position where the rear bumper or the like should be originally observed in the image picked up by the on-vehicle camera 2 is stored beforehand as a template image in a data ROM 8. In detecting the optical axis deviation of the on-vehicle camera 2, the position of the rear bumper or the like in a real image picked up by the in-vehicle camera 2 is compared with the position of the rear bumper or the like in the template image, and when there is dislocation of a prescribed value or more, the occurrence of optical axis deviation of the in-vehicle camera 2 is determined. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an optical axis deviation detecting device for an in-vehicle camera that detects an optical axis deviation that occurs in an in-vehicle camera mounted on a vehicle and capturing an image around the vehicle.
[0002]
[Prior art]
2. Description of the Related Art In recent years, attempts have been actively made to support an appropriate driving operation of a driver by capturing an image around a vehicle with a vehicle-mounted camera and displaying the image on, for example, a monitor installed in a vehicle interior. The in-vehicle camera used in such an application is designed to be mounted on the vehicle in a predetermined mounting state so as to appropriately capture a desired area around the vehicle. An optical axis shift may occur due to various factors such as shift and aging. When the optical axis shift occurs in the vehicle-mounted camera as described above, it is necessary to accurately detect the optical axis shift and correct the optical axis shift.
[0003]
As a technology for detecting an optical axis shift of an in-vehicle camera, for example, an optical axis shift detecting device disclosed in Japanese Patent Application Laid-Open No. 2000-171544 has been proposed. This optical axis deviation detection device is applied to a driving support system that controls the steering angle of a vehicle so that the vehicle travels along a white line of a driving lane, and detects a white line from an image captured by a vehicle-mounted camera. Then, a target steering angle for traveling along the white line is calculated, and when there is a difference between the target steering angle obtained by this calculation and the actual steering angle by the driver's operation, The difference is detected as the optical axis shift of the vehicle-mounted camera.
[0004]
[Problems to be solved by the invention]
However, the above-described conventional technology is based on the premise that a white line is detected from an image captured by an in-vehicle camera. Cannot be detected accurately.
[0005]
The present invention has been made in view of the above-described conventional circumstances, regardless of the location where the vehicle travels, and in any system that captures an image around the vehicle with the vehicle-mounted camera, It is an object of the present invention to provide a device for detecting an optical axis shift of an in-vehicle camera capable of accurately detecting an optical axis shift.
[0006]
[Means for Solving the Problems]
An optical axis deviation detecting device for an on-vehicle camera according to the present invention includes an on-vehicle camera, a storage unit, an image processing unit, and an optical axis deviation determination unit. The in-vehicle camera is mounted on the vehicle at a mounting position where a part of the vehicle body is within the field of view, and captures an image including a part of the vehicle body. In the storage means, a position where a part of the vehicle body should be originally observed in an image captured by the on-vehicle camera is stored in advance.
[0007]
The image processing means processes an image actually picked up by the onboard camera, and detects a position where a part of the vehicle body is observed in the actually picked up image. Then, the optical axis deviation judging means compares the originally observed position of the part of the vehicle body stored in the storage means with the actually observed position of the part of the vehicle body detected by the image processing means. Thus, the optical axis shift of the vehicle-mounted camera is determined based on the position shift.
[0008]
【The invention's effect】
According to the optical axis deviation detecting device for a vehicle-mounted camera according to the present invention, an image including a part of the vehicle body is captured by the vehicle-mounted camera, and the position where the part of the vehicle body is observed in an actual image is stored in advance. Since the optical axis deviation of the in-vehicle camera is determined based on the positional deviation by comparing with the position originally to be observed stored in the means, regardless of the road condition such as the presence or absence of a white line, any arbitrary The optical axis shift of the vehicle-mounted camera can be accurately detected at the timing.
[0009]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
[0010]
(1st Embodiment)
The optical axis deviation detecting device for an in-vehicle camera according to the present invention is applied to, for example, an in-vehicle system 1 as shown in FIG. A function of detecting an optical axis shift of the camera 2 is realized.
[0011]
An on-vehicle system 1 shown in FIG. 1 is mounted on a vehicle and presents various information to an occupant of the vehicle (hereinafter, referred to as own vehicle). A display 3 installed in the vehicle interior for displaying various images, a buzzer 4 for generating a warning sound when an optical axis shift occurs in the on-vehicle camera 2, and a display control of the display 3. And a control unit 5 for controlling the entire operation.
[0012]
The in-vehicle camera 2 is attached to the rear end of the host vehicle, and captures an image of the rear of the host vehicle as an image looking down on the ground at a predetermined depression angle. In particular, the on-vehicle camera 2 is mounted on the host vehicle such that a rear bumper provided at the rear end of the host vehicle is within the field of view, and captures an image including the rear bumper. An image of the rear side of the host vehicle (hereinafter, referred to as an actual image) captured by the on-board camera 2 is supplied to the control unit 5 and displayed on the display 3 as a back-view monitor image as needed. The back view monitor image is an image for assisting the driver's driving operation when the host vehicle is traveling backward, for example, when the host vehicle is parked.
[0013]
The display 3 includes a display such as an LCD (Liquid Crystal Display), and displays various images such as a navigation image and the above-described back view monitor image under display control by the control unit 5. When the control unit 5 detects that the on-vehicle camera 2 has an optical axis shift, the display 3 notifies the occurrence of the optical axis shift under the control of the control unit 5. Is displayed. In the vehicle-mounted system 1, when the control unit 5 detects that the vehicle-mounted camera 2 is out of optical axis, the buzzer 4 is driven under the control of the control unit 5, and a warning is issued. Notification by sound is also performed.
[0014]
The control unit 5 has a microprocessor configuration in which a CPU 6, a RAM 7, a data ROM 8, a program ROM 9, and the like are connected via a bus, and the CPU 6 executes various control programs stored in the program ROM 9 so that the control unit 5 can be mounted on the vehicle. The operation of the entire system 1 is controlled. In particular, in the control unit 5, the RAM 7 functions as an image memory that stores data of a real image behind the host vehicle that is captured by the on-board camera 2 and A / D converted. Also, in the data ROM 8, when the vehicle-mounted camera 2 is mounted at a normal position and in an ideal state in which no optical axis shift occurs, the rear bumper is originally observed in an image captured by the vehicle-mounted camera 2. An image indicating a power position (hereinafter, referred to as a template image) is stored. That is, the data ROM 8 functions as a storage unit that previously stores a position where the rear bumper of the host vehicle should be originally observed in the image captured by the on-vehicle camera 2.
[0015]
Further, in the control unit 5, a navigation program, a rearward monitoring program, and an optical axis deviation detection program are stored in a program ROM 9, and when the CPU 6 executes these programs, a navigation function, a back view monitor function, an optical The axis deviation detecting function is realized respectively.
[0016]
A vehicle speed sensor 10, a gyro sensor 11, and a GPS (Global Positioning System) receiver 12 are connected to the control unit 5, and the control unit 5 transmits a GPS signal from the GPS receiver 12 to the vehicle speed sensor 10 and the gyro. By correcting using the sensor signal from the sensor 11, the accurate current position of the own vehicle can be detected. A data reproducing device 13 such as a DVD (Digital Versatile Disc) drive is connected to the control unit 5, and a map image stored in a recording medium such as a DVD is read out by the data reproducing device 13 and the control unit 5 reads the map image. 5 is supplied. Further, an operation switch 14 such as a joystick is connected to the control unit 5, and an operation input by an occupant using the operation switch 14 is supplied to the control unit 5. During the normal operation of the in-vehicle system 1, the navigation program is executed by the CPU 6 of the control unit 5, and the current position of the vehicle and the operation input of the occupant are displayed on the map image read by the data reproducing device 13. A navigation image obtained by overlapping the searched routes and the like is displayed on the display 3.
[0017]
The control unit 5 is connected to a reverse switch 15 that is turned on when the gear of the host vehicle is shifted to the reverse position. When a reverse ON signal is supplied from the reverse switch 15, the rear monitoring program is executed by the CPU 6 of the control unit 5, and the distance scale is superimposed on the actual image behind the vehicle captured by the vehicle-mounted camera 2. The back view monitor image is displayed on the display 3.
[0018]
The control unit 5 is connected to an ignition switch 16 that is turned on when the engine of the host vehicle is started. When an ignition-on signal is supplied from the ignition switch 16, the optical axis deviation detection program is executed by the CPU 6 of the control unit 5, and the control unit 5 causes the bumper position detection unit (image The processing unit 21 functions as a bumper position deviation evaluation unit (optical axis deviation determination unit) 22 and a notification control unit 23.
[0019]
The bumper position detection unit 21 processes the data of the real image behind the host vehicle captured by the on-board camera 2 and held in the RAM 7, and the rear bumper provided at the rear end of the host vehicle is reflected in the real image. It has a function of detecting the position where the camera is located. The detection of the position of the rear bumper in the real image is performed, for example, by detecting the difference between the pixel value of each pixel constituting the real image and the adjacent pixel, and determining the difference of the pixel value between the adjacent pixels to a predetermined value. This can be easily performed by assuming that the position exceeding the boundary is the boundary between the rear bumper and the background.
[0020]
The bumper position shift evaluator 22 compares the position of the rear bumper, which should be originally observed by the template image stored in the data ROM 8, with the position of the rear bumper in the real image detected by the bumper position detector 21. It has a function of determining whether or not the optical axis shift has occurred in the vehicle-mounted camera 2 based on the degree of the position shift.
[0021]
The notification control unit 23 displays a message to that effect on the display 3 and activates the buzzer 4 when the bumper position shift evaluation unit 22 determines that the optical axis shift has occurred in the in-vehicle camera 2, and warns the buzzer 4. It has a function of controlling sound generation.
[0022]
In the in-vehicle system 1 configured as described above, during the normal operation excluding the case where the host vehicle is driven backward, the navigation program is executed by the CPU 6 of the control unit 5 and the navigation image is displayed on the display 3 as described above. It is displayed. Then, when the gear of the host vehicle is shifted to the reverse position and the reverse ON signal is supplied to the control unit 5, the CPU 6 of the control unit 5 executes the rearward monitoring program, and the display 3 displays the back view monitor image. It has become.
[0023]
At this time, an optical axis shift occurs in the vehicle-mounted camera 2 due to a position deviation at the time of mounting the vehicle-mounted camera 2, a position deviation due to physical factors such as vehicle contact, or a position deviation due to aging. In this case, the vehicle-mounted camera 2 cannot properly capture the actual image behind the vehicle, and the distance scale of the back-view monitor image is different from the actual distance. There is a problem that an image of a different area is displayed and the driver's driving operation cannot be properly supported. Therefore, in the vehicle-mounted system 1 to which the present invention is applied, when the ignition switch 16 is turned on, the CPU 6 of the control unit 5 executes the optical axis deviation detection program to determine whether the optical axis deviation has occurred in the vehicle-mounted camera 2. Is periodically determined, and when the optical axis shift occurs in the on-vehicle camera 2, the fact is displayed on the display 3 as a message, and a warning sound is generated from the buzzer 4 to notify the occupant of the vehicle. Like that.
[0024]
Here, a flow of a series of processing by the control unit 5 for determining the optical axis shift of the on-vehicle camera 2 in the above-described on-vehicle system 1 will be described with reference to a flowchart of FIG. 3 is started when the ignition switch 16 is turned on, and is repeatedly performed until the ignition switch 16 is turned off, and the vehicle is running or stopped. This is performed regardless of whether the image displayed on the display 3 is a navigation image or a back-view monitor image.
[0025]
First, when the ignition switch 16 is turned on, the optical axis shift detection program is executed by the control unit 5, and it is determined whether or not a predetermined optical axis shift detection timing has come (step S1-1). The optical axis deviation detection timing can be arbitrarily set according to the required detection accuracy, the processing capability of the control unit 5, and the like. Then, when the optical axis deviation detection timing comes, an actual image behind the host vehicle is captured by the on-vehicle camera 2, and is captured and held in the RAM 7 of the control unit 5 (step S1-2).
[0026]
Next, the bumper position detection unit 21 performs a process of detecting a position where the rear bumper is reflected in the actual image behind the host vehicle held in the RAM 7. Specifically, first, the bumper position detection unit 21 sequentially searches the real image behind the host vehicle from bottom to top and from left to right, and detects a difference between pixel values of vertically adjacent pixels, respectively. (Step S1-3). Then, a pixel having a difference in pixel value between adjacent pixels among the pixels constituting the real image exceeding a predetermined value, that is, a position in the real image at which the brightness greatly changes is specified (step S1). S1-4), the position coordinates are registered so as to be identifiable (step S1-5). Specifically, for example, the position coordinates I (x, y) of the position where the brightness greatly changes are registered as “1”, and the other position coordinates I (x, y) are registered as “0”. The above processing is repeated for each pixel of the entire real image (step S1-6).
[0027]
The position where the brightness greatly changes in the real image usually appears as a continuous line in the real image, as shown in FIG. 4A, and this position corresponds to the rear bumper provided at the rear end of the vehicle. And the background. With the above-described processing by the bumper position detection unit 21, the position coordinates of each point indicating the boundary between the rear bumper and the background are registered in an identifiable manner.
[0028]
Next, the bumper position shift evaluator 22 determines the position of the rear bumper that should be originally observed by the template image stored in the data ROM 8 and the position of the rear bumper in the real image detected by the bumper position detector 21. A comparison is performed to determine whether or not the optical axis shift has occurred in the vehicle-mounted camera 2 based on the degree of the position shift.
[0029]
In the template image stored in the data ROM 8, for example, as shown in FIG. 4B, the boundary between the rear bumper and the background, which should be originally observed, appears as a line, and the position coordinates of each point constituting the boundary are represented by J. (X, y). The bumper position shift evaluating unit 22 calculates the position coordinates J (x, y) of each point appearing as a boundary line between the rear bumper and the background in the template image, the rear bumper and the background in the real image detected by the bumper position detecting unit 21. Are compared with the position coordinates I (x, y) of each point indicating the boundary between the two. Specifically, as shown in FIG. 4C, the bumper displacement evaluation unit 22 first determines the template corresponding to the same horizontal position (x value) over the entire horizontal direction (x-axis direction) of the image. The difference α (x) between the position (y value) of the point on the image in the y-axis direction and the position (y value) of the point on the real image in the y-axis direction is obtained (step S1-7). Here, α (x) is the difference between the y value of a point on the template image at an arbitrary position (x) in the horizontal direction of the image and the y value of a point on the real image. Next, two evaluation values D1 and D2 are calculated for the calculated α (x) (step S1-8). Here, D1 is an evaluation value obtained by D1 = Σ | α (x) |, and D2 is a variance of α (x), that is, an evaluation value represented by Var (α (x)). Then, the bumper displacement evaluation unit 22 compares each of the two evaluation values obtained as described above with a predetermined threshold, and determines whether the evaluation value D1 is equal to or higher than the predetermined threshold or that the evaluation value D2 is equal to or higher than the predetermined threshold. Is determined that the optical axis shift has occurred in the on-vehicle camera 2 (step S1-9).
[0030]
When the bumper position shift evaluating unit 22 determines that the optical axis shift has occurred in the in-vehicle camera 2, next, the display control of the display 3 is performed by the notification control unit 23, and the optical axis shift occurs in the on-board camera 2. A message to the effect is displayed on the display 3. Further, the buzzer 4 is driven by the notification control unit 23 to generate a warning sound, and the fact that the optical axis shift has occurred in the on-vehicle camera 2 is notified to the occupant of the vehicle (step S1-10). The occupant of the vehicle can recognize from this notification that the optical axis shift has occurred in the in-vehicle camera 2 and take appropriate measures such as bringing the own vehicle to a repair shop and adjusting the optical axis of the in-vehicle camera 2. It becomes possible to plan. In the in-vehicle system 1 to which the present invention is applied, the above-described series of processing is repeatedly performed by the control unit 5 until the ignition switch 16 is turned off, and it is periodically determined whether or not the optical axis shift has occurred in the in-vehicle camera 2. Is determined.
[0031]
As described above, in the on-vehicle system 1 to which the present invention is applied, it is periodically determined whether or not the on-board camera 2 has an optical axis shift. Is notified to the occupant of the vehicle, so that the occupant of the vehicle is urged to take appropriate measures such as optical axis adjustment, and the vehicle-mounted camera 2 captures a necessary image and causes the display 3 to display the necessary image. Can appropriately support the driving operation. In particular, in the in-vehicle system 1, the optical axis deviation of the in-vehicle camera 2 is compared by comparing the template image stored in the data ROM 8 in advance with the actual image captured by the in-vehicle camera 2 and taken into the RAM 7. Since the determination is made, it is possible to accurately detect the optical axis shift of the vehicle-mounted camera 2 irrespective of the road condition such as the presence or absence of a white line, the use of the vehicle-mounted camera 2, and the like.
[0032]
(Second embodiment)
Next, as a second embodiment, an example in which the present invention is applied to another vehicle-mounted system 30 as shown in FIG. 5 will be described. The in-vehicle system 30 according to the second embodiment captures an image of a position at a blind spot from a driver on the left front of the own vehicle (near the left fender) with the in-vehicle camera 2 in response to an instruction input by an occupant of the own vehicle. The information is displayed on the display 3 to assist the driver's driving operation at the time of turning left or the like. The on-vehicle camera 2 is built in a left side mirror of the own vehicle. The in-vehicle camera 2 is mounted inside the side mirror so that the left turn signal provided on the side of the host vehicle enters the field of view, and captures an image including the left turn signal.
[0033]
The in-vehicle system 30 according to the second embodiment includes a display switch 31 instead of the reverse switch 15 included in the in-vehicle system 1 according to the first embodiment. The display changeover switch 31 is for inputting an instruction to display an image at a position corresponding to a blind spot taken by the vehicle-mounted camera 2 on the display 3, and is connected to the control unit 5. The display changeover switch 31 may be configured integrally with the operation switch 14 for performing an operation input related to navigation.
[0034]
In the vehicle-mounted system 30 according to the second embodiment, a turn signal switch 32 is further connected to the control unit 5. The turn signal switch 32 is turned on when the turn signal lever of the host vehicle is operated and the left turn signal is operated. Further, in the vehicle-mounted system 30 of the second embodiment, when the vehicle-mounted camera 2 is mounted in a normal position in the data ROM 8 of the control unit 5 in an ideal state in which no optical axis deviation occurs, An image (hereinafter, referred to as a template image) indicating a position where the left blinker should be originally observed in the image captured by the camera 2 is stored. That is, the data ROM 8 functions as a storage unit that previously stores a position where the left blinker of the host vehicle should be originally observed in the image captured by the on-vehicle camera 2. Further, a blind spot assisting program is stored in the program ROM 9 of the control unit 5 instead of the above-described rearward monitoring program, and the CPU 6 executes the blind spot monitoring program so that the blind spot assisting function is realized. Has become.
[0035]
Other configurations of the in-vehicle system 30 of the second embodiment are the same as those of the in-vehicle system 1 of the first embodiment described above. The same reference numerals are given and duplicate description will be omitted.
[0036]
In the in-vehicle system 30 of the second embodiment, when an ignition-on signal from the ignition switch 16 is supplied to the control unit 5, the CPU 6 of the control unit 5 executes the optical axis deviation detection program, and the control unit 5 As shown in FIG. 6, it functions as a turn signal position detecting section (image processing means) 33, a turn signal position shift evaluating section (optical axis shift determining means) 34, and a notification control section 35.
[0037]
The blinker position detection unit 33 processes data of an image (hereinafter, referred to as an actual image) of a position at a blind spot on the front left side of the own vehicle, which is captured by the vehicle-mounted camera 2 and held in the RAM 7, and within the actual image. , Which has a function of detecting a position where the left turn signal provided on the side of the own vehicle is reflected.
[0038]
The blinker position shift evaluating unit 34 compares the position of the blinker which should be originally observed by the template image stored in the data ROM 8 with the position of the blinker in the actual image detected by the blinker position detecting unit 33. It has a function of determining whether or not the optical axis shift has occurred in the vehicle-mounted camera 2 based on the degree of the position shift.
[0039]
The notification control unit 35 displays a message to that effect on the display 3 and activates the buzzer 4 when the winker position shift evaluation unit 34 determines that the optical axis shift has occurred in the in-vehicle camera 2 and warns the user. It has a function of controlling sound generation.
[0040]
The in-vehicle system 20 according to the second embodiment configured as described above performs the above-described first embodiment during normal operation except when an occupant of the own vehicle receives an instruction input using the display changeover switch 31. The navigation program is executed by the CPU 6 of the control unit 5, and a navigation image is displayed on the display 3, similarly to the vehicle-mounted system 1 of the embodiment. When an instruction input using the display changeover switch 31 is supplied to the control unit 5, the blind spot assisting program is executed by the CPU 6 of the control unit 5, and the display 3 displays the position where the driver in front of the host vehicle becomes a blind spot. Is displayed.
[0041]
At this time, an optical axis shift occurs in the vehicle-mounted camera 2 due to a position deviation at the time of mounting the vehicle-mounted camera 2, a position deviation due to physical factors such as vehicle contact, or a position deviation due to aging. In this case, there is a problem that the in-vehicle camera 2 cannot appropriately capture an image of a position that is a blind spot, and the blind spot assist function cannot be exhibited. Therefore, in the in-vehicle system 30 according to the second embodiment, similarly to the in-vehicle system 1 according to the first embodiment, when the ignition switch 16 is turned on, the CPU 6 of the control unit 5 detects the optical axis deviation. By executing the program, it is periodically determined whether or not the optical axis shift has occurred in the in-vehicle camera 2, and when the optical axis shift has occurred in the in-vehicle camera 2, the fact is displayed on the display 3 as a message. At the same time, the buzzer 4 generates a warning sound to notify the occupant of the vehicle.
[0042]
Here, a flow of a series of processing by the control unit 5 for determining the optical axis shift of the on-vehicle camera 2 in the on-vehicle system 30 of the second embodiment as described above will be described with reference to the flowchart of FIG. The series of processes shown in FIG. 7 is started when the ignition switch 16 is turned on and is repeatedly performed until the ignition switch 16 is turned off, and the vehicle is running or stopped. Also, the operation is performed regardless of whether the image displayed on the display 3 is a navigation image or an image of a position at a blind spot on the left front of the vehicle.
[0043]
First, when the ignition switch 16 is turned on, the optical axis shift detection program is executed by the control unit 5, and it is determined whether or not a predetermined optical axis shift detection timing has come (step S2-1). The optical axis deviation detection timing is determined based on, for example, the ON / OFF of the turn signal switch 32. Then, at the timing when the turn signal switch 32 is turned on, the vehicle-mounted camera 2 captures an actual image of the front left of the host vehicle when the left turn signal is turned on as shown in FIG. And stored (step S2-2). At the timing when the turn signal switch 32 is turned off, the vehicle-mounted camera 2 captures an actual image of the front left of the host vehicle when the left turn signal is not lit as shown in FIG. It is captured and held (step S2-3).
[0044]
Next, the turn signal position detection unit 33 performs a process of detecting a position where the left turn signal is reflected in the real image of the left front of the host vehicle held in the RAM 7. Specifically, the turn signal position detection unit 33 first detects a difference between the real image of the front left of the own vehicle captured when the left turn signal is turned on and the real image of the left front of the own vehicle captured when the left turn signal is not lit. (Step S2-4). As a difference between these real images, a large value is obtained only at the position of the left blinker. Next, this difference result is binarized by a predetermined threshold value, and the approximate result of the left blinker shown in FIG. The region is specified, and the center of gravity G (x, y) of the region is determined, and the center of gravity G (x, y) is specified as the position of the left blinker in the actual image on the left front of the vehicle (step). S2-5). Note that noise may be mixed in the difference result between the real image when the left turn signal is turned on and the real image when the left turn signal is not lit, and this noise hinders the identification of the exact position of the left turn signal. It is also expected that the range for obtaining the difference between the real images is limited to a certain range where the left blinker will exist, or even if the difference result exceeds the threshold, the area of the area is It is desirable to add a contrivance such as excluding from the target when the value is outside the predetermined range.
[0045]
Next, the blinker position shift evaluator 34 uses the template image stored in the data ROM 8 to originally observe the position of the left blinker and the position of the left blinker in the actual image detected by the blinker position detector 33. Are compared with each other, and processing for determining whether or not the optical axis shift has occurred in the vehicle-mounted camera 2 based on the degree of the position shift is performed.
[0046]
In the template image stored in the data ROM 8, for example, as shown in FIG. 8D, the position of the center of gravity of the left blinker that should be originally observed is shown. The blinker position shift evaluating unit 34 first compares the center of gravity position of the left blinker shown in the template image with the center of gravity of the left blinker in the actual image detected by the blinker position detecting unit 33, and compares them. By calculating the geometric distance d of the position coordinates, the positional deviation between the position of the left blinker in these template images and the position of the left blinker in the real image is calculated (step S2-6). If the calculated distance d is equal to or greater than the predetermined threshold, it is determined that the optical axis shift has occurred in the vehicle-mounted camera 2 (step S2-7).
[0047]
When the blinker position deviation evaluating unit 34 determines that the optical axis deviation has occurred in the on-vehicle camera 2, next, the display control of the display 3 is performed by the notification control unit 35, and the optical axis deviation is imposed on the on-vehicle camera 2. A message to the effect is displayed on the display 3. Further, the buzzer 4 is driven by the notification control unit 35 to generate a warning sound, and the fact that the optical axis shift has occurred in the in-vehicle camera 2 is notified to the occupant of the vehicle (step S2-8). The occupant of the vehicle can recognize from this notification that the optical axis shift has occurred in the in-vehicle camera 2 and take appropriate measures such as bringing the own vehicle to a repair shop and adjusting the optical axis of the in-vehicle camera 2. It becomes possible to plan. In the vehicle-mounted system 30 according to the second embodiment, similarly to the vehicle-mounted system 1 according to the first embodiment, the above-described series of processing is repeatedly performed by the control unit 5 until the ignition switch 16 is turned off. Whether or not the optical axis shift has occurred in the camera 2 is periodically determined.
[0048]
As described above, in the in-vehicle system 30 according to the second embodiment, similarly to the in-vehicle system 1 according to the first embodiment described above, it is periodically determined whether or not the optical axis shift has occurred in the in-vehicle camera 2. When the optical axis shift occurs in the in-vehicle camera 2, the fact is notified to the occupant of the vehicle. 2, a necessary image is captured and displayed on the display 3, so that the driving operation by the driver can be appropriately supported. In particular, in the in-vehicle system 1, the optical axis deviation of the in-vehicle camera 2 is compared by comparing the template image stored in the data ROM 8 in advance with the actual image captured by the in-vehicle camera 2 and taken into the RAM 7. Since the determination is made, it is possible to accurately detect the optical axis shift of the vehicle-mounted camera 2 irrespective of the road condition such as the presence or absence of a white line, the use of the vehicle-mounted camera 2, and the like.
[0049]
In the vehicle-mounted system 30 of the second embodiment, the vehicle-mounted camera 2 captures an actual image of the front left of the own vehicle when the left turn signal is turned on and an actual image of the front left of the own vehicle when the left turn signal is not turned on. Since the difference between these images is obtained and the position of the left blinker in the real image is specified based on the result, for example, according to various conditions such as time zone and weather, the left blinker and the left blinker in the real image are determined. Even when the difference from the background is small, the position of the left blinker can be accurately specified, and based on this, the optical axis shift generated in the vehicle-mounted camera 2 can be accurately detected.
[0050]
Further, in the in-vehicle system 30 of the second embodiment, the difference between the real image of the front left of the host vehicle when the left turn signal is turned on and the real image of the front left of the host vehicle when the left turn signal is not lit is determined by a predetermined threshold value. Since the binarization is performed and the position of the center of gravity of the obtained area is specified as the position of the left blinker, for example, the influence of the reflection on the vehicle body when the blinker is turned on is reduced, and the position of the left blinker is more accurately determined. The optical axis deviation generated in the vehicle-mounted camera 2 can be detected more accurately based on the identification.
[0051]
As described above, as an application example of the present invention, the in-vehicle system 1 having the back view monitor function and the in-vehicle system 30 having the blind spot assisting function have been specifically described. However, the present invention is not limited to the above examples. The present invention can be effectively applied to any in-vehicle system that includes a camera and in which detection of the optical axis deviation is desired.
[0052]
Further, in the above-described example, when the optical axis deviation of the on-vehicle camera 2 is detected, for example, a message to that effect is displayed on the display 3 or the buzzer 4 is driven to generate a warning sound. The optical axis shift of the camera 2 is notified to the occupant of the vehicle. However, when the optical axis shift of the on-vehicle camera 2 is detected, the optical axis shift may be automatically corrected. In this case, the in-vehicle systems 1 and 30 are provided with an actuator (camera position adjusting means) capable of adjusting the optical axis direction of the in-vehicle camera 2, and the control unit 5 determines the amount of deviation of the optical axis of the in-vehicle camera 2. The actuator may be driven accordingly. Specifically, in the in-vehicle system 1 according to the first embodiment, the control unit 5 constantly monitors the above-described evaluation values D1 and D2, and adjusts the actuator so that both the evaluation values D1 and D2 become substantially zero. May be drive-controlled. In the vehicle-mounted system 30 according to the second embodiment, the control unit 5 controls the drive of the actuator such that the geometric distance d becomes substantially zero while constantly monitoring the above-described geometric distance d. Just fine.
[0053]
As in this example, when the optical axis deviation of the vehicle-mounted camera 2 is detected and the optical axis deviation is automatically corrected, the own vehicle is brought into a repair shop and the optical axis deviation of the vehicle-mounted camera 2 is detected. It is more convenient because there is no need to make adjustments.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a vehicle-mounted system according to a first embodiment to which the present invention is applied.
FIG. 2 is a functional block diagram of a control unit realized by executing an optical axis deviation detection program in the vehicle-mounted system according to the first embodiment.
FIG. 3 is a flowchart illustrating a flow of a series of processes performed by a control unit that determines an optical axis shift of a vehicle-mounted camera in the vehicle-mounted system according to the first embodiment.
FIG. 4 is a diagram for explaining a method of detecting a displacement between the position of the rear bumper in the actual image and the position of the rear bumper in the template image in the in-vehicle system according to the first embodiment; The rear bumper boundary line in the image is shown, (b) shows the rear bumper boundary line of the template image, and (c) shows how these boundary lines are compared.
FIG. 5 is a block diagram showing a vehicle-mounted system according to a second embodiment to which the present invention is applied.
FIG. 6 is a functional block diagram of a control unit realized by executing an optical axis deviation detection program in the vehicle-mounted system according to the second embodiment.
FIG. 7 is a flowchart illustrating a flow of a series of processes performed by a control unit that determines an optical axis shift of a vehicle-mounted camera in the vehicle-mounted system according to the second embodiment.
FIG. 8 is a diagram for explaining a method of detecting a displacement between the position of the left blinker in the actual image and the position of the left blinker in the template image in the vehicle-mounted system according to the second embodiment; Shows a real image taken when the left turn signal is turned on, (b) shows a real image taken when the left turn signal is not turned on, (c) shows a difference result of these real images, and (d) is originally observed. The position of the center of gravity (template image) of the power left blinker is shown.
[Explanation of symbols]
1 In-vehicle system
2 In-vehicle camera
3 Display
7 Buzzer
5 Control unit
8 Data ROM (storage means)
21 Bumper position detector (image processing means)
22 Bumper misalignment evaluation unit (optical axis misalignment determination means)
23 Notification control unit
30 In-vehicle system
33 Turn signal position detection unit (image processing means)
34 Winker position shift evaluation unit (optical axis shift determination means)
35 Notification control unit

Claims (9)

An in-vehicle camera mounted on the vehicle at a mounting position where a part of the vehicle body enters the field of view,
Storage means for storing in advance a position at which a part of the vehicle body should be originally observed in an image captured by the on-vehicle camera,
Image processing means for processing an image actually captured by the on-board camera, and detecting a position where a part of the vehicle body is observed in the image,
The actual position of a part of the vehicle body stored in the storage means, which should be observed, is compared with the actually observed position of the part of the vehicle body detected by the image processing means. An optical axis shift detecting device for an on-board camera, comprising: The optical axis deviation detecting device for a vehicle-mounted camera according to claim 1, wherein the vehicle-mounted camera captures an image of the rear of the vehicle. The optical axis deviation detecting device for an in-vehicle camera according to claim 2, wherein the in-vehicle camera captures an image including a rear bumper provided at a rear end of the vehicle as a part of the vehicle body. 2. The optical axis deviation detecting device for a vehicle-mounted camera according to claim 1, wherein the vehicle-mounted camera captures an image in front of the vehicle from a door mirror of the vehicle. 3. The optical axis shift detecting device for an in-vehicle camera according to claim 4, wherein the in-vehicle camera captures an image including a turn signal provided on a side of the vehicle as a part of the vehicle body. The in-vehicle camera captures both images when the blinker is lit and when not lit,
6. The blinker according to claim 5, wherein the image processing unit specifies the position of the blinker in these images based on a difference between an image photographed when the blinker is turned on and an image photographed when the blinker is not turned on. Optical axis misalignment detection device for in-vehicle camera. The image processing means binarizes a difference result between an image captured when the turn signal is turned on and an image captured when the turn signal is not turned on with a predetermined threshold value, and specifies the position of the center of gravity of the obtained area as the position of the turn signal. The optical axis deviation detecting device for a vehicle-mounted camera according to claim 6, wherein: 8. The information processing apparatus according to claim 1, further comprising: a notifying unit that notifies, when the optical axis deviation determining unit determines that the optical axis deviation has occurred in the on-vehicle camera, to that effect. Optical axis misalignment detection device for in-vehicle camera. When the optical axis shift determining means determines that an optical axis shift has occurred in the on-vehicle camera, the apparatus further comprises a camera position adjusting means for adjusting the position of the on-vehicle camera to correct the optical axis shift. The optical axis deviation detecting device for a vehicle-mounted camera according to claim 1.

Images