JP2014115814A - On-vehicle camera system and camera lens abnormality detecting method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide On-Vehicle Camera System and Camera Lens Abnormality Detecting Method that can easily and flexibly adapt, in detecting any lens abnormality of on-vehicle cameras, to diversity of abnormality locations and thereby help enhance the accuracy of detection at low cost.SOLUTION: An on-vehicle camera system comprises reliability history preparing means 6 that prepares for each on-vehicle camera 2 a reliability history of road partitioning lines recognized from images photographed by the on-vehicle camera 2, and abnormality determining means 7 that compares the reliability histories of individual on-vehicle cameras 2 and determines the occurrence of lens abnormality in any on-vehicle camera 2 that manifests a relative decline of reliability.

Description

  The present invention relates to an in-vehicle camera system and a camera lens abnormality detection method, and more particularly to an in-vehicle camera system and a camera lens abnormality detection method suitable for detecting an abnormality of a lens of an in-vehicle camera.

  Conventionally, in an in-vehicle camera system that does not output an image such as a Lane Departure Warning (LDW) system, the user detects adhesion of foreign matter (dirt, water droplets, etc.) to the lens of the in-vehicle camera based on the image. Therefore, the usefulness of automatic detection of foreign matter adhesion was high.

  In this type of system, for example, as shown in Patent Document 1, for example, a real image of a part of a vehicle shown in a part of a camera, and a reference image when no foreign matter is attached to the camera, And detecting the presence or absence of a change in the image to determine the presence or absence of foreign matter adhesion.

  However, in such a technique, as shown in FIG. 14, real images of various scenes (nighttime, situations in which shadows of vehicles, etc. are reflected, rainy weather, etc.) are assumed with respect to the reference image. In view of this, it has been difficult to set a reference image and a judgment criterion for preventing erroneous detection of foreign matter adhesion, and there has been a problem that it is necessary to add a dedicated process for detecting foreign matter adhesion.

  On the other hand, Patent Document 2 proposes a technique for determining the white line recognition reliability of a captured image of an in-vehicle camera, and Patent Document 3 discloses overlapping imaging with a plurality of in-vehicle cameras having a wide angle of view. Since technologies for comparing images in regions have been proposed, if these technologies are combined, foreign matter adhesion is detected based on the difference in the degree of white line recognition between the two cameras in the overlapping shooting region without preparing a reference image. It is also possible.

JP 2008-265727 A JP 2010-69921 A JP 2011-223075 A JP 2008-60874 A

  However, in Japanese Patent Laid-Open No. 2004-260688, only the overlapping imaging region is set as a foreign matter adhesion determination region, and the foreign object is detected when “there is a change in luminance more than a predetermined value (claim 2)” or “the white line detection result is different”. It was judged that there was adhesion.

  For this reason, an abnormality cannot be determined unless a foreign object adheres to the lens portion corresponding to the overlapping photographing region.

  Therefore, the present invention has been made in view of such a point, and when detecting an abnormality (foreign matter adhesion or clouding) of a lens of an in-vehicle camera, the present invention easily and flexibly copes with a variety of places where the abnormality occurs. Therefore, an object of the present invention is to provide an in-vehicle camera system and a camera lens abnormality detection method that can improve detection accuracy at low cost.

  In order to achieve the above-described object, an in-vehicle camera system according to the present invention is an in-vehicle camera system including a plurality of in-vehicle cameras mounted on a vehicle, and is recognized on the basis of a captured image of the in-vehicle camera. The reliability history creating means for creating the reliability history of the line for each of the in-vehicle cameras and the reliability history for each of the in-vehicle cameras created by the reliability history creating means are relatively compared. The in-vehicle camera showing a decrease in reliability includes an abnormality determining means for determining that a lens abnormality including adhesion of foreign matters to the lens and / or fogging of the lens has occurred. .

  A camera lens abnormality detection method according to the present invention is a camera lens abnormality detection method for detecting lens abnormality including adhesion of foreign matter to a lens of an in-vehicle camera and / or fogging of the lens, A first step of creating a history of reliability of road lane markings recognized based on captured images of the in-vehicle camera for each of the in-vehicle cameras, and the reliability of each of the in-vehicle cameras created in the first step And a second step of determining that a lens abnormality has occurred in the in-vehicle camera in which the deterioration in reliability is relatively indicated.

  And according to such this invention, since the abnormality of the lens of a vehicle-mounted camera can be detected by comparing the reliability history of the recognition result of the road marking line for every vehicle-mounted camera, Abnormality detection can be performed easily and accurately without constraining the location of the abnormality to the lens portion corresponding to the overlapping photographing area with other in-vehicle cameras.

  The road demarcation line is a solid line drawn on a road such as a roadway center line, a lane boundary line (lane mark), and a roadway outer line (hereinafter the same).

  Further, in the in-vehicle camera system of the present invention, the abnormality determining means may be configured such that the abnormality occurs in the in-vehicle camera in which the decrease in reliability is indicated for a certain period or a predetermined number of times of recognition of the road lane marking. You may judge. Similarly, in the camera lens abnormality detection method of the present invention, in the second step, the abnormality is detected in the in-vehicle camera in which the decrease in the reliability is indicated for a certain period or the predetermined number of times of recognition of the road marking line. It may be a step of determining that it has occurred.

  And according to such this invention, since it can avoid determining with it being abnormal only by temporary fall of reliability, determination accuracy can be improved.

  Furthermore, in the in-vehicle camera system of the present invention, the abnormality determination unit can capture the same part of the same road marking line at the same time or with a time difference according to the progress of the vehicle among the plurality of in-vehicle cameras. You may compare the said log | history of the reliability of vehicle-mounted cameras. In this case, the in-vehicle camera to which the reliability history is compared by the abnormality determination unit is a front camera that captures the front of the vehicle, a side camera that captures the side of the vehicle, and a rear of the vehicle. There may be at least two of the rear cameras. Similarly, in the camera lens abnormality detection method of the present invention, in the second step, among the plurality of in-vehicle cameras, the same part of the same road marking line is simultaneously or at a time difference according to the progress of the vehicle. It may be a step of comparing the reliability histories of the in-vehicle cameras that can be photographed.

  And according to such this invention, since the log | history of reliability based on the same site | part of the same road lane marking can be compared between each vehicle-mounted camera, determination accuracy can be improved further. it can.

  Furthermore, in the in-vehicle camera system according to the present invention, the abnormality determination means may include the reliability history of the front camera and the reliability history of the rear camera. The comparison may be performed by shifting by a delay time that is considered necessary for the rear camera to image the region after the region is imaged. Similarly, in the camera lens abnormality detection method of the present invention, the second step includes a history of the reliability of the front camera as the in-vehicle camera and a history of the reliability of the rear camera as the in-vehicle camera. It may be a step in which the front camera shoots a predetermined part of the road marking line and compares it by shifting by a delay time considered to be required until the rear camera shoots the part.

  According to the present invention as described above, the history of taking into account the delay time for the cameras that shoot the same part of the road lane line with a time difference, such as the front camera and the rear camera, without overlapping imaging regions. Since the comparison can be performed, the determination accuracy can be further improved.

  In the in-vehicle camera system of the present invention, the abnormality determination unit may determine the delay time based on a vehicle speed.

  According to such a configuration, the delay time can be easily grasped based on the vehicle speed.

  Further, in the in-vehicle camera system of the present invention, the reliability history creating unit may create the history of reliability calculated based on the recognized shape of the road marking line.

  And according to such a structure, a suitable reference | standard can be used as a judgment criterion of the reliability of a road lane marking.

  Furthermore, the vehicle-mounted camera system according to the present invention may further include an image recognition control unit that automatically turns off an image recognition function corresponding to the vehicle-mounted camera determined to have the abnormality by the abnormality determination unit. Good.

  And according to such a structure, it can avoid performing meaningless image processing about the vehicle-mounted camera by which abnormality was detected.

  The vehicle-mounted camera system of the present invention may further include abnormality notification means for notifying the abnormality of the corresponding vehicle-mounted camera when the abnormality determination means determines that the abnormality has occurred.

  And according to such a structure, a user can recognize abnormality normally.

  ADVANTAGE OF THE INVENTION According to this invention, when detecting abnormality of the lens of a vehicle-mounted camera, it can respond easily and flexibly to the variety of places where the abnormality occurs, and as a result, detection accuracy can be improved at low cost.

The block diagram which shows a vehicle-mounted camera system in 1st Embodiment of this invention. 1st explanatory drawing for demonstrating recognition processing of a road lane marking 2nd explanatory drawing for demonstrating recognition processing of a road lane marking 3rd explanatory drawing for demonstrating recognition processing of a road lane marking Conceptual diagram showing the reliability history of road lane marking recognition results Schematic diagram showing the process for determining whether there is an abnormality A graph showing the change characteristics of the reliability of the recognition result of road lane markings used for explaining the abnormality determination process Explanatory drawing for explaining the delay time that may be used in the determination process of the presence or absence of abnormality Graph showing change characteristics of reliability of recognition result of road lane markings used to explain judgment processing of presence / absence of abnormality considering delay time In the first embodiment of the present invention, a first flowchart showing a camera lens abnormality detection method. Second flowchart showing camera lens abnormality detection method The schematic block diagram which shows a vehicle-mounted camera system in 2nd Embodiment of this invention. The figure which shows the aspect of the comparative evaluation of reliability in 2nd Embodiment of this invention. Explanatory diagram for explaining conventional problems

[First Embodiment]
(In-vehicle camera system)
Hereinafter, a first embodiment of an in-vehicle camera system according to the present invention will be described with reference to FIGS. 1 to 9.

  As shown in FIG. 1, the vehicle-mounted camera system 1 in this embodiment has two vehicle-mounted cameras 2f and 2r mounted on the vehicle. Among these, one in-vehicle camera 2f is a predetermined photographing centered on the front of the vehicle attached to the front of the vehicle (for example, an emblem) in such a posture as to look down the road surface in front of the vehicle from an oblique direction. A front camera 2f that captures an area is used. The other in-vehicle camera 2r is a predetermined photographing centered on the rear of the vehicle that is attached to the rear of the vehicle (for example, a rear license garnish) in a posture that looks down on the road surface behind the vehicle from an oblique direction. The rear camera 2r shoots an area. Each of the in-vehicle cameras 2f and 2r includes a solid-state imaging device (imaging surface) such as a CCD or a CMOS, and sequentially captures a corresponding imaging area at a predetermined frame rate.

  As shown in FIG. 1, a front-side captured image acquisition unit 3f is connected to the front camera 2f, and a rear-side captured image acquisition unit 3r is connected to the rear camera 2r, and these captured image acquisition units 3f and 3r are connected. Are adapted to acquire images taken by the corresponding in-vehicle cameras 2f and 2r. The captured image acquisition units 3f and 3r may be embodied by a buffer or the like.

  Furthermore, as shown in FIG. 1, the front side captured image acquisition unit 3f includes a front side road lane marking recognition unit 4f, and the rear side captured image acquisition unit 3r includes a rear side road lane marking recognition unit 4r. It is connected. The road lane marking recognition units 4f and 4r are respectively input with the captured images of the corresponding in-vehicle cameras 2f and 2r from the corresponding captured image acquisition units 3f and 3r.

  Each road lane marking recognition unit 4f, 4r performs road lane marking recognition (image recognition) based on the input captured image.

  Here, an example of a road lane marking recognition method will be described. First, the road lane marking recognition units 4f and 4r generate a grayscale luminance image from the input captured image. Then, as shown in FIG. 2, the road lane marking recognition units 4f and 4r scan the generated luminance image in the horizontal direction, and take out as a line a horizontal portion in which luminance equal to or higher than a threshold value continues for a certain interval. Such line extraction is performed on the entire luminance image. Each line taken out in this way is called a slice.

  Next, as shown in FIG. 3, the road lane marking recognition units 4f and 4r overlap at least part of the horizontal coordinate values between the slices adjacent to each other in the vertical direction. Group multiple slices. In the example shown in FIG. 3, three groups G1 to G3 of slices are extracted from the captured image.

After grouping in this way, the road lane marking recognition units 4f and 4r project the slices of the groups G1 to G3 onto the road surface (Z = 0) of the three-dimensional coordinates as shown in FIG. A straight line expression Y = a _Gi X + b _Gi (Z = 0) is extracted from the midpoint of the slice by the least square method. However, a _Gi is a linear gradient for each group Gi (i = 1 to the total number of groups), and b _Gi is a linear Y-intercept for each group Gi. In addition, the projection on the road surface of a three-dimensional coordinate means viewpoint-converting a picked-up image into the bird's-eye view image from the virtual viewpoint above a vehicle. At this time, the origin of the three-dimensional coordinates may be the center position of the vehicle, the plus side on the X axis may be the front of the vehicle, and the plus side on the Y axis may be the left side of the vehicle (see FIG. 4).

In the process of extracting such linear expressions, the road marking line recognition units 4f and 4r have a plurality of linear expressions in which the values of the inclination a _Gi and the Y intercept b _Gi are within a predetermined range. Groups corresponding to the linear expression are grouped as one group. In the example of FIG. 4, the group G2 and the group G3 are combined into one group G4. Then, the road lane marking recognition units 4f and 4r re-extract a straight line from the information of each slice included in the group grouped into one in this way. In the example of FIG. 4, Y = a _G4 X + b _G4 is a re-extracted straight line.

  In addition, the road marking line recognition units 4f and 4r use the same method as the derivation of the straight line expression based on the midpoint of each slice described above, and the straight line expression based on the left end point of each slice and the right end point of each slice. Each of the straight line expressions used as a reference is extracted. Then, the slice width is calculated by subtracting the straight line expression based on the slice left end point and the straight line expression based on the slice right end point extracted in this way to the left and right of the straight line expression based on the slice middle point. At this time, when one group is grouped, the slice width for the one group is calculated.

  As described above, recognition of road lane markings is completed.

  In addition, if it is difficult to perform recognition processing by the road lane marking recognition units 4f and 4r based on captured images for each frame, the recognition processing may be performed based on captured images for a plurality of frames. Further, the road marking line recognition units 4f and 4r may be realized by an arithmetic processing unit such as a CPU executing a program for realizing these functions. A program to be executed by the arithmetic processing unit may be stored in a program storage unit such as a ROM, and a work area such as a RAM may be used for processing of the arithmetic processing unit.

  Returning to FIG. 1, the in-vehicle camera system 1 includes a front-side reliability calculation unit 5 f and a rear-side reliability calculation unit 5 r. The front side reliability calculation unit 5f is the recognition result (recognized road lane line) of the front side road lane marking recognition unit 4f, and the rear side reliability calculation unit 5r is the recognition result of the rear side road lane line recognition unit 4r. Are to get each. Then, the reliability calculation units 5f and 5r calculate the reliability of the corresponding recognition result. At this time, as shown in FIG. 4, the reliability calculation units 5f and 5r recognize a plurality of road section lines (two road section lines corresponding to G1 and G4 in the figure) from the captured image. The reliability is calculated for each road section line.

  Here, the calculation of the reliability may be performed based on the following evaluation value reflecting the shape of the road lane marking, for example.

<Linearity evaluation value>
The linearity evaluation value is an evaluation value for the linearity of the recognition result of the road lane marking recognition units 4f and 4r. Specifically, the linear expression Y = a _Gi X + b _Gi obtained as shown in FIG. This is a calculated value based on the magnitude of the deviation from the midpoint of the slice. The linearity evaluation value MatchingValue can be expressed as the following equation, for example.

MatchingValue = 1.0-Difference / MaxDifference (1)
However, in Equation (1), Difference is an average value of the distance between the linear equation and the midpoint of each slice, and MaxDifference is a distance error threshold.

  The linearity evaluation value MatchingValue calculated in this way takes a value within the range of 0.0 to 1.0.

<Parallelity evaluation value>
The parallelism evaluation value is an evaluation value related to the parallelism between the left and right edges of the road lane markings recognized by the road lane marking recognition units 4f and 4r. Specifically, This is a quantification of the parallelism between two linear expressions, which are based on the right end point of the slice. The parallelism evaluation value EdgeMatchigValue can be expressed as the following equation, for example.

EdgeMatchigValue = EdgeDifference / MaxEdgeDifference (2)
However, in Expression (2), EdgeDifference is the difference between the slopes of the two linear expressions, and MaxEdgeDifference is the maximum value of the difference between the left end point and the right end point of each slice.

  The parallelism evaluation value EdgeMatchigValue calculated in this way also takes a value within the range of 0.0 to 1.0.

  Then, the reliability is one of these evaluation values MatchingValue, EdgeMatchigValue, the average value of both evaluation values, other elements in addition to or in place of the evaluation values (for example, the rectangularity of the recognized road marking line, It may be a value in consideration of width uniformity, luminance distribution, aspect ratio, and the like. In addition to this, the technique described in Japanese Patent Application No. 2012-057155 may be appropriately used for calculating the reliability.

  The reliability calculation units 5f and 5r may be embodied by causing the arithmetic processing unit to execute a program for realizing these functions.

  As shown in FIG. 1, the in-vehicle camera system 1 includes a front side reliability history creating unit 6f and a rear side reliability history creating unit 6r as reliability history creating means. The front side reliability history creation unit 6f obtains the calculation result of the front side reliability calculation unit 5f, and the rear side reliability history creation unit 6r obtains the calculation result of the rear side reliability calculation unit 5r. ing. The reliability history creation units 6f and 6r create and hold the history of the calculation results of the corresponding reliability calculation units 5f and 5r, respectively.

  Here, the reliability history may be one in which the correspondence between reliability and time is described as shown in FIG. 5 (a), or as shown in FIG. 5 (b). The correspondence between the reliability and the number of recognitions of road lane markings (how many times of recognition) may be described, or the correspondence between both time and number of recognitions is described. May be. In this case, the time may be acquired from a measurement result of a timer (not shown) such as a timer, and the number of times of recognition may be acquired from the road lane marking recognition units 4f and 4r. Moreover, as shown in FIG. 4, when there are a plurality of recognition results, a plurality of histories may be created.

  Note that the reliability history creating units 6f and 6r may be embodied by causing the arithmetic processing unit to execute a program for realizing these functions. For holding the history, various external storage devices (such as a hard disk drive), a main memory, or the like may be used.

  Returning to FIG. 1, the in-vehicle camera system 1 has an abnormality determination unit 7 as an abnormality determination unit, and the abnormality determination unit 7 includes the in-vehicle camera 2 f created by the reliability history creation units 6 f and 6 r, A history of reliability every 2r is acquired. Then, the abnormality determination unit 7 compares the acquired reliability histories for the in-vehicle cameras 2f and 2r with each other to the in-vehicle camera in which the decrease in the reliability is relatively indicated. It is determined that a lens abnormality including at least one of adhesion of the lens and cloudiness of the lens has occurred.

  FIG. 6 is a schematic diagram showing the operation of such an abnormality determination unit 7. In FIG. 6, the reliability corresponding to the rear camera 2r is 90%, whereas the reliability corresponding to the front camera 2f is reduced to 30%, so that a lens abnormality occurs in the front camera 2f. It is determined that

  FIG. 7A shows a history of reliability corresponding to the front camera 2f as a change in reliability (vertical axis) according to time or the number of times of recognition of road lane markings (horizontal axis). . Similarly, FIG. 7B shows a history of reliability corresponding to the rear camera 2r as a change in reliability according to time or the number of times of recognition of road lane markings. However, both histories are for the reliability of the recognition result of the same road lane marking.

  Here, as shown in FIG. 7B, the reliability corresponding to the rear camera 2r is stable regardless of the increase in time / recognition times, while as shown in FIG. The reliability corresponding to the camera 2f is relatively decreased with respect to the reliability corresponding to the rear camera 2r as the time / recognition time increases after a certain horizontal axis value v (time / recognition time). .

  In such a case, the abnormality determination unit 7 determines that an abnormality has occurred in the lens of the front camera 2f.

  The acquisition of the reliability history by the abnormality determination unit 7 may be performed for each determination period by the abnormality determination unit 7 or may be performed for each update timing of the history.

  Also, as shown in FIG. 4, when a plurality of road lane markings are recognized from the photographed image, determination is made by comparing the histories of the same road lane markings between the in-vehicle cameras 2f and 2r. Just do. The history of the same road marking line can be easily grasped according to the recognition coordinate range of the road marking line and the difference in the shooting directions of the front and rear cameras 2f and 2r. As described above, when the determination is made by comparing the reliability histories of the plurality of road marking lines between the in-vehicle cameras 2f and 2r, the determination range of the occurrence of the lens abnormality is determined for each lens portion. Will be subdivided. Specifically, for example, with respect to a road lane line recognized on the right side of the image captured by the front camera 2f, a decrease in reliability is seen with respect to a road lane line recognized on the left side of the image captured by the rear camera 2r. Therefore, it is determined that an abnormality has occurred in the right part of the front camera lens, and for the road lane line recognized on the left side of the captured image of the front camera 2f, the road segment recognized on the right side of the captured image of the rear camera 2r. Since no decrease in reliability is observed with respect to the line, it can be determined that no abnormality has occurred in the left portion of the front camera lens.

  Furthermore, the abnormality determination unit 7 may be embodied by causing the arithmetic processing unit to execute a program for realizing this function.

  According to such a configuration, the abnormality of the lenses of the in-vehicle cameras 2f and 2r can be detected by comparing the reliability histories of the recognition results of the road marking lines for the in-vehicle cameras 2f and 2r. This detection can be performed easily and accurately without limiting the location where the abnormality occurs to the lens portion corresponding to the overlapping imaging region.

  In addition to the above configuration, in the present embodiment, the abnormality determination unit 7 further reduces the reliability corresponding to the other in-vehicle camera relative to the reliability corresponding to the one in-vehicle camera for a certain period (for example, 1 minute or the like) or a predetermined number of times of recognition of road lane markings, it may be determined that a lens abnormality has occurred in the other in-vehicle camera.

  According to such a configuration, it is possible to avoid the determination that the abnormality is caused only by a temporary decrease in reliability, so that the determination accuracy can be improved.

  In addition to the above configuration, in the present embodiment, the abnormality determination unit 7 further includes a reliability history corresponding to the front camera 2f and a reliability history corresponding to the rear camera 2r, as shown in FIG. Relative comparison may be performed by shifting by a delay time DT which is considered necessary until the rear camera 2r captures the part P after the front camera 2f captures the predetermined part P of the road marking line.

  Here, FIG. 9 shows an example of such a case. FIG. 9A shows a history of reliability corresponding to the front camera 2f as a change in reliability according to time or the number of times of recognition of road lane markings (horizontal axis). Shows a history of reliability corresponding to the rear camera 2r as a change in reliability according to time or the number of times of recognition of road lane markings. As shown in FIG. 9A, the reliability corresponding to the front camera 2f is decreased after a certain horizontal axis value v1, whereas the reliability corresponding to the rear camera 2r at the simultaneous point v1 is stable. Therefore, when the reliability of the simultaneous point v1 is compared and evaluated, it is determined that the reliability is reduced only on the front camera 2f side. However, as shown in FIG. 9B, in the horizontal axis value v2 shifted by the delay time DT with respect to the horizontal axis value v1, the reliability corresponding to the rear camera 2r is also reduced similarly to the front side. Has occurred. In such a case, it is more appropriate to determine that the actual shape of the road marking line itself is deviated from the normal one, that is, that no foreign matter is attached to the camera 2f.

  Therefore, according to such a configuration, as in the case of the front camera 2f and the rear camera 2r, the delay time is not set for the cameras 2f and 2r that do not have an overlapping shooting area and shoot the same part of the road marking line with a time difference. Since the history comparison can be performed in consideration of the above, determination accuracy can be further improved. The delay time DT may be determined based on the vehicle speed detected by the vehicle speed sensor, external parameters (mounting position, mounting angle, etc.) of the cameras 2f, 2r, internal parameters (focal length, etc.), and the like.

  In addition to the above configuration, in the present embodiment, the in-vehicle camera system 1 further includes a front-side image recognition control unit 8f and a rear-side image recognition control unit 8r as image recognition control means, as shown in FIG. ing. These image recognition control parts 8f and 8r acquire the determination result about the corresponding vehicle-mounted cameras 2f and 2r by the abnormality determination part 7. Then, the image recognition control units 8f and 8r automatically turn off the recognition function of the corresponding road lane marking recognition units 4f and 4r when the acquired determination result indicates that an abnormality has occurred. It has become.

  The image recognition control units 8f and 8r may be embodied by causing the arithmetic processing unit to execute a program for realizing these functions.

  According to such a configuration, meaningless image processing can be prevented from being performed on the in-vehicle camera in which an abnormality has been detected.

  In addition to the above configuration, in the present embodiment, the in-vehicle camera system 1 further includes an abnormality notification unit 9 as abnormality notification means, as shown in FIG. When the abnormality determination unit 7 determines that an abnormality has occurred, the abnormality notification unit 9 notifies the user of the occurrence of an abnormality for the corresponding in-vehicle camera by outputting an image and / or sound. ing.

  According to such a configuration, the user can surely recognize the abnormality.

(Camera lens abnormality detection method)
Next, the operation of the in-vehicle camera system 1 will be described with reference to FIGS. 10 and 11 as a first embodiment of the camera lens abnormality detection method according to the present invention.

  In the present embodiment, first, in step 1 (ST1) of FIG. 10, captured images of the in-vehicle cameras 2f and 2r are acquired by the image acquisition units 3f and 3r.

  Next, in step 2 (ST2), the road lane marking recognition units 4f and 4r recognize road lane markings based on the captured image acquired in step 1 (ST1).

  Next, in step 3 (ST3), the reliability of the recognition result in step 2 (ST2) is calculated by the reliability calculation units 5f and 5r.

  Next, in step 4 (ST4), the reliability calculated in step 3 (ST3) is added to the history by the reliability history generation units 6f and 6r.

  Next, in step 5 (ST5), the abnormality determination unit 7 determines the presence or absence of lens abnormality based on the reliability history created in step 4 (ST4). If a positive determination result is obtained in step 5 (ST5), the process proceeds to step 6 (ST6). If a negative determination result is obtained, the process returns to step 1 (ST1).

  Next, in step 6 (ST6), the image recognition control units 8f and 8r automatically turn off the image recognition functions of the corresponding road lane marking recognition units 4f and 4r.

  Next, in step 7 (ST7), the abnormality notification unit 9 notifies the user of the lens abnormality, and the process ends.

  Next, the detailed processing of Step 5 (ST5) will be described. First, in Step 51 (ST51) of FIG. 11, the reliability histories for the in-vehicle cameras 2f and 2r are compared.

  Next, in step 52 (ST52), it is determined whether or not the reliability on one camera side is relatively lower than the reliability on the other camera side. In this determination, a threshold value of a difference in reliability for considering as relatively low may be set. If a positive determination result is obtained in step 52 (ST52), the process proceeds to step 53 (ST53), and if a negative determination result is obtained, the process proceeds to step 1 (ST1).

  Next, in step 53 (ST53), it is determined whether or not the state of low reliability (continuation state) has reached a predetermined period or the predetermined number of times of recognition of road lane markings. If a positive determination result is obtained in step 53 (ST53), the process proceeds to step 6 (ST6), and if a negative determination result is obtained, the process proceeds to step 1 (ST1).

[Second Embodiment]
Next, a second embodiment of the present invention will be described with reference to FIGS.

  As shown in FIG. 12, the in-vehicle camera system 1 according to the present embodiment further includes two cameras 2rs and 2ls in addition to the front camera 2f and the rear camera 2r. One of these two 2rs is a predetermined centered on the left side of the vehicle attached to the left side of the vehicle (for example, the left side mirror) in such a posture that the road surface on the left side of the vehicle is looked down from an oblique direction. The left side camera 2rs for photographing the photographing region. The other 2ls is a predetermined centered on the right side of the vehicle attached to the right side of the vehicle (for example, the right side mirror) in such a manner that the road surface on the right side of the vehicle is looked down from an oblique direction. The right-side camera 21s that shoots the photographic area.

  In the present embodiment, the history of reliability of road lane markings recognized based on the images taken by these two side cameras 2rs and 2ls is also used for determination by the abnormality determination unit 7. As a configuration for this, in the in-vehicle camera system 1 of the present embodiment, the side camera 2rs and 2ls also have the captured image acquisition unit 3, the road marking line recognition unit 4, the reliability, similarly to the front / rear cameras 2f and 2r. A calculation unit 5 and a reliability history creation unit 6 are provided.

  When the abnormality determination unit 7 determines, the reliability histories of in-vehicle cameras that can photograph the same part of the same road marking line at the same time or with a time difference according to the progress of the vehicle are compared.

  As such a combination of the on-vehicle cameras, a plurality of combinations are conceivable as shown in FIG. For example, by comparing the history of the front camera 2f and the history of the rear camera 2r, it is possible to determine the presence or absence of an abnormality after comparing and evaluating the reliability of the left and right white lines as road marking lines. Such an aspect has already been described in the first embodiment. Further, by comparing the history of the front camera 2f or the rear camera 2r and the history of the left side camera 2rs, it is possible to determine whether there is an abnormality after comparing and evaluating the reliability of the left white line. Furthermore, by comparing the history of the front camera 2f or the rear camera 2r with the history of the right side camera 21s, it is possible to determine whether there is an abnormality after comparing and evaluating the reliability of the right white line. However, the side cameras 2rs and 2ls are not compared because they cannot capture the same part of the same road marking line. Further, the reliability histories of the three cameras of the front, side, and rear may be compared and evaluated.

  According to such a configuration, since it is possible to compare the reliability histories based on the same part of the same road lane marking, the abnormality determination can be performed with high accuracy, and the camera to be determined Can be increased.

  Note that the present invention is not necessarily limited to the above-described embodiment, and various modifications may be made without departing from the characteristics of the present invention.

DESCRIPTION OF SYMBOLS 1 In-vehicle camera system 2 In-vehicle camera 6 Reliability history creation part 7 Abnormality judgment part

Claims (13)

An in-vehicle camera system comprising a plurality of in-vehicle cameras mounted on a vehicle,
A reliability history creating means for creating a history of reliability of road lane markings recognized on the basis of a photographed image of the in-vehicle camera for each of the in-vehicle cameras;
By comparing the reliability histories for each of the in-vehicle cameras created by the reliability history creating means, the in-vehicle camera in which the decrease in the reliability is relatively indicated, An in-vehicle camera system comprising: an abnormality determination unit that determines that a lens abnormality including fogging of the lens has occurred. The abnormality determination unit determines that the abnormality has occurred in the vehicle-mounted camera in which the decrease in reliability is indicated for a certain period or a predetermined number of times of recognition of the road marking line. The in-vehicle camera system described in 1. The abnormality determination unit is configured to record the reliability history of the in-vehicle cameras that can photograph the same part of the same road marking line at the same time or with a time difference according to the progress of the vehicle among the plurality of in-vehicle cameras. The vehicle-mounted camera system according to claim 1 or 2, wherein the comparison is performed. The in-vehicle camera to which the reliability history is compared by the abnormality determination unit includes a front camera that images the front of the vehicle, a side camera that images the side of the vehicle, and a rear camera that images the rear of the vehicle. The in-vehicle camera system according to claim 3, wherein there are at least two of them. The abnormality determination means is configured to record the reliability history of the front camera and the reliability history of the rear camera, and the rear camera detects the predetermined part of the road lane line after the front camera The vehicle-mounted camera system according to claim 4, wherein the comparison is performed while shifting by a delay time that is considered to be required before imaging the region. The in-vehicle camera system according to claim 5, wherein the abnormality determination unit determines the delay time based on a vehicle speed. The reliability history creating means includes:
The in-vehicle camera system according to any one of claims 1 to 6, wherein a history of the reliability calculated based on the shape of the recognized road marking line is created. 8. An image recognition control unit that automatically turns off an image recognition function corresponding to the in-vehicle camera that has been determined that the abnormality has occurred by the abnormality determination unit. The in-vehicle camera system according to any one of the above. The abnormality notification means for notifying the abnormality of the corresponding in-vehicle camera when the abnormality determination means determines that the abnormality has occurred. The in-vehicle camera system according to claim 1. A camera lens abnormality detection method for detecting lens abnormality including adhesion of foreign matter to a lens of an in-vehicle camera and / or clouding of the lens,
A first step of creating, for each in-vehicle camera, a history of reliability of road marking lines recognized based on a plurality of captured images of the in-vehicle camera;
Comparing the reliability histories for each of the in-vehicle cameras created in the first step, the lens in the in-vehicle camera in which the decrease in the reliability is relatively indicated A camera lens abnormality detection method comprising: a second step of determining. The second step is a step of determining that the abnormality has occurred in the in-vehicle camera in which the decrease in reliability is indicated for a certain period or a predetermined number of times of recognition of the road marking line. The camera lens abnormality detection method according to claim 10. In the second step, among the plurality of in-vehicle cameras, the reliability history of the in-vehicle cameras that can photograph the same part of the same road marking line at the same time or with a time difference according to the progress of the vehicle The camera lens abnormality detection method according to claim 10, wherein the camera lens abnormality detection method is a step of comparing the above. In the second step, the reliability history of the front camera as the in-vehicle camera and the reliability history of the rear camera as the in-vehicle camera are determined, and the front camera determines a predetermined part of the road marking line. The camera lens abnormality detection method according to claim 12, wherein the comparison is performed by shifting by a delay time that is considered to be required until the rear camera images the part after imaging.