WO2016027408A1

WO2016027408A1 - Image processing apparatus, and failure diagnosis method for image processing apparatus

Info

Publication number: WO2016027408A1
Application number: PCT/JP2015/003554
Authority: WO
Inventors: 後藤　宏明; 丙辰王
Original assignee: 株式会社デンソー
Priority date: 2014-08-22
Filing date: 2015-07-14
Publication date: 2016-02-25
Also published as: JP6350113B2; JP2016045716A

Abstract

An image processing apparatus (1) comprises: an image adjusting unit (21) which adjusts distortion of each camera image captured by a camera (3), and outputs an adjusted image, which is an image in which said distortion has been adjusted, and which has at least one vanishing point; and a failure diagnosis unit (25) which determines whether or not the image adjusting unit has failed, on the basis of whether or not there is an abnormality in the vanishing point in the adjusted image output from the image adjusting unit (21).

Description

Image processing apparatus and fault diagnosis method for image processing apparatus

Cross-reference of related applications

This application is based on Japanese Application No. 2014-169555 filed on August 22, 2014, the contents of which are incorporated herein by reference.

The present disclosure relates to an image processing device and a failure diagnosis method for the image processing device.

There is a known method for detecting abnormalities such as adhesion of foreign matter to the camera based on the luminance distribution of the captured camera image when the periphery of the vehicle is captured by the in-vehicle camera (see, for example, Patent Document 1).

JP 2003-312408 A

In the abnormality diagnosis based on the luminance distribution as described above, whether the abnormality of the camera image is abnormal due to an external environmental factor (for example, adhesion of foreign matter to the camera) or due to a malfunction of the device (for example, malfunction of hardware or software). It is difficult to distinguish whether it is abnormal.

For example, when an image processing apparatus that corrects distortion of a camera image taken by a camera fails, correcting the camera image with such an image processing apparatus may cause an abnormal luminance distribution in the corrected image. There is. However, when such an abnormality occurs, it is difficult to determine that the image processing apparatus is out of order using the conventional technology.

An object of the present disclosure is to provide a technique capable of diagnosing that the image processing apparatus is out of order.

According to one aspect of the present disclosure, the image processing apparatus includes an image correction unit and a failure diagnosis unit. The image correction unit corrects distortion of at least one camera image captured by the camera, and outputs a corrected image that is an image in which the distortion is corrected and has at least one vanishing point. The failure diagnosis unit determines whether or not the image correction unit has failed based on whether or not the vanishing point in the corrected image output from the image correction unit is abnormal.

According to this configuration, when the image correction unit fails and the distortion correction cannot be performed correctly, the vanishing point cannot be correctly obtained from the corrected image corrected by such an image correction unit. Using this, it is determined that the image correction unit is out of order.

Therefore, unlike a technique that does not focus on whether or not the vanishing point is correctly obtained from the corrected image, it is possible to specify that some kind of failure has occurred at the stage of generating the corrected image in the image correcting unit. Therefore, for example, it is possible to notify the user that there is an abnormality in the image correction unit that generates such a corrected image. Alternatively, for example, it is possible to take measures such as stopping the control based on such an abnormal corrected image.

According to another aspect of the present disclosure, a failure diagnosis method for an image processing apparatus corrects distortion of a camera image captured by a camera, and is an image in which the distortion is corrected, and includes at least one vanishing point. A failure of the image correction unit is determined for an image processing apparatus having an image correction unit that outputs a correction image that is an image having the. The failure diagnosis method for the image processing apparatus includes a procedure for determining whether or not the image correction unit is defective based on whether or not the vanishing point in the corrected image output from the image correction unit is abnormal. .

Therefore, according to such a failure diagnosis method, the operation and effect as described for the above-described image processing apparatus are achieved, and it is determined that some kind of failure has occurred at the stage of generating a corrected image in the image correction unit. it can.

The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description with reference to the accompanying drawings.
FIG. 1 is a block diagram showing the configuration of an in-vehicle image processing system Figure 2 Flow chart of real-time failure detection processing FIG. 3 is a flowchart of the first image abnormality determination process. FIG. 4 is a flowchart of the second image abnormality determination process. FIG. 5 is a flowchart of start-up failure detection processing. FIG. 6 is an explanatory diagram showing an example in which the corrected image is divided into a plurality of segments.

Next, the image processing apparatus and the failure diagnosis method for the image processing apparatus will be described with reference to exemplary embodiments.

[Configuration of image processing apparatus]
As shown in FIG. 1, an in-vehicle image processing system mounted on a vehicle described below is mainly configured by an electronic control unit 1 (Electronic Control Unit; hereinafter abbreviated as ECU 1) that functions as the above-described image processing device. Is done. In addition to the ECU 1, the camera 3, an external factor determination device 5, a display 7, a diagnosis processor 9, a vehicle speed sensor 11, a rudder angle sensor 13, and an attitude sensor 15 are provided. A vehicle equipped with an in-vehicle image processing system is also referred to as a host vehicle.

The camera 3 is mounted on the vehicle and is configured to be able to photograph the front, rear, or side of the vehicle. The camera 3 includes a wide angle lens. Therefore, each image photographed by the camera 3 (hereinafter also referred to as a camera image) has a distortion that causes the degree of reduction of the subject to increase in the peripheral portion compared to the central portion of the photographing range. One camera image can be provided as a still image. On the other hand, a plurality of camera images can be provided as moving images. Furthermore, it can be said that a still image corresponds to one frame. On the other hand, it can be said that a moving image corresponds to a plurality of frames provided at a predetermined frame rate.

The external factor determination device 5 is configured by hardware and software that can input and output images and information other than images from the camera 3. The external factor determination unit 5 is configured to be able to determine whether or not the camera 3 is functioning normally as hardware. When the camera 3 is functioning normally, the camera 1 input from the camera 3 is processed by the ECU 1. To output.

The display 7 is configured to be able to display various types of information to a vehicle driver or the like, for example, using a liquid crystal display device or the like. The diagnosis processor 9, the vehicle speed sensor 11, the rudder angle sensor 13, and the attitude sensor 15 are configured to be able to communicate with various devices including the ECU 1 via a CAN (Controller Area Network) 17.

Various diagnostic information is transmitted from the various devices including the ECU 1 to the diagnosis processor 9. The diagnosis processor 9 executes a process for storing such information as diagnosis information. Further, the diagnosis processor 9 performs processing for alerting the driver of the vehicle and the like, such as turning on a warning lamp as necessary.

The vehicle speed sensor 11 detects the speed of the vehicle and transmits the detected speed to the ECU 1 via the CAN 17. The steering angle sensor 13 detects the steering angle of the steering, and transmits the detected steering angle to the ECU 1 via the CAN 17. The attitude sensor 15 includes a gyro, a yaw rate sensor, a gradient sensor, and the like. The attitude sensor 15 includes vehicle attitude information (for example, a rotation angle (yaw) with a direction perpendicular to the traveling surface of the vehicle as an axis, and a traveling direction of the vehicle as an axis. Rotation angle (roll), rotation angle (pitch) etc. with the vehicle width direction of the vehicle as an axis) is detected, and the detected attitude information is transmitted to the ECU 1 via the CAN 17.

The ECU 1 includes an image correction unit 21, a recognition unit 23, a failure diagnosis unit 25, and the like.

Portions

21, 23, 25 are also referred to as

devices

21, 23, 25 or

modules

21, 23, 25, respectively. A camera image input from the camera 3 to the ECU 1 via the external factor determination unit 5 is subjected to distortion correction processing in the image correction unit 21. In the following description, an image subjected to the distortion correction processing in the image correction unit 21 is also referred to as a corrected image. The corrected image output from the image correction unit 21 is input to each of the recognition unit 23 and the failure diagnosis unit 25. For example, the ECU 1 can be configured as a microcomputer including a memory such as a ROM and a RAM, a CPU (Central Control Unit), and an input / output interface. In this case, the memory stores programs for various processes executed by components such as the image correction unit 21, the recognition unit 23, or the failure diagnosis unit 25. On the other hand, the ECU 1 may configure a part or all of each component as hardware without using a program. A memory is also referred to as a memory, or even a non-transitional tangible medium.

The recognition unit 23 executes processing for recognizing various targets based on the corrected image input from the image correction unit 21. For example, a white line recognition process for recognizing a marking line on a road, a sign recognition process for recognizing a sign installed on the road, and the like are executed based on a corrected image obtained by photographing the front of the vehicle. Alternatively, for example, based on a corrected image in which the rear of the vehicle is captured, a rear recognition process for recognizing a target existing behind the vehicle, or on the vehicle side based on a corrected image in which the side of the vehicle is captured. Side recognition processing for recognizing a target existing on the side is executed. And if these processes predict the occurrence of a lane departure or contact with an obstacle, for example, warning information to that effect is output to the display 7 to the driver of the vehicle. Call attention.

The failure diagnosis unit 25 executes a failure diagnosis process for the image correction unit 21 based on the corrected image input from the image correction unit 21. Details of this failure diagnosis processing will be described later. When the external factor determination unit 5 determines that the camera 3 is not functioning normally, information to that effect is transmitted from the external factor determination unit 5 to the failure diagnosis unit 25.

The failure diagnosis unit 25, when any failure is detected in at least one of the information transmitted from the external factor determination unit 5 and the result of the failure diagnosis process performed by the failure diagnosis unit 25, information to that effect Is transmitted to the diagnosis processor 9 via the CAN 17.

[Failure diagnosis processing]
Next, failure diagnosis processing executed by the above-described system will be described based on the flowcharts shown in FIGS. The failure diagnosis process described below can include a real-time failure detection process shown in FIGS. 2 to 4 and a startup failure detection process shown in FIG. The real-time failure detection process is a process for detecting a failure of the image correction unit 21 in real time while the vehicle is traveling. The start-up failure detection process is a process for detecting a failure in the image correction unit 21 at the time of vehicle start-up or factory shipment.

First, the real-time failure detection process will be described. The real-time failure detection process is a process that starts with the start of the vehicle and is repeatedly executed thereafter until the vehicle stops. When the real-time failure detection process is started, first, a still image or a moving image as a plurality of camera images provided by the camera 3 is input (S10). In S10, a moving image may be input at a predetermined frame rate or a still image may be input regularly or irregularly according to a failure determination method described later. However, in this embodiment, the description is continued on the assumption that a moving image is input at a predetermined frame rate, and when necessary, a plurality of images constituting the moving image are also used as still images.

When a still image or a moving image is input in S10, the external factor determination unit 5 executes external factor determination (S20). In S20, the abnormal state of the camera 3 itself is determined. For example, when the camera 3 is not functioning normally, such as a power supply short circuit or a signal abnormality, the external factor determination unit 5 determines that such an abnormality exists. This makes it possible to determine whether an abnormality has occurred in the image correction unit 21 or whether an abnormality has occurred in the camera 3 in the previous stage of the image correction unit 21.

In S20, when the camera 3 is functioning normally, the image correction unit 21 performs image correction (S30). In S30, the image correction unit 21 corrects distortion caused by the above-described wide-angle lens. When the image correction unit 21 functions normally, the image corrected by the image correction unit 21 is an image having at least one vanishing point.

For this reason, when there is an object to be photographed that approaches the host vehicle (for example, a target on the road shoulder in front of the vehicle), the object to be photographed is time-lapsed from the vanishing point to the image periphery in the corrected image. Appear as a pixel group (hereinafter, this pixel group is also referred to as a moving element). In addition, when there is a photographing object that moves away from the host vehicle (for example, a target on the road shoulder behind the vehicle), the photographing object moves from the image periphery to the vanishing point in the corrected image over time. It appears as a pixel group (hereinafter, this pixel group is also referred to as a moving element).

Furthermore, there is a subject to be photographed that extends linearly in the depth direction of the photographing region of the camera (for example, a white line on the road surface that extends linearly in the traveling direction of the host vehicle or a guard rail on the shoulder). In this case, when the edge of the photographing object is extracted by image processing, the extracted edge is a pixel group extending linearly toward the vanishing point in the corrected image (hereinafter, this pixel group is also referred to as a linear element). appear.

Therefore, the failure diagnosis unit 25 executes an image abnormality determination process based on the corrected image input from the image correction unit 21 (S40). In S40, the failure diagnosis unit 25 extracts the moving elements and the linear elements as described above, and determines whether or not the vanishing point is correctly obtained.

For example, in S40, the first image abnormality determination process illustrated in FIG. 3 is executed for the moving element as described above. In this image abnormality determination process, first, a feature point estimated as an image of a moving object is calculated by comparing frames before and after among a plurality of images constituting a moving image (S110). The number of feature points calculated in S110 may be any number as long as it is a number according to the calculation performance of the ECU 1, but it is preferable that there are several or more points in order to improve reliability. Subsequently, the moving direction from the previous frame is calculated for the feature point calculated in S110 (S120), and the convergence point of the moving vector of the feature point is calculated as a vanishing point (S130).

When several moving elements are extracted from the corrected image, it is possible to specify the direction in which the vanishing point is estimated to exist in association with each moving element based on the moving direction of each moving element. In S130, the direction in which the vanishing point exists for each moving element is estimated by the above-described method, and it is determined whether or not the estimated vanishing point direction converges to a predetermined position. In this determination, in consideration of noise and errors during image processing, if the vanishing point converges within a predetermined range, it may be determined that the image correction unit 21 is functioning normally.

In S110 to S130, as shown in FIG. 6, the entire corrected image is divided into a plurality of segments (nine in FIG. 6), and the convergence point is the segment in the center of the corrected image (effective segment shown in FIG. 6). Anything that deviates from may not be used. For example, when the vehicle is traveling on a curved road, the convergence point may not converge near the center of the correction image.

Therefore, in the curve, an area (for example, a segment other than the effective segment shown in FIG. 6) where a moving element expected to not converge at the center of the corrected image may appear on the corrected image in advance. Such moving elements may be excluded from the determination targets. Further, when it is estimated that the vehicle is traveling on a curved road based on information obtained from the vehicle speed sensor 11, the rudder angle sensor 13, the attitude sensor 15, and the like, the vanishing point calculation is temporarily suspended. You may comprise as follows.

In addition, when a disturbance such as sudden vibration or impact is detected based on vehicle state information or suspension load information obtained from the vehicle speed sensor 11, the steering angle sensor 13, the attitude sensor 15, etc., Are excluded from the target of failure determination.

If the vanishing point does not converge within a predetermined range as a result of obtaining the convergence point of the feature points by the procedure as described above, the image correction unit 21 diagnoses an image abnormality of the corrected image (S140), and the diagnosis processor 9 is notified of the abnormality determination result (S150).

In S40 described above, for example, the second image abnormality determination process illustrated in FIG. 4 can be executed for the linear element as described above. In this image abnormality determination process, first, image processing is performed on each of a plurality of images constituting a moving image to extract edges (linear elements) (S210). Then, of the extracted edges, an edge that can be used for vanishing point calculation is determined (S220). In S220, as an edge that can be used for vanishing point calculation, there is a length that can specify the extending direction of the edge, and its position and orientation are substantially within a predetermined allowable range in relation to the vanishing point. Is selected.

Subsequently, the convergence point is calculated as the vanishing point from the edge determined to be usable in S220 (S230). In S230, the direction in which the vanishing point is estimated is specified based on the direction in which the extracted edge extends. Also in S210 to S230, it is preferable to exclude elements that are expected not to converge to the center of the screen by a curve, such as a white line on the road surface, by the method using the effective segment as described above. Also in S210-S230, when a disturbance such as sudden vibration or impact is detected, it is preferable to exclude an image in the meantime from a failure determination target.

If the vanishing point does not converge within a predetermined range as a result of obtaining the convergence point of the feature point, the image correction unit 21 diagnoses an image abnormality of the corrected image (S240), and the diagnosis processor 9 is abnormal. The determination result is notified (S250).

2, when the above-described image abnormality determination process (the process illustrated in FIGS. 3 and 4) is executed, the failure diagnosis unit 25 subsequently performs external factor determination (S <b> 50). In S50, a temporal abnormality such as a secular change of the mounting position of the camera 3 or road surface unevenness is determined. As a result, even if the vanishing point cannot be calculated in S40, if any abnormality is detected in S50, it can be determined that the failure of the image correction unit 21 is not immediately determined. For example, even if the vanishing point is not calculated only between specific frames, if the vanishing point is calculated after that, it may be caused by a temporary abnormality such as road surface unevenness, so that the failure of the image correction unit 21 Do not judge. Alternatively, if the vanishing point is calculated at an unexpected position, but the state continues steadily, there is a possibility that the attachment position of the camera 3 may change over time, so that it is not determined that the image correction unit 21 is out of order. To.

And after S50 is complete | finished, the failure diagnosis part 25 determines the hardware abnormality in the image correction part 21 (S60). In S60, even if an exceptional case is excluded in the determination in S50 or the like, if the state in which the abnormality is detected continues for a certain time or more, it is determined that the image correction unit 21 is out of order. In that case, a failure diagnosis result is notified to the diagnosis processor 9 (S70).

Next, start-up failure detection processing will be described. The start-up failure detection process is executed only once as an initial process when the vehicle is started. Alternatively, when the above-described real-time failure detection process is employed, the start-up failure detection process may be executed as a process that is specially executed at the time of vehicle shipment or maintenance inspection. Good.

When starting failure detection processing is started, first, the image correction unit 21 inputs an evaluation image stored in the memory 25A (S310). That is, in the above-described real-time failure detection process, each camera image is input from the camera 3 in S10. Instead, in S310, an evaluation image stored in the memory 25A is input. The image for evaluation is an image having the same distortion as that taken by the camera 3.

After S310, the image correction unit 21 performs image correction (S320), and the failure diagnosis unit 25 determines a hardware abnormality in the image correction unit 21 based on the corrected image (S330). In the memory 25A, in addition to the evaluation image described above, a correction image that should be obtained when the image correction unit 21 is normal is stored in advance as a reference image. In S330, the matching between the corrected image obtained in S320 and the reference image stored in the memory is determined. If the images are not equivalent, it is determined that the image correction unit 21 has a hardware abnormality. When a hardware abnormality is detected in the image correction unit 21, a failure diagnosis result is notified to the diagnosis processor 9 (S340).

[effect]
As described above, according to the on-vehicle image processing system, when the image correction unit 21 is out of order and the distortion correction cannot be performed correctly, the failure diagnosis unit 25 is corrected by the image correction unit 21. Based on the fact that the vanishing point cannot be correctly obtained from the corrected image, it is determined that the image correction unit 21 is malfunctioning.

Therefore, unlike the technique that does not focus on whether or not the vanishing point is correctly obtained from the corrected image, at the stage of generating the corrected image in the image correction unit 21 (that is, at a stage different from the abnormality of the camera 3). ), It can be identified that some kind of failure has occurred. Therefore, for example, it is possible to notify the user that there is an abnormality in the image correction unit 21 that generates such a corrected image. Alternatively, for example, it is possible to take measures such as stopping the control based on such an abnormal corrected image.

Further, when the failure diagnosis unit 25 includes a moving element that is estimated to be an image of the moving object to be photographed in S110 to S150, whether or not the vanishing point can be estimated from the moving state of the moving element. Based on the above, it is possible to determine whether or not the image correction unit 21 is out of order. Therefore, the vanishing point can be calculated from the movement of the feature points between frames using the moving image, and the failure of the image correction unit 21 can be determined.

In addition, the failure diagnosis unit 25 includes a linear element that is presumed to be an image of an imaging target extending linearly in the depth direction of the imaging region by the camera 3 in S210 to S250. Based on whether or not the vanishing point can be estimated from the direction in which the linear elements extend, it is possible to determine whether or not the image correction unit 21 has failed. Therefore, it is possible to calculate the vanishing point by extracting the edge of the still image and determine the failure of the image correction unit 21.

As shown in FIG. 6, the failure diagnosis unit 25 selects a part of the range included in the corrected image as an effective segment, and determines whether or not the vanishing point in the corrected image is abnormal in the selected range. Based on this, it is possible to determine whether or not the image correction unit 21 is out of order. Therefore, even when the vehicle is traveling on a curve, erroneous recognition that there is a vanishing point at an unexpected location can be suppressed.

The in-vehicle image processing system is mounted on the vehicle together with the camera 3, and the image correction unit 21 corrects the distortion of the camera image taken by the camera 3 while the vehicle is moving to correct the distortion. The failure diagnosis unit 25 is configured to be able to output an image, and the image correction unit 21 determines whether the vanishing point in the corrected image output from the image correction unit 21 is abnormal while the vehicle is moving. It is configured to be able to determine whether or not a failure has occurred. Therefore, the failure of the image correction unit 21 can be detected in real time while the vehicle is traveling.

In the on-vehicle image processing system, vehicle state information and suspension load obtained from the vehicle speed sensor 11, the steering angle sensor 13, and the attitude sensor 15 (which correspond to an example of a detector) mounted on the host vehicle. If the ECU 1 (that is, the failure diagnosis unit 25) determines that a disturbance that is a force applied to the host vehicle or the camera 3 such as sudden vibration or impact is detected based on the information or the like, the disturbance is added. The image in between is excluded from the target of failure determination. Therefore, when the image is greatly shaken by the vibration of the vehicle, the image can be removed from the determination target, thereby improving the accuracy of failure diagnosis.

The in-vehicle image processing system includes a memory 25A (corresponding to an example of a storage device) that stores an evaluation image prepared in advance for failure diagnosis. Through S310-S340, the image correction unit 21 stores the memory 25A. Is corrected so as to be able to output a diagnostic correction image in which the distortion is corrected. The failure diagnosis unit 25 is configured to output the diagnostic correction image output from the image correction unit 21. A failure of the image correction unit 21 can be determined based on the abnormal state. Therefore, at the time of factory inspection or start-up, it is possible to carry out a failure diagnosis using an evaluation image prepared in advance, and in addition to the real-time failure detection processing, such as the secular change of the camera 3 and the running state of the vehicle The influence can be eliminated and more accurate fault diagnosis can be performed.

[Other Embodiments]
As described above, the image processing apparatus and the failure diagnosis method for the image processing apparatus have been described with reference to exemplary embodiments. However, the present disclosure is not limited to the above-described exemplary embodiments, and various forms are possible. Can be adopted.

For example, in the above-described embodiment, an example is shown in which both the real-time failure detection process and the start-up failure detection process are employed, but only one of them may be employed.

In the above embodiment, the correction image is divided into nine segments at a predetermined position. However, the division position and the number of divisions can be appropriately set, and are not limited to those illustrated.

In the above embodiment, for convenience of illustration, the memory 25A is illustrated as a configuration provided in the failure diagnosis unit 25 for convenience. However, any memory that can be accessed by the image correction unit 21 and the failure diagnosis unit 25 may be used. The memory provided in the image correction unit 21 may be a memory in the ECU 1 that is independent from the image correction unit 21 and the failure diagnosis unit 25. Alternatively, as long as the ECU 1 has a configuration capable of accessing a memory outside the ECU 1, a memory outside the ECU 1 may be used.

In the above embodiment, the in-vehicle image processing system is exemplified, but an equivalent system other than the vehicle may be mounted.

Further, in addition to the above-described image processing apparatus and image processing apparatus failure diagnosis method, a system including the image processing apparatus as a constituent element, a program including instructions for causing a computer to function as the image processing apparatus (also referred to as a program product) Or a program (also referred to as a program product) that includes instructions for providing a fault diagnosis method, a computer-readable ROM, RAM, or other non-transitional tangible medium that stores each of these programs, etc. The present disclosure can also be realized in various forms.

Although the present disclosure has been described with reference to the embodiments, it is understood that the present disclosure is not limited to the embodiments and structures. The present disclosure includes various modifications and modifications within the equivalent range. In addition, various combinations and forms, as well as other combinations and forms including only one element, more or less, are within the scope and spirit of the present disclosure.

Claims

An image correcting unit (21, 21) that corrects distortion of at least one camera image captured by the camera (3) and outputs a corrected image that is an image in which the distortion is corrected and has at least one vanishing point. S30)
A failure diagnosis unit (25, S40-S60) that determines whether or not the image correction unit is out of order based on whether or not the vanishing point in the corrected image output from the image correction unit is abnormal. And an image processing apparatus.
The image processing apparatus according to claim 1,
The image correcting unit is configured to output a plurality of the corrected images by correcting distortion of each of the plurality of camera images included in a moving image photographed by the camera.
When the failure diagnosis unit includes a moving element estimated as an image of a moving object to be photographed as a feature element included in the plurality of corrected images, the disappearance from the moving state of the moving element Based on whether or not the point can be estimated, it is possible to determine whether or not the image correction unit is out of order (S110 to S140).
Image processing device.
The image processing apparatus according to claim 1 or 2,
The failure diagnosis unit includes, as characteristic elements included in the corrected image, a linear element that is estimated to be an image of an object to be imaged that extends linearly in the depth direction of the imaging region of the camera. In this case, it is possible to determine whether or not the image correction unit is out of order based on whether or not the vanishing point can be estimated from the direction in which the linear element extends (S210 to S240).
Image processing device.
The image processing apparatus according to any one of claims 1 to 3,
The failure diagnosis unit selects a part of the range included in the corrected image, and based on whether the vanishing point in the corrected image is abnormal in the selected range, the image correction unit It is configured to be able to determine whether or not a failure has occurred (S110-S130, S210-S230).
Image processing device.
An image processing apparatus according to any one of claims 1 to 4, comprising:
The image processing apparatus is mounted on a moving body including at least a vehicle together with the camera,
The image correction unit is configured to be able to output a corrected image in which the distortion is corrected by correcting distortion of each camera image captured by the camera while the moving body is moving,
The failure diagnosis unit determines whether the image correction unit has failed based on whether or not the vanishing point in the corrected image output from the image correction unit is abnormal while the moving body is moving. It is configured to be able to determine whether or not (S10-S70)
Image processing device.
An image processing apparatus according to any one of claims 1 to 5, comprising:
In cooperation with a detector capable of detecting disturbances affecting the shooting by the camera,
The failure diagnosis unit is configured to exclude the camera image taken by the camera from the determination target when it is determined that the disturbance is detected by the detector (S110-S130, S210-S230). )
Image processing device.
The image processing apparatus according to any one of claims 1 to 6,
A storage device (25A) for storing an evaluation image prepared in advance for failure diagnosis;
The image correction unit is configured to be able to correct distortion of the evaluation image stored in the storage unit and output a diagnostic correction image in which the distortion is corrected (S320),
The failure diagnosis unit is configured to be able to determine a failure of the image correction unit based on an abnormal state of the diagnostic correction image output from the image correction unit (S330).
Image processing device.
An image having an image correction unit (21) that corrects distortion of at least one camera image captured by the camera and outputs a corrected image that is an image in which the distortion is corrected and that has at least one vanishing point. A method of determining a failure of the image correction unit for a processing device (1),
A failure diagnosis of an image processing apparatus including a procedure for determining whether or not the image correction unit is out of order based on whether or not the vanishing point in the corrected image output from the image correction unit is abnormal Method.