JPWO2020039812A1 - Image processing device - Google Patents

Abstract

The present invention realizes a camera image processing device capable of failure diagnosis even when an image processing device is actually used. In the present invention, a certain amount of data is set as an expected value during the vertical blanking period (c) when the image data is not output from the image sensor, and the data transmission line has a short-circuit failure or a disconnection failure. Detect if it is occurring. Failure diagnosis is possible even while the image processing device is in use. Even when there are a plurality of data transmission lines, by setting the expected value to a predetermined value, it is possible to detect which transmission line has a short circuit failure or a disconnection failure.

Description

The present invention relates to an image processing apparatus.

An in-vehicle camera image processing device mounted on a vehicle is known.

In the known technology, as described in Patent Document 1, the failure diagnosis of the in-vehicle camera image processing device is expected by the failure diagnosis unit in the image processing device by outputting a test image from the image sensor to the image processing device. I was comparing with the value. Then, when some kind of failure occurs in the path from the image sensor to the image processing device, an abnormality is detected.

WO2015 / 104915

In the above-mentioned conventional technique, since a test image output from the image sensor is used as a failure diagnosis of the in-vehicle camera image processing device, it is difficult to perform a failure diagnosis while the vehicle is actually capturing the image. For this reason, the vehicle is stopped and the ignition is turned on only when the ignition is turned on.

However, if any failure occurs in the image data transmission path from the image sensor of the in-vehicle camera image processing device to the image processing circuit after the vehicle ignition is turned on, the failure detection cannot be performed and the failure can be promptly and appropriately performed. It was difficult to take proper measures.

The present invention has been made in view of the above problems, and an object of the present invention is to realize an image processing device capable of performing failure diagnosis even when an image processing device is actually used.

In order to achieve the above object, the present invention is configured as follows.

The image processing device includes an imaging unit, a data transmission path for transmitting image data captured by the imaging unit, and an image processing unit for processing image data transmitted from the data transmission path, and the image processing unit transmits data. A data input unit that inputs image data from the path and a failure diagnosis unit that outputs a failure diagnosis signal to the data transmission line and diagnoses the failure of the data transmission line during the period when the image data is not transmitted to the data transmission line. Have.

According to the present invention, it is possible to realize an image processing device capable of failure diagnosis even when the image processing device is actually used.

It is a figure for demonstrating the structure and operation of Example 1 of this invention. It is a figure for demonstrating the structure of the data transmission part between an image pickup element and image processing in the internal structure of the vehicle-mounted camera image processing apparatus in Example 1. FIG. It is a figure for demonstrating the interface part of the data communication path of the right camera image sensor and the image processing IC in Example 1. FIG. It is a figure which showed the state that the interface circuit between an image pickup element and an image processing IC is actually composed of a plurality of lines. It is a timing diagram which showed the state that the image data in Example 1 is repeatedly transmitted. It is a timing diagram which showed the state that the image data transmission line is diagnosed in the period (c) when the image data is not transmitted. It is a schematic diagram which extracted two data transmission lines to be diagnosed from the image data transmission lines at the time of detecting a short failure mode in a period (d). It is a figure which showed the signal level or voltage of each part in the diagnosis of a short failure mode. It is a figure which showed the signal level or voltage of each part at the time of a short failure mode diagnosis. It is a schematic diagram which extracted one path of the diagnosis target from the image data transmission line at the time of detecting the disconnection failure mode in the period (e). It is a figure which showed the signal level or voltage of each part in the diagnosis of a disconnection failure mode. It is a functional block diagram of a microcomputer (fault diagnosis control unit) arranged in an image processing IC. It is a figure for demonstrating the interface part of the data communication path of the right camera image sensor and the image processing IC in Example 2 of this invention. It is a figure for demonstrating the interface part of the data communication path of the right camera image sensor and the image processing IC in Example 2 of this invention. It is a figure for demonstrating the interface part of the data communication path of the right camera image sensor and the image processing IC in Example 3 of this invention. It is a figure for demonstrating the interface part of the data communication path of the right camera image sensor and the image processing IC in Example 4 of this invention. It is a figure for demonstrating the interface part of the data communication path of the right camera image sensor and the image processing IC in Example 5 of this invention.

Hereinafter, examples of the present invention will be described with reference to the drawings.

(Example 1)
FIG. 1 is a diagram for explaining the configuration and operation of the first embodiment when the present invention is applied to an in-vehicle camera image processing device.

In FIG. 1, the vehicle-mounted camera image processing device 12 of the first embodiment processes an image obtained by capturing an image of the outside from a vehicle. Then, it has a right camera imaging module (right image imaging unit) 1 attached to the vehicle to image the outside such as the front of the vehicle, and a left camera imaging module (left image imaging unit) 2. The right camera imaging module (right image imaging unit) 1 and the left camera imaging module (left image imaging unit) 2 form an imaging unit.

The right camera image pickup module 1 and the left camera image pickup module section 2 consist of an image pickup element such as a CCD image sensor or a CMOS image sensor and a circuit section that handles electronic signals, and converts the light transmitted through the lens into an electron signal for AD conversion. Is output as a digital signal after.

The in-vehicle camera image processing device 12 monitors the front in a normal vehicle running state, and the image processing module (image processing unit) captures the captured image 3 of the right camera imaging module 1 and the captured image 4 from the left camera imaging module 2. It is output to 11. In the image processing module 11, the captured images 3 and 4 are processed by the data processing unit 5 for the purpose of removing noise and sharpening the images, respectively. Specific examples of processing include adjustment of the brightness of the entire image, adjustment of the brightness of each pixel, adjustment of the gain for each brightness, and the like.

The right camera image 6 after processing by the data processing unit 5 and the left camera image 7 after processing are used as inputs for the parallax image generation unit 8. The parallax image generation unit 8 generates the parallax image 9 by combining the two image data of the right camera image 6 and the processed left camera image 7.

The parallax image 9 shows the distance information to the object in front of the vehicle by using the difference between the right camera image 6 and the left camera image 7.

The recognition processing unit 10 determines the information in front of the vehicle based on the information of the parallax image 9, determines the danger of collision and the necessity of accelerating / decelerating the vehicle, and the vehicle communication unit 18 determines the vehicle mounted on the vehicle. Notify the control unit (not shown) of the determination result. The vehicle communication unit 18 causes the display unit 19 to display the determination result.

The image processing module 11 is a module that executes digital signal processing, and the data processing unit 5, the parallax image generation unit 8, and the recognition processing unit 10 are a combination of software and hardware such as a microcomputer and a memory, or FPGA and ASIC. It is composed.

In particular, the data processing unit 5 and the parallax image generation unit 8 that process a large number of captured images at high speed are often configured by hardware.

The vehicle communication unit 18 may be a CAN bus, a LIN bus, or a FLEX-RAY.

In FIG. 1, the transmission path 13 of the right camera image captured image 3 from the right camera imaging module 1 to the data processing unit 5, the transmission path 15 of the left camera image 4 from the left camera imaging module 2 to the data processing unit 5, and data processing. Recognition from the transmission path 14 of the right camera image 6 after processing from the unit 5 to the disparity image generation unit 8, the transmission path 16 of the left camera image 7 from the data processing unit 5 to the disparity image generation unit 8, and the disparity image generation unit 8. If any one or more of the transmission paths 17 of the disparity image 9 to the processing unit 10 fails, a normal disparity image 9 may not be output.

In this case, the misalignment of the parallax image 9 causes an erroneous determination of an object in front of the vehicle and a distance determination to be erroneously transmitted to the vehicle control unit (not shown), which may reduce the accuracy of vehicle motion control.

FIG. 2 is a diagram for explaining the configuration of the data transmission portion between the image sensor and the image processing in the internal configuration of the vehicle-mounted camera image processing device according to the first embodiment.

The image transmission paths 13 and 15 from the right camera imaging module 1 and the left camera imaging module 2 shown in FIG. 1 to the data processing unit 5 can be further divided into details.

FIG. 2 is a diagram illustrating details of an image transmission path 13 from the right camera imaging module 1 to the data processing unit 5. Since the details of the image transmission path 15 from the left camera imaging module 2 to the data processing unit 5 have the same configuration as the details of the image transmission path 13 shown in FIG. 2, the details of the image transmission path 15 are illustrated and explained in detail. Is omitted.

In FIG. 2, the image transmission path 13 includes a data transmission path 23 between the right camera imaging element 20 (CMOSIC) of the right camera imaging module 1 (CMOS module) and the connector 25 of the right camera imaging module 1, and the connector 25. It is divided into a data transmission path 26 between the connector 27 of the image processing module 11 and a data transmission path 24 between the connector 27 and the image processing IC 29 of the image processing module 11.

The image processing IC 29 is shown as a representative of the data processing unit 5.

FIG. 3 is a diagram for explaining an interface portion (interface circuit) of the right camera image sensor 20 and the image processing IC 29 shown in FIG. 2 in the first embodiment, particularly the data communication path. In FIG. 3, the connectors 25 and 27 are omitted.

In FIG. 3, the right camera image sensor 20 has a digital data output buffer 30 and an output port 31 that sends the data to the data transmission line 23. Further, the image processing IC 29 has an input / output port 33 that takes in data from the data transmission line 24. The output port 31 and the input / output port 33 correspond to the data transmission lines 23, 24, 26, and the damping resistor 32 is arranged in any of the data transmission lines 23, 24, 26.

In the first embodiment, it is assumed that the data transmission line 23 is arranged.

The inside of the image processing IC 29 is connected to the input / output port 33, and has a pull-up resistor 36 for pulling up to the IO level power supply (power supply unit) 35 of the image processing IC 29. The input / output port 33 can be pulled up by controlling the connection and disconnection of the pull-up control switch (control switch) 34 by the pull-up enable signal 37.

Further, a pull-down resistor 38 is connected to the input / output port 33, and the input / output port 33 can be pulled down by controlling the pull-down control switch 39 with the pull-down enable signal 40.

When data is taken in from the input / output port 33, the voltage of the input / output port 33 is converted into a voltage signal in the image processing IC 29 by the input buffer 42, and the data is stored in the input port register 43 in synchronization with a digital clock or the like. .. The output buffer 45 connected to the input / output port 33 can switch the output to high impedance or a voltage according to the value of the output port register 44 by the output enable signal 41.

When the output enable signal 41 is off and the output of the output buffer 45 is high impedance, if both the pull-up enable signal 37 and the pull-down enable signal 40 are off, the voltage of the input / output port 33 is the output of the image pickup element 20. It is determined by the value of the buffer 30, and the digital data output of the right camera imaging element 20 is transmitted to the input port register 43.

FIG. 4 is a diagram showing how the interface circuit between the image pickup device 20 and the image processing IC 29 shown in FIG. 3 is actually composed of a plurality of circuits. In the first embodiment, the interface circuit has twelve data lines.

In FIG. 4, the right camera imaging element 20 has an output port 51 and an output port 61 in addition to the output port 31, and although not shown, nine output ports are located between the output port 51 and the output port 61. There are a total of 12 output ports. Further, an output buffer similar to the digital data output buffer 30 is connected to the 12 output ports.

Further, the image processing IC 29 has an input / output port 53 and an input / output port 63 in addition to the input / output port 33, and although not shown, nine inputs are inserted between the input / output port 53 and the input / output port 63. There are output ports, and there are a total of 12 input / output ports (data input units).

Then, as shown in FIG. 3, the output port 31 is connected to the input / output port 33 by the data line to which the damping resistor 32 is connected. Further, the output port 51 is connected to the input / output port 53 by the data line to which the damping resistor 52 is connected, and the output port 61 is connected to the input / output port 63 by the data line to which the damping resistor 62 is connected.

Similarly, the output port between the output ports 51 and 61 and the input / output port between the input / output ports 53 and 63 are connected by a data line to which a damping resistor is connected, respectively.

FIG. 5 is a timing diagram showing how the image data in the first embodiment is repeatedly transmitted, and FIG. 5 shows data transmission for four cycles.

In FIG. 5, the period (b) is one cycle of image data transmission, which is 24.26 msec in the first embodiment. The CMOS data output (image data) shown in FIG. 5C is output from the right camera image sensor 20 in synchronization with the vertical synchronization signal Vsync shown in FIG. 5A.
The period (a) at this time is the image data transmission period, which is 21.3 msec in the first embodiment.

The optical energy that is the source of the image data in the period (a) is the CMOS exposure timing ((B) in FIG. 5) that is earlier than the timing when the CMOS data output (image data) is actually output, and the right camera image sensor. It is captured between the shutter openings set to 20.

In this way, image data based on the light energy captured by the right camera image sensor 20 is periodically transmitted, but during the period (c) (vertical blanking period) when the vertical synchronization signal Vsync is low level. The output (CMOS data output) of the right camera image sensor 20 is not output as image data and is at a low level. Then, in the vertical blanking period (c), 0 V is output as the voltage of the output port 31. Therefore, the period (c) can be set as the diagnosis period.

FIG. 6 is a timing diagram showing how the image data transmission lines 23, 26, and 24 including the connectors 25 and 27 are diagnosed during the period (c) during which the image data transmission shown in FIG. 5 is not performed. Is.

In FIG. 6, the image data transmission period of the period (a) is the expected signal value of the input / output port 33 of the image processing IC 29 ((D) of FIG. 6) is the image data output from the right camera imaging element 20 (FIG. 5). (C)).

In the diagnosis period of the period (c), if no failure has occurred, the expected value (data input) shown in the period (c) of FIG. 6 (the period (c) consists of the periods (d) and (e)). (Signal state of) is the same. Therefore, the failure can be diagnosed by confirming the data of the diagnosis period (c) based on the above expected value.

In the period (d) of the diagnosis period (c), a short failure (short failure mode) of the image data transmission lines 23, 26, 24 is detected, and in the period (e), a disconnection failure (disconnection failure mode) is detected. ..

To detect the short failure mode, the diagnostic process is repeated for the number of data. In the first embodiment having 12 data lines, the diagnostic process is performed 12 times during the period (d). Therefore, in the period (d), the expected values are "100000000000000", "010000000000000", "001000000000000", ... "00000000000010", "0000000000000001" (1 is the high level, 0 is the low level). be).

Further, in the period (e), the expected value is "000000000000".

To detect the disconnection failure mode, the detection is completed by one diagnostic process in the period (e). The specific diagnostic method will be described below.

FIG. 7 is a schematic diagram in which two data transmission lines to be diagnosed are extracted from the image data transmission lines 23, 26, and 24 when the short failure mode is detected in the period (d) shown in FIG. ..

In FIG. 7, a diagnostic method for detecting the occurrence of a short circuit between two target transmission lines will be described. During the period (d) (FIG. 6) when the transmission of image data is completed, the output enable signal 41 of the input / output buffer 45 and the output buffer 85 connected to the input / output port 33 of the image processing IC 29 and the other input / output port 63. , 81 are off, and the low level output (0V) of the output buffer 30 and the output buffer 60 of the right image pickup element 20 is directly reflected in the output port 33 and the input / output port 63.

Therefore, low-level signals are taken into the input port register 43 and the input port register 83 of the input buffer 82. An output port register 84 is connected to the output buffer 85.

Here, for diagnosis, a high level is set on the output port register 44 side of the input / output port 33 side, and the high level signal is output from the output buffer 45 by turning on the output enable signal 41. Then, assuming that the IO level power supply (power supply unit) 35 (FIG. 3) of the image processing IC 29 is 2V, the voltage level of the input / output port 33 is 2V. In this way, the failure diagnosis unit, which will be described later, controls the operation of the pull-up control switch 34 during the period when the image data is not transmitted to the data transmission line 23 or the like, and the signal state of the input / output port 33 (data input unit). To set.

At this time, since the voltage levels of the signals of the input / output port 33 and the output port 31 are different, it is necessary to select the value of the damping resistor 32 so that the current value flowing through the damping resistor 32 does not matter. Assuming that the IO level power supply 35 is 2V, if a resistance value of 100Ω is selected as the damping resistor 32, 2V ÷ 100Ω, that is, a current of 20mA flows into the damping resistor 32, and the power consumption is 2V × 20mA = 20mW (=). 1 / 25W). In this case, there is no problem if a resistor having a (1/16) W withstand capability is used for the damping resistor 32.

At this time, when the signal level is fetched from the input buffer 42, since the input / output port 33 is set to 2V, the high level signal is fetched into the input port register 43.

In diagnosing the short failure mode, the input port register 43 should have a high level and the input port register 83 should have a low level, and this is used as an expected value for diagnosis. That is, since the initial expected value of the period (d) of FIG. 6 is "100000000000000", if a short failure has not occurred between the two lines shown in FIG. 7, the input port register 83 also has a low level "100000000000000". "0" should be captured as detection data.

FIG. 8 is a diagram showing the signal level or voltage of each part in the diagnosis of the short failure mode.

Here, assuming that the input / output ports 33 and 63 are short-circuited with each other, the voltage of the input / output port 63 is also 2V due to the output buffer 45.

Therefore, unlike the expected value, a high-level signal is taken into the input port register 83, whereby a short-circuit failure can be detected. For example, the expected value is "100000000000000" and the detected value is "100000000001", which are different values from each other.

FIG. 9 is a diagram showing the signal level or voltage of each part at the time of short failure mode diagnosis. Here, if the short-circuit failure portion 90 occurs at the output ports 31 and 61 on the right camera image sensor 20 side, it cannot be detected by the same algorithm, so the range covering the short-circuit failure detection is the damping resistance 32. , 62, note that it is on the side of the transmission line on the image processing IC 29 side.

In FIGS. 7, 8 and 9, only the state of the two data transmission lines is schematically shown, but as in the first embodiment, when there are 12 data transmission lines, each transmission line is referred to as “ If "100000000000000" (1 is high level, 0 is low level) is set to be the expected value, the data transmission line that enables the output and sets 1 to the expected value, and the remaining 11 data transmission lines other than that. Can detect short-circuit failures.

By setting "010000000000000" as the next data setting and "001000000000000" as the next data setting and repeating the diagnostic process 12 times, it is possible to detect each other's short failure of 12 data transmission lines. It becomes.

In the first embodiment, the diagnosis process is performed 12 times in order to consider the failure mode of all the data transmission line pairs. However, if it is considered that a short-circuit failure can occur only in the adjacent data transmission lines, " It may be completed by two diagnostic processes of two patterns of "101010101010" and "010101010101". In this case, the data transmission line having an expected value of 1 is switched to the high level output, and the output enable signal of the data transmission line having an expected value of 0 is left off.

FIG. 10 is a schematic diagram in which one path to be diagnosed is extracted from the image data transmission lines when the disconnection failure mode is detected in the period (e) shown in FIG.

With reference to FIG. 10, a diagnostic method for detecting when a disconnection failure occurs somewhere in the data transmission line will be described.

After the above-mentioned short mode failure diagnosis is completed, in the period (e), the output enable signal 41 of the output buffer 45 connected to the input / output port 33 of the image processing IC 29 is turned off, and the output buffer of the right camera imaging element 20 is turned off. The low level output (0V) of 30 is directly reflected in the input / output port 33.

After that, the pull-up resistor 36 connected to the input / output port 33 is connected to the IO level power supply 35 of the image processing IC 29 by the pull-up enable signal 37. As the value of the pull-up resistor 36, select one that is sufficiently lower than the resistance value of the damping resistor 32, such as 1 kΩ.

In this way, the voltage of the input / output port 33 is the voltage of the IO level power supply 35 divided by the pull-up resistor 36 and the damping resistor 32. That is, 2V × {100Ω / (1kΩ + 100Ω)} = about 0.18V, and when this is taken in by the input buffer 42, the expected value of the input port register 43 becomes a low level.

Here, if the disconnection mode fails at any of the output port 31, damping resistor 32, and input / output port 33 in the middle of the data transmission line, the input / output port 33 is fixed at the pulled-up voltage, that is, 2V. Will be done. Therefore, if the voltage of the input / output port 33 is taken in by the input buffer 42 at this time, the value of the input port register 43 becomes a high level.

Since this is different from the expected value that was at the low level, it is possible to detect that a disconnection failure has occurred. FIG. 11 is a diagram showing a signal level or voltage of each part in the diagnosis of the disconnection failure mode.

In FIG. 11, a disconnection mode failure occurs in the transmission line between the damping resistor 32 and the input / output port 33 (disconnection failure location 91). At this time, the value of the input port register 43 connected to the input buffer 42 is at a high level.

In FIGS. 10 and 11, only the state of one data path is schematically shown, but when there are 12 data transmission paths as in the first embodiment, each data is set to “000000000000” (1 is high). If the pull-down resistance is set so that the level (0 is the low level) is the expected value, the expected value of the part where the disconnection failure occurs becomes 1, so any of the 12 data paths has a disconnection failure. It is possible to detect whether it is occurring.

FIG. 12 is a functional block diagram of a microcomputer (fault diagnosis control unit) 50 arranged in the image processing IC 29 (a typical example of the data capable unit 5). However, the microcomputer 50 is omitted in FIG. 3 and the like.

In FIG. 12, the microcomputer 50 includes a diagnostic data setting unit 501, a detection data input unit 502, a comparison unit 503, and a failure determination unit 504.

The diagnostic data setting unit 501 sets the failure diagnosis data (fault diagnosis data) so that the output enable signals such as the output enable signals 42 and 81 and the output port register 44 and the like become the data at the time of failure diagnosis as described above. Output and set the data transmission lines 23, 26, 24.

The detection data input unit 502 inputs the failure diagnosis data stored in a register such as the input port register 83 at the time of diagnosis as a failure diagnosis signal.

The comparison unit 503 compares the failure diagnosis data set by the diagnosis data setting unit 501 with the failure diagnosis data (detection data) input by the detection data input unit 502, and outputs the comparison result to the failure determination unit 504.

From the result output from the comparison unit 503, the failure determination unit 504 determines whether or not a failure has occurred in the image data transmission lines 23, 26, 24, and if a failure has occurred, it is a short failure or a disconnection. It is determined whether the failure occurs or which data transmission line the failure has occurred, and the determination result is output to the vehicle communication unit 18.

The vehicle communication unit 18 performs appropriate processing based on the output result from the failure determination unit 504. When the image data transmission lines 23, 26, and 24 have a failure, the vehicle communication unit 18 may display the failure occurrence part and the type of failure (short failure or disconnection failure) on the display unit 19. can.

The pull-up connection switch 34, IO level power supply 35, pull-up resistor 36, pull-down resistor 38, pull-down control switch 39, input buffer 42, input port register 43, output port register 44, output buffer 45, and microcomputer (fault diagnosis). The control unit) 50 forms a failure diagnosis unit.

Here, when the port level of the input / output port 33 is operated by the output port register 44 and the pull-up resistor 35 as in the first embodiment, the terminal voltage is changed until the terminal voltage actually changes to the changed voltage. It takes time to charge the floating capacity.

Normally, the terminal capacitance of an IC such as the image processing IC 29 is about 10 pF, and when 1 kΩ is selected as the pull-up resistor, it is sufficient to wait about 100 nsec as the time until switching.

That is, even if a wait time of about 100 nsec is inserted between each diagnosis and about 10 usec is taken for each diagnosis, the time required for the period (d) and the period (e) is 1 msec or less.

In the first embodiment, the period (c) in which the vertical synchronization signal (A) without image data transmission is at a low level is 2.96 msec, which is sufficiently diagnosable. In the first embodiment, the short circuit failure diagnosis is followed by the disconnection failure diagnosis, but the order may be reversed. At this time, it should be noted that the time until each terminal is switched differs between the short-circuit failure diagnosis and the disconnection failure diagnosis.

As in the first embodiment, port level switching and wait time insertion can be realized by the software installed in the image processing IC 29, but when the period (c) shown in FIG. 6 or the like is short or the software If the configuration does not provide sufficient speed, the above diagnostic contents may be implemented in hardware.

As described above, according to the first embodiment of the present invention, a certain amount of data is set as an expected value during the vertical blanking period (c) when the image data is not output from the image pickup element, and the data transmission path is set. Since it is configured to detect whether a short-circuit failure has occurred or a disconnection failure has occurred, it is possible to realize an image processing device capable of diagnosing the failure even during the imaging operation of the image processing device. ..

Further, according to the first embodiment, even when there are a plurality of data transmission lines, by setting the expected value to a predetermined value, it is detected which transmission line has a short circuit failure or a disconnection failure. be able to.

Although the first embodiment has been described by taking the right camera image sensor 20 as an example, the same diagnostic operation as described above can be performed on the left camera image sensor.

(Example 2)
Next, Example 2 of the present invention will be described.

13 and 14 are diagrams for explaining an interface portion (interface circuit) of the data communication path of the right camera image sensor 20 and the image processing IC 29 in the second embodiment of the present invention.

In the second embodiment, the data output level on the right camera image sensor 20 side during the low level of the vertical synchronization signal (vertical blanking period (c)) is higher than that of the first embodiment. In the first embodiment, as shown in FIG. 5C, the data output level on the right camera image sensor 20 side is low.

FIG. 13 is a diagram showing a signal level or voltage of each part when diagnosing the short failure mode in the second embodiment.

In the example shown in FIG. 13, the output of the input / output port 33 switched to the output as compared with the first embodiment is set to a low level, and the expected values of the input port registers 43 and 83 are inverted from the first embodiment accordingly. (The input port register 43 is inverted from high to low, and the input port register 83 is inverted from low to high).

FIG. 14 is a diagram showing a signal level or voltage of each part when diagnosing the disconnection failure mode in the second embodiment.

In the second embodiment, it is low level to connect to the input / output port 33 at the time of disconnection failure diagnosis. That is, the pull-down enable signal 40 is turned on, the pull-down control switch 39 is turned on, and the ground is grounded. The expected value of the input register 43 at this time becomes a high level, which is inverted from the expected value of the input register 43 in the case of the first embodiment.

Other configurations and operations are the same as those in the first embodiment, and the same configurations as in the first embodiment are designated by the same reference numerals and detailed description thereof will be omitted.

Even if the data output level on the right camera image sensor 20 side is high during the vertical synchronization signal low level (vertical blanking period (c)), based on the same concept as in the first embodiment. It is possible to realize a camera image processing device capable of failure diagnosis even while the image processing device is in use.

That is, in the second embodiment in which the specifications of the right camera image sensor 20 are different from those of the first embodiment, it is possible to realize a camera image processing device capable of performing failure diagnosis even while the image processing device is in use.

Although the right camera image sensor 20 has been described as an example in the second embodiment as in the first embodiment, the same diagnostic operation as described above can be performed on the left camera image sensor.

(Example 3)
Next, Example 3 of the present invention will be described.

FIG. 15 is a diagram for explaining an interface portion (interface circuit) of the data communication path of the right camera image sensor 20 and the image processing IC 29 in the third embodiment of the present invention.

In FIG. 15, the pull-up resistor 356 is arranged outside the image processing IC 29 and is always connected to the IO power supply 355.

In the first embodiment, the pull-up resistor 36 is turned on and off by turning the pull-up control switch 34 on and off, but the pull-up resistor 356 is used in the image data transmission period (a) and the short failure detection period (d), respectively. If the resistance value is large enough not to interfere with the signal level of (for example, 3.3 kΩ or more), it is not necessary to control the on / off of the pull-up resistor 356.

Therefore, in the third embodiment, the pull-up resistor is not arranged inside the image IC 29, but the external part pull-up resistor 356 of the image IC 29 is arranged.

Other configurations and operations are the same as those in the first embodiment, and the same configurations as in the first embodiment are designated by the same reference numerals and detailed description thereof will be omitted.

According to the third embodiment, it is not necessary to control the pull-up resistor on / off, so that the control operation can be simplified.

However, in the case of adopting the configuration of the third embodiment, if a disconnection failure occurs in the input / output port 33 portion, it should be noted that the value of the input register 43 cannot be determined and may not be detected.

Other than that, Example 3 has the same effect as that of Example 1.

Although the third embodiment has been described by taking the right camera image sensor 20 as an example as in the first embodiment, the same diagnostic operation as described above can be performed on the left camera image sensor.

(Example 4)
Next, Example 4 of the present invention will be described.

FIG. 16 is a diagram for explaining an interface portion (interface circuit) of the data communication path of the right camera image sensor 20 and the image processing IC 29 in the fourth embodiment of the present invention.

The fourth embodiment is an example applied when the image processing IC 29 has a configuration in which the image data input port cannot be switched to the output port.

In the fourth embodiment, the input-only port 433 and the output-only port (failure signal output unit) 457 that can be switched to high impedance are arranged in the image processing IC 29, and the voltage of the input port 433 is generated by using the output-only port 457. Control and perform short fault diagnosis.

Other configurations and operations are the same as those in the first embodiment, and the same configurations as in the first embodiment are designated by the same reference numerals and detailed description thereof will be omitted.

In addition to being able to obtain the same effects as in Example 3, the present invention can be applied to an image processing IC having a configuration in which the image data input port cannot be switched to the output port.

Although the right camera image sensor 20 has been described as an example in the fourth embodiment as in the first embodiment, the same diagnostic operation as described above can be performed on the left camera image sensor.

(Example 5)
Next, Example 5 of the present invention will be described.

FIG. 17 is a diagram for explaining an interface portion (interface circuit) of the data communication path of the right camera image sensor 620 and the image processing IC 29 in the fifth embodiment of the present invention.

In the fifth embodiment, a transmission path for transmitting a high-speed signal is assumed as a data transmission path. In this case, since a signal having a small voltage level that cannot be handled at a general IO port level is used for high-speed transmission, for example, an input of an AD converter is used in failure diagnosis in addition to the data transmission port.

The right camera image pickup element 620 transmits image data from the transmitter 600 to the receiver 603 of the image processing IC 29 via the output port 601 and the damping resistor 602 and the high-speed data transmission line 633. The image data transmitted to the receiver 603 is processed by the high-speed transmission signal processing circuit 606.

Since the high-speed transmission line processing circuit 606 specializes in high-speed data transmission, it is difficult to capture data in time with individual voltage changes for each voltage switch of the high-speed data transmission line 633 under diagnosis. In the fifth embodiment, there is no problem because the voltage of the high-speed data transmission path 633 under diagnosis is taken in by using the high-speed image data input port (diagnosis data input unit) 610 that makes it easy to match the diagnosis timing.

Other configurations and operations are the same as those in the first embodiment, and the same configurations as those in the first and fourth embodiments are designated by the same reference numerals and detailed description thereof will be omitted.

In addition to being able to obtain the same effects as in Example 4, the present invention can be applied to an image processing IC that performs high-speed signal transmission.

Although the right camera image sensor 20 has been described as an example in the fifth embodiment as in the first embodiment, the same diagnostic operation as described above can be performed on the left camera image sensor.

Further, according to the present invention, it is possible to perform failure diagnosis even when the image processing apparatus is not performing an image capturing operation.

The present invention is not limited to Examples 1 to 5 described above, and includes various modifications. For example, Examples 1 to 5 described above have been described in detail in order to explain the present invention in an easy-to-understand manner, and are not necessarily limited to those having all the described configurations. Further, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Further, it is possible to add / delete / replace other configurations with respect to a part of the configurations of Examples 1 and 2.

In addition, the control lines and information lines indicate those that are considered necessary for explanation, and do not necessarily indicate all the control lines and information lines in the product.

Further, the present invention can be applied not only to an in-vehicle image processing device but also to an image processing device in another moving body such as a robot.

1 ... right camera imaging module unit, 2 ... left camera imaging module unit, 3 ... right camera imaging image, 4 ... left camera imaging image, 5 ... data processing unit, 6 ... Right camera image after processing, 7 ... Left camera image after processing, 8 ... Disparity image generation unit, 9 ... Disparity image, 10 ... Recognition processing unit, 11 ... Image processing module, 12 ... In-vehicle camera image processing device, 13 ... Right camera image transmission path, 14 ... Right camera image transmission path after processing, 15 ... Left camera image transmission path, 16 ...・ Left camera image transmission path after processing, 17 ... Disparity image transmission path, 18 ... Vehicle communication unit, 19 ... Display unit, 20, 620 ... Right camera image pickup element, 23, 24, 26 ... Data transmission path, 25, 27 ... Connector, 29 ... Image processing IC, 30, 60 ... Digital data output buffer, 31, 51, 61, 601 ... Output port, 32 , 52, 62, 602 ... Damping resistance, 33, 53, 63 ... Input / output port, 34 ... Pull-up connection switch, 35 ... IO level power supply, 36, 356 ... Pull-up resistance , 37 ... Pull-up enable signal, 38 ... Pull-down resistor, 39 ... Pull-down control switch, 40 ... Pull-down enable signal, 41, 81 ... Output enable signal, 42, 82 ... Input Buffer, 43, 83 ... Input port register, 44, 84 ... Output port register, 45, 85 ... Output buffer, 50 ... Microcomputer (fault diagnosis control unit), 90 ... Short failure Location, 91: Disconnection detection location, 355: IO power supply, 433: Input-only port, 457: Output-only port (failure signal output section), 501: Diagnostic data setting section, 502・ ・ ・ Detection data input unit, 503 ・ ・ ・ Comparison unit, 504 ・ ・ ・ Failure judgment unit, 600 ・ ・ ・ Transmitter, 606 ・ ・ ・ High-speed transmission signal processing circuit, 610 ・ ・ ・ High-speed image data input port ( Diagnostic data input unit), 633 ... High-speed data transmission line

Claims (12)

Imaging unit and
A data transmission path for transmitting image data captured by the imaging unit, and
An image processing unit that processes the image data transmitted from the data transmission line, and
The image processing unit provides a data input unit for inputting the image data from the data transmission line, and a failure diagnosis signal to the data transmission line during a period when the image data is not transmitted to the data transmission line. An image processing apparatus including a failure diagnosis unit that outputs data and diagnoses a failure of the data transmission line. In the image processing apparatus according to claim 1,
A damping resistor is arranged in the data transmission line.
The image processing unit has a power supply unit and a control switch for connecting and disconnecting the power supply unit and the data input unit.
The failure diagnosis unit is characterized in that it controls the operation of the control switch, sets the signal state of the data input unit, and diagnoses the failure of the data transmission line based on the signal state of the data input unit. Image processing device. In the image processing apparatus according to claim 1,
The failure diagnosis unit sets the data input unit to the failure diagnosis state during the period when the image data is not transmitted to the data transmission line, and the failure of the data transmission line is based on the signal state of the data input unit. An image processing procedure characterized by diagnosing. In the image processing apparatus according to claim 2,
An image processing procedure characterized in that the failure diagnosis unit connects or disconnects the control switch during a period in which the image data is not transmitted to the data transmission path. In the image processing apparatus according to claim 3,
An image processing procedure comprising a plurality of the data transmission lines and the plurality of data input units, wherein the failure diagnosis unit sets each of the plurality of data input units to a failure diagnosis state. In the image processing apparatus according to claim 4,
A plurality of the data transmission lines, a plurality of the data input units, and a plurality of the control switches are provided, and the failure diagnosis unit is characterized in that connection and disconnection are set for each of the plurality of control switches. Image processing treatment. In the image processing apparatus according to any one of claims 2, 3, 4, 5, and 6.
The failure diagnosis unit sets the failure diagnosis data as the failure diagnosis signal and outputs the diagnosis data setting unit, the detection data input unit for inputting the detection data to detect the signal state of the data input unit, and the above. A comparison unit that compares the failure diagnosis data set by the diagnostic data setting unit with the detection data input by the detection data input unit, and a failure that determines a failure of the data transmission line based on the comparison by the comparison unit. An image processing procedure comprising a failure diagnosis control unit having a determination unit. In the image processing apparatus according to claim 1,
A damping resistor is arranged in the data transmission line, and includes a power supply unit connected to the data transmission line and a pull-up resistor connected between the data transmission line and the power supply unit. An image processing procedure comprising diagnosing a failure of the data transmission line based on a signal state of an input unit. In the image processing apparatus according to claim 8,
An image processing procedure characterized in that the failure diagnosis unit includes a failure signal output unit that outputs the failure diagnosis signal to the data transmission line. In the image processing apparatus according to claim 1,
A damping resistor is arranged in the data transmission line, and includes a power supply unit connected to the data transmission line and a pull-up resistor connected between the data transmission line and the power supply unit.
The image processing unit has a diagnostic data input unit for detecting the signal state of the data transmission line.
An image processing procedure characterized in that the failure diagnosis unit includes a failure signal output unit that outputs the failure diagnosis signal to the data transmission line. In the image processing apparatus according to any one of claims 1 to 10.
The imaging unit has a right image imaging unit and a left image imaging unit, and the image processing unit is based on the difference between the image data captured by the right image imaging unit and the left image imaging unit. An image processing procedure comprising a parallax image generator that generates a parallax image. In the image processing apparatus according to any one of claims 1 to 11.
The image processing device is an image processing procedure characterized in that it is mounted on a vehicle.

Images