JPH11126248A - Image processing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform self-diagnosis of a failure of a wire harness and an image pickup device or an inconvenient state on image processing within this system. SOLUTION: A synchronizing signal separator 11 divides an image signal that is sent from an image pickup device 2 to an image processor 4 into a video signal and a synchronizing signal, the synchronizing signal is used for timing control of image processing by a timing controller 16 and on the other hand, it is used as a reset signal of a clock counter 17. The counter 17 always measures elapsed time since a horizontal synchronizing signal is inputted, and if the horizontal synchronizing signal is no more inputted to the counter 17 by any chance because of the occurrence of the disconnection and short-circuit of a cable 3 or the abnormality of coming off of a connector, etc., the count value of the counter 17 becomes set value or more and an output of a comparator 18 becomes ON. As a result, an alarm 19 is operated to send a warning and notifies an operator of abnormality occurrence.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an image processing system for detecting an abnormality in a wire harness or an image sensor or an inconvenient situation in image processing by self-diagnosis.

[0002]

2. Description of the Related Art In recent years, in vehicles such as automobiles, a vehicle ahead or an obstacle is detected, a danger of colliding with them is determined, and a warning is issued to a driver, or a brake is automatically operated. Techniques such as stopping or automatically increasing / decreasing the running speed to keep the distance to the preceding vehicle safe have been actively developed. There is an image recognition technology based on a captured image.

As an example in which this image recognition technique is applied to an automobile, the applicant of the present invention disclosed in Japanese Patent Laid-Open No. Hei 5-265471 processes an image picked up by a stereo camera mounted on a vehicle to obtain a road shape, a preceding road shape, and the like. A technique has been proposed that measures the three-dimensional position of a vehicle, an obstacle, or the like, determines the possibility of collision or contact with a preceding vehicle or an obstacle, and issues a warning to provide driving assistance.

[0004]

By the way, in general, a device for handling an image is often made on the assumption that a person finally sees the image.
Since a human can ultimately determine a failure, it often does not have a special failure diagnosis function.

On the other hand, the above-described image processing apparatus mounted on a vehicle processes a measured image to notify a driver of a danger of contact or collision with an obstacle or to send a signal to a device for controlling the vehicle. The purpose is to send the image, and it is not assumed that the image is viewed directly by a human.Therefore, there is a problem with the image processing such as a defective wire harness or image sensor, dirt or clouding of the lens, or insufficient lighting at night. Occurs,
You cannot see the image directly to confirm.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an image processing system capable of performing a self-diagnosis inside a system of a defect of a wire harness or an image pickup device or a situation which is inconvenient in image processing. The purpose is.

[0007]

According to the first aspect of the present invention,
In an image processing system that separates an image signal output from an imaging device into a synchronization signal and a video signal, and processes the separated video signal, each time a synchronization signal separated from the image signal is input, Means for measuring elapsed time;
Means for determining an abnormality when the elapsed time exceeds a set time.

According to a second aspect of the present invention, in an image processing system for separating an image signal output from an imaging device into a synchronization signal and a video signal and processing the separated video signal, the amplitude of the image signal is kept constant. It is characterized by comprising means for holding with a time constant, and means for judging an abnormality when the hold value of the amplitude of the image signal becomes equal to or less than a set value.

According to a third aspect of the present invention, in an image processing system for separating an image signal output from an imaging device into a synchronization signal and a video signal and processing the separated video signal, the luminance of one screen of the video signal is obtained. It is characterized by comprising means for measuring a maximum value, and means for judging an abnormality when the maximum luminance value is equal to or less than a set value.

According to a fourth aspect of the present invention, there is provided an image processing system for separating an image signal output from an image pickup apparatus into a synchronizing signal and a video signal, and processing the separated video signal. It is characterized by comprising means for measuring the maximum value of the amount of change, and means for judging an abnormality when the maximum value of the amount of change in luminance is equal to or less than a set value.

According to a fifth aspect of the present invention, in an image processing system for separating an image signal output from an imaging device into a synchronization signal and a video signal and processing the separated video signal, the luminance of one screen of the video signal is obtained. Means for measuring a maximum value and a minimum brightness value, means for calculating a difference between the maximum brightness value and the minimum brightness value, and means for determining an abnormality when the difference is equal to or less than a set value. Features.

According to a sixth aspect of the present invention, there is provided an image processing apparatus for separating each image signal output from an image pickup apparatus having a plurality of image pickup elements into a synchronizing signal and a video signal, respectively, and processing the separated video signals. In the system, a unit for measuring a maximum luminance value of one screen of a video signal obtained from each image sensor, and at least one of a maximum luminance value of one screen of a video signal obtained from each image sensor is a first signal. Means for determining an abnormality when at least one is equal to or greater than the set value and equal to or less than the second set value.

According to a seventh aspect of the present invention, there is provided an image processing apparatus for separating each image signal output from an image pickup apparatus having a plurality of image pickup elements into a synchronizing signal and a video signal, respectively, and processing the separated video signals. In the system, the means for measuring the maximum value of the luminance change amount of one screen of the video signal obtained from each image sensor, and the maximum value of the luminance change amount of one screen of the video signal obtained from each image sensor, When at least one is equal to or more than the first set value and at least one is equal to or less than the second set value, a means for determining an abnormality is provided.

According to an eighth aspect of the present invention, there is provided an image processing apparatus for separating each image signal output from an image pickup apparatus having a plurality of image pickup devices into a synchronizing signal and a video signal, respectively, and processing the separated video signals. In the system, means for measuring a maximum luminance value and a minimum luminance value of one screen of a video signal obtained from each image sensor, respectively, and a maximum luminance value and a minimum luminance value of one screen of a video signal obtained from each image sensor. And a difference between a maximum luminance value and a minimum luminance value of one screen of a video signal obtained from each image sensor, at least one of which is equal to or more than a first set value and at least one of which is equal to a second one. Means for determining an abnormality when the value is equal to or less than the set value of.

The ninth aspect of the present invention is the third aspect of the present invention.
The invention according to any one of 5, 6, 7, and 8, wherein an alarm is issued when the state determined to be abnormal continues for the set number of screens.

The invention according to claim 10 is the third or fourth invention.
In the invention according to any one of 5, 6, 7, and 8, an alarm is issued when a state determined as abnormal continues for a set time.

The eleventh aspect of the present invention relates to the first, second, and third aspects.
In the invention described in any one of 3, 4, 5, 6, 7, and 8, the imaging device outputs an image signal of a stereo image pair obtained by imaging a subject from different viewpoints.

That is, in the image processing system according to the present invention, the image signal output from the imaging device is separated into a synchronizing signal and a video signal, and when the separated video signal is processed, occurrence of an abnormality is determined by self-diagnosis. In this self-diagnosis,
Each time a synchronization signal separated from the image signal is input, the elapsed time after input is measured, and when the elapsed time exceeds the set time, it may be determined that there is an abnormality, or the amplitude of the image signal is fixed. May be determined as abnormal when the hold value of the amplitude becomes equal to or smaller than the set value.
Further, the maximum luminance value of one screen of the video signal may be measured, and when the maximum luminance value is equal to or less than the set value, it may be determined that there is an abnormality.

Further, the maximum value of the luminance change amount of one screen of the video signal is measured, and when the maximum value of the luminance change amount is equal to or less than the set value, a self-diagnosis for judging an abnormality may be performed. A self-diagnosis may be made in which a luminance maximum value and a luminance minimum value of one screen are measured to calculate a difference between the luminance maximum value and the luminance minimum value, and when the difference is equal to or less than a set value, an abnormality is determined.
In this case, it is desirable to issue an alarm when the state determined to be abnormal continues for the set number of screens or for the set time.

When there are a plurality of image sensors, the maximum luminance value of one screen of the video signal is measured for each image sensor, and at least one of the maximum luminance values of one screen of each video signal is the first luminance.
When at least one is equal to or less than the second set value and is equal to or less than the second set value, it may be determined to be abnormal. The maximum value of the amount of change in luminance of one screen of the video signal is measured for each image sensor, and 1
An abnormality may be determined when at least one of the maximum values of the luminance change amount of the screen is equal to or more than the first set value and at least one is equal to or less than the second set value. Furthermore, the luminance maximum value and the luminance minimum value of one screen of the video signal are measured for each image sensor, and the difference between the luminance maximum value and the luminance minimum value is calculated. At least one of the differences from the minimum luminance value is equal to or greater than the first set value and at least one.
If one of them is equal to or less than the second set value, it may be determined that the abnormality is abnormal.
Also in this case, it is desirable to issue an alarm when the state determined to be abnormal continues for the set number of screens or for the set time.

The above-described self-diagnosis function can be applied to an image processing system having an image pickup device that outputs an image signal of a pair of stereo images obtained by picking up an object from different viewpoints, thereby improving the processing accuracy.

[0022]

Embodiments of the present invention will be described below with reference to the drawings. 1 to 4 show a first embodiment of the present invention.
FIG. 1 is a configuration diagram of an image processing system, and FIG.
Is an explanatory view showing an example of mounting on a vehicle, FIG. 3 is a waveform diagram of each part,
FIG. 4 is a flowchart of the abnormality determination routine.

In FIG. 1, reference numeral 1 denotes an imaging device 2;
This is an image processing system including an image processing device 4 connected to the image capturing device 2 via a cable 3. The image capturing device 2 includes one or more image capturing systems according to processing contents. The image processing device 4 includes a number of image processing systems corresponding to the number of imaging systems of the imaging device 2.

That is, the image pickup apparatus 2 includes, as one image pickup system, a lens 5 constituting an optical system, an image pickup device 6 for converting an image formed by the lens 5 into an electric signal, , An image signal (composite video signal) obtained by synthesizing a signal (video signal) output from the amplifier 7 and a synchronization signal to the image processing apparatus 4 via the cable 3 A signal synthesizer 8 is provided, and further, a timing signal generator 9 for generating a timing signal for controlling the image sensor 6 and the amplifier 7 of each system, and a synchronizing signal for synchronizing with the image processing device 4 are generated. A synchronization signal generator 10 is provided.

On the other hand, the image processing device 4 includes a synchronization signal separator 11 for separating the image signal from the imaging device 2 into a synchronization signal and a video signal, as one image processing system.
An amplifier 12 for amplifying the video signal separated by the synchronizing signal separator 11, and an AD converter 1 for converting an analog video signal amplified by the amplifier 12 into digital image data.
3. An image memory 14 for storing digital image data is provided, and the image data stored in the image memory 14 is processed by the video processor 15. Further, the image processing device 4 performs a synchronization control of the AD converter 13, the image memory 14, and the video processor 15 of each system based on the synchronization signal separated by the synchronization signal separator 11. A controller 16 is provided.

Further, the image processing apparatus 4 has a self-diagnosis function for detecting a system abnormality and issuing an alarm. The self-diagnosis function includes a time counter 17 that is reset by the synchronization signal separated by the synchronization signal separator 11 and measures the elapsed time after the input of the synchronization signal pulse. The value of the time counter 17 and the set value VREF1 Is provided for each image processing system, and the presence or absence of occurrence of an abnormality is monitored for each image processing system.

FIG. 2 shows an example in which a system using two image pickup devices 6 and 6 as a pair, such as a stereo image processing system, is mounted on a vehicle 20. In this example, a stereo camera 21 including two cameras is mounted as the imaging device 4, and an image captured by the stereo camera 21 is processed by an image recognition device 22 as an image processing device 4 to perform tertiary processing of an object outside the vehicle. The original position is obtained, and the possibility of contact or collision is determined by recognizing the preceding vehicle or obstacle.
Then, when it is determined that there is a danger of contact or collision, a warning is issued to the driver by a warning lamp 23 corresponding to the warning device 19, and the recognition result is passed to another device (not shown) to automatically perform braking. Activate and stop,
Alternatively, control is performed such as automatically increasing or decreasing the traveling speed so as to keep the inter-vehicle distance with the preceding vehicle safe.

In the image processing system 1 having the above configuration, the imaging light is periodically converted from the imaging device 6 into an electric signal in the imaging device 2 in synchronization with the timing signal from the timing signal generator 9 and output. Is done. Then, as shown in FIG. 3, the video signal from the image pickup device 6 amplified by the amplifier 7 is combined with the horizontal synchronization signal and the vertical synchronization signal from the synchronization signal generator 10 by the synchronization signal synthesizer 8 to obtain an image signal. Output to the image processing device 4 via the cable 3.

The synthesizing of the video signal and the synchronizing signal is performed for each scanning line with the horizontal synchronizing signal and for each screen with the vertical synchronizing signal.
5 usec, about 16.7 msec for the vertical synchronization signal. The vertical synchronizing signal is generated by a general method of combining a plurality of horizontal synchronizing signals having different pulse widths into a specific pattern.

The image signal sent to the image processing device 4 is separated by the synchronization signal separator 11 into a video signal and a synchronization signal. The timing controller 16 is required for image processing such as AD conversion timing in the AD converter 13, data writing timing in the image memory 14, and data reading timing from the image memory 14 by the video processor 15 based on the separated synchronization signal. Then, the separated video signal is amplified by the amplifier 12, converted into digital data by the AD converter 13, and sent to the video processor 15 via the image memory 14. Then, the video processor 15 performs processing such as image recognition.

The synchronizing signal separated by the synchronizing signal separator 11 is used for timing control by the above-mentioned timing controller 16 and is used as a reset signal of the clock counter 17. The time counter 17 always measures the elapsed time from the input of the horizontal synchronizing signal, and the count value is sent to the comparator 18 and the comparator 18 compares the count value with the set value VREF1. In a normal state, a horizontal synchronizing signal is input to the clock counter 17 at regular intervals, so that the count value of the clock counter 17 is not reset before reaching the set value VREF1 or more, and does not become the set value VREF1 or more. Is kept in the OFF state, and the alarm 19 does not operate.

Here, if an abnormality such as disconnection or short-circuit of the cable 3 or disconnection of the connector occurs and the horizontal synchronizing signal is not inputted to the time counter 17, FIG.
As shown in the figure, the count value of the time counter 17 becomes equal to or more than the set value VREF1, and the output of the comparator 18 is turned on. As a result, the alarm 19 is activated and an alarm is issued to notify the driver of the occurrence of the abnormality.

It should be noted that the abnormal time counter 17 and the comparator 1
The monitoring of the synchronization signal by 8 may be performed by software processing by the video processor 15, in which case the alarm 19 is connected to the output port of the video processor 15. In the abnormality determination by the video processor 15, both functions of the clock counter 17 and the comparator 18 can be realized by the video processor 15. However, in the abnormality determination routine shown in FIG. The function of the device 18 is substituted by the video processor 15.

In this abnormality determination routine, first, in step S10, the count value is read from the time counter 17 and the elapsed time since the horizontal synchronizing signal is input is checked. In step S11, the elapsed time corresponding to the count value is determined. It is determined whether or not the time is equal to or longer than a set time corresponding to the set value VREF1. If the elapsed time has not reached the set time, the process proceeds from step S11 to step S12 to turn off the alarm output.
After the state, the process returns to step S10 and the time counter 17
The process of reading the count value of is continued.

On the other hand, an abnormality such as disconnection or short-circuit of the cable 3 for transmitting the image signal from the imaging device 2 or disconnection of the connector occurs, so that the horizontal synchronizing signal is not inputted to the clock counter 17, and the count value of the clock counter 17 is reduced. When the time exceeds the set time, steps S11 to S13 are performed.
To turn on the alarm output to activate the alarm 19,
Notify the driver of the occurrence of an abnormality.

As a result, disconnection or short-circuit of the cable
The connector can self-diagnose an error such as a deviation inside the system and notify the driver of the error, thereby enabling quick repair work.

5 to 7 relate to a second embodiment of the present invention. FIG. 5 is a block diagram of an image processing system, FIG. 6 is a waveform diagram of each part, and FIG. 7 is a flowchart of an abnormality determination routine.

In the above-described first embodiment, when the elapsed time from the input of the synchronization signal exceeds the set time, it is determined that an abnormality has occurred. On the other hand, in the present embodiment, the amplitude of the image signal is reduced. Hold is performed, and when the hold value attenuates below a set value within a predetermined time, it is determined that an abnormality has occurred.

For this reason, the image processing system 1A of the present embodiment
As shown in FIG. 5, the configuration related to the abnormality diagnosis of the image processing apparatus 4 of the first embodiment is changed to an image processing apparatus 4A,
Instead of the time counter 17, an amplitude detector 25 connected in parallel to the synchronizing signal separator 11 to which an image signal is input from the imaging device 2 is employed, and the output of the amplitude detector 25 is compared with the set value VREF2 by the comparator 18. Is to be compared with.

In a system having a plurality of image pickup devices 6, a plurality of sets of the amplitude detector 25, the comparator 18, and the alarm device 19 are prepared in the image processing device 4A.

The amplitude detector 25 holds the peak value of the amplitude of the image signal (composite video signal including the synchronizing signal) output from the imaging device 2, and the held value is attenuated by a predetermined time constant. Is set to In a normal state, an image signal is input from the imaging device 2 at a constant interval, so that the amplitude hold value in the amplitude detector 25 does not become lower than the set value VREF2 of the comparator 18. In the present embodiment, the comparison relationship in the comparator 18 is opposite to that in the first embodiment, and when the amplitude hold value of the amplitude detector 25 is equal to or less than the set value VREF2 of the comparator 18,
Is kept in the OFF state, and the alarm 19 does not operate.

On the other hand, when an abnormality such as disconnection or short-circuit of the cable 3 or disconnection of the connector occurs and the image signal is no longer input to the image processing device 4A, as shown in FIG. Decay,
Eventually, the value becomes equal to or less than the set value VREF2. Then, the output of the comparator 18 is turned on, the alarm 19 is activated, and an alarm is issued to notify the driver of the abnormality.

As in the first embodiment, the monitoring of the image signal by the amplitude detector 25 and the comparator 18 can be performed by connecting the alarm device 19 to the output port of the video processor 15 and performing software processing by the video processor 15. It is possible.

FIG. 7 shows an abnormality determination routine in which the output value of the amplitude detector 25 is read into the video processor 15 to determine an abnormality. In step S20, the amplitude hold value of the image signal is read from the amplitude detector 25. At S21,
It is checked whether the read amplitude hold value is equal to or less than the set value VREF2. Then, the amplitude hold value of the image signal becomes the set value V
If it exceeds REF2, the process proceeds from step S21 to step S22 to turn off the alarm output, and then returns to step S20 to read the amplitude hold value from the amplitude detector 25 and continue the same processing.

On the other hand, if an abnormality such as disconnection or short circuit of the cable 3 or disconnection of the connector occurs, the image signal is not input, and the amplitude hold value of the amplitude detector 25 is attenuated. The above steps
Proceeding from S21 to step S23, the alarm output is turned on, and the alarm 19 is operated to notify the driver of the occurrence of the abnormality.

In this embodiment as well, similar to the first embodiment, the self-diagnosis function can detect an abnormality such as a disconnection or short-circuit of a cable, or a connector being displaced.

8 to 12 relate to a third embodiment of the present invention. FIG. 8 is a block diagram of an image processing system. FIGS. 9 and 10 are flowcharts of a basic routine for determining an abnormality.
FIG. 12 is a flowchart of an abnormality determination routine for an image processing system having two image sensors.

In the present embodiment, the maximum luminance value (the brightest value) of the video signal of one screen is measured, and when the maximum luminance value becomes equal to or less than the set value, it is determined that there is an abnormality. That is, in an in-vehicle image pickup device, an image pickup scene is usually captured as a video signal. Therefore, the maximum luminance value of the video signal is relatively large in a normal state. The state in which the maximum luminance value is small (dark state) is continuous. This change in characteristics appears more conspicuously in an imaging device having an exposure adjustment function.

In this case, continuation of a state in which the maximum luminance value of the video signal is small may occur when a night scenery or the like without illumination is captured. In such a case, normal image recognition processing is performed. Can be used as a warning to urge the driver to turn on the headlight.

As shown in FIG. 8, in the image processing device 4B in the image processing system 1B of the present embodiment, an alarm 19 is connected to the output port of the video processor 15, and the video processor 15 executes the normal image processing in parallel. The maximum luminance value of one screen is measured, and an abnormality is determined.

This abnormality determination processing is basically performed by the routine shown in FIGS. In this routine, first, the capture of the video signal into the image memory 14 is permitted in step S30, and the memory value of the maximum luminance value is initialized to 0 in step S31. Then, in step S32, the image memory 14 of the pixel to be read first is read out. Set the address of

Next, the process proceeds to step S33, where the image memory 1
When one pixel of image data is read from Step 4, Step S34
Then, it is checked whether or not the image data read this time exceeds the current maximum luminance value. When the brightness does not exceed the maximum value, the process jumps from step S34 to step S36.
Then, the process proceeds to step S35 to update the image data read this time as a new maximum brightness value, and then proceeds to step S36.

In step S36, it is determined whether or not the data processing for one screen has been completed. If the data processing for one screen has not been completed, the flow advances to step S37 to store the image in the image memory 14 for the pixel to be read next. The address is set, and the process returns to step S33 to repeat the same processing.

When the data processing for one screen is completed, the process proceeds from step S36 to step S38 to check whether the maximum luminance value is equal to or smaller than the set value CR. Then, the value of the determination counter that counts the number of screens in which the state of the maximum luminance value ≦ the set value CR is continuous is cleared to 0, and the process proceeds to step S41. S4
Proceeding to 0, 1 is added to the above-mentioned judgment counter and step S41
Proceed to.

In step S41, it is checked whether or not the value of the determination counter has exceeded the set number. If the value has not reached the set number, the alarm output is turned off in step S42. And the same process is repeated.

At this time, if the cable 3 is disconnected or short-circuited,
A situation where image recognition processing cannot be performed normally due to an abnormality of the image sensor 6 or defective lighting continues, and a state where the maximum luminance value of one screen is equal to or smaller than the set value CR is continuously added without clearing the determination counter. If the value is equal to or more than the set number of times, the process proceeds from step S41 to step S43 to turn on the alarm output, activate the alarm device 19, and notify the driver of the occurrence of the abnormality.

In an image processing system having a plurality of image sensors, the abnormality determination basic routine is performed for each video signal obtained from each image sensor, and an abnormality is determined for each image sensor. But for example, 2
In the case of a stereo image processing system or the like using the two imaging devices 6 and 6 as a pair, more stable diagnosis can be performed by performing the abnormality determination processing described below.

FIGS. 11 and 12 show two image pickup devices 6,
6 is a routine for judging abnormality with respect to one set of stereo image signals, wherein the maximum luminance value of a video signal of one screen input from one of the image sensors is equal to or more than a set value CR1, and the other image sensor Is the set value CR2 (where CR1> CR
2) When the following is true, and the output states of the respective image sensors are unbalanced, it is determined to be abnormal. In the following description, an image from one image sensor is referred to as a right image, and an image from the other image sensor is referred to as a left image.

In this routine, in step S50, the capture of the video signal into the image memories 14, 14 is permitted, and in steps S51 to S57, step S3 of the routine of FIG.
When the same processing as 1 to S37 is performed on the right image to obtain the maximum luminance value of one screen in the right image, in step S58,
The memory value of the maximum luminance value of the left image is initialized to 0, and the same processing is performed on the left image in steps S59 to S64 to obtain the maximum luminance value of one screen in the left image.

After obtaining the maximum luminance value of one screen in the right image and the maximum luminance value of one screen in the left image,
In step S65, first, the luminance maximum value of the right image is set to the set value C
Check whether it is greater than R1. If the luminance maximum value of the right image is greater than the set value CR1, the process advances from step S65 to step S66 to check whether the maximum luminance value of the left image is equal to or less than the set value CR2. In step S70, the flow branches to step S70 to clear the determination counter to 0. When the maximum luminance value of the left image ≦ the set value CR2, that is, the maximum luminance value of the right image> the set value CR1, and the maximum luminance value of the left image ≦ When the value is the set value CR2, it is determined that there is an abnormality, and the routine proceeds to step 69, where 1 is added to the determination counter.

On the other hand, if the maximum luminance value of the right image is equal to or less than the set value CR1 in step S65, the process branches from step S65 to step S67 to determine whether the maximum luminance value of the left image is larger than the set value CR1. Find out. When the maximum luminance value of the left image ≦ the set value CR1, the process branches from step S67 to the above-described step S70, and the maximum luminance value of the left image>
When the value is the set value CR1, the process proceeds from step S67 to step S68 to check whether or not the maximum luminance value of the right image is equal to or less than the set value CR2.

As a result, if the maximum luminance value of the right image> set value CR2 in step S68, the aforementioned step
The process branches to S70, where the determination counter is cleared to 0, and when the maximum brightness value of the right image ≦ the set value CR2, that is, the maximum brightness value of the left image> the set value CR1 and the maximum brightness value of the right image ≦ the set value CR2. Sometimes, it is determined to be abnormal
Proceed to 69 to add 1 to the judgment counter.

Thereafter, the process proceeds from step S69 or step S70 to step S71 to check whether or not the value of the determination counter has exceeded the set number. If not, the alarm output is turned off in step S72. After the state, the flow returns to step S50 to instruct the image memories 14, 14 to capture the next video signal, and the same processing is repeated.

The situation where image recognition processing cannot be performed normally due to disconnection or short-circuit of the cable 3 on either side, abnormality of the image sensor, or defective lighting continues.
The maximum luminance value of the video signal of one screen input from one of the two image sensors 6 and 6 is the set value C.
R1 or more, the maximum luminance value of the video signal of one screen input from the other image sensor is set to the set value CR2 (here, CR1
> CR2) If the value of the judgment counter continues to be equal to or more than the set number of times that is equal to or less than CR2), the process proceeds from step S71 to step S73 to turn on the alarm output, activate the alarm device 19, and cause the driver to generate an abnormality. Notify.

In each of the above routines, the number of screens determined to be abnormal by the determination counter is counted. However, the time during which the state determined to be abnormal by the determination counter or the timer is continued is counted, and the set time or more is counted. When it becomes, the alarm output may be turned on.

In the present embodiment, a cable break or short-circuit
In addition to detecting abnormalities in the image sensor, the connector can also detect an inconvenient situation in image processing due to inadequate lighting at night, preventing erroneous recognition due to factors outside the image processing system. be able to.

FIGS. 13 to 17 relate to a fourth embodiment of the present invention. FIGS. 13 and 14 are flowcharts of a basic routine for determining an abnormality. FIGS. 15 to 17 illustrate abnormalities in an image processing system having two image pickup devices. It is a flowchart of a determination routine.

This embodiment is different from the above-described third embodiment in which the maximum luminance value of one screen of a video signal is measured to determine an abnormality. Value) is measured to determine the abnormality, and the system configuration is the same as in the third embodiment.

That is, the maximum value of the luminance change amount is a relatively large value in a normal state. However, when the image pickup element is abnormal, or when the lens 5 or the glass in front of the lens 5 becomes dirty or fogged, the image is displayed. Are not sharp, and the state in which the amount of change in luminance of the video signal is small continues. Therefore, the video processor 15 measures the maximum value of the amount of change in the luminance of the video signal of one screen. Raise an alarm.

The basic routine of this abnormality determination is shown in FIGS. 13 and 14. In this routine, the capture of the video signal into the image memory 14 is permitted in step S80, and the memory of the maximum luminance change amount is stored in step S81. When the value is initialized to 0, the address of the pixel to be read first is set in the image memory 14 in step S82. In step S83, one pixel of image data is read from the corresponding address in the image memory 14 and stored in the register A. .

Next, the process proceeds to step S84, where the address of the adjacent pixel in the image memory 14 is set.
In step 5, one pixel of image data is read from the corresponding address in the image memory 14 and stored in the register B.
In step 6, the absolute value of the difference between the register A and the register B is calculated and stored in the register C as the luminance change amount.

Then, in step S87, it is checked whether or not the content of the register C, that is, the amount of change in luminance exceeds the maximum value of the amount of change in luminance at the present time. If the amount does not exceed the maximum value of change in luminance, the flow jumps to step S89. If the luminance change amount exceeds the maximum value, the process proceeds to step S88, and the content of the register C is updated as a new luminance change amount maximum value.
Proceed to step S89.

In step S89, it is determined whether or not the data processing for one screen has been completed. If the data processing for one screen has not been completed, the flow advances to step S90 to store the data in the image memory 14 of the pixel to be read next. The address is set, and the process returns to step S83 to repeat the same processing.

When the data processing for one screen is completed, the process proceeds from step S89 to step S91 to check whether the maximum value of the luminance change is equal to or less than the set value VR. In step S92, the value of the determination counter that counts the number of screens in which the state of the luminance change amount ≦ the set value VR continues is cleared to 0 in step S92, and the process proceeds to step S9.
4 and when the maximum luminance change amount ≦ the set value VR,
It is determined that there is an abnormality, and 1 is added to the determination counter in step S93, and the process proceeds to step S94.

In step S94, it is checked whether or not the value of the determination counter has exceeded the set number. If the count has not reached the set number, the alarm output is turned off in step S95. And the same process is repeated.

At this time, if the cable 3 is disconnected or short-circuited,
A situation in which image recognition processing cannot be performed normally due to dirt or fogging of the lens 5 or glass on the front surface of the lens 5, an abnormality of the image sensor 6, or defective lighting continues, and the maximum luminance change amount of one screen is equal to or less than the set value VR. When the value of the determination counter becomes equal to or more than the set number of times continuously, the process proceeds from step S94 to step S96 to turn on the alarm output, activates the alarm device 19, and notifies the driver of the occurrence of the abnormality.

In the image processing system provided with a plurality of image sensors, the abnormality determination process may be performed for each image sensor as described in the third embodiment. For a stereo image processing system or the like using pairs 6 and 6 as shown in FIG.
Is applied, the maximum value of the luminance change amount of the video signal of one screen input from one of the two imaging elements 6 and 6 is equal to or greater than the set value VR1, and the other imaging is performed. The maximum value of the luminance change amount of the video signal of one screen input from the element is the set value VR2 (where VR1> VR
2) When the following conditions are satisfied, a more stable diagnosis can be performed by determining an abnormality.

More specifically, in step S100, the capture of the video signal into the image memories 14, 14 is permitted.
In steps S101 to S110, step S81 of the routine of FIG.
When the same processing as in steps S90 to S90 is performed on the right image to obtain the maximum luminance change amount of one screen in the right image, step S1
In step 11, the memory value of the maximum luminance change amount of the left image is initialized to 0, and the same processing is performed on the left image in steps S112 to S120 to obtain the maximum luminance change amount of one screen in the left image.

After obtaining the maximum value of the luminance change amount of one screen in the right image and the maximum value of the luminance change amount of one screen in the left image, first, in step S121, the maximum value of the luminance change amount of the right image is set to the set value VR1. Check if it exceeds. Then, when the maximum value of the luminance change amount of the right image> the setting value VR1,
The process proceeds from step S121 to step S122 to check whether or not the maximum value of the luminance change amount of the left image is equal to or less than the set value VR2.
The process branches to S126, where the determination counter is cleared to 0, and when the maximum luminance change amount of the left image ≦ the set value VR2, that is, the maximum luminance change amount of the right image> the set value VR1 and the maximum luminance change amount of the left image. When the value is equal to or less than the set value VR2, it is determined that there is an abnormality, and the process proceeds to step S125 to add 1 to the determination counter.

On the other hand, if it is determined in step S121 that the maximum luminance change amount of the right image ≦ the set value VR1,
The process branches from step S121 to step S123 to check whether or not the maximum luminance change amount of the left image exceeds the set value VR1. When the maximum value of the luminance change amount of the left image ≦ the set value VR1, the process branches from step S123 to step S126 described above,
When the maximum value of the luminance change amount of the left image> the setting value VR1, the process proceeds from step S123 to step S124 to check whether the maximum value of the luminance change amount of the right image is equal to or less than the setting value VR2.

As a result, if it is determined in step S124 that the maximum value of the luminance change amount of the right image is greater than the set value VR2, the process branches to step S126, where the determination counter is cleared to 0, and the maximum value of the luminance change amount of the right image is obtained. When ≤ set value VR2, that is, when the maximum value of the luminance change amount of the left image> the set value VR1 and the maximum value of the luminance change amount of the right image ≤ the set value VR2, it is determined that there is an abnormality, and the process proceeds to step S125 described above. Add 1 to the counter.

Thereafter, the process proceeds from step S125 or step S126 to step S127 to check whether or not the value of the determination counter has exceeded the set number. If not, the alarm output is turned off in step S128.
After setting the state, the process returns to step S100 to instruct the image memories 14 to capture the next video signal, and the same processing is repeated.

A situation where image recognition processing cannot be performed normally due to disconnection or short-circuit of the cable 3 on either side, abnormality of the image pickup device, or insufficient illumination continues.
One of the image sensors 6 and 6 has a maximum luminance change amount of a video signal of one screen input from one of the image sensors and is equal to or greater than the set value VR1, and an image of one screen input from the other image sensor. The maximum value of the luminance change amount of the signal is the set value VR2.
(However, if the state of VR1> VR2) or less continues and the value of the determination counter exceeds the set number of times, the above step is performed.
The process proceeds from step S127 to step S129 to turn on the alarm output, activate the alarm device 19, and notify the driver of the occurrence of the abnormality.

In each of the routines according to the present embodiment, the time during which the state determined to be abnormal by the determination counter or the timer is continued is counted.
The alarm output may be turned on.

Also in this embodiment, not only can the cable disconnection or short circuit, the connector be separated, and the abnormality of the image pickup element be detected, but also the image processing due to the dirt or fogging of the lens or the glass in front of the lens, the insufficient illumination at night, or the like. The above-described inconvenient situation can be detected, and erroneous recognition due to a cause outside the image processing system can be prevented.

FIGS. 18 to 21 relate to the fifth embodiment of the present invention. FIGS. 18 and 19 are flowcharts of a basic routine for judging abnormality, and FIGS. 20 to 22 are abnormalities for an image processing system having two image pickup devices. It is a flowchart of a determination routine.

This embodiment is different from the fourth embodiment in that the maximum luminance value (the brightest value) and the minimum luminance value (the brightest value) of one screen of the video signal are replaced with the maximum luminance change amount of one screen of the video signal. An abnormality is determined based on the difference from the darkest value).

That is, in a normal state, the difference between the maximum luminance value and the minimum luminance value of the video signal is a relatively large value.
When the imaging device is abnormal, the lens or the glass in front of the lens becomes dirty or fogged, or when the illumination is insufficient even at night or the like, the contrast is insufficient, and the maximum luminance value and the minimum luminance value are reduced. The state in which the difference from the value is small becomes continuous.

Accordingly, the video processor 15 measures the difference between the maximum luminance value and the minimum luminance value of the video signal of one screen, and when the difference is equal to or less than the set value and the number of set screens (or set time) continues. Then, the alarm 19 is operated to notify the abnormality.

For this reason, in the abnormality determination basic routine of FIGS. 18 and 19, the capture of the video signal into the image memory 14 is permitted in step S130, and the memory value of the maximum luminance value is initialized to 0 in step S131. When the memory value of the minimum luminance value is initialized to 255, the address of the pixel to be read first in the image memory 14 is set in step S132.

Next, proceeding to step S133, when one pixel of image data is read from the corresponding address of the image memory 14, it is checked in step S134 whether the image data read this time is larger than the current maximum luminance value. If the luminance does not exceed the maximum value, step S136
Jump to step S1 if it is larger than the maximum brightness
Proceeding to 35, update the image data read this time as a new maximum brightness value, and proceed to step S136.

In step S136, it is checked whether or not the image data read this time is smaller than the minimum luminance value. If the image data is larger than the minimum luminance value, the process jumps to step S138. If smaller than the minimum luminance value, the process proceeds to step S137. The image data read this time is updated as a new minimum luminance value, and the process proceeds to step S138.

In step S138, it is determined whether the data processing for one screen has been completed. If the data processing for one screen has not been completed, the flow advances to step S139 to store the image memory of the pixel to be read next. Set 14 addresses,
Returning to step S133, similar processing is repeated.

Thereafter, when the data processing for one screen is completed, the process proceeds from step S138 to step S140, where the difference between the maximum luminance value and the minimum luminance value is calculated, and in step S141,
It is checked whether this difference is equal to or less than the set value DR. When the difference> set value DR, the process proceeds from step S141 to step S142, where the value of the determination counter for counting the number of screens in which the condition of difference ≦ set value DR continues is cleared to 0, and the process proceeds to step S144. If ≤ set value DR, it is determined that there is an abnormality, and the process proceeds from step S141 to step S143, where 1 is added to the determination counter, and the process proceeds to step S144.

In step S144, it is checked whether or not the value of the determination counter has exceeded the set number. If the set number has not been reached, the alarm output is turned off in step S145, and the process returns to step S130 to return to step S130. And the same process is repeated.

At this time, if the cable 3 is disconnected or short-circuited,
A situation in which image recognition processing cannot be performed normally due to dirt or fogging of the lens 5 or glass on the front surface of the lens 5, an abnormality of the image sensor 6, or insufficient illumination continues, and the difference between the maximum luminance value and the minimum luminance value of one screen is reduced. If the state of the set value DR or less continues and the value of the determination counter becomes equal to or more than the set number of times, the process proceeds from step S144 to step S146 to turn on the alarm output,
The alarm 19 is operated to notify the driver of the occurrence of the abnormality.

When this abnormality determination processing is applied to a stereo image processing system or the like in which two image sensors 6 and 6 are arranged on the left and right, the abnormality determination routine shown in FIGS. More stable diagnosis can be performed.

In the routines of FIGS. 20 to 22, the difference between the maximum luminance value and the minimum luminance value of the video signal of one screen input from one of the two image pickup devices 6 and 6 is set. When the difference between the maximum luminance value and the minimum luminance value of the video signal of one screen input from the other imaging element is equal to or larger than the value DR1 and equal to or smaller than the set value DR2 (where DR1> DR2), it is determined to be abnormal. .

That is, in step S150, the capture of the video signal into the image memories 14, 14 is permitted, and in step S1
At steps S131 to S159 of the routine of FIG.
When the same processing as in S139 is performed on the right image to obtain the maximum luminance value and the minimum luminance value of one screen in the right image, in step S160, the difference between the maximum luminance value and the minimum luminance value of the right image is calculated. In steps S161 to S169, the same process is performed on the left image to obtain the maximum luminance value and the minimum luminance value of one screen in the left image. In step S170, the difference between the maximum luminance value and the minimum luminance value of the left image is determined. Is calculated.

After calculating the difference between the maximum luminance value and the minimum luminance value of one screen in the right image and the difference between the maximum luminance value and the minimum luminance value of one screen in the left image, step S1 is performed.
At 71, it is first checked whether or not the difference between the right images exceeds the set value DR1. Then, difference of right image> setting value DR1
In step S171, the process proceeds from step S171 to step S172 to check whether the difference of the left image is equal to or smaller than the set value DR2. If the difference of the left image is larger than the set value DR2, the process branches to step S176 to clear the determination counter to zero. , Difference of left image ≦ setting value DR
If 2, that is, if the difference of the right image> the set value DR1 and the difference of the left image ≦ the set value DR2, it is determined that there is an abnormality, and the process proceeds to step S175 to add 1 to the determination counter.

On the other hand, when the difference of the right image ≦ the set value DR1 in the step S171, the process branches from the step S171 to the step S173 to check whether or not the difference of the left image exceeds the set value DR1. When the difference of the left image ≦ the setting value DR1, the process branches from step S173 to the above-described step S176, and when the difference of the left image> the setting value DR1,
The process proceeds from step S173 to step S174 to check whether the difference between the right images is equal to or less than the set value DR2.

As a result, if the difference of the right image is greater than the set value DR2 in step S174, the process branches to step S176, where the determination counter is cleared to 0, and the difference of the right image ≦
When the difference is the set value DR2, that is, when the difference of the left image> the set value DR1 and the difference of the right image ≦ the set value DR2, it is determined that there is an abnormality, and the process proceeds to step S175 to add 1 to the determination counter.

Thereafter, the process proceeds from step S175 or step S176 to step S177 to check whether or not the value of the determination counter has exceeded the set number. If not, the alarm output is turned off in step S178.
After the state, the process returns to step S150 to instruct the image memories 14, 14 to capture the next video signal, and the same processing is repeated.

Then, the situation where the image recognition processing cannot be performed normally due to disconnection or short circuit of the cable 3 on either side, abnormality of the image pickup device, or insufficient illumination continues.
The difference between the maximum luminance value and the minimum luminance value of the video signal of one screen input from one of the image sensors 6 and 6 is equal to or greater than the set value DR1 and is input from the other image sensor. If the state in which the difference between the maximum luminance value and the minimum luminance value of the video signal of one screen is equal to or less than the set value DR2 (where DR1> DR2) continues and the value of the determination counter exceeds the set number of times, the above-described step S177 is performed. Then, the process proceeds to step S179 to turn on the alarm output, activate the alarm device 19, and notify the driver of the occurrence of the abnormality.

In each of the routines of this embodiment as well, the time during which the state determined to be abnormal by the determination counter or the timer is continued is counted, and when it exceeds the set time,
The alarm output may be turned on.

In the present embodiment, similarly to the above-described fourth embodiment, not only can the disconnection or short-circuit of the cable, the connector be separated, and the abnormality of the image pickup device be detected, but also the lens or the glass on the front surface of the lens can be stained or fogged, An inconvenient situation in image processing due to defective lighting or the like can be detected, and erroneous recognition due to a cause outside the image processing system can be prevented.

[0107]

As described above, according to the present invention, it is possible to self-diagnose a defect in a wire harness or an image sensor or a situation inconvenient in image processing in a system.
Not only is it possible to quickly repair a failure, but it is also possible to obtain excellent effects such as preventing image recognition errors due to defects outside the system, improving the processing accuracy of image processing, and improving reliability.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of an image processing system according to a first embodiment of the present invention.

FIG. 2 is an explanatory view showing an example of mounting on a vehicle;

FIG. 3 is a waveform chart of each part of the above.

FIG. 4 is a flowchart of an abnormality determination routine according to the first embodiment;

FIG. 5 is a configuration diagram of an image processing system according to a second embodiment of the present invention.

FIG. 6 is a waveform chart of each part of the above.

FIG. 7 is a flowchart of an abnormality determination routine according to the first embodiment;

FIG. 8 is a configuration diagram of an image processing system according to a third embodiment of the present invention.

FIG. 9 is a flowchart of an abnormality determination basic routine (part 1);

FIG. 10 is a flowchart of an abnormality determination basic routine (part 2);

FIG. 11 is a flowchart of an abnormality determination routine for an image processing system having two image sensors (No. 1);

FIG. 12 is a flowchart of an abnormality determination routine for an image processing system having two image sensors (No. 2).

FIG. 13 is a flowchart (part 1) of an abnormality determination basic routine according to the fourth embodiment of the present invention;

FIG. 14 is a flowchart of an abnormality determination basic routine (part 2);

FIG. 15 is a flowchart of an abnormality determination routine for an image processing system having two image sensors (No. 1);

FIG. 16 is a flowchart of an abnormality determination routine for an image processing system having two image sensors (part 2);

FIG. 17 is a flowchart of an abnormality determination routine for an image processing system having two image sensors (No. 3).

FIG. 18 is a flowchart (part 1) of an abnormality determination basic routine according to the fifth embodiment of the present invention;

FIG. 19 is a flowchart of an abnormality determination basic routine (part 2);

FIG. 20 is a flowchart of an abnormality determination routine for an image processing system having two image sensors (No. 1);

FIG. 21 is a flowchart of an abnormality determination routine for an image processing system having two image sensors (No. 2).

FIG. 22 is a flowchart of an abnormality determination routine for an image processing system having two image sensors (part 3);

[Explanation of symbols]

 1, 1A, 1B ... image processing system 2 ... imaging device 4A, 4B ... image processing device 6 ... imaging device

Claims (11)

[Claims] An image processing system that separates an image signal output from an imaging device into a synchronization signal and a video signal, and processes the separated video signal, wherein each time a synchronization signal separated from the image signal is input. An image processing system comprising: means for measuring an elapsed time after the input; and means for determining an abnormality when the elapsed time is equal to or longer than a set time. 2. An image processing system for separating an image signal output from an imaging device into a synchronization signal and a video signal and processing the separated video signal, wherein the amplitude of the image signal is held at a constant time constant. An image processing system comprising: means for determining an abnormality when a hold value of the amplitude of the image signal becomes equal to or smaller than a set value. 3. An image processing system for separating an image signal output from an imaging device into a synchronization signal and a video signal and processing the separated video signal, wherein a maximum luminance value of one screen of the video signal is measured. An image processing system comprising: means for determining an abnormality when the maximum luminance value is equal to or less than a set value. 4. An image processing system that separates an image signal output from an imaging device into a synchronization signal and a video signal, and processes the separated video signal, wherein a maximum value of a luminance change amount of one screen of the video signal is determined. An image processing system comprising: means for measuring; and means for determining an abnormality when the maximum value of the luminance change amount is equal to or less than a set value. 5. An image processing system for separating an image signal output from an imaging device into a synchronization signal and a video signal, and processing the separated video signal, wherein a maximum luminance value and a minimum luminance value of one screen of the video signal are provided. An image processing system comprising: means for measuring a difference between the maximum luminance value and the minimum luminance value; and means for determining an abnormality when the difference is equal to or less than a set value. . 6. An image processing system which separates each image signal output from an image pickup apparatus having a plurality of image pickup devices into a synchronization signal and a video signal, and processes each of the separated video signals. Means for measuring the maximum luminance of one screen of the video signal obtained from the image sensor, and at least one of the maximum luminances of one screen of the video signal obtained from each image sensor is equal to or more than the first set value and at least 1 Means for determining an abnormality when one of them is equal to or less than a second set value. 7. An image processing system that separates each image signal output from an image pickup apparatus having a plurality of image pickup devices into a synchronization signal and a video signal, and processes each separated video signal. Means for measuring the maximum value of the luminance change amount of one screen of the video signal obtained from the image sensor, and at least one of the maximum values of the luminance change amount of one screen of the video signal obtained from each image sensor is the first value An image processing system comprising: means for determining an abnormality when at least one is equal to or more than a set value and equal to or less than a second set value. 8. An image processing system which separates each image signal output from an imaging device having a plurality of imaging elements into a synchronization signal and a video signal, and processes each of the separated video signals. Means for measuring the maximum luminance value and the minimum luminance value of one screen of the video signal obtained from the image signal, and calculating the difference between the maximum luminance value and the minimum luminance value of one screen of the video signal obtained from each image sensor, respectively And at least one of the differences between the maximum luminance value and the minimum luminance value of one screen of the video signal obtained from each image sensor is equal to or more than the first set value and at least one is equal to or less than the second set value. An image processing system comprising: means for determining an abnormality. 9. The method according to claim 3, wherein an alarm is issued when the state determined to be abnormal continues for the set number of screens.
The image processing system according to any one of 4, 5, 6, 7, and 8. 10. An alarm is issued when a state determined to be abnormal continues for a set time.
The image processing system according to any one of 4, 5, 6, 7, and 8. 11. The image pickup apparatus according to claim 1, wherein the image pickup device outputs an image signal of a stereo image pair obtained by picking up an object from different viewpoints. The image processing system according to claim 1.