JP2002300573A - Video diagnostic system on-board of video monitor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obviate the need or investigating the cause that a very small quantity of internal processing is not being performed properly, in original monitoring treatment, according to the monitoring treatment. SOLUTION: In various monitors using videos, the video diagnosis using the learning results of coordination with the input video at that time is performed, using the results of a monitoring treatment function working properly or not working properly.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video diagnosis system for a vehicle-mounted video monitoring device for diagnosing an input image of a vehicle-mounted video monitoring device for monitoring, for example, a rear side of a vehicle.

[0002]

2. Description of the Related Art In a general intruder monitoring system using images, a camera image installed at a site is
Some of them detect temporal change, detect motion in a screen, or extract characteristic points such as luminance and color change points in a screen to detect and report an intruder.

In order to improve the safety of driving a car, a device for evaluating a measurement result of an image input sensor which can be measured at the time of driving and issuing a warning or controlling driving to avoid an accident has been proposed. Many have been proposed. For example,
Uses a camera installed in front of the vehicle to extract the lane by image analysis from a single camera at the front of the road and issues a warning of a deviation from the road, and analysis of multiple camera images at the front of the road. There is a method of detecting an obstacle in front of a traveling road by using a stereo principle and measuring a distance between the host vehicle and a preceding vehicle to issue a rear-end collision warning.

Further, a device which uses a camera installed behind the vehicle to detect another vehicle approaching from behind the vehicle and overtaking the vehicle and to issue an alarm for preventing a contact accident. and so on.

Most of the devices according to the above-mentioned prior arts assume that a good input image is obtained in order to detect a specific characteristic amount, and in some cases, the detection condition is not satisfied. (Hereinafter referred to as abnormal video).

As an example of this, when the luminance of an input image is extremely insufficient, raindrops, snow, etc. due to bad weather, or any other dirt, dust, etc. adhere to the imaging surface or a normal image is intentionally blocked. In the case of image monitoring technology that requires pixel or area correspondence with multiple cameras, if there is a difference in overall or partial brightness of each camera, etc. There is.

In such a case, if the user continues to use the apparatus without noticing the abnormality, the original measurement cannot be performed and a desired result cannot be obtained, or the detection cannot be performed. It may cause an accident. As conventional examples taking measures against these abnormal images, Japanese Patent Application No. 07-121678 (Vehicle exterior monitoring device) (Reference Document 1) and Japanese Patent Application No. 08-152.
No. 330 (in-vehicle image processing device, in-vehicle road monitoring device and vehicle control device) (Reference Document 2). In the former, the image capturing timing of the imaging device is linked with the movement of the wiper, but only the influence of rain is determined as an abnormal state. In the latter case, the vibration is detected from the stereo distance measurement result, and the process is stopped.The abnormality is determined only based on whether a certain feature to be detected is detected. If so, it is necessary to add new criteria specific to each case.

[0008]

SUMMARY OF THE INVENTION The present invention relates to an in-vehicle video surveillance device which is mounted on a vehicle and which assists the operation of the vehicle by monitoring features obtained from a photographed image. It is an object of the present invention to provide a video diagnostic system of an in-vehicle video monitoring device that can easily diagnose whether a captured video is a video in an abnormal state that cannot be monitored or a video in a normal state that can be monitored without performing the monitoring. .

[0009]

An image diagnosis system for an on-vehicle image monitoring apparatus according to the present invention is provided in a vehicle, and includes a photographing means for photographing an image of a predetermined visual field, and a photographing means for acquiring an image from the image photographed by the photographing means. In an apparatus for diagnosing an image used in a vehicle-mounted image monitoring apparatus comprising monitoring means for monitoring a feature to be obtained, a feature amount is extracted from a whole or a part of a predetermined instantaneous image in the image captured by the image capturing means. Diagnostic means for diagnosing whether or not the video is in an abnormal state that cannot be monitored.

[0010]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [First Embodiment] An image diagnosis system for a vehicle-mounted image monitoring apparatus according to a first embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing a schematic configuration of a video diagnostic system of a vehicle-mounted video monitoring device used for rear side monitoring.

The image diagnosis system of the vehicle-mounted image monitoring device is a system for diagnosing the image of the vehicle-mounted image monitoring device mounted on the vehicle. The image input means 1, the image monitoring means 2, the image monitoring result display means 3 , Judgment teaching, action setting means 4, learning storage means 5, judgment means 6, alarm means 7,
It comprises an input correcting means 8, a diagnosis starting condition setting means 9, and a non-image sensor 10.

A driving support system (vehicle mounted) for issuing a warning signal about the presence of a passing vehicle from the rear side based on an image behind the vehicle by the image input means 1, the image monitoring means 2, and the image monitoring result display means 3. Type video surveillance device).

The image input means 1 is constituted by a camera as an image pickup means for picking up an image of the rear of the vehicle so as to cover a rear side view (corresponding to lanes on both sides of the own lane) from the rear of the vehicle. The image is output to the video monitoring means 2 and the judging means 6 as a digitized video. Further, the video input unit 1 outputs a video input timing signal to the judgment setting and action setting unit 4. Foreign matter such as raindrops, snow, ice, or mud adheres to the imaging surface (camera lens) of the image input means 1 or its cover surface, or the input image is changed due to backlight, sudden change in illuminance due to entering or exiting a daytime tunnel, or the like. Deterioration may make monitoring impossible.

The video monitoring means 2 uses the method described below to
The rear side monitoring process is performed using a digitized video signal as an input video from the video input means 1. The video monitoring result display means 3 displays the monitoring result of the video by the video monitoring means 2. Based on the video input timing signal from the video input unit 1, the judgment teaching and action setting unit 4 determines, for each video, the determination result of the video monitoring result display unit 3 and the determination set by the diagnosis activation condition setting unit 9. It tells what action the system should take when the result is obtained.

The learning storage means 5 stores a plurality of video scenes supplied from the judgment teaching and action setting means 4.
Based on the teaching of normal / abnormal and abnormal actions, the correspondence between the plurality of input images and the teaching result is statistically learned, and the compressed result is stored.

The judging means 6 judges the result of the judgment corresponding to the digitized video signal supplied from the video input means 1,
And actions are extracted from the information stored in the learning storage means 5. The alarm unit 7 converts the result of the determination by the determination unit 6 into audio and video and presents it to the user.

The input correcting means 8 activates, for example, an action such as dirt removal on the imaging surface in accordance with the action obtained at the time of the determination by the determining means 6. The diagnosis activation condition setting means 9 teaches an action to be performed by the system when the judgment result of the video monitoring result display means 3 is obtained. The non-video sensor 10 is a system clock, a speed sensor, a steering angle sensor, and a warper switch used in the present system.

1. In the above configuration, an outline of the first embodiment will be described. Video monitoring means 2 in this system
Means, for example, using a method as shown in Reference Document 1,
Alternatively, a warning signal about the presence of an overtaking vehicle from the rear side can be issued, although the device may be used.
For this purpose, the pixels input from the video input means 1 and having a texture of a certain intensity or higher in a designated area in a certain frame of the digitized video are associated with each other in a frame at another time point, It is assumed that a method of evaluating a movement vector (optical flow) between the frames considered to be the same between respective frames is used.

In the case of this monitoring method, for example, raindrops, snow,
The input image may be degraded due to attachment of foreign matter such as ice or mud, backlight, sudden change in illuminance due to entering or exiting a daytime tunnel, and monitoring may be impossible. As a result of raindrops, ice, backlight, and the like, the image is often an image in which the luminance is almost saturated or the contrast is insufficient, and there is no sufficient texture for the detection target.

In the case of mud, the target is mainly concealed and accurate tracking cannot be performed. In the case of snow or ice, depending on the amount of the snow or ice, either the lack of contrast or concealment,
Or both often make tracking monitoring impossible. In general, factors that cause the above-mentioned monitoring to fail will occur in combination, and it will not be possible to investigate in detail how the degree would affect the original monitoring process and make monitoring impossible. This is a difficult task.

Therefore, by extracting and evaluating a characteristic amount from the whole or a part of the predetermined instantaneous image in the video, it is possible to diagnose whether or not the video is in an abnormal state that cannot be monitored.
By extracting and evaluating feature amounts from the entire or partial image of a plurality of frames obtained in time series from the video, it is possible to diagnose whether or not the video is in an abnormal state that cannot be monitored,
A method of teaching a diagnostic result of whether an image in an abnormal state that cannot be monitored or an image in a normal state that can be monitored for each image, and statistically learning a correspondence relationship between the plurality of images taught and the result of the teaching. It is used.

Thus, a plurality of image samples are input to the image monitoring means 2 instead of performing a detailed analysis of which part of the image monitoring means 2 is affected by the abnormal state occurring in the input image. By performing only the learning that associates the success or failure of the result with the general feature quantity extracted from the input image, it is possible to predict the success or failure of the input image without the actual processing of the video monitoring unit 2 when the image is obtained. Then, it is possible to report whether the input image is abnormal or normal.

Therefore, the processing of the present system can be roughly divided into two methods. In order to determine whether or not an input is appropriate as a monitoring image, it is determined whether or not an output result obtained when a sample image is given to the image monitoring means 2 is appropriate. An abnormal image registration learning step for teaching, and an abnormality determining step of using the result to determine whether the input actually given is normal / abnormal.

The flow of the entire processing at this time will be described with reference to the flowchart shown in FIG. First, when the abnormal image registration learning is determined (ST1), the abnormal image registration learning is input by the image input means 1 installed so as to cover the rear side view from the rear of the vehicle (corresponding to the lane on both sides of the own lane). (ST2) The digitized video signal is used by the video monitoring means 2 for the rear side monitoring process which is the original purpose (ST3).

Next, it is determined whether or not the output result of the video monitoring result display means 3 is normal (ST4), and the determination result and the determination result are obtained for each video by using the determination teaching and action setting means 4. The system sometimes teaches actions to be performed (ST5, ST6). After teaching the normal / abnormal and abnormal actions for a plurality of video scenes, the learning storage unit 5 statistically learns the correspondence between the plurality of input videos and the teaching results, and stores the compressed results. (ST7, ST8).

The processing of the judgment teaching, action setting means 4 and learning storage means 5 is performed in advance by associating the statistical properties of the diagnostic features extracted from the video with the corresponding normal / abnormal states. This is the process of judging the input video, and associates not only the statistical properties of the diagnostic features extracted from the video and the corresponding normal / abnormal states, but also the countermeasures when it occurs. In this process, learning is performed in advance, and measures are presented to the user at the same time as the determination of the input video. The details of the processing here are shown in 2.

Also, in step 5, if it is obvious that when an abnormality is taught, the image is obtained and some event information obtained from a sensor other than the image (non-image sensor 10) is obvious. In addition to the action, event information from the sensor is associated with the video and learned, or simply used as a condition for judging abnormality / normality of diagnosis.

For example, when raindrops and slime adhere to the surface of the image pickup system and the input image becomes abnormal, this diagnostic processing is started only when a window wiper drive signal different from that of the image pickup system is valid. You may learn to.

As in the case of this rear side monitoring device, when it is desired to monitor passing by the rear side vehicle when the host vehicle is traveling straight, and to detect an abnormality due to mud or raindrops, a steering angle sensor or a gyro sensor is used. Learning can also be performed using the output of. Here, the selection and designation of the event from the sensor is performed by the diagnosis activation condition setting means 9.

Next, at the time of actual operation, the present system is operated as an abnormality determination step. In this case, similarly to the abnormal video registration learning step, the input and digitized video signal (ST9) is given to the determination means 6, and the determination and action corresponding to the video signal are stored in the learning storage means 5. (ST10-1)
3).

Finally, the judgment result is converted into audio and video by the alarm means 7 and presented to the user (ST).
14, 16). Further, in accordance with the action obtained at the time of the determination, the input correcting means 8 starts an action such as removal of dirt on the imaging surface (ST14, ST15).

Next, the processing of the abnormal video registration learning step will be described. As for the video captured by the video input means 1 as described above, the result of the processing by the video monitoring means 2 is displayed on the video monitoring result display means 3 as a correct answer /
The non-correct answer is output and used, the non-correct answer is regarded as abnormal, and the input video is classified into a certain N class based on the action at that time and the combination of the activation event by the input from the non-video sensor 10. Then, the learning storage unit 5 performs statistical processing on the classification, and determines whether the input
Learn the relationship between abnormalities, actions, and startup events.

Next, a registration learning step for abnormality / normal diagnosis of an input image will be described with reference to a flowchart shown in FIG. First, feature values are extracted from a certain instantaneous image in a video (ST21, ST24). Here, the following is used as a relatively general image processing feature amount in order to give the monitoring process versatility to some extent. That is, the areas D1... D which are to be used for the rear side monitoring for each frame image in the sample video scene
k, each pixel in the same area D1... Dk is extracted after spatial differentiation is performed on each pixel in the same area D1,. The luminance frequency distribution of the time difference for each pixel from the frame at the time point (for example, the next sample) is obtained and arranged as a one-dimensional vector.

As a more general feature quantity, for the above-described frame, the pixel brightness value within the designated area, the same differential pixel brightness value, and the same difference pixel brightness value are scanned in raster order to obtain 1 pixel.
Extracted as a dimensional vector and arranged them together 1
A dimensional vector may be used.

Next, the above-mentioned vectors are classified into one of Nn subclasses as a normal class and one of Na subclasses as an abnormal class (ST22, 25). At this time, based on the result of supplying the input video to the video monitoring result display means 3, it is possible to automatically classify whether the above-mentioned feature amount vector corresponding to the input is normal or abnormal.

However, regarding the image abnormality as the object of the first embodiment, since a plurality of factors as described above are considered, the vector corresponding to each class of normal / abnormal is simply 2 vectors. It is presumed that the class cannot be represented by the linear discriminant function. It is for this reason that the two classes, abnormal / normal, are further divided into subclasses.

This subclassing is generally an operation of evaluating a large number of images and classifying them. Since it is difficult for humans to classify the images, classification is performed by applying a general automatic clustering method. As a clustering method, here, SOM (Self Organiz) which is a kind of neural network is used.
ation Map), but K-means
A technique having a similar effect such as a clustering method may be used.

Learning for performing an abnormal / normal diagnosis is performed on the clustered vectors. Here, a method is used in which a discriminant analysis is performed using the above-described subclasses as belonging classes to form a discriminant space (discriminant matrix), and a projection of the barycenter vector of each subclass into this space is used as a representative vector of the class (ST23). , 25).

In this case, the discrimination matrix and the representative vector are stored as a dictionary in the learning storage unit 5 for an abnormality determination step described later. In the abnormality determination step,
A feature vector is sampled for each input frame by the same procedure as described above (ST27), and a vector obtained by projecting the feature vector onto the discrimination matrix is obtained (ST28). The distance between the vector and the representative vector of each subclass (Euclidean distance) is calculated. determined, it determines each representative and by evaluating the distance, whether the input vector V _in that belong to either / normal classes belonging to the abnormal class (ST29).

As the evaluation of the distance, _first , an average distance between the representative vectors VN ₀ to VN _Nn−1 belonging to the normal is calculated _.
vDn, representative vectors VA ₀ to VA belonging to the abnormality
The average distance from _Nn-1 is determined and is defined as AvDa. Next, among the representative vectors belonging to normal, those having a distance less than AvDn {VN}, and those representing abnormal
The average value of the distance from the object less than vDa {VA} is obtained as Dn and Da, respectively, Is determined according to the following. Where C _n :
Normal class, _{C a:} is abnormal _{class, {VN} = {VN k} / D (VN k) <AvDn}, and _{{VA} = {VA k /} D (VA k) <AvD
a｝, D (VN _k ) = | V _in −V N _k |, D (VA _k ) = |
V _in -VA _k | to. Other than such a distance evaluation, determine the distance between the input V _in and the representative vector of each subclass, leave sorted with respect to distance, extracting the top k representative vectors from close the distance, the normal / abnormal similar average distance D _'n, D' and above each class seeking _a, according to the same decision _{V in ∈C n ifD 'n <} D a V in ∈C a ifD' a <D n (2) The determination of normal / abnormal may be performed.

Further, this determination may be made according to the following equation. Here P _n and P _a are the same manner, to extract the top k representative vectors from those close in distance, weighted by the square of the reciprocal of the distance from each subclass in each class in the normal / abnormal It is the result of voting. Assuming that the value of the input vector varies according to the normal distribution, the probability P (i | x) that the input x belongs to the subclass i is shown below.

(Equation 1)

[0042] The _P n, _{P a} in the formula 3 means an approximation of the probability of this equation (4). As described above, the discriminant analysis space was previously learned from the vectors in each class, and the normal / abnormal judgment was performed by the discriminant analysis on the input image.
In contrast, the each principal component analysis for each subclass were allowed represented by the signal space defined by the upper n eigenvectors, the time of diagnosis of a normal / abnormal, the input vector V _in at the above-mentioned each subclass The projection values projected on the subspace are sorted from the larger one, and the average value P _a of the projection values up to the k-th (where k <(Nn + Na)),
_Pn may be obtained, and the determination may be made according to the following rules.

[0043] In the above learning process, the input image is converted into a plurality of N (= Nn +
Na) Classified into classes and their representative values are stored in a table 5a of the learning storage means 5, as shown in FIG. 4, and are used as a dictionary at the time of recognition. And the condition for activating the abnormality diagnosis for are stored as attributes.

Here, the start conditions include a diagnosis start at a longer time interval than the monitoring process, a case where the speed of the own vehicle measured by a speed sensor of the non-image sensors 10 is less than a certain value, and a case where an optical flow is detected. When the measured relative speed is equal to or less than a certain value, when the moving direction of the own vehicle obtained from the steering angle sensor or the gyro sensor of the non-image sensors 10 is close to straight traveling, the input image may be deteriorated due to the effect of rainy weather. It is possible to specify for each class, for example, the case of the wiper operation due to the large number.

The diagnosis for the rear side monitoring device at this time is as follows.
Among these conditions, as shown in FIG.
Stored in the table 5b. In FIG. 5, dictionary subclass names include names of abnormal causes such as raindrop, snow, ice subclass, mud splash subclass, and luminance saturation subclass. The contents of these subclasses are in the class closest to the class representative vector. It is determined by observing the image that is the source of the vector. This may be determined, for example, as the cause with the highest frequency of occurrence among the abnormal causes of the original image of k vectors near the center of gravity vector.

Next, the processing of the abnormality determination step will be described. In this step, first, it is checked whether or not a condition for activating a diagnosis for the rear side monitoring means to be monitored is obtained from the non-image sensor 10. This determination is performed by using the outputs from the system clock, speed sensor, steering angle sensor, and warper switch used in the present system as the non-video sensor 10 by the determination means 6 as follows, and using the dictionary described in the previous section. Is started.

That is, according to FIGS. 4 and 5,
It is checked whether the time interval for performing the diagnosis has been reached, whether the current speed sensor output is equal to or less than the constant speed Vth, and whether the moving direction of the vehicle from the steering angle sensor is equal to or less than the constant angle θth. Start diagnosis. At this time, if the wiper switch is ON, the "raindrop, snow, ice subclass" is also included as a subclass for performing the distance determination (use permission dictionary).

Then, use permission dictionaries satisfying these conditions are selected from those stored in the tables 5a and 5b of the learning storage unit 5, and it is confirmed which one of the currently input video signals corresponds. Specifically, this is the process described in the latter half of the previous section, in which the same feature values as in the learning are extracted from the input video,
Mapping is performed on the representative space of each class, and it is determined which class is closest (or which class has the largest angle).

When a class is further selected, an action stored as an attribute of the class is extracted.
Eventually, according to the action determined here, an operation for issuing an alarm to the user or correcting the input image (cleaning the camera lens surface in this embodiment) is started.

[Second Embodiment] Overview of Embodiment for Monitoring Obstacles Forward The configuration of a basic diagnostic system in the second embodiment is also the same as that of the example for monitoring the rear side described above. Almost the same. However, in the monitoring of this application, unlike the first embodiment, a stereo camera arranged so as to cover the same area of the front lane area as the field of view is used as the input imaging system.

Therefore, in the configuration shown in FIG. 1, the video input means 1 is a right video input means 1a, a left video input means 1b.
Becomes Also, as the video monitoring means 2 in the configuration diagram,
A stereo stereo extraction technique called "plane projection stereo method" as shown in Reference 2, or a normal stereo method that performs distance measurement based on the principle of triangulation using parallax obtained from corresponding point search results of left and right images and performs stereo extraction. Any of these may be used.

In the configuration of the second embodiment and the principle of the monitoring method to be diagnosed, what is further different from the first embodiment for the above-mentioned rear side monitoring application is an image for extracting a feature amount vector, Alternatively, it is a method of selecting an area on an image.

In the above-mentioned rear side monitoring application, a rear side field of view (corresponding to the lane on both sides of the own lane) is cut out from the rear part of the vehicle, and a feature vector is extracted from the inside. In each of the two cameras, a right image input unit 1a and a left image input unit 1b are used as a pair to cut out image regions obtained by imaging the same region in three dimensions.
The difference is that a feature vector similar to that used for the rear side application is extracted from each.

This is because a case where the luminances of the right and left cameras are significantly different due to the influence of external light or the like is taken in as a dictionary. In this case, in the dictionary, a “left / right luminance difference subclass” is added to the subclass of the abnormal class, and “left / right luminance adjustment” is added as an action attribute.

[Effects of the Embodiment] As described above, in various monitoring apparatuses using video, only the result that the monitoring processing functions well or not is used, and the learning result of the association with the input video at that time is used. Because it is a video diagnosis method, there is no need to investigate according to the monitoring process what causes the internal processing of the feature value determined in the original monitoring process to work well, and various monitoring systems using video It can be easily applied to the video diagnostic system.

Further, since the condition for starting the video diagnosis can be set for the original monitoring system, it can be started only in a situation where the video diagnosis is necessary, and the processing cost of the whole system can be suppressed.

In particular, when the purpose of the image monitoring system to be diagnosed is an on-vehicle type image monitoring and a diagnosis of a decrease in monitoring accuracy due to the weather is to be diagnosed, or when the host vehicle is traveling straight ahead, In the case of the diagnostic function of monitoring for the purpose of overtaking and interrupt detection in the above, the diagnosis cost can be further reduced by performing the diagnosis only during the time when it is acceptable to relatively spend the processing time on the diagnosis. it can.

Further, as a result of performing a video diagnosis on the original monitoring system, an action to be taken can be registered and set in advance at the time of learning, so that a video diagnosis can be performed online and an appropriate diagnosis alarm can be given to a user. And automatic start of a measure for improving the input image (for example, cleaning of a sensor) can be realized, and a user-friendly system can be constructed.

In addition, the user registers in advance whether or not to start video diagnosis in accordance with a feature obtained from a certain image or a specific event based on an external detection state or an internal state other than that, and activates according to the registration. The control is performed.

Perform a video diagnosis operation at a certain time interval.
Alternatively, an on-line monitoring process is hindered by performing a video diagnosis only when a state in which an original monitoring target is not detected within a certain period of time continues for a longer period of time separately defined.

In particular, when the object to be monitored moves, or when both the object to be monitored and the sensor (camera) as the observation system move, it is detected that either one of the speeds or the relative speed between them is low, and The video diagnosis operation is performed only in such a case so that the online monitoring process is not hindered.

In various video surveillance systems, in a video diagnostic system, in which an input video is an on-line determination that an abnormal video is unsuitable for monitoring, diagnostic features extracted from the video are used. By learning not only the statistical properties and the corresponding normal / abnormal states, but also the countermeasures when they occur, and learning in advance, it is possible to determine the input video online and take countermeasures at the same time. This is presented to the user.

In a surveillance system using images, particularly when an observation system sensor is installed on a moving object such as a vehicle and monitors the stability and safe driving of the vehicle, the input image may be rain, snow, etc. In the video diagnostic system, which is unconditionally determined to be an abnormal condition video that is unsuitable for monitoring due to the influence of precipitation, online diagnostics are performed only when the wiper is mounted on the vehicle tower. This does not hinder the monitoring processing of the server.

In a surveillance system using images, in particular, it is assumed that the own vehicle is going almost straight, such as a sensor, which is an observation system, is installed on a moving object such as a vehicle, and its stable running, overtaking, and interruption detection are performed. In the case of monitoring in a situation, in an image diagnostic method characterized in that the input image is an online unified determination that the input image is an abnormal state image unsuitable for monitoring, the observation system moves in a direction approaching straight ahead. By detecting the movement and performing the image diagnosis operation only in that case, the online monitoring process is not hindered.

[0065]

As described above in detail, according to the present invention, there is provided an on-vehicle video surveillance device which is mounted on a vehicle and which assists the driving of the vehicle by monitoring features obtained from a photographed video. It is possible to provide a video diagnostic system of an in-vehicle video monitoring device that can easily diagnose whether a captured video is a video in an abnormal state that cannot be monitored or a video in a normal state that can be monitored without performing detailed analysis of the device. .

[Brief description of the drawings]

FIG. 1 is a block diagram showing a schematic configuration of a video diagnostic system of a vehicle-mounted video monitoring device for describing an embodiment of the present invention.

FIG. 2 is a flowchart for explaining a video diagnosis process of the vehicle-mounted video monitoring device.

FIG. 3 is a flowchart for explaining an abnormal video registration learning process.

FIG. 4 is a diagram showing an example of storage of a table of activation conditions.

FIG. 5 is a diagram showing a storage example of an action table.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Video input means, 2 ... Video monitoring means, 3 ... Video monitoring result display means, 4 ... Judgment teaching, action setting means, 5 ...
Learning storage means, 5a activation condition table, 5b action table, 6 determination means, 7 determination means, 8 input correction means, 9 diagnostic activation condition setting means, 10 non-image sensor.

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G06T 7/00 300 G06T 7/00 300E G08B 21/00 G08B 21/00 U G08G 1/16 G08G 1/16 CF term (for reference) 5B057 AA16 BA11 CA08 CA12 CA16 CB18 CC01 CH08 DA15 DB02 DB09 DC30 DC36 5C054 FC00 FC01 FC12 FC16 FE28 FF07 HA30 5C086 AA53 BA22 CA28 CB36 DA07 DA33 EA45 FA06 FA18 5H180 AA01 BB17 CC04 CC04 CC07 DA03 HA04 JA11 MA07

Claims (11)

[Claims] 1. An on-vehicle video monitoring device, which is provided in a vehicle and includes a photographing means for photographing an image of a predetermined field of view, and a monitoring means for monitoring characteristics obtained from the image photographed by the photographing means. In a video diagnostic system of an in-vehicle video monitoring device for diagnosing video, a feature amount is extracted and evaluated from a whole or a part of a predetermined instantaneous image in a video taken by the above-mentioned photographing means. An image diagnostic system for a vehicle-mounted image monitoring device, comprising: diagnostic means for diagnosing whether an image is in an abnormal state. 2. The image diagnosis system according to claim 1, wherein the predetermined field of view is at a rear side of the vehicle or in front of the vehicle. 3. An on-vehicle video monitoring device provided in a vehicle, comprising: a photographing means for photographing an image of a predetermined field of view; and a monitoring means for monitoring characteristics obtained from the image photographed by the photographing means. In a video diagnosis system of a vehicle-mounted video monitoring device for diagnosing a video, extracting and evaluating a feature amount from all or a part of a plurality of frames of images obtained in time series from the video taken by the above-mentioned photography means. A diagnostic means for diagnosing whether or not the video is in an abnormal state that cannot be monitored. 4. The image diagnosis system according to claim 3, wherein the predetermined field of view is at the rear side of the vehicle or in front of the vehicle. 5. An on-vehicle video surveillance device comprising a photographing means provided in a vehicle for photographing an image of a predetermined field of view and a monitoring means for monitoring characteristics obtained from the image photographed by the photographing means. In a video diagnostic system of a vehicle-mounted video monitoring device for diagnosing a video, teaching means for teaching a diagnostic result of whether a video in an abnormal state that cannot be monitored or a video in a normal state that can be monitored for each video captured by the capturing means. And learning means for statistically learning the correspondence between the plurality of videos taught by the teaching means and the teaching results. The monitoring means of the on-vehicle video monitoring device is photographed by the photographing means. A video diagnostic system for a vehicle-mounted video monitoring device, characterized in that a feature obtained from the video is monitored based on the learning content learned by the learning means. 6. An on-vehicle video monitoring apparatus provided in a vehicle, comprising: a photographing means for photographing an image of a predetermined visual field; and a monitoring means for monitoring a characteristic obtained from the image photographed by the photographing means. In a video diagnosis system of a vehicle-mounted video monitoring device that diagnoses a video, a diagnosis unit that diagnoses whether a video captured by the imaging unit is a video in an abnormal state that cannot be monitored is longer than a monitoring process performed by the monitoring unit. Executing means for executing diagnosis by the diagnosis means at time intervals. 7. An on-vehicle video monitoring apparatus provided in a vehicle, comprising: a photographing means for photographing an image of a predetermined field of view; and a monitoring means for monitoring characteristics obtained from the image photographed by the photographing means. An image diagnosis system of an on-board type image monitoring device for diagnosing an image, wherein: a diagnosis unit for diagnosing whether an image captured by the imaging unit is an image in an abnormal state that cannot be monitored; Detecting means for detecting that the following has occurred, and executing means for executing diagnosis by the diagnostic means when the detecting means detects that the moving speed of the vehicle has become equal to or less than a predetermined speed. A video diagnostic system for a vehicle-mounted video monitoring device, characterized in that: 8. An on-vehicle video surveillance device comprising a photographing means provided in a vehicle for photographing an image of a predetermined field of view and a monitoring means for monitoring characteristics obtained from the image photographed by the photographing means. An image diagnosis system for an in-vehicle image monitoring device for diagnosing an image, comprising: a diagnosis unit that diagnoses whether an image captured by the imaging unit is an image in an abnormal state that cannot be monitored; and a detection unit that detects a straight traveling state of the vehicle. And a performing means for performing a diagnosis by the diagnosis means when the vehicle moves straight ahead by the detection means. 9. An on-vehicle video surveillance device provided in a vehicle, comprising: photographing means for photographing an image of a predetermined field of view; and monitoring means for monitoring a feature obtained from the image photographed by the photographing means. An image diagnosis system for an on-board type image monitoring device for diagnosing an image, comprising: a diagnosis unit configured to diagnose whether an image captured by the imaging unit is an image in an abnormal state that cannot be monitored; and an operation of a wiper of the vehicle detected. A video diagnosis system for a vehicle-mounted video monitoring device, comprising: a detection unit that performs a diagnosis by the diagnosis unit when the operation of the wiper of the vehicle is detected by the detection unit. 10. An in-vehicle video monitoring device provided in a vehicle, comprising: a photographing means for photographing an image of a predetermined field of view; and a monitoring means for monitoring characteristics obtained from the image photographed by the photographing means. In a video diagnosis system of a vehicle-mounted video monitoring device for diagnosing a video, a diagnosis result of whether a video in an abnormal state that cannot be monitored or a video in a normal state that can be monitored for each video captured by the capturing unit, and the diagnostic result Teaching means for teaching a process to be performed by the on-board video monitoring apparatus in the case of a video in an abnormal state; and statistically learning the correspondence between a plurality of videos taught by the teaching means and the teaching result. Learning means, and diagnosing means for diagnosing, based on the learning content learned by the learning means, whether or not the video shot by the shooting means is a video in an abnormal state that cannot be monitored. And a notifying means for notifying a process to be performed by the on-board video monitoring apparatus taught by the teaching means when the video of the unmonitored abnormal state is diagnosed by the diagnostic means. Diagnostic system for on-board video surveillance equipment. 11. The method according to claim 10, wherein the processing to be performed by the on-vehicle video monitoring device is guidance for cleaning the photographing means, guidance for driving a wiper, and guidance for danger due to backlight or reflected light. An image diagnostic system for an in-vehicle image monitoring device as described in the above.