JP2021005820A - Outside recognition device - Google Patents

Abstract

To provide an outside recognition device capable of detecting a failure of an imaging element during imaging of the outside and preventing erroneous detection of a failure of the imaging element when the outside is dark.SOLUTION: When upper bits D11 and D10 composed of one or more bits counting from the most significant bit D11 of bit strings D11 to D0 output by at least one of a plurality of imaging element 102 and 103 are not fixed to the Low level for a predetermined time, a first controller 104 executes failure determination on the plurality of imaging elements 102 and 103. When all the upper bits D11 and D10 of the plurality of imaging elements 102 and 103 are fixed to the Low level for the predetermined time, the first controller does not execute the failure determination on the plurality of imaging elements 102 and 103.SELECTED DRAWING: Figure 3

Description

The present invention relates to an outside world recognition device provided with an image pickup element that images the outside world, and particularly relates to failure detection of the image pickup element.

The vehicle control device needs a safety mechanism that avoids an event that occurs when a certain function or part fails. The vehicle control device sends a control command by communicating with a sensor that recognizes the outside world, a recognition system microcomputer that processes a signal obtained by the sensor to obtain information, and another vehicle control device based on the information. Some have two types of control microcomputers. As an example of an event caused by a failure of an external world recognition device having a vehicle control means, when abnormal image data is output due to a failure of an image sensor, the vehicle control may be mistaken and unintended sudden braking may occur. As a solution to this, there is a method of detecting a failure by sending specific data to the image sensor when the external world recognition device is activated and determining whether or not the expected data is returned.

Japanese Unexamined Patent Publication No. 2008-190868

When the above method is performed after the external world recognition device is activated, diagnosis cannot be performed unless the external world recognition function is stopped and specific data is sent. In the case of vehicle control, since the external world recognition function cannot be stopped after activation, it is not possible to detect a failure of the external world recognition device when the vehicle is moving.

When the external world recognition device detects a failure after it is activated, it may not be possible to determine the failure of the image sensor. As a failure of the image sensor, image data cannot be output, or one of a plurality of signal lines (hereinafter, image data lines) for transmitting image data (multiple bits) output from the image sensor is GND due to a foreign substance. It is conceivable that it will short-circuit and stick to the Low level. On the other hand, even when the outside world is so dark that the front cannot be confirmed, an image data line that transmits one or more bits on the upper side (hereinafter, higher side bits) among the plurality of bits constituting the image data output from the image sensor. Sticks to the Low level. Therefore, when the image data line that transmits the upper bit of the image data is fixed at the Low level, it is indistinguishable whether it is due to a failure of the image sensor or due to the darkness of the outside world. Therefore, when the image data line that transmits the upper bit of the image data is fixed at the Low level, it is not possible to determine whether the external world recognition device is normal or defective.

The present invention has been made in view of the above problems, and an object of the present invention is to be able to detect a failure of an image sensor during imaging of the outside world and to erroneously detect a failure of the image sensor when the outside world is dark. The purpose is to provide an outside world recognition device that can prevent the above.

In order to achieve the above object, the present invention comprises a plurality of image pickup elements that convert a voltage signal from a pixel that images the outside world into a bit string consisting of a plurality of bits and outputs the image sensor, and a bit string that is output from the plurality of image pickup devices. In an external world recognition device including a first controller that forms an image of the external world based on the above, the first controller is the most of the bit strings output by at least one of the plurality of image pickup elements. Failure determination of the plurality of image pickup devices based on the bit strings output from the plurality of image pickup elements when the upper bit consisting of one or a plurality of upper bits is not fixed to the Low level for a predetermined time. Is executed, and when all the upper bits of the plurality of image pickup elements are fixed to the Low level for the predetermined time, the failure determination of the plurality of image pickup elements is not executed.

According to the present invention configured as described above, when the upper bit of at least one of the plurality of image pickup devices is not fixed to the Low level for a predetermined time (when the outside world is not dark), a plurality of image pickup devices are configured. When the failure determination of a plurality of image pickup elements is executed based on the bit strings output from the image pickup elements, and all the upper bits of the plurality of image pickup elements are fixed to the Low level for the predetermined time (when the outside world is dark). , Failure determination of multiple image sensors is not executed. As a result, it is possible to detect a failure of the image sensor during imaging of the outside world, and it is possible to prevent erroneous detection of a failure of the image sensor when the outside world is dark.

According to the external world recognition device according to the present invention, it is possible to detect a failure of the image sensor during imaging of the outside world, and it is possible to prevent erroneous detection of a failure of the image sensor when the outside world is dark.

It is an overall schematic block diagram of the stereo camera which concerns on embodiment of this invention. It is a schematic circuit block diagram of the stereo camera which concerns on embodiment of this invention. It is a flowchart which shows the operation related to the failure determination of the stereo camera which concerns on embodiment of this invention.

Hereinafter, a stereo camera will be taken as an example of the vehicle control device according to the embodiment of the present invention, and will be described with reference to the drawings. In each figure, the same members are designated by the same reference numerals, and duplicate description will be omitted as appropriate.

FIG. 1 is an overall schematic configuration diagram of a stereo camera according to the present embodiment, and FIG. 2 is a schematic circuit configuration diagram of the stereo camera according to the present embodiment.

As shown in FIGS. 1 and 2, the stereo camera 101 according to the embodiment of the present invention has the image pickup devices 102 and 103 for acquiring external world information and the image pickup devices 102 and 103 via the image data lines 201 and 202. A recognition system microcomputer 104 that processes received image data consisting of 12-bit D0 to D11 and outputs information on stereoscopic objects such as people and cars and lanes, and vehicle control based on the information output from the recognition system microcomputer 104. It includes a control system microcomputer 105 that generates and outputs a command.

In the present embodiment, the recognition system microcomputer 104 determines that the image data lines 201 and 202, which transmit the image data output from the image sensors 102 and 103, do not exceed the threshold voltage for a certain period of time. The determination units 203 and 204, the failure determination unit 205 that determines the failure of the image sensor 102 and 103 based on the determination results of the sticking determination units 203 and 204, and the reset unit that resets the recognition system microcomputer 104 by an external input. It is equipped with 206.

The control system microcomputer 105 has a failure notification unit 207 that notifies other vehicle control devices of a failure based on a failure determination from the failure determination unit 205, and a reset signal output unit 208 that outputs a reset signal of the recognition system microcomputer 104. To be equipped.

Next, how to detect a failure will be described. As a failure of the image sensors 102 and 103, the image data cannot be output, or one of the image data lines 201 and 202 for transmitting the image data (plurality of bits) output from the image sensors 102 and 103 is caused by a foreign substance. It is conceivable that it will short-circuit with GND and stick to the Low level.

On the other hand, even when the outside world is so dark that the front cannot be confirmed, one or a plurality of upper bits (hereinafter, upper bits) among the plurality of bits D0 to D11 constituting the image data output from the image sensors 102 and 103. The image data line to be transmitted is fixed at the Low level. For example, in the present embodiment, of the plurality of bits D0 to D11 constituting the image data output from the image pickup devices 102 and 103, the upper two bits D10 and D11 are fixed to the Low level.

Therefore, when the image data line transmitting the upper bits D11 and D10 of the image data output from one of the image sensors is fixed at the Low level, it is due to the failure of the one image sensor or the outside world is dark. It is indistinguishable whether it is due to the fact that. However, it is unlikely that a plurality of image pickup elements 102 and 103 will fail at the same time, and if one of the image pickup elements fails, the image data lines transmitting the upper bits of the plurality of image pickup elements 102 and 103 will be fixed to the Low level at the same time. It is usually impossible to do.

Therefore, in the present embodiment, when only one of the image data lines of the image sensor 102 and 103 is fixed to the Low level, the failure determination unit 205 uses the image sensor whose image data line is fixed to the Low level. If it is determined that there is a failure and the image data lines of both the image sensors 102 and 103 are fixed at the Low level, the failure determination unit 205 determines that the outside world is dark and does not determine the failure of the image sensors 102 and 103. I will do it. In the present embodiment, since the failure determination is performed based on the image data in the state where the stereo camera 101 is operated, it is possible to detect the failure of the image pickup elements 102 and 103 when the vehicle is moving. ..

FIG. 3 is a flowchart showing an operation related to failure determination of the stereo camera 101. Hereinafter, each step will be described in order.

First, in step S1, the sticking determination unit 203 of the recognition system microcomputer 104 determines whether or not the upper bits D11 and D10 of the image sensor 102 are stuck to the Low level for a predetermined time, and determines the determination result in the failure determination unit 205. Output to.

If YES (the upper bits D11 and D10 of the image sensor 102 are fixed to the Low level for a predetermined time) is determined in step S1, the fixation determination unit 204 of the recognition system microcomputer 104 is located on the upper side of the image sensor 103. It is determined whether or not the bits D11 and D10 are stuck at the Low level for a predetermined time, and the determination result is output to the failure determination unit 205 of the recognition system microcomputer 104 (step S2).

If YES (the upper bits D11 and D10 of the image sensor 103 are fixed to the Low level for a predetermined time) is determined in step S2, the failure determination unit 205 determines that the outside world is dark, and the determination result. Is output to the failure notification unit 207 of the control system microcomputer 105 (step S3).

Following step S3, the failure notification unit 207 notifies the other vehicle control device that the stereo camera 101 has stopped functioning so that the other vehicle control device does not execute the vehicle control command from the stereo camera 101 (step S4). ), Return to step S1.

If it is determined in step S1 that NO (one of the upper bits D11 and D10 of the image sensor 102 is not fixed to the Low level for a predetermined time), the failure determination unit 205 determines that the outside world is not dark. The determination result is output to the failure notification unit 207 (step S6).

Following step S6, the failure notification unit 207 cancels the notification of the suspension of the stereo camera 101 to the other vehicle control device so that the other vehicle control device can use the function of the stereo camera 101 (step S7).

Following step S7, the sticking determination unit 203 determines whether or not any of the bits D11 to D0 of the image sensor 102 is stuck to the Low level for a predetermined time (step S8).

If it is determined in step S8 that YES (one of the bits D11 to D0 of the image sensor 102 is fixed to the Low level for a predetermined time), the failure determination unit 205 determines that the image sensor 102 is a failure and determines. The result is output to the failure notification unit 207 (step S9).

Following step S9, the failure notification unit 207 notifies the reset signal output unit 208 of the control system microcomputer 105 and other vehicle control devices of the failure of the image sensor 102 (step S10).

Following step S10, the reset signal output unit 208 outputs a reset signal to the reset unit 206 of the recognition system microcomputer 104, stops the recognition system microcomputer 104 (step S11), and ends the process. By stopping the recognition system microcomputer 104, it is possible to prevent an erroneous recognition result from being output from the recognition system microcomputer 104. Even if the image data is output after recovering from the failure, the recognition system microcomputer 104 does not recover and keeps the failure notification as it is so that the image data from the image sensor that has once failed is not processed. This state continues until the stereo camera 101 is shut down.

When it is determined in step S8 that NO (neither of the bits D11 to D0 of the image sensor 102 is fixed to the Low level for a predetermined time), the fixing determination unit 204 is one of the bits D11 to D0 of the image sensor 103. Determines whether or not is stuck at the Low level for a predetermined time, and outputs the determination result to the failure determination unit 205 (step S12).

If it is determined in step S12 that YES (one of the bits D11 to D0 of the image sensor 103 is fixed to the Low level for a predetermined time), the failure determination unit 205 determines that the image sensor 103 is a failure and determines. The result is output to the failure notification unit 207 (step S13).

Following step S13, the failure notification unit 207 notifies the reset signal output unit 208 and other vehicle control devices of the failure of the image sensor 103 (step S14), and proceeds to step S11.

In the present embodiment, the voltage signals from the pixels that image the outside world are converted into a bit string composed of a plurality of bits and output to the plurality of image pickup elements 102 and 103, and the bit strings output from the plurality of image pickup elements 102 and 103. In the external world recognition device 101 including the first controller 104 that forms an image of the external world based on the image, the first controller 104 is a bit string output by at least one of a plurality of image pickup elements 102, 103. Bit strings D0 to output from the plurality of image pickup devices 102 and 103 when the upper bits D11 and D10 consisting of one or more bits counting from the most significant bit D11 are not fixed to the Low level for a predetermined time. When the failure determinations S8 and S12 of the plurality of image pickup elements 102 and 103 are executed based on D11 and all the upper bits D11 and D10 of the plurality of image pickup elements 102 and 103 are fixed to the Low level for the predetermined time, a plurality of them. The failure determinations S8 and S12 of the image pickup elements 102 and 103 of the above are not executed.

According to the present embodiment configured as described above, when the upper bits D11 and D10 of at least one of the plurality of image pickup elements 102 and 103 are not fixed to the Low level for a predetermined time (the outside world is). When it is not dark), failure determinations S8 and S12 of the plurality of image pickup elements 102 and 103 are executed based on the bit strings D0 to D11 output from the plurality of image pickup elements 102 and 103, and all of the plurality of image pickup elements 102 and 103 are executed. When the upper bits D11 and D10 of the above are fixed to the Low level for the predetermined time (when the outside world is dark), the failure determinations S8 and S12 of the plurality of image pickup devices 102 and 103 are not executed. As a result, it is possible to detect a failure of the image pickup devices 102 and 103 during imaging of the outside world, and it is possible to prevent erroneous detection of a failure of the image pickup devices 102 and 103 when the outside world is dark. In the present embodiment, the upper bit is composed of two bits, but the upper bit may be a bit that is surely fixed to the Low level when the outside world is dark, and only one bit of the most significant bit D11 is used. , Or it may be composed of three or more consecutive bits including the most significant bit D11.

Further, the first controller 104 determines that the outside world is dark when all the high-order bits D11 and D10 of the plurality of image pickup elements 102 and 103 are fixed to the Low level for a predetermined time. As a result, when any of the image data lines 201 and 202 is fixed at the Low level, it is possible to identify whether it is due to a failure or due to the darkness of the outside world.

Further, the external world recognition device 101 according to the present embodiment includes a second controller 105, and the second controller 105 is a first controller 105 when the first controller 104 determines that one image pickup element has failed. When the controller 104 is reset and the first controller 104 determines that the outside world is dark, the first controller 104 is not reset. As a result, when the outside world is dark and the failure determination of the image pickup elements 102 and 103 is not performed, the operation of the first controller 104 is continued and any of the image pickup elements 102 and 103 is determined to be failure. In addition, the first controller 104 can be stopped.

Although the examples of the present invention have been described in detail above, the present invention is not limited to the above-mentioned examples, and includes various modifications and applications. For example, the above-described embodiment has been described in detail in order to explain the present invention in an easy-to-understand manner, and is not necessarily limited to those having all the described configurations.

101 ... Stereo camera (external world recognition device), 102, 103 ... Image sensor, 104 ... Recognition system microcomputer (first controller), 105 ... Control system microcomputer (second controller), 201, 202 ... Image data line, 203 , 204 ... Sticking determination unit, 205 ... Failure determination unit, 206 ... Reset unit, 207 ... Failure notification unit, 208 ... Reset signal output unit, D0 to D9 ... Bits, D10, D11 ... Bits (upper bit).

Claims (3)

Multiple image sensors that convert voltage signals from pixels that image the outside world into bit strings consisting of multiple bits and output them.
In an external world recognition device including a first controller that forms an image of the external world based on bit strings output from the plurality of image pickup elements.
The first controller is
When the upper bit consisting of one or more bits counting from the most significant bit in the bit string output by at least one of the plurality of image pickup elements is not fixed to the Low level for a predetermined time, the above-mentioned The failure determination of the plurality of image sensors is executed based on the bit strings output from the plurality of image sensors.
An external world recognition device, characterized in that failure determination of the plurality of image pickup elements is not executed when all the upper bits of the plurality of image pickup elements are fixed to the Low level for a predetermined time. In the external world recognition device according to claim 1,
The first controller is an external world recognition device, characterized in that, when all the upper bits of the plurality of image pickup elements are fixed to the Low level for a predetermined time, the outside world is determined to be dark. In the external world recognition device according to claim 2,
Equipped with a second controller
The second controller is
When the first controller determines that any of the plurality of image pickup elements is a failure, the first controller is reset.
An outside world recognition device, characterized in that, when the first controller determines that the outside world is dark, the first controller is not reset.

Images