JP6458579B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing apparatus that performs image processing on image data captured by a camera.

  2. Description of the Related Art Conventionally, a technique for performing various image processing such as object recognition on an image captured by a camera or the like is known. For example, Patent Document 1 describes a technique for deriving parallax of corresponding points common to both images from a pair of stereo images and acquiring distance information of the target object.

JP 2003-83742 A

  In the image processing technology as described above, image recognition may occur due to a failure of a processor that performs image processing. On the other hand, there is a need for a failure diagnosis method that can quickly detect abnormalities in image processing and that does not interfere with normal image processing routines.

  The present invention has been made to solve the above problems. An object of the present invention is to provide a technique capable of diagnosing a failure by a method having simultaneity without disturbing a normal image processing routine in a failure diagnosis of an image processing apparatus.

An image processing apparatus according to an aspect of the present disclosure includes an image processing unit, a diagnostic unit, and an output control unit. The image processing unit performs image processing for each cycle of the captured image periodically captured by the imaging unit. In parallel with the periodic image processing performed by the image processing unit on the captured image, the diagnostic unit performs image processing on diagnostic image data stored in advance in a main storage device directly accessible by the image processing unit. Let it be done. Then, the diagnostic means compares the image processing result for the diagnostic image data with the expected value data representing the standard of the normal processing result of the image processing for the diagnostic image data, and the image processing result for the diagnostic image data is normal. It is determined whether or not. The output control means outputs the processing result related to the image processing for the captured image to the predetermined control means as valid on the condition that the diagnostic processing determines that the image processing result for the diagnostic image data is normal. .

  According to the present invention, the failure diagnosis based on the diagnostic image data is periodically performed in parallel with the normal routine of the normal image processing by the image processing means, so that the normal image processing routine is not hindered. It is possible to monitor in real time whether there is a failure in the image processing means. In particular, in order to execute the processing by the diagnostic means at high speed in accordance with the normal image processing cycle, it is configured to use diagnostic image data stored in a main memory device that can be accessed at high speed. It is possible to guarantee the coincidence of failure detection with respect to.

An image processing apparatus according to another aspect of the present disclosure includes an image processing unit, a diagnostic unit, and an output control unit. The image processing unit performs image processing for each cycle on the captured image periodically captured by the imaging unit. The diagnostic unit acquires diagnostic image data stored in advance in the imaging unit and causes the image processing unit to perform image processing in parallel with the periodic image processing performed on the captured image by the image processing unit. Then, the diagnostic means compares the image processing result for the diagnostic image data with the expected value data representing the standard of the normal processing result of the image processing for the diagnostic image data, and the image processing result for the diagnostic image data is normal. It is determined whether or not. The output control means outputs the processing result related to the image processing for the captured image to the predetermined control means as valid on the condition that the diagnostic processing determines that the image processing result for the diagnostic image data is normal. .

  According to the present invention, fault diagnosis based on diagnostic image data acquired from an imaging unit can be periodically performed in parallel with a periodic routine of normal image processing by the image processing unit. By doing so, it is possible to monitor in real time whether there is a failure related to the input line from the image pickup means or the image processing means without hindering a normal image processing routine.

1 is a block diagram illustrating a configuration of an image processing apparatus (first embodiment). The flowchart showing the procedure of the image processing for every frame. The block diagram showing the structure of an image processing apparatus (2nd Embodiment). The block diagram showing the structure of an image processing apparatus (3rd Embodiment). The flowchart showing the procedure of a failure location identification process. The block diagram showing the structure of an image processing apparatus (4th Embodiment). The flowchart showing the procedure of a test process. Explanatory drawing showing an example of an embedded image. Explanatory drawing showing an example of an embedded image.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In addition, this invention is not limited to the following embodiment, It is possible to implement in various aspects.
<First Embodiment>
[Description of Configuration of Image Processing Device]
The configuration of the image processing apparatus 1a according to the first embodiment will be described with reference to FIG. The image processing apparatus 1a is configured by a computer including a CPU, a ROM, a RAM, and the like, calculates a parallax between a pair of stereo images captured by the stereo camera 10, and obtains distance information of an object included in the image. It has a function to perform image processing to be acquired. The image processing device 1a is embodied as an in-vehicle system that is mounted on a vehicle and is responsible for generating distance information used for various vehicle controls (described later). The image processing apparatus 1a includes an image processing unit 11a, a RAM 12a, a ROM 13a, a failure diagnosis unit 14a, and an output control unit 15a as a characteristic configuration in the present invention.

  The stereo camera 10a is provided with a pair of left and right imaging devices arranged in parallel equidistant positions, like a known stereo camera. The stereo camera 10a repeatedly captures image data consisting of a pair of left and right stereo images by capturing a common area with a pair of imaging devices at the same timing, and outputs the acquired image data to the image processing unit 11a.

  The image processing unit 11a performs image processing for obtaining a parallax value between stereo images based on a stereo image frame input by the stereo camera 10a. The image processing unit 11a repeatedly performs image processing at the same cycle as the frame rate (for example, 40 fps: frame par second) of images continuously acquired by the stereo camera 10a.

  The image processing unit 11a calculates a horizontal parallax between stereo images for each image block obtained by dividing the entire image into small sections, and creates a parallax value map in which the calculated horizontal parallax is associated with coordinates on the image. For the calculation of the horizontal parallax, a technique such as stereo matching can be used. Since stereo matching is a known technique, detailed description thereof is omitted here.

  The RAM 12a is a storage device (random access memory) used as a main memory of the CPU. The RAM 12a is a memory that can be accessed at the highest speed by the image processing unit 11a. The RAM 12a stores a processing result value (parallax value map) that is a result of image processing performed on the image data input by the stereo camera 10a by the image processing unit 11a.

  The RAM 12a stores a plurality of sets of test image data used for failure diagnosis of the image processing unit 11a. This test image is composed of a pair of images simulating a stereo image captured by the stereo camera 10a. In order to comprehensively check the parallax values that can be taken as the system, a plurality of sets of test images configured so that the parallax values obtained by image processing are different from each other are stored in the RAM 12a. The RAM 12a stores a plurality of expected value data corresponding to each of the plurality of sets of test images. The expected value data is information representing a normal result value when the image processing by the image processing unit 11a is performed on the test image. The RAM 12a stores a result value that is a result of image processing performed on the test image by the image processing unit 11a.

  The ROM 13a is a read-only storage device (read-only memory). The ROM 13a stores a test image developed on the RAM 12a, and an original test image and expected value data that are originals of expected value data corresponding to each test image. The test image and expected value data stored in the ROM 13a are transferred from the ROM 13a to the RAM 12a when the image processing apparatus 1a is activated, and are developed in a predetermined area of the RAM 12a.

  The failure diagnosis unit 14a causes the image processing unit 11a to execute image processing using a plurality of sets of test images developed in the RAM 12a. Then, the processing result value for the test image is compared with the expected value data corresponding to each test image to diagnose the presence or absence of a failure in the image processing unit 11a. The failure diagnosis unit 14a periodically performs failure diagnosis using test images in parallel with a routine in which the image processing unit 11a performs image processing on a stereo image acquired by the stereo camera 10a.

  The output control unit 15a performs output control related to the processing result value of the stereo image based on the diagnosis result by the failure diagnosis unit 14a. Specifically, when the diagnosis result by the failure diagnosis unit 14a indicates normality, the output control unit 15a outputs the processing result regarding the stereo image subjected to image processing as valid. The processing result output as valid from the output control unit 15a is output to the vehicle control device group 2, and is used for vehicle presentation related to information presentation to the driver of the vehicle and travel safety. On the other hand, when the diagnosis result by the failure diagnosis unit 14a indicates an abnormality, fail-safe control is executed.

  The vehicle control device group 2 includes a plurality of devices that control operations of various control objects such as a body system, a power train system, and a chassis system in the vehicle. Examples of the control target include a steering device, a speaker, a display, a braking device, a driving device, and a lamp. Each device constituting the vehicle control device group 2 controls the operation of the controlled object according to the traveling state of the vehicle and the recognition result of the object by the image processing device 1. Specifically, collision avoidance, speed warning, lane departure prevention, rear-end collision warning, inter-lane warning, lane departure warning, auto high beam, sign display, full speed adaptive cruise control (ACC), lane maintenance, lane change accident prevention, blind spot Well-known vehicle control such as warning, blind spot monitor, car line change, cross traffic warning, stepping error prevention, automatic parking, etc. is executed.

[Description of image processing for each frame]
A sequence of processing steps executed by each unit of the image processing apparatus 1a will be described with reference to the flowchart of FIG. This process is periodically executed every time a pair of stereo image frames are input from the stereo camera 10a. As a premise, when the image processing apparatus 1a is started, a plurality of sets of test images and expected value data corresponding to the test images are transferred in advance from the ROM 13a to the RAM 12a.

  In S <b> 100, the image processing unit 11 a performs image processing for obtaining a parallax value between images for two frames constituting a pair of stereo images acquired from the stereo camera 10. The image processing unit 11a stores the processing result value for the stereo image in the RAM 12a.

  In S102, before the image processing in S100 ends and before the processing result value is output, the failure diagnosis unit 14a uses the plurality of sets of test images developed in the RAM 12a to use the image processing unit 11a. The image processing for obtaining the parallax value for each test image is executed. The image processing unit 11a stores the processing result value for each test image in the RAM 12a.

  In S104, the failure diagnosis unit 14a compares the processing result value for each test image stored in the RAM 12a with the expected value data corresponding to each test image previously developed in the RAM 12a, and based on the coincidence between the two. To determine whether the image processing result is normal. Here, when the processing result value for the test image and the expected value data corresponding to the test image satisfy the matching criterion, it is determined to be normal, and when the matching criterion is not satisfied, an abnormality (that is, the image processing unit 11a has failed). Is determined). On the other hand, in S108, in parallel with the process of the failure diagnosis unit 14a, various calculation processes based on the image processing result for the stereo image by the image processing unit 11a are executed in another calculation unit (not shown).

  In S110, the process of the output control unit 15a branches according to the determination result of the failure diagnosis unit 14a in S104. If the result of determination in S104 is normal (S110: normal), the output control unit 15a proceeds to S112. In S112, the output control unit 15a outputs the processing result related to the current stereo image as an effective processing result. Further, the output control unit 15a notifies the driver of notification information indicating that the function related to image processing by the image processing unit 11a can be used via the vehicle control device group 2.

  On the other hand, if the result of determination in S104 is that there is an abnormality (S110: Abnormal), the output control unit 15a proceeds to S113. In S113, the output control unit 15a branches the process depending on whether or not the event in which the abnormality is detected continues for a specified number of times. When the event in which the abnormality is detected is not continued for the specified number of times, the output control unit 15a proceeds to S114. In S114, the output control unit 15a performs fail-safe control. Specifically, the output control unit 15a discards the processing result regarding the current stereo image without outputting it. Further, the output control unit 15a notifies the driver of notification information indicating that the function relating to image processing of the image processing unit 11a is unusable via the vehicle control device group 2.

  On the other hand, when the event in which the abnormality is detected continues for a specified number of times (S113: specified number of times continues), the output control unit 15a proceeds to S115. In S115, the output control unit 15a executes a restoration operation related to the image processing unit 11a. Specifically, the output control unit 15a retransfers the test image data from the ROM 13a to the RAM 12a, and resets the test image data in the RAM 12a. Further, the system LSI constituting the image processing unit 11a is reloaded to try to reconfigure the image processing unit 11a. Note that, as a modification, a configuration in which the restoration operation is executed may be performed even if the event that the processing result value of the test image is determined to be abnormal is one time.

[Effect of the first embodiment]
The image processing apparatus 1a according to the first embodiment has the following effects. In parallel with the image processing routine for the stereo image input in the cycle, the failure diagnosis using the test image previously developed in the RAM 12a is performed periodically, so that the normal image processing routine is not hindered. The failure of the processing unit 11a can be diagnosed in real time.

  In particular, since the test image used for failure diagnosis is preliminarily developed in the RAM 12a that can be accessed at high speed by the image processing unit 11a, the processing of the failure diagnosis unit 14a is performed in accordance with the cycle of image processing for stereo images. Can be executed at high speed. Further, by using a plurality of sets of test images having different processing result values obtained by image processing for failure diagnosis, it is possible to comprehensively check the processing result values that can be taken as a system.

  If an abnormality is detected as a result of the failure diagnosis, the output control unit 15a performs fail-safe control for discarding the image processing result or notifying the abnormality, thereby causing a malfunction based on the erroneous image processing result. Can be prevented. In addition, when an event in which an abnormality is detected in failure diagnosis continues, recovery can be attempted by the output control unit 15a executing a repair operation, and the system availability is improved.

[Modification of First Embodiment]
In the first embodiment, the case has been described in which the test image and the expected value data stored in the ROM 13a provided in the image processing apparatus 1a are transferred to the RAM 12a when the image processing apparatus 1a is activated. However, the configuration is not limited to this, and the test image and the expected value data may be transferred to the RAM 12a from a storage device connected from the outside of the image processing apparatus 1a.

Second Embodiment
The configuration of the image processing apparatus 1b according to the second embodiment will be described with reference to FIG. The image processing apparatus 1b includes an image processing unit 11b, a RAM 12b, a ROM 13b, a failure diagnosis unit 14b, and an output control unit 15b. In addition, each structure of the codes | symbols 10b-15b in 2nd Embodiment respond | corresponds with each structure of the codes | symbols 10a-15a in 1st Embodiment, respectively. In the second embodiment, items different from the image processing apparatus 1a of the first embodiment (see FIG. 1) described above will be mainly described, and descriptions of configurations and functions common to the first embodiment will be omitted.

  The second embodiment is different from the first embodiment in that test image data used for failure diagnosis is stored in the stereo camera 10b. This test image is composed of a pair of images simulating a stereo image captured by the stereo camera 10b. In order to comprehensively examine the parallax values that can be taken as a system, a plurality of sets of test images configured so that the parallax values obtained by image processing are different from each other are stored in a storage device (not shown) of the stereo camera 10b. Yes. The stereo camera 10b repeatedly acquires image data composed of a pair of left and right stereo images, and outputs the acquired image data to the image processing unit 11b. In addition, the stereo camera 10b periodically outputs a test image stored in the storage device in parallel with the captured stereo image.

  The failure diagnosis unit 14b causes the image processing unit 11b to perform image processing using a plurality of sets of test images input from the stereo camera 10b. Then, the processing result value for the test image is compared with the expected value data corresponding to each test image to diagnose the presence or absence of a failure in the image processing unit 11b. The failure diagnosis unit 14b periodically performs failure diagnosis using a test image in parallel with a routine in which the image processing unit 11b performs image processing on a stereo image acquired by the stereo camera 10b.

  The RAM 12b stores a processing result value (disparity value map) that is a result of image processing performed by the image processing unit 11b on the image data input by the stereo camera 10b. The RAM 12b stores a plurality of expected value data corresponding to each of a plurality of sets of test images. This expected value data is information representing a normal result value when the image processing by the image processing unit 11b is performed on the test image input from the stereo camera 10b. The RAM 12b stores a result value that is a result of image processing performed on the test image by the image processing unit 11b.

  The ROM 13b stores original expected value data that is an original of expected value data corresponding to each test image input from the stereo camera 10b. The expected value data stored in the ROM 13b is transferred from the ROM 13b to the RAM 12b when the image processing apparatus 1b is activated, and is expanded in a predetermined area of the RAM 12b.

  The procedure of image processing executed by each unit of the image processing apparatus 1b according to the second embodiment conforms to the procedure of the flowchart of FIG. 2 already described in the first embodiment. However, in the second embodiment, S102 in FIG. 2 is the next procedure. That is, before the end of the image processing in S100 and before the processing result value is output, the failure diagnosis unit 14b uses the plurality of sets of test images input from the stereo camera 10b to perform the image processing unit 11b. The image processing for obtaining the parallax value for each test image is executed. The image processing unit 11b stores the processing result value for each test image in the RAM 12b. In the second embodiment, instead of re-transferring the test image data to the RAM 12a in the restoration operation in S115 in FIG. 2, the configuration information of the stereo camera 10b is reloaded to attempt to reconfigure the stereo camera 10b. .

[Effects of Second Embodiment]
The image processing device 1b according to the second embodiment has the following effects. In parallel with the periodic routine of normal image processing by the image processing unit 11b, failure diagnosis based on test image data acquired from the stereo camera 10b can be periodically performed. By doing so, it is possible to monitor in real time whether there is a failure related to the input line from the stereo camera 10b or the image processing unit 11b without hindering the routine of normal image processing.

<Third Embodiment>
The configuration of the image processing apparatus 1c according to the third embodiment will be described with reference to FIG. The image processing apparatus 1c includes an image processing unit 11c, a RAM 12c, a ROM 13c, a failure diagnosis unit 14c, and an output control unit 15c. In addition, each structure of the codes | symbols 10c-15c in 3rd Embodiment respond | corresponds with each structure of the codes | symbols 10a-15a in 1st Embodiment, respectively. In the third embodiment, matters different from those in the first embodiment (see FIG. 1) and the second embodiment (see FIG. 3) will be mainly described, and common to the first embodiment and the second embodiment. A description of the configuration and functions is omitted.

  The third embodiment is different from the first and second embodiments in that test image data used for failure diagnosis is stored in both the stereo camera 10c and the RAM 12c. These test images are composed of a pair of images simulating a stereo image captured by the stereo camera 10c. In order to comprehensively check the parallax values that can be taken as a system, a plurality of sets of test images configured so that the parallax values obtained by image processing are different from each other are stored in the storage device (not shown) of the stereo camera 10c and the RAM 12c. Has been.

  The stereo camera 10c repeatedly acquires image data composed of a pair of left and right stereo images, and outputs the acquired image data to the image processing unit 11c. Further, the stereo camera 10c periodically outputs a test image stored in the storage device in parallel with the captured stereo image. Further, the ROM 13c stores a test image developed on the RAM 12c, and an original test image and expected value data which are originals of expected value data corresponding to each test image. The test image and expected value data stored in the ROM 13c are transferred from the ROM 13c to the RAM 12c when the image processing apparatus 1c is started up, and developed in a predetermined area of the RAM 12c.

  In parallel with the routine in which the image processing unit 11c performs image processing on the stereo image acquired by the stereo camera 10c, the failure diagnosis unit 14c periodically switches the test images of the stereo camera 10c and the RAM 12c alternately to cause a failure. Make a diagnosis. Specifically, the failure diagnosis unit 14c causes the image processing unit 11c to perform image processing using a plurality of sets of test images developed in the RAM 12c. Then, the processing result value for the test image is compared with the expected value data corresponding to each test image to diagnose the presence or absence of abnormality. Further, the failure diagnosis unit 14c causes the image processing unit 11c to perform image processing using a plurality of sets of test images input from the stereo camera 10c. Then, the processing result value for the test image is compared with the expected value data corresponding to each test image to diagnose the presence or absence of abnormality. In addition, about the procedure which performs a failure diagnosis using each test image, it applies to the procedure of S100-S114 in the flowchart of FIG.

  Furthermore, the failure diagnosis unit 14c identifies a failure location from the diagnosis results regarding the test images of the stereo camera 10c and the RAM 12c. Then, the output control unit 15c executes a repair operation according to the failure location specified by the failure diagnosis unit 14c. This series of processing procedures will be described with reference to the flowchart of FIG.

  In S200, the failure diagnosis unit 14c refers to the result of the failure diagnosis alternately performed on the test images of the stereo camera 10c and the RAM 12c. In S202, the failure diagnosing unit 14c specifies a failure location related to the stereo camera 10c, the image processing unit 11c, or the test image based on the combination pattern of the diagnosis results related to the test images of the stereo camera 10c and the RAM 12c.

  Specifically, when the test images of the stereo camera 10c and the RAM 12c are regularly diagnosed as abnormal, the failure diagnosis unit 14c determines that the image processing unit 11c is out of order. Further, when an abnormality is diagnosed only when the test image of the stereo camera 10c is used, the failure diagnosis unit 14c determines that the test camera data stored in the stereo camera 10c or the stereo camera 10c is defective. . Further, when an abnormality is diagnosed only when the test image in the RAM 12c is used, the failure diagnosis unit 14c determines that the data of the test image stored in the RAM 12c has failed.

  In S204, the output control unit 15c executes a repair operation corresponding to the failure location specified in S202. Specifically, when it is determined that the image processing unit 11c is out of order, the output control unit 15c reloads the configuration information of the system LSI that configures the image processing unit 11c, and the image processing unit 11c re-reads. Try composition. When the test camera data stored in the stereo camera 10c or the stereo camera 10c is identified as malfunctioning, the output control unit 15c reloads the configuration information of the stereo camera 10b, and the stereo camera 10b. Attempt to reconfigure. If it is determined that the test image data stored in the RAM 12c is out of order, the output control unit 15c retransfers the test image data from the ROM 13c to the RAM 12c, and the test image data in the RAM 12c is transferred to the RAM 12c. Reset.

[Effect of the third embodiment]
By performing failure diagnosis using the test images of the stereo camera 10c and the RAM 12c, specifying the failure location, and performing a repair operation according to the specified failure location, it is possible to isolate the failure location and improve availability.

<Fourth embodiment>
The configuration of the image processing apparatus 1d according to the fourth embodiment will be described with reference to FIG. The image processing apparatus 1d includes an image processing unit 11d, a RAM 12d, a ROM 13d, a failure diagnosis unit 14d, and an output control unit 15d. In addition, each structure of the codes | symbols 10d-15d in 4th Embodiment respond | corresponds with each structure of the codes | symbols 10a-15a in 1st Embodiment, respectively. In the fourth embodiment, matters different from those in the first embodiment (see FIG. 1) are mainly described, and descriptions of configurations and functions common to the first embodiment are omitted.

  The fourth embodiment is different from the first embodiment in that a fraud test image used for a test of a fault diagnosis function is stored in the ROM 13d in addition to a normal test image used for fault diagnosis. The normal test image is the same as that described in the first embodiment. The fraud test image is composed of an image simulating a stereo image captured by the stereo camera 10d in the same manner as a normal test image. However, the fraud test image is modified so that the processing result value when the image processing is normally performed in the image processing unit 11d is illegal with respect to the expected value data.

  In the image processing apparatus 1d according to the fourth embodiment, at the stage of initialization accompanying activation, first, the fraud test image stored in the ROM 13d is transferred to the RAM 12d and developed in a predetermined area of the RAM 12d. The failure diagnosis unit 14d tests whether or not the failure diagnosis function operates normally by using the fraud test image transferred to the RAM 12d at the time of initialization. Then, on the condition that the failure diagnosis function is confirmed to operate normally, a normal test image is transferred from the ROM 13d to the RAM 12d, and thereafter normal image processing (see FIG. 2) is started. . A series of processing procedures will be described with reference to the flowchart of FIG.

  In S300, the failure diagnosis unit 14d transfers the fraud test image stored in the ROM 13d to the RAM 12d on the condition that the initialization associated with the activation of the image processing apparatus 1d has been performed. The transferred fraud test image is developed in a predetermined area of the RAM 12d. In S302, the failure diagnosis unit 14d causes the image processing unit 11d to execute image processing for obtaining a parallax value for the fraud test image using the fraud test image developed in the RAM 12d. The image processing unit 11d stores the processing result value for the fraud test image in the RAM 12d.

  In S304, the failure diagnosis unit 14d compares the processing result value for the fraud test image stored in the RAM 12d with a predetermined failure expectation value that defines “failure”, and determines a match between them. When the processing result value for the fraud test image and the expected failure value satisfy the matching criterion, it is determined to match, and when the matching criterion is not satisfied, it is determined to be mismatched. In S306, the failure diagnosis unit 14d branches the process according to the determination result in S304.

  If it is determined that the processing result value for the fraud test image matches the expected failure value (S306: match), the failure diagnosis unit 14d proceeds to S308. In this case, since it has been confirmed that the processing result value for the illegal image data is correctly determined as “failure”, it is determined that the failure diagnosis function operates normally. In S308, the failure diagnosis unit 14d resets the RAM 12d and transfers a normal test image stored in the ROM 13d to the RAM 12d. The processing procedure after S308 follows the procedure of the flowchart of FIG. 2 already described in the first embodiment.

  On the other hand, when it is determined that the processing result value for the fraud test image does not match the expected failure value (S306: mismatch), the failure diagnosis unit 14d proceeds to S310. In this case, since it has not been confirmed that the processing result value for the illegal image data is correctly determined as “failure”, it is determined that the failure diagnosis function is not operating normally. In S310, the output control unit 15d performs fail-safe control. Specifically, the output control unit 15d notifies the driver of notification information indicating that the function relating to the image processing of the image processing unit 11d cannot be used via the vehicle control device group 2.

[Other ideas]
Hereinafter, various devices applicable to the embodiment will be described.
When causing the image processing unit to perform image processing on the test image, the test image alone may be configured to perform image processing, or a test image pattern is incorporated in a predetermined area of a real image input from a stereo camera. In this state, the image processing for the actual image and the test image may be collectively performed.

  As a specific example of the latter, as illustrated in FIG. 8, it is conceivable to incorporate test image patterns 31 a to 30 d into blank areas adjacent in the vertical and horizontal directions of the actual image 30 actually captured by the stereo camera.

  FIG. 9 shows an example in which a test image pattern 41 a is incorporated between lines (raster lines) constituting the actual image 40 actually captured by the stereo camera 10. In this example, when the parallax calculation method is different for each region divided on the boundary line defined at a certain position in the horizontal direction of the actual image 40, it is conceivable to incorporate the test image pattern 41a between the boundaries. FIG. 9 shows a case where two different types of test images 41b and 41c are incorporated in the right region of the actual image 40. In this way, the configuration may be such that the pattern of the test image changes from a certain boundary in the region where the test image is incorporated.

  1a, 1b, 1c, 1d ... image processing apparatus, 10a, 10b, 10c, 10d ... stereo camera, 11a, 11b, 11c, 11d ... image processing unit, 12a, 12b, 12c, 12d ... RAM, 13a, 13b, 13c , 13d ... ROM, 14a, 14b, 14c, 14d ... fault diagnosis unit, 15a, 15b, 15c, 15d ... output control unit, 2 ... vehicle control device group.

Claims (8)

Image processing means (11) that performs image processing for each of the cycles of the picked-up images periodically picked up by the image pickup means (10);
In parallel with the periodic image processing performed by the image processing unit on the captured image, the image processing unit performs image processing on diagnostic image data stored in advance in a main storage device directly accessible by the image processing unit. By performing the comparison, the image processing result for the diagnostic image data is compared with the expected value data representing the standard of the normal processing result of the image processing for the diagnostic image data, and the image processing result for the diagnostic image data Diagnostic means (14) for determining whether or not is normal;
On the condition that the image processing result for the diagnostic image data is determined to be normal by the diagnostic means, the processing result relating to the image processing for the captured image is output as valid to the predetermined control means (2). Output control means (15) for
The diagnostic unit causes the image processing unit to perform image processing on the embedded image in which the diagnostic image data is embedded in a predetermined region of the captured image input from the imaging unit, thereby performing image processing on the captured image. In parallel with the image processing means to perform image processing on the diagnostic image data,
An image processing apparatus. Image processing means (11) that performs image processing for each of the cycles of the picked-up images periodically picked up by the image pickup means (10);
In parallel with the periodic image processing performed by the image processing unit on the captured image, the image processing unit performs image processing on diagnostic image data stored in advance in a main storage device directly accessible by the image processing unit. By performing the comparison, the image processing result for the diagnostic image data is compared with the expected value data representing the standard of the normal processing result of the image processing for the diagnostic image data, and the image processing result for the diagnostic image data Diagnostic means (14) for determining whether or not is normal;
On the condition that the image processing result for the diagnostic image data is determined to be normal by the diagnostic means, the processing result relating to the image processing for the captured image is output as valid to the predetermined control means (2). Output control means (15) for
The diagnostic unit causes the image processing unit to perform image processing on image data for fraud diagnosis prepared so that an image processing result is an invalid value with respect to the expected value data at the time of start-up. Determine whether the image processing result for the diagnostic image data meets a predetermined failure condition,
On the condition that the diagnostic processing determines that the image processing result for the fraud diagnostic image data meets the failure condition, the diagnostic storage unit stores the diagnostic image data from the auxiliary storage unit to the main storage device. Transferring the image data, and thereafter, the diagnostic means determines whether or not the image processing result for the diagnostic image data is normal using the diagnostic image data stored in the main storage device;
An image processing apparatus. Image processing means (11) that performs image processing for each of the cycles of the picked-up images periodically picked up by the image pickup means (10);
In parallel with the periodic image processing performed by the image processing unit on the captured image, the image processing unit performs image processing on diagnostic image data stored in advance in a main storage device directly accessible by the image processing unit. By performing the comparison, the image processing result for the diagnostic image data is compared with the expected value data representing the standard of the normal processing result of the image processing for the diagnostic image data, and the image processing result for the diagnostic image data Diagnostic means (14) for determining whether or not is normal;
On the condition that the image processing result for the diagnostic image data is determined to be normal by the diagnostic means, the processing result relating to the image processing for the captured image is output as valid to the predetermined control means (2). Output control means (15) for
The diagnostic unit causes the image processing unit to alternately perform image processing on diagnostic image data stored in advance in the imaging unit and diagnostic image data stored in the main storage device, respectively. To determine whether the image processing result for the diagnostic image data is normal,
A specifying unit (14c) for specifying a failure location according to a pattern of a determination result by the diagnostic unit for diagnostic image data of each of the imaging unit and the main storage device;
Comprising specific repairing means (15c) for performing a repairing operation on the fault location specified by the specifying means;
An image processing apparatus. The image processing apparatus according to any one of claims 1 to 3 ,
A plurality of types of diagnostic image data corresponding to a plurality of types of image processing results by the image processing means are stored in the main storage device,
The diagnostic unit causes the image processing unit to perform image processing on each of the plurality of types of diagnostic image data, thereby performing image processing results on the plurality of types of diagnostic image data and a plurality of types of the diagnostics. Comparing a plurality of types of expected value data corresponding to each of the image data, and determining whether the image processing result is normal,
An image processing apparatus. It claims 1 image processing apparatus according to any one of claims 4,
Transferring the diagnostic image data from the auxiliary storage means storing the diagnostic image data to the main storage device at the time of startup;
An image processing apparatus. The image processing apparatus according to any one of claims 1 to 5 ,
When the diagnostic means diagnoses that the image processing result for the diagnostic image data is abnormal, the processing result relating to the image processing for the captured image is discarded without being output to the control means, and Providing fail-safe means (15) for notifying abnormality,
An image processing apparatus. The image processing apparatus according to any one of claims 1 to 6 ,
A repairing means (15a) for executing a repairing operation on the image processing means is provided on the condition that an event diagnosed by the diagnostic means as an abnormal image processing result for the diagnostic image data has continued for a specified number of times. ,
An image processing apparatus. Image processing means (11b) that performs image processing for each cycle of the picked-up images picked up periodically by the image pickup means (10b);
In parallel with periodic image processing performed by the image processing unit on the captured image, diagnostic image data stored in advance in the imaging unit is acquired and the image processing unit performs image processing. The image processing result for the diagnostic image data is compared with the expected value data representing the standard of the normal processing result of the image processing for the diagnostic image data, and the image processing result for the diagnostic image data is normal Diagnostic means (14b) for determining whether or not,
On the condition that the image processing result for the diagnostic image data is determined to be normal by the diagnostic means, the processing result relating to the image processing for the captured image is output as valid to the predetermined control means (2). Output control means (15b) for
The diagnostic unit causes the image processing unit to perform image processing on the embedded image in which the diagnostic image data is embedded in a predetermined region of the captured image input from the imaging unit, thereby performing image processing on the captured image. In parallel with the image processing means to perform image processing on the diagnostic image data,
An image processing apparatus.

Images