JP5207823B2 - Image judging apparatus and image judging method - Google Patents

Description

  The present invention relates to an image determination apparatus and an image determination method, and more particularly, to an image determination apparatus and an image determination method for notifying that an image is abnormal by calculating a total amount of pixel values related to all pixels of an inspection image. Is.

  Currently, systems that support driving of a vehicle by acquiring captured images from an in-vehicle camera that continuously captures the periphery of the vehicle and outputting the images as a video (moving image) on the in-vehicle monitor are becoming popular. In the system, when some failure occurs, the same image may be repeatedly output to the in-vehicle monitor. In this case, the video output to the in-vehicle monitor freezes (is stopped without changing). When the video output to the in-vehicle monitor freezes, there is a risk that the occupant misidentifies the situation around the vehicle due to the frozen video. Therefore, when the same image is repeatedly output to the in-vehicle monitor, it is necessary to immediately notify the passenger that the video output to the in-vehicle monitor is frozen. This is to make the passenger immediately notice that the video is frozen so that the passenger does not mistake the situation around the vehicle.

An imaging display system for notifying that a captured image output from an in-vehicle camera has not changed is known in connection with a technique for notifying a passenger that a video output to an in-vehicle monitor is frozen. (For example, refer to Patent Document 1). The imaging display system described in Patent Literature 1 notifies that the captured image output from the in-vehicle camera has not changed by comparing the luminance value of each pixel of the current and previous captured images.
Japanese Patent Laid-Open No. 2004-064440

  However, the imaging display system described in Patent Document 1 has a high processing load because pixel values (luminance values) for each pixel of a captured image are compared. Therefore, the imaging display system described in Patent Document 1 indicates that the captured image output from the in-vehicle camera has not changed, that is, as a result, the video output to the in-vehicle monitor is frozen. There is a problem that the passenger may not be notified immediately. In particular, when the imaging speed of the in-vehicle camera is fast or when the number of pixels of the captured image is large, the processing load is further increased, which becomes serious.

  The present invention has been made to solve such problems. That is, it is an object to be able to immediately notify the passenger that the same image is repeatedly output to the in-vehicle monitor, that is, that the video output to the in-vehicle monitor is frozen.

In order to solve the above-described problem, in the present invention, an image continuously acquired from the outside or an image continuously generated based on the image is continuously acquired as an inspection image, and the acquired inspection image A total amount calculation unit that calculates a total amount that is a sum of pixel values related to all pixels, and a difference amount calculation unit that calculates a difference amount between both total amounts calculated by the total amount calculation unit from both inspection images acquired last time and this time And whether each of the one or more difference amounts calculated by the difference amount calculation unit from two or more total amounts calculated by the total amount calculation unit from two or more predetermined number of consecutively acquired inspection images is zero. A determination unit for determining, a notification unit for notifying that there is an abnormality in the image when the determination unit determines that all of the difference amounts of 1 or more are zero, and continuous imaging of the vehicle periphery at a predetermined timing Continuously capture images from in-vehicle camera A captured image acquisition unit that Tokusuru is provided with an output image generating unit which generates in succession the output image to be output on the output screen based on the captured image acquired by the captured image acquisition unit.
Here, the total amount calculation unit continuously acquires the captured images continuously acquired by the captured image acquisition unit as the inspection image, calculates the total amount related to the captured image as the captured image total amount, and outputs the output image. The output image continuously generated by the generation unit is continuously acquired as the inspection image, and the total amount related to the output image is calculated as the output image total amount.
The difference amount calculation unit calculates a captured image difference amount that is a difference amount related to the total amount of both captured images calculated by the total amount calculation unit from both captured images acquired by the captured image acquisition unit last time and this time, and an output image An output image difference amount that is a difference amount related to both output image total amounts calculated by the total amount calculation unit from both output images generated by the generation unit last time and this time is calculated.
The determination unit is configured to calculate one or more picked-up image difference amounts calculated by the difference amount calculating unit from two or more picked-up image total amounts calculated by the total amount calculating unit from the picked-up images acquired by the predetermined number of consecutive images. The difference amount calculation unit calculates whether or not each is zero, and the difference amount calculation unit calculates from two or more output image total amounts calculated by the total amount calculation unit from the output images successively generated by the output image generation unit. It is determined whether each of the one or more output image difference amounts is zero.
When the determination unit determines that all of the one or more captured image difference amounts are zero, or when the determination unit determines that all of the one or more output image difference amounts are zero, the abnormality is detected. Notify that there is.

  According to the present invention configured as described above, compared to a method of comparing pixel values for each pixel of both inspection images (both captured images in the imaging display system described in Patent Document 1), Since it is only necessary to compare the sum of the pixel values of the pixels, it can be determined with a small amount of calculation whether the video output to the in-vehicle monitor is frozen. Accordingly, it is possible to immediately notify the passenger that the video output to the in-vehicle monitor is frozen.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a diagram illustrating a configuration example of an image determination apparatus 10 according to the first embodiment of the present invention. The image determination device 10 is mounted on a vehicle. The image determination apparatus 10 is connected to the in-vehicle camera 5, the monitor 190, and the speaker 192 via an in-vehicle network. As shown in FIG. 1, the image determination device 10 includes a captured image acquisition unit 100, an input buffer 110, an output image generation unit 120, an output buffer 130, an output control unit 140, a total amount calculation unit 152, a difference amount calculation unit 162, A determination unit 172, a notification unit 180, and a notification information storage unit 182 are provided. The in-vehicle camera 5 continuously captures the periphery of the vehicle at a predetermined timing (for example, 30 frames / second).

  The captured image acquisition unit 100 continuously acquires captured images continuously captured by the in-vehicle camera 5. The captured image acquisition unit 100 stores the captured image acquired from the in-vehicle camera 5 in the input buffer 110 every time it acquires the captured image from the in-vehicle camera 5. Further, the captured image acquisition unit 100 supplies the captured image acquired from the in-vehicle camera 5 to the total amount calculation unit 152 every time the captured image is acquired from the in-vehicle camera 5.

  The output image generation unit 120 sequentially reads captured images that are sequentially stored in the input buffer 110, and outputs an image in a format (hereinafter referred to as “output image”) that is output on the monitor 190 (output screen) from the read captured image. Generate. The output image generation unit 120 stores the generated output image in the output buffer 130 every time an output image is generated.

  The output control unit 140 sequentially reads the output images that are sequentially stored in the output buffer 130 and outputs the read output images to the monitor 190. That is, captured images continuously captured by the in-vehicle camera 5 are output as a video (moving image) to the monitor 190 by the captured image acquisition unit 100, the input buffer 110, the output image generation unit 120, the output buffer 130, and the output control unit 140. The

  The notification information storage unit 182 stores information for notifying that the video output to the monitor 190 is frozen (hereinafter referred to as “freeze notification information”). Specifically, the notification information storage unit 182 stores text-format freeze notification information to be output to the monitor 190 (for example, the phrase “video is frozen”).

  The total amount calculation unit 152 sequentially acquires captured images from the captured image acquisition unit 100 as images for inspecting whether or not the video output to the monitor 190 is frozen (hereinafter referred to as “inspection image”). To do. Each time the captured image is acquired from the captured image acquisition unit 100, the total amount calculation unit 152 calculates the total amount that is the sum of the pixel values of all the pixels for each captured image acquired from the captured image acquisition unit 100.

  Specifically, when the total amount calculation unit 152 acquires a colored captured image from the captured image acquisition unit 100, the G component value (hereinafter referred to as “G value”) in the RGB color space is determined according to the image format. Or the Y component (luminance value) values in the YUV color space for the total number of pixels to calculate the total amount for each captured image. Further, when a colorless (monochrome) captured image is acquired from the captured image acquisition unit 100, the total amount calculation unit 152 calculates a total amount using a gray value (tone value). The total amount calculation unit 152 sequentially supplies the calculated total amount to the difference amount calculation unit 162 every time the total amount is calculated for each captured image.

  The difference amount calculation unit 162 sequentially acquires the total amount of all pixel values related to each captured image from the total amount calculation unit 152. The difference amount calculation unit 162 temporarily stores the total amount acquired from the total amount calculation unit 152. Each time the difference amount calculation unit 162 acquires the total amount from the total amount calculation unit 152, the difference amount calculation unit 162 acquired the previous time (the total amount previously calculated by the total amount calculation unit 152. And the difference amount calculation unit 162 acquired this time (the total amount calculated this time by the total amount calculation unit 152, hereinafter referred to as “the current total amount”). calculate. The difference amount calculation unit 162 sequentially supplies the calculated difference amount to the determination unit 172 every time the difference amount is calculated.

  The determination unit 172 sequentially acquires the difference amount from the difference amount calculation unit 162. Whenever the difference amount is acquired from the difference amount calculation unit 162, the determination unit 172 calculates the difference amount calculation unit from two or more total amounts calculated from the captured images that the total amount calculation unit 152 continuously acquires a predetermined number of two or more. It is determined whether each of the one or more difference amounts calculated by 162 is zero. That is, the determination unit 172 determines whether or not each of the one or more difference amounts acquired continuously from the difference amount calculation unit 162 is zero.

  For example, when the predetermined number is 2, the determination unit 172 determines the difference amount acquired by the determination unit 172 this time (the difference amount calculated by the difference amount calculation unit 162 this time from the previous total amount and the current total amount. It is determined whether or not the “difference amount” is zero. When the predetermined number is 3, the determination unit 172 determines the difference amount previously acquired by the determination unit 172 (the difference amount previously calculated by the difference amount calculation unit 162 from the previous total amount and the previous total amount. Whether or not the difference amount of this time is zero). That is, the determination unit 172 has zero each of n−1 difference amounts in a predetermined number n (n is an integer of 2 or more) of captured images that the captured image acquisition unit 100 continuously acquires from the in-vehicle camera 5. Judge whether there is. When the predetermined number is 3 or more, the determination unit 172 temporarily stores the difference amount acquired from the difference amount calculation unit 162.

  The determination unit 172 has zero of one or more difference amounts calculated by the difference amount calculation unit 162 from two or more total amounts calculated from the captured images continuously acquired by the total amount calculation unit 152 by two or more predetermined numbers. If it is determined that the total amount is constant, a notification indicating that the total amount is constant (hereinafter referred to as “total amount constant notification”) is supplied (notified) to the notification unit 180.

  The notification unit 180 obtains a total sum constant notification from the determination unit 172. When the notification unit 180 acquires the constant total amount notification from the determination unit 172, the notification unit 180 reads out the freeze notification information in the text format from the notification information storage unit 182 and outputs it to the monitor 190.

  Hereinafter, the operation of the image determination apparatus 10 will be described. FIG. 2 is a flowchart showing an operation example of the image determination apparatus 10 shown in FIG. The flowchart shown in FIG. 2 shows the operation from when the captured image acquisition unit 100 acquires a captured image from the in-vehicle camera 5 to when the notification unit 180 notifies the freeze notification information.

  In FIG. 2, the total amount calculation unit 152 acquires a captured image as an inspection image from the captured image acquisition unit 100 (step S100). The total amount calculation unit 152 calculates the total amount of pixel values related to all pixels of the captured image acquired from the captured image acquisition unit 100 (step S110). The total amount calculation unit 152 supplies the calculated total amount to the difference amount calculation unit 162. The difference amount calculation unit 162 acquires the total amount from the total amount calculation unit 152. The difference amount calculation unit 162 calculates the difference amount between the previous total amount and the current total amount (step S120). The difference amount calculation unit 162 supplies the calculated difference amount to the determination unit 172.

  The determination unit 172 acquires the difference amount from the difference amount calculation unit 162. The determination unit 172 has zero of one or more difference amounts calculated by the difference amount calculation unit 162 from two or more total amounts calculated from the captured images continuously acquired by the total amount calculation unit 152 by two or more predetermined numbers. Is determined (step S130).

  When the determination unit 172 determines that any of the one or more difference amounts is zero (step S130: Yes), the determination unit 172 supplies the notification of the total amount to the notification unit 180. The notification unit 180 that has acquired the constant total amount notification from the determination unit 172 reads out the text-format freeze notification information “video in freeze” stored in the notification information storage unit 182 and outputs the read-out notification information to the monitor 190 (step S140). On the other hand, when the determination unit 172 determines that any one of the one or more difference amounts is not zero (step S130: No), the process skips step S140 and proceeds to step S150.

  Subsequent to step S130 (No) or step S140, an end determination unit (not shown) determines whether or not to stop the output function of the freeze notification information (step S150). If the end determination unit determines that the freeze notification information output function is to be stopped (step S150: Yes), this flowchart ends. On the other hand, when the end determination unit determines that the freeze notification information output function is not stopped (step S150: No), the process returns to step S100. The end determination unit determines to stop the function of outputting freeze notification information when, for example, key-off is detected.

  As described above, according to the image determination device 10, it is only necessary to compare the total amount of the pixel values of all the pixels of the inspection image. It is possible to determine with a small amount of calculation whether or not the video output to is frozen. Accordingly, it is possible to immediately notify the passenger that the video output to the monitor 190 is frozen.

  Next, another embodiment of the present invention will be described with reference to the drawings. FIG. 3 is a diagram illustrating a configuration example of the image determination apparatus 20 according to the second embodiment of the present invention. FIG. 4 is a diagram illustrating an example of information stored in the notification information storage unit 282 illustrated in FIG. Similar to the image determination device 10, the image determination device 20 is mounted on a vehicle and is connected to the in-vehicle camera 5, the monitor 290, and the speaker 292 via the in-vehicle network.

  As shown in FIG. 3, the image determination device 20 includes a captured image acquisition unit 200, an input buffer 210, an output image generation unit 220, an output buffer 230, an output control unit 240, a total amount calculation unit 250, a difference amount calculation unit 260, A determination unit 270, a notification unit 280, and a notification information storage unit 282 are provided. The total amount calculation unit 250 further includes a first total amount calculation unit 252, a second total amount calculation unit 254, a third total amount calculation unit 256, and a fourth total amount calculation unit 258. The difference amount calculation unit 260 further includes a first difference amount calculation unit 262, a second difference amount calculation unit 264, a third difference amount calculation unit 266, and a fourth difference amount calculation unit 268. The determination unit 270 further includes a first determination unit 272, a second determination unit 274, a third determination unit 276, and a fourth determination unit 278.

  The captured image acquisition unit 200 continuously acquires captured images continuously captured by the in-vehicle camera 5 as in the captured image acquisition unit 100 of the image determination device 10 illustrated in FIG. 1. The captured image acquisition unit 200 stores the captured image acquired from the in-vehicle camera 5 in the input buffer 210 every time the acquired image is acquired from the in-vehicle camera 5. Further, the captured image acquisition unit 200 supplies the captured image acquired from the in-vehicle camera 5 to the first total amount calculation unit 252 every time the captured image is acquired from the in-vehicle camera 5.

  The output image generation unit 220 sequentially reads captured images sequentially stored in the input buffer 210 and generates an output image. The output image generation unit 220 stores the generated output image in the output buffer 230 every time an output image is generated. The output image generation unit 220 supplies the generated output image to the third total amount calculation unit 256 every time an output image is generated.

  The output control unit 240 sequentially reads the output images sequentially stored in the output buffer 230 and outputs the read output images to the monitor 290, similarly to the output control unit 140 of the image determination apparatus 10 illustrated in FIG. That is, captured images continuously captured by the in-vehicle camera 5 are output as a video (moving image) to the monitor 290 by the captured image acquisition unit 200, the input buffer 210, the output image generation unit 220, the output buffer 230, and the output control unit 240. The

  The notification information storage unit 282 stores freeze notification information and information for notifying a failure location that can be regarded as a cause of the freeze (hereinafter referred to as “failure location notification information”). Specifically, as shown in FIG. 4, the notification information storage unit 282 determines whether or not there is a constant total amount notification from the first determination unit 272, the second determination unit 274, the third determination unit 276, or the fourth determination unit 278. In association with the notification presence / absence pattern based on the text, freeze notification information and failure location notification information in text format are stored.

  In the example shown in FIG. 4, the notification information storage unit 282 is associated with the notification presence / absence pattern 1 in the form of text-type freeze notification information (the word “video is frozen”) and the text-format failure location notification information (“ The word “There is an abnormality in either the in-vehicle camera or the captured image acquisition unit”) is stored. In addition, the notification information storage unit 282 associates with the notification presence / absence pattern 2 into text-format freeze notification information (the word “video freeze”) and text-format failure location notification information (“input buffer is abnormal”). ")" Is memorized. In addition, the notification information storage unit 282 is associated with the notification presence / absence pattern 3 in a text-format freeze notification information (the word “video freeze”) and a text-format failure location notification information (“in the output image generation unit”). The word “abnormal” is stored. In addition, the notification information storage unit 282 associates with the notification presence / absence pattern 4 in a text-format freeze notification information (the word “video frozen”) and a text-format failure location notification information (“the output buffer is abnormal”). ")" Is memorized.

  The first total amount calculation unit 252 sequentially acquires captured images as inspection images from the captured image acquisition unit 200. The second total amount calculation unit 254 sequentially acquires captured images sequentially read from the input buffer 210 as inspection images. The third total amount calculation unit 256 sequentially acquires output images from the output image generation unit 220 as inspection images. The fourth total amount calculation unit 258 sequentially acquires output images sequentially read from the output buffer 230 as inspection images.

  Each time the first total amount calculation unit 252 acquires a captured image from the captured image acquisition unit 200, the first total amount calculation unit 252 calculates the total amount of all the pixel values for each captured image, similar to the total amount calculation unit 152 of the image determination device 10. It supplies to the 1st difference amount calculation part 262 sequentially. Similarly, every time a captured image is acquired from the input buffer 210, the second total amount calculation unit 254 calculates the total amount of all pixel values for each captured image and sequentially supplies the total amount to the second difference amount calculation unit 264. Each time the output image is acquired from the output image generation unit 220, the third total amount calculation unit 256 calculates the total amount of all the pixel values for each output image, and sequentially supplies it to the third difference amount calculation unit 266. Every time an output image is acquired from the output buffer 230, the fourth total amount calculation unit 258 calculates a total amount of all pixel values for each output image and sequentially supplies the total amount to the fourth difference amount calculation unit 268.

  The first difference amount calculation unit 262 sequentially acquires the total amount related to each captured image from the first total amount calculation unit 252 and temporarily stores the total amount. Similarly, the second difference amount calculation unit 264 sequentially acquires the total amount related to each captured image from the second total amount calculation unit 254 and temporarily stores the total amount. The third difference amount calculation unit 266 sequentially acquires the total amount related to each output image from the third total amount calculation unit 256 and temporarily stores it. The fourth difference amount calculation unit 268 sequentially acquires the total amount related to each output image from the fourth total amount calculation unit 258 and temporarily stores it.

  Each time the first difference amount calculation unit 262 acquires the total amount from the first total amount calculation unit 252, the first difference amount calculation unit 262 calculates the difference amount in the same manner as the difference amount calculation unit 162 of the image determination apparatus 10 and sends it to the first determination unit 272. Supply sequentially. Similarly, every time the second difference amount calculation unit 264 acquires the total amount from the second total amount calculation unit 254, the second difference amount calculation unit 264 calculates the difference amount and sequentially supplies it to the second determination unit 274. The third difference amount calculation unit 266 calculates the difference amount and sequentially supplies it to the third determination unit 276 every time the total amount is acquired from the third total amount calculation unit 256. The fourth difference amount calculation unit 268 calculates the difference amount and sequentially supplies it to the fourth determination unit 278 every time the total amount is acquired from the fourth total amount calculation unit 258.

  The first determination unit 272 sequentially acquires the difference amount from the first difference amount calculation unit 262. Similarly, the second determination unit 274 sequentially acquires the difference amount from the second difference amount calculation unit 264. The third determination unit 276 sequentially acquires the difference amount from the third difference amount calculation unit 266. The fourth determination unit 278 sequentially acquires the difference amounts from the fourth difference amount calculation unit 268.

  Each time the first determination unit 272 acquires the difference amount from the first difference amount calculation unit 262, the first total amount calculation unit 252 continuously receives a predetermined number of two or more, like the determination unit 172 of the image determination apparatus 10. It is determined whether each of the one or more difference amounts calculated by the first difference amount calculation unit 262 is zero from the two or more total amounts calculated from the acquired captured image, and all the difference amounts are zero. If it is determined, the total amount constant notification is supplied to the notification unit 280.

  Similarly, every time the second determination unit 274 acquires the difference amount from the second difference amount calculation unit 264, the second total amount calculation unit 254 calculates 2 from the captured images acquired successively by a predetermined number of 2 or more. It is determined whether each of the one or more difference amounts calculated by the second difference amount calculation unit 264 is zero from the above total amount, and when it is determined that any of the difference amounts is zero, the total amount is constant. The notification is supplied to the notification unit 280.

  Each time the third determination unit 276 acquires a difference amount from the third difference amount calculation unit 266, the third total amount calculation unit 256 calculates two or more sums calculated from two or more predetermined numbers of output images continuously acquired. It is determined whether or not each of the one or more difference amounts calculated by the third difference amount calculation unit 266 from the amount is zero, and when it is determined that any of the difference amounts is zero, a notification of the total amount constant is notified. Part 280.

  Each time the fourth determination unit 278 acquires a difference amount from the fourth difference amount calculation unit 268, the fourth total amount calculation unit 258 calculates two or more sums calculated from two or more predetermined consecutively acquired output images. It is determined whether or not each of the one or more difference amounts calculated by the fourth difference amount calculation unit 268 from the amount is zero, and when it is determined that any of the difference amounts is zero, a total sum amount constant notification is notified. Part 280. Note that the determination unit 270 (the first determination unit 272, the second determination unit 274, the third determination unit 276, and the fourth determination unit 278) has a predetermined number as in the determination unit 172 of the image determination apparatus 10 illustrated in FIG. In the case of 3 or more, the acquired difference amount is temporarily stored.

  The notification unit 280 acquires the total amount constant notification from the determination unit 270 (the first determination unit 272, the second determination unit 274, the third determination unit 276, and the fourth determination unit 278). When the notification unit 280 acquires the total amount constant notification from the first determination unit 272 (regardless of whether the second determination unit 274, the third determination unit 276, and the fourth determination unit 278 acquire the notification), the notification presence / absence pattern The text-format freeze notification information “video is frozen” and the text-format failure location notification information “There is an abnormality in either the in-vehicle camera or the captured image acquisition unit” stored in the notification information storage unit 282 in association with 1 To the monitor 290.

  On the other hand, when the notification unit 280 does not acquire the total amount constant notification from the first determination unit 272 and acquires from the second determination unit 274 (presence / absence of acquisition from the third determination unit 276 and the fourth determination unit 278) The text-format freeze notification information “video being frozen” and the text-format failure location notification information “input buffer is abnormal” are stored in the notification information storage unit 282 in association with the notification presence / absence pattern 2 To the monitor 290.

  In addition, the notification unit 280 does not acquire the constant total amount notification from the second determination unit 274 and does not acquire the notification from the third determination unit 276 (whether or not it is acquired from the first determination unit 272 and the fourth determination unit 278) Text format freeze notification information “video being frozen” and text format failure location notification information “There is an abnormality in the output image generation unit”. Is output to the monitor 290.

  Further, the notification unit 280 does not acquire the total amount constant notification from the third determination unit 276 and acquires it from the fourth determination unit 278 (presence / absence of acquisition from the first determination unit 272 and the second determination unit 274) The text-format freeze notification information “video being frozen” and the text-format failure location notification information “the output buffer is abnormal” are stored in the notification information storage unit 282 in association with the notification presence / absence pattern 4 To the monitor 290.

  Hereinafter, the operation of the image determination apparatus 20 will be described. FIG. 5 is a flowchart illustrating an operation example of the image determination apparatus 20 illustrated in FIG. 3. The flowchart shown in FIG. 5 shows the operation from when the captured image acquisition unit 200 acquires a captured image from the in-vehicle camera 5 until the notification unit 280 notifies the freeze notification information and the failure location notification information.

  In FIG. 5, step S <b> 10 is a process of calculating the difference amount based on the captured image before being stored in the input buffer 210. Specifically, first, the first total amount calculation unit 252 acquires a captured image as a test image from the captured image acquisition unit 200. The first total amount calculation unit 252 calculates the total amount of all pixel values for the captured image acquired from the captured image acquisition unit 200 and supplies the calculated total amount to the first difference amount calculation unit 262. And the 1st difference amount calculation part 262 which acquired the sum total amount from the 1st total amount calculation part 252 calculates a difference amount (step S10). The first difference amount calculation unit 262 supplies the calculated difference amount to the first determination unit 272.

  The first determination unit 272 that has acquired the difference amount from the first difference amount calculation unit 262 calculates the first difference from two or more total amounts calculated from the captured images continuously acquired by the first total amount calculation unit 252 by a predetermined number of two or more. It is determined whether or not any one or more difference amounts calculated by the one difference amount calculation unit 262 are zero (step S12). When the first determination unit 272 determines that any of the one or more difference amounts is zero (step S12: Yes), the first determination unit 272 supplies the total amount constant notification to the notification unit 280.

  The notification unit 280 that has acquired the constant total amount notification from the first determination unit 272 is associated with the notification presence / absence pattern 1 and is stored in the notification information storage unit 282 in the text-format freeze notification information “in video freeze” and in the text format. The failure location notification information “There is an abnormality in either the in-vehicle camera or the captured image acquisition unit” is read and output to the monitor 290 (step S14). And step S44 is skipped from step S20 and it progresses to step S50.

  On the other hand, when the first determination unit 272 determines that any one of the one or more difference amounts is not zero (step S12: No), the process proceeds to step S20. Step S20 is a process of calculating the difference amount based on the captured image read from the input buffer 210. Specifically, first, the second total amount calculation unit 254 acquires a captured image read from the input buffer 210 as an inspection image. The second total amount calculation unit 254 calculates the total amount of all pixel values for the captured image acquired from the input buffer 210 and supplies the calculated total amount to the second difference amount calculation unit 264. And the 2nd difference amount calculation part 264 which acquired the sum total amount from the 2nd total amount calculation part 254 calculates a difference amount (step S20). The second difference amount calculation unit 264 supplies the calculated difference amount to the second determination unit 274.

  The second determination unit 274 that has acquired the difference amount from the second difference amount calculation unit 264 is configured to calculate the second difference from the two or more total amounts calculated from the captured images continuously acquired by the second total amount calculation unit 254 by a predetermined number of two or more. It is determined whether or not any of the one or more difference amounts calculated by the two difference amount calculation unit 264 is zero (step S22). When the second determination unit 274 determines that any one of the one or more difference amounts is zero (step S22: Yes), the second determination unit 274 supplies the total amount constant notification to the notification unit 280.

  The notification unit 280 that has acquired the constant total amount notification from the second determination unit 274 is associated with the notification presence / absence pattern 2 and is stored in the notification information storage unit 282. The failure location notification information “There is an abnormality in the input buffer” is read and output to the monitor 290 (step S24). And step S44 is skipped from step S30 and it progresses to step S50.

  On the other hand, when the second determination unit 274 determines that any one of the one or more difference amounts is not zero (step S22: No), the process proceeds to step S30. Step S30 is a process of calculating the difference amount based on the output image before being stored in the output buffer 230. Specifically, first, the third total amount calculation unit 256 acquires an output image from the output image generation unit 220 as an inspection image. The third total amount calculation unit 256 calculates the total amount of all pixel values for the output image acquired from the output image generation unit 220 and supplies the calculated total amount to the third difference amount calculation unit 266. And the 3rd difference amount calculation part 266 which acquired the sum total amount from the 3rd total amount calculation part 256 calculates a difference amount (step S30). The third difference amount calculation unit 266 supplies the calculated difference amount to the third determination unit 276.

  The third determination unit 276 that has acquired the difference amount from the third difference amount calculation unit 266 calculates the second sum from the two or more total amounts calculated from the output images continuously acquired by the third total amount calculation unit 256 by a predetermined number of two or more. It is determined whether or not any of the one or more difference amounts calculated by the three difference amount calculation unit 266 is zero (step S32). When the third determination unit 276 determines that any of the one or more difference amounts is zero (step S32: Yes), the third determination unit 276 supplies the total amount constant notification to the notification unit 280.

  The notification unit 280 that has acquired the total amount constant notification from the third determination unit 276 is associated with the notification presence / absence pattern 3 and is stored in the notification information storage unit 282 in the text-format freeze notification information “video being frozen” and the text format The failure location notification information “There is an abnormality in the output image generation unit” is read and output to the monitor 290 (step S34). And step S44 is skipped from step S40 and it progresses to step S50.

  On the other hand, if the third determining unit 276 determines that any one of the one or more difference amounts is not zero (step S32: No), the process proceeds to step S40. Step S <b> 40 is processing for calculating a difference amount based on the output image read from the output buffer 230. Specifically, first, the fourth total amount calculation unit 258 acquires an output image read from the output buffer 230 as an inspection image. The fourth total amount calculation unit 258 calculates the total amount of all pixel values for the output image acquired from the output buffer 230 and supplies the calculated total amount to the fourth difference amount calculation unit 268. Then, the fourth difference amount calculation unit 268 that has acquired the total amount from the fourth total amount calculation unit 258 calculates the difference amount (step S40). The fourth difference amount calculation unit 268 supplies the calculated difference amount to the fourth determination unit 278.

  The fourth determination unit 278 that has acquired the difference amount from the fourth difference amount calculation unit 268 calculates the second difference amount from the two or more total amount amounts calculated from the output image that the fourth total amount calculation unit 258 has continuously acquired two or more predetermined numbers. It is determined whether or not any of the one or more difference amounts calculated by the four difference amount calculation unit 268 is zero (step S42). When the fourth determination unit 278 determines that any of the one or more difference amounts is zero (step S <b> 42: Yes), the fourth determination unit 278 supplies the total amount constant notification to the notification unit 280.

  The notification unit 280 that has acquired the constant total amount notification from the fourth determination unit 278 is associated with the notification presence / absence pattern 4 and is stored in the notification information storage unit 282. The failure location notification information “There is an abnormality in the output buffer” is read and output to the monitor 290 (step S44). On the other hand, if it is determined by the fourth determination unit 278 that any one of the above-described difference amounts is not zero (step S42: No), the process skips step S44 and proceeds to step S50.

  Subsequent to step S14, step S24, step S34, step S42 (No) or step S44, the end determination unit (not shown) stops the output function of the freeze notification information as in step S150 of the flowchart shown in FIG. It is determined whether or not (step S50). If the end determination unit determines that the freeze notification information output function is to be stopped (step S50: Yes), this flowchart ends. On the other hand, when the end determination unit determines that the freeze notification information output function is not stopped (step S50: No), the process returns to step S10.

  When the image determination device 20 includes a plurality of output buffers, the fourth total amount calculation unit 258 continuously uses the output images sequentially stored in any one of the plurality of output buffers as an inspection image. To obtain the total amount. FIG. 6 is a diagram for explaining the operation of the fourth total amount calculation unit 258 when the image determination device 20 shown in FIG. 3 includes two output buffers 231 and 232.

  As illustrated in FIG. 6A, the output image generation unit 220 stores the generated output image in the output buffer 231 and the output buffer 232. Specifically, the output image generation unit 220 stores the output image generated this time in another output buffer different from one output buffer that stores the output image generated last time. For example, as illustrated in FIG. 6B, the output image generation unit 220 stores the first generated output image 1 in the output buffer 231, and stores the second generated output image 2 in the output buffer 232. The operation of storing the third generated output image 3 in the output buffer 231 again is repeated.

  As shown in FIG. 6A, the output control unit 240 reads out the output images stored in the output buffer 231 and the output buffer 232 and outputs them to the monitor 290. At this time, the output control unit 240 outputs the output image twice from the output buffer to which the output image is not written by the output image generation unit 220 at a speed twice as high as that of writing (60 frames / second). The data are successively read and output to the monitor 290.

  For example, when the output image is stored in the output buffer 231 and the output buffer 232 as shown in FIG. 6B by the output image generation unit 220, the output control unit 240 causes the output image generation unit 220 to output the output buffer 232. While the output image 2 is being written, the output image 1 is read twice from the output buffer 231 at a double speed and output to the monitor 290. Next, while the output image generation unit 220 writes the output image 3 in the output buffer 231, the output control unit 240 reads the output image 2 from the output buffer 232 twice at a double speed and outputs it to the monitor 290. . Thereafter, the same operation is repeated.

  As shown in FIG. 6A, the fourth total amount calculation unit 258 continuously acquires output images read from the output buffer 231 as inspection images, and calculates the total amount. For example, when the output image is stored in the output buffer 231 and the output buffer 232 by the output image generation unit 220 as illustrated in FIG. 6B, the fourth total amount calculation unit 258 outputs the output from the output buffer 231. The operation of acquiring image 1, output image 1, output image 3, and output image 3 (hereinafter the same) in order and calculating the total amount of each is repeated.

  In the case shown in FIG. 6A, the fourth determination unit 278 at the latter stage calculates the third sum from three or more sums calculated by the fourth sum total calculation unit 258 from three or more predetermined consecutively acquired output images. It is determined whether each of the two or more difference amounts calculated by the four difference amount calculation unit 268 is zero. In the case shown in FIG. 6A, since the same output image is read twice from the output buffer 231, when the predetermined number is 2, one difference amount calculated by the fourth difference amount calculation unit 268 is This is because it always becomes zero and correct judgment cannot be made.

  As described above, according to the image determination device 20, as with the image determination device 10, it can be determined with a small amount of calculation whether the video output to the monitor 290 is frozen. Accordingly, it is possible to immediately notify the passenger that the video output to the monitor 290 is frozen. Furthermore, since the failure location notification information is output, it is possible to easily identify the failure location that is considered to be the cause of the freeze.

  When the image determination apparatus 20 includes a plurality of output buffers 231 and 232, the fourth total amount calculation unit 258 acquires an output image from any one of the output buffers 231 (or the output buffer 232), and totals the amount. Is calculated. When there are a plurality of output buffers, a plurality of output images that do not change are continuously output from the output buffers 231 and 232 alternately even when the video output to the monitor 290 is substantially frozen. For this reason, if the images output from both the buffers 231 and 232 are set as inspection targets (inspection images), it is erroneously recognized that the output image continues to change. On the other hand, if only the output image of any one of the output buffers is set as the inspection target, it can be surely notified that the video output to the monitor 290 is frozen.

  That is, the output images stored in the output buffers 231 and 232 are different from each other, but are output to the monitor 290 when the output images stored in the output buffers 231 and 232 do not change with time. The video that is actually frozen. Even if the video is substantially frozen when the fourth total amount calculation unit 258 acquires output images from both the output buffers 231 and 232, the fourth determination unit 278 notifies the total amount constant notification. Cannot be supplied to the unit 280. This is because the difference amount between the output image acquired from the output buffer 231 and the output image acquired from the output buffer 232 does not become zero except when the predetermined number is 2. When the predetermined number is 2, as described above, one difference amount calculated by the fourth difference amount calculation unit 268 is always zero, so that it does not make sense. On the other hand, when the fourth sum total calculation unit 258 acquires the output image only from the output buffer 231 and the video is substantially frozen, the fourth determination unit 278 notifies the notification unit 280 of the sum total amount constant. Can be supplied. This is because the difference amount between the output images acquired from the output buffer 231 becomes zero.

  The image determination device 20 includes four types of inspection images (a captured image supplied from the captured image acquisition unit 200, a captured image read from the input buffer 210, an output image supplied from the output image generation device 220, and The output image read from the output buffer 230) is used to determine whether or not the video output to the monitor 290 is frozen. However, it is not necessary to use all the four types of images.

  For example, the image determination apparatus 20 may determine the above using only one of the four types of images. When the image determination apparatus 20 determines the above using only the captured image read from the input buffer 210, the total amount calculation unit 250 includes only the second total amount calculation unit 254, and the difference amount calculation unit 260 includes the second difference. The determination unit 270 includes only the quantity calculation unit 264, and the determination unit 270 includes only the second determination unit 274. When the image determination device 20 determines the above using only the output image supplied from the output image generation device 220, the total amount calculation unit 250 includes only the third total amount calculation unit 256, and the difference amount calculation unit. 260 includes only the third difference amount calculation unit 266, and the determination unit 270 includes only the third determination unit 276.

  When the image determination apparatus 20 determines the above using only the output image read from the output buffer 230, the total amount calculation unit 250 includes only the fourth total amount calculation unit 258, and the difference amount calculation unit 260 includes the first difference amount calculation unit 260. The four difference amount calculation unit 268 is configured only, and the determination unit 270 is configured by only the fourth determination unit 278. Note that when the image determination device 20 determines the above using only the captured image supplied from the captured image acquisition unit 200, the total amount calculation unit 250 only includes the first total amount calculation unit 252, as in the image determination device 10. The difference amount calculation unit 260 includes only the first difference amount calculation unit 262, and the determination unit 270 includes only the first determination unit 272.

  Further, the image determination device 20 determines whether or not the video output to the monitor 290 is frozen using any two of the four types of images or any three types of images. May be. As an example, when the image determination device 20 determines the above using the captured image supplied from the captured image acquisition unit 200 and the output image supplied from the output image generation device 220, the total amount calculation unit 250 uses the first total amount. The difference calculation unit 260 includes a first difference calculation unit 262 and a third difference calculation unit 266, and the determination unit 270 includes a first determination unit 272 and a third difference calculation unit 266. The third determination unit 276 is configured.

  In this case, when the notification unit 280 obtains the total amount constant notification only from the first determination unit 272, the text-format freeze notification information “video is frozen” and the text-format failure location notification information “vehicle camera or captured image acquisition "There is an abnormality in any part" is output to the monitor 290. In addition, when the notification unit 280 obtains the total amount constant notification only from the third determination unit 276, the text-type freeze notification information “video being frozen” and the text-format failure location notification information “input buffer or output video generation unit” Is output to the monitor 290.

  As another example, when the image determination apparatus 20 determines the above using the captured image read from the input buffer 210 and the output image read from the output buffer 230, the total amount calculation unit 250 includes the second total amount calculation unit 254 and The fourth total amount calculation unit 258 includes a second difference amount calculation unit 264 and a fourth difference amount calculation unit 268. The determination unit 270 includes the second determination unit 274 and the fourth determination unit. 278.

  In this case, when the notification unit 280 acquires the total amount constant notification only from the second determination unit 274, the text-format freeze notification information “video is frozen” and the text-format failure location notification information “vehicle camera, captured image acquisition Output to the monitor 290. In addition, when the notification unit 280 obtains the total amount constant notification only from the fourth determination unit 278, the text-type freeze notification information “video being frozen” and the text-format failure location notification information “output video generation unit or output buffer” Is output to the monitor 290.

  In addition, the notification information storage unit 182 of the image determination device 10 stores image format freeze notification information (for example, an image depicting the word “in video freeze”) instead of or in addition to the text-format freeze notification information. May be. Also, the notification information storage unit 182 may store audio format freeze notification information (a voice message issued as “video frozen”) to be output to the speaker 192 instead of or in addition to the text-format freeze notification information. Good. In this case, the notification unit 180 of the image determination apparatus 10 may output the image format freeze notification information to the monitor 190 instead of or in addition to the text format freeze notification information. In addition, the notification unit 180 may output the audio format freeze notification information to the speaker 192 instead of or in addition to the text format freeze notification information.

  Similarly, the notification information storage unit 282 of the image determination device 20 may store freeze notification information and failure location notification information in image format or audio format instead of or in addition to the text format. In this case, the notification unit 280 of the image determination device 20 may output freeze notification information and failure location notification information in image format or audio format to the monitor 290 or the speaker 292 instead of or in addition to the text format. .

  Further, when the total amount calculation unit 152 of the image determination apparatus 10 acquires a colored captured image from the captured image acquisition unit 100, the R component value (hereinafter referred to as “R value”) or B instead of the G value is used. The total amount may be calculated using component values (hereinafter referred to as “B values”). Further, the total amount calculation unit 152 may calculate the total amount using any two or more of the R value, the G value, and the B value. The same applies to the first total amount calculation unit 252, the second total amount calculation unit 254, the third total amount calculation unit 256, and the fourth total amount calculation unit 258 of the image determination device 20.

  Moreover, although the predetermined number may be constant, it may be changed according to the traveling speed of the vehicle. Thereby, when the speed of the vehicle is high, it is possible to notify the passenger that the video is frozen earlier. This is because when the vehicle speed is high, the scenery around the vehicle changes significantly compared to when the vehicle speed is low, and the number of inspection images can be reduced.

  As an example of changing the predetermined number according to the traveling speed of the vehicle, the image determination device 10 further includes a traveling speed acquisition unit (not shown) that acquires the traveling speed of the vehicle. Then, when the vehicle traveling speed acquired from the traveling speed acquisition unit is less than the threshold, the determination unit 172 determines that each of the X−1 difference amounts in the X (integer of 2 or more) inspection images (captured images). It is determined whether or not it is zero, and if it is equal to or greater than the threshold value, whether or not each of Y−1 difference amounts in Y (an integer greater than or equal to 2 and less than X) inspection images (captured images) is zero. Judging. The same applies to the image determination device 20.

  Further, the predetermined number may be changed according to the brightness around the vehicle. As a result, when the periphery of the vehicle is bright, the passenger can be notified more quickly that the video is frozen. When the periphery of the vehicle is bright, the total amount of a plurality of different inspection images (a plurality of inspection images in which one or more pixels among the pixels at the same pixel position are different from each other) is accidentally compared to a dark case. This is because the number of inspection images can be reduced because the probability of matching is further reduced. When the brightness is bright, the probability that the total amount of the plurality of inspection images coincides by chance is even lower because the pixel values of the captured image captured when the periphery of the vehicle is dark are concentrated on a low value. This is because the pixel values of the captured image captured when the periphery of the image is bright are widely distributed from a low value to a high value.

  As an example of changing the predetermined number according to the brightness, the determination unit 172 acquires the total amount from the total amount calculation unit 152, and when the total amount acquired from the total amount calculation unit 152 is less than the threshold, X (X Is an integer greater than or equal to 2) It is determined whether or not each of the X−1 difference amounts in the inspection image (captured image) is zero. It is determined whether or not each of Y−1 difference amounts in the inspection image (captured image) is zero. The same applies to the image determination device 20.

  As another example of changing the predetermined number according to the brightness, the image determination device 10 acquires a brightness around the vehicle from an in-vehicle sensor that detects the brightness around the vehicle (not shown). Is further provided. Then, when the brightness value around the vehicle acquired from the brightness acquisition unit is less than the threshold value, the determination unit 172 includes X−1 X (X is an integer of 2 or more) inspection images (captured images). It is determined whether each difference amount is zero or not. If the difference amount is equal to or greater than the threshold value, each of Y−1 difference amounts in Y (an integer greater than or equal to 2 and less than X) inspection images (captured images) is determined. Determine if it is zero. The same applies to the image determination device 20.

  As another example of changing the predetermined number according to the brightness, the image determination device 10 further includes a time acquisition unit (not shown) that acquires the current time. Then, when the current time acquired from the time acquisition unit is within a predetermined range (for example, evening 18:00 to early morning 6:00), the determination unit 172 includes X inspection images (X is an integer of 2 or more). It is determined whether or not each of the X−1 difference amounts in the (captured image) is zero. It is determined whether or not each of the Y−1 difference amounts at zero is zero. The same applies to the image determination device 20.

  In addition, each of the above-described embodiments is merely an example of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner. In other words, the present invention can be implemented in various forms without departing from the spirit or main features thereof.

It is a figure which shows the structural example of the image judgment apparatus which is the 1st Embodiment of this invention. 3 is a flowchart illustrating an operation example of the image determination apparatus illustrated in FIG. 1. It is a figure which shows the structural example of the image judgment apparatus which is the 2nd Embodiment of this invention. It is a figure which shows an example of the information memorize | stored in the alerting | reporting information storage part shown in FIG. It is a flowchart which shows the operation example of the image judgment apparatus shown in FIG. It is a figure for demonstrating operation | movement of the 4th total amount calculation part in case the image judgment apparatus shown in FIG. 3 is provided with two output buffers.

Explanation of symbols

10, 20 Image determination device 100, 200 Captured image acquisition unit 110, 210 Input buffer 120, 220 Output image generation unit 130, 230 Output buffer 140, 240 Output control unit 152, 250 Total amount calculation unit 162, 260 Difference amount calculation unit 172, 270 Determination unit 180, 280 Notification unit 182, 282 Notification information storage unit 231 Output buffer 232 Output buffer 252 First total amount calculation unit 254 Second total amount calculation unit 256 Third total amount calculation unit 258 Fourth total amount calculation Unit 262 first difference amount calculating unit 264 second difference amount calculating unit 266 third difference amount calculating unit 268 fourth difference amount calculating unit 272 first determining unit 274 second determining unit 276 third determining unit 278 fourth determining unit

Claims (6)

An image obtained continuously from the outside or an image continuously generated based on the image is continuously acquired as an inspection image, and a total amount that is a sum of pixel values related to all pixels of the acquired inspection image A total amount calculation unit for calculating
A difference amount calculation unit that calculates a difference amount of both total amounts calculated by the total amount calculation unit from both inspection images acquired last time and this time;
Whether or not each of the one or more difference amounts calculated by the difference amount calculation unit from two or more total amounts calculated by the total amount calculation unit from two or more predetermined numbers of consecutively acquired inspection images is zero. A determination unit for determining whether or not
A notification unit for notifying that there is an abnormality in the image when the determination unit determines that any of the one or more difference amounts is zero ;
A captured image acquisition unit that continuously acquires captured images from an in-vehicle camera that continuously captures the periphery of the vehicle at a predetermined timing; and
An output image generation unit that continuously generates an output image to be output on an output screen based on the captured image acquired by the captured image acquisition unit ;
The total amount calculation unit continuously acquires the captured image continuously acquired by the captured image acquisition unit as the inspection image and calculates the total amount related to the captured image as a captured image total amount. The output image generated continuously by the output image generation unit is continuously acquired as the inspection image, and the total amount related to the output image is calculated as an output image total amount.
The difference amount calculation unit calculates a captured image difference amount that is the difference amount related to the total amount of both captured images calculated by the total amount calculation unit from both captured images acquired by the captured image acquisition unit last time and this time. And calculating the output image difference amount, which is the difference amount related to the total amount of both output images calculated by the total amount calculation unit from both output images generated by the output image generation unit last time and this time,
The determination unit calculates the difference amount calculation unit from the two or more captured image total amounts calculated by the total amount calculation unit from the captured images acquired continuously by the predetermined number of times by the captured image acquisition unit. It is determined whether or not each of the above-described captured image difference amounts is zero, and two or more calculated by the total amount calculation unit from the output images continuously generated by the output image generation unit by the predetermined number Determining whether each of the one or more output image difference amounts calculated by the difference amount calculation unit from the total output image amount is zero,
When the determination unit determines that any of the one or more captured image difference amounts is zero, or when the determination unit determines that any of the one or more output image difference amounts is zero An image determination device that notifies that there is an abnormality. An image obtained continuously from the outside or an image continuously generated based on the image is continuously acquired as an inspection image, and a total amount that is a sum of pixel values related to all pixels of the acquired inspection image A total amount calculation unit for calculating
A difference amount calculation unit that calculates a difference amount of both total amounts calculated by the total amount calculation unit from both inspection images acquired last time and this time;
Whether or not each of the one or more difference amounts calculated by the difference amount calculation unit from two or more total amounts calculated by the total amount calculation unit from two or more predetermined numbers of consecutively acquired inspection images is zero. A determination unit for determining whether or not
A notification unit for notifying that there is an abnormality in the image when the determination unit determines that any of the one or more difference amounts is zero ;
An input buffer for sequentially storing captured images continuously acquired from an in-vehicle camera that continuously captures the periphery of the vehicle at a predetermined timing;
An output buffer that sequentially stores output images that are continuously generated as images output on an output screen based on the captured images sequentially stored in the input buffer ;
The total amount calculation unit sequentially acquires the captured images sequentially stored in the input buffer as the inspection image, calculates the total amount related to the captured image as a captured image total amount, and outputs the output buffer. The output image stored sequentially is sequentially acquired as the inspection image, and the total amount related to the output image is calculated as an output image total amount.
The difference amount calculation unit calculates a captured image difference amount that is the difference amount related to both the captured image total amounts calculated by the total amount calculation unit from both captured images acquired from the input buffer last time and this time. Calculating an output image difference amount that is the difference amount related to both output image sum amounts calculated by the sum amount calculation unit from both output images obtained from the previous and current output buffers from the output buffer;
The determination unit includes one or more difference amount calculation units calculated by the difference amount calculation unit from two or more of the captured image total amounts calculated by the total amount calculation unit from the captured images continuously acquired from the input buffer by the predetermined number. It is determined whether or not each of the captured image difference amounts is zero, and the two or more output images calculated by the total amount calculation unit from the output images obtained continuously from the output buffer by the predetermined number Determining whether each of the one or more output image difference amounts calculated by the difference amount calculation unit from the total amount is zero;
When the determination unit determines that any of the one or more captured image difference amounts is zero, or when the determination unit determines that any of the one or more output image difference amounts is zero An image determination device that notifies that there is an abnormality. When the determination unit determines that any of the one or more captured image difference amounts is zero, the notification unit notifies that there is an abnormality in either the in-vehicle camera or the input buffer,
The notification unit outputs the output when the determination unit determines that any one of the one or more captured image difference amounts is not zero and any one of the one or more output image difference amounts is zero. The image determination apparatus according to claim 2 , wherein the fact that there is an abnormality in the buffer is notified. An image obtained continuously from the outside or an image continuously generated based on the image is continuously acquired as an inspection image, and a total amount that is a sum of pixel values related to all pixels of the acquired inspection image A total amount calculation unit for calculating
A difference amount calculation unit that calculates a difference amount of both total amounts calculated by the total amount calculation unit from both inspection images acquired last time and this time;
Whether or not each of the one or more difference amounts calculated by the difference amount calculation unit from two or more total amounts calculated by the total amount calculation unit from two or more predetermined numbers of consecutively acquired inspection images is zero. A determination unit for determining whether or not
A notification unit for notifying that there is an abnormality in the image when the determination unit determines that any of the one or more difference amounts is zero ;
A captured image acquisition unit that continuously acquires captured images from an in-vehicle camera that continuously captures the periphery of the vehicle at a predetermined timing; and
An input buffer for sequentially storing the captured images successively acquired by the captured image acquisition unit;
An output image generation unit that continuously generates an output image to be output on an output screen based on the captured image sequentially stored in the input buffer;
An output buffer for sequentially storing the output images continuously generated by the output image generation unit ,
The total amount calculation unit continuously acquires the captured image continuously acquired by the captured image acquisition unit as the inspection image and calculates the total amount related to the captured image as a first captured image total amount. Then, the captured image sequentially stored in the input buffer is continuously acquired as the inspection image, the total amount related to the captured image is calculated as the second captured image total amount, and is continuously generated by the output image generation unit. The output image generated in this manner is continuously acquired as the inspection image, the total amount related to the output image is calculated as the first output image total amount, and the output image sequentially stored in the output buffer is calculated. Continuously acquiring the inspection image and calculating the total amount of the output image as the second output image total amount;
The difference amount calculation unit is a first captured image difference that is the difference amount relating to both first captured image total amounts calculated by the total amount calculation unit from both captured images previously and currently acquired by the captured image acquisition unit. Calculating a quantity, calculating a second captured image difference amount that is the difference amount related to both second captured image total amounts calculated by the total amount calculation unit from both captured images acquired from the previous and current times from the input buffer; A first output image difference amount, which is the difference amount related to both first output image total amounts calculated by the total amount calculation unit from both output images generated by the output image generation unit last time and this time, is calculated, and the output Calculating a second output image difference amount, which is the difference amount related to both the second output image total amounts calculated by the total amount calculation unit from both output images acquired from the buffer last time and this time,
The determination unit calculates the difference amount calculation unit from two or more first captured image total amounts calculated by the total amount calculation unit from the captured images continuously acquired by the captured image acquisition unit by the predetermined number. It is determined whether or not each of the one or more first captured image difference amounts is zero, and the total amount calculation unit calculates 2 from the captured images continuously acquired from the input buffer by the predetermined number. It is determined whether each of the one or more second captured image difference amounts calculated by the difference amount calculation unit from the second captured image total amount is zero, and the output image generation unit performs the predetermined number. Each of the one or more first output image difference amounts calculated by the difference amount calculation unit from two or more first output image sum amounts calculated by the total amount calculation unit from the continuously generated output images. Is zero 1 calculated by the difference amount calculation unit from two or more second output image total amounts calculated by the total amount calculation unit from the output images continuously acquired by the predetermined number from the output buffer. It is determined whether each of the above second output image difference amounts is zero,
The notification unit determines that any of the one or more second captured image difference amounts is zero when the determination unit determines that any of the one or more first captured image difference amounts is zero. When it is determined that any of the one or more first output image difference amounts is zero, or when it is determined that any of the one or more second output image difference amounts is zero, An image determination apparatus that notifies that there is an abnormality. When the determination unit determines that any of the one or more first captured image difference amounts is zero, the notification unit notifies that either the in-vehicle camera or the captured image acquisition unit is abnormal. And
The notification unit determines that any of the one or more first captured image difference amounts is not zero and any of the one or more second captured image difference amounts is zero. , Inform you that there is an abnormality in the input buffer,
The notification unit determines that any of the one or more second captured image difference amounts is not zero and any of the one or more first output image difference amounts is zero. Informing that there is an abnormality in the output image generation unit,
The notification unit determines that any of the one or more first output image difference amounts is not zero and any of the one or more second output image difference amounts is zero. The image determination apparatus according to claim 4 , further reporting that there is an abnormality in the output buffer. An inspection image acquisition step of continuously acquiring images continuously acquired from the outside or images continuously generated based on the above images as inspection images;
A total amount calculation step of calculating a total amount that is a sum of pixel values related to all pixels of the inspection image acquired by the inspection image acquisition step;
A difference amount calculating step for calculating a difference amount between the two total amounts calculated from the previous and current inspection images by the total amount calculating step;
Each of the one or more difference amounts calculated by the difference amount calculation step from two or more sum amounts calculated from the inspection image acquired continuously by a predetermined number of two or more is zero. A determination step for determining whether or not,
A notification step of notifying that there is an abnormality when it is determined that any of the one or more difference amounts is zero by the determination step ;
A captured image acquisition step of continuously acquiring captured images from an in-vehicle camera that continuously captures the periphery of the vehicle at a predetermined timing; and
An output image generation step of continuously generating an output image to be output on an output screen based on the captured image acquired by the captured image acquisition step ;
In the total amount calculation step, the captured image continuously acquired in the captured image acquisition step is continuously acquired as the inspection image, and the total amount related to the captured image is calculated as a captured image total amount. The output image continuously generated in the output image generation step is continuously acquired as the inspection image, and the total amount related to the output image is calculated as the output image total amount.
In the difference amount calculation step, a captured image difference amount, which is the difference amount related to the total amount of both captured images calculated in the total amount calculation step, is calculated from both the captured images acquired in the captured image acquisition step and the previous time. And calculating an output image difference amount that is the difference amount related to the total amount of both output images calculated in the total amount calculation step from the output images generated last time and in the current time in the output image generation step,
In the determination step, 1 calculated in the difference amount calculation step from two or more of the captured image total amounts calculated in the total amount calculation step from the captured images acquired continuously in the predetermined number in the captured image acquisition step. It is determined whether or not each of the above-described captured image difference amounts is zero, and two or more calculated in the total amount calculation step from the output images generated continuously in the output image generation step by the predetermined number Determining whether each of the one or more output image difference amounts calculated in the difference amount calculation step from the total output image amount is zero.
In the notification step, when it is determined in the determination step that all of the one or more captured image difference amounts are zero, or when it is determined that any of the one or more output image difference amounts is zero. An image determination method characterized by notifying that there is an abnormality.