JP2019102929A - Video processing system, video processing apparatus, and video processing method - Google Patents

Abstract

To accurately detect a malfunction generated in a captured video even when the malfunction is difficult to detect.SOLUTION: A video processing system comprises: a plurality of capturing units 1, 2 provided on a mobile body; a first feature amount detection unit 12 for detecting a feature amount from video in a first capturing range in video captured by the first capturing unit 1 of the plurality of capturing units 1, 2; a second feature amount detection unit 14 for detecting a feature amount from video in the first capturing range in video captured by the second capturing unit 2 other than the first capturing unit 1; and feature amount comparison detection units 15, 16 that compare the feature amount in the first capturing range detected by the first feature amount detection unit 12 with the feature amount in the first capturing range detected by the second feature amount detection unit 14 to detect at least any one of a malfunction region included in the video captured by the first capturing unit 1 and a malfunction region included in the video captured by the second capturing unit 2 other than the first capturing unit 1.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a video processing system and a video processing apparatus mounted on a moving body, and a video processing method used in a video processing system mounted on a moving body.

  2. Description of the Related Art In recent years, vehicles equipped with cameras for capturing images of the rear or the side have been increasing in order to quickly confirm the situation around the host vehicle. When the camera is disposed outside the vehicle, dirt such as mud may be attached to the lens of the camera, and the dirt may be reflected in the image to be captured. In addition, even when the camera is placed in a car, the glass surface (for example, the rear window) covering the lens of the camera may be stained or scratched, and the dirt or scratches may appear in the captured image. I can get stuck.

  On the other hand, in an on-vehicle camera device for imaging the rear or side of a vehicle, there is known a technique for automatically detecting foreign matter such as mud when adhering to the lens of the camera or the glass surface covering the camera lens. (See, for example, Patent Document 1).

  In the on-vehicle camera device disclosed in Patent Document 1, a foreign matter attached to the lens surface of the camera is detected by detecting a region where the display mode does not change in the captured image captured by the camera.

JP 2007-38773 A

  By the way, in the technique disclosed in Patent Document 1, in order to increase the detection accuracy of foreign matter, it is necessary to accurately detect an area where the display mode in the captured image does not change. However, if an area in which the display mode does not change is to be detected accurately, white lines on the road continuously reflected in the image are also detected as an area in which the display mode does not change, and the detection accuracy of foreign matter decreases. There was a fear.

  As such, even when a defect that is difficult to detect occurs in a captured image, such as when a minute foreign substance or highly permeable dirt adheres to the lens of the camera, the defect is detected with high accuracy. A video processing system is required.

  An object of the present invention is to provide an image processing system, an image processing apparatus, and an image processing method capable of detecting a defect with high accuracy even if a defect that is difficult to detect occurs in a captured video. I assume.

  In order to achieve the above object, a video processing system according to the present invention comprises a plurality of imaging units installed on a moving object, and at least a first imaging in an image captured by a first imaging unit of the plurality of imaging units. A first feature amount detection unit that detects a feature amount from a video of a range, and a feature amount from a video of at least a first imaging range in a video captured by an imaging unit other than the first imaging unit among the plurality of imaging units Of the first imaging range detected by the first feature amount detection unit, and the features of the first imaging range detected by the second feature amount detection unit At least one of the defect area included in the image captured by the first imaging unit and the defect area included in the image captured by the imaging unit other than the first And a feature amount comparison detection unit to detect.

  Further, in order to achieve the above object, according to the video processing device of the present invention, a video of at least a first imaging range in a video captured by a first imaging unit among a plurality of imaging units installed on a moving object A first feature amount detection unit for detecting a feature amount, and a feature amount is detected from an image of at least a first imaging range in an image captured by an imaging unit other than the first imaging unit among the plurality of imaging units Comparison of the feature quantities of the first imaging range detected by the first feature quantity detection unit with the feature quantities of the first imaging range detected by the second feature quantity detection unit And a feature amount for detecting at least one of the defect area included in the image captured by the first imaging unit and the defect area included in the image captured by the imaging unit other than the first imaging unit And a comparison detection unit.

  Further, in order to achieve the above object, according to the image processing method of the present invention, an image of at least a first imaging range in an image imaged by a first imaging unit of a plurality of imaging units installed on a moving object A first feature amount detection step of detecting a feature amount, and detecting a feature amount from an image of at least a first imaging range in an image captured by an imaging unit other than the first imaging unit among the plurality of imaging units; Comparison of the feature amounts of the first imaging range detected in the first feature amount detection step with the second feature amount detection step and the feature amounts of the first imaging range detected in the second feature amount detection step And a feature amount for detecting at least one of the defect area included in the image captured by the first imaging unit and the defect area included in the image captured by the imaging unit other than the first imaging unit And comparing and detecting.

  According to the present invention, even when a defect that is difficult to detect occurs in a captured image, it is possible to detect the defect with high accuracy.

Block diagram showing an example of the function of each part according to Embodiment 1 of the present invention The figure which shows an example of the installation position of the 1st imaging part which concerns on embodiment of this invention, a 2nd imaging part, and a display part. The figure which shows the image of the synthetic | combination processing which the 1st video synthetic | combination part which concerns on embodiment of this invention performs A flowchart showing an example of the foreign matter detection process according to the first embodiment of the present invention The figure which shows an example of the image of the synthetic | combination image output to a display part when a foreign material is detected by the foreign material detection process which concerns on Embodiment 1 of this invention. The figure which shows another example of the image of the synthetic | combination image output to a display part when a foreign material is detected by the foreign material detection process which concerns on Embodiment 1 of this invention. Block diagram showing an example of the function of each part according to Embodiment 2 of the present invention The figure which shows the image of the synthetic | combination image output to a display part when a foreign material is detected by the foreign material detection process which concerns on Embodiment 2 of this invention. Block diagram showing an example of the function of each part according to Embodiment 3 of the present invention Flowchart showing an example of the foreign matter detection process according to the third embodiment of the present invention

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In addition, the same code | symbol is attached | subjected to the same component. Also, the drawings schematically show the respective components in order to facilitate understanding.

Embodiment 1
FIG. 1 is a block diagram showing an example of the function of each unit according to Embodiment 1 of the present invention. The video processing system according to Embodiment 1 of the present invention includes a first imaging unit 1, a second imaging unit 2, a video processing apparatus 10, and a display unit 4.

  The first imaging unit 1 is an imaging device such as, for example, a charge coupled device (CCD) camera or a complementary metal oxide semiconductor (CMOS) camera installed in a vehicle.

  The first imaging unit 1 is communicably connected to the video processing apparatus 10 by wire or wirelessly. The first imaging unit 1 transmits video data of a captured video to the video processing apparatus 10 in real time.

  The second imaging unit 2 is an imaging device such as, for example, a charge coupled device (CCD) camera or a complementary metal oxide semiconductor (CMOS) camera installed in a vehicle.

  The second imaging unit 2 is communicably connected to the video processing apparatus 10 by wire or wirelessly. The second imaging unit 2 transmits video data of the captured video to the video processing apparatus 10 in real time.

  The display unit 4 is, for example, a display device such as an electronic mirror installed at the front of a ceiling of a vehicle compartment. The display unit 4 is communicably connected to the video processing apparatus 10 by wire or wirelessly.

  FIG. 2 shows an example of installation positions of the first imaging unit 1, the second imaging unit 2 and the display unit 4 according to the embodiment of the present invention.

  The first imaging unit 1 is, for example, a rear camera 5 installed in an inner portion of a rear window of the vehicle, and images the rear of the vehicle. However, the present invention is not limited to this. The first imaging unit 1 includes the right side camera 6 installed near the outside window of the right front door, the left side camera 7 installed near the outside window of the left front door, and the outside of the vehicle It may be a back camera 8 disposed behind the.

  The second imaging unit 2 is any one of the rear camera 5, the right side camera 6, the left side camera 7, and the back camera 8, and is different from the first imaging unit 1.

  As described above, the display unit 4 is, for example, an electronic mirror installed at the front of the ceiling of the cabin, and is provided instead of the rearview mirror of the vehicle. Moreover, the display part 4 may be installed in the instrument panel of a vehicle.

  It returns to the explanation of FIG. The video processing apparatus 10 includes a foreign object erasing unit 10 a, a third video extracting unit 17, a fourth video extracting unit 18, and a first video combining unit 19.

  The foreign material elimination unit 10 a includes a first video extraction unit 11, a second video extraction unit 13, a first feature amount detection unit 12, a second feature amount detection unit 14, and a feature amount comparison unit 15. And a foreign matter detection unit 16.

  The foreign matter erasing unit 10 a receives video data of a video imaged by the first imaging unit 1 and the second imaging unit 2. The foreign matter erasing unit 10a determines whether or not there is a defect in the image captured by the first imaging unit 1 or the second imaging unit 2 based on the received image data, and if there is a defect in the image, Detect where in the image there is a defect.

  In the case where there is a defect in the image captured by the first imaging unit 1 or the second imaging unit 2, dirt adheres to the camera lens of the first imaging unit 1 or the second imaging unit 2 and imaging is performed. This is the case when dirt appears on the image. Alternatively, the foreign matter such as dust adheres to the imaging element of the camera of the first imaging unit 1 or the second imaging unit 2, and the foreign matter is captured in the captured image. Or when installing a camera inside the rear window of vehicles, it is a case where a foreign material adheres to the outer side of a rear window. However, the case where there is a defect in the image captured by the first imaging unit 1 or the second imaging unit 2 is not limited thereto, but includes any defect in the image such as a case where a disturbance occurs in the captured image.

  The first video extraction unit 11 is a video extraction unit that extracts a desired video in order to detect a feature amount from the video captured by the first imaging unit 1. The first video extraction unit 11 performs video extraction in the video cutout unit 11a, the resolution change unit 11b, and the image quality change unit 11c.

  The video cutout unit 11 a is a cutout unit that cuts out a video of a predetermined imaging range from the video captured by the first imaging unit 1. The video cutout unit 11a cuts out a video of a predetermined range from the video captured by the first imaging unit 1 so that the video of the predetermined imaging range which is clipped by the video cutout unit 13a described later matches the video capturing range. The range of the video clipped by the video clipper 11a is predetermined.

  The resolution changing unit 11 b is a video processing unit that changes the resolution of the video clipped by the video clipping unit 11 a to a predetermined resolution. The resolution changing unit 11 b changes the resolution of the video clipped by the video clipping unit 11 a so as to match the resolution of the video changed by the resolution changing unit 13 b described later.

  The image quality change unit 11 c is a video processing unit that changes the image quality of the video whose resolution has been changed by the resolution change unit 11 b. Here, the image quality refers to, for example, the color tone of an image. The image quality change unit 11 c performs, for example, tone curve correction to change the color tone of the image. The image quality change unit 11 c changes the image quality of the image whose resolution has been changed by the resolution change unit 11 b so as to match the image quality of the image whose image quality has been changed by the image quality change unit 13 c described later.

  The first feature amount detection unit 12 is a detection unit that performs edge detection of the image extracted by the first image extraction unit 11 and detects an accumulated edge amount of the detected edge.

  First, the first feature quantity detection unit 12 converts the input image into gray scale. The first feature quantity detection unit 12 performs edge detection of the image converted to gray scale using, for example, a Canny filter.

  Further, the first feature quantity detection unit 12 performs an accumulation process of the edge detected from the video. The edge accumulation processing refers to taking the logical product of a video for which edge detection has been newly performed and a video for which edge detection has been performed in the past each time an edge of the video is detected. As a result, only the overlapping portion of the edge between the image newly subjected to edge detection and the image previously subjected to edge detection is detected as a new edge.

  By performing such edge accumulation processing, the first feature amount detection unit 12 detects the accumulated edge amount of the image captured by the first imaging unit 1. When the above processing is performed on the image captured during movement, the edge of the non-problem area changes, so the accumulated edge amount decreases, and the accumulated time increases and the accumulated edge amount approaches zero. On the other hand, since the edge does not change in the area where there is a defect, the accumulated edge amount shows a constant value.

  The second video extraction unit 13 is a video extraction unit that extracts a desired video in order to detect the feature amount from the video captured by the second imaging unit 2. The second video extraction unit 13 extracts a video in the video cutout unit 13a, the resolution change unit 13b, and the image quality change unit 13c.

  The video cutout unit 13 a is a video cutout unit that cuts out a video in a predetermined imaging range from the video captured by the second imaging unit 2. The video cutout unit 13a cuts out a video of a predetermined range from the video captured by the second imaging unit 2 so that the video of the predetermined video capturing range which is clipped by the video clipping unit 11a described above matches the video capturing range. The range of the video clipped by the video clipper 13a is predetermined.

  The resolution changing unit 13 b is a video processing unit that changes the resolution of the video clipped by the video clipping unit 13 a to a predetermined resolution. The resolution changing unit 13 b changes the resolution of the video clipped by the video clipping unit 13 a so as to match the resolution of the video changed by the resolution changing unit 11 b described above.

  The image quality change unit 13c is a video processing unit that changes the image quality of the video whose resolution has been changed by the resolution change unit 13b. The image quality change unit 13 c performs, for example, tone curve correction to change the color tone of the image. The image quality change unit 13c changes the image quality of the image whose resolution has been changed by the resolution change unit 13b so as to match the image quality of the image whose image quality has been changed by the image quality change unit 11c.

  The second feature amount detection unit 14 detects an edge of the image extracted by the second image extraction unit 13 and detects an accumulated edge amount of the detected edge.

  First, the second feature quantity detection unit 14 converts the input image into gray scale. The second feature quantity detection unit 14 performs edge detection of the image converted into gray scale using, for example, a Canny filter.

  Further, the second feature quantity detection unit 14 performs an accumulation process of the edge detected from the video. By performing the edge accumulation process, the second feature amount detection unit 14 detects the accumulated edge amount of the image captured by the second imaging unit 2.

  The feature amount comparison unit 15 is a comparison unit that compares the accumulated edge amount detected by the first feature amount detection unit 12 with the accumulated edge amount detected by the second feature amount detection unit 14.

  The feature amount comparison unit 15 divides the video input from the first feature amount detection unit 12 into a plurality of areas, and calculates an accumulated edge amount for each of the divided areas. Also, the feature amount comparison unit 15 divides the video input from the second feature amount detection unit 14 into a plurality of areas, and calculates an accumulated edge amount for each of the divided areas.

  An image input from the first feature amount detection unit 12 and the second feature amount detection unit 14 to the feature amount comparison unit 15 is divided into, for example, 12 areas in the same manner. However, the number of divisions of the video is not limited to this, and may be appropriately set according to the processing capacity of the video processing apparatus 10 or the like.

  The same area number is given to the areas at the same positions of the images input from the first feature amount detection unit 12 and the second feature amount detection unit 14 to the feature amount comparison unit 15. Then, the accumulated edge amounts are compared between areas to which the same area number is assigned in the video input from the first feature detection unit 12 and the video input from the second feature detection unit 14.

  When the cumulative edge amount of the area of a certain area number out of the divided areas in the video input from the first feature amount detection unit 12 becomes equal to or more than a predetermined threshold, the feature amount comparison unit 15 detects the area Set as the first foreign matter adhesion candidate. In addition, when the cumulative edge amount of the area of an area number among the divided areas in the video input from the second feature amount detection unit 14 becomes equal to or more than a predetermined threshold, the feature amount comparison unit 15 The area is set as a second foreign matter adhesion candidate.

  The feature amount comparison unit 15 calculates the cumulative edge amount of the area set as the first foreign matter adhesion candidate and the area having the same area number as the area in the video input from the second feature amount detection unit 14 Compare with the accumulated edge amount. Also, the feature amount comparison unit 15 adds the same area number as the area in the cumulative edge amount of the area set as the second foreign matter adhesion candidate and the image input from the first feature amount detection unit 12 Compare with the cumulative edge amount of the area.

  When the difference between the accumulated edge amounts of the compared areas becomes equal to or greater than a predetermined threshold value, the feature amount comparison unit 15 sets the area set as the first foreign matter adhesion candidate or the second foreign matter adhesion candidate as the final foreign matter adhesion candidate Set to The feature amount comparison unit 15 outputs, to the foreign matter detection unit 16, information indicating that the final foreign matter adhesion candidate is set in any area or not set in any area.

  The foreign matter detection unit 16 is a detection unit that detects, based on the information output from the feature amount comparison unit 15, to which area of the areas set as the final foreign matter adhesion candidate the foreign matter adheres.

  The foreign substance detection unit 16 is provided with a final foreign substance adhesion candidate counter that counts the set number of times while being set as the final foreign substance adhesion candidate for each divided area. When the continuity of the event in which an area is set as the final foreign matter adhesion candidate is interrupted, the counter value of the final foreign matter adhesion candidate counter of the area is reset.

  When the count value of the final foreign matter adhesion candidate counter in a certain area reaches a predetermined number N, the foreign matter detection unit 16 sets the area as the foreign matter adhesion area. Further, the foreign matter detection unit 16 outputs the area number set in the foreign matter adhesion area to the first video combining unit 19.

  In addition, although the information which shows the area set to the foreign material adhesion area was demonstrated as an area number here, it is not restricted to this. For example, the information indicating the area may be coordinate values indicating the position of the foreign matter adhesion area.

  The third video extracting unit 17 is a video captured by the first imaging unit 1 in order to combine the video captured by the first imaging unit 1 with the video captured by the second imaging unit 2. To extract a desired image from the image.

  The third video extraction unit 17 cuts out a video showing a predetermined imaging range from the video captured by the first imaging unit 1. In addition, the third video extracting unit 17 changes the resolution of the clipped video and the image quality.

  The fourth video extraction unit 18 is a video captured by the second imaging unit 2 in order to combine the video captured by the second imaging unit 2 with the video captured by the first imaging unit 1. To extract a desired image from the image.

  The fourth video extraction unit 18 cuts out a video of a predetermined imaging range from the video captured by the second imaging unit 2. In addition, the fourth video extracting unit 18 changes the resolution of the clipped video and the image quality.

  The first video combining unit 19 is a combining processing unit that generates a combined video obtained by combining the video extracted by the third video extracting unit 17 and the video extracted by the fourth video extracting unit 18.

  When the foreign matter adhesion area is detected in the foreign matter elimination unit 10a, the first video synthesis unit 19 is extracted by the third video extraction unit 17 based on the detected information (area number) of the foreign matter adhesion area. The composite state of the video and the video extracted by the fourth video extraction unit 18 is changed.

  Here, changing the composite state means that the composite video of the video captured by the first imaging unit 1 and the video captured by the second imaging unit 2 is output to the display unit 4. By changing the display area ratio of the video captured by the first imaging unit 1 to the video captured by the second imaging unit 2 or by changing the α value at the time of combining the two videos by the coefficient (α value) is there. In addition, in order to change the composition state, in a state where only the video captured by the first imaging unit 1 is output to the display unit 4, the second imaging may be performed on the video captured by the first imaging unit 1. The process of combining and outputting the images captured by the imaging unit 2 is included.

  FIG. 3 is a view showing an image of the combining process performed by the first video combining unit 19 according to the embodiment of the present invention.

  The first video synthesis unit 19 may, for example, compare the video 101 captured by the first imaging unit 1 (rear camera 5) and the video 102 captured by the second imaging unit 2 (right side camera 6). Arrange on the left and right.

  The first video combining unit 19 performs alpha blending in the vicinity of the border of these videos to perform video combining. At that time, as shown in FIG. 3, the first video synthesis unit 19 directs the α value of the video 101 captured by the first imaging unit 1 to the video 102 captured by the second imaging unit 2. Decrease according to On the other hand, the α value of the image 102 captured by the second imaging unit 2 is increased toward the image 101 captured by the first imaging unit 1.

  That is, when the first video combining unit 19 combines the video 101 captured by the first imaging unit 1 and the video 102 captured by the second imaging unit 2, the first video combining unit 19 starts from the boundary portion of these videos. The transmittance of the image is changed according to the distance of

  Thereby, the first video combining unit 19 combines the video so that the boundary between the video 101 captured by the first imaging unit 1 and the video 102 captured by the second imaging unit 2 is not noticeable. It can be carried out.

  Similarly, in the process of combining the image 101 captured by the first imaging unit 1 (rear camera 5) and the image 103 captured by the second imaging unit 2 (left side camera 7), By performing alpha blending in the vicinity of a part, it is possible to combine images so that the border part is not noticeable.

  The first video combining unit 19 combines the video extracted by the first video extracting unit 11 and the second video extracting unit 13 in order to reduce distortion of the combined video and the like. A part of the image may be rotated, enlarged, or reduced.

  When the composite video shown in the middle part of FIG. 3 is displayed on the display unit 4, the first video combining unit 19 outputs video data obtained by inverting the right and left sides of the composite video to the display 4.

  Next, the foreign matter detection processing performed by the foreign matter eraser 10a will be described. FIG. 4 is a flowchart showing an example of the foreign matter detection process according to the first embodiment of the present invention. In FIG. 4, steps S10 to S30 show the processing in the first video extraction unit 11 and the first feature amount detection unit 12, but the second video extraction unit 13 and the second feature It is assumed that the same processing is performed in the amount detection unit 14 as well.

  First, the first video extraction unit 11 executes a video extraction process (step S10). The video cutout unit 11 a in the first video extraction unit 11 cuts out a video of a predetermined imaging range from the video captured by the first imaging unit 1 in the video extraction processing.

  Further, the resolution changing unit 11b in the first video extracting unit 11 changes the resolution of the clipped video in the video extracting process.

  Further, the image quality changing unit 11c in the first video extracting unit 11 changes the color tone of the video whose resolution has been changed in the video extracting process.

  Next, the first feature quantity detection unit 12 executes a feature quantity detection process (step S20). In the feature amount detection process, the first feature amount detection unit 12 detects an edge of the image extracted by the first image extraction unit 11, and detects an accumulated edge amount of the detected edge.

  In the feature amount detection process, the first feature amount detection unit 12 first converts the video extracted by the first video extraction unit 11 into gray scale. Next, the first feature quantity detection unit 12 performs edge detection of the image converted into gray scale using a Canny filter. In addition, the first feature quantity detection unit 12 executes accumulation processing of the edge detected by the edge detection.

  Next, the first feature quantity detection unit 12 determines whether the feature quantity detection process (step S20) has been performed M times (step S30). If it is determined that the feature amount detection process has not been performed M times (No in step S30), the first video extraction unit 11 performs the video extraction process (step S10) again.

  When it is determined that the feature amount detection process has been performed M times (in the case of Yes in step S30), the feature amount comparison unit 15 executes the foreign substance adhesion candidate determination process (step S40). At this time, the number of times of execution of the feature amount detection process is reset.

  The feature amount comparison unit 15 divides the video input from the first feature amount detection unit 12 and the second feature amount detection unit 14 into a plurality of areas in the foreign substance adhesion candidate determination process. The feature amount comparison unit 15 calculates the accumulated edge amount accumulated for each of the divided areas. If the calculated accumulated edge amount is equal to or larger than a predetermined threshold value, the feature amount comparison unit 15 sets the area as a foreign matter adhesion candidate.

  Next, the feature amount comparison unit 15 determines whether or not there is an area to which a foreign matter adhesion candidate is set among the divided areas (step S50).

  If no area is set as a foreign substance adhesion candidate in the foreign substance adhesion candidate determination processing (No in step S50), the first video extraction unit 11 executes the video extraction processing (step S10) again.

  If any area is set as a foreign substance adhesion candidate in the foreign substance adhesion candidate judgment processing (step S40) (if Yes in step S50), the feature amount comparison unit 15 performs final foreign substance adhesion candidate judgment processing (step S60). Run.

  When the feature amount comparison unit 15 executes the final foreign matter adhesion candidate determination processing, the cumulative edge amount of the area set as the foreign matter adhesion candidate in the image input from the first feature amount detection unit 12 and the second feature The cumulative edge amount of the area to which the same area number as the area in the image input from the amount detection unit 14 is added is compared. Also, the feature amount comparison unit 15 calculates the cumulative edge amount of the area set as the foreign matter adhesion candidate in the image input from the second feature amount detection unit 14 and the image input from the first feature amount detection unit 12 And the accumulated edge amount of the area to which the same area number is attached.

  If the difference between the accumulated edge amounts in the compared areas is equal to or greater than a predetermined threshold value, the feature amount comparison unit 15 sets the area set as the foreign matter adhesion candidate as the final foreign matter adhesion candidate.

  Next, the feature amount comparison unit 15 determines whether or not there is an area to which the final foreign matter adhesion candidate is set among the divided areas (step S70).

  In the final foreign matter adhesion candidate determination process, when any area is not set as the final foreign matter adhesion candidate (No in step S70), the first video extraction unit 11 executes the video extraction process (step S10) again. Do. At this time, the counter value of the final foreign matter adhesion candidate counter is reset.

  If any area is set as the final foreign matter adhesion candidate in the final foreign matter adhesion candidate determination processing (in the case of Yes in step S70), the foreign matter detection unit 16 determines that the counter value of the final foreign matter adhesion candidate counter of the area is predetermined. It is determined whether the number of times N has been reached (step S80).

  If there is no area in which the counter value of the final foreign matter adhesion candidate counter reaches N (in the case of No in step S80), the first video extracting unit 11 executes the video extracting process (step S10) again.

  On the other hand, if there is an area where the final foreign matter adhesion candidate counter value reaches N (if Yes at step S80), the foreign matter detection unit 16 sets the area as the foreign matter adhesion area (step S90). End the detection process.

  As described above, when the continuity of the event that an area is set as the final foreign matter adhesion candidate is interrupted, the counter value of the final foreign matter adhesion candidate counter of the area is reset.

  Further, the foreign matter detection processing need not be performed for all the frames of the video input to the foreign matter elimination unit 10a, and may be performed at an appropriate frame interval according to the processing capability of the video processing device 10. The values of the constant M and the constant N may be appropriately set according to the detection accuracy of the foreign object to be obtained.

  Next, when the foreign matter adhesion area is set by the foreign matter detection process, the video combining process performed in the first video combining unit 19 will be described. FIG. 5 is a view showing an example of a composite video image output to the display unit 4 when a foreign matter is detected by the foreign matter detection processing according to the first embodiment of the present invention.

  In the upper left of FIG. 5, the image 101 captured by the first imaging unit 1 when the image 101 captured by the first imaging unit 1 (rear camera 5) is displayed on the display unit 4. Is a diagram of a display image showing the foreign object 110 captured in the lower left portion of the image.

  The drawing on the upper right of FIG. 5 shows that the lower left portion of the image 101 captured by the first imaging unit is replaced with the image 103 captured by the second imaging unit 2 (left side camera 7). It is a figure of a display image.

  In the lower left of FIG. 5, the image 101 captured by the first imaging unit 1 when the image 101 captured by the first imaging unit 1 (the rear camera 5) is displayed on the display unit 4. Is a diagram of a display image showing the foreign object 110 captured in the lower right portion of the image.

  In the lower right part of FIG. 5, the lower right portion of the image 101 captured by the first imaging unit 1 is replaced with the image 102 captured by the second imaging unit 2 (right side camera 6). It is a figure of the display image which shows.

  When the lower left portion of the image 101 captured by the first imaging unit 1 (rear camera 5) is set as the foreign matter adhesion area in the foreign matter elimination unit 10a, the first video combining unit 19 selects the image of the area The image 103 is replaced with the image 103 captured by the second imaging unit 2 (left side camera 7).

  When the lower right portion of the image 101 captured by the first imaging unit 1 (rear camera 5) is set to the foreign matter adhesion area in the foreign matter elimination unit 10a, the first video combining unit 19 determines The video is replaced with the video 102 captured by the second imaging unit 2 (right side camera 6).

  Thereby, the image of the area in which the field of view is blocked by the foreign matter 110 attached to the first imaging unit 1 is replaced with the images 102 and 103 imaged by the second imaging unit 2 and displayed on the display unit 4 it can.

  FIG. 6 is a view showing another example of the image of the composite image output to the display unit 4 when the foreign matter is detected by the foreign matter detection processing according to the first embodiment of the present invention.

  In the upper left of FIG. 6, a composite image of the image 101 captured by the first imaging unit 1 (rear camera 5) and the image 102 captured by the second imaging unit 2 (right side camera 6) is displayed. It is a figure of a display picture showing signs that a foreign material 110 is reflected in the lower left part of picture 102 picturized by the 2nd image pick-up part 2 (right side camera 6), when displaying on part 4.

  The left side of the image 102 captured by the second imaging unit 2 (right side camera 6) is the image 101 captured by the first imaging unit 1 (rear camera 5). It is a figure of the display image which shows a mode that it was substituted.

  The lower left side of FIG. 6 displays a composite image of the image 101 captured by the first imaging unit 1 (rear camera 5) and the image 103 captured by the second imaging unit 2 (left side camera 7). FIG. 10 is a view of a display image showing a state in which a foreign object 110 is reflected in the lower right portion of the image 103 captured by the second imaging unit 2 when displayed on the unit 4;

  In the lower right part of FIG. 6, the right part of the image 103 captured by the second imaging unit 2 (left side camera 7) is the image 101 captured by the first imaging unit 1 (rear camera 5). It is a figure of the display image which shows a mode that it was substituted.

  When the lower left portion of the image 102 captured by the second imaging unit 2 (right side camera 6) is set as the foreign matter adhesion area, the first image combining unit 19 changes the area ratio of the combined image (boundary portion To move the video of the area to the video 101 captured by the first imaging unit 1 (rear camera 5).

  In addition, when the lower right portion of the image 103 captured by the second imaging unit 2 (left side camera 7) is set as the foreign matter adhesion area, the first image combining unit 19 changes the area ratio of the combined image Thus, the video of the area is replaced with the video 101 captured by the first imaging unit 1 (rear camera 5).

  As a result, it is possible to replace the image of the area in which the field of view is blocked by the foreign matter attached to the second imaging unit 2 with the image 101 imaged by the first imaging unit 1 and to display the image on the display unit 4.

  In the first embodiment of the present invention, the feature amount comparison unit 15 and the foreign matter detection unit 16 are separately provided, but these are the same as the feature amount comparison unit 15 and the foreign matter detection unit 16 as one comparison detection unit. May have the function of

Second Embodiment
Next, a second embodiment of the present invention will be described. FIG. 7 is a block diagram showing an example of the function of each unit according to Embodiment 2 of the present invention.

  The video processing system according to Embodiment 2 of the present invention includes a first imaging unit 1, a second imaging unit 2, a third imaging unit 21, a fourth imaging unit 22, and an image processing apparatus 20. And the display unit 4.

  The video processing device 20 includes a foreign object erasing unit 20a, a third video extracting unit 17, a fourth video extracting unit 18, a seventh video extracting unit 33, an eighth video extracting unit 34, and a first. And a video synthesis unit 19 of The seventh video extraction unit 33 and the eighth video extraction unit 34 are video extraction units having the same functions as the third video extraction unit 17 and the fourth video extraction unit 18.

  The foreign matter removing unit 20 a includes a first video extracting unit 11, a second video extracting unit 13, a fifth video extracting unit 25, and a sixth video extracting unit 27. Also, the foreign matter elimination unit 20a includes the first feature amount detection unit 12, the second feature amount detection unit 14, the third feature amount detection unit 26, the fourth feature amount detection unit 28, and the feature amounts. A comparison unit 15 and a foreign matter detection unit 16 are provided.

  The fifth video extraction unit 25 and the sixth video extraction unit 27 are video extraction units having the same functions as the first video extraction unit 11 and the second video extraction unit 13. The third feature amount detection unit 26 and the fourth feature amount detection unit 28 are feature amount detection units having the same functions as the first feature amount detection unit 12 and the second feature amount detection unit 14.

  That is, in the video processing system according to the second embodiment of the present invention, in addition to the configuration of the video processing system according to the first embodiment, the third imaging unit 21 and the fourth imaging unit 22 and the corresponding video extraction The units 25 and 27 and the feature amount detection units 26 and 28 are added.

  Here, the first imaging unit 1 is, for example, the rear camera 5. The second imaging unit 2 is, for example, the right side camera 6. The third imaging unit 21 is, for example, the left side camera 7. The fourth imaging unit 22 is, for example, a back camera 8.

  In the second embodiment of the present invention, as in the first embodiment of the present invention, the foreign matter adhesion area is detected by the foreign matter detection processing shown in FIG. However, the cumulative edge amount in the final foreign matter adhesion candidate determination process (step S60) is the same as the area number in the area set as the foreign matter adhesion candidate and the image input from the plurality of other feature quantity detection units Are compared with other areas (corresponding areas).

  Then, when the difference between the accumulated edge amount of the area set as the foreign matter adhesion candidate and the amount accumulated edge amount of the corresponding area in the video input from the plurality of other feature amount detection units is equal to or more than a predetermined threshold, The area set as the foreign matter adhesion candidate is set as the final foreign matter adhesion candidate.

  Then, when the foreign matter adhesion area is set in the foreign matter adhesion area setting processing (step S90), the first video combining unit 19 selects the third video extraction unit 17 based on the information of the detected foreign matter adhesion area, The combining processing of the video extracted by the fourth video extracting unit 18, the seventh video extracting unit 33, and the eighth video extracting unit 34 is performed.

  Next, an image output by the combining process performed by the first image combining unit 19 according to the second embodiment of the present invention will be described. FIG. 8 is a view showing an image of a composite image output to the display unit 4 when foreign matter is detected by the foreign matter detection processing according to the second embodiment of the present invention.

  FIG. 8 illustrates the feature amount comparison unit 15 dividing the input video into 12 areas of 3 rows and 4 columns.

  On the left side of FIG. 8, when the image 101 captured by the first imaging unit 1 (rear camera 5) is displayed on the display unit 4, the first row, the first column to the third row in the image 101 are displayed. It is a figure of the display image which shows a mode that the foreign material 110 was reflected in the area located in a row | line | column, 2nd row 1st column and 3rd row 3rd column and 4th column.

  The right side of FIG. 8 shows that the area of the first row and the first column and the second column and the second row and the first column in the image 101 captured by the first imaging unit 1 is the third imaging unit 21 (left It is a figure of a display picture showing signs that it was replaced by picture 103 picturized by side camera 7). In the right side of FIG. 8, the area of the first row and the third column and the third row, the third column, and the fourth column in the image 101 captured by the first imaging unit is the second imaging unit 2 ( It is a figure of a display picture showing signs that it was replaced by picture 102 picturized by right side camera 6).

  The first image combining unit 19 is configured to use the third imaging unit 21 as an area located in the first row, the first column, the second column, and the second row, the first column in the image 101 captured by the first imaging unit. It is replaced with the image 103 captured by (left side camera 7). In addition, the first video combining unit 19 is configured to set the area located in the first row, the third column and the third row, the third column, and the fourth column in the image 101 captured by the first imaging unit The image 102 is replaced with the image 102 captured by the imaging unit 2 (right side camera 6) of

  As a result, the area where the field of view is blocked by the foreign matter attached to the first imaging unit 1 is replaced with the images 102 and 103 imaged by the second imaging unit 2 and the third imaging unit 21 and displayed on the display unit 4. It can be displayed.

  Although the video captured by the first imaging unit 1 is not replaced with the video captured by the fourth imaging unit 22 here, the video 101 captured by the first imaging unit 1 is converted to the fourth You may replace with the image | video imaged by the imaging part 22. FIG.

Third Embodiment
Next, a third embodiment of the present invention will be described. FIG. 9 is a block diagram showing an example of the function of each unit according to Embodiment 3 of the present invention.

  In the third embodiment of the present invention, a second video combining unit 45 and a fifth feature detection unit are substituted for the second feature detection unit 14 and the third feature detection unit 26 in the second embodiment. It has 46 and. However, in order to make it easier to understand the description according to the third embodiment, in FIG. 9, the fourth imaging unit 22, the sixth video extraction unit 27, the eighth video extraction unit 34, and the fourth feature detection unit Illustration of 28 is omitted and the explanation about these is omitted.

  The video system according to Embodiment 3 of the present invention includes a first imaging unit 1, a second imaging unit 2, a third imaging unit 21, an image processing device 40, and a display unit 4. .

  The first imaging unit 1 is a rear camera 5, the second imaging unit 2 is a right side camera 6, and the third imaging unit 21 is a left side camera 7.

  The video processing device 40 includes a foreign matter erasing unit 40a, a third video extracting unit 17, a fourth video extracting unit 18, a seventh video extracting unit 33, and a first video combining unit 19. .

  The foreign matter eraser 40 a includes a first video extractor 11, a second video extractor 13, a fifth video extractor 25, a first feature detector 12, and a second video synthesizer 45. And a fifth feature amount detection unit 46, a feature amount comparison unit 15, and a foreign matter detection unit 16.

  Here, the second video combining unit 45 and the fifth feature detection unit 46, which are characteristic parts of the video system according to the third embodiment, will be described.

  The second video combining unit 45 is a combining processing unit that combines the video extracted by the second video extracting unit 13 and the video extracted by the fifth video extracting unit 25. The second video combining unit 45 performs alpha blending of the video in the vicinity of the border of the composite video as in the combining processing performed in the first video combining unit 19 so that the border between the two videos is not noticeable. Perform video composition processing.

  In addition, when the second video combining unit 45 performs combining processing on the video extracted by the second video extracting unit 13 and the fifth video extracting unit 25, the second video combining unit 45 reduces the distortion of the combined video, etc. Performs processing such as rotation, enlargement, and reduction on a part of the image.

  The fifth feature detection unit 46 performs edge detection of the composite image synthesized by the second video synthesis unit 45 by the same processing as the first feature detection unit 12, and detects the accumulated edge of the detected edge. Detect the quantity.

  Next, foreign object detection processing executed by the video processing system according to the third embodiment of the present invention will be described.

  FIG. 10 is a flowchart showing an example of the foreign matter detection process according to the third embodiment of the present invention. Although the process performed by the first video extracting unit 11 and the first feature amount detecting unit 12 is omitted in FIG. 10, it is assumed that the process described in the first embodiment is performed. Further, although the process performed by the fifth video extracting unit 25 is omitted in FIG. 10, the fifth video extracting unit 25 performs the same process as the second video extracting unit 13. Do.

  First, the second video extraction unit 13 executes a video extraction process (step S10). The video cutout unit 13a in the second video extraction unit 13 cuts out a video of a predetermined imaging range from the captured video captured by the second imaging unit 2 in video extraction processing.

  Further, the resolution changing unit 13b in the second video extracting unit 13 changes the resolution of the clipped video in the video extracting process.

  Further, the image quality changing unit 13c in the second video extracting unit 13 changes the color tone of the video whose resolution has been changed in the video extracting process.

  Next, the second video combining unit 45 combines the video extracted by the second video extracting unit 13 and the video extracted by the fifth video extracting unit 25 (step S15).

  Next, the fifth feature quantity detection unit 46 executes a feature quantity detection process (step S20). In the feature amount detection process (step S20), the edge detection of the video synthesized by the second video synthesis unit 45 is performed, and the accumulated edge amount of the detected edge is detected.

  Next, the fifth feature quantity detection unit 46 determines whether the feature quantity detection process (step S20) has been performed M times (step S30). If it is determined that the feature amount detection process has not been executed M times (No in step S30), the second video extraction unit 13 executes the video extraction process (step S10) again.

  When it is determined that the feature amount detection process has been performed M times (in the case of Yes in step S30), the feature amount comparison unit 15 executes the foreign substance adhesion candidate determination process (step S40). At this time, the count number of execution times of the feature amount detection process is reset.

  The feature amount comparison unit 15 divides the video input from the first feature amount detection unit 12 and the fifth feature amount detection unit 46 into a plurality of areas in the foreign substance adhesion candidate determination process. The feature amount comparison unit 15 calculates the accumulated edge amount accumulated for each of the divided areas. If the calculated accumulated edge amount is equal to or larger than a predetermined threshold value, the feature amount comparison unit 15 sets the area as a foreign matter adhesion candidate.

  Next, the feature amount comparison unit 15 determines whether or not there is an area to which a foreign matter adhesion candidate is set among the divided areas (step S50).

  If any area is not set as a foreign matter adhesion candidate in the foreign matter adhesion candidate determination process (No in step S50), the second video extraction unit 13 executes the video extraction process (step S10) again.

  If any area is set as a foreign substance adhesion candidate in the foreign substance adhesion candidate judgment processing (step S40) (if Yes in step S50), the feature amount comparison unit 15 performs final foreign substance adhesion candidate judgment processing (step S60). Run.

  The feature amount comparison unit 15 determines, in the final foreign matter adhesion candidate determination process, the cumulative edge amount of the area set as the foreign matter adhesion candidate in the video input from the first feature amount detection unit 12 and the fifth feature amount detection unit The cumulative edge amount of the area with the same area number as the area in the video input from 46 is compared. Also, the feature amount comparison unit 15 calculates the cumulative edge amount of the area set as the foreign matter adhesion candidate in the image input from the fifth feature amount detection unit 46 and the image input from the first feature amount detection unit 12 And the accumulated edge amount of the area to which the same area number is attached.

  If the difference between the accumulated edge amounts in the compared areas is equal to or greater than a predetermined threshold value, the feature amount comparison unit 15 sets the area set as the foreign matter adhesion candidate as the final foreign matter adhesion candidate.

  Next, the feature amount comparison unit 15 determines whether or not there is an area to which the final foreign matter adhesion candidate is set among the divided areas (step S70).

  If any area is not set as the final foreign matter adhesion candidate in the final foreign matter adhesion candidate determination process (No in step S70), the second video extraction unit 13 executes the video extraction process (step S10) again. Do. At this time, the counter value of the final foreign matter adhesion candidate counter is reset.

  If any area is set as the final foreign matter adhesion candidate in the final foreign matter adhesion candidate determination processing (in the case of Yes in step S70), the foreign matter detection unit 16 determines that the counter value of the final foreign matter adhesion candidate counter of the area is predetermined. It is determined whether the number of times N has been reached (step S80).

  If there is no area in which the counter value of the final foreign matter adhesion candidate counter reaches N (in the case of No in step S80), the first video extracting unit 11 executes the video extracting process (step S10) again.

  On the other hand, when there is an area in which the value of the final foreign matter adhesion candidate counter reaches N (in the case of Yes in step S80), the foreign matter detection unit 16 sets the area as the foreign matter adhesion area and ends the foreign matter detection processing. .

  When the continuity of the event in which an area is set as the final foreign matter adhesion candidate is interrupted, the counter value of the final foreign matter adhesion candidate counter of the area is reset.

  Further, the foreign matter detection processing need not be performed for all the frames of the video input to the foreign matter elimination unit 40a, and may be performed at an appropriate frame interval according to the processing capability of the video processing device 40. The values of the constant M and the constant N may be appropriately set according to the detection accuracy of the foreign object to be obtained.

  In the third embodiment of the present invention, the second video combining unit 45 performs video combining processing before the feature amount comparison unit 15 executes foreign object deposition candidate determination processing and final foreign particle deposition candidate determination processing.

  That is, a comparison is made between the composite image obtained by subjecting the video input from the right side camera 6 and the video input from the left side camera 7 to the video input from the rear camera 5. In this case, the feature amount comparing unit 15 separates the image input from the rear camera 5, the image input from the right side camera 6, the image input from the rear camera 5, and the image input from the left side camera 7 separately. The processing load of the video processing device 40 can be reduced compared to the case of FIG.

  Although the embodiment in which two, three, or four imaging units are provided is shown in the embodiment of the present invention, the number of imaging units is not limited to this, and another imaging unit may be provided.

  Further, in the embodiment of the present invention, as the case where there is a defect in the image captured by the imaging unit, the case where the dirt adheres to the lens of the camera of the imaging unit is exemplified, but the present invention is not limited thereto. For example, the case where the image becomes unclear due to the disturbance of the frequency of the image is included.

  In addition, when there is a defect in the image captured by the imaging unit, the user may be notified of that. For example, when an image on which dirt is attached to the lens of the rear camera 5 and a portion where the dirt is attached is displayed is replaced with an image captured by the right side camera 6, this may be displayed on the display unit 4.

  Further, foreign matter detected by the foreign matter detection processing may be automatically cleaned using air, washer fluid or the like.

  INDUSTRIAL APPLICABILITY The present invention is widely applicable to video processing of video captured by an imaging unit mounted on a mobile object.

DESCRIPTION OF SYMBOLS 1 first imaging unit 2 second imaging unit 4 display unit 5 rear camera 6 right side camera 7 left side camera 8 back camera 10 video processing device 10 a foreign object erasing unit 11 first video extracting unit 11 a video extracting unit 11 b resolution Change unit 11c Image quality change unit 12 First feature amount detection unit 13 Second image extraction unit 13a Video clipping unit 13b Resolution change unit 13c Image quality change unit 14 Second feature amount detection unit 15 Feature amount comparison unit 16 Foreign matter detection unit 17 Third Video Extraction Unit 18 Fourth Video Extraction Unit 19 First Video Synthesis Unit 20 Video Processing Device 20a Foreign Material Elimination Unit 21 Third Imaging Unit 22 Fourth Imaging Unit 25 Fifth Video Extraction Unit 25a Video Cropping unit 25b Resolution changing unit 25c Image quality changing unit 26 Third feature amount detection unit 27 Sixth video extraction unit 28 Fourth feature amount detection unit 33 Seventh Image extraction unit 34 Eighth image extraction unit 40 Image processing device 40a Foreign material elimination unit 45 Second image combining unit 46 Fifth feature amount detection unit 101 Image captured by first imaging unit 102 Second imaging unit 103 Images captured by the second imaging unit

Claims (14)

A plurality of imaging units installed on the mobile object;
A first feature amount detection unit that detects a feature amount from an image of at least a first imaging range in an image captured by a first imaging unit of the plurality of imaging units;
A second feature amount detection unit that detects the feature amount from an image of at least the first imaging range in an image captured by an imaging unit other than the first imaging unit among the plurality of imaging units;
The feature amount of the first imaging range detected by the first feature amount detection unit is compared with the feature amount of the first imaging range detected by the second feature amount detection unit, Feature amount comparison detection for detecting at least one of a defect area included in a video captured by a first imaging unit and a defect area included in a video captured by an imaging unit other than the first imaging unit Department,
A video processing system comprising:   The image processing system according to claim 1, wherein the imaging unit other than the first imaging unit is a second imaging unit. A first video extraction unit that extracts a first video from the video captured by the first imaging unit;
And a second video extraction unit that extracts a second video from the video captured by the second imaging unit.
The first video extraction unit and the second video extraction unit may be configured to match the image quality of the first video and the second video or the detection range of the feature amount. The image processing system according to claim 2, wherein the second image is extracted. And a first video combining unit that combines the video captured by the first imaging unit and the video captured by the second imaging unit.
The first image combining unit is included in the image captured by the first imaging unit when the defect area is detected in the image captured by the first imaging unit by the feature amount comparison detection unit. The first imaging unit to capture the defective area included in the image captured by the second imaging unit so as to replace the defective area to be replaced with the image captured by the second imaging unit 4. The method according to claim 2, wherein a combination state of the video captured by the first imaging unit and the video captured by the second imaging unit is changed so as to replace the video with the selected video. Video processing system. The feature amount comparison and detection unit outputs information indicating coordinate values of the defect area included in the image captured by the first imaging unit,
The first video synthesis unit may replace the defect area included in the video captured by the first imaging unit with the video captured by the second imaging unit using the information indicating the coordinate value. 5. The composite image according to any one of claims 2 to 4, wherein the composite video is generated by combining the video captured by the first imaging unit and the video captured by the second imaging unit. Video processing system. The feature amount comparison and detection unit may include information indicating coordinate values of the defect area included in the image captured by the first imaging unit or the coordinates of the defect area included in the image captured by the second imaging unit. Output information indicating the value,
The first video synthesis unit uses the information indicating the coordinate value to replace the defective area, the video captured by the first imaging unit and the video captured by the second imaging unit The image processing system according to any one of claims 2 to 4, wherein the display area ratio of is changed.   The first video combining unit is configured to combine the video captured by the first imaging unit with the video captured by the second imaging unit so as to replace the defect area. The video captured by the imaging unit and the video captured by the second imaging unit are divided into a plurality of areas, and the video captured by the first imaging unit and the image are divided for each of the divided areas. The video processing system according to any one of claims 4 to 6, wherein the video is combined with the video captured by the second imaging unit.   The first image combining unit captures an image by the first imaging unit when generating a composite image that combines the image captured by the first imaging unit and the image captured by the second imaging unit. Of the image captured by the first imaging unit and the image captured by the second imaging unit according to the distance from the boundary between the captured image and the image captured by the second imaging unit The image processing system according to any one of claims 4 to 7, wherein the rate is changed. The imaging units other than the first imaging unit are a second imaging unit and a third imaging unit,
And a first video combining unit that combines the video captured by the first imaging unit, the video captured by the second imaging unit, and the video captured by the third imaging unit.
The first video synthesis unit
When the defect area is detected in the image captured by the first imaging unit by the feature amount comparison detection unit, the defect area included in the image captured by the first imaging unit is the second defect area The image captured by the first imaging unit, the image captured by the second imaging unit, and the image captured by the third imaging unit so as to replace the image captured by the imaging unit or the third imaging unit Change the composite state with the selected video,
When the defect area is detected in the image captured by the second imaging unit by the feature amount comparison detection unit, the defect area included in the image captured by the second imaging unit is the first defect area The image captured by the first imaging unit, the image captured by the second imaging unit, and the image captured by the third imaging unit so as to replace the image captured by the imaging unit or the third imaging unit Change the composite state with the selected video,
When the defect area is detected in the image captured by the third imaging unit by the feature amount comparison detection unit, the defect area included in the image captured by the third imaging unit is the first defect area The image captured by the first imaging unit, the image captured by the second imaging unit, and the image captured by the third imaging unit so as to replace the image captured by the imaging unit or the second imaging unit The image processing system according to claim 1, wherein the combination state with the selected image is changed.   10. The image processing according to claim 9, wherein the first imaging unit is a rear camera, the second imaging unit is a right side camera, and the third imaging unit is a left side camera. system. The second feature amount detection unit is configured to generate at least the image obtained by combining the images captured by the second and third imaging units other than the first imaging unit among the plurality of imaging units. Detecting the feature amount from the image of the first imaging range;
The feature amount comparison and detection unit detects at least one of the defect area included in the image captured by the first imaging unit and the defect area included in the combined image. The image processing system according to claim 1.   The image processing according to claim 11, wherein the first imaging unit is a rear camera, the second imaging unit is a right side camera, and the third imaging unit is a left side camera. system. A first feature amount detection unit that detects a feature amount from an image of at least a first imaging range in an image captured by a first imaging unit of the plurality of imaging units installed on the mobile object;
A second feature amount detection unit that detects the feature amount from an image of at least the first imaging range in an image captured by an imaging unit other than the first imaging unit among the plurality of imaging units;
The feature amount of the first imaging range detected by the first feature amount detection unit is compared with the feature amount of the first imaging range detected by the second feature amount detection unit, Feature amount comparison detection for detecting at least one of a defect area included in a video captured by a first imaging unit and a defect area included in a video captured by an imaging unit other than the first imaging unit Department,
An image processing apparatus comprising: A first feature amount detection step of detecting a feature amount from an image of at least a first imaging range in an image captured by a first imaging unit of the plurality of imaging units installed on the mobile object;
A second feature amount detection step of detecting the feature amount from an image of at least the first imaging range in an image captured by an imaging unit other than the first imaging unit among the plurality of imaging units;
The feature amount of the first imaging range detected in the first feature amount detection step is compared with the feature amount of the first imaging range detected in the second feature amount detection step, Feature amount comparison detection for detecting at least one of a defect area included in a video captured by a first imaging unit and a defect area included in a video captured by an imaging unit other than the first imaging unit Step and
A video processing method comprising:

Images