JP2015215645A - Analyzer, analysis method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide novel means for detecting contamination adhered to a camera or a moving body to which a camera is installed.SOLUTION: An analyzer 10 comprises: a camera 11 installed to a moving body; a determination part 12 for determining whether or not the moving body is moving; a non-movement object detection part 13 for detecting a non-movement object which is shown in a same place in frames, over the plural frames, by analyzing pieces of image data which are created by the camera 11 during movement of the moving body; a contamination detection part 14 for evaluating a visual field state using one or more sizes of one or more non-movement objects detected by the non-movement object detection part 13, and when an evaluation value is in a predetermined range, detecting a state there is contamination; and a notification part 15 for outputting information indicating the state there is contamination, when the contamination detection part 14 detects the state there is contamination.

Description

  The present invention relates to an analysis apparatus, an analysis method, and a program.

  Patent Document 1 discloses an in-vehicle camera device having a function of detecting dirt on a windshield and an optical surface. The invention is based on causing a phenomenon that a photographed image is out of focus when dirt is attached to a windshield or an optical surface.

  The in-vehicle camera device disclosed in Patent Document 1 detects one or more light sources that are sufficiently bright with respect to the brightness of a running environment being shot by the camera, and blurs the light source from the luminance distribution around the light source. The condition is detected, and the dirt is detected depending on the size of the blur. Specifically, the in-vehicle camera device includes a storage unit that stores in advance an ideal image obtained by performing binarization processing on a captured image captured by a capturing unit whose optical surface is not soiled, and a captured image, A processing unit that performs binarization processing based on two predetermined threshold values, compares the binarized image with an ideal image stored in the storage unit, and performs stain detection based on the comparison result; Have.

JP 2014-43121 A

  In the case of the invention disclosed in Patent Document 1, ideal images corresponding to various types and sizes of light sources (e.g., headlights of compatible vehicles, traffic lights, etc.) that may appear in the captured image are stored in advance. It is necessary to keep. However, there are a wide variety of types and sizes of light sources that may appear in the captured image. For this reason, it is difficult to hold an ideal image covering all of them.

  An object of the present invention is to provide a new means for detecting dirt adhering to a camera or a moving body in which the camera is installed.

According to one embodiment of the present invention,
A camera installed on a moving object;
Determining means for determining whether or not the moving object is moving;
Non-moving object detection means for analyzing image data generated by the camera while the moving body is moving and detecting non-moving objects appearing at the same position in a frame over a plurality of frames;
The visual field state is evaluated using any one or more sizes of the one or more non-moving objects detected by the non-moving object detection means, and there is contamination when the evaluation value falls within a predetermined range. Dirt detection means for detecting the state of
When the dirt detection means detects a state with dirt, a notification means for outputting information indicating that,
Is provided.

Also, according to one embodiment of the present invention,
Computer
A determination step of determining whether or not the moving object is moving;
A non-moving target detection step of analyzing image data generated by a camera installed on the moving body while the moving body is moving, and detecting a non-moving target that appears in the same position in the frame over a plurality of frames;
When the visual field state is evaluated using any one or more of the one or more non-moving objects detected in the non-moving object detection step, and the evaluation value falls within a predetermined range, there is contamination. A dirt detection step for detecting the state of
When a state with dirt is detected in the dirt detection step, a notification step for outputting information indicating that,
An analysis method for performing the above is provided.

Also, according to one embodiment of the present invention,
Computer
Determining means for determining whether or not the moving object is moving;
Non-moving object detection means for analyzing image data generated by a camera installed on the moving object while the moving object is moving, and detecting a non-moving object appearing at the same position in the frame over a plurality of frames;
The visual field state is evaluated using any one or more sizes of the one or more non-moving objects detected by the non-moving object detection means, and there is contamination when the evaluation value falls within a predetermined range. Dirt detection means for detecting the state of
When the dirt detection means detects a state with dirt, a notification means for outputting information indicating that,
A program for functioning as a server is provided.

  ADVANTAGE OF THE INVENTION According to this invention, the new means which detects the stain | pollution | contamination adhering to the mobile body in which the camera and the camera were installed is provided.

It is a figure which shows notionally an example of the hardware constitutions of the analyzer of this embodiment. It is a figure for demonstrating the effect of this embodiment. It is a figure which shows an example of the functional block diagram of the analyzer of this embodiment. It is a schematic diagram for demonstrating an example of the process of the non-movement object detection part of this embodiment. It is a flowchart which shows an example of the flow of a process of the analyzer of this embodiment. It is a figure which shows an example of the functional block diagram of the analyzer of this embodiment. It is a figure which shows an example of the functional block diagram of the analyzer of this embodiment. It is a conceptual diagram which shows the example which attached the analyzer of this embodiment to the motor vehicle. It is a figure which shows an example of the functional block diagram of the analyzer of this embodiment.

  First, an example of the hardware configuration of the apparatus according to the present embodiment will be described. Each unit included in the apparatus according to the present embodiment includes a CPU (Central Processing Unit) of an arbitrary computer, a memory, a program loaded in the memory, a storage unit such as a hard disk for storing the program, and a network connection interface. And any combination of software. It will be understood by those skilled in the art that there are various modifications to the implementation method and apparatus. Here, the program includes a program downloaded from a storage medium such as a CD (Compact Disc), a server on the Internet, and the like in addition to a program stored in the memory in advance from the stage of shipping the apparatus.

  FIG. 1 is a diagram conceptually illustrating a hardware configuration example of an analysis apparatus 10 in the present embodiment. The analysis device 10 is a so-called computer, and includes a CPU (Central Processing Unit) 1A, a memory 2A, an input / output interface (I / F) 3A, a communication device 4A, a camera 5A, and the like that are connected to each other via a bus 6A. The memory 2A is a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk, a portable storage medium, or the like.

  The input / output I / F 3A can be connected to a device such as a display device (not shown), an input device (not shown), a vehicle control device (not shown), or the like. The communication device 4A communicates with other devices via a network (not shown). The camera 5A is configured to be able to take a so-called moving image. The camera 5A may be a fixed focus camera. The present embodiment does not limit the hardware configuration of the analysis apparatus 10.

  Hereinafter, this embodiment will be described. Note that the functional block diagram used in the following description of the embodiment shows functional unit blocks rather than hardware unit configurations. In these drawings, each device is described as being realized by one device, but the means for realizing it is not limited to this. That is, it may be a physically separated configuration or a logically separated configuration. In addition, the same code | symbol is attached | subjected to the same component and description is abbreviate | omitted suitably.

<First Embodiment>
First, an outline of the present embodiment will be described. Image data obtained by continuously photographing the outside of a moving body with a camera installed on the moving body such as an automobile has the following characteristics.

“When a plurality of frames are compared, the types of objects (e.g., forward vehicles, pedestrians, etc.) and their modes (e.g., position, size, shape, etc. within the frames) appear different from each other. However, an object that is captured due to dirt attached to the camera or moving object is captured over a plurality of frames, and its mode (eg, position, size, shape, etc. within the frame) hardly changes. . "

  Note that “frame” means each of a plurality of still images constituting a moving image. Hereinafter, the data of each frame is referred to as “frame data”. A group of a plurality of frame data is called “image data”.

  The above features will be described using a specific example. FIG. 2 is a schematic diagram of a front image obtained by photographing the front through the windshield of the automobile with a camera installed in the automobile (moving body). FIG. 2A shows a frame shot at the first timing, and FIG. 2B shows a frame shot at the second timing immediately after that.

  In the frame of FIG. 2A, the forward vehicle 101 running ahead, the white lines 102-1 to 102-4, and the street trees 103-1 and 103-2 are shown. In addition, the non-moving object 200 caused by dirt attached to the windshield is shown at a position covering the back surface of the forward vehicle 101. Note that the non-moving target 200 due to the dirt may be the dirt itself, or may be a blurred part due to the dirt.

  On the other hand, in the frame of FIG. 2B, the forward vehicle 101 running ahead, the white lines 102-2 to 102-4, and the street trees 103-1 and 103-2 are shown. Moreover, the non-moving object 200 resulting from the dirt adhering to the windshield is shown on the back of the forward vehicle 101 and the position on the road.

  Comparing the frames of FIG. 2A and FIG. 2B, it can be seen that, for example, the white line 102-1 is shown in FIG. 2A, but this is not shown in FIG. 2B. Further, the shape of the forward vehicle 101 shown in FIG. 2A (the shape in the captured image) is different from the shape of the forward vehicle 101 reflected in FIG. 2B (the shape in the captured image). I understand that. Further, the positions of the street trees 103-1 and 103-2 shown in FIG. 2A in the frame, and the street trees 103-1 and 103-2 shown in FIG. It can be seen that the positions in the frame are different from each other. Thus, with the movement of the moving body (movement from the first timing to the second timing), the types and / or modes of the majority of objects shown in the frame change.

  However, the non-moving object 200 (the object reflected in the frame due to the dirt adhering to the camera or the moving body) due to the dirt adhering to the windshield is both in FIG. 2 (a) and FIG. 2 (b). The mode (e.g. position, size, shape, etc. in the frame) is almost the same.

  Such a feature is that as the moving body moves, the relative position of the majority of the objects changes with the moving body, and more specifically, the relative positional relation with the camera installed on the moving body changes. The dirt attached to the camera installed on the moving body or the moving body itself is due to the fact that the relative positional relationship between the moving body and the camera installed on the moving body does not change.

  Based on this feature, the analysis device of the present embodiment is configured to detect dirt attached to the camera or the moving body by detecting an object appearing at the same position in the frame over a plurality of frames. .

  By the way, some of the dirt adhering to the camera or the moving body may seriously affect a predetermined process (eg, lane departure determination) using the image data generated by the camera. There are some that do not affect the processing so much. When the former is detected, it is preferable to take measures such as notifying the user, executing a process for removing dirt, and stopping the affected process. However, in the latter case, it can be safely ignored. When any of the stains is detected, if the above-described countermeasure is performed, an unnecessary countermeasure is performed, which is not preferable.

  Therefore, when the analysis apparatus of the present embodiment detects a non-moving object that is captured at the same position in a frame over a plurality of frames, the visual field state (predetermined processing is performed) based on the size of the detected non-moving object. The degree of impact). Then, when the evaluation result falls within a predetermined range, the analysis apparatus according to the present embodiment determines that the “contaminated state”. That is, the state detected by the analysis apparatus of the present embodiment (the state with dirt) is a non-moving object that is captured at the same position in a frame over a plurality of frames, and a visual field state using the size. This is a state where the evaluation result is within a predetermined range. Even when a non-moving target is detected, a state where the evaluation result of the visual field state (degree of influence on predetermined processing) is not within the predetermined range is out of the detection target.

  Note that the splashed condition means that the surface of the camera lens or filter (Photographic Filter) is splashed or the camera is placed inside the windshield. It refers to the state where a certain amount or more of droplets are attached to the inside or outside of the windshield. Raindrops, muddy water, condensation are examples of splashes, but are not limited thereto. The technical concept of the present invention can be applied even to general objects such as dirt, dust, or leaves of trees, and is included in the droplets in the present invention. Further, the target to which the droplets or the like adhere is not limited to the above-described example. Any object that is located between the camera's imaging target and the optical path that connects the camera, moves with the moving body on which the camera is mounted, and transmits or reflects light to guide the light coming from the imaging target to the camera. It is assumed as a target to which droplets or the like in the invention are attached.

  According to the analysis apparatus of the present embodiment as described above, dirt attached to the camera or the moving body may affect a predetermined process (eg, lane departure determination) using image data generated by the camera. Dirt can be detected.

  Next, the configuration of the present embodiment will be described in detail. FIG. 3 shows an example of a functional block diagram of the analysis apparatus 10 of the present embodiment. As illustrated, the analysis apparatus 10 includes a camera 11, a determination unit 12, a non-moving target detection unit 13, a dirt detection unit 14, and a notification unit 15. Hereinafter, each part will be described.

  The camera 11 is installed on a moving body. Further, the camera 11 is configured to be able to take a so-called moving image. Such a configuration of the camera 11 is not particularly limited, and any conventional configuration can be adopted. For example, the camera 11 may be a fixed focus camera. The camera 11 continuously generates image data by real-time processing, and inputs the generated image data to the non-moving target detection unit 13 described below. In addition, although a mobile body corresponds to a motor vehicle, a cargo vehicle, a motorcycle etc., for example, it is not limited to these.

  The determination unit 12 determines whether or not the moving body is moving. Hereinafter, although an example of the determination means by the determination part 12 is demonstrated, it is not limited to this.

  For example, the determination unit 12 may hold an acceleration sensor. Based on the measurement result of the acceleration sensor, the determination unit 12 determines whether the moving body is moving at predetermined time intervals (eg, every 0.5 seconds, every 1 second, every 5 seconds, every 10 seconds, etc.). It may be determined whether or not. In addition, when the moving body has a function of determining whether or not it is moving, the determination unit 12 obtains a determination result (eg, moving or stopped) of the moving body from the moving body every predetermined time. May be. Then, the determination unit 12 may determine whether or not the moving body is moving by analyzing the determination result of the moving body.

  The determination unit 12 inputs a determination result (for example, moving or stopped) to the non-moving target detection unit 13 described below in real time processing.

  When the moving body on which the camera 11 is mounted is moving, objects and scenery around the moving body appear to always move as viewed from the camera 11. Among these, the non-moving target detection unit 13 analyzes the image data generated by the camera 11 while the moving body is moving, and detects a non-moving target that appears at the same position in the frame over a plurality of frames. Specifically, when the non-moving target detection unit 13 acquires image data (a plurality of frame data) from the camera 11 by real time processing, the following (Step 1) to (Step) are sequentially performed on the acquired plurality of frame data. The process of 4) is performed. The non-moving target detection unit 13 may perform the following process on all acquired frame data, or perform the following process only on frame data for each predetermined number of frames. Also good.

(Step 1) It is determined whether or not the frame data to be processed is generated while the moving body is moving.
(Step 2) If the moving object is generated while moving, the object shown in the frame is extracted. If the moving object is not generated during movement, the process of (Step 1) is performed on the next frame data as a processing target.
(Step 3) The position information in the frame of each extracted object and other characteristics (eg, shape, size, etc.) are calculated.
(Step 4) Using the calculated position information and other features, and the calculation result for the previous frame data, a non-moving object that is captured at the same position in the frame over a plurality of frames is detected.

  Hereinafter, each step will be described in order.

  In step 1, the non-moving target detection unit 13 determines whether the processing target frame data is generated while the moving body is moving. This determination is made using the determination result by the determination unit 12. Although various modes may exist as the details, the non-moving target detection unit 13 of the present embodiment can adopt all the modes. An example will be described below, but the present invention is not limited to this.

  For example, the non-moving target detection unit 13 obtains a determination result (e.g., moving) every predetermined time from the determination unit 12 (e.g., every 0.5 seconds, every 1 second, every 5 seconds, every 10 seconds, etc.). Or, the latest state of the moving body (eg, moving or stopped) is managed on the basis of For example, by continuously updating the flag indicating the latest state of the moving body (e.g. moving or stopped) based on the determination result (e.g. moving or stopped) acquired from the determination unit 12, The latest state of the moving body (eg, moving or stopped) may be managed.

  Then, the non-moving target detection unit 13 adds to the frame data to be processed the “latest latest information at a predetermined timing related to the frame data (eg, timing acquired from the camera 11, timing specified as the processing target in Step 1, etc.). Corresponds to the state of the moving object.

  In step 2, if the frame data to be processed is generated while the moving object is moving, the object shown in the frame is extracted. The extraction can be realized using any conventional technique. An example will be described below, but the present invention is not limited to this.

  For example, the analysis unit 12 performs edge detection processing on the processing target frame. And the judgment part 12 specifies the area enclosed by the edge as an area where the object is reflected.

  In step 3, position information in the frame of each extracted object and other features (eg, feature quantities indicating shape, size, etc.) are calculated.

  There are various methods for expressing the position information. For example, a representative value obtained from an arbitrary number of arbitrary positions in the area occupied by each target may be used as the position information of the target. The arbitrary position may be, for example, the center point in the area A (see FIG. 4) occupied by each object, the left end point A1, the right end point A2, the upper end point A3, the lower end point A4, and the like.

  The other characteristics calculated in this step can be used to determine the identity of the object shown in the first frame and the object shown in the second frame. As long as such processing can be realized, the “other features” calculated in this step may be of any type. For example, the shape of the object, the size of the object, etc. can be considered. Such means for calculating other characteristics can be realized in accordance with the prior art, and therefore the description thereof is omitted here.

  In step 4, using the calculated position information and other features and the calculation results (position information and other features) for the previous frame data, the non-images appearing at the same position in the frame over a plurality of frames. Detect moving objects.

  The detection compares the position information and other features of the target extracted from the frame data to be processed with the position information and other features of the target extracted from the previous frame data, and the position information and other features are compared. This can be realized by specifying an object that matches or whose difference is within a predetermined value. Hereinafter, an example of a detailed processing flow will be described with reference to a flowchart, but the processing flow of the step is not limited to that described here.

  Through Steps 1 to 4 described above, non-moving objects appearing at the same position in the frame are detected over a plurality of frames (the number is a design matter). When there are a plurality of non-moving objects separated from each other, the non-moving object detection unit 13 can detect a plurality of non-moving objects.

  The dirt detection unit 14 evaluates the visual field state using one or more sizes of one or more non-moving targets detected by the non-moving target detection unit 13. And the dirt detection part 14 detects the state with dirt, when an evaluation value becomes a predetermined range.

  The visual field state is a visual field state from the camera 11, and the larger the non-moving target due to the dirt existing in the frame, the worse the visual field state. The dirt detection unit 14 can evaluate the visual field state by the following method, for example.

(Method 1) Area of non-moving target In the case of this method, the area of the non-moving target is an evaluation value. The area of the non-moving object can be expressed by, for example, the number of pixels occupied by the non-moving object. When the non-moving object detection unit 13 detects a plurality of non-moving objects, the sum of all the areas may be used as an evaluation value, or a statistical value of each area of the non-moving objects (eg, maximum value, It may be an average value).

(Method 2) Length of the portion related to the non-moving target when raster scanning is performed In the case of this method, the length of the portion related to the non-moving target is the evaluation value. For example, the length of the portion related to the non-moving target (the portion occupied by the non-moving target) may be calculated for each row, and the total number thereof may be used as the evaluation value. The length can be expressed by the number of pixels, for example. The raster scan may not be performed for all rows but may be performed at a predetermined line interval.

(Method 3) Light quantity value captured by the camera 11 When the dirt is present, the light quantity that the camera 11 can capture is less than when the dirt is not present. In addition, the greater the dirt, the less light that the camera 11 can capture. Therefore, when light of a predetermined intensity is emitted from a predetermined light source, the size of the non-moving target can be evaluated based on the light amount value captured by the camera 11. In the case of this method, the evaluation value is a light amount value captured by the camera 11 when a predetermined light source emits light having a predetermined intensity, or a value calculated from this light amount value.

  For example, the analysis apparatus 10 may include a predetermined light source and a shielding mechanism that shields intrusion of external light into the camera 11. The shielding mechanism may be configured to be openable / closable. When the evaluation value is evaluated, the dirt detection unit 14 controls the shielding mechanism to shield the intrusion of external light into the camera 11. Then, the light source is controlled to emit light having a predetermined intensity. Thereafter, the dirt detection unit 14 acquires the light amount value captured by the camera 11. The dirt detection unit 14 may use the light amount value as an evaluation value, or may use a comparison value (eg, difference, ratio, etc.) with a predetermined reference value as an evaluation value.

(Method 4) Vertical length and / or horizontal length of non-moving target In the case of this method, the vertical length and / or horizontal length of the non-moving target is an evaluation value. Note that the shape of the non-moving target tends to be irregular. Hereinafter, an example of the definition of the vertical and horizontal lengths of the non-moving target will be described. For example, the length of the portion of the non-moving target that has the largest spread in the vertical direction may be the vertical length, and the length of the portion that has the largest spread in the horizontal direction may be the horizontal length. In addition, the vertical length from the upper end point (A3 in FIG. 4) to the lower end point (A4 in FIG. 4) of the non-moving object A is the vertical length, and the left end point (A1 in FIG. 4) to the right end point (see FIG. 4). The horizontal length up to A2) of 4 may be the horizontal length. The length can be expressed by the number of pixels, for example.

  When the non-moving object detection unit 13 detects a plurality of non-moving objects, the sum of all the vertical lengths and / or the total of the horizontal lengths may be used as the evaluation value, or A statistical value (eg, maximum value, average value, etc.) of the vertical length and / or horizontal length of each moving object may be used as the evaluation value.

  The dirt detection unit 14 can calculate the evaluation value by the above-described method, for example. Note that the dirt detection unit 14 holds in advance a range of evaluation values that are determined to be dirty. Then, the dirt detection unit 14 determines whether or not the evaluation value is within the predetermined range by comparing the evaluation value calculated by the above method with the predetermined range. As a result of the determination, when the evaluation value is within a predetermined range, the dirt detection unit 14 detects a state with dirt.

  Note that the range of the evaluation value determined to be a dirty state is, for example, a predetermined value when a predetermined process (eg, lane departure determination) is executed based on the evaluation value and image data at the time of each evaluation value state. The relationship with the accuracy of the process may be analyzed in advance and determined based on the analysis result.

  When the dirt detection unit 14 detects a state with dirt, the notification unit 15 outputs information indicating that fact. The output means is not particularly limited, and lighting of a warning lamp, output of a warning sound via a speaker, output of information via a display connected to the analysis device 10 (for example, an in-vehicle display), and the like can be considered. Not.

  Next, an example of the processing flow of the analysis method of this embodiment will be described using the flowchart shown in FIG.

  First, the non-moving target detection unit 13 acquires first first frame data from image data generated by the camera 11 (S101). Next, the non-moving target detection unit 13 determines whether or not the first frame data is generated while the moving body is moving (S102). The details of the determination here are as described above.

  When the first frame data is not generated during the movement of the moving body (No in S102), the non-moving target detection unit 13 acquires the next frame data (S103). Then, the process of S102 is performed on the frame data.

  When the determination result in S102 is “generated during movement of the moving object” (Yes in S102), the non-moving target detection unit 13 extracts a target shown in the frame (S104). Then, position information within the frame of each extracted object and other characteristics (eg, shape, size, etc.) are calculated (S105). The details of the processing of S104 and S105 are as described above.

  Next, the non-moving target detection unit 13 registers the target extracted in S104 as a non-moving target candidate, and associates the number of repetitions (the number of repeated frames) “1” with each target (S106). The non-movement target candidate is associated with the position information calculated in S105 and other features.

  Thereafter, the non-moving target detection unit 13 acquires the next frame data (S107). Then, it is determined whether or not the frame data is generated while the moving body is moving (S108). The details of the determination here are as described above.

  When the frame data acquired in S107 is not generated during the movement of the moving body (No in S108), the non-moving target detection unit 13 acquires the next frame data (S107). Then, the process of S108 is performed on the frame data.

  When the determination result in S108 is “generated during movement of the moving object” (Yes in S108), the non-moving object detection unit 13 extracts an object shown in the frame (S109). Then, position information within the frame of each extracted object and other characteristics (eg, shape, size, etc.) are calculated (S110). The details of the processing of S109 and S110 are as described above.

  Next, the non-moving target detection unit 13 identifies one of the targets extracted in S109 as a processing target (S111). Then, the target specified in S111 is compared with the non-moving target candidate registered at that time (S112). Specifically, the positional information and other characteristics (eg, shape, size, etc.) of the target specified in S111 are compared with the positional information and other characteristics of the non-moving target candidate, and the target specified in S111 It is determined whether there is a non-moving target candidate that matches (matches or has a difference within a predetermined range).

  When there is a matching non-moving target candidate (Yes in S113), the non-moving target detection unit 13 increases the number of repetitions of the non-moving target candidate by “1” (S114). Thereafter, the process proceeds to S116.

  On the other hand, when there is no matching non-moving target candidate (No in S113), the non-moving target detecting unit 13 registers the target specified in S111 as a non-moving target candidate and associates the repetition number “1” (S115). . Then, the position information calculated in S110 and other features are associated with the non-moving target candidates. Thereafter, the process proceeds to S116.

  In S116, it is determined whether there is any target that is not specified as a processing target among the targets extracted in S109. When remaining (Yes in S116), the non-moving target detection unit 13 specifies another target (S117), and repeats the processes after S112. On the other hand, if it does not remain (No in S116), a non-moving target candidate whose number of repetitions has not increased in the process for the current frame data is identified and deleted from the non-moving target candidate (S118).

  Next, the non-moving target detection unit 13 determines whether there are non-moving target candidates having a repetition count equal to or greater than a predetermined value (S119). If not (No in S119), the process returns to S107 and the process is repeated. On the other hand, if there is (Yes in S119), the non-moving target detection unit 13 identifies the non-moving target candidate as a non-moving target. Then, the dirt detection unit 14 evaluates the visual field state using the specified size of the non-moving target (S120). The details of the visual field state evaluation are as described above.

  When the evaluation value in S120 is within the predetermined range, the dirt detection unit 14 detects a state with dirt (Yes in S121). And the alerting | reporting part 15 outputs the information which shows having detected the state with dirt according to the detection (S122). Then, it returns to S107 and repeats a process. On the other hand, when the evaluation value in S120 is not within the predetermined range, the dirt detection unit 14 does not detect a state with dirt (No in S121). Then, it returns to S107 and repeats a process. Thereafter, the above process is repeated until an input for terminating the process at an arbitrary timing is made.

  According to the analysis device 10 of the present embodiment described above, the image data generated by the camera 11 while the moving body is moving is analyzed, and as a result, the non-moving image that appears in the same position in the frame over a plurality of frames. The object can be detected. Then, according to the analysis device 10 of the present embodiment, the visual field state is evaluated using any one or more sizes of the detected one or more non-moving objects, and the evaluation value falls within a predetermined range. A dirty state can be detected.

  According to the analysis apparatus 10 of the present embodiment as described above, it is dirt that adheres to the camera 11 and the moving body, and affects predetermined processing (eg, lane departure determination, etc.) that uses image data generated by the camera 11. It is possible to detect a stain that can give a notification to the user. As a result, the user can recognize the presence of dirt. For this reason, the user can execute measures such as removing the dirt.

<Second Embodiment>
When the analysis apparatus 10 according to the present embodiment detects a dirt state, the analysis apparatus 10 stops predetermined processing (eg, lane departure determination) performed using the image data generated by the camera 11. For this reason, it is possible to avoid the inconvenience of performing processing such as image recognition using a frame in which a non-moving object is captured and executing predetermined processing based on an erroneous recognition result.

  Next, the configuration of the present embodiment will be described in detail. FIG. 6 shows an example of a functional block diagram of the analysis apparatus 10 of the present embodiment. As illustrated, the analysis apparatus 10 includes a camera 11, a determination unit 12, a non-moving target detection unit 13, a dirt detection unit 14, a notification unit 15, an execution unit 16, and a control unit 17. The configurations of the camera 11, the determination unit 12, the non-moving target detection unit 13, the dirt detection unit 14, and the notification unit 15 are the same as those in the first embodiment. Hereinafter, configurations of the execution unit 16 and the control unit 17 will be described.

  The execution unit 16 analyzes the image data generated by the camera 11 and specifies an object existing around the moving body, and executes predetermined processing according to the specified result. The processing content by the execution unit 16 is not particularly limited.

  For example, the execution unit 16 may analyze the image data and specify objects such as white lines on the road, road shoulders, vehicles ahead, and pedestrians. The execution unit 16 then determines the lane departure, the lane departure warning, the steering operation control for maintaining the position in the lane, the moving object and the object based on the positional relationship between the identified object and the moving object. May be performed, the speed control of the moving body, the control of the braking operation of the moving body, and the like may be executed.

  The control unit 17 controls the execution unit 16 so as not to execute the predetermined processing when the contamination detection unit 14 detects the presence of contamination.

  Further, the control unit 17 controls the execution unit 16 so as not to execute the predetermined process in response to the detection of the contamination state by the contamination detection unit 14, and then the evaluation value is out of the predetermined range by the contamination detection unit 14. If it is detected, the execution unit 16 may be controlled to resume the predetermined process in response to detecting that the evaluation value is out of the predetermined range.

  For example, in the step of S121 in the flowchart shown in FIG. 5, the stain detection unit 14 detects that the evaluation value is included in a predetermined range and a state with contamination is detected, and the evaluation value is included in the predetermined range. In any case, the result may be input to the control unit 17 when the dirty state is not detected. And the control part 17 may manage the newest state (whether it is a state with a dirt) based on the input result, and may control the execution part 16 according to the newest state managed.

  In addition to or instead of the above processing, the control unit 17 controls the execution unit 16 so as not to execute the predetermined processing in response to the detection of the presence of contamination by the contamination detection unit 14, and then detects the non-moving object. When it is detected that the non-moving target detected by the unit 13 has disappeared from the frame, the execution unit 16 is controlled to resume predetermined processing in response to the detection that the non-moving target has disappeared from the frame. May be.

  For example, if the non-moving target detection unit 13 determines in step S119 in the flowchart illustrated in FIG. 5 that there is no repetition number equal to or greater than a predetermined value (No in S119), the result is input to the control unit 17. May be. Then, when the control unit 17 controls the execution unit 16 so as not to execute the predetermined process, an input indicating that there is no repetition number equal to or greater than the predetermined value (no non-moving target) is the non-moving target. If received from the detection unit 13, the execution unit 16 may be controlled to resume predetermined processing accordingly.

  Note that the analysis apparatus 10 may notify the user of the fact while the processing of the execution unit 16 is stopped under the control of the control unit 17. Further, when the processing of the execution unit 16 resumes under the control of the control unit 17, the analysis device 10 may notify the user of that fact. There are no particular restrictions on the means of notification.

  According to the present embodiment described above, the same function and effect as those of the first embodiment can be realized. Further, according to the present embodiment, it is possible to stop the predetermined processing by the execution unit 16 while detecting the state with the contamination.

  Non-moving objects due to dirt that adheres to the camera 11 or the moving body and that is so large that it can affect other processes (eg, lane departure determination) that use image data generated by the camera 11 is a frame. If a predetermined process is executed by the execution unit 16 in a state that is reflected in the image, there is a risk of malfunction due to deterioration in accuracy of image analysis. For example, a non-moving object is erroneously determined as an object existing around the moving body, and the steering is controlled to avoid an object that is braked at an unnecessary timing or erroneously recognized (eg, leftward) Moving). As a result, other accidents such as a collision with a following vehicle and a collision with an object (eg, a car) located on the left and right can be caused.

  According to the present embodiment, since the predetermined process by the execution unit 16 can be stopped while the dirty state is detected, such inconvenience can be avoided.

  Further, when it is detected that the state with the contamination is no longer present (the evaluation value is out of the predetermined range), the processing by the execution unit 16 can be resumed according to the detection. For this reason, it can be set as a user-friendly structure which does not require a restart instruction operation by the user.

<Third Embodiment>
The moving body of this embodiment is an automobile. The camera of the analysis apparatus 10 of this embodiment is installed so that the exterior of a motor vehicle may be image | photographed through the windshield of a motor vehicle. In this case, dirt adhering to the windshield is also detected as a non-moving target. Depending on the degree of the contamination, a state with contamination may be detected.

  When the analysis apparatus 10 according to the present embodiment detects a dirt state, it outputs a signal for operating a wiper attached to the windshield of the automobile. According to the present embodiment as described above, if dirt is attached to the windshield, it is possible to eliminate the dirt state without any work by the user.

  Next, the configuration of the present embodiment will be described in detail. In FIG. 7, an example of the functional block diagram of the analyzer 10 of this embodiment is shown. As illustrated, the analysis device 10 includes a camera 11, a determination unit 12, a non-moving target detection unit 13, a dirt detection unit 14, a notification unit 15, and an operation signal output unit 18. Note that the analysis device 10 of the present embodiment may further include an execution unit 16 and a control unit 17. The configurations of the camera 11, the determination unit 12, the non-moving target detection unit 13, the dirt detection unit 14, the notification unit 15, the execution unit 16, and the control unit 17 are the same as those in the first and second embodiments. Hereinafter, the configuration of the operation signal output unit 18 will be described.

  As described above, the moving object of this embodiment is an automobile. And the camera 11 is installed so that the exterior of a motor vehicle may be image | photographed through the windshield of a motor vehicle. FIG. 8 shows an installation example of the analysis apparatus 10 having the camera 11.

  FIG. 8 is a schematic plan view of the windshield 20 of the automobile as viewed from the front. The analysis device 10 is attached to the upper middle of the windshield 20. The installation example is merely an example, and the present invention is not limited to this.

  The operation signal output unit 18 outputs a signal for operating the wiper attached to the windshield when the contamination detection unit 14 detects a state of contamination. The signal is input to a control device that controls the operation of a predetermined part of the automobile. The said control apparatus is comprised so that the operation | movement (control of an action | operation, a stop, etc.) of the wiper attached to the windshield may be controlled at least. When a signal for operating the wiper is input from the operation signal output unit 18, the control device operates the wiper according to the signal.

  According to the present embodiment described above, the same operational effects as those of the first and second embodiments can be realized. Further, according to the present embodiment, the wiper attached to the windshield of the automobile (moving body) can be operated in response to the detection of the dirty state. For this reason, it is also possible to eliminate the dirty state without any work by the user.

<Fourth Embodiment>
The dirt detection unit 14 of the present embodiment evaluates the visual field state using the position in the frame and / or the distribution in the frame in addition to the size of the non-moving target. Different from the embodiment.

  An example of a functional block diagram of the present embodiment is the same as in the first to third embodiments. The configurations of the camera 11, the determination unit 12, the non-moving object detection unit 13, the notification unit 15, the execution unit 16, the control unit 17, and the operation signal output unit 18 are the same as those in the first to third embodiments. Hereinafter, the configuration of the dirt detection unit 14 will be described.

  The dirt detection unit 14 evaluates the visual field state using the position in the frame and / or the distribution in the frame (degree of congestion) in addition to the size of the non-moving target, and calculates an evaluation value.

  First, an example in which a position in a frame is used will be described. Even if the size of the non-moving target in the frame is the same, if the position in the non-moving target frame is different, other processing using the image data generated by the camera 11 (eg, lane departure determination etc.) The impact can vary. Therefore, by evaluating the visual field state in consideration of not only the size of the non-moving object but also the position in the frame, it is a dirt that adheres to the camera 11 and the moving body with high accuracy and is generated by the camera 11. It is possible to detect dirt that may affect a predetermined process (eg, lane departure determination) using image data.

  For example, the stain detection unit 14 may divide the frame into a plurality of areas and hold weighting information in which weighting coefficients are defined for each area. The weighting coefficient for each area is determined according to the influence on the predetermined processing.

  And the dirt detection part 14 is good also considering the value which multiplied the weighting coefficient determined according to the position of a non-moving object to the evaluation value computed by either of the said methods 1 thru | or 4 as an evaluation value. In addition, when the non-moving target is straddling a plurality of areas, the largest weighting coefficient may be applied to the non-moving target. In addition, when there are a plurality of non-moving objects, a weighting coefficient for each non-moving object may be specified according to the position of each non-moving object, and the largest weighting coefficient among them may be adopted.

  In addition, for example, the dirt detection unit 14 multiplies each evaluation value calculated by the method 1 or 4 for each non-moving target by a weighting coefficient for each non-moving target determined according to the position of each non-moving target. The sum of the values may be used as the evaluation value. In addition, when the non-moving target is straddling a plurality of areas, the largest weighting coefficient may be applied to the non-moving target.

  Next, an example in which the distribution in the frame (the degree of congestion) is used will be described. Even if the size of the non-moving target (occupation ratio in the frame) is the same, the camera 11 is used when the non-moving target is densely located in a predetermined place and when the non-moving target is evenly distributed in the frame. The influence on other processing (eg, lane departure determination etc.) using the generated image data may be different. Therefore, by evaluating the field of view in consideration of not only the size of the non-moving object but also the distribution in the frame, it is a dirt that adheres to the camera 11 or the moving body with high accuracy and is generated by the camera 11. It is possible to detect dirt that may affect a predetermined process (eg, lane departure determination) using image data.

  For example, the dirt detection unit 14 sets a window having a predetermined size, scans the window with a predetermined movement width in a predetermined direction within the frame, and determines the occupation ratio of the non-moving target in the window at each window position. calculate. The occupation ratio can be calculated by, for example, an expression of (number of pixels occupied by a non-moving target in the window) / (number of pixels included in the window). And the dirt detection part 14 specifies the largest value as a representative value among the occupation ratio calculated in each window position. The larger the representative value, the denser the non-moving objects are at a predetermined location.

  For example, when the evaluation value calculated by any one of the above methods 1 to 4 is within a predetermined range and the degree of distribution (representative value) calculated by the above method is within the predetermined range, the dirt detection unit 14 The state with dirt may be detected.

  According to the present embodiment described above, the same function and effect as those of the first to third embodiments can be realized. In addition, according to the present embodiment, the state of contamination can be detected based not only on the size of the non-moving object but also on the position and distribution in the frame, so that the camera 11 and the moving body are attached with high accuracy. It is possible to detect dirt that may affect a predetermined process (eg, lane departure determination) using image data generated by the camera 11.

<Fifth Embodiment>
First, an outline of the present embodiment will be described. When a target moving in the same direction at the same speed as the moving object is shown over a plurality of frames, the analysis apparatus 10 detects this target as a non-moving target. In addition, there is a possibility that a state with dirt is erroneously detected. The present embodiment includes means for reducing the occurrence of such inconvenience.

  Next, the configuration of the present embodiment will be described in detail. FIG. 9 shows an example of a functional block diagram of the analysis apparatus 10 of the present embodiment. As illustrated, the analysis apparatus 10 includes a camera 11, a determination unit 12, a non-moving target detection unit 13, a dirt detection unit 14, a notification unit 15, and an attribute determination unit 19. Note that the analysis device 10 of this embodiment may further include at least one of the execution unit 16, the control unit 17, and the operation signal output unit 18. The configurations of the camera 11, the determination unit 12, the non-moving target detection unit 13, the dirt detection unit 14, the notification unit 15, the execution unit 16, the control unit 17, and the operation signal output unit 18 are the same as those in the first to fourth embodiments. It is. Hereinafter, the configuration of the attribute determination unit 19 will be described.

  The attribute determination unit 19 determines an attribute of an object existing around the moving body. Then, when the attribute determination unit 19 determines that the target (target extracted from the frame) has an attribute that can move at the same speed as the moving object on which the camera is installed, the attribute determination unit 19 determines from the evaluation target of the dirt detection unit 14. Remove the subject.

  In addition, the attribute determination unit 19 is a predetermined object that may be included in a frame captured by the camera 11 installed on the moving body and may move in the same direction at the same speed as the moving body. By pattern matching using an image (eg, car, person, bicycle, etc.), an attribute (eg, type of object, such as car, person, bike, etc.) may be specified for each object extracted from the frame. Then, the attribute determination unit 19 detects the non-moving target detection process by the non-moving target detection unit 13 and / or the dirt detection unit for the target identified as the predetermined target among the targets extracted from the frame. 14 may be excluded from the process of detecting a dirty state. This is because it is unlikely that the shape of the non-moving target due to the dirt will be the same shape as the predetermined target.

  According to the present embodiment, even when a target moving in the same direction at the same speed as the moving object is captured over a plurality of frames, the inconvenience of erroneously detecting a dirty state based on the target Can be reduced. As a result, it is possible to improve the detection accuracy of the dirty state.

  As mentioned above, although embodiment of this invention was described with reference to drawings, these are illustrations of this invention and various structures other than the above may be employ | adopted.

1A CPU
2A Memory 3A I / O I / F
4A Communication device 5A Camera 6A Bus 10 Analysis device 11 Camera 12 Judgment unit 13 Non-moving target detection unit 14 Dirt detection unit 15 Notification unit 16 Execution unit 17 Control unit 18 Operation signal output unit 19 Attribute determination unit 20 Windshield 101 Front vehicle 102 -1 to 102-4 White line 103-1 and 103-2 Roadside tree 200 Non-moving object

Claims (9)

A camera installed on a moving object;
Determining means for determining whether or not the moving object is moving;
Non-moving object detection means for analyzing image data generated by the camera while the moving body is moving and detecting non-moving objects appearing at the same position in a frame over a plurality of frames;
The visual field state is evaluated using any one or more sizes of the one or more non-moving objects detected by the non-moving object detection means, and there is contamination when the evaluation value falls within a predetermined range. Dirt detection means for detecting the state of
When the dirt detection means detects a state with dirt, a notification means for outputting information indicating that,
Analyzing device. The analysis device according to claim 1,
Analysis means for analyzing the image data, specifying an object existing around the moving body, and executing predetermined processing according to the specified result;
A control means for controlling the execution means so as not to execute the predetermined process when the dirt detection means detects a dirty state;
An analysis device further comprising: The analysis device according to claim 2,
The control means controls the execution means so as not to execute the predetermined process in response to detection of a dirt state by the dirt detection means, and then the evaluation value is deviated from the predetermined range by the dirt detection means. An analysis device that controls the execution unit to resume the predetermined process in response to detection that the evaluation value is out of the predetermined range when it is detected that the evaluation value has deviated. The analysis device according to claim 2,
The control means controls the execution means so as not to execute the predetermined processing in response to detection of a dirt state by the dirt detection means, and then the non-moving target detected by the non-moving target detection means When it is detected that the non-moving object has disappeared from the frame, the analyzing apparatus controls the execution means to resume the predetermined processing in response to the detection that the non-moving object has disappeared from the frame. In the analysis device according to any one of claims 1 to 4,
Attribute determination means for determining an attribute of an object existing around the moving object;
When the attribute determination unit determines that the target has an attribute that can move at a speed equivalent to a moving object in which the camera is installed, the attribute determination unit removes the target from the evaluation target of the dirt detection unit.
Analysis device. In the analysis device according to any one of claims 1 to 5,
The moving body is an automobile;
The camera is installed so as to photograph the outside of the automobile through the windshield of the automobile,
An analysis apparatus further comprising an operation signal output means for outputting a signal for operating a wiper attached to the windshield when the dirt detection means detects a dirt state. In the analysis device according to any one of claims 1 to 6,
The said dirt detection means is an analyzer which evaluates the said visual field state using at least one of the position and distribution in a flame | frame in addition to the magnitude | size of the said non-moving object. Computer
A determination step of determining whether or not the moving object is moving;
A non-moving target detection step of analyzing image data generated by a camera installed on the moving body while the moving body is moving, and detecting a non-moving target that appears in the same position in the frame over a plurality of frames;
When the visual field state is evaluated using any one or more of the one or more non-moving objects detected in the non-moving object detection step, and the evaluation value falls within a predetermined range, there is contamination. A dirt detection step for detecting the state of
When a state with dirt is detected in the dirt detection step, a notification step for outputting information indicating that,
Analysis method to execute. Computer
Determining means for determining whether or not the moving object is moving;
Non-moving object detection means for analyzing image data generated by a camera installed on the moving object while the moving object is moving, and detecting a non-moving object appearing at the same position in the frame over a plurality of frames;
The visual field state is evaluated using any one or more sizes of the one or more non-moving objects detected by the non-moving object detection means, and there is contamination when the evaluation value falls within a predetermined range. Dirt detection means for detecting the state of
When the dirt detection means detects a state with dirt, a notification means for outputting information indicating that,
Program to function as.

Images