JP2019053625A - Moving object detection device, and moving object detection method - Google Patents

Abstract

To provide an inexpensive and highly safe moving object detection device and moving object detection method that can detect moving objects from motion images of a monocular camera with high accuracy and quickly.SOLUTION: A representative configuration of a moving object detection device according to the present invention comprises a moving object detection unit that sets a detection frame to a moving object of a frame image. The moving object detection unit is configured to: extract a foreground pixel from a current frame, using a background differential method; determine a frame surrounding the foreground pixel as a primary candidate frame; acquire the detection frame of a previous frame from a storage unit; determine, of the primary candidate frames, the primary candidate frame overlapping the detection frame of the previous frame in an area more than a prescribed ratio as a secondary candidate frame; group the secondary candidate frames overlapping the detection frame of the previous frame, of the secondary candidate frames; determine, of the grouped secondary candidate frames, the secondary candidate frame overlapping the detection frame thereof in the area more than prescribed ratio in a y-axis direction as an integrated detection frame; and determine the secondary candidate frame non-overlapping the detection frame thereof in the area more than prescribed ratio in the y-axis direction as a separated detection frame.SELECTED DRAWING: Figure 6

Description

  The present invention relates to a moving object detection apparatus and a moving object detection method capable of detecting a moving object with high accuracy and high speed from a moving image of a monocular camera.

  Currently, many preventive safety technologies are being researched and developed to prevent traffic accidents. One of these is a request to prevent traffic accidents at the intersection of cars and pedestrians, bicycles, cars, etc. at intersections. In such a system, it is conceivable to detect a moving object using an in-vehicle sensor or an in-vehicle camera.

  As an object detection technique using a vehicle-mounted sensor, there is a system that uses an auxiliary device such as a laser range sensor. However, the laser device is expensive and has a problem that it is difficult to increase the output because it cannot irradiate the human body with a strong laser.

  When using an in-vehicle camera, it is conceivable to use the background subtraction method. The background subtraction method can be processed at high speed and is a highly reliable method using dynamic scene changes. However, when the object has a flat texture or when a part of the object has a color similar to the background, only a part of the object may be detected.

  In Patent Document 1, when there are a plurality of candidates for the detection target region (face portion and foot portion), the size that the detection target region should occupy in the captured image is specified, and other candidates exist within the size range. If included, an image processing apparatus has been proposed that extracts a region obtained by combining the candidate and another candidate as a new candidate for one detection target region.

JP 2010-092353 A

  However, the technique of Patent Literature 1 specifies the size to be occupied based on the position of the detection target region candidate in the captured image. Then, it is necessary to first determine the distance to be detected. In Patent Document 1, the imaging device is fixedly installed so as to look down on the intersection (paragraph 0038, FIG. 1). Although it is possible to measure the distance from the position on the image under this limited condition, it is difficult to measure the distance based on the image of the in-vehicle camera.

  In Patent Document 1, it is necessary to determine the type of detection target (whether it is a person) and to specify the size to be occupied. Such a determination is very uncertain. Further, in Patent Document 1, since an infrared camera is used, a portion where the skin is exposed is detected. However, there is uncertainty in that the area where the skin can be seen is completely different depending on the direction of the person, clothes, and hairstyle. It will increase.

  Further, when an inexpensive monocular camera is used instead of an infrared camera, or when a fish-eye lens or a wide-angle lens is used for photographing a wider range, noise may be larger than when an infrared camera is used.

  Accordingly, an object of the present invention is to provide an inexpensive and highly safe moving object detection apparatus and moving object detection method capable of detecting a moving object with high accuracy and high speed from a moving image of a monocular camera.

  In order to solve the above problems, a typical configuration of the moving object detection device according to the present invention includes an input unit that inputs a moving image, a frame acquisition unit that continuously extracts a plurality of frame images from the moving image, and a moving unit. A moving object detection unit configured to set a detection frame on the object; and a storage unit configured to store the set detection frame. The moving object detection unit extracts a foreground pixel from the current frame using a background subtraction method. The enclosing frame is set as a primary candidate frame, the detection frame of the previous frame is acquired from the storage unit, and the primary candidate frame is defined as a secondary candidate frame if the area of the detection frame of the previous frame overlaps by a predetermined amount or more. Of the frames, those that overlap with the detection frame of the same previous frame are grouped, and among the grouped secondary candidate frames, those that overlap more than a predetermined amount in the y-axis direction are combined into a detection frame, and more than a predetermined overlap in the y-axis direction. What does not become a separate detection Characterized by a.

  According to the above configuration, the current frame candidate frame is integrated and separated within a range that overlaps the detection frame of the previous frame. It is not necessary to determine the type and distance of the moving object, and the candidate frames can be integrated and separated even if there is a lot of noise. As a result, it is possible to set a detection frame appropriately corresponding to a moving object, whether it is an image of a monocular camera or an image taken while moving like a vehicle-mounted camera. Therefore, a moving object approaching can be detected at high speed with high accuracy, and safety can be improved at low cost.

  A background image generation unit that combines the background image and the current frame image to update the background image may be further included. Thus, a background image can be generated as appropriate even for a moving image shot while moving.

  A filter unit for removing a detection frame that does not satisfy a predetermined area or aspect ratio with respect to one or more detection frames detected in the current frame may be further provided. The fact that the moving image is taken while moving and that it is an image of a monocular camera causes noise to increase. According to the above configuration, noise can be removed and detection accuracy of a moving object can be improved.

  A filter unit may be further provided that detects a horizon in the current frame and removes detection frames that do not overlap the horizon at a predetermined ratio with respect to one or more detection frames detected in the current frame.

  In a moving image taken while moving with an in-vehicle camera or the like, there is a possibility that a road sign or a pedestrian crossing is detected as a moving object below, and a building, a signboard, a traffic light, an electric wire, a street light, or the like is detected as a moving object. However, the moving object to be detected in the present invention moves on the ground. And according to said structure, the detection frame which is not a mobile body can be removed.

  The filter unit may use a genetic algorithm for detecting the horizon. As a result, it is possible to detect with high accuracy a horizon that is obstructed by the shape of the building or road and is not easily understood.

  A typical configuration of the moving object detection method according to the present invention is that a moving image is input, a plurality of frame images are continuously extracted from the moving image, a frame image to be processed is set as a current frame, and the previous frame image Is the previous frame, the foreground pixels are extracted from the current frame using the background subtraction method, the frame surrounding the foreground pixels is set as the primary candidate frame, the detection frame surrounding the moving object in the previous frame is acquired, and the primary candidate is acquired. Of the frames, a frame that overlaps the detection frame of the previous frame by a predetermined amount or more is a secondary candidate frame, and among the secondary candidate frames, those that overlap the detection frame of the same previous frame are grouped, and the grouped secondary candidates Among the frames, those that overlap a predetermined amount or more in the y-axis direction are integrated detection frames, and those that do not overlap a predetermined amount or more in the y-axis direction are separated detection frames.

  According to the above method, a moving object can be detected from a moving image of a monocular camera at high speed with high accuracy, and safety can be improved at low cost. The same technical idea as that of the moving object detection apparatus can be applied to the moving object detection method.

  According to the present invention, it is possible to provide an inexpensive and highly safe moving object detecting apparatus and moving object detecting method capable of detecting a moving object with high accuracy and high speed from a moving image of a monocular camera.

It is a figure explaining the whole structure of the moving object detection apparatus concerning this embodiment. It is a flowchart explaining the moving object detection method concerning this embodiment. It is a flowchart explaining the procedure of the detection of a moving object. It is an example of an image of a background difference method. It is an example of an image explaining expansion and contraction of a foreground pixel. It is an example of an image explaining setting of a detection frame. It is another example of an image explaining integration and separation. It is an example image explaining filtering processing of a detection frame. It is an example of an image explaining generation of a background image. It is an example of an image explaining filtering processing using a horizon. It is a figure explaining the conditions etc. of a genetic algorithm. It is a flowchart of a genetic algorithm. It is a figure explaining the objective function of a genetic algorithm.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The dimensions, materials, and other specific numerical values shown in the embodiments are merely examples for facilitating understanding of the invention, and do not limit the present invention unless otherwise specified. In the present specification and drawings, elements having substantially the same function and configuration are denoted by the same reference numerals, and redundant description is omitted, and elements not directly related to the present invention are not illustrated. To do.

[First Embodiment]
A first embodiment of a moving object detection device and a moving object detection method according to the present invention will be described. FIG. 1 is a diagram illustrating the overall configuration of a moving object detection device 100 according to the present embodiment. FIG. 2 is a flowchart for explaining the moving object detection method according to the present embodiment.

  As shown in FIG. 1A, the moving object detection device 100 is an in-vehicle system that is mounted on an automobile 10. As a representative example, a moving image captured by the in-vehicle camera 12 of the automobile 10 is input to the moving object detection device 100 to detect a moving object. Information on the moving object detected by the moving object detection apparatus 100 is sent to the braking system 14 or a monitor (not shown) for use.

  The in-vehicle camera 12 is an inexpensive monocular camera and captures a general moving image. The moving image may be a color image or a gray scale image (monochrome image). The in-vehicle camera 12 may include a fisheye lens or a wide angle lens. According to the present invention, it is possible to detect a moving object with high accuracy and high speed even from a moving image of a monocular camera, as will be described later. Can be provided.

  FIG. 1B shows the configuration of the moving object detection device 100. Specifically, the moving object detection apparatus 100 can be constructed by a computer system. Hereinafter, the configuration of the moving object detection device 100 will be described with reference to the flowchart shown in FIG.

  A moving image captured by the in-vehicle camera 12 is input to the input unit 110 (step 210). When the moving image is a video signal, a video encoder chip having a composite terminal or an HDMI (registered trademark) terminal corresponds to the input unit 110. When the moving image is encoded digital data, the USB or IEEE 1394 interface corresponds to the input unit 110. In any case, the input unit 110 only needs to be able to input moving image data so that it can be processed in the moving object detection apparatus 100.

  The frame acquisition unit 112 continuously acquires a plurality of frame images (still images) from the input moving image (step 212). A specific process for acquiring a frame from a moving image depends on the format of the moving image. For example, if the moving image is a format in which still images are arranged, such as Motion JPEG, each frame may be simply extracted. In addition, you may extract by a fixed time unit (for example, 0.1 second unit), without depending on fps (frame per second) of a moving image. What is differentially compressed like MPEG may be extracted for each G frame (Group Of Pictures) I frame.

  The preprocessing unit 114 performs preprocessing on the acquired frame image (step 214). In the present embodiment, the input moving image is a color image, and is converted into a grayscale image as preprocessing. Note that, as preprocessing, noise removal may be performed as necessary, or crop processing for cutting out an area to be subjected to image processing may be performed.

  When the frame image to be processed is the current frame, the moving object detection unit 118 detects the moving object from the current frame using the background subtraction method, and sets the detection frame (step 216). In the following description, the frame before the frame to be processed is referred to as “previous frame”, and the next frame is referred to as “next frame”.

  FIG. 3 is a flowchart for explaining the procedure (subroutine) for detecting a moving object. The moving object detection unit 118 first extracts foreground pixels using the background subtraction method (step 232).

  FIG. 4 is an image example of the background subtraction method. In the image of the current frame shown in FIG. 4A, a person riding on a bicycle located on the left side and two cars located in the center are moving objects. The background image illustrated in FIG. 4B is acquired from the background image generation unit 116. These moving objects are also shown in the background image, but their positions are slightly different. When the background difference method is applied to these images (luminance subtraction is performed), different pixels are extracted as in the difference image shown in FIG. A pixel having this difference is a foreground pixel.

  FIG. 5 is an image example illustrating the expansion and contraction of the foreground pixels. The foreground pixels extracted by the background subtraction method are in a state of low brightness and finely divided. Therefore, smoothing (blurring processing) using a Gaussian filter or the like is performed to connect adjacent foreground pixels, and then binarized as shown in FIG. 5A to clarify (step 234). Even in this case, since there are still voids and fine divisions, the foreground pixels are contracted as shown in FIG. 5C after the foreground pixels are expanded as shown in FIG. 5B (step 236). .

  The expansion is a process of setting pixels within the filter diameter centering on all white pixels to white when the background pixel is black and the foreground pixel is white and the filter diameter is 5 × 5, for example. Shrinkage is the reverse, and is the process of making the pixels within the filter diameter centered on all black pixels black. When expansion and contraction are performed, the outline of the outer edge returns to the original position, but the voids and divisions (discontinuities) that are filled by expansion remain buried even after contraction, so adjacent foreground pixels are connected together. can do.

  Here, as explained at the beginning, in the background subtraction method, when the object has a flat texture or when a part of the object has a color similar to the background, the brightness of the background image and the current frame image Since no difference occurs, only a part of the object may be detected. In this case, since the foreground pixels exist apart from each other, the foreground pixels cannot be connected only by smoothing as described above, expansion and contraction. Therefore, in the present invention, the candidate frames described below are integrated and separated.

  The integration is a process of determining that a single moving object is originally extracted as a plurality of detection frames due to erroneous detection to form one detection frame. Separation is a process of determining adjacent detection frames as a plurality of adjacent moving objects and making them independent detection frames.

  FIG. 6 is an example of an image for explaining detection frame setting. The moving object detection unit 118 detects the outline of the extracted foreground pixels, and sets the frames surrounding the foreground pixels as primary candidate frames 150a to 150f as shown in FIG. 6A (step 238). Then, the moving object detection unit 118 acquires the detection frame of the previous frame from the storage unit 120 (step 240). If there is no previous frame detection frame in the current frame, this integration and separation processing (steps 240 to 244) is skipped.

  Then, the moving object detection unit 118 has, as shown in FIG. 6B, primary detection frames 152a to 150f shown in FIG. % Or more) are set as secondary candidate frames 154a to 154d (step 242). In FIG. 6B, the detection frame 152 of the previous frame is indicated by a one-dot chain line, and the secondary candidate frame 154 is indicated by a broken line. Of the primary candidate frames 150a to 150f, the primary candidate frames 150e and 150f that have not become secondary candidate frames are used as detection frames as they are ("detection frames" are not candidates).

  The moving object detection unit 118 groups the secondary candidate frames 154 that overlap with the detection frame 152 of the same previous frame. Since the secondary candidate frames 154a and 154b overlap the detection frame 152a of the previous frame on the left side in FIG. 6B, these are grouped. Then, among the grouped secondary candidate frames 154a and 154b, as shown in FIG. 6C, an overlapping detection frame 156a that overlaps in the y-axis direction by a predetermined amount (for example, 30% or more). Those that do not overlap a predetermined amount or more in the y-axis direction are separated (independent) detection frames (step 244). Note that “overlap in the y-axis direction by a predetermined amount or more” is synonymous with a range of coordinates in the x-axis direction of the frame overlapping a predetermined amount or more.

To summarize the above, there are the following three detection frames.
-Of the primary candidate frames, those that did not become secondary candidate frames (150e, 150f)
-Integrated secondary candidate frame (156a)
-Secondary candidate frames that remain separated without being merged (154c, 154d)
When there are a large number of grouped secondary candidate frames, a plurality of integrated detection frames may be set. For example, when there are four secondary candidate frames in one group, two detection frames may be set by integrating two candidate frames.

  FIG. 7 is another image example for explaining the integration and separation, and is a processing example when two pedestrians that have been seen overlapping each other are separated. FIG. 7A shows an image of the current frame. Two pedestrians are walking on the road. FIG. 7B shows the foreground pixels extracted in the current frame and corresponds to the center of the current frame. The foreground pixel of the right person is one primary candidate frame 150g, while the foreground pixel of the left person is two primary candidate frames 150h and 150i cut off at the top and bottom. There are a total of three foreground pixel blocks for two pedestrians. FIG. 7C shows the foreground pixels and detection frames extracted in the previous frame. In the previous frame, two pedestrians still seemed to overlap each other, so two people form one large detection frame 152d.

  Then, as shown in FIG. 7D, the moving object detection unit 118 superimposes the detection frame 152d of the previous frame on the primary candidate frame 150, and sets an area that overlaps a predetermined area or more as a secondary candidate frame. In this example, the three primary candidate frames 150g-i are all three secondary candidate frames 154e-g. Further, since the three secondary candidate frames 154e to 154g overlap the detection frame 152d of the same previous frame, they are grouped.

  In FIG. 7D, since the secondary candidate frames 154f and 154g overlap each other by a predetermined amount or more in the y-axis direction, they are set as an integrated detection frame 156b as shown in FIG. Since the secondary candidate frame 154e does not overlap by a predetermined amount or more in the y-axis direction, it remains separated without being integrated. As a result, as shown in FIG. 7F, detection frames (secondary candidate frame 154e and integrated detection frame 156b) are set for two pedestrians, respectively. When determining the degree of overlap, the size (length) of the two candidate frames overlapping in the x-axis direction may be compared with the smaller of the two candidate frames in the x-axis direction.

  In this manner, it is possible to appropriately integrate the three foreground pixels and set the detection frames for the two moving objects. At the same time, the detection frame that was one in the previous frame can be separated into two detection frames in the current frame.

  FIG. 8 is an image example illustrating detection frame filtering processing (step 246). FIG. 8A shows a detection frame after the integration and separation are completed (see FIG. 6C). In FIG. 8A, the person riding on the bicycle on the left side of the image becomes an integrated detection frame 156a, and the two cars in the center of the image are detected as secondary candidate frames 154c and 154d. The primary candidate frames 150e and 150f remain as detection frames, but are extremely small and can be considered as noise.

  Therefore, in the present embodiment, the filter unit 122 removes detection frames that do not satisfy a predetermined area or aspect ratio from one or more detection frames detected in the current frame (ASF: Area Size Filtering). That is, an extremely small detection frame or an extremely long detection frame is removed.

  In the example of FIG. 8B, extremely small detection frames (primary candidate frames 150e and 150f) are removed by filtering processing, and integrated detection frames 156a, secondary candidate frames 154c and 154d remain. In FIG. 8B, these final detection frames are shown superimposed on the grayscale image of the current frame instead of the binary image.

  The fact that the moving image is taken while moving, the image of a monocular camera, and the use of a fish-eye lens or a wide-angle lens are factors that increase noise. Therefore, by performing the above filtering process, it is possible to remove noise and improve the detection accuracy of the moving object.

  The moving object detection unit 118 registers the final detection frame subjected to the filtering process by the filter unit 122 in the storage unit 120 (step 218). The detection frame is stored in association with the frame number, and a detection frame that has been detected for an image of an arbitrary frame can be read out. In particular, the detection frame of the current frame is used for image processing of the next frame, like the detection frame of the previous frame is used for image processing of the current frame in the above description. The detection frame set by the moving object detection device 100 is sent to the braking system 14 or a monitor (not shown) by the output unit 124.

  The background image generation unit 116 generates (updates) a background image for processing the next frame image (step 220). Of the plurality of frame images acquired by the frame acquisition unit 112, the foreground pixels are not extracted from the first frame image and are used only as the background image. When foreground pixels are extracted from the second frame image, the first frame image is used as the background image. When extracting the foreground pixels from the third and subsequent frame images, the background image generated (updated) by the background image generation unit 116 is used.

  FIG. 9 is an image example illustrating generation of a background image. The background image generation unit 116 combines the background image and the current frame image to update the background image, and sets it as a background image when processing the next frame. As a specific example, as shown in FIG. 9, a background image is generated by synthesizing an image in which the luminance of the current frame is reduced to 25% and an image in which the luminance of the background image is reduced to 75%. By sequentially combining (updating) the background image in this way, the background image can be obtained even if it is a moving image shot while moving or a moving image without a moving object. It can generate | occur | produce suitably.

  In addition, in the background subtraction method, updating the background model based on learning has a high calculation cost, so that it has been difficult to apply to high-speed moving image processing. However, combining the past background image and the current frame image as in this embodiment requires a small amount of calculation, so that the calculation cost can be improved as a whole system.

  When the generation of the background image is completed, it is determined whether there is a next frame image (step 222). If there is an image of the next frame, the above series of processing is repeated. If there is no next frame image, the process is terminated.

  As described above, according to the moving object detection device and the moving object detection method according to the present embodiment, the candidate frame of the current frame is integrated and separated within the range overlapping the detection frame of the previous frame. It is not necessary to determine the type and distance of the moving object, and the candidate frames can be integrated and separated even if there is a lot of noise.

  As a result, it is possible to set a detection frame appropriately corresponding to a moving object, whether it is an image of an inexpensive monocular camera or an image taken while moving like an in-vehicle camera. The background subtraction method can be processed at high speed, and the above-described candidate frame integration and separation processing and filtering processing can also be processed at high speed. Therefore, a moving object approaching can be detected at high speed with high accuracy, and safety can be improved.

  In particular, since the integration and separation algorithms are simple, the amount of calculation is small compared with the conventional simple background subtraction method, although the detection rate of moving objects is greatly increased. Compared with the prior art, this method has high practicality and sufficiently high technical superiority. In addition, since only one monocular camera is used in the present invention, there is a significant cost advantage compared to conventional methods using a plurality of cameras, laser radars, infrared cameras, and the like.

  INDUSTRIAL APPLICABILITY The present invention can be applied to an in-vehicle safe driving system that prevents a traffic accident at the head of a meeting or an automatic traveling system of a mobile device such as a robot. Further, the present invention is not limited to a system mounted on an automobile or a robot, and can be applied to a fixed installation monitoring system using a wide-angle security camera.

[Second Embodiment]
A second embodiment of the moving object detection device and the moving object detection method according to the present invention will be described. In the first embodiment, it has been described that the filter unit 122 removes a detection frame that does not satisfy a predetermined area or aspect ratio (ASF). In contrast, in the present embodiment, the horizon is detected instead of or in addition to the ASF, and one or more detection frames detected in the current frame are not overlapped with the horizon at a predetermined rate. The detection frame is removed (HLF: Horizontal Line Filtering).

  FIG. 10 is an image example illustrating filtering processing using the horizon. FIG. 10A shows a binarized image and a detection frame (completed until integration and separation). FIG. 10B shows the detection frame after filtering processing superimposed on the grayscale image of the current frame.

  As can be seen by comparing FIG. 10 (a) with FIG. 10 (b), in moving images taken while moving with an in-vehicle camera or the like, road markings and pedestrian crossings (pedestrian crossing detection frame 160b) are shown below. There is also a possibility that buildings, signboards, traffic lights, electric wires, street lights, etc. (signboard detection frame 160c, electric wire detection frame 160d, street light detection frame 160e) may also be detected as moving objects. However, the moving body to be detected in the present invention is one that moves on the ground such as a pedestrian or automobile (automobile detection frame 160a).

  Therefore, as in the determination formula shown in FIG. 10C, the areas a and b divided by the horizon HL are obtained for all the detection frames 160, and the detection frames that do not overlap the horizon HL at a predetermined ratio are removed. To do. In the present embodiment, a detection frame that satisfies the determination formula: | a−b | / (a + b) ≦ 0.6 is left (correct = 1), and other detection frames are removed (correct = 0).

  As a specific example of the determination formula, when the horizon HL passes through the center of the detection frame, since a = b, the value of the determination formula is 0 and Correct = 1, and the detection frame 160 remains. When the horizon HL does not overlap the detection frame, the value of the determination formula is 1, and Correct = 0 and the detection frame 160 is removed.

  Note that the threshold value is set to 0.6, and the threshold value may be set as appropriate. For example, it may be | a−b | / (a + b) <1. In this case, only the detection frame in which the horizon HL does not overlap the detection frame at all is removed.

  In this way, the detection frame 160 that is not a moving body can be removed by appropriate and high-speed processing. As a result, it is possible to increase the detection accuracy of a moving object that is desired to be detected for avoiding danger.

  There are various methods for detecting the horizon. In this embodiment, the horizon is detected by using a genetic algorithm for detecting the horizon. The genetic algorithm is the most preferable method because it is a multi-point search that always improves a solution while maintaining a plurality of good solutions locally, so that the best solution can be searched globally. As a result, it is possible to detect with high accuracy a horizon that is obstructed by the shape of the building or road and is not easily understood.

FIG. 11 is a diagram for explaining conditions and the like of the genetic algorithm. Briefly explaining the detection of the horizon described below, a boundary line between the road surface and other areas (such as a structure such as a building or the sky) is detected by a genetic algorithm, and this is defined as the horizon. P _i (x _i , y _i ) is the i-th pixel (point) on the horizon. x _i is the x coordinate of the i th point on the horizon. y _i is the y coordinate of the i th point on the horizon, and is represented by the baseline height B + constant a + variable b _i . Then, a, b ₀ , b ₁ ,... B _M are set as the chromosomes of the genetic algorithm.

  FIG. 12 is a flowchart of the genetic algorithm. The filter unit 122 first initializes the algorithm using a predetermined initial value (step 250). For example, the initial value of the algorithm may be 50 generations, 50 initial individuals, 0.7 crossover rate, and 0.01 mutation rate.

  Next, provisional parameters are given to the initial individual to generate a horizon candidate line (step 252). The fitness of the candidate line with respect to the objective function is evaluated while repeating crossover and mutation based on the logic of the genetic algorithm (step 254). Since crossover and mutation in the genetic algorithm are known techniques, detailed description thereof is omitted here.

FIG. 13 is a diagram for explaining an objective function of the genetic algorithm. The objective function F is the number of pixels Diff (j, x) where a difference in pixel values exceeding a threshold value is recognized. j is an evaluation width, and the evaluation range L is the maximum value. x is an x coordinate of an intermediate position between the pixels P _i (x _i , y _i ) and P _{i + 1} (x _{i + 1} , y _{i + 1} ). y _x is a function of x representing the y coordinate of the pixel P by linear interpolation (linear equation).

represents i th pixel _P i _(x i, _{y i)} the pixel value of the (gray scale _{value) P (x i, y i} ) and. The determination of the pixel Diff (j, x) where the difference is recognized is 1 if the pixel value difference | P (x, y _x + j) −P (x, y _x −j) | 0 if there is. DiffTB (j, x) is 1 when the pixel values (grayscale values) differ by more than a threshold value by comparing pixel values at positions 2j above and below in the y-axis direction. Therefore, the value of the objective function F increases as the horizon formed by the chromosomes a, b ₀ , b ₁ ,... B _M extends along the boundary line between the road surface and the rest. In other words, the horizon detection by this genetic algorithm is the contour extraction of the upper edge of the road surface.

DiffTN and DiffBN in the evaluation function F compare pixel values of P (x, y _x −j) and P (x, y _x + j) and the next (next) pixel, respectively. As a result, the value of the evaluation function F increases not only when it reaches the horizontal line (horizontal line) but also the outline (vertical line) of a building or the like. Since the value increases at the intersection of the horizontal line and the vertical line, that is, the value increases at the intersection of the building and the ground. Thereby, the accuracy of horizon detection can be improved.

  When finishing the calculation of the predetermined number of generations, the filter unit 122 outputs the selected candidate line as a horizon line (step 256). The output horizon is used in the determination formula described in FIG.

  Then, the filter unit 122 determines whether there is a next frame (step 258). If there is a next frame, the horizon detection by the genetic algorithm is repeated, and if there is no next frame, the horizon detection processing is terminated.

  Detection of the horizon using a genetic algorithm can be performed on a single image. In this case, the processing may be performed for all the frames, but the speed can be significantly increased by performing the processing for the frames at regular intervals. Further, when the image is not changed, processing is performed at regular intervals, and processing is performed even when the image is significantly changed, so that the processing speed can be increased and the accuracy can be improved.

  In addition, if an acceleration sensor is mounted or an acceleration sensor signal can be received from the navigation system, processing may be performed when turning a corner or when the up / down direction changes. This makes it easier to follow changes in the horizon and improve detection accuracy.

  In addition, the detection of the horizon using a genetic algorithm can improve the speed and accuracy by using the calculation result of the previous frame (not using the image of the previous frame). Specifically, the calculation is performed using the chromosome information of all individuals of the genetic algorithm including the optimal solution calculated in the previous frame as the initial chromosome of the initial individual population (evolutionary moving image processing). In particular, when processing is performed on all frames, or when processing is performed on frames at short intervals, the movement of the horizon in the image is slight. Convergence up to is extremely short. Therefore, the CPU of the in-vehicle computer can calculate at a speed sufficient for real-time processing, and can continue to detect the horizon.

  According to the second embodiment, the detection accuracy of the moving object can be improved by removing the detection frame that does not overlap the horizon at a predetermined rate. Further, by using a genetic algorithm for detecting the horizon, the horizon can be detected with high accuracy.

  INDUSTRIAL APPLICABILITY The present invention can be used as an inexpensive and highly safe moving object detection apparatus and moving object detection method capable of detecting a moving object with high accuracy and high speed from a moving image of a monocular camera.

DESCRIPTION OF SYMBOLS 10 ... Automobile, 12 ... Car-mounted camera, 14 ... Braking system, 100 ... Moving object detection apparatus, 110 ... Input part, 112 ... Frame acquisition part, 114 ... Pre-processing part, 116 ... Background image generation part, 118 ... Moving object Detection unit, 120 ... storage unit, 122 ... filter unit, 124 ... output unit, 150 ... primary candidate frame, 152 ... detection frame of previous frame, 154 ... secondary candidate frame, 156 ... integrated detection frame, 160 ... detection frame

Claims (6)

An input unit for inputting a moving image;
A frame acquisition unit that continuously extracts a plurality of frame images from a moving image;
A moving object detection unit for setting a detection frame on the moving object;
A storage unit for storing the set detection frame,
The moving object detector is
Extract foreground pixels from the current frame using the background subtraction method,
A frame surrounding the foreground pixels is a primary candidate frame,
Obtain the detection frame of the previous frame from the storage unit,
Among the primary candidate frames, those that overlap the detection frame of the previous frame by a predetermined amount or more are defined as secondary candidate frames,
Grouping the secondary candidate frames that overlap with the detection frame of the same previous frame,
Among the grouped secondary candidate frames, those that overlap more than a predetermined amount in the y-axis direction are integrated detection frames, and those that do not overlap more than a predetermined amount in the y-axis direction are separated detection frames. apparatus.   The moving object detection device according to claim 1, further comprising a background image generation unit configured to synthesize the background image and the current frame image to update a background image.   The filter unit for removing a detection frame that does not satisfy a predetermined area or aspect ratio with respect to one or a plurality of the detection frames detected in a current frame. Moving object detection device. Detect the horizon in the current frame,
4. The apparatus according to claim 1, further comprising a filter unit that removes a detection frame that does not overlap the horizon at a predetermined ratio with respect to one or a plurality of the detection frames detected in a current frame. The moving object detection device according to item.   The moving object detection apparatus according to claim 4, wherein the filter unit uses a genetic algorithm to detect the horizon. Enter a video,
Extract multiple frame images from a moving image,
When the frame image to be processed is the current frame and the previous frame image is the previous frame,
Extract foreground pixels from the current frame using the background subtraction method,
A frame surrounding the foreground pixels is a primary candidate frame,
Obtain a detection frame that surrounds the moving object in the previous frame,
Among the primary candidate frames, those that overlap the detection frame of the previous frame by a predetermined amount or more are defined as secondary candidate frames,
Grouping the secondary candidate frames that overlap with the detection frame of the same previous frame,
Among the grouped secondary candidate frames, those that overlap more than a predetermined amount in the y-axis direction are integrated detection frames, and those that do not overlap more than a predetermined amount in the y-axis direction are separated detection frames. Method.