JP2007188294A - Method for detecting moving object candidate by image processing, moving object detection method for detecting moving object from moving object candidate, moving object detection apparatus, and moving object detection program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To quickly and highly accurately detect an intruder by a monitoring camera. <P>SOLUTION: This method includes: movement suppressing processing for detecting a pixel whose moving vector is different from that of peripheral pixels out of a difference image of a photographed image and detecting a small area including the pixel out of small areas obtained by dividing the photographed image, color suppressing processing for preparing large areas each of which is larger than each of small areas obtained by dividing the photographed image and detecting a large area in which the value of the brightness or color of a pixel is different from that of peripheral areas as a moving object candidate area in accordance with the absolute value of the moving vector of a small area included in the large area, and moving object determination processing for determining whether the moving object exists in the moving object candidate area or not by pattern recognition. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a method for detecting a moving object candidate by image processing for extracting a moving object candidate from a captured image, a moving object detection method for detecting a moving object from a moving object candidate, a moving object detection device, and a moving object detection program. More specifically, the present invention relates to a method of extracting a moving object candidate from an image captured by a moving camera and detecting the moving object at high speed and with high accuracy.

  As an intruder monitoring method, an intruder detection method using image processing is known. Since an intruder may move at high speed, it is necessary to accurately detect the intruder moving at high speed.

  Here, for the purpose of high-speed motion recognition of a person, techniques related to a motion detection method using MHI (Motion History Images) focusing on the motion of the person have been proposed (Non-Patent Documents 1 and 2).

  In addition, as a method for automatically monitoring a wide area, a method of repeatedly detecting a moving object with a background difference or a frame difference while a monitoring camera is stationary in a certain direction for a certain period of time, or creating a background image in all directions in which the monitoring camera turns, A method of detecting the moving object by reading the turning value of the surveillance camera as a digital value or taking in the rotation instruction value of the camera by the surveillance staff and taking the difference from the background image created in advance in that direction is also proposed. (Non-patent Document 3).

Also, the previous image is compared with the current image, the amount of camera rotation is estimated from the movement change of the entire image, the previous image is moved by the amount of movement, and the change area obtained by the difference from the latest image is calculated. A method for obtaining a moving body has been proposed (Non-Patent Document 4).
Davis, WJ: Recognizing Movement using Motion Histograms, MIT Media Laboratory Perceptual Computing Section Technical Report, No. 487 (1998). Davis, WJ: Hierarchical Motion History Images for Recognizing Human Motion, IEEE Workshop on Detection and Recognition of Events in Video, pp.39-46 (2001). Kang S., Paik J., Koschan A., Abidi MA: "Real-time video tracking using PTZ cameras," Proc. Of SPIE 6th International.Conference on Quality Control by Artificial Vision, Vol. 5132, pp. 103-111 , Gatlinburg, TN, May 2003. Yusuke Takahashi, Toshio Kamei: Moving object tracking system using pan, tilt and zoom, Proceedings of the 17th Sensing Forum, Vol.17, pp.55-60, 2000.

The moving object candidate detection method using the techniques of Non-Patent Documents 1 and 2 (hereinafter referred to as MHI) is performed as follows. In MHI, the image is updated by Equation 1. Here, Ψ (x, y) is a preset background difference or inter-frame difference, etc., and 0 is output when the difference value at coordinates (x, y) is less than or equal to the threshold value T, and 1 otherwise. Function to output. Further, τ indicates a time stamp at the time of image acquisition, and a portion acquired before a specific delay time δ is set to zero.

Next, MHI is converted into a grayscale image of 255 gradations. At that time, by looking at the time stamp recorded in the MHI, a portion close to the current time is darkened (luminance value: low) and a delay portion is lightened (luminance value: high). Further, the gradient of the luminance value of the gray scale image is obtained by Equation 3 using a Sobel filter (Sobel operator) represented by Equation 2 for the gray scale image. From the vertical and horizontal gradients Fx and Fy, the movement direction of the pixel is obtained by Expression 4. Note that m = n = 1. The Sobel filter is a difference operator for calculating a partial differential in the xy direction more accurately using a neighborhood region whose size is determined.

  Next, in MHI, the motion direction is calculated only when the difference between the maximum value and the minimum value of the grayscale image is between the upper limit threshold value and the lower limit threshold value in the vicinity region of the pixel of interest. Furthermore, in MHI, using the API function floodfill from the outline of a person on a grayscale image, the process of integrating neighboring pixels lower than the luminance value of the pixel of interest is repeated, and then the region above a certain threshold is moved. Detected as a body candidate.

  However, MHI focuses on only the motion history between frames and detects moving object candidates. In this way, since MHI focuses only on movement, it is impossible to accurately detect a moving object candidate unless image processing is performed retroactively to the past several frames for processing of one frame. As a result, the amount of data to be processed has become enormous, and quick detection processing has not been possible.

  In addition, for example, when a plurality of surveillance cameras are installed in the premises of a power plant or a dam outlet, etc., it is necessary to monitor a wide area. There are many cases of continuous monitoring. That is, surveillance cameras often shoot while moving the shooting direction continuously up and down, left and right. In this way, even when a wide area is automatically monitored while panning or tilting the surveillance camera, if a moving body such as an intruder is detected and the detection can be made, the surveillance staff can be alerted. It is possible to greatly reduce the burden on the monitoring personnel who are monitoring a plurality of monitoring points on a multi-screen.

  Further, the present invention is not limited to the case where the surveillance camera is photographed while being panned or tilted in a state where the surveillance camera is fixed at a fixed position. Such a video camera is small and light, and the photographer can take a picture of a landscape or the like while moving freely or from a car window. In this specification, a camera that moves while the frame is fixed at a fixed position, that is, a surveillance camera that takes a picture while panning or tilting, and a camera that can take a picture while moving by hand are called a moving camera.

  However, MHI assumes that images are taken with a fixed camera, and when applied to a monitoring state where the camera moves, unintended areas are detected due to the movement history of the background and the moving object, and the history of both. There was a problem of being. Thus, in the state where the surveillance camera has moved, the intruder detection method using the conventional image processing cannot accurately detect the intruder.

  Also, if the moving object detection is repeated with background difference or frame difference while the surveillance camera is stationary in a certain direction for a certain period of time, there is a problem that the surveillance camera needs to be stationary for a certain period of time, so that the moving object passing outside the image range is missed appear. The technique described in Non-Patent Document 3 has a problem that it is necessary to update the entire background image in the turning direction every time a physical change or a change in sunshine occurs in the background. I can't say that.

  Further, in the technique described in Non-Patent Document 4, in order to estimate the rotation amount of the camera from continuous images, it is necessary to obtain the movement amount of the entire image while repeating local correlation processing or the like. This local correlation calculation requires a huge amount of calculation, and there is a problem that it is difficult to perform processing in real time.

  Therefore, the present invention provides a conventional technique for monitoring a moving object by a monitoring camera by suppressing a portion where the brightness and color values of the pixels of the monitoring object and the surrounding area differ from the surroundings in addition to the movement history of the monitoring object. A moving object detection method and a moving object detection apparatus for detecting a moving object from moving object candidates by using image processing capable of significantly reducing data processing and shortening detection time And a moving body detection program.

  In order to achieve such an object, the invention described in claim 1 detects pixels having different magnitudes and directions of motion vectors with respect to surrounding pixels from the difference image of the captured image, and includes the sub-region of the small region obtained by dividing the captured image. A motion suppression process for detecting a small region including pixels, a large region larger than the small region by dividing the captured image, and a large region according to the magnitude of the absolute value of the motion vector of the small region included in the large region. Color suppression processing is performed in which a large area in which the brightness or color value of the pixels in the area is different from the surrounding area is detected as a moving object candidate area.

  Therefore, paying attention to the fact that the moving body moves differently from the surroundings, the pixels that move differently from the surroundings than the pixels that move the same as the surroundings from the inter-frame difference image, that is, the motion vectors are the surrounding pixels. By detecting pixels that differ from the surrounding pixels, the surrounding pixels are suppressed by movement, and the moving object has a different brightness or color value from the surrounding pixels. The moving object candidate area is detected by performing a two-stage suppression of detecting the surrounding area by detecting a large area whose brightness or color value is different from that of the surrounding area, not a large area approximating to . That is, detection of the same movement and color portion as the surroundings is suppressed, and portions different from the surroundings are positively detected.

  According to a second aspect of the present invention, in the method for detecting a moving object candidate by the image processing according to the first aspect, the captured image is an image captured while moving the imaging means. According to a ninth aspect of the present invention, in the moving object detection apparatus according to the eighth aspect of the present invention, the captured image is an image captured while moving the imaging means. Therefore, the moving object candidate is detected for an image captured while the imaging unit is panned and tilted or an image that is captured while the imaging unit itself is moving horizontally and vertically, such as when the photographer is carrying around. be able to.

  According to a third aspect of the present invention, in the method for detecting a moving object candidate by the image processing according to the first or second aspect, the inter-frame difference image is a binary image of a difference between two frames. The invention according to claim 10 is the method for detecting a moving object candidate by image processing according to any of claims 8 or 9, wherein the difference image in the motion suppressing means is a binary image of a difference between two frames. It is supposed to be. Therefore, the amount of calculation is reduced by obtaining a difference image from an image only between two frames.

  According to a fourth aspect of the present invention, in the method for detecting a moving object candidate by the image processing according to the third aspect, the motion vector is an x-axis direction and a y-axis direction of an edge between two frames at each coordinate of the binary image. It is a vector whose component is a change in motion. Therefore, the movement amount of the edge of the binary image between the current frame and the immediately preceding frame is expressed as a vector.

  According to a fifth aspect of the present invention, in the method for detecting a moving object candidate by the image processing according to any one of the first to fourth aspects, the color suppression process ranks a large region according to the magnitude of the absolute value of the motion vector. In accordance with the ranking, a large area in which the brightness or color value of the pixels in the large area differs from the surrounding area is detected as a moving object candidate area. According to an eleventh aspect of the present invention, in the moving object detection apparatus using the image processing according to any one of the eighth to tenth aspects, the color suppression means ranks the large region according to the magnitude of the absolute value of the motion vector. In accordance with the ranking, a large area in which the brightness or color value of the pixels in the large area differs from the surrounding area is detected as a moving object candidate area. Therefore, color suppression processing is performed in descending order of the absolute value of the motion vector.

  The invention according to claim 6 is the method of detecting a moving object candidate by image processing according to any one of claims 1 to 5, wherein the color suppression processing is a middle region that is a region smaller than the large region in the large region. And compare the brightness or color values of the pixels with a plurality of neighboring areas having the same number of pixels as the middle area using a histogram, and the difference between the neighboring areas where the difference in pixel brightness or color value is the smallest Is set as a color difference, and a predetermined number of large areas are detected as the moving object candidate areas in descending order of the color difference. Therefore, the brightness value or color value of the pixel between the middle area set in the large area and the surrounding area is compared using a histogram, and the number of moving object candidate areas set in advance based on the difference in the histogram Only minutes are detected.

  The invention according to claim 7 is a moving object that determines whether or not a moving object is photographed in the moving object candidate area detected by the moving object candidate detection method according to any one of claims 1 to 6. By performing the determination process, the moving object is detected.

  The moving body determination device according to claim 8 detects pixels having different motion vector sizes and directions with respect to surrounding pixels from the difference image of the captured image, and the pixel in the small region obtained by dividing the captured image. And a motion suppression means for detecting a small area including a large area larger than the small area by dividing the captured image, and according to the magnitude of the absolute value of the motion vector of the small area included in the large area Color suppression means for detecting a large area in which the brightness or color value of a pixel in the pixel differs from the surrounding area, and a moving body for determining whether or not a moving body previously stored in a storage device is photographed in the large area A determination means is provided.

  The moving body detection program according to claim 12 detects pixels having different motion vector sizes and directions with respect to surrounding pixels from the difference image of the captured image, and the pixel in the small region obtained by dividing the captured image. A motion suppression means for detecting a small area including the coordinates and storing the coordinate position of the small area, and creating a large area larger than the small area by dividing the captured image, and the absolute value of the motion vector included in the small area is large The large areas are ranked in order, and among the ranked large areas, a large area in which the brightness or color value of a pixel in the large area differs from the surrounding area is detected, and the coordinate position of the large area is recorded. The computer functions as a color suppressing means and a moving body determining means for determining whether or not a moving body previously stored in a storage device is photographed in a large area by pattern recognition.

  Therefore, paying attention to the fact that the moving body moves differently from the surroundings, pixels that move differently from the surroundings than the pixels that move the same as the surroundings from the inter-frame difference image, that is, the motion vector is the surrounding pixels. By detecting pixels that differ from the surrounding pixels, the surrounding pixels are suppressed by movement, and the moving object has a different brightness or color value from the surrounding pixels. The moving object candidate area is detected by performing a two-stage suppression of detecting the surrounding area by detecting a large area whose brightness or color value is different from that of the surrounding area, not a large area approximating to . That is, detection of the same movement and color portion as the surroundings is suppressed, and portions different from the surroundings are positively detected. Furthermore, a mobile body determination process is performed to determine whether or not a mobile body registered in advance in the mobile body candidate area has been photographed.

  As described above, according to the moving object candidate detection method of the present invention, the amount of calculation can be reduced by obtaining the motion history by processing the minimum inter-frame difference. For this reason, it becomes possible to realize the detection of the moving object candidate region at high speed as a whole.

  Further, by performing two-stage suppression of not only suppression by movement but also suppression by lightness or color value, it is possible to detect moving object candidates more accurately than in the conventional technique that performs only suppression by motion. . In addition, since it is possible to prevent the movement history between the background and the moving object from occurring when the difference between multiple frames is changed, it is possible to detect the moving object candidate region with high accuracy.

  Further, according to the moving object detection method, the moving object detection device, and the moving object detection program according to the present invention, the moving object detection is further performed by determining whether or not the moving object is photographed in the moving object candidate region. It can be performed with high accuracy.

  In addition, the mobile body determination process can be performed in a state where mobile body candidate areas are detected with high accuracy, that is, in a state where the number of mobile body candidate areas is reduced as compared with the prior art. For this reason, since the area | region which performs a mobile body determination process within a picked-up image can be limited, the calculation amount of a mobile body determination process can be reduced, and a higher-speed mobile body detection can be performed.

  Furthermore, according to the method for detecting a moving object candidate according to claim 2 and the moving object detection device according to claim 9, it is possible to detect a moving object candidate even for an image taken by a moving camera. it can.

  Furthermore, according to the method for detecting a moving object candidate according to claim 3 and the moving object detection apparatus according to claim 10, it is possible to reduce the amount of calculation by considering only two frames, the current frame and the immediately preceding frame. And moving object candidates can be detected quickly.

  Furthermore, according to the method for detecting a moving object candidate described in claim 4, since the movement of the moving object can be detected only by processing the two frames of the current frame and the immediately preceding frame, the amount of calculation can be reduced. The moving object candidate can be detected quickly.

  Furthermore, according to the moving object candidate detection method according to claim 5 and the moving object detection apparatus according to claim 11, the amount of calculation is reduced by performing color suppression processing in the order of the absolute value of the motion vector. Therefore, the mobile object candidate can be detected quickly.

  Furthermore, according to the method for detecting a moving object candidate according to claim 6, in addition to the suppression by the movement of the moving object, by adding a two-stage suppression that detects a region having a brightness or color value different from that of the surrounding region. Thus, the detection accuracy of the moving object candidate region can be increased.

  Hereinafter, the configuration of the present invention will be described in detail based on embodiments shown in the drawings. The moving body detection method of the present invention detects pixels having different motion vector sizes and directions with respect to surrounding pixels in a difference image of a captured image, and includes a small region including the pixel in a small region obtained by dividing the captured image. And a motion suppression process that detects a large area larger than the small area by dividing the captured image, and according to the magnitude of the absolute value of the motion vector of the small area included in the large area, A color suppression process for detecting a large area whose brightness or color value is different from the surrounding area as a moving object candidate area, and whether or not there is a moving object stored in the storage device in advance in the moving object candidate area A moving body determination process that is determined by recognition is performed. In the moving object detection method of the present invention, the candidate area of the moving object is performed in two stages, narrowing down by movement and narrowing down by color.

  In the present embodiment, a method for detecting an intruder by a moving camera that performs monitoring while moving will be described. However, the monitoring target is not limited to a person, and may be a vehicle, an animal, or the like. For example, if the monitoring target is an animal, it can be used for monitoring the behavior of animals in a zoo. In particular, it is possible to detect at high speed a moving body showing a moving direction different from the surroundings, a moving body having a color different from the surroundings, or an object whose shape can be registered in advance, such as a person.

  First, the moving body detection apparatus of the present invention will be described. FIG. 1 shows an example of the configuration of the moving object detection apparatus. The moving body detection device includes an imaging unit 1 such as a moving camera, an input interface 2 for inputting a captured image, an output device 3 for displaying the captured image, an input device 4 such as a keyboard and a mouse, and arithmetic processing. A central processing unit (CPU) 5 to be performed, a main storage device 6 in which temporary work data and the like are recorded, and an auxiliary storage device 7 such as a hard disk in which counting results and the like are recorded are provided. The above hardware resources are electrically connected through the bus 8, for example. The captured image input from the imaging unit 1 may be input directly from the imaging unit 1 via the input interface 2 or may be input after being recorded on a recording medium or the like. The input interface 2 has a function of converting, for example, video input from a video camera or video recorded on a video tape into data that can be processed by a computer, and an auxiliary storage device using each frame constituting the video as image data. 6 has the function of recording. The output device 3 is a display, for example, and displays a captured image, a user interface screen, and the like.

  The auxiliary storage device 7 stores the moving object detection program of the present invention, and the computer functions as a moving object detection device by the CPU 5 reading and executing the program. In addition, the CPU 5 of the moving object detection device detects pixels having different motion vector sizes and directions with respect to surrounding pixels in the difference image of the captured image, and includes a small region including the pixels among the small regions obtained by dividing the captured image. And a motion suppression means 9 for detecting the image and dividing the captured image to create a large area larger than the small area, and according to the magnitude of the absolute value of the motion vector included in the small area, the brightness or color of the pixels in the large area The moving body in which the color suppressing means 10 for detecting a large area having a different value from the surrounding area and the moving body determining means 11 for determining whether a moving body exists in the large area by pattern recognition are read Run the detection program. In addition, the structure of the said mobile body detection apparatus is an example, and is not restricted to this. For example, although not illustrated, a buzzer, a lamp, or the like that can issue an alarm when a moving body is detected may be provided.

  In the processing described below, all processing involving temporary recording and storage of data is performed by the CPU 5 of the mobile unit detection device in the cache memory in the CPU 5, the main storage device 6, or the buffer area of the storage device. is there. In the following, when the storage device is simply used, it means any one of the cache memory in the CPU 5, the main storage device 6, and the auxiliary storage device 7. In addition, the determination result, the detection image of the mobile object, and the like may be recorded in an external storage device via the Internet, LAN, or the like without being limited to the auxiliary storage device 7 of the mobile object detection device. The threshold value and parameter are stored in advance in a storage device. The parameter setting screen may be displayed on the output device 3 so that registration is possible.

  Next, a moving object detection method using image processing according to the present invention will be described. In this embodiment, for example, as shown in FIG. 7, monitoring image setting (S1), motion suppression processing (S2), color suppression processing (S3), moving body determination processing (S4), and moving body detection image recording (S5) ) To detect intruders.

  The monitoring image setting (S1) is for capturing a moving image from a moving camera into a moving object detection apparatus. Note that the video data can be acquired by capturing the video data captured by the mobile camera in real time or by recording the video captured by the mobile camera on a video tape, etc. You may make it process with respect to the video data memorize | stored in the memory | storage device 7 and memorize | stored. Furthermore, the image captured in the computer in real time may be stored in the storage device at the same time. Note that the image processing described below is performed on each frame image of video data captured by a video camera. In this embodiment, the frame rate is set to, for example, 10 fps (sheets / second), but is not limited thereto.

  Next, a motion suppression process (S2) is performed on the set monitoring image. In the motion suppression process, a candidate area of a moving object is detected from a “motion” that is a feature of the moving object, from a monitoring image taken by a moving camera.

  In the present embodiment, since the camera constantly moves, the subject in the monitoring image photographed by the moving camera appears to move constantly. For example, when attention is paid to one subject (stationary object) in the monitoring image, this subject appears to move in a direction opposite to the moving direction of the camera. For example, when a tree is photographed in the center of the monitoring image, and when the photographing is performed while panning the moving camera leftward, the tree apparently moves to the right in the monitoring image. As described above, all the stationary objects in the monitoring image seem to move in the direction opposite to the moving direction of the camera. In other words, even a fixed background moves relatively. However, if there is a moving object in the subject, the object moves in the monitoring image regardless of the moving direction of the camera and the moving speed of the camera.

  In the present embodiment, extraction is performed as a moving object candidate by paying attention to such a portion that moves differently from the surrounding movement. Therefore, for example, even a person cannot detect a moving body that is in a fixed position and does not cause a slight movement while the camera is shooting while panning and tilting. This is because such a moving body is determined as a background. However, since the object to be monitored is an intruder or the like, for example, an intruder into a dangerous area, the intruder does not intentionally move. In addition, the moving body and the background may move in the same direction, but the background and the moving body can be detected because the absolute values of the motion vectors are different.

  Hereinafter, the motion suppression process will be described in detail. In the present embodiment, in the motion suppression process, first, an inter-frame difference is obtained. In the present embodiment, for example, a binary image is created depending on whether a change in luminance value between two frames exceeds a certain threshold. Furthermore, a pixel with motion is detected by comparing binary images of two frames. As a result, the overlap of the movement history between the background and the moving object that occurs when the difference between multiple frames is changed is prevented, and the amount of calculation is reduced. For example, if the time constraint is moderate, the number of frames is not necessarily limited to two frames, and the same processing may be performed. Further, an edge image obtained by a Sobel filter or the like may be used. If the number of frames to be processed is increased, the amount of calculation naturally increases.

  Next, in the motion suppression process, motion is detected. In the present embodiment, since a certain threshold is set for the luminance value of the pixel, it is preferable to convert the monitoring image to be processed in the motion suppression process into a grayscale image of 255 gradations in advance. This is because the amount of calculation can be reduced and a quick process can be performed. However, for example, if the time constraint is mild, the monitoring image does not necessarily have to be converted into a grayscale image.

In the present embodiment, the first original image I _t, the binary image Et of the inter-frame difference between two frames of the time t (current frame) and t-1 (previous frame) is set to be determined by Equation 5. Note that T is a threshold value and can be arbitrarily set in advance. For example, when T = 50, a binary image in which the value of the corresponding pixel is 1 when the luminance value is different by 50 or more is created.

Here, if the luminance value changes greatly between the previous coordinates at the same coordinates in the monitoring image, it means that the background has changed at those coordinates. In the present embodiment, since the image is taken by a moving camera, in principle, all coordinates do not indicate the same background as the previous frame. However, the change exceeding the threshold value of the luminance value set by the difference between successive frames mainly occurs in a moving object such as an edge of a stationary object or a pedestrian. Accordingly, attention is first focused on the portion of the binary image _Et (x, y) = 1.

Next, in the present embodiment, two binary images of the current frame and the previous frame are compared, and an area with motion is detected as a small area. It can be said that the portion of the binary image E _t (x, y) = 1 obtained by Expression 5 indicates the amount of movement of the edge of the background object, moving object, etc. in the original image It by the line segment width. That is, the area coordinates of the values of the binary image E _t is 1, the area value is 1 in the binary image E _t-1 coordinates, the position in the image of the edge portion of the background objects and mobile It can be said that it shows a change. FIGS. 2A and 2B _show two images, a binary image _Et (x, y) = 1 and a binary image _Et-1 (x, y) = 1 in the previous frame. The figure which overlap | superposed the binary image is shown. As shown in FIG. 2, when a region having a coordinate value of 1 is viewed locally, it can be seen that the line segments appear to move continuously in parallel. In addition, the arrow in FIG. 2 shows the moving direction of a moving body.

Then, in the present embodiments, n of a central pixel binary image boundary portion E _t side of E _t-1 in the binary image E _t and the previous frame (2n + 1) sets the local region of pixels The value of binary image _Et (x, y) = 1 included in each local region is obtained. In the present embodiment, it is checked whether there is an adjacent and binary image E _t and E _t-1 to jump one pixel, the E _t are adjacent, as a whole (2n + 1) × (2n + 1) The local region of the pixel is set (FIG. 3), but is not limited to this. Further, for example, n = 5, but is not limited to this. The setting of the local area is an example and is not limited to this. In the present embodiment, the center pixel is included in the binary image _Et side of the boundary portion, but the present invention is not limited to this. The purpose of setting the local region is to determine how much pixels of the binary image Et (x, y) = 1 are included in a certain region. FIGS. 3A and 3B show an example in which local regions N _i and P _i of n (2n + 1) pixels are set in a portion extending over _Et and _Et-1 . Note that _Ni is set to obtain a change in horizontal direction, and P _i is set to obtain a change in vertical direction. Reference numeral 12 denotes a central pixel. In the present embodiment, it is assumed that calculates the value of the binary image E _t included in each local region N _i and P _i according to Equation 6.

Then, the difference between the values of the binary image Et included in each N _i and P _i, divided by the number of pixels in the local region n (2n + 1), is set to ask the motion component gamma _i (Equation 7). In the present embodiment, for example, the motion changes in the x-axis direction and the y-axis direction at each coordinate (x, y) in the image are represented by Equations 8 and 9, respectively, as the motion components in the local region.

Furthermore, in this embodiment, in order to detect a portion where the motion is different from the coordinates (x, y) and the surrounding pixels, the entire image is represented by a small region Φ _i (l ₁ × l ₁ pixel: i = 1, 2, ., M × N / l ₁ ² ). M and N are the vertical and horizontal sizes of the image, and the average of the motion vectors in each small area Φ _i is A _i (Equation 10). Note that m is the number of pixels in the small region Φ. Further, the sum of the average motion vectors in the neighboring region centered on the small region Φ _i is expressed by Equation 11. The neighborhood is, for example, 8 neighborhoods, but is not limited to this, and may be 4 neighborhoods, 24 neighborhoods, and the like.

Here, it is assumed that the average A _i of motion vectors and the sum V _i thereof satisfy the formula 12 as a small region Φ _i ^′ . As a result, small regions having different movements in the center and the surroundings are extracted. Note that θ is an angle parameter and can be arbitrarily set. In this embodiment, for example, θ = 10 degrees, but the present invention is not limited to this.

According to Expression 12, only a portion having a different direction is detected as a small region Φ _i ^′ , not a portion where the center and the periphery are moving in the same direction. The coordinate position of the small area Φ _i ^′ is stored in the storage device, and the motion suppression process ends.

The motion suppression process described above is an example, and the present invention is not limited to this. It is only necessary that a small area including a moving object candidate can be extracted using an inter-frame difference image. For example, the extraction of the small region Φ _i ^′ is not limited to Formula 12.

  In this way, in the motion suppression process (S2), a small area including coordinates with motion is extracted using only the inter-frame difference between the currently processed frame and the immediately preceding frame, and a moving object candidate is extracted. As an area. Thereby, compared with the conventional technique which detects a movement area | region using many frames, the computational complexity is reduced significantly and high-speed processing is enabled. Furthermore, in the present invention, by performing color suppression processing in addition to motion suppression processing, detection accuracy is improved as compared with the conventional technology that performs only suppression by motion. Even if the color suppression process is performed, it is possible to realize a high-speed process as a whole as compared with the prior art due to a decrease in the calculation amount in the motion suppression process.

  For example, when the walking direction of the person is the same as the moving direction of the background caused by the pan of the camera, the surroundings and the person move in the same direction. You can think of not. However, when a person walks, even if they move in the same direction as a whole, the movements of their feet, arms, heads, shoulders, etc. move in various directions. It is possible to detect a person by detecting a simple movement.

Next, the color suppression process (S3) will be described. In the color suppression process, a moving object candidate is determined for the small area Φ _i ^′ detected by the motion suppression process in addition to suppression by color. In this embodiment, the color literally means a color value of a pixel, more specifically, an RGB color space, but is not limited thereto. For example, only the luminance value may be used. For example, if the monitoring image is a monochrome image, the processing amount is reduced by using only the luminance value for suppression, so that the processing can be performed more quickly. In addition, this is effective under conditions where time constraints are severe, and when it is applied to an image having a large number of frame rates. Further, any one of HSV (hue, saturation, brightness) may be used, or a combination thereof may be used. Further, the color value is not limited to the RGB color space, and other color spaces such as CIEL * a * b * may be used.

In the present embodiment, first, an image is divided into large regions Ω _k (l ₂ × l ₂ pixels). In the present embodiment, l ₂ is larger than l ₁ , that is, the large region Ω _k is larger than the small region Φ _k . For example, the small region Φ _i ^′ can be set to 10 × 10 pixels, and the large region Ω _k can be set to 50 × 50 pixels. In this case, the number of small areas for one large area is 25.

Furthermore, in the present embodiment, the absolute values | A ′ _i | of the motion vectors of the small region Φ ′ _i extracted by the motion suppression process included in the large region Ω _k are added, and the larger the added value B _k in descending order. The region Ω _k is ranked (Equation 13). This is because the color suppression process is preferentially executed on a region including a small region having a large motion among the small regions Φ ′ _i extracted by the motion suppression process.

Next, with respect to the large region Ω _k ranked by Equation 13, priority is given to a region having a color value different from the surroundings for the first large region Ω _k up to the _first t ₁ according to a preset parameter. It is supposed to be detected as a moving object candidate.

FIG. 4 shows the relationship between the large region Ω _k and the middle region Λ _k . In the present embodiment, the middle region Λ _k (l ₃ × l ₃ pixels) in the center of the large region Ω _{k is set} for the large region Ω _k . Note that l ₂ > l ₃ . Therefore, for example, the small region Φ _k can be set to 10 × 10 pixels, the middle region Λ _k can be set to 30 × 30 pixels, and the large region Ω _k can be set to 50 × 50 pixels.

In this embodiment, since each of the color values R, G, and B has a value of 0 to 255, each value is compared by a histogram, and a square error of the difference is obtained. Specifically, for large area Ωk to upper t1 th ordered, a region within the surrounding 8-neighborhood and area lambda _k in the large region Omega _k the central _{Λ j (j = 1 ...,} 8) RGB histogram difference F _k (j) is obtained (Equation 14), and the minimum value thereof is defined as the color difference G _{k from} the periphery in the large region Ω _k (Equation 15). Note that f _k is an RGB histogram in the middle region Λ _k , f _j is an RGB histogram in the vicinity 8 of Λ _k and near Λ _j , and H is the number of bins (number of sections) in the histogram.

The ordered large areas Ω _k up to the _first t ₁ are rearranged in descending order of the color difference G _k , and up to the second highest t ₂ as moving object candidates. Note that t ₁ ≧ t ₂ .

Here, t ₁ and t ₂ are parameters that can be arbitrarily set, and are previously stored in the storage device. In calculating the color difference G _k , it is necessary to repeatedly calculate {9l ₃ ² + 8H} per one large region Ω _k, so the number t ₁ of processing regions can be appropriately selected according to the ability of the computer to be used. That's fine.

  As described above, according to the moving object candidate detection method of the present invention, a moving object candidate area can be quickly and accurately obtained by adding suppression by movement and color to an image taken by a moving camera. Can be extracted. Further, the moving object can be detected by using an image determination technique on the extracted candidate area of the moving object.

  In this embodiment, for example, determination by OpenCV is performed as the moving object determination processing. Note that OpenCV (Intel Open Source Vision Library) is a library for computer vision provided by Intel laboratories. For the moving body determination process, for example, various known image determination processes may be appropriately selected according to the situation, and are not particularly limited. For example, a conventional template matching method or a determination method such as ARToolkit may be used instead of OpenCV.

  Hereinafter, the moving object determination process by OpenCV in this embodiment will be described. First, AdaBoost, which is a machine learning method used in OpenCV, will be described. AdaBoost is a method for reducing the importance of learning samples that are already correctly classified by intensively learning difficult-to-classify learning samples.

AdaBoost uses a method in which the convergence of learning is theoretically compensated for weighting of temporary learning. AdaBoost generates a temporary learning error with a finite number of learning temporary generations if the basic learning algorithm can always search for a learning temporary with a learning error ε _t of less than 1/2 for any weighted learning sample. can do.

The learning algorithm of AdaBoost is shown below. As a learning sample, a total of n samples obtained by collecting images of moving objects to be detected and other images are input (Formula 16). Note that y _i is a binary label of x _i , and y _i = 0, 1 indicates an image to be detected and others.

The initial value of the weight corresponding to n samples is set as in Equation 17. Note that m is the total number of y _i = 0, and l is the total number of y _i = 1.

Next, the following processing is performed for a preset number of times. First, weight normalization is performed (Equation 18), and a learning error for the classification h _j using each image feature j is obtained by the following equation (Equation 19).

Further, the classification h _t having the smallest ε _t is selected, and the weight is updated (Formula 20). In Equation 20, e _i is 0 when x _i is correctly classified and is 1 when x _i is incorrectly classified. Β is updated by Equation 21.

The learning result of AdaBoost generated by the learning count T generates a minimum margin from the boundary in the same manner as the support vector machine (SVM). AdaBoost is a hard margin. As a result of the above processing, the determination by AdaBoost is the sum of the products of the previous hypothesis h _t and the weight α _t , and if the value is ½ or more of the total weight, it is recognized as a detection target (Formula 22). The weight α _t is expressed by Equation 23.

  In person detection and face detection using OpenCV, various filters as shown in FIG. 5 are applied to the target image, and image characteristics obtained as a result are used to make a determination using AdaBoost. The white part is -1 and the black part is 1.

  In the present embodiment, a method called cascade is used for the determination of AdaBoost. FIG. 6 shows an example of cascade processing. As shown in FIG. 6, in cascade processing, a filter is used to determine AdaBoost with the lowest error rate by “determination 1”, and then the determination with the lowest error rate is performed by “determination 2”. It is said. Such a determination is repeated n times, and finally the part remaining as a candidate is determined as the part where the person is shown. That is, in the cascade system, the place to be viewed is limited every time one determination process is passed, so that the entire determination process can be speeded up.

  Note that the determination processing by AdaBoost provided by OpenCV described above is an example and is not limited to this. For example, as a moving object determination process, for example, an image of a moving object stored in advance as learning data in a support vector machine (SVM) may be used for the determination process. Thereby, it is possible to perform highly accurate moving body determination processing. In addition to the image of the moving body, an image other than the moving body (background image or the like) is also stored as learning data in the same manner, so that the accuracy of the determination result can be increased. Specifically, a moving body template is created in advance by a support vector machine using images obtained by converting two types of image sets other than the moving body image and the moving body using, for example, the Haar Weblet method. The moving body template is not limited to a template created by the support vector machine, and an existing template is stored in advance in a storage device, or is stored in an external storage device separately from the moving body detection device. It may be remembered.

  In the present embodiment, for example, if a moving body is detected based on the result of the moving body determination process, image recording is started as an image in which an intruder is detected (S5). Note that the intruder's image may be recorded continuously during photographing regardless of the presence or absence of detection. Alternatively, the monitoring image may be recorded while updating the monitoring image storage area at regular time intervals.

  The recording of the moving object detection image is performed in order to leave the recording of the moving object. In the present invention, since a warning can be issued simultaneously with detection of a moving body, recording of a moving body detection image is not always necessary. In addition, when the detected image is recorded, it is not always necessary to use the moving body detection apparatus of the present invention, and for example, it may be stored in a storage unit of other hardware through the Internet network and notified.

  Furthermore, processing such as issuing an alarm at the same time as detecting an intruder may be performed. Thus, the processing for one frame image of the moving image is completed, and the next frame is processed in the same manner except when the last frame of the monitoring image or the monitoring is interrupted.

  Processing performed by the moving object detection program of the present invention is shown in the flowcharts of FIGS. The moving body detection program of the present invention first acquires a captured image (S1), and then performs motion suppression processing.

FIG. 8 shows a flowchart of the motion suppression process. In the motion suppression process (S2), a difference between frames is obtained (S201), and motion is detected (S202). Next, a moving part different from the surrounding is detected as a small area Φ _i ^′ (S203), and the coordinate position of the small area Φ _i ^′ is recorded in a storage device (S204). With the above, the motion suppression process ends, and the color suppression process is performed.

FIG. 9 shows a flowchart of the color suppression process (S3). In the color suppression processing, the captured image is divided into large regions Ω _k (S301), and the large regions Ω _k are ranked in descending order of the motion obtained in the motion suppression processing (S302). Next, the minimum value of the color difference G _k with respect to the surroundings is _obtained in each large region Ω _k (S303). Furthermore Sort in descending order of color difference G _k (S304), it records the coordinate position of the large region Omega _k to the upper t ₂ in the storage device (S305). With the above, the color suppression process ends, and the moving body determination process is performed.

FIG. 10 shows a flowchart of the moving object determination process (S4). In the moving object determination process, first, a learning sample is input (S401), then a weight is set (S402), the number t of loop processing is set to 1 (S403), and the weight is first normalized. (S404) Further, a learning error is obtained for the classification hj (S405). As a result, the classification with the smallest learning error is selected (S406), and the weight is updated (S407). Finally, the number of processes is incremented by 1 (S408), and an end determination is made (S409). If the termination condition is satisfied (S409: Yes), the products of the previous hypothesis h _t and the weight α _t are summed, and if the value is ½ or more of the sum of the weights, it is recognized as a detection target. (S410). Thus, the moving object determination process ends.

  When a moving body is detected as a result of the moving body determination process, for example, a process of starting recording in the auxiliary storage device 6 as a monitoring image as a separately detected image or a process of issuing an alarm provided in the moving body detection apparatus is performed. Is. In principle, the process is performed for each frame of the captured image, and the process moves to the process of the next frame. Above, the process which the mobile body detection program of this invention performs is complete | finished.

  As described above, the present invention can detect a moving object with high accuracy and is a high-speed method compared to existing software. Moreover, since it is realized by software, it can be easily introduced into an existing image monitoring system. In the present invention, it is possible to perform a moving body detection process that does not require a calculation amount and operates at high speed on a general computer without using a special dedicated computer without using an existing monitoring camera without adding equipment. High-accuracy and high-speed intruder monitoring is realized without investing capital investment.

  The present invention is devised on the assumption that, for example, a person entering the monitoring area moves differently from the surrounding background, has a different color from the surroundings, and has a known shape. Therefore, for example, a person wearing clothes of the same color as the background may be difficult to detect. However, it can be said that such a person is difficult to find even when the monitoring person visually monitors the monitoring image.

  The above-described embodiment is an example of a preferred embodiment of the present invention, but is not limited to this, and various modifications can be made without departing from the gist of the present invention. For example, in the present embodiment, the mobile camera has been described, but it is needless to say that the mobile body detection method, apparatus, and program of the present invention can be applied to a surveillance camera whose frame position is fixed in advance. In this case, it is possible to detect the moving body with higher accuracy. In addition, since the amount of processing can be reduced by changing the parameter setting, higher speed processing can be realized.

  For example, in the above-described embodiment, each process is performed using a color image as an original image, but each process may be performed using a grayscale image as an original image. Similar processing can be performed by using lightness instead of color values. In this case, since the amount of calculation can be further reduced, high-speed processing can be realized.

  Further, each parameter such as a threshold value, for example, a standard deviation threshold value, may be appropriately adjusted in accordance with imaging conditions. Further, for example, the number of detected moving bodies may be counted and stored in an auxiliary storage device or the like. As a result, it is also possible to keep a record of the number of detected moving bodies.

  Although the case where the moving body detection method of the present invention is applied to the detection of an intruder has been described so far, the moving body detection method of the present invention can be used for various purposes. For example, at present, in biological observations for environmental measurement, an observer carries a video camera outdoors, shoots observation targets such as birds and animals, and visually detects target animals from recorded images. Yes. Such visual work can also be automated by using the moving body detection method of the present invention, and work efficiency can be improved.

  In the present embodiment, suppression by movement is first performed, and then suppression by color is performed. However, as a method for detecting a moving object candidate, first, the brightness or color value of a pixel differs from that of a surrounding pixel. It is also possible to extract a moving body candidate region by extracting a region and performing a suppression process by movement on the extracted region. In addition, the moving object candidate area extraction process based on the suppression process based on motion and the suppression process based on pixel brightness or color value may be processed in parallel, and the area detected by both processes may be used as the moving object candidate area. Is possible. In that case, whichever is selected as the moving pair candidate region may be selected with respect to the region detected by the suppression by movement and the region detected by the brightness or color value of the pixel. An optimal method may be selected depending on the monitoring environment and the subject to be photographed.

Example 1
The comparison experiment of the mobile body detection by MHI of this invention and a nonpatent literature 1 and 2 is shown. In this embodiment, a computer equipped with an Intel Pentium 4 (registered trademark) 2.24 GHz CPU was used. The experiment was performed with the image size of 360 × 240 pixels. In addition, in a present Example (Example 1), the result which did not perform a mobile body determination process is shown.

  The moving object detection by MHI uses the function for moving object detection by MHI provided by Motion Analysis and Object Tracking included in OpenCV. The inter-image difference Ψ is the inter-frame difference with a threshold T = 50, and the lower limit threshold of the detection area size is S = 50 (total length in length and width is 50 or more). In addition, MHI was calculated with three consecutive images, and the upper limit delay threshold of motion calculation was U = 0.1 and the lower limit threshold was D = 0.05. Others used parameter values provided as examples of arguments for MHI functions in OpenCV.

In this experiment, parameters were set as follows. The threshold T of the difference between frames T = 50, the size l ₁ of the small region Φ _i for calculating the motion is 10 pixels, and the angle parameter θ in Expression 12 is 10 degrees. In addition, the calculation of the suppression of the small region Φ _i is set to the vicinity of 24 surroundings. In the color suppression processing, the size of the large region Ω is l ₂ = 50 pixels, the size of Λ is l ₃ = 30 pixels, and the number of bins in the RGB histogram is H = 75. In addition, the number of color processing regions (t ₁ = 10) and the final number of detected moving objects are (t ₂ = 5). In the examples in the present specification, the above parameter values were used unless otherwise specified.

(Moving object detection by fixed camera)
First, an experiment was performed using an image obtained by photographing a pedestrian with a fixed camera. In the experiment, 200 consecutive images from a sample data set of pedestrian images were used. The experimental image is a scene of about 6.6 seconds (hereinafter referred to as scene 1) where a person crosses from right to left in front of a fixed camera. The experimental results are shown in FIG. FIG. 11A shows the processing result of MHI, and the moving object detection result is shown by a circle. A straight line in the circle indicates the moving direction of the region, and this is the same in FIGS. 12A, 13A, and 18A. In contrast, FIG. 11B shows a moving object detection result by the moving object detection program of the present invention. A rectangular frame is a moving object detection result. The lines and points in the image indicate the magnitude and direction of the movement, which is the same as in FIGS. 12B, 13B, 15, 16, 17, and 18B. To do. For scene 1, both the MHI and the moving object detection program of the present invention were able to detect the human area with a high accuracy of 96 to 97 percent.

(Detecting moving objects with a constant-velocity camera)
Next, a scene where a pedestrian was photographed for about 9.1 seconds while panning the moving camera installed to monitor the building entrance and the vicinity of the shutter at a constant speed (about 2 degrees / second) from right to left (Scene 2) ) Was used for comparative experiments. The experimental results are shown in FIG. The pedestrian approached the shutter from the lower left side of the screen and moved to the lower right side of the image. FIG. 12A shows the MHI processing result, and FIG. 12B shows the moving object detection result by the moving object detection program of the present invention. The human area was detected with an accuracy of 85% in the MHI process for a scene of about 9.1 seconds, and the moving object detection program of the present invention was able to detect the human area with an accuracy of 89%. In addition, as shown in FIG. 12A, MHI detects an average of seven backgrounds other than a person on average, and a maximum of 17 backgrounds per image, whereas the moving object detection program of the present invention uses a pedestrian. We succeeded in detecting the neighborhood. This indicates how many candidate regions for which the determination process is performed exist for one correct answer. That is, in the case where the same method is used for the determination of the moving body, the processing speed increases because the number of candidate areas detected in MHI is large. For scene 2, the moving object detection program of the present invention had higher detection accuracy than MHI.

(Detecting moving objects with a randomly moving camera)
Furthermore, an experiment was conducted to detect pedestrians from images taken while moving the mobile camera up and down randomly. As an experimental image, a scene (Scene 3) was used that was shot while the camera was tilting from the bottom to the top and from the middle to the bottom again for about 7 seconds when the pedestrian passed through the image from right to left. The experimental results are shown in FIG. FIG. 13A shows the moving object detection result by MHI, and FIG. 13B shows the moving object detection result by the moving object detection program of the present invention. As for the scene 3 as well, the MHI results detected 11 moving averages per image, and a maximum of 23 moving areas that were not detected from one image, whereas the present invention concentrated on the vicinity of the human area. Successfully detected. Thereby, a quick and highly accurate moving body determination process can be performed. In this experiment, the detection rate of the pedestrian part was about 70% in the moving object detection program of the present invention, while MHI was about 70%.

(Comparison of processing speed)
Table 1 shows the result of comparing the time taken for the processes of scenes 1 to 3 described above. For the moving object detection program of the present invention, two types of processing times were measured with t ₁ = 10 and 20 as the number of areas for executing the color suppression processing.

For example, the processing time required for the scene 3 was 5.2 seconds for MHI, 4.1 seconds when t ₁ = 10, and 6.0 seconds when t ₁ = 20 for the moving object detection program of the present invention. In this way, it has succeeded in reducing the processing speed itself for the detection process of the moving object candidate. Furthermore, the moving object detection program of the present invention has better detection accuracy of the moving object candidate itself as in the above-described example, compared to MHI. That is, since the number of candidate areas for one actual moving body is small, the same processing for the person determination processing further increases the difference in processing time, and the processing can be speeded up.

The processing speed of MHI is about 41 to 51 FPS, which is as fast as 51 FPS when using a fixed camera. However, if the camera moves, the speed decreases to 41 FPS. On the other hand, the speed of the moving object detection program of the present invention is stable, and the processing can be performed at 51 to 52 FPS when the number of color processing areas t ₁ = 10 and at 35 to 36 FPS when t ₁ = 20. . Since the change area is small in the fixed camera image, the MHI processing does not change with the moving object detection program of the present invention. However, when the camera moves, the change area in the image increases and the processing speed of the MHI decreases. On the other hand, the moving object detection program of the present invention has a constant processing speed regardless of the state in which the camera is fixed or moving.

(Processing speed and moving object detection rate)
In addition, since the processing speed depends on the ability of the computer, an experiment was conducted on the relationship between the calculation processing speed and the moving object detection rate. In the experiment, moving object detection was performed by changing the processing speed of the moving object detection program of the present invention in a scene 3 from 2 FPS to 30 FPS in a pseudo manner. FIG. 14 shows the analysis results of the processing speed and the moving object detection rate. FIG. 14 shows the human detection rate when the number of color processing regions t ₁ = 50 and the number of moving object detections t ₂ = 20. The detection rate when only the motion suppression processing is performed, and the motion suppression and color suppression are performed. The detection rate in the case of performing is shown. In addition, at a processing speed of 3FPS or more, human detection using motion and color showed a higher detection rate than human detection of motion alone. An example of moving object detection is shown in FIG.

(Results of carrying video camera)
An experiment was carried out to detect children using a 36.2-second recorded image (scene 4) of a photographer holding a video camera in his hand and moving the child outdoors while moving moderately. The experimental results are shown in FIG. FIG. 18A shows the moving object detection result by MHI, and FIG. 18B shows the moving object detection result by the moving object detection program of the present invention. In the moving object detection result by MHI, many movements of background trees are erroneously detected as moving objects. However, in the moving object detection program of the present invention, erroneous detection occurred only in 5% of images. Moving object detection by MHI occurred in 67% of all images, and an average of 6 errors per image and 19 errors at the maximum were detected. In this way, not only an image captured while being panned or tilted in a fixed state, but also an image captured while the photographer is holding it by hand, that is, an image captured while the camera itself is moving It was found that the moving object can be detected with higher accuracy than the method.

(Example 2)
(Combination with moving object judgment processing)
Next, FIG. 16 shows a moving object detection result by the moving object detection program of the present invention when the moving object determination process is performed. For the moving object determination, the person's upper determination process using AdaBoost provided by OpenCV was used. The front and back images of the person were learned with AdaBoost, and the person in the image was determined by cascade processing to increase the processing speed.

Table 2 shows a comparison between the processing time when performing person determination on the entire image and the processing time when performing person determination by narrowing down the search range to five locations using the moving object detection program of the present invention.

  The processing speed when performing person determination on the entire image is about 5.6 FPS, but in the moving object detection program of the present invention, the processing speed is about 23.3 FPS. When the moving body determination process is not performed (Example 1), other than the person is detected as shown in FIG. 12, for example, but when the moving body determination process is added (FIG. 16), only the person is detected. Succeeded. Thereby, it turned out that the moving body detection program of this invention is faster than searching the whole image by a cascade system.

  FIG. 17 shows a moving object detected by the moving object detection program of the present invention when the moving camera and the person are moving in the same direction. In this way, it has been found that it is possible to detect a person even if the movement direction of the camera and the person is the same direction.

  According to the present embodiment, it was found that the moving object detection program of the present invention can detect a moving object at a high speed from the surrounding background, and can detect a moving object from an image of a surveillance camera during panning or tilting. . In addition, the mobile object detection program of the present invention was compared with a conventional mobile object detection method, and a higher processing speed and a higher accuracy mobile object detection rate were obtained.

It is a figure which shows an example of a structure of a mobile body detection apparatus. Is a diagram showing an example of a case superimposed binary image E _t-1 of the previous frame and the (A) the binary image E _t. (B) is a diagram illustrating another example of superimposed binary image E _t-1 in the binary image E _t and the previous frame. (A) It is a figure which shows an example of the setting of a local region. (B) It is a figure which shows the other example of the setting of a local area | region. It is a figure which shows an example of the setting of large area | region (omega) and middle area (LAMBDA). It is a figure which shows an example of the filter used for a mobile body determination process. It is a figure which shows an example of a cascade process. It is a flowchart which shows an example of the mobile body detection process of this invention. It is a flowchart which shows an example of a motion suppression process. It is a flowchart which shows an example of a color suppression process. It is a flowchart which shows an example of a mobile body determination process. FIG. 6 is an image showing an example of (A) a moving object detection result by MHI and (B) a moving object detection result by the moving object detection program of the present invention when a moving object determination process for scene 1 is not performed. FIG. 6 is an image showing an example of (A) a moving object detection result by MHI and (B) a moving object detection result by the moving object detection program of the present invention when a moving object determination process for scene 2 is not performed. FIG. 7 is an image showing an example of (A) a moving object detection result by MHI when (B) a moving object detection result of the present invention is not performed when a moving object determination process for scene 3 is not performed. It is a graph which shows an example of the relationship between a processing speed and a mobile body detection rate. It is an image which shows an example of the moving body detection result by the moving body detection program of this invention at the time of performing a moving body determination process. It is an image which shows the other example of the moving body detection result by the moving body detection program of this invention at the time of performing a moving body determination process. It is an image which shows an example of the moving body detection result by the moving body detection program of this invention when a moving camera and a person move to the same direction. FIG. 4 is an image showing an example of (A) a moving object detection result by MHI when (B) a moving object detection result of the present invention is not performed when a moving object determination process is not performed on scene 4;

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Imaging means 9 Movement suppression means 10 Color suppression means 11 Moving body determination means

Claims (12)

  A motion suppression process for detecting pixels having different magnitudes and directions of motion vectors with respect to surrounding pixels from a difference image of the captured image, and detecting a small region including the pixels among the small regions obtained by dividing the captured image; The captured image is divided to create a large area larger than the small area, and according to the magnitude of the absolute value of the motion vector of the small area included in the large area, the brightness or color of the pixels in the large area A method for detecting a moving object candidate by image processing, wherein color suppression processing is performed to detect the large area whose value is different from a surrounding area as a moving object candidate area.   The method for detecting a moving object candidate by image processing according to claim 1, wherein the photographed image is an image photographed while moving an imaging unit.   The method of detecting a moving object candidate by image processing according to claim 1, wherein the difference image is a binary image of a difference between two frames.   4. The image processing according to claim 3, wherein the motion vector is a vector whose component is a motion change in an x-axis direction and a y-axis direction of an edge between two frames at each coordinate of the binary image. A method for detecting moving object candidates.   The color suppression processing ranks the large areas according to the magnitude of the absolute value of the motion vector, and according to the ranking, the brightness or color value of pixels in the large area differs from the surrounding areas. The method of detecting a moving object candidate by image processing according to any one of claims 1 to 4, wherein the area is detected as a moving object candidate area.   The color suppression processing creates a middle region that is smaller than the large region within the large region, and histograms the brightness or color values of the pixels with a plurality of neighboring regions having the same number of pixels as the middle region. The difference between the brightness value and the color value of the pixel is a color difference, and a predetermined number of large areas are detected as the moving object candidate areas in order of increasing color difference. The method for detecting a moving object candidate by image processing according to any one of claims 1 to 5.   Moving by performing a moving body determination process for determining whether or not a moving body is photographed in the moving body candidate area detected by the moving body candidate detection method according to claim 1. A moving body detection method for detecting a body.   A motion suppressing unit that detects pixels having different sizes and directions of motion vectors with respect to surrounding pixels from a difference image of the captured image, and detects a small region including the pixel among the small regions obtained by dividing the captured image; The captured image is divided to create a large area larger than the small area, and according to the magnitude of the absolute value of the motion vector of the small area included in the large area, the brightness or color of the pixels in the large area Color suppression means for detecting the large area whose value is different from the surrounding area, and mobile object determination means for determining whether or not the mobile object previously stored in the storage device is photographed in the large area. A moving object detection apparatus characterized by the above.   The moving body detection apparatus according to claim 8, wherein the captured image is an image captured while moving an imaging unit.   The moving object detection apparatus according to claim 8, wherein the difference image in the motion suppressing unit is a binary image of a difference between two frames.   The color suppression unit ranks the large areas according to the magnitude of the absolute value of the motion vector, and the brightness or color value of the pixels in the large area differs from the surrounding areas according to the ranking. 11. The moving body detection apparatus according to claim 8, wherein the area is detected as a moving body candidate area.   Pixels having different magnitudes and directions of motion vectors with respect to surrounding pixels are detected from the difference image of the captured image, and a small region including the pixel is detected from among the small regions obtained by dividing the captured image, and the coordinates of the small region are detected. A motion suppressing means for storing a position, and creating a large area larger than the small area by dividing the captured image, ranking the large areas in descending order of absolute values of the motion vectors included in the small area; Among the ranked large areas, color suppression means for detecting the large areas in which brightness or color values of pixels in the large areas are different from the surrounding areas, and recording the coordinate positions of the large areas; A moving body detection program for causing a computer to function as moving body determination means for determining whether or not a moving body previously stored in a storage device is photographed in the large area by pattern recognition.