JP4610005B2 - Intruding object detection apparatus, method and program by image processing - Google Patents

Description

  The present invention relates to an image processing method, apparatus, and program. More specifically, the present invention relates to an intruding object detection apparatus, method, and program suitable for use in a place where there is an illumination change or a background change by image processing.

  As an intruder monitoring method, an intrusion detection method using image processing is known. This utilizes the fact that a luminance change occurs in an image when a person enters the image taken by an ITV camera or the like, and a method of capturing the luminance change is widely used. In particular, as a method of capturing changes in brightness, a background difference that uses a difference between the brightness values of a background image and a captured image prepared in advance or a difference in pixel values between images of the same background that are slightly separated in time is detected. By doing so, there is an inter-frame difference for creating a difference image. In addition, when detecting nearby objects, inter-frame difference is effective, and when detecting distant objects, background difference is effective, so intrusion detection using a hybrid method that combines both methods. A scheme has been proposed. (See Patent Document 1).

  On the other hand, a method for detecting a person from an image taken by an ITV camera by comparison with a human body template has been proposed (see Non-Patent Document 1). Here, it is shown that it is possible to detect a person stably against a change in illuminance or a change in background.

JP 2002-176640 A A. Mohan, C. Papageorgiou and T. Poggio: Example-based Object Detection in Images by Components, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 23, No. 4,349-361, 2001

  However, in the intrusion detection method using the background difference, the interframe difference, and the hybrid method described above, there may be many false detections for detecting background changes such as outdoor illumination changes and vegetation movements. In addition, in order to detect a person or the like, it is necessary to perform a difference process on the entire image, and further determine a candidate area where a person is present by noise removal or labeling processing, and perform person determination processing on the candidate area. There is a problem that the amount increases and it is not suitable for real-time processing.

  In addition, in the method of detecting a person by using a still image and comparing it with a human body template, since the comparison process is performed while scanning the human body template over the still image, data processing of the entire image is required, and the amount of data processing is large. Become more. For this reason, it takes time for data processing, and it is not suitable for real-time processing such as issuing an alarm simultaneously with detection of an intruding object, and may not function as a monitoring device.

  Therefore, the present invention provides an intruding object detection apparatus and method, and a program that can reduce false detection and further reduce the processing time by reducing the amount of data processing and enable real-time processing. The purpose is to do.

  In order to achieve such an object, the intruding object detection method according to claim 1 includes a monitoring line setting process for setting at least one line segment area in a captured image, and a luminance value change on the line segment area. Line monitoring processing for extracting large pixels, and the captured image is divided into predetermined sections, and in order of sections in which it is estimated that an intruding object including pixels extracted by the line monitoring processing and having a high moving speed exists. According to the priority section determination process for assigning priorities and the priority of the sections set by the priority section determination process, the image captured in the section and the intruding object template prepared in advance are intruded into the section. Intrusion object determination processing for determining whether or not an object exists is performed.

  The intruding object detection device according to claim 8 is a monitor line setting unit that sets at least one line segment area in a captured image, and a pixel having a large change in luminance value of the pixel on the line segment area. The line monitoring means for extracting, the captured image is divided into fixed sections, and the priority order is determined in the order of the sections including the pixels extracted by the line monitoring processing and presumed that an intruding object having a high moving speed exists. An intruding object exists in the section by matching a pre-prepared intrusion object template with an image captured in the section in accordance with the priority section determining means to be attached and the priority order of the section set by the priority section determining means. An intruding object determination means for determining whether or not

  The intruding object detection program according to claim 15 is a monitoring line setting means for storing, in the main storage device, an address of a portion corresponding to a line segment area set on an output device for displaying a captured image, as a monitoring line; Line monitoring means for storing the address of a pixel having a large change in luminance value on the line segment area in a main storage device, and the captured image is divided into predetermined sections and stored in the main storage device, and stored by the line monitoring means. A priority partition determination unit for assigning priorities in the order of the partitions in which it is estimated that an intruding object having a high moving speed is included, and storing the priorities in the main storage device, In accordance with the priority order of the sections stored in the main storage device by the priority section determining means, the image photographed in the sections and the intruding object template read from the auxiliary storage device So that causes a computer to function as determining intruding object determination unit configured to determine whether the intruding object is present in said compartment by matching.

  Therefore, when an intruding object such as an unauthorized intruder passes through a line segment area (hereinafter referred to as a monitoring line), the luminance value of the pixel on the monitoring line changes. Pixels with a large change in luminance value are extracted, and a section including pixels with a large change in luminance value is extracted from the captured image divided in advance, and priorities are set for the sections. In accordance with the priorities, it is determined whether or not an intruding object exists in the section by matching an image photographed in the section with an intruding object template prepared in advance. In this way, the data processing range is limited to the pixels on the monitoring line, and only pixels with a large change in luminance value of the pixels on the monitoring line are extracted and set as processing targets, so that they are stationary. Except for objects and objects with little movement, only pixels that may be intruding objects are targeted for data processing. By reducing the data processing amount, the data processing time is greatly shortened, real-time processing (approximately 30 images / second) is possible, and detection of an intruding object with few false detections is realized.

  According to a second aspect of the present invention, in the intruding object detection method according to the first aspect, the monitoring line setting process sets the line segment region to a position and an inclination according to necessity in the captured image. According to a ninth aspect of the present invention, in the intruding object detection device according to the eighth aspect, the monitoring line setting process sets the line segment region to a position and an inclination according to necessity in the captured image. . Therefore, since it is necessary for the intruding object to pass through the monitoring line in order to detect the intruding object, a position where the intruding object can be detected is set in a portion where the intruding object is likely to pass as necessary. The monitoring line can be set by tilting.

  According to a third aspect of the present invention, in the intruding object detection method according to the first or second aspect, in the line monitoring process, for each line segment area, all pixels on the line segment area are arranged over an arbitrary recording time. A threshold value of a standard deviation set in advance with respect to an average value of luminance values over an arbitrary recording time obtained at a pixel position on the line segment area. A pixel having a luminance value exceeding the double range is detected, and the length of a line segment formed on the spatiotemporal image by the detected pixel is obtained. The invention according to claim 10 is the intruding object detection apparatus according to claim 8 or 9, wherein the line monitoring means sets all the pixels on the line segment area to an arbitrary recording time for each line segment area. A standard deviation set in advance with respect to an average value of luminance values over an arbitrary recording time obtained at a pixel position on the line segment region is created by creating a spatio-temporal image recorded over A pixel having a luminance value exceeding the range of the threshold times is detected, and the length of the line segment formed on the spatiotemporal image by the detected pixel is obtained. Therefore, all the pixels on the monitoring line are memorized over an arbitrary recording time to create a spatiotemporal image, and further, when the intruding object passes on the monitoring line, the change in the luminance value of the pixel occurs. And detecting a pixel whose luminance value at a pixel position on the monitoring line exceeds a preset threshold value, and determining a length of a line segment formed on the spatiotemporal image by the detected pixel. I want to ask. The amount of data processing is reduced only for pixel data processing on the monitoring line.

  According to a fourth aspect of the present invention, in the method for detecting an intruding object according to the third aspect, the line monitoring process is performed when the detected pixels are detected over a number of image frames equal to or greater than a preset threshold. It is extracted as a line segment formed on the spatial image. The invention according to claim 11 is the intruding object detection device according to claim 10, wherein the line monitoring means detects the detected pixels over a predetermined number of image frames or more. This is extracted as a line segment formed on the spatiotemporal image. Therefore, when a pixel whose luminance value has changed greatly is detected over the number of image frames equal to or greater than a preset threshold, it is detected as a line segment on the spatiotemporal image. This excludes the detection of a single change in luminance value.

  According to a fifth aspect of the present invention, in the method for detecting an intruding object according to the fourth aspect, the priority section determining process is performed by calculating the length of the line segment obtained by the line monitoring process and the slope of the line segment from the number of shooting frames. Then, the moving speed of the intruding object is estimated from the magnitude of the inclination. According to a twelfth aspect of the present invention, in the intruding object detection device according to the eleventh aspect, the priority section determining unit determines the line segment from the length of the line segment obtained by the line monitoring unit and the number of frames taken. The inclination is obtained, and the moving speed of the intruding object is estimated from the magnitude of the inclination. Therefore, the inclination of the line segment is obtained from the length of the line segment and the number of frames taken, and the moving speed of the intruding object is faster for the line segment with the larger inclination, and the moving speed of the intruding object is slower for the line segment with the smaller inclination. The moving speed of the intruding object is estimated only by data processing of pixels on the monitoring line.

  According to a sixth aspect of the present invention, in the intruding object detection method according to the fifth aspect of the invention, the priority section determining process sets a weight value for pixels on the line segment area in accordance with the slope of the line segment. The priority order is set for the section including pixels having a high weight value. The invention according to claim 13 is the intruding object detection device according to claim 12, wherein the priority section determining means assigns a weight value to the pixels on the line segment area according to the inclination of the line segment. The priority order is set for the section including pixels having a high weight value. Therefore, by setting a high weight value for the pixels on the monitoring line having a large inclination of the line segment and setting a high priority for performing the intruding object determination process for the section including the high weight value, A high priority is set for a section in which an intruding object having a high moving speed is estimated to exist.

  According to a seventh aspect of the present invention, in the intruding object detection method according to the sixth aspect, when it is determined that the intruding object does not exist in the priority section in the intruding object determination process, the priority section determining process For the priority partition, the weight value is updated to a low value to suppress the priority. According to a fourteenth aspect of the present invention, in the intruding object detection device according to the thirteenth aspect, the priority section determining means determines that no intruding object exists in the priority section of the intruding object determining means. In addition, for the priority section, the weight value is updated to a low value to suppress the priority order. Therefore, if it is determined that there is no intruding object in the previous frame, it is determined that there is no intruding object once by updating the weight value to a lower value to reduce the priority in the next frame processing. For the section, the intruding object determination process is not performed for a while.

  As described above, according to the intruding object detection method according to claim 1, the intruding object detection device according to claim 8, and the intruding object detection program according to claim 15, monitoring of the intruding object is limited to the monitoring line. Since the intruder is detected by image data processing on the monitoring line, the data processing amount can be reduced as compared with the conventional method for processing the entire image. In addition, since the matching range with the intruding object template is limited to the priority section on the monitoring image, the data processing amount can be reduced as compared with the method of performing the matching process on the entire captured image. That is, by reducing the amount of data processing, the data processing time can be greatly shortened and processing can be performed at higher speed. As a result, real-time processing (about 30 images / second) is possible, and an alarm or the like can be issued simultaneously with detection of an intruding object of the monitoring device. Moreover, since it can be realized by software using an existing surveillance camera, a new monitoring facility is not required, and the capital investment cost can be suppressed. Further, false detection can be reduced by performing template matching.

  In addition, according to the intruding object detection method according to claim 2 and the intruding object detection device according to claim 9, monitoring the position and inclination at which the intruding object can be detected in consideration of the movement of the intruding object in the captured image. Line can be set.

  In addition, according to the intruding object detection method according to claim 3 and the intruding object detection device according to claim 10, the intruding object is detected only by the pixel data processing on the line segment area. This eliminates the need for data processing of the entire photographed image, shortens the processing time, and enables real-time processing.

  Further, according to the intruding object detection method according to claim 4 and the intruding object detection apparatus according to claim 11, noise can be removed by removing detection of a single change in luminance value.

  According to the intruding object detection method of claim 5 and the intruding object detection device of claim 12, the moving speed of the intruding object can be estimated only by data processing of pixels on the monitoring line.

  According to the intruding object detection method according to claim 6 and the intruding object detection device according to claim 13, the intruding object determination process is performed from the intruding object having a high moving speed only by the data processing of the pixels on the monitoring line. Is possible. Compared with the method of processing the entire image, the data processing amount can be reduced.

  Further, according to the intruding object detection method according to claim 7 and the intruding object detection device according to claim 14, the priority can be suppressed by the result of the intruding object determination process in the previous frame. For the section determined that there is no intruding object, the intruding object determination processing can not be performed for a while, and efficient data processing is possible.

  Hereinafter, the configuration of the present invention will be described in detail based on embodiments shown in the drawings.

  FIG. 1 to FIG. 29 show an embodiment of an intruding object detection method and apparatus and program by image processing according to the present invention. This intruding object detection method first sets a monitoring line in the captured image, creates a spatio-temporal image for each monitoring line, detects pixels on the monitoring line that have large time-series changes in pixel luminance values, and detects them. The magnitude of the slope of a line segment (hereinafter referred to as a change path) formed on the spatio-temporal image by the obtained pixels is obtained. Further, a section in which a captured image is divided in advance, including pixels on a monitoring line that estimates the moving speed of an intruding object based on the obtained gradient of the change path and captures an intruding object having a high moving speed. In accordance with the priority order of the section, it is determined whether or not an intruding object exists in the section by matching an image photographed in the section with a prepared intruding object template. ing.

  As shown in FIG. 1, for example, as shown in FIG. 1, an intruding object detection method, apparatus, and program according to the present invention are used for monitoring line setting (S1), new captured image input (S2), line monitoring processing (S3), and priority section determination. The process includes a process (S4), an intruding object determination process (S5), and an intruding object detection image recording (S6). FIG. 21 shows an example of a hardware configuration diagram of the intruding object detection device.

  In the monitoring line setting (S1), the user arbitrarily sets a line segment area at a necessary position and inclination on an image using the output device 14 such as a display and the input interface 11 such as a mouse or a tablet. . It is assumed that a monitoring image is displayed on the output device 14. The setting of the monitoring line is set on the output device 14 by designating the start point and end point of the line segment with a mouse or the like, or dragging the line segment with the mouse or the like. The main storage device 13 stores the address on the monitoring image of the part corresponding to the designated line segment area as the monitoring line.

  In the present invention, an intruding object is detected when the intruding object passes on a monitoring line. For this reason, it is necessary to set the monitoring line with a position and an inclination so that the intruding object passes through a place where the entering or passing of the intruding object is prohibited or restricted. For this purpose, it is desirable that the monitoring line can be set not only at a specific position and inclination but also at an arbitrary position and inclination. In the present invention, since it is assumed that a monitoring line can be set on the monitoring image with a position and an inclination according to necessity, it can be set with a position and an inclination necessary for capturing the passage of an intruding object. Note that the number of monitoring lines set is not particularly limited, and a necessary number of monitoring lines can be set for one monitoring image.

  The monitoring lines are set on the monitoring image by using the input interface 11. For example, a plurality of monitoring lines may be set so as to be a passage through which an intruding object is likely to pass or to surround an intrusion prohibited area. It is effective. When the monitoring target is a person, it is effective to set a monitoring line in the restricted entry area or restricted entry area. Specifically, an intruder can be detected in advance by setting it in a passage that leads to a restricted area. Further, by setting a plurality of monitoring lines so as to surround the prohibited entry area, it is possible to detect an intruder into the area. The monitoring target is not limited to a person, and may be a vehicle, an animal, or the like. For example, if the monitoring target is a vehicle, it is possible to detect the passing of the vehicle by setting a monitoring line in the vehicle passage prohibition area. If the monitoring target is an animal, monitoring the behavior of animals in a zoo It can also be used for such as.

  FIG. 5 shows an example of monitoring line settings. In the present embodiment, for example, it is assumed that the monitoring of the approach of the person to the charging unit in the power company and the intrusion monitoring in the intrusion prohibited area, and from the monitoring image for monitoring the approach of the person to the charging unit in the power company, It is supposed to detect intruders. The charging unit in the power company premises is a place where access is restricted because it is dangerous for a person to approach carelessly. In such a case, by setting a monitoring line so as to surround the charging unit in the electric premises, it is possible to detect the approach of the person to the charging unit and to prevent an accident. In this embodiment, monitoring is performed outdoors, but the monitoring location is not limited to the outdoors, and indoor monitoring is also possible. For example, three monitoring lines can be set so as to surround the charging unit in the electric facility. As described above, the monitoring line can be set at an arbitrary position and inclination so as to surround a place where the entry of persons is restricted.

  In the newly-captured image input (S2), a monitoring image (for example, 720 × 480 pixels) captured by the monitoring camera is acquired at a TV rate (30 images / second). The imaging means is not limited to the monitoring camera, and may be an imaging device such as a video camera. In addition, the imaging unit may be installed separately from the present apparatus. For example, the captured image may be acquired through the Internet network or the like. Further, the number of frame rates is not limited to the TV rate. The captured image may be a color image or a gray scale image. Also, the resolution is not limited to the above resolution.

  The line monitoring process (S3) creates a spatiotemporal image for each monitoring line. FIG. 2 shows an example of a spatiotemporal image. The spatio-temporal image 2 refers to an image created by arranging all the pixels on the monitoring line 3 set on the monitoring image 1 in order of shooting frames (from 1 to n). Here, n indicates the number of frames of the monitoring image. The spatio-temporal image is updated, and after monitoring images for n frames are acquired, the space-time image is constantly updated thereafter. Note that n is a parameter and can be arbitrarily set. The recording time t (shooting time) can also be obtained from the number of frames and the number of frame rates.

  FIG. 3 and FIG. 4 show the results of measuring the change in luminance value at a certain pixel value on the monitoring image through which a person passes. FIG. 4 shows a time-series change in the luminance value (about 600 frames) of the x point portion 8 in the monitoring image shown in FIG. FIG. 4 shows how a person passes through about 400 to 500 frames. When a person passes, the luminance value changes greatly (standard deviation 11.4 over 400 to 500 frames), but in other frames, a slight change (standard luminance value for 100 images) changes before and after the average luminance value. The deviation was found to be 1.4). X Asphalt is usually photographed at the point portion 8. While the same background was photographed, the luminance value of the pixel hardly changed, but a large change in the luminance value was captured at a point where the intruding object (person) passed the x point. As a result, it has been found that it can be used for detecting an intruding object by capturing a time-series change in luminance value of each pixel in each frame.

  From the above, it has been found that the time-series change of the luminance value of the pixel on the monitoring line shows a large change when the intruding object passes (FIGS. 3 and 4). In other words, an intruding object can be detected by extracting a pixel in which the luminance value of each pixel on the monitoring line provided in the photographed image has changed significantly over time. In the present embodiment, in order to capture the time series change of the luminance value of the pixel, a pixel in which the luminance value of the pixel on the monitoring image is equal to or greater than a preset threshold value is detected. In the present embodiment, the luminance value of the pixel is used as a reference as a reference for capturing the time-series change. However, a pixel that is equal to or more than a preset threshold value based on the color information of each pixel may be extracted.

  Pixels whose luminance values have changed greatly (hereinafter referred to as change points) are extracted on the monitoring line. In the present embodiment, a sigma band using standard deviation is used to extract the change point. Specifically, the difference between the luminance value at each pixel position of the monitoring line of the monitoring image newly added to the spatiotemporal image and the average value of the luminance values obtained at each pixel position one frame before is one frame. The change point is extracted depending on whether or not it falls within a preset threshold value multiple of the standard deviation obtained with respect to the average value of the luminance values at each previous pixel position. As a result, it is judged how much the luminance value at each pixel position of the monitoring line of the newly added monitoring image has changed with respect to the luminance value at the same pixel position, and has changed greatly. To capture.

  Note that the threshold value set for the standard deviation can be arbitrarily set, and can be set as appropriate according to the intruding object and the background change. For example, if the threshold value is set low, a smaller change in luminance value can be detected, which is effective for indoor monitoring with little noise. In addition, it is possible to reduce erroneous detection of noise by previously obtaining a threshold value that can be detected most effectively according to an intruding object to be detected through experiments or the like. In this embodiment, extraction of change points is performed using a sigma band, but extraction of change points is not limited to this. For example, a pixel that indicates a luminance value that is more than a preset threshold value from the average luminance value may be extracted as a change point. Note that it is not always necessary to set the threshold value, and pixels exceeding the standard deviation (1σ) may be extracted without setting the threshold value.

  Further, in the present embodiment, only when the change points are detected over a plurality of frames (hereinafter, the reference frame number is referred to as the change determination image number), the continuous change points are extracted as change paths. It is said. The change point can be caused not only by the passage of an intruding object but also by a single change in luminance value due to noise (black noise, change in sunlight, heat, passage of insects and birds in front of the camera, etc.). For this reason, only a change in the brightness value is extracted by extracting as a change path only when change points are detected over the number of change determination images. As a result, noise is removed. Since the number of change determination images is a parameter that can be set arbitrarily, it can be set arbitrarily according to the shooting environment and the monitoring target.

  In addition, although continuous change points over a plurality of frames are extracted as change paths, even if change points are continuously captured due to the passage of an intruding object, they are not detected as change points due to the occurrence of noise. By indicating the luminance value of the range, there are cases where the continuation of change points is interrupted without being extracted as change points. In this case, since it may occur that the change path is not extracted, a process of extending the change point on the monitoring line to the surrounding pixels is performed. For example, in the present embodiment, processing is performed in which the pixels located on both sides of the pixel position detected as the change point on the monitor line are also recorded as the change point. In this way, the change point extension process prevents the change points from being interrupted and not being detected as a change path. The extended range can be arbitrarily set, and is not limited to the pixels on both sides. This extended process is not necessarily a necessary process. For example, it may be omitted when it is considered that the generation of noise is small.

  The space-time image is updated using First In First Out (hereinafter referred to as FIFO). The number n of frames for creating the spatiotemporal image can be arbitrarily set in advance. For example, with 60 images, a spatiotemporal image over 60 frames (2 seconds) can be recorded. Specifically, the update of the spatiotemporal image is performed by adding a pixel on the monitoring line of the latest captured image to the lowest row of the spatiotemporal image and deleting the uppermost row (the previous captured image). Is. As a result, data processing is always performed on the latest frame, and frames after a certain period of time are deleted, so that the amount of data processing can be reduced compared to a method of simultaneously processing a plurality of frames. Note that the spatio-temporal image update method is not limited to FIFO, and for example, a plurality of frames may be added and deleted simultaneously.

  Next, FIG. 6 shows the change points of the intruding object and the stationary object in the spatiotemporal image, which are shown as line segments (hereinafter referred to as change point recording lines). A spatiotemporal image 2 created from a monitoring line 3 on the monitoring image 1 is shown. Reference numeral 4a denotes a change point recording line for an intruding object, and change 5 denotes a change point recording line for a stationary object. For example, the change point recording line 4a represents a change point recording line when the intruding object 6 advances in the direction of the arrow. As the intruding object 6 passes on the monitoring line, the changing point also moves on the monitoring line, but since the spatiotemporal image is constantly updated, the changing point recording line that captures the continuation of the changing point. Draws a line segment having an inclination like the line segment 4a. Note that the monitoring line 3 is actually a line segment area, but in order to facilitate understanding, in FIG. 6 and FIG. 7, it is described with a certain width.

  Further, in the part where the intruding object does not pass on the monitoring line 3, since the stationary object such as the same background or building is photographed over the photographing time, the change in the luminance value of the pixel is minute. Originally, since the pixel value of a stationary object hardly changes, it is not extracted as a change point, but here, in order to facilitate understanding, the pixel value of a stationary object is extracted and an intruding object is displayed on a spatio-temporal image. Similarly, when the change point recording line 5 of the stationary object is shown, a perpendicular line drawn in parallel with the time series direction is drawn. That is, on the spatio-temporal image, it can be seen that the intruding object draws a line segment having an inclination and the stationary object draws a perpendicular line.

  FIG. 7 shows an example of change point recording lines of two intruding objects in the spatiotemporal image. The change point recording line 4b of the intruding object indicates that it has passed through the monitoring line over the recording time t. Further, the change point recording line 4c of the intruding object indicates that it has passed through the monitoring line in about half the recording time t. That is, since the intruding object having a high moving speed passes through the monitoring line in a short time, it is estimated that the intruding object 4c has a higher moving speed than the intruding object 4b. Here, in this embodiment, when the change point is recorded over the change determination image number c (for example, c = 4) or more, the change point is extracted as a change path. Focusing on the change path 7b and the change path 7c, the change path 7b is a line segment for four images of the change point recording line 4b, and the change path 7c is a line segment for four images of the change point recording line 4c. . Here, focusing on the angle between the monitoring line and the change path, it can be seen that the change path 7c has a larger inclination than the change path 7b (the angle between the monitoring line and the change path is smaller). As the moving speed of the intruding object changes faster, the inclination of the changing path increases (the angle between the monitoring line and the changing path decreases). The inclination becomes small (the angle between the monitoring line and the change path becomes large). The present invention uses this to estimate the moving speed of the intruding object from the magnitude of the gradient of the change path in the spatiotemporal image. In the present embodiment, at each pixel position on the monitoring line, a variable proportional to the angle between the monitoring line and the change path is obtained and recorded in a storage device or the like.

  In addition, if the intruding object passes perpendicular to the monitoring line, the change path draws a perpendicular line, so it may be impossible to detect it, but in reality the intruding object is an object with a certain width, so it is monitored. Passing completely perpendicular to the line (perpendicular for several frames) is unlikely. In the present invention, the monitoring line can be set so as to avoid such a situation. For example, if two monitoring lines are set side by side with different inclinations, an object passing vertically can be detected by either monitoring line. Furthermore, the data processing described above is all processing for one frame of a captured image, and in reality, the processing for the captured frame is repeated. For this reason, it is assumed that, in a certain frame, the intruding object happens to move vertically with respect to the monitoring line, and the angle between the monitoring line and the change path increases to detect that the moving speed is slow. In this case, no matter how fast the moving speed of the intruding object is, the image is taken over several frames before the intruding object passes. Considering this, even if the detection fails in the processing of one frame, if the detection processing is possible in any frame during the passage of the intruding object, it can serve as a monitoring device.

  In addition, since the intruding object is an object having a certain width, the change point detected when the intruding object passes through the monitoring line is usually detected continuously by a plurality of pixels on the monitoring line. It becomes. That is, even if the change path is obtained for all of the detected change points, the same object is detected if it is an adjacent pixel. Therefore, in this embodiment, the spatiotemporal image in which the change points are continuous is detected. The middle point at the 0th row point of the upper pixel and the middle point at the cth row (c = number of change determination images) are connected to be extracted as a change path. Thereby, one change path is extracted for one intruding object.

  The priority section determination process (S4) is for dividing the monitoring image into at least one section in advance and setting a priority order for performing the intruding object determination process (S5) for the divided sections. is there. If an intruding object determination process is performed on all pixels for which a change path can be detected on the monitoring line, the amount of data processing increases, and a rapid process cannot be performed. Therefore, the monitoring image is divided into at least one or more sections, priorities are set for the sections, and the intruding object determination process is performed on the entire monitoring image in order from the higher priority section. It is supposed to detect an intruding object without doing. As a result, the amount of data processing can be reduced and real-time processing is possible as compared with a method in which the entire monitoring image is processed.

  Note that the division of the monitoring image is set in advance. The monitoring image is divided, for example, by dividing the monitoring image into p × q sections in units of m pixels. For example, when the entire image is 100 × 100 pixels, if p = 10 and q = 10 are set, the image is classified into 100 sections in units of 100 pixels. The section is set by designating p and q on the output device 14 or the like and storing the section of the monitoring image in the main storage device 13. The section setting method is not limited to this. For example, the division may be set by inputting the number of divisions to be divided.

  In the present embodiment, the priority order is set such that the priority order is determined in the order in which the fast intruding objects are included in each of the divided sections so as not to miss the intruding objects having a high moving speed. This is because it is considered that an intruding object with a high moving speed is likely to be an intruding object to be monitored, and thus monitoring can be performed more effectively. Since the change path in the spatio-temporal image of the intruding object having a high moving speed has a large inclination, the priority order is determined according to the order of the sections including the change path having a large inclination.

  A weight coefficient (intrusion object determination process in the previous frame) set in the intrusion object determination process described later in a variable proportional to the angle between the monitoring line and the change path obtained in the line monitoring process (S3). The weight value for each pixel on the monitoring line is obtained by multiplying the coefficient determined by the result (1). Next, the weight value of the pixel that is maximum in the section is set as the section value of the section, and the priority order is set in the order of the section value in descending order. Of the sections obtained by dividing the entire monitoring image, only the section including the monitoring line is targeted for data processing. Specifically, a section containing at least one pixel address of the line segment area (monitor line) stored in the monitor line setting (S1) is extracted, and further, an intrusion with a fast moving speed is extracted. Priorities are set in descending order of movement speed for sections including objects, and intruding object determination processing is performed according to the priorities. For example, the priority order may be given to the sections in order of increasing moving speed of the intruding object, such as 1, 2, 3,..., But the method of assigning the priority order is not limited to this. Note that it is also possible to set a threshold for actually performing the intruding object determination process for the priority order. For example, when the threshold is set to 3, the next intruding object determination process is performed for three sections with priority levels up to 3. It is also possible to set the threshold value to 1 and perform the intruding object determination process only for the section showing the maximum section value in the processing for one frame of the captured image. Note that the intruding object determination process may be performed on all pixels that have detected a change path on the monitoring line without setting the priority order.

  Since data processing is limited to the section including the monitoring line, it is not necessary to perform data processing for the entire image, and the data processing amount can be reduced. Furthermore, since the intruding object determination process is performed only for the sections for which priority is set, the data processing amount can be greatly reduced. Note that the intruding object determination process may be performed for all pixels in which the change path is detected. The reason why the entire monitoring image is divided into sections in advance is that the sections are overlapped, so that it is possible to avoid duplication of data processing for the same pixel. Therefore, if processing for excluding overlapping pixel data processing is performed, or if overlapping portion data processing is performed a plurality of times, only the vicinity of the monitoring line in the captured image is divided into sections, and the sections are prioritized sections. You may make it set as.

  The intruding object determination process (S5) performs a process of determining a section and an intruding object template by template matching according to the priority set in the priority section determination process (S4). As a result of the intruding object determination process, a different value is set for the weighting factor depending on whether it is determined that there is no intruding object in the section and when it is determined that there is an intruding object. The weighting coefficient is used when obtaining a variable proportional to the gradient of the change path in the image processing of the next frame, and updates the value of the variable. As a result, the weight value for determining the priority order is updated. As a result, the priority order of the partitions is updated.

  In this embodiment, an image of an intruding object stored in advance as learning data in a support vector machine is used for determination processing. This makes it possible to perform highly accurate intruding object determination processing. Note that in addition to the image of the intruding object, an image other than the intruding object (background image or the like) is also stored as learning data in the same manner, so that the accuracy of the determination result can be improved. Specifically, an intruding object template is created in advance by a support vector machine using an image obtained by converting two types of image sets other than the intruding object image and the intruding object using, for example, the Haar weblet method. The Haar weblet is one of the simplest weblet functions. The intruding object template is not limited to a template created by a support vector machine, and an existing template is stored in advance in a storage device of a computer, or an external storage device separately from this device. It may be remembered.

  In the present embodiment, two types of image sets are prepared for the template matching: a person image and an image other than a person (an image obtained arbitrarily from a background image of a monitoring image in which no person is photographed). Images other than a person are not necessarily required, and can be determined if there is a person image template. Determination accuracy can be improved by storing images other than a person in the support vector machine in advance. In addition, when performing vehicle monitoring, template matching is performed in the same manner by preparing a vehicle image instead of a person image and preparing an animal image when monitoring an animal.

  In addition, the matching method uses, for example, a method (see Non-Patent Document 1) of detecting a person by comparing with an intruding object template from an image taken by a monitoring camera, but is not limited thereto. . For example, other template matching methods such as an area correlation method may be used.

  The intruding object detection image recording (S6) stores an image in which an intruding object is detected among the monitoring images. The monitoring image may be continuously recorded during shooting regardless of whether or not an intruding object is detected. Alternatively, the monitoring image may be recorded while updating the monitoring image storage area at regular time intervals. In addition, since the recording of the intruding object detection image is for recording the intruding object, in the present invention capable of real-time processing, an alarm can be issued simultaneously with the detection of the intruding object. Recording of is not necessary. In addition, when recording an image, it is not necessary to use this apparatus. For example, the image may be recorded by other hardware through the Internet network and notified.

  The present invention is capable of real-time processing (about 30 images / second), and realizes a high-speed method compared to human detection using existing software. For this reason, it can be used in situations where a monitoring processing speed such as monitoring of the charging unit is required. Moreover, since it is realized by software, it can be easily introduced into an existing image monitoring system. For example, about 30 or more monitoring cameras are currently installed per power plant. In the future, the number of surveillance cameras that use the Internet network is expected to increase. However, it is difficult to implement a dedicated safety surveillance device for determining the proximity of live parts for all of these cameras because it is too expensive. . In this regard, in the present invention, since realization is possible by software using existing hardware connected to an existing surveillance camera, real-time processing can be performed without investing in new capital investment. Monitoring is realized.

  Furthermore, according to the present invention, a method for extracting a part of an intruding object area from a change in the value of a pixel and collating with an intruding object template within the intruding object area candidate. Further, by setting a monitoring line in the captured image, the image processing range before the detection of the intruding object is limited to the monitoring line. For this reason, the data processing amount can be reduced as compared with the conventional method of processing the entire image. Therefore, the data processing time can be shortened, and an alarm or the like can be issued simultaneously with detecting an intruding object.

  The intruding object detection method described above can be implemented as an intruding object detection device using, for example, a known computer. For example, FIG. 21 shows the intruding object detection device 10. An input interface 11 to which a video image of a monitoring image is input, a hard disk as an external storage device 12 in which data such as original images and template images are recorded, and a main storage device in which temporary work data and the like are recorded 13, a RAM 14, an output device 14 for outputting an image in which an abnormality is detected, a central processing unit (CPU) 15, and the like. The hardware resources are electrically connected through the bus 16, for example. The input interface 11 converts, for example, a signal input from a surveillance camera or read from a medium such as a DVD on which video is recorded into data that can be processed by a computer, and each frame constituting the video is an image. It has a function of recording in the external storage device 12 as data. As such an input interface 11, for example, an existing NTSC-RGB converter, a frame grabber, an image capturing board for a personal computer, or the like may be used. The output device 14 is a display, for example, and displays an image in which an abnormality is detected, a user interface screen, and the like. Further, the intruding object detection program of the present invention is recorded in the external storage device 12, and the computer functions as the intruding object detection device 10 when the program is read and executed by the CPU 15.

  Next, an example of processing executed when the intruding object detection apparatus 10 monitors the intrusion of a person by the intruding object detection program according to the present embodiment will be described with reference to the flowcharts shown in FIGS. FIG. 22 shows a schematic flowchart of processing executed by the intruding object detection device 10. First, a captured image is captured from the surveillance camera (S101). Next, the monitoring line is set (S102). [No] is a program variable indicating a monitoring line. Since a plurality of monitoring lines [no] can be set, the following processing (S104 to S108) is repeated for the set number of monitoring lines (S103). In addition, the description of “no = 1, number of monitoring lines, 1” in S103 is performed by adding 1 to no from the first monitoring line [no] to the number of monitoring lines to be set (hereinafter referred to as increment). , Loop processing is performed. (The description of “a, b, c” means that the process is looped up to b while adding c to a. The same applies hereinafter.) Next, the luminance at each pixel position on the monitoring line. The standard deviation of the value is calculated (S104), the spatiotemporal image of the monitoring line is updated and the changing point is detected (S105), and the changing point is expanded (S106) in the spatiotemporal image of the monitoring line. If there is a change over the specified number of frames (the number of change determination images) (S107; Yes), the change length (slope) in the spatiotemporal image is detected (S108), and if there is no change (S107). No) The processing for detecting the length (inclination) of the change in the spatiotemporal image is not performed. A section including a pixel whose change has the maximum weight value is selected from all the monitoring lines (S109), matching with the human body template is performed, and person determination is performed (S110). The weighting coefficient is updated based on the person determination result (S111). The result of the person determination result is output (S112). When a person is detected, a process for issuing an alarm simultaneously with the output of the detection result is performed. Since the above processing is executed for all the frames of the monitoring image, it is executed while the monitoring image is captured. This process ends when the monitoring image ends.

  FIG. 23 shows a detailed flowchart of the monitoring line setting process (S102). The intruding object detection device 10 stores the addresses on the start point and end point images designated on the output device 14 (S102-2), and all the addresses on the image of the line segment area determined by the designation of the start point and end point. Is stored in the main memory (S102-3). The above processing (from S102-2 to S102-3) is repeated for the number of monitoring lines to be set (S102-1). This completes the monitoring line setting process.

  After the monitoring line setting process is completed, a line monitoring process is performed (S103 to S108). FIG. 24 shows a flowchart showing a calculation process of an average value and a standard deviation among the flowcharts detailing the line monitoring process. In this process, first, the average value counter a is set to 0 (S104-2). Then, the process of adding the luminance value at the horizontal pixel position x to the average value counter a (S104-4) is repeated for the recording time t (vertical pixel direction) (S104-3). The average value counter a obtained as a result is divided by the recording time t to calculate the average luminance at the horizontal pixel position x (S104-5). Next, 0 is set to the standard deviation counter s (S104-6). The process of adding the square value of the difference between the luminance value and the average value at each pixel position to the standard deviation counter s (S104-8) is repeated for the recording time t (104-7). The standard deviation counter s obtained as a result is divided by the number of data (t-1) and the square root is calculated to calculate the standard deviation (S104-9). The above processing (from S104-2 to S104-9) is repeated for obtaining standard deviation at all pixel positions on the monitoring line (S104-1). This completes the average value and standard deviation calculation processing in the line monitoring processing.

  FIG. 25 is a flowchart showing the spatiotemporal image update process (S105) in the detailed flowchart of the line monitoring process. In this process, the latest captured image is acquired, and a process (FIFO) for deleting the image on which the most time has elapsed on the spatiotemporal image is performed. First, the luminance value at the horizontal pixel position x of the monitoring line is set to -1 row in the vertical pixel direction of the spatiotemporal image (S105-2). If the deviation between the newly added luminance value and the standard deviation from the average of each pixel is within α times which is a preset threshold (S105-3; Yes), each pixel of the recording image DIF 0 is substituted into the flag area corresponding to (S105-4). Hereinafter, the flag area refers to an area for storing flag data in the data structure. If it exceeds α times (S105-3; No), 1 is substituted into the flag area corresponding to each pixel of the recording image DIF (S105-5), and the point is changed. The above process (from S105-2 to S105-5) is repeated at each pixel position on the monitoring line (S105-1). Next, the process of shifting the spatiotemporal image by one line upward in the vertical pixel direction (S105-7) is repeated for all the pixels of the spatiotemporal image (S105-6). The spatio-temporal image update processing is thus completed.

  In the extension process of the change point in the spatio-temporal image of the monitoring line of this embodiment, the line segment is expanded by adding the line segment of the recording image DIF by one coordinate to the x coordinate, and this is expanded into the working array. Newly recorded as BW. Furthermore, a portion that continuously changes over an arbitrary designated time (number of change determination images c) in the work array BW (a portion in which the flag area of the work array BW is 1 continuously) is detected. Specifically, it is determined whether or not the portion where the flag area of the work array BW is 1 can be continuously detected from the 0th line to the cth line of the work array BW. A series of change points over line c is detected as a change path, and an inclination detection process in the spatiotemporal image shown when the change path is detected is performed.

  FIG. 26 shows a flowchart showing the extension process (S106) of the changing point in the spatiotemporal image of the monitoring line, out of the detailed flowcharts of the line monitoring process. In this process, first, the flag area of the work array BW is initialized (S106-1). Next, if the flag area of the recording image DIF is 1 (S106-4; Yes), the change point area in the spatiotemporal image is expanded (S106-5). The above processing (from S106-4 to S106-5) is repeated for all the pixels of the spatiotemporal image (S106-2, S106-3). This completes the process of extending the changing point in the spatiotemporal image of the monitoring line.

  FIG. 27 shows a flowchart showing a process (S107) for determining a change over the number c of change determination images among the detailed flowcharts of the line monitoring process. It is determined whether or not there is a continuous change from the 0th line to the change determination image number c in the spatiotemporal image (from S107-1 to S107-2), and there is a portion where the change continues between the change determination image number c lines. If (S107-4; Yes), the process A (FIG. 28) proceeds to the process B (FIG. 28) if there is no portion where the change between the change determination image count c lines continues (S107-4; No). . A process of repeating this operation for all the pixels on the monitoring line (S107-3) is performed. Above, the process which determines the change over the change determination image number c is complete | finished.

  FIG. 28 is a flowchart showing the inclination detection process (S108) in the spatiotemporal image, out of the detailed flowcharts of the line monitoring process. In this process, a continuous change from the 0th row to the change determination image number c in the spatiotemporal image is detected as a change path (S108-1). Next, a process of counting the number of changed paths (S108-2) is performed, and serial numbers are assigned to the changed paths (S108-3). Next, the midpoint of line 0 and the midpoint of line t are obtained (S108-4, S108-5). Further, the process of calculating the length of the change path given the number (S108-6) and the process of measuring the path length (S108-7) are performed. This completes the tilt detection process in the spatiotemporal image.

  FIG. 29 shows a flowchart of the priority section determination process (S109), the person determination process (S110), and the weight update process (S111). The x coordinate in the monitoring image of the pixel of the monitoring line is substituted for x and the y coordinate in the monitoring image of the pixel of the monitoring line is substituted for y (S109-2), and the length of the change path is the maximum value at each pixel position. The process which sets the pixel which shows (S109-3) is performed. The above processing (S109-2 to S109-3) is looped over all the pixels of the monitoring line (S109-1). Next, if the weight value is 0 or more (S109-5; Yes) and the length of the change path is 0 (S109-6; Yes), the value obtained by subtracting the parameter γ from the weight value ω is set as the weight value ω. (However, the minimum value is 0) (S109-7). If the length of the change path is not 0 (S109-6; No), a value obtained by multiplying the weight value ω by the weight coefficient β and the length of the change path is substituted (however, the maximum value of ω is 2). ) Process (S109-7) is performed. A process (S109-4) for repeating the above process (from S109-5 to S109-9) is performed. Further, the value of the weight coefficient β is initialized (from S109-10 to 109-11), and the coordinates on the monitoring image where the weight value ω is maximum are detected (S109-12). Template matching is performed between the section including the coordinates and the person template image (S110). As a result, if the person is a person, 1.0 is assigned to the weighting coefficient β, and if the person is not a person, 0.5 is assigned to the weighting coefficient β (S111-1, S-111-2). Yes. The processing for one frame of the monitoring image executed by the intruding object detection device 10 by the intruding object detection program according to the present embodiment is completed as described above, and the processing proceeds to the next frame. When a person is determined, the result of the determination result is output (S112), and a process for issuing an alarm simultaneously with the output of the detection result is performed.

  The present invention is capable of real-time processing (about 30 images / second), and is a high-speed method compared to human detection using existing software. For this reason, it can be used in situations where a monitoring processing speed such as monitoring of the charging unit is required. Moreover, since it is realized by software, it can be easily introduced into an existing image monitoring system.

  Also, by limiting the monitoring area to the monitoring line, the data processing range of the image is limited, and the processing speed is increased. Furthermore, even in the intruding object determination processing, which requires a large amount of data processing and takes a long time, by first narrowing down the sections where there is a high possibility that an intruding object exists, by performing pattern recognition only on the sections where there is a high possibility, The amount of data processing is reduced. As described above, the processing speed is increased by reducing the data processing amount, and real-time processing is possible.

  The above-described embodiment is an example of a preferred embodiment of the present invention, but is not limited thereto, and various modifications can be made without departing from the gist of the present invention. For example, in the above-described embodiment, each process is performed using a color image as an original image, but each process may be performed using a grayscale image as an original image. Further, each parameter such as a threshold value, for example, a standard deviation threshold value, may be appropriately adjusted in accordance with imaging conditions. Further, for example, the number of intruding objects detected through the monitoring line may be counted and stored in the external storage device 12. Thereby, it is also possible to record the number of detected intruding objects.

(Example)
A sigma band using standard deviation was used for extraction of change points. First, with respect to the luminance value of each pixel on the monitoring line, an average value and a standard deviation value with respect to the height of the spatiotemporal image were obtained (Formula 1). Next, when the spatio-temporal image is updated using a new captured image, the above-described standard deviation σ is used to determine how much the luminance value of the pixel to be newly added to the lower row of the spatio-temporal image deviates from the average. Judgment was made using a sigma band (Formula 2).

Here, k represents the k-th new input image, and M _i ^k−1 and I _i ^k represent the average value and standard deviation of the luminance values of n images up to the previous image. Judgment was made based on whether or not the difference between the new input value of each pixel i and the average value of the previous image was within a certain magnification α of the standard deviation. It is assumed that 1 is set in the parameter when the standard magnification is equal to or larger than the constant magnification α times, and 0 is set in the parameter otherwise. In this embodiment, α = 4. As a result, a pixel having a luminance value exceeding the sigma band in the range of four times the standard deviation was extracted as a change point.

  Next, when a pixel having a parameter of 1 is detected as a change point and the change points are detected continuously for the number of change determination images, that is, when a continuous portion (change path) of the change points is detected, a monitoring line is detected. It was decided that an approaching object had appeared. In this embodiment, the number of change determination images is set to 4 images (c = 4) so that a person can be detected at a processing speed of about 7.5 images or more per second. As a result, it was found that noise can be removed.

Next, the variable λ _i ^k proportional to the moving speed of the intruding object was obtained by the following method. FIG. 8 shows an example of a spatiotemporal image. The inclination angle of the change path created by the intruding object in the spatiotemporal image was θ _i, and the moving speed of the intruding object was measured as follows using the spatiotemporal image. The variable λ _i ^k is proportional to the slope θ _i of the line segment in the spatiotemporal image at each pixel i of the monitoring line, and is obtained by the following equation. Here, c is the number of change determination images, and c ′ is the number of change determination images c and the movement trajectory length (≧ c) in the image. β _i ^k−1 is a weighting factor determined in the intruding object determination process one frame before, and a different weighting factor is set when the intruding object is detected and when it is not detected. The weight coefficient is determined by the intruding object determination process.

Next, the weight value ω _i ^k for each pixel on the monitoring line was obtained by Equation 4. Note that this is a variable proportional to the slope θ _i of the line segment in the spatiotemporal image at each pixel i of the λ _i ^k monitoring line. The weighting factor β is a weighting factor determined based on the determination result of the intruding object determination process, and different weighting factors are set depending on whether the intruding object is detected or not. (In this embodiment, 1.0 is set when it is determined to be a person, 0.5 is set when it is determined that it is not a person, and the upper limit value of the weight value ω _i ^k is set to 2.0.)

<Equation 4>
ω _i ^k = min (λ _i ^k , 2.0)

However, it is assumed that the weight value ω _i ^k is ^obtained using Equation 4 only when λ _i ^k ≠ 0 (change path length is not 0). For pixel values where λ _i ^k = 0 (change path length is 0), the change path cannot be detected, but the intruding object may have stopped on the spot. For this reason, it was determined by Equation 5. (In this embodiment, the minimum value of ω _i ^k is set to 0, and is set to 0 if the calculation result is 0 or less. If the weight value is set negative, the correct priority cannot be set. Note that γ is a parameter and can be set arbitrarily. This makes it possible to cope with a short stoppage of an intruding object. When the value of the parameter γ is increased, the weight value ω _i ^k is decreased, and therefore, it is difficult to be extracted as a priority section for a while. That is, it is possible to control how much the weight value of the portion where the changed path is not detected by the parameter γ is lowered and how much the priority order is lowered.

<Equation 5>
ω _i ^k = max (ω _i ^k−1 −γ, 0)

In this embodiment, the weight value of the pixel whose weight value ω _i ^k is the maximum value in each section obtained by Expression 4 or 5 is set as the section value of the section, and the section value Priorities were decided in descending order.

  FIG. 9 shows an image showing an example of a person image used for learning in the support vector machine. In this embodiment, template matching between a human body template created in advance by a support vector machine and a section Rmax is performed. (The section Rmax indicates a section having the maximum section value among the sections.) Weblet conversion was performed on the region of the section Rmax, and matching with the human body template was performed.

  In this embodiment, as an image monitoring mode, an experiment was performed in which real-time processing was performed to monitor a person approaching a charging unit on a premises using a monitoring camera image obtained by photographing the premises of an electric power company. The person was photographed in a size of about 30 × 25 pixels to 120 × 60 pixels, and assumed a walking speed of about 6 to 15 pixels per frame. This is a situation where a pedestrian is assumed at a TV rate (30 images / second). Therefore, in order to determine the passage of a person at a certain point, processing of about 7.5 images per second is required at the minimum. The experimental results are shown below.

  The experimental result by three types of picked-up images shown to Fig.10 (a)-(c) is shown. In the experiment on the monitoring image shown in FIG. 10A, the detection speed of one subject walking in the vicinity of the equipment was measured in order to examine the processing speed of the system. Further, in the experiment on the monitoring image shown in FIG. 10B, the human images collected in the first half (7 minutes) of the captured image 15 minutes were used to detect an unspecified number of people in the second half 8 minutes. Especially, in the power company premises, asphalt and flower beds may be the background in addition to the equipment, so we compared human detection when the backgrounds were different. Further, in the experiment on the monitoring image shown in FIG. 10C, in order to examine the robustness against the change in the situation, the proximity monitoring around the power facility was performed using the system learned in FIG. 10B. In this experiment, a monitoring image (720 × 480 pixels) was read from the monitoring camera with IEEE 1394, and the experiment was performed with a computer (CPU clock number: 2.4 GHz) equipped with a 1 GB memory. About each monitoring image, the processing speed of this system was measured by the person detection experiment of about 1 minute 4 second (1900 images).

  FIG. 10A is an image with little change in illuminance photographed on a cloudy day, and consists of a scene in which one subject repeats approaching and leaving the power facility. Table 1 shows various parameters used in the processing speed measurement experiment. As a person image used for learning of the person determination process, a 551 image cut out from a video image of about 1 minute in FIG. FIG. 9 shows an example of a person image used for learning. Moreover, 2838 images cut out from the unmanned background of FIG. 10A were used as background images other than the person images used for learning. Three monitoring lines were set so as to surround the power equipment fence in the image (FIG. 11).

  FIG. 11 shows an example of person detection, and FIG. 12 shows a time-series result of person detection for all scenes taken. The upper side of the graph in FIG. 12 indicates an image number in which the line monitoring process detects a change, and the lower side indicates an image number in which the person determination process detects a person. Also, in Table 2, if a person responds at least once before entering the monitoring line and then exits, the answer is correct. If there is no reaction before entering the monitoring line and then exits, no reaction occurs. The results of counting cases as false reactions are shown. (Hereinafter, this tabulated result is referred to as “experiment summary result.”) In the line monitoring process, there are three erroneous reactions, but in the person determination process, there were zero erroneous reactions. As a result of experiments, it was found that this system can detect people in real time. It was also found that the person can be detected with higher accuracy by using the learning person image and the background image extracted from the experimental image.

  Thus, it was found that a captured image (about 1900 images) of about 1 minute 4 seconds can be processed by real-time processing (30 images / second). Table 3 shows the correct rate of person detection for a captured image (approximately 1900 images) of approximately 1 minute 4 seconds. The line monitoring processing column in Table 3 shows the number of images in which a change by a person is correctly detected (correct answer), the number of images that show a false reaction other than a person (false reaction), and an image that does not show a response while passing a person. The number (unreacted) is shown. The person determination process column shows the result of performing the person determination process on the 470 images that have reacted to the line monitoring process. In the final result column, reaction ratios in all 1900 images including images in which no person appears are shown.

  Next, an unspecified number of person detection experiment results using another person's image will be shown. In addition, when monitoring outdoors, artificial parts such as asphalt and trees are often the background. Therefore, experimental results for two types of background are shown. In this experiment, 1486 images of 25 people cut out from the first half of the image of about 15 minutes in FIG. 10B and 2410 images of the pedestrian image database (924 people) of Massachusetts Institute of Technology A learning image was used. As a background image, the 9281 image of FIG. 10B in which no person is shown was used for machine learning. Thereafter, an unspecified number of person detection experiments appearing in the remaining 8 minutes of images were performed. Table 1 shows various parameters used in the experiment.

  The video image of 15 minutes in FIG. 10B is an image including many large illuminance changes such as the sun being hidden by clouds, and is a scene in which an unspecified number of people pass through the image. The experimental results against asphalt and trees are shown.

  FIG. 13 shows an example of monitoring results with asphalt in the background. Table 4 shows the accuracy rate of the latter half of the video image (14400 images). FIG. 14 shows the time series results of the proposed method for 14400 images. Table 5 shows the summary results of the experiment. As shown in Table 4, in the line monitoring process, an erroneous reaction of 155 images was observed due to the influence of people other than people. Both were false reactions that occurred when the sun was hidden and appearing in the clouds, but there were no false reactions in the person determination process. As a result, it has been found that an erroneous reaction of the line monitoring process due to a change in sunlight can be completely eliminated by the person determination process.

  Next, FIG. 15 shows an example of a monitoring result with a tree as a background, and Table 6 shows a correct answer rate and the like. FIG. 16 shows the time series results of the proposed method for 14400 images. Table 7 shows the summary results of the experiment. In the monitoring at the asphalt portion shown in Table 4, the variation in luminance value was small, so even in the shade, it reacted to changes in sunlight and caused a false reaction in the line monitoring process. The result shows that the variation in the brightness of the tree part is large, so the result of following the change in sunlight compared to the asphalt part was obtained, but any false reaction of the line monitoring process could be eliminated by the person judgment process . From the results of this experiment, we were able to confirm that human detection can be performed stably for unspecified persons different from the learning examples. It was also confirmed that stable human detection was possible even when the background was asphalt or trees.

  The learning result of the person image and the background image obtained by the above experiment is applied to the photographed image of FIG. The captured image in FIG. 10C includes a scene in which three subjects touch or climb the fence or stop in front of the fence. Three monitoring lines are set so as to surround the fence of the power equipment (FIG. 17), and Table 1 shows various parameters.

  FIG. 18 shows an example of the monitoring experiment result. Table 8 shows the accuracy rate for the 6000 image in FIG. Further, FIG. 19 shows time-series results, and Table 11 shows experimental summary results. If the person stops on the monitoring line, the person is regarded as the background, so the line monitoring process does not respond. For this reason, the line monitoring process is unresponsive to 91 images in which a person is still on the monitoring line. However, there is always a reaction when entering and exiting the monitoring line, and the non-detection becomes 0 when considering the change upon entering the monitoring line (Table 9). Since it is impossible to enter the monitoring line without moving, it was found that the objective of monitoring the approach to live parts could be achieved.

In addition, the line monitoring process misreacts outside the person due to the shaking of the fence, and an image was generated in which the neighborhood was misjudged as a person in the person judgment process. In order to investigate, the person determination process was performed every 10 pixels in the vicinity of the monitoring line. FIG. 20 shows the result. It can be seen from FIG. 20 that the background column is erroneously detected as a person in addition to the person. 20 is a place where one erroneous reaction shown in Table 9 has occurred. In this experiment, a system in which only a background image obtained from one unmanned image in FIG. 10B was learned was used. For this reason, a pillar that is not in the background image used for learning is misjudged as a person, but by learning more diverse background images such as fences and power facilities, misjudgment of person judgment is eliminated, with higher accuracy. Detection is considered possible.

  A person determination formula in the support vector machine in this embodiment is shown in Formula 6. Table 10 shows a learning result by a support vector machine using an image acquired from the monitoring image shown in FIG. The learning rate result for the linear support vector machine was 97.7%. Note that the processing speed of one person determination process by the linear support vector machine was about 0.013 seconds including the feature calculation. Note that the linear kernel K (x, y) = x · y (inner product) is obtained in advance by Equation 7, so that φ is not calculated but replaced with a kernel function calculation (referred to as kernel trick), and φ is The processing is speeded up without calculating directly. Table 11 shows the learning result by the support vector machine using the image acquired from the monitoring image of FIG.

<Equation 7>
φ = Σd _i x _i

Here, x _i is a case image (human body template image), x is a new input image (person candidate image), d _i is a parameter, and K is a kernel function for calculating the distance between the case image and the input image. Whether or not a person is permitted is determined by the sign of the person determination formula f (x). In this embodiment, the input image x is an image obtained by converting the image by Haar Weblet transform so that the input image x is not affected by slight changes in the input image.

  In this embodiment, if the detection on the monitoring line fails, there is a possibility that an intruding object is overlooked. In order to improve detection accuracy, it was found that operational measures such as providing double monitoring lines at important parts were necessary. Also, in order to improve the accuracy of intruding object determination, it is considered that operations such as preparing more images of intruding objects and background images and learning the support vector machine for each monitoring scene are effective.

It is a flowchart which shows the outline | summary of the intruding object detection method. It is a conceptual diagram explaining a spatiotemporal image. It is the image used for the experiment for measuring the time series change of the luminance value of a pixel. FIG. 4 is an example of a graph showing a time-series change in luminance value at a pixel position indicated by a cross in FIG. 3, where the vertical axis indicates the luminance value and the horizontal axis indicates the number of frames. It is an example of the image which set the three monitoring lines in the form surrounding the charge part in an electric power company premise. It is a conceptual diagram explaining the change point recording line of an intruding object and a stationary object in a spatiotemporal image. It is a conceptual diagram explaining the change point recording line and change path of an intruding object in a spatiotemporal image. It is an image showing the inclination of the change path created by the intruding object in the spatiotemporal image. It is an image which shows an example of a person picture used for learning with a support vector machine. It is an image which shows an example of a monitoring image, (a) is an example of the monitoring image which image | photographed the electric power equipment periphery from the upper part, (b) is an example of the monitoring image containing asphalt in the background, (c) is electric power It is an example of the monitoring image around equipment. It is an image which shows an example of a person's detection image. 12 is a graph illustrating an example of a time-series change in luminance value in the image of FIG. 11. It is an image which shows an example of the monitoring image which uses asphalt as a background. It is the graph which showed an example of the time-sequential change of the luminance value in the image of FIG. It is an image which shows an example of the monitoring image which uses a tree as a background. It is the graph which showed an example of the time-sequential change of the luminance value in the image of FIG. It is an image which shows an example of the detection image of a person in the present invention. It is an image which shows an example of the detection image of a person in the present invention. It is the graph which showed an example of the time-sequential change of the luminance value in the image of FIG. It is an image which shows an example of the detection image of a person in the present invention. It is a figure which shows an example of the hardware constitutions of an intruding object detection apparatus. It is a flowchart which shows the whole intruding object detection process. It is a flowchart which shows an example of the monitoring line setting process. It is a flowchart which shows an example of the calculation process of the average value in a line monitoring process, and a standard deviation. It is a flowchart which shows an example of the update of the spatio-temporal image of the monitoring line in a line monitoring process, and the detection process of a change point. It is a flowchart which shows an example of the expansion process of the change point in the spatiotemporal image of the monitoring line in a line monitoring process. It is a flowchart which shows an example of the change determination process over the fixed test | inspection time in a line monitoring process. It is a flowchart which shows an example of the inclination detection process in the spatiotemporal image in a line monitoring process. It is a flowchart of an example of a priority division determination process, a person determination process, and a weight update process.

Explanation of symbols

10 Intruder detection device

Claims (15)

A monitoring line setting process for setting at least one line segment area in the captured image, a line monitoring process for extracting pixels having a large luminance value change on the line segment area, and dividing the captured image into fixed sections In addition, a priority section determination process that prioritizes a section order that includes pixels extracted by the line monitoring process and that is estimated to have an intruding object with a high moving speed, and is set by the priority section determination process. In accordance with the priority order of the sections, an intruding object determination process is performed to determine whether or not an intruding object exists in the section by matching an image captured in the section with a prepared intruding object template. An intruding object detection method using image processing. 2. The method for detecting an intruding object by image processing according to claim 1, wherein in the monitoring line setting process, the line segment area is set to a position and an inclination in a captured image as required. The line monitoring process creates a spatio-temporal image in which all pixels on the line segment area are recorded over an arbitrary recording time for each line segment area, and the luminance at the pixel position on the line segment area is determined. A pixel having a luminance value that exceeds a threshold value times a preset standard deviation with respect to an average value of luminance values over an arbitrary recording time obtained at the pixel position is detected, and the detected pixel detects the pixel The method for detecting an intruding object by image processing according to claim 1 or 2, wherein the length of a line segment formed on the spatiotemporal image is obtained. The line monitoring process is characterized in that when the detected pixel is detected over a number of image frames equal to or greater than a preset threshold, it is extracted as a line segment formed on the spatiotemporal image. A method for detecting an intruding object by image processing according to claim 3. In the priority zone determination process, the slope of the line segment is obtained from the length of the line segment obtained by the line monitoring process and the number of shooting frames, and the moving speed of the intruding object is estimated from the magnitude of the slope. The method for detecting an intruding object by image processing according to claim 4. In the priority section determination process, a weight value is set for pixels on the line segment area according to the slope of the line segment, and a priority order is set for the section including pixels having a high weight value. The method for detecting an intruding object by image processing according to claim 5. In the priority zone determination process, when it is determined that no intruding object exists in the priority zone in the intrusion object determination process, the priority value is updated to a low value for the priority zone, and the priority The method of detecting an intruding object by image processing according to claim 6, wherein the order is suppressed. Monitoring line setting means for setting at least one line segment area in the photographed image, line monitoring means for extracting pixels having a large change in luminance value of the pixels on the line segment area, and a predetermined section of the photographed image A priority section determining means for assigning priorities in order of sections estimated to include an intruding object that includes pixels extracted by the line monitoring process and has a high moving speed; and the priority section determining means An intruding object determining means for determining whether or not an intruding object exists in the section by matching an image photographed in the section with a prepared intruding object template according to the set priority order of the section; An intruding object detection device comprising: The intruding object detection device according to claim 8, wherein the monitoring line setting unit sets the line segment region at a position and an inclination according to necessity in the captured image. The line monitoring means creates, for each line segment area, a spatiotemporal image in which all the pixels on the line segment area are recorded over an arbitrary recording time, and the luminance at the pixel position on the line segment area A pixel having a luminance value that exceeds a threshold value times a preset standard deviation with respect to an average value of luminance values over an arbitrary recording time obtained at the pixel position is detected, and the detected pixel detects the pixel The intruding object detection device according to claim 8 or 9, wherein a length of a line segment formed on the spatiotemporal image is obtained. The line monitoring means extracts the detected pixels as line segments formed on the spatiotemporal image when the detected pixels are detected over a number of image frames equal to or greater than a preset threshold value. Intrusion object detection device. The priority section determining means obtains the inclination of the line segment from the length of the line segment obtained by the line monitoring means and the number of captured frames, and estimates the moving speed of the intruding object from the magnitude of the inclination. The intruding object detection device according to claim 11. The priority section determining means sets a weight value for pixels on the line segment area according to the slope of the line segment, and sets a priority order for the section including pixels having a high weight value. The intruding object detection device according to claim 12. The priority section determining means updates the weight value to a low value for the priority section when the intruding object determination means determines that no intruding object exists in the priority section, and the priority section The intruding object detection device according to claim 13, which suppresses the rank. Monitoring line setting means for storing, in the main memory, the address of the portion corresponding to the line segment area set on the output device for displaying the photographed image, and for the pixel having a large change in luminance value on the line segment area; Line monitoring means for storing addresses in the main storage device, and the captured image is previously divided into predetermined sections and stored in the main storage device, including pixel addresses stored by the line monitoring means, and has a high moving speed. Priorities are determined in order of partitions in which an intruding object is presumed to exist, and priority partition determination means for storing the priorities in the main storage device; and the priority storage determination means stored in the main storage device by the priority partition determination means. According to the priority order, there is an intruding object in the section by matching the image captured in the section with the intruding object template read from the auxiliary storage device. Intruding object detecting program for causing a computer to function as determining intruding object determination means for determining whether or not.