JP4900506B2 - Moving object detection device using images - Google Patents

Description

  The present invention relates to a moving object detection apparatus using an image for detecting a moving object such as a person using images obtained by continuously capturing a target space with time.

Traditionally,
2. Description of the Related Art Conventionally, a technique for detecting the presence or absence of a person in a target space using an image obtained by imaging the target space is known. In addition, by comparing images between predetermined frames, it is detected whether or not the image content is different, and further, by detecting in which range the content difference occurs in the image, the movement in the image There has been proposed a technique for enlarging an image using an electronic zoom for a predetermined period when it is determined whether or not there is motion in the image (see, for example, Patent Document 1).

JP 2000-350080 JP

  The technique described in Patent Document 1 enlarges an image using an electronic zoom when a motion is detected in the image, and therefore enlarges a region having motion in the image without requiring manual operation. It becomes possible to confirm. However, since the period during which the image is magnified using the electronic zoom is a predetermined period and the movement within the image is not monitored during the period during which the image is magnified, the human There is a problem that even if they leave, they cannot be recognized. As a result, for example, when an image is displayed on a display device, once the image is enlarged, the enlarged image continues to be displayed regardless of the presence or absence of movement in the image. There will be many periods for displaying.

  The present invention has been made in view of the above-described reasons, and its purpose is to detect whether a moving object is present in an image even during the image enlargement, so that the image can be displayed even during the image enlargement. In addition, it is possible to provide a moving object detection apparatus using an image that enables the user to check the presence or absence of a moving object and that does not need to generate an enlarged image retroactively.

  According to the first aspect of the present invention, there is provided an imaging unit that continuously captures the target space with time, an image storage unit that stores a plurality of images captured by the imaging unit, and an image stored in the image storage unit. A first moving area extracting means for extracting a moving area in the image of interest by comparing the image of interest with a predetermined number of images in the past, and the first moving area in the image of interest; When the moving area is extracted by the extracting means, an enlarged image generating means for generating an enlarged image obtained by enlarging the partial area set to include the moving area to the size of one screen, and an image of interest among the enlarged images A second moving region extracting means for detecting the presence or absence of the moving region in the image of interest by comparing the image with a predetermined number of images in the past, and the second moving region extracting means includes: And detecting the presence or absence of the moving region from the difference image of the enlarged image of each two adjacent in the column.

  According to this configuration, since the presence / absence of a moving object in the image is detected even during the image enlargement period, the presence / absence of the moving object in the image can be confirmed even during the image enlargement. In addition, when the processing is handed over from the first moving region extracting unit that extracts the moving region of the 1 × image captured by the imaging unit to the second moving region extracting unit that detects the presence or absence of the moving region using the enlarged image. Since the presence / absence of the moving region in the enlarged image can be detected from the time when the second enlarged image is obtained, it is not necessary to generate the enlarged image retroactively. In addition, since the presence / absence of the moving area is detected using the difference image between two enlarged images that are adjacent in time series, the processing load is small compared to the case of using three or more images, and the presence / absence of the moving area is determined. The storage capacity for storing the enlarged image used for detection can also be reduced.

  According to a second aspect of the present invention, there is provided an imaging unit that continuously captures the target space with time, an image storage unit that stores a plurality of images captured by the imaging unit, and an image stored in the image storage unit. A first moving area extracting means for extracting a moving area in the image of interest by comparing the image of interest with a predetermined number of images in the past, and the first moving area in the image of interest; When the moving area is extracted by the extracting means, an enlarged image generating means for generating an enlarged image obtained by enlarging the partial area set to include the moving area to the size of one screen, and an image of interest among the enlarged images A second moving area extracting means for detecting the presence or absence of the moving area in the image of interest by comparing the image with a predetermined number of images in the past. A common area in each difference image between the enlarged image and the plurality of previously specified enlarged images is detected as a moving area, and after the processing is shifted from the first moving area extracting means, the plurality of prescribed images The period until the enlarged image is obtained is characterized in that the presence / absence of the moving region is detected from a difference image between two enlarged images adjacent in time series.

  According to this configuration, since the presence / absence of a moving object in the image is detected even during the image enlargement period, the presence / absence of the moving object in the image can be confirmed even during the image enlargement. In addition, when the processing is handed over from the first moving region extracting unit that extracts the moving region of the 1 × image captured by the imaging unit to the second moving region extracting unit that detects the presence or absence of the moving region using the enlarged image. Since the presence / absence of the moving region in the enlarged image can be detected from the time when the second enlarged image is obtained, it is not necessary to generate the enlarged image retroactively. Moreover, if a process for extracting a common area as a moving area in two or more difference images becomes applicable, the process proceeds to that process and the detection accuracy of the presence / absence of the moving area is increased. It is possible to achieve both reducing the processing load and ensuring the detection accuracy regarding the presence or absence of the moving region.

  According to a third aspect of the present invention, in the first or second aspect of the present invention, the image storage means uses, as a reference image, an image in which no moving area is detected before the enlarged image generating means generates the enlarged image. And the second moving area extracting means no longer detects the moving area and the first moving area extracting means returns to the process of extracting the moving area using the difference between the image of interest and the reference image. A region is extracted.

  According to this configuration, it is possible to extract the moving area using the image immediately after the processing is shifted from the second moving area extracting means to the first moving area extracting means, and the moving area exited from the enlarged image is 1 It is possible to extract without delay in the doubled image.

  According to a fourth aspect of the present invention, in any one of the first to third aspects of the present invention, an image having a detection area that coincides with a detection area for detecting a person as a moving area from the image captured by the imaging unit is used. A human sensor for detecting the presence or absence of a person in the detection area is added, and the first movement region extracting means and the second movement are combined by combining the results of the human detection by the image and the human detection by the human sensor. It is characterized by comprising a general control unit that alternatively selects processing with the region extraction means.

  According to this configuration, the first moving region extracting unit and the second moving region extracting unit determine not only the person using the image but also the outputs of other human sensors to determine the presence of the person. Therefore, the first moving area extracting means and the second moving area extracting means can be appropriately selected according to the situation.

  According to a fifth aspect of the present invention, in the fourth aspect of the invention, the integrated control unit detects that the second moving region extracting means does not detect the presence of a person and the human sensor detects the presence of the person. In some cases, the process is returned to the process of extracting the moving area by the first moving area extracting means.

  According to this configuration, even if the presence of a person is no longer detected in the enlarged image, if the person is present in the detection area, the presence of the person is detected by the human sensor. The processing can be quickly switched from the means to the first moving area extracting means.

  According to a sixth aspect of the present invention, in the fourth aspect of the invention, the comprehensive control unit can change a determination condition for selecting the first moving region extracting means and the second moving region extracting means, In the period in which at least one of the moving region extraction means and the human sensor detects the presence of a person, it is easier to detect the presence of the person than in the period in which both do not detect the presence of the person The determination condition is set so that

  According to a seventh aspect of the present invention, in the sixth aspect of the invention, the determination condition is the number of times that a person is detected within a specified time. After that, the number of times is set to be smaller than that before detection.

  In the invention of claim 8, in the invention of claim 6, the judgment condition is a threshold used for judging whether or not a person is present, and when the threshold is exceeded, it is judged that a person exists, The threshold is set lower after detection than before detection.

  According to the configuration of the invention of any one of claims 6 to 8, both detect the presence of a person during a period when at least one of the first moving region extraction means and the human sensor detects the presence of the person. Since the judgment condition is relaxed compared to the period when the person is not present, the judgment condition is made slightly stricter before detection of the presence of a person to prevent false detections, and the judgment condition for once confirming the existence of a person is relaxed to enlarge the enlarged image. It is possible to quickly shift to the processing to be used.

  According to a ninth aspect of the present invention, in the fourth aspect of the present invention, the comprehensive control unit focuses on a period in which both the first moving region extracting means and the human sensor do not detect the presence of a person. A common area in each difference image between the image and a plurality of past prescribed images is detected as a movement area, and whether or not the movement area is a person is determined, and the first movement area extraction means and the human sensor In the period during which at least one of them detects the presence of a person, the presence / absence of a moving region is detected from a difference image between two adjacent images in time series.

  According to this configuration, when the presence of a person is not detected, the first moving area extracting unit increases the accuracy of detecting the moving area using three or more images, and the first moving area extracting unit and the person are detected. After the presence of a person is detected on at least one of the sensation sensors, the presence of the moving area is simply detected using the difference image of the two images, while reducing the possibility of human false detection. Once a person is detected, it is possible to reduce the processing load and quickly determine whether a person exists.

  According to the configuration of the present invention, since the presence / absence of a moving object in the image is detected even during the image enlargement period, the presence / absence of the moving object in the image can be confirmed even during the image enlargement. There are advantages. Moreover, there is an advantage that it is not necessary to generate an enlarged image retroactively.

It is a block diagram which shows embodiment. FIG. 3 is an operation explanatory diagram illustrating the first embodiment. FIG. 6 is an operation explanatory diagram illustrating Embodiment 2. FIG. 10 is an operation explanatory diagram illustrating the third embodiment. FIG. 10 is an operation explanatory diagram illustrating the fourth embodiment. FIG. 10 is an operation explanatory diagram illustrating the fifth embodiment. It is operation | movement explanatory drawing same as the above. It is operation | movement explanatory drawing same as the above.

(Embodiment 1)
In the present embodiment, as shown in FIG. 1, a target space is imaged by an imaging unit 10 such as a TV camera, and a grayscale image captured by the imaging unit 10 (in the following description, a black and white image is assumed. An image or an image in a specific wavelength region can also be used) and is temporarily stored in the grayscale image storage unit 11. An object moving in the target space is detected by performing processing described later on the grayscale image stored in the grayscale image storage unit 11.

  The imaging means 10 is fixed at a fixed position, and a plurality of images are obtained in time series by continuously imaging the target space with time. Therefore, an object moved in the image is detected using a relationship between a plurality of images arranged in time series. A person is assumed as a moving object in the target space. Basically, when a moving object is detected in an image, whether the person is a person is determined based on the size.

  In order to extract an object existing in the target space from the image, the edge image generation unit 12 first generates an edge image. The edge image is an image obtained by extracting a portion where the density change is maximized in the grayscale image and thinning it so as to become a line image having a width of 1 pixel. Performs processing to emphasize density change. For example, if a commonly used Sobel filter is applied to a grayscale image, the density change can be enhanced. Therefore, if binarization is performed with an appropriate threshold after the Sobel filter is applied, an area near the maximum value of the density change is obtained. Can be extracted. If the binarized image is thinned to one pixel width and the discontinuous portions are made continuous by extending the end points, an edge image from which the contour of the object is extracted can be obtained. Other known techniques may be applied to the edge image generation technique. The edge image generated by the edge image generation unit 12 is stored in the edge image storage unit 13.

  Since a plurality of edge images arranged in time series are stored in the edge image storage unit 13, a region that is stationary and a region that is moving can be easily obtained by using the relationship between the plurality of edge images. Can be separated. Here, the time interval between the edge images does not need to be the time interval of each frame of the image obtained by the imaging means 10, and in time series when a person is moving at the assumed speed in the target space. The time interval is such that a change occurs in the image between two adjacent edge images. This is because if the time interval between two edge images is extremely short, a region that has moved substantially between the two edge images cannot be found.

  In order to extract a moving region using the edge image, the focused edge image and the two previous edge images are used. The time interval between these three edge images is constant, and hereinafter the time interval is assumed to be ΔT. Therefore, if the focused edge image is E (t), there are two past edge images, E (t−ΔT) and E (t−2ΔT), used for detecting the moving region. In order to extract a moving area from these three edge images E (t), E (t−ΔT), and E (t−2ΔT), first, the focused edge image E (t) and each past image After generating an image for which an exclusive OR (corresponding to a difference) is obtained for each of the edge images E (t−ΔT) and E (t−2ΔT), two images consisting of the exclusive OR An image obtained by calculating a logical product for each pixel is generated. A logical operation between the edge images E (t), E (t−ΔT), and E (t−2ΔT) is performed in the logical operation unit 14a.

  More specifically, when an image composed of exclusive OR for each pixel is generated for the edge image E (t) and the edge image E (t−ΔT), the edge image E (t) and the edge image E (t−ΔT) are generated. ) And the background edge that overlaps the moved area are extracted. Further, when an image composed of exclusive OR for each pixel is generated for the edge image E (t) and the edge image E (t−2ΔT), the edge image E (t) and the edge image E (t−2ΔT) are generated. The edge of the moved area and the edge of the background that overlaps the moved area are extracted. Generating an image consisting of the logical product of each pixel for two images consisting of exclusive OR is to generate an image consisting of a common part of both images, and as a result, the edge image E ( The area moved in t) and the background overlapping the area are extracted.

  As described above, a technique for extracting a moved region in an image using a relationship between three or more edge images arranged in time series is hereinafter referred to as silhouette matching. The region moved from the focused edge image E (t) can be extracted by silhouette matching. However, in this embodiment, the purpose is to extract a person and to distinguish insects and small animals from humans. is necessary. Therefore, for the movement region extracted by the logical operation unit 14a, the movement determination unit 15a determines whether the person is a person using the area, height dimension, aspect ratio, and the like occupied in the image. Since the logical operation unit 14a and the movement determination unit 15a extract the movement area from the edge image, they constitute a first movement area extraction unit.

  When the movement determination unit 15a determines that the person is a person, the three edge images used to determine the person are provided from the edge image storage unit 13 to the enlarged image generation unit 16. Further, the subsequent edge image is also given to the enlarged image generation unit 16. The enlarged image generation unit 16 sets a rectangular partial region including the region determined as the movement region by the movement determination unit 15a, and expands the partial region so that the partial region becomes the size of one screen. An electronic zoom function is used to enlarge the partial area. That is, the pixel position is converted so that the pixels included in the rectangular partial area are assigned to the pixel positions on one screen. Here, when setting the partial area, it is desirable to match the aspect ratio of the partial area with the aspect ratio of one screen. Further, the partial area is set to a size that includes the entire moving area included in the three edge images used to extract the moving area.

  In this embodiment, in principle, it is assumed that one moving area is included in one screen, but a plurality of moving areas may be detected in one screen. In such a case, the technology described below is adopted by limiting the number of partial areas to be expanded by setting priority order in the moving area and expanding only the moving area having the highest priority order. can do.

  As described above, when the movement determination unit 15a extracts the movement region, the partial image is set by the enlarged image generation unit 16, and the partial region is enlarged for the edge image, an enlarged image obtained by enlarging the partial region is obtained thereafter. It is stored in the enlarged image storage unit 17. The enlarged image stored in the enlarged image storage unit 17 is given to the logical operation unit 14b having the same function as the logical operation unit 14a. In the logical operation unit 14b, first, the enlarged image of the edge image in which the partial area is set is set as the first image of interest, and three enlarged images of this enlarged image and the past two enlarged images are used. Use, generate two images with pixel values of exclusive OR for each pixel of the enlarged image of interest and each of the past two enlarged images, and further calculate the logical product of each pixel from both of the generated images Generate an image as a value.

  As described above, when the partial area is set as an edge image, the edge images of the previous screen and the previous two screens are read from the edge image storage unit 13 with respect to the edge image in which the partial area is set. Since the partial area of the edge image is enlarged, it is possible to prepare as many enlarged images as necessary for silhouette matching when the partial area is set.

  The image generated by the logical operation unit 14b is given to the movement determination unit 15b, and the movement region in the enlarged image is extracted in the same manner as the movement determination unit 15a. However, the movement determination unit 15b does not set a partial area, and only determines whether the movement area extracted by the logical operation unit 14b is a person. Since the logical operation unit 14b and the movement determination unit 15b extract the movement area from the enlarged image, they constitute a second movement area extraction unit. If the presence / absence of a person is detected using an enlarged image, it can be detected even if the movement of the person is minute, so that it is easy to detect the presence / absence of a person.

  As described above, the logical operation unit 14a handles the edge image of the original size (hereinafter referred to as “whole image” in the sense of the image of the entire field of view of the imaging means 10), and the logical operation unit 14b handles the enlarged image. However, the logical operation unit 14a and the logical operation unit 14b are almost the same operation, and the movement determination unit 15a and the movement determination unit 15b are different in whether or not a partial area is set, but whether or not a person is a person. This determination is performed in the same procedure. Therefore, when the functions of the logical operation units 14a and 14b and the movement determination units 15a and 15b are realized by executing an appropriate program on the microcomputer, a flag for identifying whether to use the whole image or the enlarged image is set. If the processing of the movement determination units 15a and 15b is changed by the flag, the logical operation units 14a and 14b and the movement determination units 15a and 15b can be realized by one subroutine.

  When no person is detected in the enlarged image, the movement determining unit 15b instructs the first moving area extracting means to detect the moving area using the entire image. That is, the process is handed over from the second moving area extracting means to the first moving area extracting means. When processing is handed over from the second moving area extracting means to the first moving area extracting means, if the edge images for the past two screens are left in the edge image storage unit 13, one sheet restored from the enlarged image By using the entire image of the eye as the first image of interest and reading and using the past two edge images from the edge image storage unit 13, the moving region can be tracked.

  In the above-described example, the edge image storage unit 13 stores at least two edge images before the target edge image, and the enlarged image is generated by the electronic zoom technique. When returning to the whole image from the state where the moving area is extracted, the edge image stored in the edge image storage unit 13 is used as a past edge image, so that silhouette matching can be performed using the whole image. it can.

  However, in the case of changing the size of an image used for silhouette matching using the optical zoom in the imaging unit 10 (for example, in an enlarged image, an optical zoom is used instead of an electronic zoom in order to increase the resolution of a human face. When the moving region is restored from the state obtained from the enlarged image to the state obtained from the entire image, the number of images necessary for silhouette matching is not available in the edge image storage unit 13, and therefore after returning to the entire image. The silhouette matching process can be performed only for the third and subsequent images.

  As described above, when the moving area is detected in the entire image, an enlarged image can be obtained without delay. Therefore, when the moving area is tracked with the enlarged image, there is no time delay and the moving area is enlarged. The image can be reliably supplemented. On the other hand, when returning from the enlarged image to the entire image, the enlarged image is not necessary immediately after that, and no inconvenience arises even if silhouette matching is not performed on several images (two in this embodiment).

  As described above, silhouette matching is performed to detect the presence / absence of a moving region using an edge image. When the moving region is extracted, an enlarged image is generated by enlarging a partial region including the moving region to the size of one screen. When the moving area is extracted from the enlarged image, and the moving area is not extracted from the enlarged image, the moving image is extracted from the enlarged image by returning to the entire image.

  By the way, in this embodiment, it has the function to output the grayscale image of the range which has extracted the movement area | region to a display apparatus (not shown). For this reason, area switching means 18 is provided for reading out pixels in the area from which the movement areas are extracted by the movement determination sections 15a and 15b in the grayscale image captured by the imaging means 10 and stored in the grayscale image storage section 11. The area switching means 18 normally reads out the pixels of the entire area of the grayscale image captured by the imaging means 10 and switches to a state in which pixels corresponding to the partial area are read out when the partial area is set in the movement determination unit 15a. Further, when the movement determination unit 15b instructs to extract the moving area in the entire image, the state returns to the state of reading the pixels in the entire area of the grayscale image.

  The pixels read by the area switching unit 18 are output to the display device through the output unit 19 that converts the pixel group read from the grayscale image storage unit 11 into a video signal. By the way, when returning from the state where the moving area is extracted using the enlarged image to the state where the moving area is extracted from the entire image, silhouette matching may not be performed for the two images. Since the means 18 only changes the area for reading out pixels from the grayscale image storage unit 11, no dropout occurs in the image displayed on the display device.

  By the above operation, the image of each part changes as shown in FIG. In the figure, the time interval between images is ΔT, and nine images from time (T−4ΔT) to time (T + 4ΔT) are shown. Among these, an enlarged image is generated at three times of time (T−ΔT), time (T), and time (T + ΔT). In other words, at the time (T−ΔT), the time (T), and the time (T + ΔT), the moving area Dm is extracted using the second moving area extracting means, and the time (T−4ΔT) The first moving region extracting means is used until T-2ΔT) and from time (T + 2ΔT) to time (T + 4ΔT).

  Therefore, as shown in FIG. 2A, at the times (T−4ΔT), (T−3ΔT), and (T−2ΔT), the edge images E (T−4ΔT) and E (T −3ΔT) and E (T−2ΔT). By using these three edge images E (T−4ΔT), E (T−3ΔT), and E (T−2ΔT), the moving region Dm (change) Therefore, the partial region Dp including the moving region Dm is set at the time (T−2ΔT). Further, since silhouette matching is performed using the enlarged image at time (T-ΔT), two edge images are read from the edge image storage unit 13 retroactively from time (T-ΔT), and the result shown in FIG. As described above, two enlarged images in which the pixels corresponding to the partial region Dp are enlarged to the size of one screen are generated. That is, enlarged images L (T-3ΔT) and L (T-2ΔT) are generated from the edge images E (T-3ΔT) and E (T-2ΔT) at times (T-3ΔT) and (T-2ΔT), respectively. The

  As shown in FIG. 2C, since the silhouette matching performed by the first moving area extracting means is before enlargement before the time (T-2ΔT), the moving area Dm is relatively small in the screen, but the time (T− In the period from ΔT) to time (T + ΔT), the silhouette region matching is performed by the second moving region extraction means using the enlarged image, so the moving region Dm occupies a relatively large area in the screen.

  At time (T + ΔT), as shown in FIG. 2 (c), the moving area Dm cannot be detected in the enlarged image. Therefore, as in the state at time (T + 2ΔT) in FIG. The state where the image is used is returned to the state where the entire image E (T + 2ΔT) is used. At this time (T + 2ΔT) and the next time (T + 3ΔT), the three edge images necessary for silhouette matching cannot be obtained. Silhouette matching resumes after becoming. Of course, if the entire image is always stored in the edge image storage unit 13, it is possible to continue silhouette matching at times (T + 2ΔT) and (T + 3ΔT) by using the edge image.

  Since the area of the image for detecting the movement area Dm is changed as described above, the area of the image to be displayed on the display device is switched as shown in FIG. At time (T-4ΔT), (T-3ΔT), (T-2ΔT) and time (T + 2ΔT), (T + 3ΔT), (T + 4ΔT), a grayscale image A (T-4ΔT) corresponding to the entire image area, A (T-3ΔT), A (T-2ΔT), A (T + 2ΔT), A (T + 3ΔT), A (T + 4ΔT) are displayed, and enlarged images are displayed at times (T−ΔT), (T), and (T + ΔT). Gray images A (T−ΔT), A (T), and A (T + ΔT) corresponding to the region are displayed.

  With the above-described configuration, the presence / absence of a moving area is always determined in a relatively large area of the target space. The details of the area can be checked on the screen. In the configuration example described above, a configuration is adopted in which the first moving region extracting unit returns to the state in which the moving region is extracted as soon as the moving region is not extracted by the second moving region extracting unit. After the state in which the moving area is extracted by the second moving area extracting unit is continued for a certain period of time, it may be returned to the state in which the moving area is extracted by the first moving area extracting unit.

(Embodiment 2)
In the first embodiment, the presence / absence of a moving region is always extracted by silhouette matching using the entire image, and an enlarged image is generated when the moving region is extracted, and the presence / absence of the moving region is extracted by silhouette matching using the enlarged image. The structure to be adopted is adopted. Therefore, when a moving region is extracted and an enlarged image is generated while using the entire image, it is necessary to generate an enlarged image from an edge image obtained before the enlarged image is obtained.

  In the present embodiment, when an enlarged image is generated, a difference image between two edge images adjacent in time series is generated without performing silhouette matching, and the enlarged image is extracted while the moving region is extracted in the difference image. The configuration in which the processing according to is continued is adopted. Therefore, the second moving area extracting means uses a difference calculating section that generates a difference image instead of the logic calculating section 14b, and a movement determining section that determines the presence or absence of a moving area from the difference image instead of the movement determining section 15b. Is used.

  In addition, when the moving region is extracted by silhouette matching using the entire image, the enlarged image is not generated by electronic zooming back to the past edge image as in the first embodiment, and the second moving region is not generated. A difference image is generated using only an enlarged image generated during extraction of the moving region by the extraction means. Since two images are required to generate the difference image, only one image immediately after shifting to the state of extracting the moving area by the second moving area extracting means can generate the difference image. In other words, the moving area cannot be extracted only once immediately after the moving area is extracted by the second moving area extracting means. However, the period during which the moving area cannot be extracted is a period required for one update of the image immediately after the moving area is detected, and is a short period. Therefore, the moving area almost disappears from the partial area during this period. There is no problem in practical use. Further, similarly to the first embodiment, the moving area cannot be extracted twice immediately after the shift from the moving area extraction by the difference image to the moving area extraction by silhouette matching.

  By the above operation, the image of each part changes as shown in FIG. In the figure, (n + 4) images from time (T-3ΔT) to time (T + nΔT) are shown. Among these, an enlarged image is generated in a period from time (T−2ΔT) to time (T + (n−1) ΔT).

  The edge images stored in the edge image storage unit 13 before the time (T-3ΔT) and after the time (T + nΔT) in FIG. 3A..., E (T-3ΔT), E (T + nΔT),. The moving region Dm can be extracted as shown in FIG. 3B by silhouette matching using three edge images. In the illustrated example, it is assumed that a partial region Dp including the moving region Dm is set at the time (T-3ΔT). When the moving area Dm is extracted and the partial area Dp is set at the time (T-3ΔT), at the next time (T-2ΔT), the enlarged image E (T-2ΔT) is displayed in the enlarged image generation unit 16 (see FIG. 1). ) Is generated. Further, since the enlarged image E (T−ΔT),... Is generated after the next time (T−ΔT), the difference image is generated using the adjacent edge images in time series after the time (T−ΔT). Can be sought.

  That is, when the difference image is generated using the enlarged image E (T-2ΔT),... After the time (T-2ΔT), the difference image is displayed at the time (T-2ΔT) as shown in FIG. The difference image D (T−ΔT),... Can be obtained after the time (T−ΔT). Since the obtained difference image D (T−ΔT),... Includes the moving area Dm, the movement determining unit determines whether the person is a person based on the size of the moving area Dm.

  In the illustrated example, the process for obtaining the moving area based on the difference image D (T + (n-1) ΔT) at the time (T + (n−1) ΔT) ends, and the moving area is obtained by silhouette matching at the time (T + nΔT). Transition to processing. However, since an edge image necessary for silhouette matching is not obtained at time (T + nΔT), the process for obtaining a moving region by silhouette matching is after time (T + (n + 2) ΔT).

  As in the first embodiment, since the image area for detecting the movement area Dm is changed, the area of the image displayed on the display device is changed in accordance with the change of the image area. Thus, in the period from time (T−2ΔT) to time (T + (n−1) ΔT), the enlarged grayscale image A (T−2ΔT),..., A (T + (n−1) ΔT) Is displayed, and a grayscale image corresponding to the entire image area is displayed at other times.

  Other configurations and operations are the same as those of the first embodiment, and in the configuration of the present embodiment, a difference image obtained only by the difference calculation of two images is used while a moving region is detected by an enlarged image. Since the presence / absence of the moving region is determined, processing with a small processing amount and a small time lag is possible as compared with silhouette matching that requires three or more images. Further, since the enlarged image storage unit 17 only needs to store two enlarged images, the capacity of the semiconductor memory used for the enlarged image storage unit 17 can be reduced.

(Embodiment 3)
The present embodiment is a combination of the first embodiment and the second embodiment, and when a moving area is detected in the entire image and an enlarged image is generated, the moving image is first generated by generating a differential image of the enlarged image. The presence / absence of a region is confirmed, and then the presence / absence of a moving region is confirmed by silhouette matching when three enlarged images necessary for silhouette matching are prepared. Therefore, substantially only one difference image is generated.

  The image of each part changes as shown in FIG. As in FIG. 3, (n + 4) images from time (T-3ΔT) to time (T + nΔT) are shown. The difference from the second embodiment is that, as shown in FIG. 3B, silhouette matching is not performed in the period in which the enlarged image is obtained in the second embodiment, but in this embodiment, FIG. As shown, silhouette matching is performed while an enlarged image is obtained.

  That is, at the time (T-2ΔT) at which the first enlarged image E (T-2ΔT) is obtained, the difference image is not substantially selected, and the time at which the second enlarged image E (T−ΔT) is obtained. In (T−ΔT), a difference image D (T−ΔT) is obtained. Up to this point, the operation is the same as that of the second embodiment. Next, at time (T), E (T−2ΔT), E (T−ΔT), and E (T) are obtained for the three enlarged images, so that silhouette matching is possible, and no difference image is obtained. Further, as shown in FIG. 3B, the presence / absence of the moving region is detected by silhouette matching. The subsequent processing is the same as in the first embodiment.

  Therefore, similarly to the operation shown in FIG. 3D, as shown in FIG. 4D, in the period from the time (T−2ΔT) to the time (T + (n−1) ΔT), the enlarged shading is performed. Images A (T−2ΔT),..., A (T + (n−1) ΔT) are displayed, and gray images corresponding to the entire image area are displayed at other times.

  Other configurations and operations are the same as those of the first and second embodiments. In the configuration of the present embodiment, the difference obtained only by the difference calculation of the two images while the moving area is detected by the enlarged image. Since the presence / absence of a moving region is determined using an image, the processing amount is small and the time lag is small immediately after shifting to an enlarged image, as compared with silhouette matching that requires three or more images.

(Embodiment 4)
In each of the above-described embodiments, silhouette matching is performed when returning from a state in which a moving region is extracted using an enlarged image to a state in which a moving region is extracted using an entire image. The moving region can be extracted only from the eye image. In order to extract a moving area from the first image, as described in the first embodiment, when an enlarged image is obtained by electronic zoom, a past edge image stored in the edge image storage unit 13 is used. Though conceivable, this configuration cannot be adopted when using an optical zoom to obtain an enlarged image.

  Therefore, in the present embodiment, the entire edge image is stored in advance in the edge image storage unit 13 as a reference image, which is an edge image including only a background that does not include a moving region. Therefore, when returning from the state in which the moving area is extracted using the enlarged image to the state in which the entire image is used, a difference image from the reference image stored in the edge image storage unit 13 is generated, and silhouette matching is possible 3 Until the first image is obtained, the presence / absence of a moving region is determined from the difference image. Since silhouette matching becomes possible when the third image is obtained, silhouette matching is performed thereafter. Here, since the reference image is an image in which the moving area is not detected, three or more past images are used for the image in which the moving area is detected.

  Now, as shown in FIG. 5, during the period from time (T-3ΔT) to time (T + (n + 2) ΔT), as shown in FIG. 5 (b), it moves at time (T) by performing silhouette matching. Assume that the region Dm is extracted. Here, the edge image E (T-3ΔT), which is a reference image not involved in silhouette matching, of the edge images shown in FIG. 5A is considered to be an image of only a background that does not include a moving region. The edge image E (T-3ΔT) is stored in the edge image storage unit 13.

  The process of determining the presence or absence of a moving region using an enlarged image can employ any of the configurations of the first to third embodiments. When the state of confirming the presence / absence of the moving area is terminated using the enlarged image and the process returns to the process of extracting the moving area from the entire image at time (T + nΔT), the edge image E (T + nΔT) and the edge image at this time (T + nΔT) A difference image D (T + nΔT) from the edge image E (T-3ΔT) that is the reference image read from the storage unit 13 is generated. Since this difference image D (T + nΔT) is based on the edge image E (T−3ΔT) that does not include a moving object, it can be considered that a moving region exists when a significant difference occurs. In the illustrated example, a case where the moving area Dm is included in the difference image D (T + nΔT) is shown. The same is true at time (T + (n + 1) ΔT), and the difference image D (T + (n + 1)) between the edge image E (T + (n + 1) ΔT) and the edge image E (T−3ΔT) as the reference image. ΔT) is generated, and the presence / absence of the moving region Dm is determined based on the difference image D (T + (n + 1) ΔT).

  At time (T + (n + 2) ΔT), three images are prepared after returning from the process of determining the presence or absence of the moving region based on the enlarged image, and silhouette matching is possible at this point. Therefore, thereafter, the moving region Dm shifts from the state of extracting the moving region using the difference image to the state of extracting the moving region Dm by silhouette matching.

  In the present embodiment, the moving region can be extracted without causing a time lag from the time when the process proceeds to the process of extracting the moving region based on the entire image from the state where the presence or absence of the moving region is determined based on the enlarged image. Other configurations and operations are the same as those of the other embodiments described above.

(Embodiment 5)
This embodiment adds a human sensor for detecting a person using a pyroelectric infrared sensor (not shown) separately from the configuration of any of the above-described embodiments, and makes a determination using the pyroelectric infrared sensor. By combining the detection of the detected person and the detection of the person determined using the image, it is possible to determine the presence or absence of the person with higher accuracy than detecting the person alone. Hereinafter, a sensor that detects the presence or absence of a person using an image is referred to as an image sensor, and a sensor that detects the presence or absence of a person using a pyroelectric infrared sensor is referred to as a pyroelectric sensor. Therefore, this embodiment adds a pyroelectric sensor to the image sensor having any one of the configurations of the above-described embodiments, and comprehensively controls the operation by comprehensively outputting the image sensor and the pyroelectric sensor. (Not shown) is added. In the present embodiment, the detection areas of the image sensor and the pyroelectric sensor are assumed to be the same.

  The configuration of the pyroelectric sensor is well known, and it uses a pyroelectric infrared sensor that detects the heat rays radiated from a person, and triggers and holds a retriggerable holding timer when an output change occurs in the pyroelectric infrared sensor The output of the timer is inverted, and a detection signal indicating that a person is present in the detection area of the pyroelectric infrared sensor is output during a period in which the output of the holding timer is inverted. The pyroelectric infrared sensor is a differential sensor, and generates an output corresponding to the magnitude of change in incident heat dose. Therefore, even if a person exists in the detection area, if the person is stationary, the heat dose does not change, and it is mistaken that there is no person in the detection area. In view of this, a configuration is adopted in which a holding timer is provided to re-trigger the holding timer and maintain the detection signal when a change in heat dose occurs within the time limit of the holding timer. In front of the light receiving surface of the pyroelectric infrared sensor, an optical element (lens or mirror) that causes nonuniform sensitivity in the field of view of the pyroelectric infrared sensor is placed, so that minute movements of people in the detection area The heat dose incident on the pyroelectric infrared sensor is changed.

  The operation of the general control unit used in this embodiment is shown in FIG. Steps S1 and S2 in FIG. 6 are operations of the image sensor in each embodiment described above. However, in the embodiment described above, the determination when returning from the state in which the presence or absence of the moving region is detected using the enlarged image to the state in which the moving region is extracted using the entire image is performed in the above-described embodiment. However, the present embodiment is different in that the determination of steps S4 to S8 is added.

  That is, when a moving region (hereinafter referred to as a person) is detected by the image sensor (S1) and an enlarged image is generated (S2), the presence or absence of the moving region is confirmed using the enlarged image (S3). That is, in step S1, as shown in FIG. 7A, the partial area Dp is set by detecting the person M by the image sensor, and in step S2, the partial area Dp is set as shown in FIG. 7B. An enlarged image enlarged to one screen is generated. In FIG. 7, a range indicated by reference sign B is a detection area of the image sensor and pyroelectric sensor. Here, if the person M is moving in the partial area Dp, the person M is also detected in the pyroelectric sensor and a detection signal is output, but the person M is detected by the image sensor even if the detection signal is not output. If so, the enlarged image is maintained.

  Thereafter, if the person M is present in the partial region Dp but the person M is stationary, the person M may not be detected by either the image sensor or the pyroelectric sensor. That is, if the person M is not detected in the enlarged image in step S3 and the person M is not detected by the pyroelectric sensor in step S4, it is determined that the person M is still present in the partial region Dp but is still. The enlarged image is maintained until a predetermined time limit elapses (S5). When the time limit expires, there is a possibility that the person M has left the partial area Dp, so that the whole image is restored (S6), and the state where the person M is detected by the image sensor is restored (S1).

  On the other hand, if the person M continues to exist in the detection area B even after the person M leaves the partial area Dp as shown in FIG. 7C after step S2, the enlargement set by the image sensor is performed. The person M is not present in the image, but the person M is detected by the pyroelectric sensor. Therefore, even if the person M is not detected in step S3, when the person M is detected by the pyroelectric sensor (S4), it is determined that the person M exists in the detection area B, and the entire image is restored from the enlarged image. (S7). Thereafter, if the person M is no longer detected by the pyroelectric sensor (S8), it is determined that the person M has left the detection area B as shown in FIG. 7D, and the person M is detected by the image sensor. Return (S1).

  An image displayed on the display device through the output unit 19 by the above-described operation is, for example, as shown in FIG. 7A, since the partial area Dp is set in the entire image, an image in which the partial area Dp is set in the entire image as shown in FIG. 8A is generated. In this relationship, the person M moves in the enlarged image as shown in FIG. Further, if the person M leaves the partial area Dp as shown in FIG. 7C, or if the person M leaves the detection area B as shown in FIG. 7D, FIGS. In this way, the person M does not exist in the enlarged image. However, when the time limit expires, the displayed image becomes the entire image.

  By the operation described above, it is possible to detect the person M by linking the image sensor and the pyroelectric sensor, and therefore it is possible to improve the certainty of detection regarding the presence or absence of the person M in the detection area B. In addition, switching between the enlarged image and the entire image can be performed appropriately, and when the presence or absence of the person M can be determined from the enlarged image, detailed information based on the enlarged image can be acquired. When the person M leaves the partial area Dp, It becomes possible to immediately return to the whole image and track the person M in the image. Other configurations and operations are the same as those of the other embodiments described above.

(Embodiment 6)
The present embodiment is similar to the fifth embodiment in that the image sensor and the pyroelectric sensor are configured to suppress erroneous detection before detection of a person and to easily maintain the detection state after detection of a person. Therefore, the general control unit (not shown) adopts a configuration in which the judgment conditions used for detecting the presence of a person with the image sensor and the pyroelectric sensor are changed before and after the person is detected. The determination condition in the present embodiment can be rephrased as sensitivity for detecting a person.

  The sensitivity with which the person is detected by the image sensor can be adjusted by at least one of the number of times the moving area is detected within a specified time and the threshold value for determining whether the moving area is a person. That is, in each of the above-described embodiments, an enlarged image is generated when the moving region is detected once. However, if the enlarged image is generated when the moving region is detected by a plurality of continuous images, substantially Therefore, the sensitivity of detecting a person is reduced, and the erroneous detection of a person can be suppressed. Also, if the area of the moving region is used to determine whether a person is a person, increasing the threshold value reduces the probability of detecting a person or small animal that is present in the distance, and consequently suppresses false detection. Will do.

  The sensitivity of detecting a person by the pyroelectric sensor is adjusted by at least one of the number of detection pulses generated by binarizing the output of the pyroelectric sensor with a threshold value within a specified time and the threshold value. can do. In other words, if the enlarged image is generated when a plurality of detection pulses are obtained within a predetermined time instead of generating the enlarged image with only one detection pulse, the sensitivity is substantially reduced. That's right. Also, if the threshold for binarizing the output of the pyroelectric sensor to generate a detection pulse is set high, it will be detected if the heat dose incident on the pyroelectric sensor is small, such as a person or small animal that exists far away. No pulse is generated, and as a result, erroneous detection can be suppressed.

  Therefore, in this embodiment, the sensitivity of the image sensor and the pyroelectric sensor can be set in two steps, high and low, and both the image sensor and the pyroelectric sensor are set to low sensitivity before detecting the presence of a person. Both are set to high sensitivity after the presence of a person is detected. For example, when high sensitivity is set, the image sensor shifts to an enlarged image when the moving region is detected once, and the pyroelectric sensor outputs a detection signal when the detection pulse is output once. When the low sensitivity is set, the image sensor shifts to an enlarged image when the moving region is detected twice, and the pyroelectric sensor is configured to output a detection signal when the detection pulse is output twice.

  By adopting this configuration, it is possible to prevent false detection of distant people and small animals at all times with low sensitivity, and detect the presence of a person in either the image sensor or pyroelectric sensor. When they do, both are highly sensitive to make it easier to detect the presence of a person. Therefore, even when a person is detected only by the pyroelectric sensor, the sensitivity of the image sensor is increased, so that it is possible to quickly shift to an enlarged image. Further, even when a person is detected only by the image sensor, the sensitivity of the pyroelectric sensor is increased, so that the above-described holding timer is easily re-triggered and the detection signal is easily maintained.

  In the present embodiment, while one of the image sensor and the pyroelectric sensor is detecting a person, both are set to high sensitivity, but the time for setting high sensitivity is limited to a certain time. You may employ | adopt the structure to do. Other configurations and operations are the same as those of the above-described embodiments.

(Embodiment 7)
In the configuration of the sixth embodiment, both the sensitivity of the image sensor and the pyroelectric sensor are switched in two stages, but this embodiment includes the image sensor and the pyroelectric sensor as in the fifth and sixth embodiments. And a configuration in which the accuracy of detecting a person in the image sensor is switched in two stages. The basic operation is the same as in the second embodiment, and a person is detected by silhouette matching until a person is detected, and after the person is detected, a difference image is used to detect the presence or absence of the person. is doing. However, in the second embodiment, only the image sensor is used. In the present embodiment, when a person is detected in either the image sensor or the pyroelectric sensor, the difference image is used in the image sensor. Is to be detected. That is, when a person is detected by the image sensor, the difference image of the enlarged image is used, but when a person is detected by the pyroelectric sensor and not detected by the image sensor, the person is detected using the difference image of the whole image. Become.

  Since a difference image can extract a person by using two edge images, the time required to set an enlarged image is shorter than when using three edge images as in silhouette matching. Even when extracting a person using a difference image for the entire image, a partial area including a person is set and an enlarged image is generated by enlarging the partial area to the size of one screen as in the case of silhouette matching. To do.

  With this configuration, the timing for generating an enlarged image is one image earlier than when extracting a person by silhouette matching, and the timing at which a person can be confirmed by the enlarged image can be advanced accordingly. That is, it is possible to lengthen the period for confirming the presence / absence of a person using an enlarged image, compared to the case of extracting a moving region by silhouette matching. Other configurations and operations are the same as those of the other embodiments described above. In addition, although the example which used the pyroelectric sensor in Embodiment 5 thru | or Embodiment 7 was shown, it replaced with the passive human sensor like a pyroelectric sensor, and the active type which transmits / receives infrared rays and transmits / receives an ultrasonic wave It is also possible to combine the human sensor with an image sensor.

DESCRIPTION OF SYMBOLS 10 Imaging means 11 Gray image storage part (image storage means)
12 edge image generation unit 13 edge image storage unit (image storage means)
14a Logical operation unit (first moving region extracting means)
14b Logical operation unit (second moving region extracting means)
15a Movement determination unit (first movement area extraction means)
15b Movement determination unit (second movement area extraction means)
16 Enlarged image generation means 17 Enlarged image storage unit (image storage means)

Claims (9)

  An imaging unit that continuously captures the target space over time, an image storage unit that stores a plurality of images captured by the imaging unit, an image of interest among the images stored in the image storage unit, and the image A first moving area extracting means for extracting a moving area from the image of interest by comparing with a predetermined number of images in the past, and a moving area is extracted by the first moving area extracting means in the image of interest. Then, an enlarged image generating means for generating an enlarged image obtained by enlarging the partial area set to include the moving area to the size of one screen, an image of interest among the enlarged images, and a past specified number of the images Second moving region extracting means for detecting the presence or absence of the moving region in the image of interest by comparing the image with the second image, and the second moving region extracting means is adjacent in time series Moving object detection apparatus using an image and detecting the presence or absence of the moving region from the difference image by the two enlarged image.   An imaging unit that continuously captures the target space over time, an image storage unit that stores a plurality of images captured by the imaging unit, an image of interest among the images stored in the image storage unit, and the image A first moving area extracting means for extracting a moving area from the image of interest by comparing with a predetermined number of images in the past, and a moving area is extracted by the first moving area extracting means in the image of interest. Then, an enlarged image generating means for generating an enlarged image obtained by enlarging the partial area set to include the moving area to the size of one screen, an image of interest among the enlarged images, and a past specified number of the images Second moving region extracting means for detecting the presence or absence of the moving region in the image of interest by comparing with the image of the second image. A common area in each difference image with the plurality of prescribed enlarged images is detected as a moving area, and the prescribed plural enlarged images are obtained after the processing is shifted from the first moving area extracting means. A moving object detection apparatus using an image characterized in that the presence or absence of a moving region is detected from a difference image between two enlarged images that are adjacent in time series for a period until it is determined.   The image storage means stores, as a reference image, an image in which no moving area is detected before the enlarged image generating means generates the enlarged image, and the moving area is not detected by the second moving area extracting means. 3. The moving area is extracted using a difference between an image of interest and a reference image when returning to the process of extracting the moving area by the first moving area extracting means. Moving object detection device using images.   A human sensor for detecting the presence or absence of a person in the detection area without using an image having a detection area that coincides with a detection area for detecting a person as a moving area from the image picked up by the image pickup means is added. And a general control unit that selectively selects processing of the first moving region extracting unit and the second moving region extracting unit based on a combination of results of human detection and human detection by a human sensor. A moving object detection apparatus using the image according to any one of claims 1 to 3.   The comprehensive control unit extracts a moving area by the first moving area extracting unit when the second moving area extracting unit does not detect the presence of the person and the human sensor detects the presence of the person. The moving object detection apparatus using an image according to claim 4, wherein the process is returned to the process.   The comprehensive control unit can change a determination condition for selecting the first moving region extracting unit and the second moving region extracting unit, and is at least one of the first moving region extracting unit and the human sensor. In the period in which the presence of a person is detected, a judgment condition is set so that it is easier to detect the presence of a person compared to a period in which both do not detect the presence of a person. Item 5. A moving object detection apparatus using the image according to item 4.   The determination condition is the number of times that a person is detected within a specified time, and when the number of times is reached, it is determined that a person exists, and after the detection of the presence of a person, the number of times is set to be less than before detection The moving object detection apparatus using an image according to claim 6.   The determination condition is a threshold used to determine whether or not a person is present. When the threshold is exceeded, it is determined that a person is present, and the threshold is set lower than before detection after the presence of a person is detected. The moving object detection apparatus using the image according to claim 6.   In the period when both the first moving region extracting means and the human sensor do not detect the presence of a person, the comprehensive control unit is configured to display each of the focused image and a plurality of past prescribed images. A period in which a common area in the difference image is detected as a moving area, it is determined whether the moving area is a person, and at least one of the first moving area extracting means and the human sensor detects the presence of a person 5. The moving object detection device using an image according to claim 4, wherein the presence / absence of a moving region is detected from a difference image between two adjacent images in time series.

Images