JP4628860B2 - Image sensor - Google Patents

Description

  The present invention relates to an image sensor that tracks a moving object based on image information, and more particularly, to an image sensor that improves the tracking accuracy of a moving object when information from the vicinity of the camera to a distance is included in the image.

2. Description of the Related Art Conventionally, there is known an image monitoring apparatus that detects a region indicating characteristics of a moving object such as an intruder and abnormally outputs it from a change mode between an image input from a camera as an imaging device and a past image. As this image monitoring apparatus, for example, a currently captured input image is compared with a previously stored past image, a difference between both images is extracted as a variable region, and the area and moving speed of the extracted region are extracted. Such a device is known that determines whether or not an intruder exists by calculating the above.

With regard to such an image monitoring device, in particular, Patent Document 1 uses information on installation conditions such as the height and depression angle of a camera to detect information such as the height and size of an intruder and the moving speed, and identifies the moving object. An intruder detection device is described. In this intruder detection device, based on the position of the moving object extracted on the screen, measure the distance from the camera to the object, calculate the size of the moving object, the moving speed, etc. based on this distance, It is determined whether or not the moving object is an intruder.
JP-A-8-136251

In the conventional intruder detection apparatus described above, the extracted moving object is tracked for a plurality of frames, and the moving speed of the object is calculated based on the actual (in real space) distance from the camera to the object. Whether or not it is an intruder is determined.
However, in the conventional intruder detection device, since the moving speed of the moving object is calculated by obtaining the moving distance in real space from the image, the accuracy of measuring the moving speed is lowered for a moving object that exists far away. There is a problem.

Generally, in a captured image, a nearby object is large (large number of pixels) and a far object is small (small number of pixels). That is, a moving object present in the distance has a low distance resolution per pixel on the image (a large distance in real space per pixel), and the measurement accuracy of the moving distance is lowered. In other words, since the distance in the real space per pixel in the captured image increases as the subject moves further away, the calculated movement distance in the real space for a distant moving object includes a large amount of error. Become.

For this reason, in the prior art document, the moving speed calculated based on the moving distance in the real space and the moving distance is determined to be a person only when the distant moving object is shifted by several pixels on the image. There is a risk of exceeding the threshold value and becoming a very large value. Therefore, it is difficult for the conventional intruder detection device to determine that a distant moving object is a person based on a moving distance or speed in real space, and is not suitable for a wide space monitoring application. There is a problem.

Such a decrease in the measurement accuracy of the real space distance is particularly noticeable when the depression angle of the camera optical axis is reduced. When the depression angle of the camera optical axis is close to zero, the photographed image includes a horizontal line, and the point where the distance is theoretically infinite is overlooked.
For example, when the camera is installed so that the depression angle of the camera optical axis is reduced in order to widen the imaging range, the input image includes from the vicinity of the camera to the distant place. Only intruders in the vicinity of the camera can be detected, and it is difficult to detect intruders in the distance.

Therefore, an object of the present invention is to provide an image sensor that can accurately detect an intruder even when an image includes from the vicinity of the camera to a distance.

In order to achieve the above object, an image sensor according to the present invention extracts a variation area from a difference between a captured image of a monitoring space sequentially acquired by an imaging unit and a past image and tracks the movement of the variation area. A distant area setting unit that sets an area in which the distant area of the monitoring space is reflected in the captured image as a distant area; installation information including an installation height, depression angle, and angle of view of the image capturing unit; and the distant area And a storage unit for storing the position of the extracted variation area, a presence position determination unit for determining whether or not the variation region is located in the far region, and a distance on the captured image based on the installation information an image distance calculating unit that converts into an actual distance on the monitor space, the be determined at least one of the position of the present position determining unit is extracted with temporally preceding variations region and the distal region variant The distance on the captured image between the regions as a condition of correspondence, the present position determining unit if the it is determined both the position outside the distal region of the variation region extracted by chronologically successive between the variable domain A tracking unit that tracks the movement of the variable region using the actual distance as a condition for association, and the far region setting unit has the actual distance between adjacent pixels in the captured image exceeding a preset reference. A region is set as the far region .

The farther the position of the object reflected in the captured image is, the smaller the object on the image becomes, and the object shape is roughly reflected. Further, the distance resolution becomes lower as the distance increases, and the reliability of the distance in real space calculated per pixel becomes lower. For this reason, the farther the position of the object reflected in the captured image is, the shorter the moving distance on the image becomes.However, depending on the posture of the object reflected in the fluctuation area and the shape extracted as the fluctuation area, The distance in the real space obtained by converting the movement distance may be a very large value including a lot of errors. Therefore, if a distant object is associated with a distance in real space, it may not be associated.
On the other hand, the closer the position of the object reflected in the captured image is, the longer the moving distance on the image becomes. However, the distance in the real space obtained by converting the moving distance on the image is reliable. High value.

Therefore, the image sensor of the present invention associates and tracks a distant object with the number of pixels that is the moving distance on the image, and corresponds to an object reflected from a place that is not far away by a distance in real space. Track and track.
According to the present invention, it is possible to track (track) a moving object without depending on the distance in the real space for a distant area where the reliability of the distance in the real space is low. Even so, it becomes possible to track.

Preferably, the far area setting unit calculates a position of a vanishing line in the captured image based on the installation information, and sets a periphery of the vanishing line as the far area. As a result, by calculating the position of the vanishing line that is a set of vanishing points, it is possible to set a region in which a distant image is captured in the captured image by simple calculation.

In a preferred embodiment, the far region setting unit sets, as the far region , a region where an actual distance between the adjacent pixels is equal to or greater than a first threshold value. As a result, it is possible to set a region with a low distance resolution calculated as the distance in the real space is greater than or equal to the threshold as the far region.

Further, preferably said far region setting unit includes a wherein calculating the actual distance rate of change between adjacent pixels, said alteration rate sets a region is the second threshold or more as the far region To do. Accordingly, an area with a low distance resolution, which is calculated as a change rate of the distance in the real space with respect to the pixel row in the depth direction of the monitoring space, can be set as a far area.

In addition, the present invention is not limited to the above-described image sensor. The present invention may be expressed and specified in the form of a method, program or system. For example, another aspect of the present invention is a method for controlling an image sensor that performs the operation of the above configuration. Another aspect of the present invention is a program that causes a computer to execute such a method.

According to the present invention, it is possible to track (track) a moving object without depending on the distance in the real space for a distant area reflected in an image obtained by photographing the monitoring space. Can track.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
The image sensor of the present invention extracts a distant area reflected in a captured image from an image captured in a monitoring space, and executes tracking processing of a moving object by using different logic in the distant area and other areas. Thus, a moving object is detected.
In the present embodiment, an example will be described in which a human (intruder) who has entered the monitoring space as a moving object to be detected is detected.

FIG. 1 is a block diagram showing the configuration of the image sensor 1.
As shown in FIG. 1, the image sensor 1 includes an imaging unit 10, a storage unit 20, an image processing unit 30, and an output unit 40. Each part will be described below.

  The imaging unit 10 includes an imaging device such as a camera. The imaging unit 10 converts the optical image of the monitoring space into a digitized image composed of discrete pixel groups, and outputs the image to the image processing unit 30 as a captured image in which each pixel is expressed by a luminance value. . The imaging unit 10 acquires a captured image of one frame at a predetermined time interval (for example, every 1/5 second) and outputs it to the image processing unit 30.

The storage unit 20 stores information used for various processes of the image sensor 1. The information stored in the storage unit 20 includes installation information.
The installation information is information indicating the installation environment of the imaging unit 10, the installation height H of the imaging unit 10, the depression angle Φ of the imaging unit 10, the vertical direction (Y-axis) number of pixels Y of the image obtained from the imaging unit 10, and It includes the number of pixels X in the horizontal direction (X axis), the vertical field angle θh, and the horizontal field angle θv. The number of pixels Y and the number of pixels X may be obtained from the captured image output by the imaging unit 10, and the vertical field angle θh and the horizontal field angle θv are calculated from the focal length of the imaging device included in the imaging unit 10. Also good.
The far region information is information for specifying a region in which a distant region spreading around the vanishing line in the image is captured. The far area information is calculated and stored by the far area setting unit 31 based on the installation information.

The image processing unit 30 is configured by a computer having a CPU and the like, receives input of a digitized captured image, performs difference processing, binarization processing, labeling (object candidate detection) processing, tracking (object candidate detection). Tracking) processing, feature calculation processing, and abnormality determination processing.
The image processing unit 30 includes a far region setting unit 31, a fluctuation region extraction unit 32, an object candidate detection unit 33, an existence position determination unit 34, an object candidate tracking unit 35, an image distance calculation unit 36, and a feature calculation. A unit 37 and an abnormality determination unit 38 are included. The object candidate tracking unit 35 includes an actual distance tracking unit 351 and a pixel distance tracking unit 352.

The far region setting unit 31 extracts a far region reflected in the captured image and sets the far region. FIG. 2 is a diagram illustrating a far region set in the captured image. In FIG. 2, the distant area is hatched for explanation.
Based on the installation information stored in the storage unit 20, the far region setting unit 31 uses, as a far region, a region that extends around the vanishing line that is a set of vanishing points (infinity points) in the image captured by the imaging unit 10. Coordinate information for calculating and specifying this far region is stored in the storage unit 20 as far region information. The details of the far region calculation method will be described later.

As illustrated in FIG. 3, the fluctuation region extraction unit 32 calculates a difference value between pixels corresponding to each other in the captured image input from the imaging unit 10 and the reference background image, and generates a difference image. The background image is an image captured in a state where no intruder or the like exists in the monitoring space, and is acquired in advance in the past and stored in the storage unit 20. The variation area extraction unit 32 binarizes the difference image based on the magnitude of the luminance value of each pixel included in the generated difference image and a predetermined threshold value, and the variation of the luminance value becomes equal to or greater than the threshold value. A differential binary image is generated by extracting the fluctuation pixels. As a result, a fluctuation region composed of fluctuation pixels whose luminance value fluctuation is equal to or greater than the threshold value is extracted. At this time, it is preferable to generate a binary image from which noise has been removed by filtering for expansion / contraction and micro-area exclusion. The calculated binary difference image is stored in the storage unit 20 to serve as a reference for tracking processing for the next captured image.
The background image may be updated with an image captured every predetermined time. Further, a difference image between frames may be obtained by using a captured image acquired before a predetermined frame as a background image.

  The object candidate detection unit 33 groups substantially continuous variation pixel groups included in the binary difference image generated by the variation region extraction unit 32 as one variation region, and labels each variation region with a unique label. Then, a grayscale pattern matching (normalized correlation) process is performed on each labeled variation region and a background image (or a captured image before a predetermined frame) corresponding to each variation region to calculate a texture similarity. A fluctuation area in which the calculated texture similarity is equal to or less than a predetermined value is determined as an object candidate area, and is excluded as a disturbance in other cases.

The presence position determination unit 34 determines whether or not the variable region determined as the object candidate region is included in the far region specified by the far region information stored in the storage unit 20. The presence position determination unit 34 determines that the object candidate region exists on the far region when a predetermined ratio or more of the pixel group constituting the object candidate region is located on the far region. As the predetermined ratio, for example, a majority is set, and when the majority of the pixel group constituting the object candidate region is located on the far region, it is determined that the pixel exists on the far region. For example, in the example of FIG. 2, it is determined that the object candidate area A and the object candidate area B exist on the far area, and the object candidate area C is determined to exist outside the far area.
Note that the centroid coordinates of the object candidate area may be calculated, and if the centroid coordinates are located on the far area, it may be determined that the object candidate area exists on the far area. It may be determined that the object candidate region exists on the far region only when it is completely included.

The object candidate tracking unit 35 determines the object candidate area and the change area determined as the object candidate area in the frame (current frame) acquired by the current imaging and the object candidate area in the immediately preceding frame (may be several cycles ago). Object candidate region tracking processing is performed by comparison with the determined variation region. In the current frame, if an object candidate area exists in an area corresponding to the tracking search range set around the object candidate area in the immediately preceding frame, each other is based on the similarity of feature quantities such as the size and shape of the area. Object candidate regions estimated to be regions obtained by imaging the same object are associated with each other. The same label is assigned to the object candidate region associated with the same object imaged as the object candidate region of the immediately preceding frame. The labeling information is stored in the storage unit 20.
When the object candidate area in the immediately previous frame is not associated with the object candidate area, the object candidate area extracted in the current frame is determined as an object candidate area newly appearing in the monitoring space. The coordinates of the barycentric point of the newly appearing object candidate area are stored in the storage unit 20 as the appearance position coordinates.

The tracking search range is calculated by different logic depending on whether or not the object candidate region determined by the presence position determination unit 34 is located on the far region.
First, a case where the object candidate region is not located on the far region will be described. FIG. 4 is a diagram illustrating an example in which a binary image of consecutive frames is extracted, and an object candidate area is extracted outside a far area.
As shown in FIG. 4, when neither the object candidate area of the previous frame nor the object candidate area of the current frame is located on the far area (is located outside the far area), tracking is performed by the real distance tracking unit 351. Processing is performed. The real distance tracking unit 351 sets a tracking search range based on a distance (real distance) in the real space in the monitoring space, and searches for an object candidate region within the search range. That is, the tracking process is performed based on the actual distance that the object candidate area has moved (between the object candidate areas in both frames). The actual distance tracking unit 351 searches the current frame for an object candidate area in which the actual distance between the centroid coordinates of the object candidate area of the immediately preceding frame and the centroid coordinates of the object candidate area of the current frame is within the tracking search range. To do. The actual distance set as the tracking search range is set based on the moving speed of the person. For example, when the captured image is acquired at 5 frames / second, the actual distance is set to 200 cm. The actual distance used for the tracking process by the actual distance tracking unit 351 is calculated by the image distance calculating unit 36.

  The image distance calculation unit 36 converts the distance between arbitrary coordinates on the captured image into an actual distance. Here, a case will be described in which the actual distance between the object imaged in the object candidate area of the immediately preceding frame and the object imaged in the object candidate area of the current frame is calculated. By assuming that the floor surface of the monitoring space is flat and the object imaged in the object candidate area is in contact with the floor surface, the installation information of the storage unit 20 and the captured image of each object candidate area Based on the position, it is possible to estimate the linear distance from the object imaged in the object candidate area to the imaging device, and thereby estimate the linear distance between the objects imaged in each object candidate area.

Hereinafter, a method for calculating the actual distance by the image distance calculation unit 36 will be described.
In this embodiment, an example in which the actual distance between objects is calculated using the distance between the centers of gravity will be described. Here, it is assumed that the gravity center position of the object candidate region is at a position of Hman / 2 from the floor surface. Hman is a value set based on the height of the person who is the object to be detected, and is stored in the storage unit 20 in advance.
FIG. 5 is an explanatory diagram when the space between the imaging unit and the object is viewed from the side.
First, assuming that the center of gravity coordinates of an object candidate area in which an object is reflected is (x _g , y _g ), as shown in FIG. 5, the floor when the space between the imaging device included in the imaging unit 10 and the object is viewed from the side. The distance dg on the surface can be calculated using Formula 1 based on the installation information stored in the storage unit 20.

FIG. 6 is an explanatory diagram when the space between the imaging unit and the object is viewed from the vertical direction.
Next, as shown in FIG. 6, the distance ag on the floor when the imaging apparatus and the object are viewed from the vertical direction (the direction of the arrow Q in FIG. 5) is based on the installation information stored in the storage unit 20. It can be calculated using the following formula.

From Equations 1 and 2, the center coordinates of the two object candidate region _{(x g1, y g1) (} x g2, y g2) 2 points when the given _{(x g1, y g1) (} x g2, y g2 ) Can be calculated using the equation (3).

Based on the actual distance calculated by the image distance calculation unit 36, the actual distance tracking unit 351 searches for the object candidate region of the current frame that exists within the search range of the object candidate region of the immediately preceding frame. When an object candidate area is detected within the search range, the rate of change in the number of pixels included in the object candidate area and the circumscribing of the object candidate area between the object candidate area in the previous frame and the object candidate area in the current frame are detected. The similarity is calculated based on the aspect ratio change rate of the rectangle, and the object candidate regions with the highest similarity are labeled in association with the same object imaged region. Here, when a captured image is acquired at an object candidate area whose similarity is equal to or less than a predetermined threshold, for example, 5 frames / second, the change rate of the number of constituent pixels ΔS is 1/10 ≦ ΔS ≦ 10, and the aspect ratio change The object candidate area that does not satisfy each condition of 1/4 ≦ ΔP ≦ 4 for the rate ΔP is determined not to be an area in which the same object is imaged, and is not associated.

Next, the case where the object candidate region is located on the far region will be described. FIG. 7 is a diagram illustrating an example in which a binary image of consecutive frames is extracted, and an object candidate region is extracted on a far region.
As shown in FIG. 7, when at least one of the object candidate region of the immediately preceding frame or the object candidate region of the current frame is located on the far region, the pixel distance tracking unit 352 performs tracking processing. The pixel distance tracking unit 352 sets a tracking search range between both frames based on the number of pixels (pixel distance) on the captured image, and searches for an object candidate region within the search range. That is, tracking processing is performed based on the pixel distance to which the object candidate area has moved (between the object candidate areas in both frames). The pixel distance tracking unit 352 searches for an object candidate region in which the pixel distance between the centroid coordinate of the object candidate region in the previous frame and the centroid coordinate of the object candidate region in the current frame is within the tracking search range. The pixel distance set as the search range is set to a value that can remove disturbance, and is set to 70 pixels, for example, when a captured image is acquired at 5 frames / second.

In this way, the pixel distance tracking unit 352 searches for the object candidate region of the current frame that exists within the search range of the object candidate region of the immediately preceding frame. When an object candidate area is detected within the search range, the rate of change in the number of pixels included in the object candidate area and the circumscribing of the object candidate area between the object candidate area in the previous frame and the object candidate area in the current frame are detected. The similarity is calculated based on the aspect ratio change rate of the rectangle, and the object candidate regions with the highest similarity are labeled in association with the same object imaged region. The calculation of the similarity is the same as that of the actual distance tracking unit 351.

Here, a method of calculating the far region by the far region setting unit 31 will be described.
The far region setting unit 31 calculates the change rate of the actual distance corresponding to the coordinates (pixels) on the captured image by the image distance calculation unit 36, and sets the far region based on the calculated change rate of the actual distance.
Information in the depth direction of the monitoring space appears in the pixel column in the vertical direction (Y-axis) of the image. Therefore, the rate of change of the actual distance with respect to the barycentric coordinate y _g1 of the object candidate region on the Y axis of the captured image, specifically, the coordinate y _{g1 is} calculated when the value of the coordinate y _g1 on the Y axis is shifted by one pixel. To find out how much the actual distance D changes, the derivative of the actual distance D can be calculated. As a result, where the value of the derivative is large, the change in the actual distance D is large when the value of the coordinate y _g1 changes, and the distance resolution is low, and vice versa, the opposite is true when the value of the coordinate y _g1 changes. The change in the distance is small and the distance resolution is high.
Now, the actual distance D between any two barycentric coordinates of the object candidate region _{(x g1, y g1) (} x g2, y g2) when viewed as a function of the coordinates y _g1 on the Y axis, with respect to the coordinate y _g1 the rate of change (derivative) r _g of actual distance D can be calculated using equation 4 expression.

_Equation 4 is a quadratic fractional function with respect to y _g1 , and its outline is as shown in FIG. FIG. 8 is a diagram showing the relationship between the coordinate y _g1 on the Y axis and the rate of change r _g of the actual distance D corresponding to the coordinate y _g1 . 8, the change rate r _g of actual distance D indicates the actual distance D when the value of the coordinate y _g1 is shifted 1 pixel. Further, since the rate of change r _g of actual distance D is the lower range resolution of 1 pixel per degree becomes greater, FIG. 8 shows the coordinate y _g1 on the Y-axis, the distance resolution with respect to the coordinate y _g1 (the inverse of) It can also be seen as a diagram.
8, asymptote, the change rate r _g of actual distance D with respect to the coordinate y _g1 on the Y axis represents the position becomes infinite, represents the vanishing line. That is, the y _g1 axis coordinate of the asymptotic line in FIG. 8 is the coordinate of the disappearance line on the Y axis of the captured image.

FIG. 8 shows that there are points α and β at which the dependence of the actual distance D on the Y axis of the captured image changes dramatically. That is, if the center of gravity coordinates on the Y axis of the object candidate region are closer to the disappearance line than the point α or β, the actual distance D will change drastically when the center of gravity coordinates are shifted by only a few pixels. The reliability becomes low. Point alpha, beta, as shown in FIG. 8, to set the threshold Th in the rate of change r _g of actual distance D, obtained from the point at which r _{g =} Th on curve r _{g (y g1).} Similarly, a threshold is set for the actual distance D per pixel and the distance resolution at each coordinate on the Y axis, and the point where the actual distance D is equal to or greater than the threshold or the distance resolution point is equal to or less than the threshold. The points α and β may be obtained from the following points. The points α and β may be obtained from the slope of the curve r _g (y _g1 ). In this case, the points α and β are obtained from the point where the differential coefficient is 1 (the point where the tangent is ± 45 °).

Far region setting unit 31 calculates the rate of change r _g of actual distance D in the coordinate y _g1 at image distance calculating unit 36 calculates the value of y _g1 when the r g ₌ Th. That, alpha in FIG. 8, calculates the value alpha _y and beta _y of y _g1 axes beta. Then, alpha and _y and beta _y are obtained as coordinates on Y axis of the captured image, as shown in FIG. 2, a pixel that exists between alpha _{y-beta y} on the Y-axis, or Y on axis alpha _y-beta A pixel group continuous in the X-axis (horizontal) direction between _y is set as a far region and stored in the storage unit 20 as far region information. As a result, in response to distance resolutions that differ depending on the resolution of the captured image, an area in which the reliability of the actual distance is reduced in the captured image is extracted, and a distant area that spreads in a band around the disappearance line Can be set.

The far region calculation method is not limited to the above-described method. Based on the installation information stored in the storage unit 20, the position of the vanishing line in the image captured by the imaging unit 10 is extracted, and this vanishing line is extracted. The surrounding predetermined area may be calculated as a far area. Since vanishing points generally exist on the horizon, vanishing lines that are a set of vanishing points can be extracted from the captured image by calculating the position of the horizon in the monitoring space from the installation information. In this case, the far region setting unit 31 calculates the Y-axis coordinates of the vanishing line for the captured image, and sets the points _{y y} and β moved from the Y-axis coordinates of the vanishing line on the Y axis by a preset number of pixels. _The pixel group existing between α _{y and} β _y is set as a far region, and coordinate information for specifying this far region is stored in the storage unit 20 as the far region information.

  As described above, the object candidate tracking unit 35 tracks an object based on the actual distance when the object candidate area is located in an area where the reliability of the actual distance is high, and on the far area where the reliability of the actual distance is low. When the object candidate area is located, tracking the object based on the pixel distance makes it possible to appropriately perform the tracking process by changing the logic for setting the search range according to the position where the object exists.

The feature calculation unit 37 calculates an object feature amount that is a parameter indicating the humanity of the object candidate region labeled as the same object by the object candidate tracking unit 35.
In the present embodiment, an example will be described in which the amount of movement of an object reflected in an object candidate region is calculated as the object feature amount. The amount of movement is based on the distance between object candidate regions associated with the same object by the object candidate tracking unit 35 in captured images acquired at different times.

The feature calculation unit 37 reads the appearance position coordinates stored for the object candidate area labeled as the same object by the real distance tracking unit 351 from the storage unit 20, and the appearance position coordinates and the corresponding current frame The actual distance between the object candidate area and the barycentric coordinate is calculated as the actual distance movement amount of the object.
In addition, the feature calculation unit 37 reads out the appearance position coordinates stored for the object candidate region labeled as the same object by the pixel distance tracking unit 352 from the storage unit 20, and corresponds to the appearance position coordinates. The pixel distance between the object candidate area of the current frame and the barycentric coordinate is calculated as the pixel distance movement amount of the object.

Note that the object feature amount is not limited to the above-described movement amount, and the image feature amount (luminance average value and luminance dispersion value), geometric feature amount (area, aspect ratio, barycentric position) of the object candidate region, background You may calculate from the comparison value (normalization correlation, difference average value), etc. with an image.

The abnormality determination unit 38 functions as a moving body determination unit, and determines whether the object reflected in the object candidate area is an intruder based on the object feature amount calculated by the feature calculation unit 37. When detecting the presence of an intruder, the abnormality determination unit 38 determines that an abnormality has occurred and outputs an abnormality signal to the output unit 40.

The abnormality determination unit 38 determines that the corresponding object candidate region is an intruder when the actual distance movement amount input from the feature calculation unit 37 is equal to or greater than the predetermined actual distance, and determines that an abnormality has occurred. Here, the predetermined actual distance is a threshold value for determining whether or not the object candidate area tracked by the actual distance tracking unit 351 is a person, such as an actual distance when a person moves in the monitoring space. As a reference, it is set to a predetermined distance that can remove disturbance, and is arbitrarily set to 100 cm, for example.
Further, the abnormality determination unit 38 determines that the corresponding object candidate region is an intruder when the pixel distance movement amount input from the feature calculation unit 37 is equal to or greater than the predetermined pixel distance, and determines that an abnormality has occurred. The predetermined pixel distance is a threshold value for determining whether or not the object candidate area tracked by the pixel distance tracking unit 352 is a person, such as a pixel distance when a person moves in a distant area in the monitoring space. As a reference, the number of pixels is set such that disturbance can be removed, and is arbitrarily set to, for example, 50 pixels.

In addition, in addition to the determination process using the threshold value, the abnormality determination unit 38 preferably performs abnormality determination when the area of the object candidate region is equal to or larger than a predetermined area. At this time, as the area of the object candidate region, the number of pixels included in the object candidate region is calculated as the object area for the object candidate region located on the far region, and the object candidate region located outside the far region is the object The actual distance between the vertex coordinates of the circumscribed rectangle of the candidate area may be obtained, and the area in the real space may be obtained from this actual distance. Thereby, disturbance can be removed and intruder detection accuracy can be improved.

The output unit 40 is a unit for outputting the abnormality determined by the image processing unit 30 to the outside, and includes an LED, a buzzer, or the like. When an abnormal signal is input from the image processing unit 30, the output unit 40 drives the LED and the buzzer to notify the occurrence of the abnormality.
Note that the output unit 40 may be configured as a communication I / F that is connected to a security device, a remote monitoring center, or the like via a communication line and that sends an abnormal signal input from the image processing unit 30 to the communication line.

Hereinafter, a process of detecting an intruder in the monitoring space from the captured image acquired by the imaging unit 10 using the image sensor 1 will be described with reference to a flowchart illustrated in FIG. 9. By making each step of the flowchart shown in FIG. 9 a control program that can be executed by a computer, it can be executed by a processing device of the computer.

First, the far region setting unit 31 calculates a far region in the captured image from the installation information stored in the storage unit 20 and stores it in the far region information (ST1). Here, as described above, the far region calculates the change rate of the actual distance with respect to the Y-axis coordinate of the captured image, and is located around the vanishing line and the change rate of the actual distance is equal to or greater than the threshold value. Based on the coordinates, it is set as a region extending in a band around the vanishing line.
Next, an image of the monitoring space is acquired by the imaging unit 10. The captured image is output to the image processing unit 30 (ST2).

The variation area extraction unit 32 takes a difference value between the captured image and the background image, generates a difference image, binarizes, and extracts a variation area from the captured image (ST3). If the picked-up image does not include the fluctuation region (ST3-No), the process returns to step ST2, and the picked-up image that becomes the next frame is acquired at the next picked-up image acquisition timing.
On the other hand, if the captured image includes a variation region (ST3-Yes), the object candidate detection unit 33 assigns a unique label to each variation region and calculates the texture similarity with the background image. A variation area having a similarity of not more than a predetermined value is detected as an object candidate area (ST4).

In step ST5, a tracking process for tracking the temporal movement of the object candidate area is performed. This step ST5 is made into a subroutine, and is executed by the presence position determination unit 34 and the object candidate tracking unit 35 according to the flowchart shown in FIG.

When the tracking process is started, first, a differential binary image of the current frame is read from the storage unit 20 (ST501), and it is determined whether or not an object candidate area exists (ST502). If an object candidate area does not exist in the current frame (ST502-No), the process proceeds to step ST523.

On the other hand, if an object candidate region exists in the current frame (ST502-Yes), the counter m is initialized to 1 (ST503). The counter m is used as a label for specifying the object candidate area in the current frame, and is used to associate the object candidate area in the current frame with the object candidate area in the immediately preceding frame in the following processing.

Next, the difference binary image of the immediately preceding frame is read from the storage unit 20 (ST504), and it is determined whether or not an object candidate area exists (ST505). If an object candidate area does not exist in the immediately preceding frame (ST505-No), the process proceeds to step ST525.

If an object candidate area exists in the current frame (ST505-Yes), the counter n is initialized to 1 (ST506). The counter n is used as a label for specifying the object candidate area in the immediately preceding frame, and is used to associate the object candidate area in the immediately preceding frame with the object candidate area in the current frame specified in the counter m in the following processing. It is done.

Next, the existence positions of the object candidate area m of the current frame and the object candidate area n of the immediately preceding frame are determined (ST507).
When both the object candidate region m and the object candidate region n are not present on the far region calculated in step ST1, that is, when located outside the far region (see FIG. 4) (ST507-Yes), the actual distance Tracking processing by actual distance is performed in the tracking unit 351 (ST508).

The actual distance tracking unit 351 sets a tracking search range based on the distance in the real space (actual distance) around the position corresponding to the position where the object candidate region n in the previous frame exists in the current frame (ST509). The actual distance (first predetermined actual distance) set as the tracking search range is set based on the movement speed of the person, and is set to 200 cm, for example, when a captured image is acquired at 5 frames / second. The

Next, based on the actual distance between the barycentric coordinates of the object candidate region n and the object candidate region m calculated by the image distance calculation unit 36, the actual distance tracking unit 351 determines that the object candidate region m of the current frame is the previous frame. It is determined whether or not the object candidate area n is within the search range (ST510).
If the object candidate area m exists in the search range, that is, if the actual distance between the object candidate area n and the object candidate area m is within the search range (ST510-Yes), the object candidate area n and the object candidate The degree of similarity with the area m is calculated (ST511).

The similarity is calculated based on the distance between the object candidate areas, the change rate of the size of both object candidate areas, and the change rate of the aspect ratio. The shorter the distance, the more the change rate of the size and aspect ratio becomes 1. The higher the degree of approximation (the smaller the change), the higher the similarity is calculated. When the calculated similarity is equal to or less than the threshold value (ST511-No), or when the object candidate region m does not exist in the search range of the object candidate region n (ST510-No), the process proceeds to step ST518. On the other hand, if the calculated similarity is greater than or equal to the threshold (ST511-Yes), the calculated similarity is stored in storage unit 20 (ST512), and the process proceeds to step ST518.

In step ST507, if at least one of the object candidate region m and the object candidate region n exists in the far region calculated in step ST1 (see FIG. 7) (ST507-No), the pixel distance The tracking unit 352 performs tracking processing based on the pixel distance (ST513).

The pixel distance tracking unit 352 sets a tracking search range based on the number of pixels (pixel distance) around the position corresponding to the position of the object candidate region n in the previous frame in the current frame (ST514). The pixel distance (first predetermined pixel distance) set as the tracking search range is set to a value that can remove disturbance, and is, for example, 70 pixels when a captured image is acquired at 5 frames / second. Is set.

Next, the pixel distance tracking unit 352 positions the object candidate region m of the current frame within the search range of the object candidate region n of the previous frame based on the pixel distance between the centroids of the object candidate region n and the object candidate region m. It is determined whether or not to perform (ST515).
If the object candidate area m exists in the search range, that is, if the pixel distance between the object candidate area n and the object candidate area m is within the search range (ST515-Yes), the object candidate area n and the object candidate The similarity with the area m is calculated (ST516).

The similarity is calculated in the same manner as in step ST511, and when the calculated similarity is equal to or less than the threshold value (ST516-No), or when the object candidate area m does not exist in the search range of the object candidate area n (ST515). -No) progresses to step ST518. On the other hand, if the calculated similarity is greater than or equal to the threshold value (ST516-Yes), the calculated similarity is stored in storage unit 20 (ST517), and the process proceeds to step ST518.

In ST518, it is determined whether or not there is an object candidate region that has not been subjected to tracking processing in the immediately preceding frame, with respect to the object candidate region m of the current frame. Then, when there is an object candidate area that has not been subjected to tracking processing in the immediately preceding frame with respect to the object candidate area m, that is, when the value of the counter n is less than the total number of object candidate areas in the immediately preceding frame (ST518-Yes). , N is incremented by 1 (ST519), and the processes after step ST507 are repeated.

When the object candidate region m in the current frame has been tracked with all the object candidate regions in the immediately preceding frame (ST518-No), the process proceeds to step ST520. In step ST520, the object candidate area n _m having the highest similarity calculated in the object candidate areas of the immediately preceding frame is determined with reference to the similarity stored in the storage unit 20, and this object candidate area n _{m is determined.} And the object candidate area m of the current frame are labeled in association with each other as areas where the same object is imaged.

Next, the process proceeds to step ST521, where it is determined whether or not there is an object candidate area that has not been subjected to tracking processing in the current frame. When there is an object candidate area that is not tracked in the current frame, that is, when the value of the counter m is less than the total number of object candidate areas in the current frame (ST521-Yes), the value of m is increased by 1 ( (ST522), the process after step ST506 is repeated. When the tracking processing for all object candidate areas is completed in the current frame (ST521-No), the process proceeds to step ST523.

If there is an object candidate area n _{lost that} has not been associated with the object candidate area of the current frame in the previous frame (ST523-Yes), the object reflected in the object candidate area n _lost disappears from the monitoring space. It determines with having carried out and it memorize | stores in the memory | storage part 20 (ST524).
Next, when there is an object candidate area m _{new that} has not been associated with any object candidate area in the previous frame in the current frame (ST525-Yes), the object reflected in the object candidate area m _new is monitored space. The center of gravity coordinates of the object candidate area m _new are stored in the storage unit 20 as appearance position coordinates.
When the processing in the subroutine is completed, the processing is returned to the main routine.

In step ST6, a feature amount calculation process indicating the humanity of the object candidate area labeled as the same object in step ST5 is performed. This step ST6 is converted into a subroutine, and is executed by the feature calculation unit 37 along the flowchart shown in FIG.

When the feature amount calculation process is started, first, the counter m is initialized to 1 (ST601). The counter m is used as a label for specifying an object candidate area in the current frame.
Next, it is determined whether or not the object candidate area m is an object (object candidate area m _new ) that has newly appeared in the monitoring space (ST602). If it is a newly appearing object (ST602-Yes), the process proceeds to ST607.

  On the other hand, if the object candidate area m is not an object newly appearing in the current frame (ST602-No), the appearance position coordinates of the object candidate area m are read from the storage unit 20 (ST603). Then, in the tracking process between the current frame and the immediately preceding frame, it is determined whether the object candidate region m has been tracked at an actual distance or a pixel distance (ST604).

If tracking processing is performed at the actual distance (ST604-Yes), the actual distance between the current center-of-gravity coordinates and the appearance position coordinates of the object candidate region m is calculated and stored in the storage unit 20 as the actual distance movement amount ( ST605).
On the other hand, if tracking processing is performed using the pixel distance (ST604-No), the pixel distance between the current center-of-gravity coordinates and the appearance position coordinates of the object candidate region m is calculated and stored in the storage unit 20 as the pixel distance movement amount. (ST605).

  Next, in ST607, it is determined whether or not there is an object candidate region that has not been subjected to feature amount calculation processing in the current frame. If there is an object candidate area that has not been subjected to feature quantity calculation processing in the current frame, that is, if the value of the counter m is less than the total number of object candidate areas in the current frame (ST607-Yes), the value of m is increased by 1. (ST608), and the processes after step ST602 are repeated. When the feature amount calculation processing for all object candidate regions is completed in the current frame (ST607-No), the processing in the subroutine is terminated and the processing is returned to the main routine.

In step ST7, the abnormality determination unit 38 determines whether the object reflected in the object candidate area is an intruder based on the feature amount calculated by the feature amount calculation process. The abnormality determination unit 38 determines that the corresponding object candidate region is an intruder when the actual distance movement amount calculated in the feature amount calculation process is equal to or greater than a predetermined actual distance (second predetermined actual distance). (ST7-Yes). The predetermined actual distance is a threshold value for determining whether or not the object candidate area tracked by the actual distance tracking unit 351 is a person, and the actual distance when the person moves in the monitoring space is used as a reference. It is set to a value that can remove the disturbance, and is arbitrarily set to 100 cm, for example.

In addition, the abnormality determination unit 38 determines that the corresponding object candidate region is an intruder when the pixel distance movement amount calculated in the feature amount calculation process is equal to or greater than a predetermined pixel distance (second predetermined pixel distance). Determine (ST7-Yes). The predetermined pixel distance is a threshold value for determining whether or not the object candidate area tracked by the pixel distance tracking unit 352 is a person, such as a pixel distance when a person moves in a distant area in the monitoring space. Is set to a value that can remove disturbance, and is arbitrarily set to 50 pixels, for example.

In ST8, based on the determination of the abnormality determination unit 38, the output unit 40 operates to notify the occurrence of the abnormality.
When the abnormality determination unit 38 determines that there is no intruder (ST7-No) or after the output unit 40 notifies the occurrence of abnormality, the process returns to step ST2 and becomes the next frame at the next captured image acquisition timing. A captured image is acquired.

  As described above, according to the present embodiment, in the distant area reflected in the captured image, the far distance reflected in the captured image is based on the knowledge that the reliability of the calculated distance in the real space is low. The movement distance is calculated based on the number of pixels for the fluctuation area extracted in the far area, and based on the distance in the real space for the fluctuation area extracted outside the far area. By determining the moving distance, it is possible to improve the detection accuracy of an intruder even when the vicinity of the imaging apparatus to the far side is used as a monitoring space.

1 is a block diagram of an image sensor according to an embodiment of the present invention. The figure explaining the distant area | region set to the captured image which concerns on embodiment of this invention. The figure explaining the production | generation method of the difference image which concerns on embodiment of this invention. The figure which shows the moving object outside a distant area | region which concerns on embodiment of this invention. The figure explaining the method of calculating the distance in real space which concerns on embodiment of this invention. The figure explaining the method of calculating the distance in real space which concerns on embodiment of this invention. The figure which shows the moving object on the far field which concerns on embodiment of this invention. The figure explaining the calculation method of a far field concerning an embodiment of the invention. The flowchart of the intruder detection which concerns on embodiment of this invention. The flowchart of the tracking process which concerns on embodiment of this invention. The flowchart of the feature-value calculation process which concerns on embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image sensor, 10 Image pick-up part, 20 Storage part, 30 Image processing part, 31 Far area setting part, 32 Fluctuation area extraction part, 33 Object candidate detection part, 34 Existence position determination part, 35, Object candidate tracking part, 351 Distance tracking unit, 352 Pixel distance tracking unit, 36 Image distance calculation unit, 37 Feature calculation unit, 38 Abnormality determination unit, 40 Output unit

Claims (3)

An image sensor that extracts a fluctuation region from a difference between a captured image of a monitoring space sequentially acquired by an imaging unit and a past image and tracks the movement of the fluctuation region,
A distant area setting unit for setting an area in which the distant part of the monitoring space is reflected in the captured image as a distant area;
A storage unit that stores installation information including the installation height, depression angle, and angle of view of the imaging unit, the far region, and the position of the extracted variation region;
A presence position determination unit that determines whether or not the variable region is located in the far region;
An image distance calculation unit that converts a distance on the captured image into an actual distance on the monitoring space based on the installation information;
If the presence position determination unit determines that at least one position of the fluctuation area extracted before and after in time is within the far area, the distance on the captured image between the fluctuation areas is set as a matching condition, If the presence position determination unit determines that both positions of the variable region extracted before and after in time are outside the far region, the movement of the variable region is tracked using the actual distance between the variable regions as a matching condition. A tracking unit;
Equipped with a,
The far region setting unit sets the region where the actual distance between adjacent pixels exceeds a preset reference in the captured image as the far region . The image sensor according to claim 1, wherein the far region setting unit sets a region where an actual distance between the adjacent pixels is equal to or greater than a first threshold value as the far region. The far region setting unit, an image sensor according to claim 1, wherein calculating the actual distance rate of change between adjacent pixels, said alteration rate sets a region is the second threshold or more as the far region .

Images