JP3894038B2 - Image processing apparatus, protrusion detection method and program for image processing apparatus - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus that captures a moving image of a person, and more particularly, to an image processing apparatus that detects that a person protrudes from a captured image.
[0002]
[Prior art]
In order to leave a moving image as a record, a moving image may be captured by an image capturing device such as a video camera. An image captured by an image capturing device for leaving a moving image as a record often includes a person or a person's face. It is desirable that the person and his / her face do not protrude from the image captured by the image capturing device. However, when shooting a moving image, the person to be shot is not always stationary, and usually moves constantly. Therefore, it is conceivable that a person or face protrudes from the field of view. Therefore, it is desirable that the shooting direction and shooting range of the video camera be corrected according to the movement of the person.
[0003]
Conventionally, when a moving image is shot by an image shooting device, an operator who operates the image shooting device takes a picture so that the person does not protrude from the field of view or the person who has protruded is placed in the field of view again. The shooting direction was adjusted by panning and tilting according to the movement of the target person. A conventional video camera described in Japanese Patent Laid-Open No. 01-243673 has a driving device for panning or tilting in a grip. Therefore, an operator of a conventional video camera can easily perform panning or tilting without using an electric pan head device.
[0004]
[Problems to be solved by the invention]
In recent years, image recognition devices that recognize people and their faces from moving images taken with a video camera have been applied to human authentication for security and criminal investigations using criminal images taken with a security camera. ing.
[0005]
Some image recognition apparatuses recognize a person by comparing an image of a person photographed by a video camera with an image of a person recorded in advance. For example, a conventional face image matching device described in Japanese Patent Application Laid-Open No. 2000-331167 compares two face images and evaluates the degree of coincidence between two faces appearing in the face image. A conventional face image matching device first normalizes a face image in order to match the size of the face appearing in the face image and the position of the face in each face image. Next, the face image matching device extracts a face area by masking a predetermined area of two face images based on predetermined mask data. Next, the face image collation device converts each face image into a monochrome image as necessary. Furthermore, the face image collation device collates after reducing the gradation of each face image. In this way, the conventional image recognition apparatus recognizes a person or a face from an image taken with a video camera.
[0006]
However, since a video camera is restricted by the lens system image acquisition range and the size of an image that can be output from the photoelectric conversion device, there is a finite field of view in an image that can be captured by the video camera. Since the image recognition apparatus recognizes a person or face from an image taken by a video camera, the person or face must be in the field of view in order to recognize the person or face normally.
[0007]
However, when shooting a moving image including a person with a video camera, the person is not necessarily stationary. Therefore, the person who is the recognition target of the image recognition apparatus is not always in the field of view of the video camera. Therefore, it is desired that the image recognition apparatus corrects the shooting direction and shooting range of the video camera in accordance with the movement of the person.
[0008]
As a method for correcting the shooting direction and the shooting range, there has conventionally been a method in which an operator of a video camera observes the movement of a person to be shot and pans or tilts accordingly.
[0009]
However, an image recognition apparatus applied to personal authentication and security camera for ensuring security is not always operated by an operator, and needs to operate even without an operator. Therefore, when the conventional image recognition apparatus cannot recognize a person, it cannot know whether or not it is due to the protrusion from the image. For this reason, conventional image recognition devices cannot automatically correct the shooting direction or shooting range when a person or their face protrudes from the image, and warn that a person protrudes from the image. I couldn't.
[0010]
For this reason, the conventional image recognition apparatus is installed so as to fixedly shoot an appropriate range in which a person and his / her face will enter the image, so that the person does not protrude from the image as much as possible.
[0011]
An object of the present invention is to provide an image processing apparatus that can detect that a person or a person's face protrudes from an image taken by a video camera, and a protrusion detection method and program thereof.
[0012]
[Means for Solving the Problems]
In order to achieve the above object, an image processing apparatus of the present invention is provided. The person An image processing apparatus for detecting an overhang from a captured image of an object, the image processing unit continuously acquiring images including the person, and a frame for calculating a difference of each pixel between the images acquired by the image capturing unit For each pixel column in the vertical direction of the image, a difference value calculated by the inter-frame difference calculation unit is determined to obtain a pixel closest to the upper end of the image among pixels having a predetermined value or more, and the pixel A horizontal curve is drawn by connecting the pixels obtained for each column, and the image area is vertically divided into a plurality of divided areas at a point where the curve reaches the lower end of the image and a point where the curve reaches a downward peak. And a divided area having the largest number of pixels having a difference value equal to or larger than a predetermined value among the plurality of divided areas obtained by the area dividing means as a person area where the person exists Area selection means to select and previous Determining means for determining that the person protrudes from the image if there are more pixels than the predetermined threshold near the image edge of the image within the person area, the difference value being greater than or equal to a predetermined value; Have.
[0057]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described in detail with reference to the drawings.
(First embodiment)
The first embodiment of the present invention is an image processing device that detects that a person protrudes from an image based on changes in the image of the upper, lower, left, and right portions of the image.
[0058]
FIG. 1 is a block diagram illustrating a configuration of the image processing apparatus according to the first embodiment. Referring to FIG. 1, the image processing apparatus 10 includes an imaging unit 11, an inter-frame difference calculation unit 12, a binarization unit 13, an image upper pixel number unit 14, an image right pixel count unit 15, and an image left pixel number. A count unit 16, an image lower pixel number count unit 17, and a determination unit 18 are included.
[0059]
The imaging unit 11 is, for example, a video camera, and continuously captures images one after another and digitizes the captured images to obtain data for each pixel. In the present embodiment, it is assumed that the captured image is a monochrome image as an example. Therefore, the data for each pixel is indicated by a numerical value representing luminance.
[0060]
The inter-frame difference calculation unit 12 receives the image digitized by the imaging unit 11 and stores a predetermined number of past images in a memory (not shown), and stores the latest image and the past image stored in the memory. The difference from each image is calculated for each pixel. The difference of each pixel is a difference between the luminance of the pixel in the latest image and the luminance of the pixel in the past image. The number of past images stored in the memory is at least one, and may be plural, for example, set to be changeable as a parameter.
[0061]
The binarization unit 13 compares the difference of each pixel calculated by the inter-frame difference calculation unit 12 with a predetermined difference threshold value, and binarizes it according to the magnitude of the threshold value. Note that the difference between the pixels being larger than the difference threshold indicates that the pixel has a large difference from the past image, that is, the change of the pixel is large. Here, the difference of each pixel is binarized to “1” when it is equal to or greater than the difference threshold, and is binarized to “0” when it is smaller than the difference threshold.
[0062]
In the first embodiment, the portions near the upper, lower, left, and right sides of the image captured by the imaging unit 11 are defined as an upper region, a lower region, a left region, and a right region. 2 shows an upper area 91 in the image 90, FIG. 3 shows a left area 92, FIG. 4 shows a right area 93, and FIG. 5 shows a lower area 94.
[0063]
The image upper pixel number counting unit 14 counts the number of pixels in the upper region 91 that are equal to or higher than the difference threshold by the binarizing unit 13, that is, the number of pixels binarized to “1”. .
[0064]
The image right pixel count counting unit 15 counts the number of pixels in the right region 93 that are equal to or greater than the difference threshold by the binarization unit 13.
[0065]
The image left pixel number counting unit 16 counts the number of pixels in the left region 92 that are equal to or greater than the difference threshold by the binarization unit 13.
[0066]
The image lower pixel number counting unit 17 counts the number of pixels in the lower region 94 that are equal to or greater than the difference threshold by the binarizing unit 13.
[0067]
If the count value of the upper image pixel count unit 14 is larger than a predetermined upper count threshold value, the determination unit 18 determines that a person is protruding from the image. The determination unit 18 determines that the person protrudes to the right of the image if the count value of the image right pixel number counting unit 15 is greater than a predetermined right part count threshold. The determination unit 18 determines that the person protrudes to the left of the image if the count value of the image left pixel number counting unit 16 is larger than a predetermined left part count threshold value. If the count value of the image lower pixel number counting unit 17 is larger than a predetermined lower count threshold value, the determining unit 18 determines that the person protrudes below the image.
[0068]
FIG. 6 is a flowchart showing the operation of the image processing apparatus of this embodiment. Referring to FIG. 6, first, the imaging unit 11 captures an image and sends the image to the inter-frame difference calculation unit 12 (step A1). Here, it is assumed that an image captured t-th after the video camera of the imaging unit 11 is activated is im (t).
[0069]
Next, the inter-frame difference calculation unit 12 calculates a difference between each pixel of the latest image im (t) and each pixel of the past image stored in the memory, and each difference is set as the value of each pixel. The difference image diff (t) to be generated is generated (step A2).
[0070]
Next, the inter-frame difference image calculation unit 13 stores the image im (t) in the memory as a past image (step A3). At the same time, the inter-frame difference image calculation unit 13 discards one oldest past image stored in the memory.
[0071]
Next, the binarizing unit 14 compares the value of each pixel (the above-described difference) of the difference image diff (t) with the difference threshold value, binarizes the difference image diff (t), and binarizes the binarized image bin. Conversion into (t) (step A4). At this time, the binarization unit 14 sets the pixel value to “1” when the value of each pixel is equal to or greater than the difference threshold. Also, the binarization unit 14 sets the pixel value to “0” when the value of each pixel is smaller than the difference threshold.
[0072]
Next, the image upper pixel number counting unit 14 counts the number of pixels having a pixel value “1” in the upper region 91 of the binarized image bin (t) (step A5). Here, the count value is a (t).
[0073]
Next, the image right pixel number counting unit 15 counts the number of pixels having a pixel value “1” in the right region 92 of the binarized image bin (t) (step A6). Here, the count value is b (t).
[0074]
Next, the image left pixel number counting unit 16 counts the number of pixels having a pixel value “1” in the left region 93 of the binarized image bin (t) (step A7). Here, the count value is c (t).
[0075]
Next, the image lower pixel number counting unit 17 counts the number of pixels having a pixel value “1” in the lower region 94 of the binarized image bin (t) (step A8). Here, the count value is d (t).
[0076]
Next, the determination unit 18 determines whether or not the count value a (t) is larger than the upper count threshold th1 (step A9). If the count value a (t) is larger than the upper count threshold th1, the determination unit 18 detects that a person protrudes from the image (step A10).
[0077]
Next, the determination unit 18 determines whether or not the count value b (t) is greater than the right count threshold th2 (step A11). If the count value b (t) is larger than the right part count threshold th2, the determination part 18 detects that a person protrudes to the right of the image (step A12).
[0078]
Next, the determination unit 18 determines whether or not the count value c (t) is greater than the left count threshold th3 (step A13). If the count value c (t) is larger than the right part count threshold th3, the determination unit 18 detects that the person protrudes to the left of the image (step A14).
[0079]
Next, the determination unit 18 determines whether or not the count value d (t) is greater than the lower count threshold th4 (step A15). If the count value d (t) is larger than the right part count threshold th4, the determination unit 18 detects that a person protrudes under the image (step A16).
[0080]
When the above operation is completed, the image processing apparatus 10 increments “t” by one (step A17), and returns to step A1.
[0081]
As described above, the image processing apparatus 10 according to the present embodiment calculates the difference between the latest image and the past image by the inter-frame difference calculation unit 12, and binarizes the difference by the binarization unit 13. The number of pixels that have changed more than a predetermined number is counted for each of the upper, lower, left, and right areas by determining whether or not the above has changed, by the upper, right, left, or lower pixel number counting units 14, 15, 16, and 17, The determination unit 18 compares the count value of each region with the threshold value and determines that a person has protruded from the image in the region where the count value exceeds the threshold value. Therefore, the region where the count value exceeds the threshold value due to a large image change is determined. By detecting this, it is possible to automatically detect the protrusion of a person from the image.
[0082]
In the present embodiment, the case where the imaging unit 11 captures a monochrome image has been illustrated, but the present invention is not limited to this and may be a color image. When a color image is captured by the imaging unit 11, for example, the inter-frame difference calculation unit 12 represents each pixel value as a vector whose elements are RGB three colors, and the difference between the pixel of the past image and the vector. Ask for. And the binarization part 13 may produce | generate a binarized image by comparing the magnitude | size of the difference vector calculated | required by the interframe difference calculation part 12 with a difference threshold value.
[0083]
Further, in the present embodiment, the case where the object for detecting the protrusion from the image is a person is exemplified, but the present invention is not limited to this. Any object can be used as long as it can move when continuously photographed.
[0084]
In addition, by adjusting parameters such as the number of past images, each threshold, and the size of each of the upper, lower, left, and right areas used in the present embodiment, the accuracy with which the image processing apparatus detects the protrusion is improved. Can do.
(Second Embodiment)
FIG. 7 is a block diagram showing the configuration of the image processing apparatus according to the second embodiment of the present invention. Referring to FIG. 7, the image processing apparatus 20 includes an imaging unit 21, an inter-frame difference calculation unit 22, a binarization unit 23, an area division unit 24, an area selection unit 25, an image upper pixel number counting unit 26, and an image right part. A pixel number counting unit 27, an image left pixel number counting unit 28, an image lower pixel number counting unit 29, and a determination unit 210 are provided.
[0085]
The imaging unit 21 is a video camera, for example, and continuously captures images one after another, and digitizes the captured images to obtain data for each pixel. In the present embodiment, it is assumed that the captured image is a monochrome image as an example. Therefore, the data for each pixel is indicated by a numerical value representing luminance.
[0086]
The inter-frame difference calculation unit 22 receives an image digitized by the imaging unit 21 and stores a predetermined number of past images in a memory (not shown), and stores the latest image and the past image stored in the memory. The difference from each image is calculated for each pixel. The difference of each pixel is a difference between the luminance of the pixel in the latest image and the luminance of the pixel in the past image. The number of past images stored in the memory is at least one, and may be plural, for example, set to be changeable as a parameter.
[0087]
The binarization unit 23 compares the difference of each pixel calculated by the inter-frame difference calculation unit 22 with a predetermined difference threshold value, and binarizes the difference based on the magnitude of the threshold value. Note that the difference between the pixels being larger than the difference threshold indicates that the pixel has a large difference from the past image, that is, the change of the pixel is large. Here, the difference of each pixel is binarized to “1” when it is equal to or greater than the difference threshold, and is binarized to “0” when it is smaller than the difference threshold.
[0088]
The area dividing unit 24 divides the image into a plurality of divided areas. A method for dividing an image into a plurality of divided regions will be described later.
[0089]
The area selection unit 25 selects a divided area that is most likely to be a person from among a plurality of divided areas divided by the area dividing unit 24, and further converts the binarized image according to the selected divided area. The method for selecting the most likely divided area and the method for converting the binarized image will be described later.
[0090]
In the second embodiment, similarly to the first embodiment, the upper, right, left, and lower regions 93, 94, 95, and 96 shown in FIGS. 2 to 5 are defined.
[0091]
The image upper pixel number counting unit 26 counts the number of pixels having a pixel value “1” in the upper region 91 in the binarized image converted by the region selecting unit 25.
[0092]
The image right pixel count counting unit 27 counts the number of pixels having a pixel value “1” in the right region 93 in the binarized image converted by the region selection unit 25.
[0093]
The image left pixel number counting unit 28 counts the number of pixels having a pixel value “1” in the left region 92 in the binarized image converted by the region selecting unit 25.
[0094]
The image lower pixel number counting unit 29 counts the number of pixels having a pixel value “1” in the lower region 94 in the binarized image converted by the region selecting unit 25.
[0095]
If the count value of the image upper pixel number counting unit 26 is larger than a predetermined upper count threshold value, the determining unit 210 determines that a person protrudes from the image. Further, the determination unit 210 determines that the person protrudes to the right of the image if the count value of the image right pixel number counting unit 27 is larger than a predetermined right part count threshold. The determination unit 210 determines that the person protrudes to the left of the image if the count value of the image left pixel number counting unit 28 is larger than a predetermined left part count threshold value. If the count value of the image lower pixel number counting unit 29 is larger than a predetermined lower count threshold value, the determination unit 210 determines that the person protrudes below the image.
[0096]
FIG. 8 is a flowchart showing the operation of the image processing apparatus of this embodiment. Referring to FIG. 8, first, the imaging unit 21 captures an image and sends it to the inter-frame difference calculation unit 22 (step B1). Here, it is assumed that an image captured t-th after the video camera of the imaging unit 11 is activated is im (t).
[0097]
Next, the inter-frame difference calculation unit 22 calculates a difference between each pixel of the latest image im (t) and each pixel of the past image stored in the memory, and each difference is set as the value of each pixel. The difference image diff (t) to be generated is generated (step B2).
[0098]
Next, the inter-frame difference image calculation unit 22 stores the image im (t) in the memory as a past image (step B3). At the same time, the inter-frame difference image calculation unit 22 discards one oldest past image stored in the memory.
[0099]
Next, the binarization unit 23 compares the value of each pixel of the difference image diff (t) (the difference described above) with the difference threshold value to binarize the difference image diff (t), and binarizes the binarized image bin. Conversion to (t) (step B4). At this time, the binarization unit 23 sets the pixel value to “1” when the value of each pixel is equal to or greater than the difference threshold. The binarization unit 23 sets the pixel value to “0” when the value of each pixel is smaller than the difference threshold.
[0100]
Next, the region dividing unit 24 divides the binarized image bin (t) obtained by the binarizing unit 23 into a plurality of divided regions (step B5).
[0101]
An example of a region dividing method by the region dividing unit 24 will be described. FIG. 9 is a diagram for explaining a region dividing method. As shown in FIG. 9, the horizontal direction of the binarized image bin (t) is the X axis (rightward is positive), the vertical direction is Y axis (downward is positive), and the xth pixel from the upper left pixel of the image to the right Then, let the pixel value of the yth pixel be v (x, y) below.
[0102]
When a binarized image bin (t) as shown in FIG. 9A is obtained, as shown in FIG. 9B, v (x, y) becomes 1 for all x constituting the image. Find the minimum y value. Next, a point at which the obtained y value becomes a maximum with respect to the change of x (maximum point) and a point reaching the lower side of the image (lower side contact) are obtained, and a perpendicular passing through the maximum point and the lower side contact is obtained. The image is divided into a plurality of divided areas by lines.
[0103]
As another example of the region dividing method, a method in which pixels having the same pixel value (“1” or “0”) and connected to each other are grouped into one divided region can be considered. That is, a portion where pixels having a pixel value “1” are connected (a group of pixel values “1”) is defined as one divided region, and similarly, a portion where pixels having a pixel value “0” are connected (pixel value). A group of “0”) is defined as one divided area.
[0104]
Next, the area selection unit 25 selects a divided area that is most likely to be a person from the divided areas obtained by the area dividing unit 24. Then, the area selection unit 25 creates a binarized image sbin (t) that is converted so as to have the same value as bin (t) in the divided area and “0” in the other divided areas. (Step B6).
[0105]
As an example of a method for selecting a region most likely to be a person by the region selection unit 25, among the divided regions obtained by the region dividing unit 24, a divided region having the largest number of pixels having a pixel value “1” is selected. A method of selecting is conceivable.
[0106]
Next, the image upper pixel number counting unit 26 counts the number of pixels having a pixel value “1” in the upper region 91 of the converted binary image sbin (t) (step B7). Here, the count value is a (t).
[0107]
Next, the image right pixel number counting unit 27 counts the number of pixels having a pixel value “1” in the right region 92 of the converted binarized image sbin (t) (step B8). . Here, the count value is b (t).
[0108]
Next, the image left pixel number counting unit 28 counts the number of pixels having a pixel value “1” in the left region 93 of the converted binarized image sbin (t) (step B9). . Here, the count value is c (t).
[0109]
Next, the image lower pixel number counting unit 29 counts the number of pixels having a pixel value “1” in the lower region 94 of the converted binary image sbin (t) (step B10). Here, the count value is d (t).
[0110]
Next, the determination unit 210 determines whether or not the count value a (t) is greater than the upper count threshold th1 (step B11). If the count value a (t) is larger than the upper count threshold th1, the determination unit 210 detects that a person is protruding from the image (step B12).
[0111]
Next, the determination unit 210 determines whether or not the count value b (t) is greater than the right count threshold th2 (step B13). If the count value b (t) is larger than the right-side count threshold th2, the determination unit 210 detects that a person protrudes to the right of the image (step B14).
[0112]
Next, the determination unit 210 determines whether or not the count value c (t) is greater than the left count threshold th3 (step B15). If the count value c (t) is greater than the right count threshold th3, the determination unit 210 detects that a person protrudes to the left of the image (step B16).
[0113]
Next, the determination unit 210 determines whether or not the count value d (t) is greater than the lower count threshold th4 (step B17). If the count value d (t) is larger than the right count threshold th4, the determination unit 210 detects that a person is protruding below the image (step B18).
[0114]
When the above operation is completed, the image processing apparatus 20 increments “t” by 1 (step B19), and returns to step B1.
(Third embodiment)
The third embodiment is an image processing apparatus that detects that a person is too close to a video camera and protrudes from an image.
[0115]
FIG. 10 is a block diagram illustrating a configuration of the image processing apparatus according to the third embodiment. Referring to FIG. 10, the image processing device 30 includes a first imaging unit 31, a second imaging unit 32, an image center part correlation calculation unit 33, a reliability determination unit 34, a distance calculation unit 35, and a distance determination unit 36. Have.
[0116]
The first imaging unit 31 is, for example, a video camera, and continuously captures images one after another and digitizes the captured images to obtain data for each pixel.
[0117]
Similar to the first imaging unit 31, the second imaging unit 32 continuously captures images one after another, digitizes the captured images, and obtains data for each pixel. In addition, the field of view for capturing the image of the second imaging unit 32 has a portion that overlaps the field of view of the first imaging unit 31 and has a relative positional relationship with the field of view of the first imaging unit 31. It is fixed.
[0118]
The image center correlation calculation unit 33 calculates a correlation value at the image center 95 (see FIG. 12) of images obtained from the first imaging unit 31 and the second imaging unit 32 almost simultaneously.
[0119]
The reliability determination unit 34 determines that the reliability is low if the change of the correlation value obtained by the image center portion correlation calculation unit 33 with respect to the coordinate position is gentle, and if the change is sudden, the reliability is high. Judge.
[0120]
The distance calculation unit 35 calculates the central portion of the image from the positional relationship between the first imaging unit 31 and the second imaging unit 32 and the shift amount when the correlation value between the two images is maximized by shifting the two images. The distance to the object shown in 95 is calculated.
[0121]
If the distance obtained by the distance calculating unit 35 is equal to or less than a predetermined threshold and the reliability determining unit 34 determines that the reliability is high, the distance determining unit 36 is too close to detect the person from the image. Judge that it protrudes.
[0122]
FIG. 11 is a flowchart showing the operation of the image processing apparatus of this embodiment. Referring to FIG. 11, first, the first and second imaging units 31 and 32 respectively capture images and send them to the image center correlation calculation unit 33 (step C1). Assume that the first imaging unit 31 and the second imaging unit 32 are activated at the same time, and images taken t-th after the video camera is activated are im1 (t) and im2 (t), respectively.
[0123]
Next, the image center part correlation calculating unit 33 calculates the correlation value corr (x) at the image center part 95 of the images im1 (t) and im2 (t) (step C2).
[0124]
An example of a correlation value calculation method will be described.
[0125]
Here, the horizontal direction of the image im1 (t) is the X axis (rightward is positive), the vertical direction is the Y axis (downward is positive), and the xth pixel to the right and the yth pixel to the right from the upper left pixel of the image. Let the value be v1 (x, y). Similarly, v2 (x, y) is defined for the image im2 (t). Further, it is assumed that the image center portion 95 is set as shown in FIG.
[0126]
If the set of coordinate values of the image central portion 95 is C, the correlation value is corr (x) = Σ _ {(t, s) ¥ in C} | v1 (t−x, s) −v2 (t, s ) |
[0127]
Next, the reliability determination unit 34 determines whether or not the change in the correlation value corr (x) is gentle (step C3).
[0128]
An example of a method for determining whether or not the correlation value corr (x) is gentle will be described. Let x1 be the x that maximizes the correlation value corr (x). At this time, if corr (x1) −corr (x1−g) <thc1 and corr (x1) −corr (x1 + g) <thc2 in the entire area of the image central portion 95, it is determined that corr (x) is gentle. It is judged that it is not gentle.
[0129]
If the change in the correlation value corr (t) is not gentle, the distance calculation unit 35 determines the positional relationship between the first imaging unit 31 and the second imaging unit 32 and the x of the correlation value corr (x) that maximizes the correlation value corr (x). The distance d (t) is obtained from the value (step C4). The method for calculating the distance may be a well-known method. For example, it can be realized by using a method described in the document “3D Vision” (Xu, Tsuji, Kyoritsu Publishing (1998)).
[0130]
Next, the distance determination unit 36 determines whether or not the distance d (t) is greater than a predetermined threshold th (step C5). The threshold th is calculated in advance from the size of the object (here, a person), the focal length of the lens, and the size of the image, the distance that the object will protrude from the image. If the distance d (t) is less than or equal to the threshold th, the distance determination unit 36 detects that a person is protruding from the image (step C6).
[0131]
After Step C6, when the distance d (t) is larger than the threshold th in Step C5 or when the change is gentle in Step C3, the image processing apparatus 30 increments “t” by one (Step C7), returning to step C1.
[0132]
As described above, the image processing apparatus 30 according to the present embodiment simultaneously captures two images that are shifted from each other with the first and second imaging units 31 and 32, and the image center portion correlation calculation unit 33 performs 2 The correlation value at the image central portion 95 of one image is calculated, and the reliability determination unit 34 determines whether the reliability is high or low from the correlation value. When the reliability is high, the distance calculation unit 35 performs the video camera operation. And the distance determination unit 36 automatically determines that the person protrudes from the image if the distance is too short. can do.
[0133]
In the present embodiment, the case where the target for detecting the protrusion from the image is a person, but the present invention is not limited to this. Any thing can be used as long as a certain size can be assumed.
(Fourth embodiment)
FIG. 13 is a block diagram showing a configuration of an image processing apparatus according to the fourth embodiment of the present invention. Referring to FIG. 13, the image processing apparatus 40 includes a first imaging unit 41, a second imaging unit 42, a small region correlation calculation unit 43, a distance calculation unit 44, a minimum distance selection unit 45, and a distance determination unit 46. is doing.
[0134]
The first imaging unit 41 is, for example, a video camera, and continuously captures images one after another and digitizes the captured images to obtain data for each pixel.
[0135]
Similar to the first imaging unit 41, the second imaging unit 42 captures images one after another and digitizes the captured images to obtain data for each pixel. The field of view for capturing the image of the second imaging unit 42 has a portion that overlaps the field of view of the first imaging unit 41, and the field of view for capturing the image of the second imaging unit 42 is the first imaging. The relative positional relationship with the visual field of the part 41 is fixed.
[0136]
The small area correlation unit 43 calculates the correlation values in the plurality of small areas 96 a, 96 b, 96 c, and 96 d of the image center part 96 of the image obtained from the first imaging unit 41 and the second imaging unit 42 almost simultaneously. Calculate (see FIG. 15).
[0137]
For each of the small regions 96a, 96b, 96c, and 96d, the distance calculation unit 44 shifts the position of the first image capturing unit 41 and the second image capturing unit 42 and shifts the two images to maximize the correlation value between the two images. The distance from the object shown in each small area is calculated from the amount of deviation when.
[0138]
The minimum distance selection unit 45 selects the smallest one of the distances of the small areas calculated by the distance calculation unit 44.
[0139]
If the distance of the small area selected by the minimum distance selection unit 45 is equal to or smaller than a predetermined threshold, the distance determination unit 46 determines that the person is protruding from the image because the distance is too close.
[0140]
FIG. 14 is a flowchart showing the operation of the image processing apparatus of this embodiment. Referring to FIG. 14, first, the first and second imaging units 41 and 42 respectively capture images and send them to the small region correlation calculation unit 43 (step D1). It is assumed that the first imaging unit 41 and the second imaging unit 42 are activated at the same time, and images taken t-th after the video camera is activated are im1 (t) and im2 (t), respectively.
[0141]
Next, the small area correlation calculation unit 43 calculates correlation values in the small areas 96a, 96b, 96c, and 96d of the images im1 (t) and im2 (t), and the distance calculation unit 44 calculates the small areas. The distance between the video camera and the object in 96a, 96b, 96c, 96d is calculated (step D2). The calculation method of the correlation value and the distance used at this time is the same method as in the third embodiment.
[0142]
Next, the distance determination unit 46 determines whether or not the shortest distance among the distances in each small region is equal to or less than a predetermined threshold th (step D3). As in the third embodiment, the threshold th is calculated in advance from the size of the object (here, a person), the focal length of the lens, and the size of the image. It is what I left. If the distance is equal to or smaller than the threshold th, the distance determination unit 46 detects that the person is protruding from the image (step D4).
[0143]
After step D4 or when the distance is not less than or equal to the threshold th in step D3, the image processing device 40 increments “t” by one (step D5) and returns to step D1.
[0144]
【The invention's effect】
According to the present invention, the inter-frame difference calculating means calculates the difference of each pixel indicating the change of the image from the images continuously taken by the imaging means, and the image is calculated in the region near the upper, lower, left and right sides of the image. If the change is large, the determination means determines that the object is protruding from the image, so that it is possible to automatically detect that the object has protruded from the image due to deviation in the vertical and horizontal directions.
[0145]
In addition, the inter-frame difference calculating means calculates the difference of each pixel indicating the change of the image from the images continuously captured by the imaging means, and the area dividing means divides the image into a plurality of divided areas. If the selection means selects a divided area that is most likely to be an object, and the determination means is based on the image of the divided area that is most likely to be an object, if the change in the image is large in areas near the top, bottom, left, and right sides of the image, Since it is determined that the object protrudes from the image, it is possible to automatically and reliably detect the protrusion of the object from the image.
[0146]
Further, from the image captured by the imaging unit, the distance calculating unit calculates the distance to the object captured at the center of the image, which is highly likely to be an object, and the reliability determining unit is calculated. When the reliability of the distance is obtained, and as a result, the reliability is high and the distance is too close, the determination means determines that the object is protruding from the image, so the object is too close and protrudes from the image. Can be automatically detected.
[0147]
Further, the image center correlation calculation means calculates the correlation value of these images from the two images taken by the first and second imaging means and shifted from each other, and the distance calculation means knows from the correlation value. The distance to the object is calculated from the amount of deviation between the two images. As a result, when the distance is too close, the determination means determines that the object protrudes from the image. It is possible to automatically detect that the image protrudes from the image.
[0148]
Further, the image center correlation calculation means calculates the correlation value of these images from the two images taken by the first and second imaging means and shifted from each other, and the distance calculation means knows from the correlation value. From the amount of deviation between the two images, the distance to the object is calculated by using the object in each small area at the center of the image and having the smallest distance as a target. However, since it is determined that the object protrudes from the image, it can be automatically and reliably detected that the object protrudes from the image because the object is too close.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a configuration of an image processing apparatus according to a first embodiment.
FIG. 2 is a diagram illustrating an upper region in an image.
FIG. 3 is a diagram illustrating a left region in an image.
FIG. 4 is a diagram illustrating a right region in an image.
FIG. 5 is a diagram illustrating a lower region in an image.
FIG. 6 is a flowchart illustrating an operation of the image processing apparatus according to the first embodiment.
FIG. 7 is a block diagram showing a configuration of an image processing apparatus according to a second embodiment of the present invention.
FIG. 8 is a flowchart illustrating an operation of the image processing apparatus according to the second embodiment.
FIG. 9 is a diagram for explaining a region dividing method;
FIG. 10 is a block diagram illustrating a configuration of an image processing apparatus according to a third embodiment.
FIG. 11 is a flowchart illustrating an operation of the image processing apparatus according to the third embodiment.
FIG. 12 is a diagram showing an image center portion in an image.
FIG. 13 is a block diagram illustrating a configuration of an image processing apparatus according to a fourth embodiment.
FIG. 14 is a flowchart illustrating an operation of the image processing apparatus according to the fourth embodiment.
FIG. 15 is a diagram illustrating each small region in the center of an image in the image.
[Explanation of symbols]
10, 20, 30 Image processing apparatus
11, 21 Imaging unit
12, 22 Difference calculation unit between frames
13, 23 Binarization part
14, 26 Image upper pixel part
15, 27 Image right side pixel number counting part
16, 28 Image left pixel count section
17, 29 Image lower pixel count section
18,210 Judgment part
24 Area division part
25 Area selection part
31, 41 First imaging unit
32, 42 Second imaging unit
33 Image center correlation calculation unit
34 Reliability Judgment Department
35, 44 Distance calculator
36, 46 Distance judgment part
43 Small region correlation calculator
45 Minimum distance selector
90 images
91 Upper area
92 Left area
93 Right area
94 Lower area
95,96 Image center
96a, 96b, 96c, 96d Small area
A1-A17, B1-B19, C1-C7 steps

Claims (3)

An image processing device for detecting a protrusion of a person from a captured image,
Imaging means for continuously acquiring images including the person;
Inter-frame difference calculating means for calculating a difference of each pixel between images acquired by the imaging means;
For each pixel row in the vertical direction of the image, a pixel closest to the upper end of the image among pixels having a difference value calculated by the inter-frame difference calculation means equal to or greater than a predetermined value is obtained, and the pixel row obtained for each pixel row is obtained. An area dividing unit that draws a horizontal curve through the pixels and divides the image area vertically into a plurality of divided areas at a point where the curve reaches the lower end of the image and a point where the curve reaches a downward peak;
A region selecting unit that selects a divided region having the largest number of pixels having a difference value equal to or larger than a predetermined value among the plurality of divided regions obtained by the region dividing unit as a person region in which the person exists; ,
Determining means for determining that the person protrudes from the image if there are more pixels than the predetermined threshold value near the image edge of the image within the person area, An image processing apparatus. In an image processing apparatus for processing a captured image, a protrusion detection method for detecting protrusion of a person from the captured image,
Continuously acquiring images including the person;
Calculating a difference of each pixel between the acquired images;
For each pixel column in the vertical direction of the image, a pixel closest to the upper end of the image among the pixels having the calculated difference value equal to or larger than a predetermined value is obtained, and the pixel obtained for each pixel column is divided in the horizontal direction. Dividing the region of the image vertically into a plurality of divided regions at a point where the curve reaches the lower end of the image and a point where the curve reaches a downward peak;
Selecting a divided area having the largest number of pixels having a difference value equal to or greater than a predetermined value among the plurality of divided areas as a person area where the person exists;
Determining that the person protrudes from the image if there are more pixels than the predetermined threshold value near the image edge of the image within the person region, the difference value being greater than or equal to a predetermined value; Protruding detection method. A protrusion detection program for detecting protrusion of a person from the captured image, which is executed by a computer that processes the captured image,
A procedure for continuously acquiring images including the person;
Calculating a difference of each pixel between the acquired images;
For each pixel column in the vertical direction of the image, a pixel closest to the upper end of the image among the pixels having the calculated difference value equal to or larger than a predetermined value is obtained, and the pixel obtained for each pixel column is divided in the horizontal direction. A step of dividing the region of the image vertically into a plurality of divided regions at a point where the curve reaches the lower end of the image and a point where the curve reaches a downward peak;
A procedure for selecting a divided area having the largest number of pixels having a difference value equal to or greater than a predetermined value as the person area where the person exists, among the plurality of divided areas;
A procedure for determining that the person protrudes from the image if there are more pixels than the predetermined threshold value near the image edge of the image within the person region. A protrusion detection program to be executed by a computer.

Images