JP4530175B2 - Image processing apparatus, protrusion detection method and program for image processing apparatus - Google Patents

Description

  The present invention relates to an image processing apparatus that captures a moving image of a person, and more particularly, to an image processing apparatus that detects that a person protrudes from a captured image.

  In order to leave a moving image as a record, a moving image may be captured by an image capturing device such as a video camera. An image captured by an image capturing device for leaving a moving image as a record often includes a person or a person's face. It is desirable that the person and his / her face do not protrude from the image captured by the image capturing device. However, when shooting a moving image, the person to be shot is not always stationary, and usually moves constantly. Therefore, it is conceivable that a person or face protrudes from the field of view. Therefore, it is desirable that the shooting direction and shooting range of the video camera be corrected according to the movement of the person.

Conventionally, when a moving image is shot by an image shooting device, an operator who operates the image shooting device takes a picture so that the person does not protrude from the field of view or the person who has protruded is placed in the field of view again. The shooting direction was adjusted by panning and tilting according to the movement of the target person. The conventional video camera described in Patent Document 1 has a driving device for panning or tilting in a grip. Therefore, an operator of a conventional video camera can easily perform panning or tilting without using an electric pan head device.
Japanese Patent Laid-Open No. 01-243673

  In recent years, image recognition devices that recognize people and their faces from moving images taken with a video camera have been applied to human authentication for security and criminal investigations using criminal images taken with a security camera. ing.

  Some image recognition apparatuses recognize a person by comparing an image of a person photographed by a video camera with an image of a person recorded in advance. For example, a conventional face image matching device described in Japanese Patent Application Laid-Open No. 2000-331167 compares two face images and evaluates the degree of coincidence between two faces appearing in the face image. A conventional face image matching device first normalizes a face image in order to match the size of the face appearing in the face image and the position of the face in each face image. Next, the face image matching device extracts a face area by masking a predetermined area of two face images based on predetermined mask data. Next, the face image collation device converts each face image into a monochrome image as necessary. Furthermore, the face image collation device collates after reducing the gradation of each face image. In this way, the conventional image recognition apparatus recognizes a person or a face from an image taken with a video camera.

  However, since a video camera is restricted by the lens system image acquisition range and the size of an image that can be output from the photoelectric conversion device, there is a finite field of view in an image that can be captured by the video camera. Since the image recognition apparatus recognizes a person or face from an image taken by a video camera, the person or face must be in the field of view in order to recognize the person or face normally.

  However, when shooting a moving image including a person with a video camera, the person is not necessarily stationary. Therefore, the person who is the recognition target of the image recognition apparatus is not always in the field of view of the video camera. Therefore, it is desired that the image recognition apparatus corrects the shooting direction and shooting range of the video camera in accordance with the movement of the person.

  As a method for correcting the shooting direction and the shooting range, there has conventionally been a method in which an operator of a video camera observes the movement of a person to be shot and pans or tilts accordingly.

  However, an image recognition apparatus applied to personal authentication and security camera for ensuring security is not always operated by an operator, and needs to operate even without an operator. Therefore, when the conventional image recognition apparatus cannot recognize a person, it cannot know whether or not it is due to the protrusion from the image. For this reason, conventional image recognition devices cannot automatically correct the shooting direction or shooting range when a person or their face protrudes from the image, and warn that a person protrudes from the image. I couldn't.

  For this reason, the conventional image recognition apparatus is installed so as to fixedly shoot an appropriate range in which a person and his / her face will enter the image, so that the person does not protrude from the image as much as possible.

  An object of the present invention is to provide an image processing apparatus capable of detecting that a person protrudes from an image photographed by a video camera, and a protrusion detection method and program thereof.

In order to achieve the above object, an image processing apparatus of the present invention is an image processing apparatus that detects protrusion of an image of an object to be photographed.
Imaging means for acquiring an image including the object with a plurality of cameras ;
When an image is acquired by the imaging unit, a distance calculation unit that calculates a distance from the imaging unit to an object that is reflected in the center of the image;
Reliability determining means for determining that the reliability is high if the change in the correlation value at the center of the image obtained from the plurality of cameras by the imaging means is greater than a predetermined rate of change ;
Distance determination for determining that the object is protruding from the image when the distance obtained by the distance calculation means is less than or equal to a predetermined threshold value and the reliability determination means determines that the reliability is high. Means.

  Therefore, the distance calculation means calculates the distance to the object photographed at the center of the image, which is highly likely to be an object, from the image captured by the imaging means, and the reliability determination means is calculated. The reliability of the distance is obtained, and as a result, when the reliability is high and the distance is too close, the determination unit determines that the object is protruding from the image.

  The predetermined value and the threshold value may be parameters that can be changed.

Another image processing apparatus of the present invention is an image processing apparatus that detects a protrusion from an image of an object to be photographed,
First imaging means for acquiring an image including the object;
A second imaging unit that includes the object and acquires an image shifted from the image acquired by the first imaging unit simultaneously with the first imaging unit;
An image center part correlation calculating means for calculating a correlation value at an image center part of images simultaneously obtained by the first image pickup means and the second image pickup means;
Reliability determination means for determining that the reliability is high if the change in the correlation value obtained by the image center correlation calculation means is greater than a predetermined change rate;
From the positional relationship between the first imaging means and the second imaging means and the amount of deviation when the correlation value between the two images is maximized by shifting the two images, Distance calculating means for calculating the distance of;
Distance determining means for determining that the object is protruding from the image when the distance obtained by the distance calculating means is less than or equal to a predetermined threshold value and the reliability determining means determines that the reliability is high. And have.

  Therefore, the image center correlation calculation means calculates the correlation value of these images from the two images taken by the first and second imaging means and shifted from each other, and the distance calculation means knows from the correlation value. The distance to the object is calculated from the amount of deviation between the two images, and as a result, when the distance is too close, the determination means determines that the object protrudes from the image.

  The change rate and the threshold may be parameters that can be changed.

Still another image processing apparatus according to the present invention is an image processing apparatus that detects a protrusion from an image of an object to be photographed.
First imaging means for acquiring an image including the object;
A second imaging unit that includes the object and acquires an image shifted from the image acquired by the first imaging unit simultaneously with the first imaging unit;
A small area correlation calculating means for calculating a correlation value in each of a plurality of small areas in an image central portion of an image obtained simultaneously by the first imaging means and the second imaging means;
From the positional relationship between the first image pickup means and the second image pickup means and the shift amount when the correlation value of the two images obtained by shifting the two images is maximized to the object reflected in each of the small areas Distance calculating means for calculating the distances of
Minimum distance selection means for selecting the smallest of the distances in each of the small regions calculated by the distance calculation means;
Distance determining means for determining that the object protrudes from the image if the minimum distance obtained by the minimum distance selecting means is less than or equal to a predetermined threshold value;

  Therefore, the image center correlation calculation means calculates the correlation value of these images from the two images taken by the first and second imaging means and shifted from each other, and the distance calculation means knows from the correlation value. Based on the amount of deviation between the two images, the distance to the object is calculated by using the object in each small area at the center of the image with the smallest distance as the object. However, it is determined that the object protrudes from the image.

  Note that the threshold may be a parameter that can be changed.

  According to the present invention, since an object is too close, it can be automatically detected that the object protrudes from the image.

Embodiments of the present invention will be described in detail with reference to the drawings.
(First embodiment)
The first embodiment of the present invention is an image processing device that detects that a person protrudes from an image based on changes in the image of the upper, lower, left, and right portions of the image.

  FIG. 1 is a block diagram illustrating a configuration of the image processing apparatus according to the first embodiment. Referring to FIG. 1, the image processing apparatus 10 includes an imaging unit 11, an inter-frame difference calculation unit 12, a binarization unit 13, an image upper pixel number unit 14, an image right pixel count unit 15, and an image left pixel number. A count unit 16, an image lower pixel number count unit 17, and a determination unit 18 are included.

  The imaging unit 11 is, for example, a video camera, and continuously captures images one after another and digitizes the captured images to obtain data for each pixel. In the present embodiment, it is assumed that the captured image is a monochrome image as an example. Therefore, the data for each pixel is indicated by a numerical value representing luminance.

  The inter-frame difference calculation unit 12 receives the image digitized by the imaging unit 11 and stores a predetermined number of past images in a memory (not shown), and stores the latest image and the past image stored in the memory. The difference from each image is calculated for each pixel. The difference of each pixel is a difference between the luminance of the pixel in the latest image and the luminance of the pixel in the past image. The number of past images stored in the memory is at least one, and may be plural, for example, set to be changeable as a parameter.

  The binarization unit 13 compares the difference of each pixel calculated by the inter-frame difference calculation unit 12 with a predetermined difference threshold value, and binarizes it according to the magnitude of the threshold value. Note that the difference between the pixels being larger than the difference threshold indicates that the pixel has a large difference from the past image, that is, the change of the pixel is large. Here, the difference of each pixel is binarized to “1” when it is equal to or greater than the difference threshold, and is binarized to “0” when it is smaller than the difference threshold.

  In the first embodiment, the portions near the upper, lower, left, and right sides of the image captured by the imaging unit 11 are defined as an upper region, a lower region, a left region, and a right region. 2 shows an upper area 91 in the image 90, FIG. 3 shows a left area 92, FIG. 4 shows a right area 93, and FIG. 5 shows a lower area 94.

  The image upper pixel number counting unit 14 counts the number of pixels in the upper region 91 that are equal to or higher than the difference threshold by the binarizing unit 13, that is, the number of pixels binarized to “1”. .

  The image right pixel count counting unit 15 counts the number of pixels in the right region 93 that are equal to or greater than the difference threshold by the binarization unit 13.

  The image left pixel number counting unit 16 counts the number of pixels in the left region 92 that are equal to or greater than the difference threshold by the binarization unit 13.

  The image lower pixel number counting unit 17 counts the number of pixels in the lower region 94 that are equal to or greater than the difference threshold by the binarizing unit 13.

  If the count value of the upper image pixel count unit 14 is larger than a predetermined upper count threshold value, the determination unit 18 determines that a person is protruding from the image. The determination unit 18 determines that the person protrudes to the right of the image if the count value of the image right pixel number counting unit 15 is greater than a predetermined right part count threshold. The determination unit 18 determines that the person protrudes to the left of the image if the count value of the image left pixel number counting unit 16 is larger than a predetermined left part count threshold value. If the count value of the image lower pixel number counting unit 17 is larger than a predetermined lower count threshold value, the determining unit 18 determines that the person protrudes below the image.

  FIG. 6 is a flowchart showing the operation of the image processing apparatus of this embodiment. Referring to FIG. 6, first, the imaging unit 11 captures an image and sends the image to the inter-frame difference calculation unit 12 (step A1). Here, it is assumed that an image captured t-th after the video camera of the imaging unit 11 is activated is im (t).

  Next, the inter-frame difference calculation unit 12 calculates a difference between each pixel of the latest image im (t) and each pixel of the past image stored in the memory, and each difference is set as the value of each pixel. The difference image diff (t) to be generated is generated (step A2).

  Next, the inter-frame difference image calculation unit 13 stores the image im (t) in the memory as a past image (step A3). At the same time, the inter-frame difference image calculation unit 13 discards one oldest past image stored in the memory.

  Next, the binarizing unit 14 compares the value of each pixel (the above-described difference) of the difference image diff (t) with the difference threshold value, binarizes the difference image diff (t), and binarizes the binarized image bin. Conversion into (t) (step A4). At this time, the binarization unit 14 sets the pixel value to “1” when the value of each pixel is equal to or greater than the difference threshold. Also, the binarization unit 14 sets the pixel value to “0” when the value of each pixel is smaller than the difference threshold.

  Next, the image upper pixel number counting unit 14 counts the number of pixels having a pixel value “1” in the upper region 91 of the binarized image bin (t) (step A5). Here, the count value is a (t).

  Next, the image right pixel number counting unit 15 counts the number of pixels having a pixel value “1” in the right region 92 of the binarized image bin (t) (step A6). Here, the count value is b (t).

  Next, the image left pixel number counting unit 16 counts the number of pixels having a pixel value “1” in the left region 93 of the binarized image bin (t) (step A7). Here, the count value is c (t).

  Next, the image lower pixel number counting unit 17 counts the number of pixels having a pixel value “1” in the lower region 94 of the binarized image bin (t) (step A8). Here, the count value is d (t).

  Next, the determination unit 18 determines whether or not the count value a (t) is larger than the upper count threshold th1 (step A9). If the count value a (t) is larger than the upper count threshold th1, the determination unit 18 detects that a person protrudes from the image (step A10).

  Next, the determination unit 18 determines whether or not the count value b (t) is greater than the right count threshold th2 (step A11). If the count value b (t) is larger than the right part count threshold th2, the determination part 18 detects that a person protrudes to the right of the image (step A12).

  Next, the determination unit 18 determines whether or not the count value c (t) is greater than the left count threshold th3 (step A13). If the count value c (t) is larger than the right part count threshold th3, the determination unit 18 detects that the person protrudes to the left of the image (step A14).

  Next, the determination unit 18 determines whether or not the count value d (t) is greater than the lower count threshold th4 (step A15). If the count value d (t) is larger than the right part count threshold th4, the determination unit 18 detects that a person protrudes under the image (step A16).

  When the above operation is completed, the image processing apparatus 10 increments “t” by one (step A17), and returns to step A1.

  As described above, the image processing apparatus 10 according to the present embodiment calculates the difference between the latest image and the past image by the inter-frame difference calculation unit 12, and binarizes the difference by the binarization unit 13. The number of pixels that have changed more than a predetermined number is counted for each of the upper, lower, left, and right areas by determining whether or not the above has changed, by the upper, right, left, or lower pixel number counting units 14, 15, 16, and 17, The determination unit 18 compares the count value of each region with the threshold value and determines that a person has protruded from the image in the region where the count value exceeds the threshold value. Therefore, the region where the count value exceeds the threshold value due to a large image change is determined. By detecting this, it is possible to automatically detect the protrusion of a person from the image.

  In the present embodiment, the case where the imaging unit 11 captures a monochrome image has been illustrated, but the present invention is not limited to this and may be a color image. When a color image is captured by the imaging unit 11, for example, the inter-frame difference calculation unit 12 represents each pixel value as a vector whose elements are RGB three colors, and the difference between the pixel of the past image and the vector. Ask for. And the binarization part 13 may produce | generate a binarized image by comparing the magnitude | size of the difference vector calculated | required by the interframe difference calculation part 12 with a difference threshold value.

  Further, in the present embodiment, the case where the object for detecting the protrusion from the image is a person is exemplified, but the present invention is not limited to this. Any object can be used as long as it can move when continuously photographed.

Also, by adjusting parameters such as the number of past images, each threshold, and the size of each area in the top, bottom, left, and right used in this embodiment, the accuracy of detecting the protrusion of the image processing apparatus is improved. Can do.
(Second Embodiment)
FIG. 7 is a block diagram showing the configuration of the image processing apparatus according to the second embodiment of the present invention. Referring to FIG. 7, the image processing apparatus 20 includes an imaging unit 21, an inter-frame difference calculation unit 22, a binarization unit 23, an area division unit 24, an area selection unit 25, an image upper pixel number counting unit 26, and an image right part. A pixel number counting unit 27, an image left pixel number counting unit 28, an image lower pixel number counting unit 29, and a determination unit 210 are provided.

  The imaging unit 21 is a video camera, for example, and continuously captures images one after another and digitizes the captured images to obtain data for each pixel. In the present embodiment, it is assumed that the captured image is a monochrome image as an example. Therefore, the data for each pixel is indicated by a numerical value representing luminance.

  The inter-frame difference calculation unit 22 receives an image digitized by the imaging unit 21 and stores a predetermined number of past images in a memory (not shown), and stores the latest image and the past image stored in the memory. The difference from each image is calculated for each pixel. The difference of each pixel is a difference between the luminance of the pixel in the latest image and the luminance of the pixel in the past image. The number of past images stored in the memory is at least one, and may be plural, for example, set to be changeable as a parameter.

  The binarization unit 23 compares the difference of each pixel calculated by the inter-frame difference calculation unit 22 with a predetermined difference threshold value, and binarizes the difference based on the magnitude of the threshold value. Note that the difference between the pixels being larger than the difference threshold indicates that the pixel has a large difference from the past image, that is, the change of the pixel is large. Here, the difference of each pixel is binarized to “1” when it is equal to or greater than the difference threshold, and is binarized to “0” when it is smaller than the difference threshold.

  The area dividing unit 24 divides the image into a plurality of divided areas. A method for dividing an image into a plurality of divided regions will be described later.

  The area selection unit 25 selects a divided area that is most likely to be a person from among a plurality of divided areas divided by the area dividing unit 24, and further converts the binarized image according to the selected divided area. The method for selecting the most likely divided area and the method for converting the binarized image will be described later.

  In the second embodiment, the upper, right, left, and lower regions 93, 94, 95, and 96 shown in FIGS. 2 to 5 are defined as in the first embodiment.

  The image upper pixel number counting unit 26 counts the number of pixels having a pixel value “1” in the upper region 91 in the binarized image converted by the region selecting unit 25.

  The image right pixel count counting unit 27 counts the number of pixels having a pixel value “1” in the right region 93 in the binarized image converted by the region selection unit 25.

  The image left pixel number counting unit 28 counts the number of pixels having a pixel value “1” in the left region 92 in the binarized image converted by the region selecting unit 25.

  The image lower pixel number counting unit 29 counts the number of pixels having a pixel value “1” in the lower region 94 in the binarized image converted by the region selecting unit 25.

  If the count value of the image upper pixel number counting unit 26 is larger than a predetermined upper count threshold value, the determining unit 210 determines that a person protrudes from the image. Further, the determination unit 210 determines that the person protrudes to the right of the image if the count value of the image right pixel number counting unit 27 is larger than a predetermined right part count threshold. The determination unit 210 determines that the person protrudes to the left of the image if the count value of the image left pixel number counting unit 28 is larger than a predetermined left part count threshold value. If the count value of the image lower pixel number counting unit 29 is larger than a predetermined lower count threshold value, the determining unit 210 determines that the person is protruding below the image.

  FIG. 8 is a flowchart showing the operation of the image processing apparatus of this embodiment. Referring to FIG. 8, first, the imaging unit 21 captures an image and sends it to the inter-frame difference calculation unit 22 (step B1). Here, it is assumed that an image captured t-th after the video camera of the imaging unit 11 is activated is im (t).

  Next, the inter-frame difference calculation unit 22 calculates a difference between each pixel of the latest image im (t) and each pixel of the past image stored in the memory, and each difference is set as the value of each pixel. The difference image diff (t) to be generated is generated (step B2).

  Next, the inter-frame difference image calculation unit 22 stores the image im (t) in the memory as a past image (step B3). At the same time, the inter-frame difference image calculation unit 22 discards one oldest past image stored in the memory.

  Next, the binarization unit 23 binarizes the value of each pixel of the difference image diff (t) (difference described above) with the difference threshold value, and binarizes the difference image diff (t). Conversion to (t) (step B4). At this time, the binarization unit 23 sets the pixel value to “1” when the value of each pixel is equal to or greater than the difference threshold. The binarization unit 23 sets the pixel value to “0” when the value of each pixel is smaller than the difference threshold.

  Next, the region dividing unit 24 divides the binarized image bin (t) obtained by the binarizing unit 23 into a plurality of divided regions (step B5).

  An example of a region dividing method by the region dividing unit 24 will be described. FIG. 9 is a diagram for explaining a region dividing method. As shown in FIG. 9, the horizontal direction of the binarized image bin (t) is the X axis (rightward is positive), the vertical direction is Y axis (downward is positive), and the xth pixel from the upper left pixel of the image to the right Then, let the pixel value of the yth pixel be v (x, y) below.

  When a binarized image bin (t) as shown in FIG. 9A is obtained, as shown in FIG. 9B, v (x, y) becomes 1 for all x constituting the image. Find the minimum y value. Next, a point at which the obtained y value becomes maximum with respect to the change of x (maximum point) and a point reaching the lower side of the image (lower side contact point) are obtained, and a perpendicular passing through the maximum point and the lower side contact point is obtained. The image is divided into a plurality of divided areas by lines.

  As another example of the region dividing method, a method in which pixels having the same pixel value (“1” or “0”) and connected to each other are grouped into one divided region can be considered. That is, a portion where pixels having a pixel value “1” are connected (a group of pixel values “1”) is defined as one divided region, and similarly, a portion where pixels having a pixel value “0” are connected (pixel value). A group of “0”) is defined as one divided area.

  Next, the area selection unit 25 selects a divided area that is most likely to be a person from the divided areas obtained by the area dividing unit 24. Then, the area selection unit 25 creates a binarized image sbin (t) that is converted so as to have the same value as bin (t) in the divided area and “0” in the other divided areas. (Step B6).

  As an example of a method for selecting a region most likely to be a person by the region selection unit 25, among the divided regions obtained by the region dividing unit 24, a divided region having the largest number of pixels having a pixel value “1” is selected. A method of selecting is conceivable.

  Next, the image upper pixel number counting unit 26 counts the number of pixels having a pixel value “1” in the upper region 91 of the converted binary image sbin (t) (step B7). Here, the count value is a (t).

  Next, the image right pixel number counting unit 27 counts the number of pixels having a pixel value “1” in the right region 92 of the converted binarized image sbin (t) (step B8). . Here, the count value is b (t).

  Next, the image left pixel number counting unit 28 counts the number of pixels having a pixel value “1” in the left region 93 of the converted binarized image sbin (t) (step B9). . Here, the count value is c (t).

  Next, the image lower pixel number counting unit 29 counts the number of pixels having a pixel value “1” in the lower region 94 of the converted binary image sbin (t) (step B10). Here, the count value is d (t).

  Next, the determination unit 210 determines whether or not the count value a (t) is greater than the upper count threshold th1 (step B11). If the count value a (t) is larger than the upper count threshold th1, the determination unit 210 detects that a person protrudes from the image (step B12).

  Next, the determination unit 210 determines whether or not the count value b (t) is greater than the right count threshold th2 (step B13). If the count value b (t) is larger than the right-side count threshold th2, the determination unit 210 detects that a person protrudes to the right of the image (step B14).

  Next, the determination unit 210 determines whether or not the count value c (t) is greater than the left count threshold th3 (step B15). If the count value c (t) is greater than the right count threshold th3, the determination unit 210 detects that a person protrudes to the left of the image (step B16).

  Next, the determination unit 210 determines whether or not the count value d (t) is greater than the lower count threshold th4 (step B17). If the count value d (t) is larger than the right count threshold th4, the determination unit 210 detects that a person is protruding below the image (step B18).

When the above operation is completed, the image processing apparatus 20 increments “t” by 1 (step B19), and returns to step B1.
(Third embodiment)
The third embodiment is an image processing apparatus that detects that a person is too close to a video camera and protrudes from an image.

  FIG. 10 is a block diagram illustrating a configuration of the image processing apparatus according to the third embodiment. Referring to FIG. 10, the image processing device 30 includes a first imaging unit 31, a second imaging unit 32, an image center part correlation calculation unit 33, a reliability determination unit 34, a distance calculation unit 35, and a distance determination unit 36. Have.

  The first imaging unit 31 is, for example, a video camera, and continuously captures images one after another and digitizes the captured images to obtain data for each pixel.

  Similar to the first imaging unit 31, the second imaging unit 32 continuously captures images one after another, digitizes the captured images, and obtains data for each pixel. In addition, the field of view for capturing the image of the second imaging unit 32 has a portion that overlaps the field of view of the first imaging unit 31 and has a relative positional relationship with the field of view of the first imaging unit 31. It is fixed.

  The image center correlation calculation unit 33 calculates a correlation value at the image center 95 (see FIG. 12) of images obtained from the first imaging unit 31 and the second imaging unit 32 almost simultaneously.

  The reliability determination unit 34 determines that the reliability is low if the change of the correlation value obtained by the image center portion correlation calculation unit 33 with respect to the coordinate position is gentle, and if the change is sudden, the reliability is high. Judge.

  The distance calculation unit 35 calculates the central portion of the image from the positional relationship between the first imaging unit 31 and the second imaging unit 32 and the shift amount when the correlation value between the two images is maximized by shifting the two images. The distance to the object shown in 95 is calculated.

  If the distance obtained by the distance calculating unit 35 is equal to or less than a predetermined threshold and the reliability determining unit 34 determines that the reliability is high, the distance determining unit 36 is too close to detect the person from the image. Judge that it protrudes.

  FIG. 11 is a flowchart showing the operation of the image processing apparatus of this embodiment. Referring to FIG. 11, first, the first and second imaging units 31 and 32 respectively capture images and send them to the image center correlation calculation unit 33 (step C1). Assume that the first imaging unit 31 and the second imaging unit 32 are activated at the same time, and images taken t-th after the video camera is activated are im1 (t) and im2 (t), respectively.

  Next, the image center part correlation calculating unit 33 calculates the correlation value corr (x) at the image center part 95 of the images im1 (t) and im2 (t) (step C2).

  An example of a correlation value calculation method will be described.

  Here, the horizontal direction of the image im1 (t) is the X axis (rightward is positive), the vertical direction is the Y axis (downward is positive), and the xth pixel to the right and the yth pixel to the right from the upper left pixel of the image. Let the value be v1 (x, y). Similarly, v2 (x, y) is defined for the image im2 (t). Further, it is assumed that the image center portion 95 is set as shown in FIG.

  If the set of coordinate values of the image central portion 95 is C, the correlation value is corr (x) = Σ _ {(t, s) ¥ in C} | v1 (t−x, s) −v2 (t, s ) |

  Next, the reliability determination unit 34 determines whether or not the change in the correlation value corr (x) is gentle (step C3).

  An example of a method for determining whether or not the correlation value corr (x) is gentle will be described. Let x1 be the x that maximizes the correlation value corr (x). At this time, if corr (x1) −corr (x1−g) <thc1 and corr (x1) −corr (x1 + g) <thc2 in the entire area of the image central portion 95, it is determined that corr (x) is gentle. Otherwise, it is judged that it is not gentle.

  If the change in the correlation value corr (t) is not gentle, the distance calculation unit 35 determines the positional relationship between the first imaging unit 31 and the second imaging unit 32 and the x of the correlation value corr (x) that maximizes the correlation value corr (x). The distance d (t) is obtained from the value (step C4). The method for calculating the distance may be a well-known method. For example, it can be realized by using a method described in the document “3D Vision” (Xu, Tsuji, Kyoritsu Publishing (1998)).

  Next, the distance determination unit 36 determines whether or not the distance d (t) is greater than a predetermined threshold th (step C5). The threshold th is calculated in advance from the size of the object (here, a person), the focal length of the lens, and the size of the image, the distance that the object will protrude from the image. If the distance d (t) is less than or equal to the threshold th, the distance determination unit 36 detects that a person is protruding from the image (step C6).

  After Step C6, when the distance d (t) is larger than the threshold th in Step C5 or when the change is gentle in Step C3, the image processing apparatus 30 increments “t” by one (Step C7), returning to step C1.

  As described above, the image processing apparatus 30 according to the present embodiment simultaneously captures two images that are shifted from each other with the first and second imaging units 31 and 32, and the image center portion correlation calculation unit 33 performs 2 The correlation value at the image central portion 95 of one image is calculated, and the reliability determination unit 34 determines whether the reliability is high or low from the correlation value. When the reliability is high, the distance calculation unit 35 performs the video camera operation. And the distance determination unit 36 automatically determines that the person protrudes from the image if the distance is too short. can do.

In the present embodiment, the case where the target for detecting the protrusion from the image is a person, but the present invention is not limited to this. Any thing can be used as long as a certain size can be assumed.
(Fourth embodiment)
FIG. 13 is a block diagram showing a configuration of an image processing apparatus according to the fourth embodiment of the present invention. Referring to FIG. 13, the image processing apparatus 40 includes a first imaging unit 41, a second imaging unit 42, a small region correlation calculation unit 43, a distance calculation unit 44, a minimum distance selection unit 45, and a distance determination unit 46. is doing.

  The first imaging unit 41 is, for example, a video camera, and continuously captures images one after another and digitizes the captured images to obtain data for each pixel.

  Similar to the first imaging unit 41, the second imaging unit 42 captures images one after another and digitizes the captured images to obtain data for each pixel. The field of view for capturing the image of the second imaging unit 42 has a portion that overlaps the field of view of the first imaging unit 41, and the field of view for capturing the image of the second imaging unit 42 is the first imaging. The relative positional relationship with the visual field of the part 41 is fixed.

  The small area correlation unit 43 calculates the correlation values in the plurality of small areas 96 a, 96 b, 96 c, and 96 d of the image center part 96 of the image obtained from the first imaging unit 41 and the second imaging unit 42 almost simultaneously. Calculate (see FIG. 15).

  For each of the small regions 96a, 96b, 96c, and 96d, the distance calculation unit 44 shifts the position of the first image capturing unit 41 and the second image capturing unit 42 and shifts the two images to maximize the correlation value between the two images. The distance from the object shown in each small area is calculated from the amount of deviation when.

  The minimum distance selection unit 45 selects the smallest one of the distances of the small areas calculated by the distance calculation unit 44.

  If the distance of the small area selected by the minimum distance selection unit 45 is equal to or smaller than a predetermined threshold, the distance determination unit 46 determines that the person is protruding from the image because the distance is too close.

  FIG. 14 is a flowchart showing the operation of the image processing apparatus of this embodiment. Referring to FIG. 14, first, the first and second imaging units 41 and 42 respectively capture images and send them to the small region correlation calculation unit 43 (step D1). It is assumed that the first imaging unit 41 and the second imaging unit 42 are activated at the same time, and images taken t-th after the video camera is activated are im1 (t) and im2 (t), respectively.

  Next, the small area correlation calculation unit 43 calculates correlation values in the small areas 96a, 96b, 96c, and 96d of the images im1 (t) and im2 (t), and the distance calculation unit 44 calculates the small areas. The distance between the video camera and the object in 96a, 96b, 96c, 96d is calculated (step D2). The calculation method of the correlation value and the distance used at this time is the same method as in the third embodiment.

  Next, the distance determination unit 46 determines whether or not the shortest distance among the distances in each small region is equal to or less than a predetermined threshold th (step D3). As in the third embodiment, the threshold th is calculated in advance from the size of the object (here, a person), the focal length of the lens, and the size of the image. It is what I left. If the distance is equal to or smaller than the threshold th, the distance determination unit 46 detects that the person is protruding from the image (step D4).

  After step D4 or when the distance is not less than or equal to the threshold th in step D3, the image processing device 40 increments “t” by one (step D5) and returns to step D1.

1 is a block diagram illustrating a configuration of an image processing apparatus according to a first embodiment. It is a figure which shows the upper area | region in an image. It is a figure which shows the left part area | region in an image. It is a figure which shows the right part area | region in an image. It is a figure which shows the lower area | region in an image. 3 is a flowchart illustrating an operation of the image processing apparatus according to the first embodiment. It is a block diagram which shows the structure of the image processing apparatus of the 2nd Embodiment of this invention. It is a flowchart which shows operation | movement of the image processing apparatus of 2nd Embodiment. It is a figure for demonstrating the method of area | region division. It is a block diagram which shows the structure of the image processing apparatus of 3rd Embodiment. 10 is a flowchart illustrating an operation of the image processing apparatus according to the third embodiment. It is a figure which shows the image center part in an image. It is a block diagram which shows the structure of the image processing apparatus of 4th Embodiment. It is a flowchart which shows operation | movement of the image processing apparatus of 4th Embodiment. It is a figure which shows each small area | region of the image center part in an image.

Explanation of symbols

10, 20, 30 Image processing apparatus 11, 21 Imaging unit 12, 22 Inter-frame difference calculation unit 13, 23 Binarization unit 14, 26 Image upper pixel number unit 15, 27 Image right side pixel number counting unit 16, 28 image Left pixel count unit 17, 29 Image lower pixel count unit 18, 210 Judgment unit 24 Region division unit 25 Region selection unit 31, 41 First imaging unit 32, 42 Second imaging unit 33 Image center part correlation calculation Unit 34 Reliability determination unit 35, 44 Distance calculation unit 36, 46 Distance determination unit 43 Small region correlation calculation unit 45 Minimum distance selection unit 90 Image 91 Upper region 92 Left region 93 Right region 94 Lower region 95, 96 Image center Part 96a, 96b, 96c, 96d Small area A1-A17, B1-B19, C1-C7 Step

Claims (18)

An image processing apparatus for detecting an overhang from an image of a shooting object,
Imaging means for acquiring an image including the object with a plurality of cameras ;
When an image is acquired by the imaging unit, a distance calculation unit that calculates a distance from the imaging unit to an object that is reflected in the center of the image;
Reliability determining means for determining that the reliability is high if the change in the correlation value at the center of the image obtained from the plurality of cameras by the imaging means is greater than a predetermined rate of change ;
Distance determination for determining that the object is protruding from the image when the distance obtained by the distance calculation means is less than or equal to a predetermined threshold value and the reliability determination means determines that the reliability is high. And an image processing apparatus.   The image processing apparatus according to claim 1, wherein the predetermined value and the threshold value are parameters set to be changeable. An image processing apparatus for detecting an overhang from an image of a shooting object,
First imaging means for acquiring an image including the object;
A second imaging unit that includes the object and acquires an image shifted from the image acquired by the first imaging unit simultaneously with the first imaging unit;
An image center part correlation calculating means for calculating a correlation value at an image center part of images simultaneously obtained by the first image pickup means and the second image pickup means;
Reliability determination means for determining that the reliability is high if the change in the correlation value obtained by the image center correlation calculation means is greater than a predetermined change rate;
From the positional relationship between the first imaging means and the second imaging means and the amount of deviation when the correlation value between the two images is maximized by shifting the two images, Distance calculating means for calculating the distance of;
Distance determining means for determining that the object is protruding from the image when the distance obtained by the distance calculating means is less than or equal to a predetermined threshold value and the reliability determining means determines that the reliability is high. An image processing apparatus.   The image processing apparatus according to claim 3, wherein the change rate and the threshold are parameters that are set to be changeable. An image processing apparatus for detecting an overhang from an image of a shooting object,
First imaging means for acquiring an image including the object;
A second imaging unit that includes the object and acquires an image shifted from the image acquired by the first imaging unit simultaneously with the first imaging unit;
A small area correlation calculating means for calculating a correlation value in each of a plurality of small areas in an image central portion of an image obtained simultaneously by the first imaging means and the second imaging means;
From the positional relationship between the first image pickup means and the second image pickup means and the shift amount when the correlation value of the two images obtained by shifting the two images is maximized to the object reflected in each of the small areas Distance calculating means for calculating the distances of
Minimum distance selection means for selecting the smallest of the distances in each of the small regions calculated by the distance calculation means;
An image processing apparatus comprising: a distance determining unit that determines that the object is protruding from the image if the minimum distance obtained by the minimum distance selecting unit is equal to or less than a predetermined threshold value.   The image processing apparatus according to claim 5, wherein the threshold is a parameter set to be changeable. A protrusion detection method of an image processing apparatus for detecting protrusion of an object to be photographed from images continuously acquired by an imaging means,
When an image captured by a plurality of cameras is acquired by the imaging unit, a step of calculating a distance from the imaging unit to an object that is reflected in an image center part of the image;
Determining that the reliability is high if the change in the correlation value at the center of the image obtained from the plurality of cameras by the imaging means is greater than a predetermined rate of change;
Said distance is equal to or less than a predetermined threshold value, and wherein the reliability is determined to be high, protrusion detection method comprising the step of determining that the object from the image is overrunning.   The protrusion detection method according to claim 7, wherein the predetermined value and the threshold are parameters set to be changeable. Acquired by a second imaging unit that acquires an image continuously acquired by the first imaging unit and an image shifted from the image acquired by the first imaging unit simultaneously with the first imaging unit. An image processing method for detecting a protrusion of an object to be imaged from an image using an image,
Calculating a correlation value at an image central portion of images simultaneously obtained by the first imaging means and the second imaging means;
If the change in the correlation value is greater than a predetermined rate of change, determining that the reliability is high;
From the positional relationship between the first imaging means and the second imaging means and the amount of deviation when the correlation value between the two images is maximized by shifting the two images, Calculating the distance of
And a step of determining that the object is protruding from the image when it is determined that the distance is equal to or less than a predetermined threshold and the reliability is high.   The protrusion detection method according to claim 9, wherein the change rate and the threshold are parameters that are set to be changeable. Acquired by a second imaging unit that acquires an image continuously acquired by the first imaging unit and an image shifted from the image acquired by the first imaging unit simultaneously with the first imaging unit. An image processing method for detecting a protrusion of an object to be imaged from an image using an image,
Calculating respective correlation values in a plurality of small regions at the center of an image obtained simultaneously by the first imaging means and the second imaging means;
From the positional relationship between the first image pickup means and the second image pickup means and the shift amount when the correlation value of the two images obtained by shifting the two images is maximized to the object reflected in each of the small areas Calculating each of the distances,
Selecting the smallest of the distances in each of the subregions;
And a step of determining that the object is protruding from the image if the selected minimum distance is equal to or smaller than a predetermined threshold.   The protrusion detection method according to claim 11, wherein the threshold is a parameter set to be changeable. A projecting detection program for causing a computer to detect projecting of an object to be photographed from images continuously acquired by an imaging means,
When an image captured by a plurality of cameras is acquired by the imaging unit, a process of calculating a distance from the imaging unit to an object reflected in the center of the image;
A process of determining that the reliability is high if the change in the correlation value at the center of the image obtained from the plurality of cameras by the imaging means is greater than a predetermined change rate;
Said distance is equal to or less than a predetermined threshold value, and wherein the reliability is determined to be high, protrusion detection program having a process to determine that the object from the image is overrunning.   The protrusion detection program according to claim 13, wherein the predetermined value and the threshold value are parameters set to be changeable. Acquired by a second imaging unit that acquires an image continuously acquired by the first imaging unit and an image shifted from the image acquired by the first imaging unit simultaneously with the first imaging unit. A protrusion detection program for causing a computer to detect protrusion from an image of an object to be photographed using an image,
Processing for calculating a correlation value at an image central portion of images simultaneously obtained by the first imaging means and the second imaging means;
If the change in the correlation value is greater than a predetermined rate of change, a process for determining that the reliability is high;
From the positional relationship between the first imaging means and the second imaging means and the amount of deviation when the correlation value between the two images is maximized by shifting the two images, Processing to calculate the distance of
A protrusion detection program comprising: a process for determining that the object protrudes from the image when it is determined that the distance is equal to or less than a predetermined threshold and the reliability is high.   The protrusion detection program according to claim 15, wherein the change rate and the threshold are parameters set to be changeable. Acquired by a second imaging unit that acquires an image continuously acquired by the first imaging unit and an image shifted from the image acquired by the first imaging unit simultaneously with the first imaging unit. A protrusion detection program for causing a computer to detect protrusion from an image of an object to be photographed using an image,
Processing for calculating respective correlation values in a plurality of small regions in the image central portion of an image obtained simultaneously by the first imaging means and the second imaging means;
From the positional relationship between the first image pickup means and the second image pickup means and the shift amount when the correlation value of the two images obtained by shifting the two images is maximized to the object reflected in each of the small areas A process for calculating the distances of
A process of selecting the smallest of the distances in each of the small areas;
A protrusion detection program including a process of determining that the object protrudes from the image if the selected minimum distance is equal to or less than a predetermined threshold.   The protrusion detection program according to claim 17, wherein the threshold value is a parameter that is set to be changeable.

Images