JP2010186274A - Sunglasses wearing detection apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an apparatus for determining whether or not a person who is imaged in an input image by a camera etc. which is provided at a side of an ATM is wearing sunglasses, and for detecting a person who is still unnaturally wearing the sunglasses. <P>SOLUTION: Determination of wearing the sunglasses is made from distribution of black pixels which satisfy a predetermined condition within an input image by focusing a shape particular to the sunglass. An index indicating the fact that the black pixels are symmetrically distributed at both sides of a face and an index indicating the fact that a lower portion of a lens and a bridge portion are separated up and down within the image are used in the determination. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an apparatus for determining whether or not a person in an image sequentially photographed in image processing is wearing sunglasses. In particular, the present invention relates to an apparatus that can detect a person who operates while wearing sunglasses as a suspicious person by being attached to an ATM (automated teller machine).

In recent years, the number of incidents has increased, and so-called “transfer fraud” is a crime that has become a social problem. For example, this is a scam in which an elderly person who has been tricked to transfer money from a family due to a car accident transfers money to an account designated by the criminal, and then a member of the criminal group withdraws cash from the account. It is a trick.
In this scam technique, a face image of a member of the criminal group who plays a role of withdrawing cash is often taken by a security camera or a camera provided in an ATM, which is one clue that leads to arrest of the criminal.

  However, the fact that the person who withdraws cash from the ATM is a member of the criminal group is known for the first time after being withdrawn, and is generally disguised such as wearing sunglasses so that the human phase is not known. Therefore, if a person standing in front of an ATM wears sunglasses so that the human phase is not known, it is desirable to request that the person remove the sunglasses or, in some cases, report to the security officer.

  In Patent Document 1, if a face image of a person standing in front of an ATM is acquired and compared with a pre-stored standard pattern of the peripheral part of the eye, the person wears sunglasses when the person is not more than a predetermined similarity However, a technique for determining that a person is a suspicious person when attempting to operate an ATM as it is is disclosed.

JP 09-091432 A

  In the technique of Patent Document 1, since the determination is based on the brightness distribution of the peripheral portion of the eye, even if the comparison with the standard pattern is performed, depending on the illumination conditions around the ATM, the eye The surroundings appear dark, and in this case, it is not similar to the standard pattern, and an unnecessary suspicious person determination may be performed.

  The frequent occurrence of unnecessary suspicious person judgments causes great discomfort to users and greatly impairs the reliability of financial institutions with suspicious person judgment equipment installed, and the distribution of brightness depending on lighting conditions and hairstyle Judgment methods that are not affected by this are needed.

  Therefore, an object of the present invention is to provide a sunglasses wearing detection device that detects whether or not a person in an image sequentially photographed in image processing is wearing sunglasses based on a shape peculiar to sunglasses. . When a person standing in front of an ATM is photographed using the apparatus according to the present invention, a suspicious person can be detected.

In order to solve the above problems, a sunglasses wearing detection device according to the present invention includes an image input unit that acquires an input image, and an image processing unit that detects a person wearing sunglasses from the input image, The image processing unit includes a head extracting unit that extracts a head region corresponding to the head of a person from the input image, and a black pixel extracting unit that extracts a black pixel having a substantially black color included in the head region. A feature amount calculating means for calculating a feature amount of the head region based on the distribution of the black pixels, and using the feature amount, when the black pixels are distributed substantially symmetrically, the person wears sunglasses. It is determined that the person wears, and if the black pixels are not distributed substantially symmetrically, the person is provided with a determining unit that determines that the person is not wearing sunglasses.

  Focusing on the symmetrical shape of sunglasses, after extracting the head region including the human face, we examined the distribution of the left and right faces of the black pixels that appear to be wearing sunglasses inside, The configuration in which the wearing of the sunglasses is determined when it is determined that has a symmetry makes it less likely to be affected by the shadow of the hair and the lighting conditions, and the wearing of the sunglasses can be determined with high accuracy.

  In the sunglasses wearing detection device according to the present invention, it is preferable that a vertical histogram relating to the frequency obtained by projecting the black pixel on the vertical axis is created, and the feature amount is calculated at a vertical position indicated by the maximum frequency of the vertical histogram. is there.

  With such a configuration, attention is particularly paid to a row where black pixels that are considered to be capturing sunglasses are concentrated, and therefore, there is an effect that the black pixels caused by causes other than sunglasses, that is, the influence of noise is less likely to be affected.

  In the sunglasses wearing detection device according to the present invention, it is preferable that the feature amount is calculated with the reference for determining the left-right symmetry as the midline of the human head.

  The shape of sunglasses is generally a narrow bridge (a frame that connects the left and right lenses), and there are lenses on both sides of the bridge, so if you determine the symmetry of the left and right with reference to the midline where the bridge is located in the head region Wearing sunglasses can be determined with high accuracy.

  In the apparatus for detecting wearing sunglasses according to the present invention, the feature amount calculating means is located below the vertical position, and is separated from the median line by a predetermined number of pixels at a boundary between the black pixel and other color pixels. Of the certain boundary pixels, the number of vertical pixels between the pixel located at the bottom and the vertical position is calculated, and the determination means does not wear sunglasses if the number of vertical pixels is equal to or less than a predetermined threshold value. It is preferable not to determine wearing sunglasses.

  Sunglasses generally do not have a lens in the bridge portion, and therefore, even in the case of a black pixel, the distance in the vertical direction is large between the black pixel existing on the midline and the black pixel corresponding to the lower part of the lens. When it is small, since it is a case where a black pixel is extracted for reasons other than sunglasses, high precision is realizable by not determining with sunglasses wearing.

  According to the present invention, the presence / absence of wearing sunglasses can be detected with high accuracy based on the shape unique to sunglasses without being affected by the distribution of brightness depending on illumination conditions and hairstyle.

It is a sketch of ATM which is one Embodiment of this invention It is a block diagram of a suspicious person detection device which is one embodiment of the present invention. It is a schematic diagram of the image for demonstrating calculation of the feature-value used for detection of sunglasses. It is a figure explaining the ellipse template for extracting a human head. It is a figure for demonstrating the calculation method of the similarity degree using an ellipse template It is a figure for demonstrating the area | region processed except a hair and an ear unnecessary for the detection of sunglasses. It is a flowchart which shows the whole operation | movement of the suspicious person detection apparatus which is one Embodiment of this invention. It is a flowchart which shows the detail of a sunglasses detection process.

    As a preferred embodiment of the present invention, the sunglasses wear detection device according to the present invention is applied to a suspicious person detection device attached to an ATM of a financial institution, and a person who operates the ATM while wearing sunglasses is suspicious by image processing. An example of detecting a person will be described.

    The suspicious person detection device according to the present embodiment is attached to an ATM, and the imaging device images a person who operates the ATM. And when it determines with the said person wearing sunglasses, it has a function which issues a predetermined warning or notifies a security officer.

  FIG. 1 is a diagram showing an example of an ATM provided with a suspicious person detection device according to an embodiment of the present invention. As shown in the example of FIG. 1, the ATM 40 is provided with a sensor (hereinafter, “human sensor”) 30 for detecting the approach of a person and an imaging device 20. The main body (not shown in FIG. 1) of the suspicious person detection device according to the present embodiment is installed in a so-called backyard where only the employee of the financial institution can enter so that the user cannot touch it. And

  When it is determined that there is a person who approaches the ATM 40 in order to operate the ATM 40, the human sensor 30 outputs that fact to the suspicious person detection device. The human sensor 30 may have an arbitrary detection mechanism as long as it can detect the presence or approach of a person. For example, a distance measuring sensor using ultrasonic waves or microwaves can be used. Further, since the user does not always stand directly in front of the ATM 40, it is desirable that a plurality of users be installed side by side as shown in FIG.

  The imaging device 20 is a so-called camera. A person standing in front of the ATM 40 is photographed at predetermined time intervals, and the photographed images are sequentially sent to the suspicious person detection device. Hereinafter, the unit of time recorded in the predetermined time is referred to as time. As the imaging device 20, for example, a known device such as an internal camera or an external camera configured to include an imaging device such as a CCD device or a C-MOS device, an optical system component, an A / D converter, or the like is used. it can. Further, the resolution of the imaging device 20 can be selected according to a required angle of view, and the NTSC standard, SDTV standard, or HDTV standard can be used, for example. Further, as the wavelength band used for image capturing, color information is used for distinguishing between skin color and black as described later, and therefore the visible light wavelength is selected.

  FIG. 2 is a block diagram illustrating a configuration of the suspicious person detection device 1 on which the sunglasses wearing detection device 10 is mounted, which is an embodiment of the present invention. As shown in FIG. 2, the sunglasses wearing detection device 10 of the present invention includes an image input unit 110, an image processing unit 130, a storage unit 140, and a notification unit 150, and is connected to the imaging device 20 and the human sensor 30. Yes.

  The image input unit 110 is an interface for receiving image signals sequentially transmitted from the imaging device 20, and receives images from the imaging device 20 via wired or wireless wiring such as SCSI, USB, LAN, and dedicated cables. The image input unit 110 sends the image received from the imaging device 20 to each unit of the sunglasses wearing detection device 10. In the present embodiment, the color information is used to determine whether or not each of the pixels constituting the image is a pixel that is determined to represent human skin. As the input unit 110, an input unit capable of processing a color image is used.

  The storage unit 140 can store various programs and various data. For example, a semiconductor memory such as RAM or ROM, EPROM, a magnetic recording medium such as a hard disk, an optical recording medium such as a CD-ROM, DVD-R / W, or the like. Can be used. The storage unit 140 is connected to the image input unit 110, the image processing unit 130, and the reporting unit 150, and reads and writes various programs and various data according to requests from each unit. The storage unit 140 stores a reference image 142 and a suspicious person image 144.

  The reference image 142 is an image that serves as a reference for comparison when extracting the change area. For example, when the human sensor 30 is OFF, that is, when no person is present before the ATM 40, the reference image 142 is taken by the imaging device 20. The image is stored in the storage unit 140. Further, as will be described later, it is appropriately updated by the updating means 137.

  The suspicious person image 144 stores an image of the person when the image processing unit 130 determines that the person included in the image photographed by the imaging device 20 is wearing sunglasses as described later. Is.

  The image processing unit 130 processes the images sequentially received from the image input unit 110 and determines whether a person included in the image is wearing sunglasses. As shown in FIG. 2, the image processing unit 130 includes a change area extraction unit 131, a head extraction unit 131, a black pixel extraction unit 133, a feature amount calculation unit 135, and a determination unit 136.

  The change area extraction unit 131 includes a reference image 142 in which no person exists in front of the ATM 40 stored in the storage unit 140 and an image captured by the imaging device 20 and received via the image input unit 110 (hereinafter referred to as input). The difference between the images is calculated in black and white by a known method, and a pixel group having a difference equal to or greater than a certain threshold is extracted as a change region. The threshold value is obtained by experiment. In addition to the person standing in front of the ATM 40, noise may be extracted due to a person or light shining, so that a known expansion and contraction process is appropriately performed in the image processing technique to remove the noise.

  The head extraction unit 132 extracts the head of a person standing in front of the ATM 40 in the change area extracted by the change area extraction unit 131 from the input image acquired by the imaging device 20.

  First, the head extraction unit 132 converts the input image to black and white by a known method, applies an edge extraction filter, and obtains the edge strength and the edge angle (edge direction with respect to the horizontal). Then, an edge strength image and an edge angle image are created and temporarily stored in the storage unit 140. At the same time, the image obtained by converting the input image to black and white is also temporarily stored in the storage unit 140 for each subsequent process. These temporarily stored images are sequentially deleted when a new input image is acquired at the next time. The edge extraction filter may be a known filter such as a Sobel filter.

  Next, the head extracting means 132 performs matching by shifting a plurality of ellipse templates created in advance and stored in the storage unit 140 on the edge strength image and the edge angle image, and using the edge strength and the edge angle. The similarity is obtained for each pixel, and the position where the similarity is high is extracted as a head candidate of a person standing in front of the ATM 40.

  An elliptic template for extracting head candidates will be described with reference to FIG. The ellipse template is an elliptical partial image having a predetermined width, as indicated by a hatched portion in FIG. 4A. A plurality of types of ellipse templates 300 are prepared so as to correspond to the size of the head of a person standing in front of the ATM 40.

  As shown in FIG. 4B, each pixel 313 on the ellipse template 300 has an X axis (horizontal) when the direction vector 310 is considered in the radial direction as indicated by reference numerals 311a to 311d from the center 312 of the ellipse. An angle θ with respect to the axis).

  The similarity between the ellipse template 300 and the edge intensity image is obtained by the sum of the edge intensities of the pixels that are included in the ellipse template 300 when the ellipse template 300 is superimposed on the edge intensity image. However, when the difference between the angle defined in the ellipse template 300 and the angle on the edge angle image is large for the pixel, the pixel is not included in the calculation of the similarity. This will be described with reference to FIG.

  In FIG. 5, among the edge intensity images obtained by the head extracting means 132, among the pixels whose intensity is equal to or higher than a threshold value capable of removing noise, an overlap with the ellipse template 300 (shown by a dotted line for convenience of drawing). Is schematically shown in the form of a line at reference numerals 322 and 324. A pixel included in reference numeral 322 is indicated by reference numeral 326 for the sake of explanation. Similarly, a pixel included in reference numeral 324 is indicated by reference numeral 328 for the sake of explanation.

The edge direction of the pixel 326 is denoted by reference numeral 332 with reference to the edge angle image. If the direction vector from the center of the ellipse template for the pixel is denoted by reference numeral 330, the angle formed by the reference numeral 332 and the reference numeral 330 is δ1. When δ1 exceeds 180 degrees, the value is subtracted from 360 degrees. When δ1 is smaller than a predetermined edge angle threshold, the edge strength is added to the similarity by referring to the edge strength image for the pixel 326.
Similarly, for the pixel 328, the angle δ2 is obtained from the edge direction 334 and the direction vector 324. This angle δ2 is larger than the edge angle threshold. In this case, the edge strength is not added to the similarity.

  As described above, it is possible to remove the influence of the edge component that is not related to the elliptical shape by referring to the angle information for the pixel having a certain intensity or higher and overlapping the elliptical template 300. Since there is a low possibility that the head of a person standing in front of the ATM 40 is in a part that deviates from the change area extracted by the change area extraction unit 132, the shift matching process using the ellipse template 300 has the center 312 at the change area. It is preferable to carry out only when the change area is extracted by the extraction means 132.

  As a result of the matching process using the ellipse template 300, if there is only one position of the ellipse template 300 that is suitable as the head candidate in the input image, that is, the degree of similarity equal to or greater than a predetermined value is obtained, The head of a person standing in front of ATM 40 is determined. When a plurality of appropriate parts are extracted as head candidates, a determination process for selection as described below is further performed to narrow down to one.

  For each head candidate, the head extraction unit 132 includes (i) an upward degree a1 at the center, (ii) an edge coverage a2, (iii) a skin color pixel ratio a3, (iv) a lateral eccentricity a4 at the center, (V) The change area boundary coincidence a5 is calculated, and the head of the person is extracted from the input image by linear sum of these and (vi) the similarity a6 based on the elliptical template 300.

  (I) The upper degree a1 of the center is a feature amount indicating how far the center point of the head candidate is in the input image. This is because the possibility that the head candidate is erroneously extracted at the body portion of the person cannot be denied, while the head is usually positioned upward in the input image. It is assumed that the center point of the head candidate is linearly proportional to the Y coordinate as 1.0 when the center point is at the top of the input image, and 0 when it is at the bottom. Determine by examining the coordinates.

  (Ii) The edge coverage rate a2 is obtained by dividing the average of the edge strengths included in the head candidate by 255 which is the maximum edge strength. This is based on the fact that edges with a certain degree of strength are extracted for each part of the face, but strong edges are not extracted with clothes and the like.

  (Iii) The skin color pixel ratio a3 is a ratio of the number of pixels considered to be skin color to the total number of pixels included in the head candidate. The skin color is determined by dividing the color component of each pixel into H (hue) S (saturation) V (lightness), and whether the possibility of skin color is sufficiently large from the range of the predetermined HSV color system. Depending on. In this determination, it is only necessary to specify a pixel that satisfies the so-called skin color condition by using a known detection process similar to that used in the human face detection function that has been practically used in digital cameras in recent years. Details are omitted.

  (Iv) The lateral eccentricity a4 at the center is 1.0 when the X coordinate of the center of the head candidate is at the center in the X-axis direction of the input image, and is 0 when it is at the left and right ends of the image, and in the meantime It is assumed that the value changes linearly in proportion to the X coordinate. This is because the ATM user often operates in front of the ATM, so that the head is likely to be located at the center in the input image.

  (V) The change area boundary coincidence a5 is the degree to which the elliptical templates corresponding to the head candidates match on the curve connecting the uppermost ends of the change areas extracted by the change area extracting unit 132. It is defined by the ratio of the number of matching pixels in the pixels constituting the elliptical template 300 to the total number of pixels included in the uppermost curve. Usually, since the head of a person standing in front of ATM 40 is located at the uppermost end of the change area, the characteristic is that the uppermost curve and the upper side of the elliptical template are likely to coincide with each other.

  (Vi) Although the method of calculating the similarity using the elliptical template 300 has already been described, normalization is performed so that the value falls within the range of 0 to 1. For this purpose, the normalized similarity a6 is calculated by multiplying the number of pixels included in the ellipse template 300 by 255 which is the maximum edge strength and dividing the obtained similarity.

  As shown in the following formula (1) from the above a1 to a6, weight coefficients w1 to w6 for each are multiplied to obtain a linear sum to obtain a head degree H that is a head-like degree. The copy candidate is determined as the head of a person standing in front of the ATM 40.

  H = w1 * a1 + w2 * a2 + w3 * a3 + w4 * a4 + w5 * a5 + w6 * a6 (1)

  The weighting factors w1 to w6 are determined experimentally as appropriate depending on the shooting conditions. In the present embodiment, all of them are simply set to 1.

  In order to detect the presence or absence of sunglasses, the entire head may be processed, but hair, ears, and chin are not required. Therefore, as shown in FIG. 6, a rectangular area such as reference numeral 352 that does not include hair or the like is considered for the ellipse template 300, and only the portion included in the rectangular area 352 in the input image is processed thereafter. It is suitable for the object. An image obtained by cutting out a portion included in the rectangular area 352 from the input image is temporarily stored in the storage unit 140 as a head area image.

  The black pixel extraction unit 133 determines whether or not each pixel included in the rectangular area image indicated by the reference numeral 352 in FIG. 6 extracted by the head extraction unit 132 is a pixel that is likely to capture sunglasses. Judgment is made and a highly likely pixel (hereinafter referred to as “black pixel”) is extracted. This black pixel extraction processing will be described with reference to FIG.

A reference numeral 200 shown in FIG. 3A indicates a head region image extracted by the head extraction means 132.
Although the human face has a characteristic for each individual, the positional relationship of each part such as eyes and nose constituting the face is approximately constant regardless of the person. Therefore, when the head region image 200 is obtained, a sunglasses search region as shown by reference numeral 202 in FIG. 3A is set so as to include a portion where the eyes are likely to be located. Black pixels are extracted, and processing for determining the presence of sunglasses from the distribution is performed as described below.

In many cases, ATMs 40 are installed at ATM corners and unattended ATM booths in addition to financial institutions, and in recent years, they are also often installed in convenience store shops.
Therefore, in obtaining the input image, the illumination conditions vary greatly depending on the location where the ATM 40 is installed, and accordingly, the brightness and color tone of the face of the person operating the ATM 40 in the input image varies, and the accuracy of the determination due to this varies. The decline cannot be ignored. Therefore, the following processing is performed after normalizing the pixel values of the head region image.

  Based on the color component of each pixel included in the head region image, a pixel having a high possibility of human skin color is extracted. As the method, a known method used in the field of digital cameras as described above may be employed.

  Next, the average value is calculated for the color balance (RGain = R / G, BGain = B / G) of the pixels in the extracted pixel with a high possibility of skin color, and the luminance value when black and white is obtained by a known method. And RGainAve, BGainAve, and GrayAve, respectively. Then, the color balance and the luminance value are normalized for all the pixels in the head region image 200. Normalization is performed according to the following equations (2) to (4). Note that RT and BT are an R component and a B component, respectively, when the skin color is ideally extracted.

  R = R * RT / RGainAve * GrayT / GrayAve (2)

  G = G × GrayT / GrayAve (3)

  R = B × BT / BGainAve × GrayT / GrayAve (4)

  If each pixel after normalization uses S (saturation), V (lightness), RGain, and BGain and satisfies either of the following equations (5) or (6), the possibility of sunglasses Are extracted as black pixels which are large pixels. However, it is assumed that the values of S (saturation) and V (lightness) are normalized to a range of 0 to 255.

  (S <30 or 70 <S) and V <70 (5)

  (RGain <0.8 or BGain> 1.2) and 70 ≦ V <120 (6)

Expression (5) indicates that the pixel is black, that is, a condition where the luminance is low and the skin color is not expressed as a range of S (saturation) when grayscale is formed by a known method.
Since sunglasses are usually black, V (brightness) is low. On the other hand, human skin is also darkened due to lighting conditions, hairstyles, and shadows of hats, and brightness is often low. Therefore, it is difficult to distinguish whether a pixel shows sunglasses or a shadowed skin only by lightness. Therefore, in the expression (5), the determination accuracy is improved by referring to the value of S (saturation).

Equation (6) corresponds to the case where specular reflection occurs on the surface of the sunglasses and the brightness increases. When the specular reflection occurs, the brightness is higher than that of the portion where the specular reflection does not occur, but the brightness does not become so bright as to exceed the dynamic range of the imaging device 20, for example, when directly photographing a light source.
Further, the color of the specular reflection is a color generated by the painting of other objects reflected by the sunglasses, for example, the wall of the ATM corner or the ATM body, and it is difficult to determine the range in advance. Therefore, by referring to the values of RGain and BGain, only pixels that are clearly not skin color are extracted as black pixels.

  However, in the edge intensity image obtained by the head extracting unit 132, the pixel in the head region image 200 corresponding to the pixel having a high edge intensity is not a black pixel. This is because eyes and eyebrows are black, and it is difficult to distinguish them from black pixels simply by examining the color components. Therefore, although there is no texture in the lens of the sunglasses, attention is paid to the texture in the eyes and eyebrows, and the portion with high luminance edge strength is not set as a black pixel.

  The feature quantity calculating means 135 calculates a sunglasses feature quantity that indicates whether or not a person standing in front of the ATM 40 is wearing sunglasses based on the distribution state of black pixels extracted by the black pixel extracting means 133. . The sunglasses feature amount is divided into (i) symmetry, (ii) edge coincidence, and (iii) shape.

  (I) The degree of symmetry is such that the black pixel is bilaterally symmetric in the head region image 200, that is, when the X axis and the Y axis are defined as shown in FIG. This is a feature amount indicating that the X-coordinate constant line is distributed approximately equally on the left and right sides of the line. A method for obtaining this degree of symmetry will be described below.

  First, black pixels are projected on the vertical axis (Y axis), and a histogram relating to the frequency is created. Of course, the higher the frequency, the higher the possibility that a lens of sunglasses actually exists in the head region image 200. An example of the histogram is indicated by reference numeral 211 in FIG. 3B, and is hereinafter referred to as “vertical histogram” in the present specification. Further, the Y coordinate indicating the maximum frequency of the vertical histogram is defined as HmaxY. A hatched area indicated by reference numeral 213 indicates an area including the black pixels extracted by the black pixel extracting means 133.

  Next, in the row where the Y coordinate is HmaxY, the frequency is examined. If the frequency is smaller than the threshold value th1 relating to the frequency, it is determined that the sunglasses are not detected by the determination unit 136 described later. This is a case where pixels like sunglasses are not included so much, and therefore, it is not necessary to perform the subsequent processing. The threshold th1 is set to an appropriate value in consideration of the resolution and angle of view of the imaging device 20.

  Further, when HmaxY is larger than the threshold value th <b> 2 regarding coordinates, that is, when HmaxY is positioned below the head region image 200, it is determined that the sunglasses are not detected by the determination unit 136 described later. This utilizes the fact that sunglasses are positioned in the vicinity of the upper eye in the structure of a human face. The threshold th2 is determined in consideration of the resolution and angle of view of the imaging device 20.

  Next, the number SPixALL of black pixels included in the row where the Y coordinate is HmaxY is counted, and the ratio of the horizontal width of the head region image 200 to the number of pixels is set to SRatio. The larger this SRatio is, the higher the possibility that the line is a line that captures sunglasses, but the line whose Y coordinate is HmaxY is considered to be a line near the frame such as a bridge in consideration of the shape of a general sunglasses. It is done.

  On the other hand, for example, in the case of a hairstyle that droops bangs largely, the hair is black, so SRatio may increase even if it is not sunglasses. However, as in the head region image indicated by reference numeral 240 in FIG. 3 (e), even if hair covers one eye, if both eyes are covered, the front cannot be seen and it is difficult to operate the ATM. It is thought that there is little covering. Therefore, it is possible to pay attention to the degree of symmetry using the fact that the shape of the sunglasses is symmetrical on the left and right. The degree of symmetry is obtained by the following equation (7).

  Symmetry = Min (SPixL / SPixALL, SPixR / SPixALL) × 2 (7)

  Here, SPixR is the number of black pixels included in the portion corresponding to the right half of the human face from the reference position in the head region image 200, and SPixL is also included in the portion corresponding to the left half. This is the number of black pixels. Min (,) indicates the minimum value among the numerical values in parentheses.

  The degree of symmetry is 0 when a black pixel exists only on the half face of the face with a Y coordinate HmaxY, and is 1 when the black pixel is distributed symmetrically. A value of 1 indicates higher symmetry.

  As a method of determining the reference position for distinguishing the right half surface and the left half surface of the human face, a method based on the structural information related to the human face can be appropriately adopted. However, in the present embodiment, the head region image 200 is simply determined. The left and right midpoints are used as reference positions. As will be described later, the midline of the human head may be obtained and used as a reference.

(Ii) The degree of edge matching is determined based on the degree of matching between the upper part and the lower part. Wearing sunglasses and the shadow of a hat or hair on the eyes, It was introduced to distinguish it from darkening.
That is, it is generally difficult to distinguish black sunglasses from dark hair or shadowed skin only by the brightness and color of the pixels. On the other hand, in the vicinity of eyes with hats, shadows of hair, and deeply carved faces, even if black pixels are fixed and extracted, focusing on the surrounding area rarely extracts strong edges, even if extracted In many cases, it does not coincide with the position of the black pixel. Naturally, if sunglasses are worn, a strong edge is extracted at the edge of the lens, and it is easy to match the boundary of the candidate sunglasses region.

  A method for obtaining the upper coincidence will be described with reference to FIG. The feature amount calculation unit 135 specifies a pixel that is a black pixel for a row in which the Y coordinate of the head region image 200 is HmaxY, and moves upward in the X coordinate of each pixel (a direction in which the Y coordinate decreases). Check for black pixels. Then, the pixels where the black pixels are interrupted (hereinafter referred to as “upper boundary pixels”) are identified and counted. The identified upper boundary pixel EUPix is represented by a curved line and is indicated by reference numeral 220 in FIG. Since there are two lenses of sunglasses, there are two curves 220 in FIG.

  Next, the feature quantity calculating unit 135 refers to the edge intensity image extracted by the head extracting unit 132, is a pixel having a certain edge intensity or more, and matches the upper boundary pixel EUPix in the head region image 200. Pixel SUPix is counted. Then, the upper coincidence is obtained by the following equation (8).

  Upper part coincidence = number of SUPix / number of EUPix (8)

  Next, how to obtain the lower coincidence will be described with reference to FIG. The feature amount calculation unit 135 specifies a pixel that is a black pixel in a row in which the Y coordinate of the head region image 200 is HmaxY, and the downward direction (the direction in which the Y coordinate increases) of the X coordinate of each pixel. Then, the presence or absence of black pixels is examined, and the pixels where the black pixels are interrupted (hereinafter referred to as “lower boundary pixels”) are identified and counted. The identified lower boundary pixel ELPix arranged in a curve is indicated by reference numeral 230 in FIG. Since there are two lenses of sunglasses, there are two curves 230 in FIG.

  Although the curve 220 in FIG. 3C and the curve 230 in FIG. 3D are drawn in two parts as an example, the shape of the sunglasses is typically as shown by reference numeral 201 in FIG. However, the lens is not necessarily elliptical and the bridge is not always thin. If the lens, bridge, and other frame parts are made of the same plastic material and have a thick design, the curves 220 and 230 may be one.

  Next, the feature amount calculating unit 135 refers to the edge intensity image extracted by the head extracting unit 132, is a pixel having a certain edge intensity or more, and matches the lower boundary pixel ELPix in the head region image 352. Pixel SLPix is counted. Then, the lower coincidence is obtained by the following equation (9).

  Lower coincidence = number of SLPix / number of ELPix (9)

  Then, the edge coincidence is obtained by the following equation (10).

  Edge coincidence = Max (upper coincidence, lower coincidence) (10)

  Here, Max (,) represents obtaining the maximum value among the values in parentheses. The edge coincidence becomes a value closer to 1 as the boundary between a black pixel and a pixel other than that matches with a strong edge pixel, and becomes 0 when there is no coincidence.

  (Iii) The degree of shape is that sunglasses wears on the human face, so the area near the bridge is recessed compared to the bottom of the lens, and the bridge is located above the bottom of the lens when viewed from the front. It captures something.

  A method for obtaining the shape will be described with reference to FIGS. First, among the lower boundary pixels ELPix specified when obtaining the lower matching degree, the lowermost pixel in the head candidate image among those belonging to the left half of the face, that is, the pixel PL having the largest Y coordinate is specified. Similarly, the lower boundary pixel ELPix belonging to the right half of the face is identified as the lowest pixel, that is, the pixel PR having the largest Y coordinate. The respective coordinates are (XL, YL) and (XR, YR).

  However, the pixel PL and the pixel PR are positions that are more than a certain distance from the midpoint of the head region image 200 that is a reference position for distinguishing the right and left half faces of a human face, for example, the head region image 200. The lower boundary pixel ELPix included in a region having a width of 1/8 of the width of the head region image 200 with a point separated by 1/4 of the width of the head region image 200 as a center. This is to capture that the shape of the sunglasses is concave in the vicinity of the bridge as described above and the black pixel region is concave downward.

  Next, among the lower boundary pixels ELPix, the Y coordinate of the pixels satisfying XR <x <XL is sequentially examined with respect to the X coordinate, and the minimum value, that is, the pixel PC at the uppermost position in the head region image 200 is specified. . Let the coordinates be (XC, YC). However, depending on the shape of the sunglasses, the curve 230 may be divided into two as shown in FIG. 3D, and the pixel PC may not be identified from the lower boundary pixel ELPix. If the pixel PC is mechanically specified, YC = 0 as shown in FIG. 3D, that is, the uppermost end of the head region image 200. In such a case, HmaxY is defined as the minimum value of YC. The pixel PC ′ having the X coordinate as the midpoint of the head region image 200 is used instead of the pixel PC.

  The degree of shape is defined as 1 when the following expression (11) is satisfied, and 0 when it is not satisfied. Here, th3 is a threshold value related to the shape of the sunglasses and is appropriately determined experimentally.

  MIN (YL, YR) −YC> th3 (11)

  This shape degree is used for discrimination from the case where the vicinity of the sunglasses search area indicated by reference numeral 202 in FIG. It is. In other words, when wearing a hat deeply, unlike in the case of wearing sunglasses, black pixels also exist near the center, which is a portion where the nose bridge is shown. Therefore, when the pixel PC as described above and the pixel PR or PL are not largely separated in the vertical direction, it is determined that the user is not wearing sunglasses.

The determination unit 136 determines whether the person standing in front of the ATM 40 is wearing sunglasses based on each sunglasses feature amount obtained by the feature amount calculation unit 135. In addition, when the person standing in front of the ATM 40 clearly does not satisfy the conditions for wearing sunglasses during the calculation of the sunglasses feature value, it is determined that the sunglasses are not worn.
For the determination, the likelihood of sunglasses defined by the following equation (12) is obtained, and if it is equal to or greater than the predetermined sunglasses threshold th4, it is determined that the person standing in front of the ATM 40 is wearing sunglasses. The sunglasses threshold value th4 is obtained by experiment.

  Sunglasses-likeness = (symmetry) x (edge coincidence) x (shape) (12)

  As described above, the value of the degree of shape is either 0 or 1. Therefore, when the expression (11) is not satisfied, the likelihood of sunglasses is 0. Therefore, the determination unit 136 must always wear sunglasses. Output the result.

The update unit 137 updates the reference image 142 stored in the storage unit 140.
For updating, the pixel value of each pixel of the input image acquired by the imaging device 20 at each time point is multiplied by a predetermined weighting factor and the corresponding pixel of the already stored reference image 142 is added and updated. To do. The entire input image may be simply replaced with the reference image 142.
However, the reference image 142 is updated when the human sensor 30 is OFF, that is, when there is no person before the ATM 40.

When the person standing in front of ATM 40 is wearing sunglasses based on the result of determination means 136, reporting section 150 outputs that fact to the outside as a suspicious person. An alarm lamp such as an LED, an alarm buzzer, or the like may prompt the user to remove the sunglasses, or a prestored message may be played as sound through a speaker or displayed on a display. Further, the suspicious person image 144 stored in the storage unit 140 can be output via an output device (not shown) such as a display or a printer.
Furthermore, by providing the reporting unit 150 with a communication I / F function and connecting it to a security system, it is possible to report to a financial institution or a security company and take a quick response.

  Next, the operation of the suspicious person detection device 1 according to the present embodiment will be described with reference to a flowchart. FIG. 7 is a main flow diagram thereof. First, assume that various initial settings have been completed after the suspicious person detection device 1 is powered on.

  In step S <b> 100, the imaging device 20 captures an area before the ATM 40 at each time, and transmits the acquired input image to the image input unit 110.

The sunglasses wearing detection device 10 checks whether or not the human sensor 30 is responding (step S120). If it is not responding (in the case of OFF), it is determined that the ATM 40 is unattended and the flag is turned OFF. Set (step S140).
This flag is used to refer to whether or not sunglasses detection processing has already been performed for the person at a subsequent time when it is detected that a person has stood before the ATM 40. This is stored in the unit 140.

  If the human sensor 30 is not responding, the reference image 142 in the storage unit 140 is updated in step S160. This is to cope with sunshine fluctuations.

  If the human sensor 30 is reacting in step S120, that is, if it is determined that a person is standing in front of the ATM 40, the process proceeds to step S180 to check the state of the flag. If the flag is set to ON, the sunglasses detection process has already been completed, and it is assumed that a warning announcement or operation guidance for a normal visitor has been given as described later, and the process returns to step S100 (Y branch of step S180). ).

  If the flag is OFF in step S180, there is a person in front of ATM 40, but the sunglasses detection process has not yet been performed, so the process proceeds to step S200 to perform the sunglasses detection process.

  In step S200, the change area extraction unit 131 converts the reference image 142 stored in the storage unit 140 and the input image sent from the image input unit 110 into black and white, and performs difference processing for each pixel (background difference processing). , And a change area generated in the input image is extracted.

  In step S220, the head extracting unit 132 extracts a portion corresponding to the head of a person standing in front of the ATM 40 from the changed region extracted by the changed region extracting unit 131.

  In step S240, based on the black pixel distribution extracted by the black pixel extraction unit 133, the likelihood of sunglasses is obtained from each feature amount calculated by the feature amount calculation unit 135, and the determination unit 136 stands in front of the ATM 40 based thereon. It is determined whether the person is wearing sunglasses. Details of this processing will be described later.

  In step S260, the flag is set to ON. Thus, once the detection process of wearing sunglasses is performed, the branch of Y is followed in step S180, so that it is not necessary to repeat the sunglasses detection process with a high processing load until the person standing in front of the ATM 40 leaves.

  In step S280, referring to the processing result in step S240, if sunglasses are not detected, that is, if it is determined that the person standing in front of ATM 40 is not wearing sunglasses, the N branch is followed. A normal announcement such as “Welcome” is played (step S300), and the process returns to step S100.

  Referring to the processing result of step S240, if sunglasses are detected (Y branch in step S280), a warning announcement such as “Please remove sunglasses and use” is sent to the person standing in front of ATM 40. When the reporting unit 150 has a communication I / F function and is connected to the ATM 40, the sunglasses wearing detection device 10 may output a signal for stopping the use of the ATM 40. Further, an input image when sunglasses are detected may be stored in the suspicious person image 144 of the storage unit 140 and transmitted to the outside via the notification unit 150 in response to an external transmission request or automatically. good.

  Next, the details of the sunglasses detection process in step S240 will be described with reference to FIG.

  In step S <b> 500, the black pixel extraction unit 133 extracts a pixel having a black color that has a high possibility of taking sunglasses from the head region image 200 extracted by the head extraction unit 132.

  In step S510, the feature amount calculating unit 135 creates a vertical histogram from the extracted black pixel distribution.

  In step S520, the feature amount calculation unit 135 specifies the Y coordinate of the row having the maximum frequency in the vertical histogram created in step S510, and sets it as HmaxY.

In step S530, the feature amount calculating unit 135 refers to the maximum value of the frequency of the vertical histogram and compares it with the threshold th1. When it is smaller than the threshold th1, the fact is output to the determination means 136, and it is assumed that the person standing in front of the ATM 40 is not wearing sunglasses (N branch), and the process of step S240 is terminated (step S610). ). When the frequency of the vertical histogram is equal to or greater than the threshold th1 (Y branch), the process proceeds to step S540.
Here, instead of thresholding the maximum frequency of the vertical histogram, SRatio, which is the ratio of the number of black pixels to the horizontal width of the head region image 200, may be thresholded.

  In step S540, the feature amount calculating unit 135 compares HmaxY specified in step S520 with a threshold th2 related to coordinates. If HmaxY is greater than or equal to the threshold th2, the fact is output to the determination means 136, and the person standing in front of the ATM 40 is not wearing sunglasses (N branch), and the process of step S240 is terminated (step S610). ). When HmaxY is smaller than the threshold th2 (Y branch), the process proceeds to step S550.

  In step S550, the feature amount calculation unit 135 calculates the degree of symmetry from the distribution of black pixels extracted by the black pixel extraction unit 133. The calculation method is as described in the description of the feature amount calculation unit 135.

  In step S560, the feature amount calculating unit 135 calculates the edge from the edge intensity image created by the head extracting unit 132 and temporarily stored in the storage unit 140 and the black pixel distribution extracted by the black pixel extracting unit 133. The degree of coincidence is calculated. The calculation method is as described in the description of the feature amount calculation unit 135.

  In step S570, the feature amount calculating unit 135 calculates the shape degree from the distribution of black pixels extracted by the black pixel extracting unit 133. The calculation method is as described in the description of the feature amount calculation unit 135.

  In step S580, the feature amount calculation unit 135 calculates the likelihood of sunglasses from the symmetry, the edge coincidence, and the shape by Equation (12).

  In step S590, the determination unit 136 compares the likelihood of sunglasses calculated by the feature amount calculation unit 135 with the threshold th4, and if it is less than th4 (N branch), the person standing in front of the ATM 40 wears sunglasses. If not (step S610), the process of step S240 is terminated. If the likelihood of sunglasses is greater than the threshold th4 (Y branch), the person is wearing sunglasses (step S620), and the process of step S240 is terminated.

In the present embodiment, as a method of determining the reference position for distinguishing the right half surface and the left half surface of the human face, the midpoint of the width of the head region image 200 is simply used, but other methods may be used.
For example, the major axis of the ellipse template 200 used for head extraction may be superimposed on the head region image 200 and used to determine the reference position.
In addition, a nose bridge can also be used as a reference. In that case, a template relating to an edge representing the standard shape of a person's nose is prepared and stored in the storage unit 140, and the size is changed in various ways. The nasal bridge can be obtained by setting the position that most closely matches the edge intensity image extracted by the head extracting unit 132 as the position of the nose.
Furthermore, the mouth and nose may be extracted by a known method, and a straight line connecting the mouth point and the nasal head point may be used as a midline to determine the reference position. The midline may be a perpendicular bisector of a straight line connecting the left and right mouth corner points.

  In addition, the feature amount calculation unit 135 forms a histogram of the frequency of black pixels included in each X-axis coordinate column for all rows in the sunglasses search area 202 shown in FIG. In this specification, it is referred to as “horizontal histogram” (illustrated by reference numeral 212 in FIG. 3B), and the X coordinate corresponding to the position at which the frequency becomes a minimum value in the vicinity of the approximate center of the horizontal histogram is also obtained. it can.

In this embodiment, the likelihood of sunglasses is obtained by multiplying the degree of symmetry, the degree of edge coincidence, and the degree of shape, but it is not necessary to use all of them. For example, with particular attention to the fact that sunglasses have a bilaterally symmetric shape, only the degree of symmetry may be used to output the detection result.
Furthermore, not only in the row where the Y coordinate value is HmaxY, but referring to the above-mentioned horizontal histogram, and the distribution curve shows bimodality as indicated by reference numeral 212 in FIG. It may be determined that is high and output.

    In the present embodiment, when extracting the head, the reference image 142 is prepared in the storage unit 140, and the change area extraction unit 131 performs a difference process with the input image obtained by the imaging device 20. It is not limited to this. The same effect can be obtained even if a large number of data representing human heads are prepared in advance, and a learned discriminator (detector) is applied to the input image to directly extract the head regardless of the difference processing. Can do.

By providing the present invention in an ATM, a suspicious person who wants to withdraw cash while wearing sunglasses can be detected before the operation, and this can be useful for early detection and prevention of a criminal group of wire fraud.

DESCRIPTION OF SYMBOLS 1 Suspicious person detection apparatus 10 Sunglasses wear detection apparatus 132 Head extraction means 133 Black pixel extraction means 135 Feature-value calculation means

Claims (4)

An image input unit for acquiring an input image;
An image processing unit for detecting a person wearing sunglasses from the input image;
A sunglasses wearing detection device comprising:
The image processing unit
Head extraction means for extracting a head region corresponding to the head of a person from the input image;
Black pixel extraction means for extracting black pixels that are substantially black in the head region;
Feature amount calculating means for calculating the feature amount of the head region based on the distribution of the black pixels;
If the black pixels are distributed substantially symmetrically using the feature amount, it is determined that the person wears sunglasses. If the black pixels are not distributed substantially symmetrically, the person is wearing sunglasses. A sunglasses wear detection device comprising a determination means for determining that the wearer is not worn. The feature amount calculating means calculates a vertical histogram related to a frequency obtained by projecting the black pixel on a vertical axis, and determines whether or not the black pixel is substantially symmetrical in the horizontal direction at a vertical position indicating the maximum frequency of the vertical histogram. The sunglasses wearing detection apparatus according to claim 1, wherein the feature quantity to be expressed is calculated. The feature amount calculation unit calculates a median line from the head region, and calculates a feature amount indicating whether or not the black pixel is substantially symmetric with respect to the median line. Sunglasses wear detection device described in one. The feature amount calculating means is located at a lowermost position among boundary pixels that exist below the vertical position and are separated from the median line by a predetermined number of pixels and are boundaries between the black pixels and other color pixels. Calculating the number of vertical pixels between the vertical pixel and the vertical position,
4. The sunglasses wearing detection apparatus according to claim 3, wherein the judging means judges that sunglasses are not worn if the number of vertical pixels is equal to or less than a predetermined threshold value.