JP2006092396A - Apparatus for detecting lone person and person in group - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately detect a lone person or person in a group and a kind of the group form from information for sex, age and the like of persons, and further to accurately and rapidly detect a lost child from information such as physical size, sound of crying and the like by acquiring a moving image of each point, and grouping persons based on information for the behaviors of persons acquired from the moving image. <P>SOLUTION: This detector comprises a person detecting means detecting persons and the positions of the persons from a moving image for every lapse of a predetermined time; a person group determining means performing a group determination for grouping the persons from the positional relations of the persons; a person group form determining means determining lone persons or persons in a group and a group form thereof, by focusing on the number of persons from the result of the group determination; and a person group reporting means reporting the group form. The group determination is performed for the persons contained in the moving image. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a single actor and group actor detection device.

  Conventionally, whether each person is acting alone in a place where a large number of unspecified people gather, such as large commercial facilities such as shopping centers, department stores, shopping malls, public facilities such as railway stations, underground malls, etc. Alternatively, information such as whether the group is acting as a group and what kind of form the group is in is required from the viewpoint of marketing and security.

  For example, from a marketing point of view, retailers, restaurants, etc. manually survey passers-by, and obtain information on what types of groups exist based on the passer's age, gender, etc. There is.

  Also, from the viewpoint of security, there is a strong demand for finding a lost child that is separated from the parent, a method for finding a lost child by hand, a method for finding a lost child by wireless communication such as an ID tag, a portable information terminal, etc. Several methods have been proposed. For example, as a method for finding a lost child by wireless communication, a method for preventing a lost child by issuing a warning when the distance between the parent and the child is separated (see, for example, Patent Document 1), and recognizing the lost child with an ID card after becoming lost. A method (for example, refer to Patent Document 2), a method for tracking a lost child with a transmitter (for example, refer to Patent Document 3), and the like have been proposed.

  However, most of the lost child discovery is done manually. In this case, if a facility such as a lost child information center is prepared in the facility, and either one of the lost child or the parent comes to the lost child information center, broadcasts on the site, etc. It is common to search for the other using.

  Here, when a lost child comes to the Lost Child Information Center and searches for a parent, the facility employees and passers-by show that the child goes to the center and the child who is lonely and crying alone. There are two patterns: a pattern to discover and take to a lost child information center.

In addition, when a parent comes to a lost child information center and searches for a lost child, information on the appearance of the child's clothes, height, etc. is notified by a pattern that calls the lost child directly using broadcast, and the employee or passerby of the facility There are two patterns to find with the help of. For example, many commercial facilities in the United States have introduced a system called Code Adam that seeks lost children by mobilizing all of the facility's employees. In this case, as a supplementary means, the child may be preliminarily attached with a lost child tag in which the names of the person and the parent and the contact information are written, and may be used as a key when searching for lost children.
JP 11-242790 A JP 9-61519 A JP 2000-293785 A

  However, the conventional system has various problems. For example, in general, when manually checking whether a person is a single actor or a group actor, it takes time, manpower, and cost, and a relatively long observation time is required for the determination. In addition, while the observation is continued for a long time, the judgment criteria of the observer changes, and the data is likely to be inaccurate. Furthermore, it is extremely difficult for one observer to determine many people at once at the same time.

  Also, when a facility employee or passerby finds a lost child and calls the parent by broadcast at the lost child information center, the child may be walking alone, crying, or repeatedly calling out to the parent, etc. It is necessary to behave like a lost child and to be discovered by employees and passers of the facility. However, when a lost child is discovered, it is necessary to watch the child for a certain amount of time in order to determine whether or not he / she is really lost. For this reason, when employees and passersby are busy or crowded, it is difficult to keep a close eye on the child for a long time, and it is difficult to determine whether the child is lost.

  Also, a lost child does not necessarily cry, and if it does not cry, it will be even more difficult to determine whether the child is lost. Therefore, it takes time for the child to be determined to be lost and take action. ) It will be easy to be broken.

  Furthermore, when a child who is lost gets out of the way to the lost child information center and calls the parent by broadcast, the child needs some judgment and action. However, lost children are often unemployed children, who are too young to make their own names. For this reason, it is rare for a lost child to make a name.

  Conversely, when a parent goes to a lost child information desk, it is difficult to find lost children with the help of facility employees and passersby. In other words, it is difficult to accurately capture the lost child, and it takes time and manpower, so that the employees of the facility cannot concentrate on the original work. In addition, calling directly to a lost child through broadcasting may not be effective due to causes such as the child being young, losing calmness, or being noisy in the facility.

  Broadcasts to call the lost child or their parents are difficult to hear when the facility is noisy, and the broadcast sound destroys the atmosphere of the venue or calls the individual's name in front of the public. The privacy of your company will be violated. In addition, the personal information of the child and the parent wearing the lost tag is leaked from the written information.

  Furthermore, in the case of wireless communication, it is necessary for people gathered in advance to be equipped with all the necessary equipment, so there is a cost depending on the number of people gathered, there is a risk of complexity and loss of carrying, database capacity restrictions There is a problem such as exhaustion of ID.

  The present invention solves the above-mentioned conventional problems, acquires a moving image of each point, and groups the persons based on the information of the behavior of the person acquired from the moving images, so that the person can act independently. Can accurately detect whether they are doing group actions or not, and can detect what the group form is based on information such as the gender and age of the person, as well as physique and cry It is an object of the present invention to provide a single actor and group actor detection device capable of accurately and quickly detecting a lost child from such information.

  For this purpose, in the single and group actor detection device of the present invention, in a place where a large number of unspecified persons gather, moving image acquisition means for acquiring a moving image at each point, and a person from the acquired moving image And a person detecting means for detecting the position of the person at every elapse of a predetermined time and a state in which the relative distance is below a threshold value from the mutual positional relationship of the detected person continues for a time longer than the threshold value. It is determined that a group of persons belongs to a group performing the same action, and a group determination unit for performing group determination for grouping the detected persons, and a single action focusing on the number of persons from the result of the group determination A person group form determining means for determining a person or group actor and a group form, and a person group notifying means for notifying the operator of the group form. Performing group judgment about a person included in the image.

  In another single actor and group actor detection device of the present invention, the person group determination means further includes the same set of persons whose relative distance and relative speed are below a threshold for a time longer than the threshold. Judge as a group.

  In still another single-behavior and group-behavior detection device of the present invention, the person group determination means further performs statistical processing on all values of the detected mutual positional relationship of the persons, and the group of persons Person group space statistical means for calculating a threshold value used in the division is provided, and the group determination is performed with a threshold value adapted to the degree of congestion of the evaluation target.

  In still another single actor and group actor detection device of the present invention, the person group form determination means further includes the face image, physique, age, and sex of the person based on the detected position of the person. And a person feature extracting means for extracting and determining the presence or absence of clothes or a disorder as feature information, and determining the group form by combining the feature information of the person and the group determination information.

  In still another single actor and group actor detection device of the present invention, the person group form determination means further determines that the detected person is a single action by the person group determination means, and the person When it is determined by the feature extraction means that the child is based on the physique and age, the group form of the person is determined to be lost.

  In still another single actor and group actor detection device of the present invention, the person group form determination means determines that the detected person is acting in a group of two people, and The person feature extraction means determines that the group form of the group is a couple when it is determined that all of the persons are of adult age, close to age, one is male and the other is female.

  In still another single actor and group actor detection device of the present invention, the person group form determination means further determines that the detected person is acting in a group of a plurality of persons, and When it is determined by the person feature extraction means that there are one or more adult-aged persons and children in the group, the group form of the group is determined to be accompanied by a parent and child.

  In still another single actor and group actor detection device of the present invention, the person group form determination means further determines that the detected person is acting in a group of more than a threshold number, and When the person feature extraction means determines that the clothes of the number of persons equal to or greater than the threshold in the group are the same, the group form of the group is determined to be an organization.

  In still another single-behavior and group-behavior detection device of the present invention, the person group form determination means further determines, from the detected person, a feature that requires a companion by the person feature extraction means. And when the said group is not provided with the said characteristic and there are not enough persons determined to be a companion, the detected person is determined to be a companion without a companion.

  In still another single actor and group actor detection device of the present invention, the feature that requires the accompanying person is either that the age is below a threshold value or that a disorder exists.

  In still another single actor and group actor detection device of the present invention, the person group notification means is further specified based on the detected position of the person, group determination information or personal attribute information. Person search means for narrowing down and searching for persons who meet the conditions is provided, and information on persons who meet the specified conditions is filtered and output.

  In still another single actor and group actor detection device of the present invention, the person group form determination means further acquires a face image of the detected person and stores it as face authentication dictionary data. And storing means and face authentication means for comparing the face image extracted by the person feature extraction means with the face authentication dictionary data and outputting an authentication score, and the authentication score and the person feature information are used for person group form determination. Use.

  In still another individual actor and group actor detection device of the present invention, the person group form determination means further recognizes a facial expression, facial color, and face orientation from the face image extracted by the person feature extraction means. An expression recognizing means is provided, and group form determination is performed using the expression and face color.

  In still another single-behavior and group-behavior detection device according to the present invention, the person group form determining means determines that the detected person is a child acting alone and the facial expression recognition means If the facial expression is recognized as “crying” or if the facial change frequency is higher than the threshold and the amplitude of the change is higher than the threshold, the person is lost Is determined.

  In still another single actor and group actor detection device of the present invention, the person group form determination means determines that the detected person is a group actor, and When the facial expression recognition means recognizes the facial expression of each person as “angry”, the group form of the group of persons is determined to be proximity abnormal behavior.

  In still another individual actor and group actor detection device of the present invention, the person group form determination means further recognizes a facial expression, facial color, and face orientation from the face image extracted by the person feature extraction means. When the facial expression recognition means recognizes that the detected facial color is “red” or “palm white”, the person is determined to be a companion without a companion.

  In still another single actor and group actor detection device of the present invention, the person group form determination means further includes an acoustic acquisition means for acquiring the sound at each of the points, and a group acoustic from the acquired sound. Human acoustic feature extraction means for determining a feature, and determining a person group form based on the acoustic feature.

  In still another single actor and group actor detection device of the present invention, the sound acquisition unit acquires sound with directivity, and the detected person is acquired by the sound acquisition unit. In the case where the sound level exists in a strong direction, the person group form determination means determines that the person has emitted the acquired sound.

  In still another single actor and group actor detection device of the present invention, the person acoustic feature extraction means recognizes the detected utterance of the person.

  In still another single actor and group actor detection device of the present invention, the person group form determination means further includes a crying voice of a child from the person, an utterance calling a companion, etc. by the person acoustic feature extraction means. If the acoustic characteristics of the lost child are extracted and it is determined that the person is a child acting alone, the person is determined to be a lost child.

  In still another single actor and group actor detection device of the present invention, the person group form determination means further performs a yelling sound, a beating sound, a scream by the person acoustic feature extraction means for a group of persons belonging to the same group. When the acoustic features of criminal acts such as the above are extracted, the group form of the group of persons is determined as a criminal act.

  In still another single actor and group actor detection device of the present invention, in addition to the person group form determination means, information on group determination, person characteristics, face authentication score, facial expression, and acoustic characteristics is weighted and added. Then, the personal characteristic scoring means for scoring is provided, and the personal information is output based on the value calculated as the score.

  According to the present invention, the single-behavior and group-behavior detection apparatus acquires a moving image of each point, and groups persons based on information on the behavior of the person acquired from the moving image. . This makes it possible to accurately detect whether a person is acting alone or in a group, and detecting the group form from information such as the gender and age of the person. Furthermore, it is possible to accurately and quickly detect a lost child from information such as physique and cry.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a diagram showing a configuration of a detection apparatus according to the first embodiment of the present invention.

  In the figure, reference numeral 10 denotes a detection device as a single-behavior and group-behavior detection device according to the present embodiment, and includes a calculation means such as a CPU and MPU, a storage means such as a semiconductor memory and a magnetic disk, an input / output interface, and the like. , A kind of computer or computer system that operates according to a program, and includes moving image acquisition means 11, person detection means 12, person group determination means 13, person group form determination means 14, and person group notification means 15.

  The moving image acquisition means 11 outputs in real time a camera device that captures an image of a point where a person is desired to be detected and a moving image captured by the person detection means 12 in a place where an unspecified number of persons gather. Provide an interface. The place where a large number of unspecified persons gather is, for example, convenience stores, supermarkets, department stores, home centers, shopping centers and other commercial facilities, restaurants, restaurants and other dining facilities, banks, post offices, credit unions and other financial institutions , Railway stations, railway vehicles, underpasses, buses, airfields, entertainment facilities such as theaters, theme parks, amusement parks, accommodations such as hotels and inns, public facilities such as schools and government offices, private houses, gatherings Housing facilities such as houses, entrance halls in ordinary buildings, interiors of buildings such as elevators, construction sites, work facilities such as factories, exhibition facilities such as museums, art galleries, exhibition halls, etc. Good. Furthermore, the camera device generally uses an industrial surveillance camera, but if it has a function of capturing a moving image as a surveillance image, it may be a broadcast station camera or a home video camera. Any form may be used as long as a moving image can be acquired by photographing a predetermined range.

  The person detecting means 12 includes an interface for inputting a moving image from the moving image acquiring means 11, an image processing apparatus or software for detecting a person passing through the moving image and its position at each time, and a person group determination. The means 13 includes an interface for outputting person position information indicating the position of the person. Then, the person and the position of the person are detected from the acquired moving image every predetermined time.

  Further, the person group determining means 13 is an interface for inputting person position information from the person detecting means 12, and a computing device for determining which person and which person belong to the same group from the person position information. Alternatively, the software and the person group notification means 15 are provided with an interface for outputting person group information as information relating to the group of persons. Then, based on the mutual positional relationship of the detected persons, it is determined that a group of persons whose relative distance is less than the threshold value continues for a time longer than the threshold value belongs to the group performing the same action, and the detected person Perform group judgment to categorize.

  The person group form determination unit 14 includes an interface for receiving person position information and person group information from the person group determination unit 13, a computing device or software for determining the form of each group, and the person group notification unit 15. An interface for outputting person group form information as information relating to the group form is provided. And from the result of the group determination, paying attention to the number of persons, the single actor or the group actor and the group form are determined.

  Further, the person group notification means 15 includes an interface for inputting person group form information from the person group form determination means 14, an arithmetic unit or software for statistically processing the group information, and an operator as information on the group. A display device for notifying the person group information is provided. Then, the group form is notified to the operator.

  Next, the operation of the detection apparatus 10 having the above configuration will be described.

  FIG. 2 is a diagram illustrating an example of a person detected by the person detection unit according to the first embodiment of the present invention, and FIG. 3 is a first diagram illustrating the relative distance of the person according to the first embodiment of the present invention. 4 is a second diagram showing the relative distance of the person in the first embodiment of the present invention, FIG. 5 is a third diagram showing the relative distance of the person in the first embodiment of the present invention, FIG. 6 is a first diagram showing the relative speed of the person in the first embodiment of the present invention, FIG. 7 is a second diagram showing the relative speed of the person in the first embodiment of the present invention, and FIG. FIG. 9 is a diagram showing a relative speed of a person in the first embodiment of the present invention, and FIG. 9 is a diagram showing an example of notification display of the person group notifying means in the first embodiment of the present invention. 3 to 5, the horizontal axis represents time, the vertical axis represents distance, and in FIGS. 6 to 8, the horizontal axis represents time and the vertical axis represents speed.

  First, a person is detected in real time by the person detection means 12 based on the images of the respective points taken by the moving image acquisition means 11, and the person group determination means 13 based on the person position detection information output by the person detection means 12. Thus, the group of persons is determined, and the person group notification means 15 notifies the operator of the person group information.

  The person detecting means 12 detects each person using a method based on optical flow calculation and obtains the position of the real space coordinate system for each time (see, for example, Patent Document 4). Here, a plurality of cameras may be used (for example, refer to Patent Document 5), or the acquired moving image may be enlarged or reduced based on the position of the detection target to improve accuracy, or detection may be performed over a wide range. (For example, refer to Patent Document 6).

  Also, in order to convert from image coordinates to the real space coordinate system, the correspondence between the image coordinate values of each pixel in the image captured by each camera device and the world coordinate values on the floor surface is determined in advance by calibration or the like. It is necessary to keep it. As this method, for example, there is a method in which four reference points are set on the floor surface and a conversion matrix between world coordinate values and image coordinate values is calculated (for example, see Non-Patent Document 1).

[Patent Document 4] Japanese Patent Laid-Open No. 10-91793 [Patent Document 5] Japanese Patent Laid-Open No. 10-48008 [Patent Document 6] Japanese Patent Laid-Open No. 2001-266131 [Non-Patent Document 1] “Cue-based camera calibration with it” ”practical application to digital image production”, Yuji Nakazawa et al., IEICE Tech.
Then, the person group determination means 13 calculates the relative distance and the relative speed for all combinations of the persons from the positions of the persons detected by the person detection means 12 for each time, and the person of each combination is calculated. It is determined whether they belong to the same group. For example, whether or not the relative distance continues below a certain threshold for a certain period of time, or whether both the relative distance and the relative speed maintain a value smaller than a predetermined threshold in a certain time interval Is the criterion.

  Here, a case where four persons are detected by the person detecting means 12 will be described.

  In this case, as shown in FIG. 2, the person b moves closer and in the same direction with respect to the person a, the person c moves closer and in the opposite direction, and the person d moves far and moves in the same direction. Shall. In addition, the arrow in FIG. 2 has shown the mode of movement of each person. At this time, the relative distance between the person a and the person b is as shown in FIG. 3, the relative distance between the person a and the person c is as shown in FIG. 4, and the relative distance between the person a and the person d is as shown in FIG. Is as shown in FIG. Further, the relative speed between the person a and the person b is as shown in FIG. 6, the relative speed between the person a and the person c is as shown in FIG. 7, and the relative speed between the person a and the person d is As shown in FIG.

  Here, predetermined thresholds are set for the relative distance and the relative speed, respectively. 2 to 8, the set threshold value is indicated by a dotted line. In this case, only the person b is in a state where the relative distance with respect to the person a is less than the threshold value for a long time. The same result is obtained when it is determined whether or not both the relative distance and the relative speed are below the threshold value. Therefore, the person group determination unit 13 determines that the person a and the person b belong to the same group, and the person a and the person c and the person a and the person d belong to different groups.

  Then, the person group form determination means 14 determines the group form as a single action person, a large group, or the like based on the person group information output by the person group determination means 13.

  Subsequently, the person group notification means 15 superimposes the person group form information and the flow line information output from the person group form determination means 14 with a moving image, a floor plan, etc. if necessary, for example, as shown in FIG. As shown in FIG. 4, the data is displayed on the display device and output to the operator. The output contents vary depending on the application, such as a group of two people, a person acting alone, or a group of many people. Further, instead of outputting as shown in FIG. 9, the group form determination results acquired over a long period of time may be aggregated and statistically processed for each group.

  As described above, in the present embodiment, the detection device 10 acquires a moving image at each point to detect a person, examines a relative distance and a relative speed for all detected combinations of persons, Whether or not a person belongs to the same group is determined based on whether or not both the distance and relative speed values are below a predetermined threshold value set in advance. That is, the person group determination unit 13 determines a group of persons whose relative distance and relative speed are below the threshold for a time longer than the threshold as the same group. Therefore, it is possible to know whether a person is acting alone or in a group without relying on human hands, and it is possible to know what kind of group is formed. It is possible to save labor and improve the efficiency of the work of grasping the above.

  As a result, opportunities for grasping the group form of persons greatly increase. For example, marketing of a coffee shop (for example, if there are many two people on the street, prepare two seats), grasp the relationship between monkeys in a zoo ( For example, the present invention can be widely applied to who (who) and who are close friends and who are out of friends.

  Next, a second embodiment of the present invention will be described. In addition, about the thing which has the same structure as 1st Embodiment, the description is abbreviate | omitted by providing the same code | symbol. The description of the same operation and the same effect as those of the first embodiment is also omitted.

  FIG. 10 is a diagram showing a configuration of a detection device according to the second embodiment of the present invention.

  In the present embodiment, the detection apparatus 10 includes a person group space statistical unit 16 in addition to the configuration described in the first embodiment. The person group space statistics means 16 is an interface for inputting person position information from the person detection means 12, an arithmetic device or software for performing statistical processing spatially based on the person position information and calculating an appropriate threshold for group determination And an interface for outputting a group determination threshold value to the person group determination means 13.

  Next, operation | movement of the detection apparatus 10 in this Embodiment is demonstrated.

  FIG. 11 is a diagram showing the distribution of the number of appearances of the relative distance of persons in the second embodiment of the present invention, and FIG. 12 is a diagram showing the probability density function of the relative distance of persons in the second embodiment of the present invention. It is. In FIG. 11, the horizontal axis represents distance, the vertical axis represents frequency, and in FIG. 12, the horizontal axis represents distance, and the vertical axis represents probability density.

  In the present embodiment, statistical processing is spatially performed from the person detection information, and a threshold appropriate for group determination and other determination criteria are output to the person group determination means 13. In this case, the person group space statistics means 16 spatially counts the person position information acquired by the person detection means 12, performs statistical processing on the distance and relative speed between the persons, and determines threshold values and other determinations. A reference is calculated and output to the person group determination means 13.

  For example, all the distances between persons at a certain time are totaled as shown in FIG. 11, and the distribution is applied to a certain probability density function. Then, based on the probability density, as shown in FIG. 12, a threshold value adapted to the current situation is determined. Thereby, for example, when the distance between two persons is a value represented by a solid line arrow (downward arrow) shown in FIGS. 11 and 12, the value is smaller than the calculated threshold value. The two people are determined to be in the same group.

  Thus, in the present embodiment, statistical processing is performed on the distance and relative speed between persons, and a threshold value and other determination criteria are calculated based on the result of the statistical processing and used for group determination. That is, the person group determination unit 13 includes a person group space statistical unit 16 that performs statistical processing on all the values of the mutual positions of the detected persons and calculates a threshold value used for grouping the persons. Group determination is performed with a threshold adapted to the degree of congestion. Therefore, group determination can be performed based on an appropriate standard that matches the degree of congestion at each point, and the accuracy of group determination can be improved.

  For example, since many people are approaching at a crowded point, in the first embodiment, it is easy to misjudg a person in another group nearby as the same group. In this embodiment, since the distance threshold is lowered, this erroneous determination is reduced. In addition, for example, since the distance between each other is easily separated even in the same group at the point where the person is sitting, in the first embodiment, it is easy to erroneously determine that it is a different group. Since the distance threshold value increases, this erroneous determination is reduced.

  Next, a third embodiment of the present invention will be described. In addition, about the thing which has the same structure as 1st and 2nd embodiment, the description is abbreviate | omitted by providing the same code | symbol. Also, the description of the same operations and effects as those of the first and second embodiments is omitted.

  FIG. 13 is a diagram showing a configuration of a detection device according to the third embodiment of the present invention.

  In the present embodiment, the detection apparatus 10 includes a person feature extraction unit 17 in addition to the configuration described in the first embodiment. The person feature extracting unit 17 is an interface for acquiring a moving image from the moving image acquiring unit 11, and an interface for acquiring an existing position on the image of a person whose features are to be extracted as person position information from the person group determining unit 13. The personal attribute information of the extracted person is output to the computing device or software for extracting the personal attribute information of each person detected based on the moving image and the person position information, and the person group form determining means 14 Provide an interface.

  Next, operation | movement of the detection apparatus 10 in this Embodiment is demonstrated.

  FIG. 14 is a diagram showing an example of notification display of the person group notification means in the third embodiment of the present invention.

  In the present embodiment, personal attribute information such as the detected face image and physique of each person is extracted by the person feature extraction unit 17 and used as additional information for determination processing and output by the person group determination unit 13. It is like that.

  In this case, the person feature extraction means 17 cuts out the area of each person from the moving image acquired from the moving image acquisition means 11 on the basis of the person position information acquired from the person group determination means 13, and The attributes are extracted and output to the person group form determination means 14. Here, as necessary, the acquired image may be enlarged or reduced based on the person position information. Alternatively, the person feature extraction unit 17 may first detect an image region that is known to be a person from a face image, specific clothes, and the like, and then perform person detection by narrowing down the person detection unit 12 based on the information. Good.

  Here, there are various personal attributes of a person that can be extracted from an image, such as a face image, a physique, an age, a sex, and clothes. For example, a method for extracting a face region and clothes (for example, see Patent Documents 7 and 8), a method for extracting a physique (for example, see Patent Document 9), and a method for extracting age and sex (for example, Patent Document 10). And 11). In addition, methods for detecting wheelchairs (chairs), elderly people, visually handicapped persons, and other vulnerable persons (see, for example, Patent Documents 12 and 13) are also known.

[Patent Document 7] JP 2001-216515 [Patent Document 8] JP 11-283001 [Patent Document 9] JP 10-105748 [Patent Document 10] JP 2001-167110 [Patent Document 9] [Patent Document 11] Japanese Patent Laid-Open No. 11-175724 [Patent Document 12] Japanese Patent Laid-Open No. 2001-101576 [Patent Document 13] Japanese Patent Application Laid-Open No. 2002-260161. For example, the following events (1) to (5) are determined from the person group information sent from the person and the person attribute information sent from the person feature extracting means 17.
(1) IF group = 1 person AND character feature = children THEN = event lost (2) IF group = 2 people AND person 1 = adult male AND person 2 = adult female AND age difference <TH AGE DIF THEN Event = couple (3) IF 2 people < group <TH NUM AND Number of adults > 1 AND Number of children > 1 THEN Event = Parents and children (4) IF group > TH NUM1 AND Number of people in same outfit > TH NUM2 THEN event = organization (5) IF group. Accompanying person > TH NUM3 AND group. Companion <TH NUM4 THEN Event = no companion required *** is a threshold value arbitrarily set by the operator. A companion is a person who needs a companion, such as a disabled person or a young person. The operator may set it arbitrarily.

  The person group notification means 15 adds the personal attribute means of the person acquired from the person feature extraction means 17 to the display in the first embodiment, and displays the display device as shown in FIG. 14, for example. To display. When the person group determination results are totaled, attribute persons are used as the key for the totalization.

  Thus, in the present embodiment, attribute information can be extracted for a detected person, added to the person group determination person, and provided to the operator, or used as a key for statistical processing. That is, the person group form determination unit 14 extracts and determines the person's face image, physique, age, gender, clothes or presence / absence as feature information based on the detected position of the person. And determining the group form by combining the feature information of the person and the group determination information. Therefore, the operator can grasp the relationship between the person group and the personal attribute, and can analyze the group form of the person in more detail. In addition, when a face image is extracted, the operator can know who is a person who belongs to the group.

  Furthermore, the person group form determination unit 14 determines whether the detected person is a single action person by the person group determination unit 13 and the person feature extraction unit 17 determines that the person is a child from the physique and age. Is determined to be lost. Furthermore, the person group form determination means 14 determines that the detected person is acting in a group of two persons, and the person feature extraction means 17 indicates that both of the persons are adult ages and the age is close. When one is determined to be male and the other is determined to be female, the group form of the group is determined to be a couple. Further, the person group form determining means 14 determines that the detected person is acting in a group of a plurality of persons, and the person feature extracting means 17 includes one person of an adult age and one child in each group. When it is determined that more than one person exists, the group form of the group is determined to be accompanied by a parent and child. Furthermore, the person group form determination means 14 determines that the detected person is acting in a group having a number greater than or equal to the threshold, and the person feature extraction means 17 determines that the number of persons in the group equal to or greater than the threshold is dressed. If it is determined that they are the same, the group form of the group is determined to be an organization. Further, the person group form determination means 14 determines the feature that requires a companion from the detected person by the person feature extraction means 17 and does not have the feature in the same group and determines that the person is a companion. When there are not enough persons, the detected person is determined to be a companion without a companion. The feature requiring the accompanying person is either that the age is below the threshold value or that a disorder exists.

  For example, FIG. 14 shows a notification display when gender, height, estimated age, and clothing characteristics are extracted for all persons and a face image is also extracted for a single actor. According to this, since it can be estimated that the left two-person group is a “couple”, it is possible to take measures such as preparing a two-seater seat at the coffee shop. In addition, since the right single person is 5 years old, it can be estimated that he / she is “lost”, so the employee of the facility goes out to protect the child with the extracted characteristic person as a clue, and at the same time A staff member of the lost child information center can take a response such as showing a face image to a parent searching for his child and confirming it.

  In addition, information such as “with parents and children” and “group” is useful in dealing with customers for retail stores and transportation facilities. In addition, information such as “Non-accompanied companion” is required for game centers that require young people to be accompanied, and for the railway company to provide assistance to disabled persons who are not accompanied by a railway company. It can be used to improve the soundness and service of facility management.

  In addition, since the person feature extraction unit 17 detects a face image, specific clothes, and the like before the person detection by the person detection unit 12, the image area is narrowed down in the person detection. Compared to this form, the speed and accuracy of person detection may be improved.

  Next, a fourth embodiment of the present invention will be described. In addition, about the thing which has the same structure as the 1st-3rd embodiment, the description is abbreviate | omitted by providing the same code | symbol. Explanation of the same operations and effects as those of the first to third embodiments is also omitted.

  FIG. 15 is a diagram showing a configuration of a detection device according to the fourth embodiment of the present invention.

  In the present embodiment, the detection apparatus 10 includes a person search unit 18 in addition to the configuration described in the third embodiment. The person search means 18 has a function of performing a narrowing search on the person information sent to the person group notification means 15, and transmits group form information to an interface through which the operator inputs search conditions and the person group determination means 13. An interface, an interface for outputting the extraction result to the person group notification means 15, and an arithmetic unit or software for narrowing down and searching for a search result that matches the search condition.

  Next, operation | movement of the detection apparatus 10 in this Embodiment is demonstrated.

  FIG. 16 is a diagram showing an example of notification display of the person group notification means in the fourth embodiment of the present invention.

  In the present embodiment, the person search means 18 narrows down the output of the person group notification means 15.

  In this case, the person search means 18 receives a search condition input for searching for a person at the place from the operator. Next, information about the group form among the search conditions is transmitted to the person group determination means 13 so that only persons who meet the condition are sent from the person group determination means 13 to the person feature extraction means 17. Then, the information output from the person feature extraction means 17 is received, and only the information that matches the search condition is narrowed down and extracted and sent to the person group notification means 15. The person group notification means 15 displays the extraction result together with the search condition, for example, as shown in FIG.

  As the search condition, it is possible to specify what group form the person is acting on. In this respect, personal attribute information such as the physique, gender, age, clothes, etc. described in the third embodiment. This is different from the method of searching using only as a key.

  As described above, in the present embodiment, the operator arbitrarily designates the extracted person based on the group action form of the person and personal attribute information such as the person's physique, gender, age, and clothes. Information such as the position of each person is obtained by narrowing down the search conditions. That is, the person group notifying unit 15 includes a person searching unit 18 that narrows down and searches for persons who meet a specified condition based on the detected position of the person, group determination information, or personal attribute information. The information of the person who meets the specified conditions is narrowed down and output. Therefore, the operator can check only the group action form that is of interest to the user and the information of the person with the personal attribute, and can reduce the labor of the operator in the checking work.

  Next, a fifth embodiment of the present invention will be described. In addition, about the thing which has the same structure as 1st-4th embodiment, the description is abbreviate | omitted by providing the same code | symbol. Explanation of the same operations and effects as those of the first to fourth embodiments is also omitted.

  FIG. 17 is a diagram showing a configuration of a detection apparatus according to the fifth embodiment of the present invention.

  In the present embodiment, in addition to the configuration described in the third embodiment, the detection apparatus 10 includes a face authentication unit 20 connected to a subsequent stage of the person feature extraction unit 17 and the face authentication unit 20. It has a connected face authentication dictionary storage means 19. The face authentication dictionary storage means 19 includes a disk device for storing face authentication dictionary data to be collated in face authentication, and a camera or scanner device for acquiring data.

  Further, the face authentication unit 20 receives an interface for inputting person attribute information such as a face image from the person feature extraction unit 17, and the input face image and the face authentication dictionary data stored in the face authentication dictionary storage unit 19. An arithmetic processing device or software that collates and outputs an authentication score, and an interface that outputs a face authentication result to the person group notification means 15 are provided.

  Next, operation | movement of the detection apparatus 10 in this Embodiment is demonstrated.

  FIG. 18 is a diagram showing an example of notification display of the person group notification means in the fifth embodiment of the present invention.

  In the present embodiment, face authentication is performed on the face image extracted by the person feature extraction means 17 and an authentication score is output.

  In this case, the face authentication dictionary storage unit 19 acquires a face image of the person to be extracted, and creates and stores face authentication dictionary data. The face authentication means 20 collates the face image extracted by the person feature extraction means 17 with the face authentication dictionary data stored in the face authentication dictionary storage means 19 and outputs an authentication score.

  Further, the person group notifying means 15 adds the authentication score to the display of each person, for example, as shown in FIG. In this case, if necessary, the personal information may be output limited to those having a somewhat high authentication score.

  Further, the face authentication dictionary storage means 19 acquires a face image in advance using a camera device or a scanner device, converts the acquired face image into a face authentication dictionary data format, and stores it. The face authentication unit 20 reads the stored face authentication dictionary data, compares it with the face image output from the person feature extraction unit 17, and outputs an authentication score for each combination. The creation of the face authentication dictionary data and the face authentication can be performed using a known method (for example, see Patent Document 14).

[Patent Document 14] Japanese Patent Laid-Open No. 2002-183734 As described above, in the present embodiment, face authentication is performed on the extracted face image, the authentication score is output, and position information, group information, and personal information are output. Are output together with typical attribute information. That is, the person group form determination unit 14 acquires the face image of the detected person and stores the face image extracted by the face authentication dictionary storage unit 19 and the person feature extraction unit 17 as face authentication dictionary data. Face authentication means 20 for outputting an authentication score by collating with the face authentication dictionary data is provided, and the authentication score and person characteristic information are used for person group form determination. Therefore, for example, when a person who belongs to the same group, such as a lost child, is separated from a friend, it is determined whether or not the person at each point is a searched person using face authentication It is possible to know exactly and quickly where the person being searched is.

  In addition, compared to the method of searching for a person only with face authentication, face authentication is performed only for persons who have been narrowed down to some extent by group behavior information and other personal attribute information. Can be reduced.

  Next, a sixth embodiment of the present invention will be described. In addition, about the thing which has the same structure as 1st-5th embodiment, the description is abbreviate | omitted by providing the same code | symbol. Explanation of the same operations and effects as those of the first to fifth embodiments is also omitted.

  FIG. 19 is a diagram showing a configuration of a detection device according to the sixth embodiment of the present invention.

  In the present embodiment, the detection apparatus 10 includes a facial expression recognition unit 21 connected to the subsequent stage of the person feature extraction unit 17 in addition to the configuration described in the third embodiment. The facial expression recognition means 21 includes an arithmetic processing device or software for recognizing a human facial expression from a face image, and an interface for receiving the facial image from the person feature extraction means 17.

  Next, operation | movement of the detection apparatus 10 in this Embodiment is demonstrated.

  FIG. 20 is a diagram showing an example of notification display of the person group notification means in the sixth embodiment of the present invention.

  In the present embodiment, the facial expression recognition means 21 recognizes a human facial expression from a facial image and uses it as additional information for search and determination.

  In this case, the facial expression recognizing means 21 recognizes the facial expression of the person from the face image output from the person feature extracting means 17 by a known method (for example, see Patent Document 15). If necessary, the face color of a person is recognized by another known method (for example, see Patent Document 16), or by another known method (for example, see Patent Document 17). The direction may be recognized.

[Patent Document 15] Japanese Patent Application Laid-Open No. 4-342078 [Patent Document 16] Japanese Patent Application Laid-Open No. 2000-275997 [Patent Document 17] Japanese Patent Application Laid-Open No. 2002-352228. It is sent to the determination means 14, and person group form determination as shown in the following (6) to (9) is performed.
(6) IF group = 1 person AND Characteristic feature = Child AND Facial expression = Crying THEN Event = Lost child (7) IF group = 1 person AND Characteristic feature = Child AND Face direction change angle amplitude> TH ANG AMP AND face change count / sec> TH ANG FRQ THEN event = lost child (8) IF group > 2 people AND facial expression = get angry THEN event = proximity abnormal behavior (9) IF group. Accompanying person > TH NUM3 AND group. Accompanying person <TH NUM4 THEN Event = no companion required companion The companion must be red (drunk) or pale (sick), except for the fact that the focus is on the complex. It is the same as the form.

  Further, the person group notification means 15 performs output as shown in FIG. 20, for example, by person group form determination based on facial expression recognition.

  As described above, in this embodiment, the facial expression recognition means 21 is provided to recognize the facial expression and facial color of a person and use it for person group form determination. That is, the person group form determination unit 14 includes a facial expression recognition unit 21 that recognizes an expression, a face color, and a face direction from the face image extracted by the person feature extraction unit 17, and uses the expression and the face color to determine the group form. I do. Therefore, it is possible to know in more detail what the form of the person group is based on the facial expression and facial color of the person, and it is possible to output a result more in line with the operator's wishes.

  Furthermore, the person group form determination means 14 determines that the detected person is a child who acts alone and the facial expression recognition means 21 recognizes that the expression is “crying”, or the face group orientation determination means 14 When it is recognized that the frequency of change is higher than a threshold and the amplitude of the change is higher than the threshold, the person is determined to be lost. Furthermore, the person group form determination means 14 determines that the detected person is the group action person, and the facial expression recognition means 21 recognizes the facial expression of each person as “angry”. In addition, it is determined that the group form of the group of persons is proximity abnormal behavior. Furthermore, the person group form determination unit 14 includes a facial expression recognition unit 21 that recognizes a facial expression, a facial color, and a face direction from the facial image extracted by the person feature extraction unit 17, and the person from which the facial expression recognition unit 21 is detected. When the face color is recognized as “red” or “white”, the person is determined to be an accompanying person without an accompanying person.

  For example, in the case of detecting a lost child, in the third embodiment, a child who is not a lost child but is separated from his / her parent, such as a child playing in a kids space to detect a single person who is young. Are prone to be detected incorrectly. However, in this embodiment, the expression of “crying” frequently appearing on the face of a lost child or the fact that the face direction is frequently moved as a key is added, so that it can be detected more accurately. it can.

  Also, for example, proximity abnormal behavior such as fighting of fighting and criminal acts can be detected with certainty by recognizing that a person belonging to the same group has an angry face. . In particular, when the face is red, it can be detected with certainty as a fight of drunkness.

  Furthermore, there are cases where a person who needs monitoring drunkness or a poor physical condition requiring protection may be detected as an excessive number of persons who are drunk or have poor physical condition simply by looking at the complexion. However, in this embodiment, by adding information indicating that it is a single action, that is, information indicating that there is no person to be monitored or protected in the vicinity, it is particularly limited to a person who needs to be monitored or protected. Can be detected.

  Next, a seventh embodiment of the present invention will be described. In addition, about the thing which has the same structure as 1st-6th embodiment, the description is abbreviate | omitted by providing the same code | symbol. Explanation of the same operations and effects as those of the first to sixth embodiments is also omitted.

  FIG. 21 is a diagram showing a configuration of a detection apparatus according to the seventh embodiment of the present invention.

  In the present embodiment, in addition to the configuration described in the third embodiment, the detection apparatus 10 includes a human sound connected to the subsequent stage of the sound acquisition means 22 and the person feature extraction means 17 as input means. It has feature extraction means 23. The sound acquisition unit 22 includes a microphone device and an A / D converter. In addition, the human acoustic feature extraction unit 23 includes an interface that receives an acoustic signal from the acoustic acquisition unit 22, a storage device that stores a pattern signal related to the human feature, and an operation that recognizes a pattern related to the human feature from the acoustic signal. A processing device or software is provided.

  Next, operation | movement of the detection apparatus 10 in this Embodiment is demonstrated.

  FIG. 22 is a diagram showing the azimuth and level characteristics of characteristic sounds in the seventh embodiment of the present invention, and FIG. 23 is a diagram showing the position of a person detected from an image in the seventh embodiment of the present invention. .

  In the present embodiment, the acoustic characteristics of the person detected using the acoustic acquisition means 22 and the human acoustic feature extraction means 23 are extracted and used as additional information for person group determination.

  In this case, the sound acquisition unit 22 acquires a sound signal using a microphone or the like disposed at the same location as the camera device of the moving image acquisition unit 11 and sends the sound signal to the person sound feature extraction unit 23. The person acoustic feature extracting means 23 stores the person's acoustic feature pattern in the database in advance, and each time an acoustic signal is sent, a pattern indicating the feature of the person's action is included in the acoustic signal. It is detected and sent to the person group determination means 13 as person acoustic feature information. A method for detecting this feature from an input acoustic signal is known (see, for example, Patent Document 18). As a result, when a certain characteristic sound is detected, it is estimated that any of the persons detected at that time emits the sound.

  Here, the sound acquisition means 22 is provided with directivity, the sound generation direction is estimated using a known sound source direction estimation method (see, for example, Patent Document 19), and the direction information and the person detection means 12 are used. It may be combined with person position information. For example, when the level for each direction is measured for the specific sound that is the sound acquisition means 22, the upper left level is high and a person is detected at the position shown in FIG. 23 as shown in FIG. The person whose number is detected at the upper left in FIG. 23 is the person who is emitting the specific sound.

  In addition, the human acoustic feature extraction means 23 may recognize an acoustic signal that seems to be a person's utterance from the acoustic signal using a known speech recognition method (see, for example, Patent Document 20).

[Patent Document 18] JP-A-8-54891 [Patent Document 19] JP-A-5-87903 [Patent Document 20] JP-A-5-66790 Taking into account the acoustic features of the person extracted by the means 23, person group form determination is performed as shown in the following (10) to (13).
(10) IF group = 1 person AND person feature = child AND feature sound = child crying THEN event = lost child (11) IF group = 1 person AND person feature = child AND utterance content = remarks that can be estimated to be lost ( THEM Event = Lost Child (12) IF Group > 2 people AND Utterance Content = Remarks that can be presumed to be a crime or a fight ("Drobo", "Kill", "Konoyaro", "Tame", "Kisama" Etc.) THEN event = criminal action (13) IF group > 2 people AND characteristic sound = sound that can be presumed to be a crime or a fight (scream, beating sound, etc.) THEN event = criminal action The person group notification means 15 Performs output by person group form determination based on the extracted person acoustic features. Here, the extracted characteristic sounds may be output simultaneously.

  As described above, in the present embodiment, the sound acquisition means 22 and the person sound feature extraction means 23 are provided and used to extract the person's feature sound and determine the form of the person group. Yes. That is, the person group form determination unit 14 includes an acoustic acquisition unit 22 that acquires sounds at each point, and a human acoustic feature extraction unit 23 that determines the acoustic characteristics of the group from the acquired sounds. Based on this, the person group form is determined. Further, the sound acquisition unit 22 acquires sound with directivity, and when the detected person exists in a direction in which the sound level acquired by the sound acquisition unit 22 is strong, the person group form determination unit 14 It is determined that the person has made the acquired sound. Furthermore, the person acoustic feature extraction unit 23 recognizes the detected utterance of the person. Furthermore, the person group form determination means 14 is a child in which the acoustic characteristics of a lost child such as a child's cry or utterance calling a companion is extracted from the person by the person acoustic feature extraction means 23 and the person acts alone. If it is determined that there is a person, the person is determined to be lost. Furthermore, the person group form determination means 14 determines the person group belonging to the same group when the acoustic characteristics of criminal acts such as yelling, beating sound, scream, etc. are extracted by the person acoustic feature extraction means 23. Is determined to be a criminal act. Therefore, it is possible to know in detail the form of the person group based on the sound emitted by the person such as a voice, and it is possible to output a result according to the operator's desire.

  Next, an eighth embodiment of the present invention will be described. In addition, about the thing which has the same structure as 1st-7th embodiment, the description is abbreviate | omitted by providing the same code | symbol. Explanation of the same operations and effects as those of the first to seventh embodiments is also omitted.

  FIG. 24 is a diagram showing a configuration of a detection device according to the eighth embodiment of the present invention.

  In the present embodiment, in addition to the configuration described in the seventh embodiment, the detection apparatus 10 includes a face authentication unit 20 and a facial expression recognition unit 21 connected to the subsequent stage of the person feature extraction unit 17, and A face authentication dictionary storage unit 19 connected to the face authentication unit 20 is provided, and a person feature score conversion unit 24 is provided instead of the person group form determination unit 14 in the first to seventh embodiments.

  The person feature scoring unit 24 converts all the information such as the input person group information, person feature information, facial expression information, face authentication score, and person acoustic feature person into a score and outputs the score. And an interface for an operator to set conditions for score conversion, and an arithmetic processing unit or software.

  Next, operation | movement of the detection apparatus 10 in this Embodiment is demonstrated.

  FIG. 25 is a diagram showing an example of notification display of the person group notification means in the eighth embodiment of the present invention.

  In the present embodiment, the person characteristic scoring means 24 converts a score into consideration by adding a plurality of pieces of information related to the detected person, such as person group information, and uses it as person group form information.

  In this case, the person feature scoring means 24 receives information on the number of groups sent from the person group determination means 13, person feature information such as age and clothes sent from the person feature extraction means 17, and facial expression recognition means 21. All the information such as the facial expression information of the sent person, the face score sent from the face authenticating means 20, the person acoustic feature information sent from the person acoustic feature extracting means 23, etc. are taken into account, and the setting conditions are arbitrarily set The person feature score is calculated by weighted addition based on the above. This setting condition can be freely set by the operator prior to processing.

  Then, the person group notifying unit 15 performs output in descending order of the person feature score, for example, as shown in FIG. 25, based on the calculated person feature score. For example, a five-year-old child wearing red clothes is lost and needs to search for the child. In this case, since the child is a lost child, there is a high probability of crying by acting alone, that is, a crying expression with a crying expression. In addition, as a person feature, there is a high probability that a person whose clothes are red and whose age is determined to be 5 years old is a child to be searched. Further, when face authentication is performed, there is a high probability that a person with a high face authentication authentication score is a child to be searched. Therefore, the score is high when the condition that the person is acting alone, the clothes are red and the age is 5 years old, the face is crying, the cry is raised, and the face authentication score is high is satisfied. Such a calculation is performed, a score is calculated for each detected person, and person information is output in descending order of the calculated score.

  Thus, in the present embodiment, the person feature scoring means 24 is provided, and a plurality of features of the person are converted into a single score unified based on conditions arbitrarily set by the operator, By using it for person group form determination, a plurality of person determination methods can be combined and the result can be output centrally. In other words, instead of the person group form determination means 14, there is provided a person feature scoring means 24 for scoring information by weighted addition of group determination, person characteristics, face authentication score, facial expression, and acoustic feature information. The person information is output based on the calculated value. Therefore, a desired person group form determination result can be viewed without referring to a plurality of pieces of person information, and the operator's workload can be reduced.

  In addition, recognition information such as person position, group determination result, output person characteristics such as age and clothes, facial expressions, face authentication score, etc., acquired image and sound status, lighting conditions, congestion of the point to be evaluated, etc. May cause significant blurring. However, in this embodiment, by combining a plurality of these and outputting them as a unified score, even if some of the recognition means are blurred, accurate recognition information can be obtained by other recognition means. If so, it is possible to suppress the adverse effects of blurring and to output a more reliable recognition result.

  In the first to eighth embodiments, there is no means for obtaining personal information other than images and sounds, but other means for obtaining personal information such as a card pass and a non-contact tag. Even if there is, the person group form determination means 14 etc. can use the information in consideration.

  The present invention is not limited to the above-described embodiment, and various modifications can be made based on the spirit of the present invention, and they are not excluded from the scope of the present invention.

It is a figure which shows the structure of the detection apparatus in the 1st Embodiment of this invention. It is a figure which shows the example of the person detected by the person detection means in the 1st Embodiment of this invention. It is a 1st figure which shows the relative distance of the person in the 1st Embodiment of this invention. It is a 2nd figure which shows the relative distance of the person in the 1st Embodiment of this invention. It is a 3rd figure which shows the relative distance of the person in the 1st Embodiment of this invention. It is a 1st figure which shows the relative speed of the person in the 1st Embodiment of this invention. It is a 2nd figure which shows the relative speed of the person in the 1st Embodiment of this invention. It is a 3rd figure which shows the relative speed of the person in the 1st Embodiment of this invention. It is a figure which shows the example of the notification display of the person group notification means in the 1st Embodiment of this invention. It is a figure which shows the structure of the detection apparatus in the 2nd Embodiment of this invention. It is a figure which shows distribution of the appearance number of the relative distance of the person in the 2nd Embodiment of this invention. It is a figure which shows the probability density function of the relative distance of the person in the 2nd Embodiment of this invention. It is a figure which shows the structure of the detection apparatus in the 3rd Embodiment of this invention. It is a figure which shows the example of the notification display of the person group notification means in the 3rd Embodiment of this invention. It is a figure which shows the structure of the detection apparatus in the 4th Embodiment of this invention. It is a figure which shows the example of the notification display of the person group notification means in the 4th Embodiment of this invention. It is a figure which shows the structure of the detection apparatus in the 5th Embodiment of this invention. It is a figure which shows the example of the notification display of the person group notification means in the 5th Embodiment of this invention. It is a figure which shows the structure of the detection apparatus in the 6th Embodiment of this invention. It is a figure which shows the example of the notification display of the person group notification means in the 6th Embodiment of this invention. It is a figure which shows the structure of the detection apparatus in the 7th Embodiment of this invention. It is a figure which shows the azimuth | direction and level characteristic of characteristic sound in the 7th Embodiment of this invention. It is a figure which shows the position of the person detected from the image in the 7th Embodiment of this invention. It is a figure which shows the structure of the detection apparatus in the 8th Embodiment of this invention. It is a figure which shows the example of the notification display of the person group notification means in the 8th Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Detection apparatus 11 Moving image acquisition means 12 Person detection means 13 Person group determination means 14 Person group form determination means 15 Person group notification means 16 Person group space statistics means 17 Person feature extraction means 18 Person search means 19 Face authentication dictionary storage means 20 Face authentication means 21 Facial expression recognition means 22 Sound acquisition means 23 Human acoustic feature extraction means 24 Human feature score conversion means

Claims (9)

(A) a moving image acquisition means for acquiring a moving image at each point in a place where an unspecified number of persons gather;
(B) a person detecting means for detecting a person and the position of the person from the acquired moving image every predetermined time;
(C) Based on the mutual positional relationship between the detected persons, it is determined that a state in which the relative distance is less than the threshold value belongs to a group performing the same action for a group of persons that lasts longer than the threshold value. Person group determination means for performing group determination for grouping the persons;
(D) From the result of the group determination, a person group form determination unit that determines a single form or group action and a group form by paying attention to the number of persons;
(E) a person group notification means for notifying an operator of the group form;
(F) A single-behavior and group-behavior detection device that performs group determination on a person included in the moving image. 2. The single-behavior / group-behavior detection device according to claim 1, wherein the person group determination unit determines that a group of persons whose state in which the relative distance and the relative speed are below a threshold value continues for a longer time than the threshold is the same group. . The person group determination unit includes a person group space statistical unit that performs statistical processing on all the values of the detected mutual positional relationship of the persons and calculates a threshold value used for grouping the persons, The single-behavior and group-behavior detection device according to claim 1 or 2, wherein the group determination is performed with a threshold adapted to the degree. The person group form determining means is a person feature extracting means for extracting and determining the face image, physique, age, sex, clothes or presence / absence of the person as feature information based on the detected position of the person. The single-behavior and group-behavior detection device according to any one of claims 1 to 3, wherein a group form is determined by combining the feature information of the person and the group determination information. The person group notification means includes person search means for narrowing down and searching for persons who meet a specified condition based on the detected position of the person, group determination information, or personal attribute information. The single-behavior and group-behavior detection device according to any one of claims 1 to 4, wherein information on a person corresponding to a condition is narrowed down and output. The person group form determination means acquires the detected face image of the person and stores it as face authentication dictionary data, and the face authentication extracted by the person feature extraction means. The single actor and group actor detection device according to claim 4 or 5, further comprising face authentication means for collating with dictionary data and outputting an authentication score, and using the authentication score and person characteristic information for person group form determination. The person group form determination means comprises a facial expression recognition means for recognizing an expression, face color and face direction from the face image extracted by the person feature extraction means, and performs group form determination using the expression and face color. Item 7. The single actor and group actor detection device according to any one of Items 4 to 6. The person group form determination means includes sound acquisition means for acquiring sounds at the respective points, and person sound feature extraction means for determining acoustic characteristics of the group from the acquired sounds, based on the acoustic features. The single action person and group action detection device given in any 1 paragraph of Claims 1-7 which judge a person group form. In place of the person group form determining means, there is a person feature scoring means for weighting and adding information of group determination, person characteristics, face authentication score, facial expression, and acoustic characteristics, and the value calculated as the score The single action person and group action person detection device according to claim 1 which outputs person information based on it.

Images