JP7208051B2 - State recognition device - Google Patents

Description

The present invention relates to a state recognition device and a state recognition method for recognizing the state of an object to be measured based on the output of a measuring device.

In recent years, based on information such as image data and distance data acquired by measurement devices such as surveillance cameras and distance sensors, state recognition recognizes the state within the measurement range of the measurement device and the state of the measurement target (person, vehicle, etc.) Demand for technology is increasing. For example, technology for detecting abnormal conditions (such as traffic jams) in the flow of people and traffic, and technology for detecting abnormal behavior of people and vehicles (fallen people, violent people, broken down vehicles, etc.).

With this type of state recognition technology, processing resources are small if only the amount of state change in the measurement range is roughly detected, but if advanced algorithms are used to recognize the detailed behavior of people and vehicles. requires a large amount of processing resources, it is necessary to take measures to reduce the processing resources. Therefore, processing resources are often reduced by setting a part of a wide measurement range in advance as a recognition area for executing state recognition.

However, when recognizing an abnormal state with a surveillance camera installed in a public place, it is unclear in which area the abnormality will occur, so the preset recognition area may not be appropriate. Therefore, in state recognition technology using advanced algorithms, there are high expectations for technology that determines a recognition area according to the actual situation.

As a method of determining the recognition area according to the actual situation, there is a method described in Patent Document 1. In this document, the object to be measured is a vehicle, and by analyzing time change information such as the amount of movement of the vehicle, it is possible to specify a recognition area where an abnormal state is assumed to occur. By executing state recognition processing using advanced image analysis after identifying the recognition area, it is possible to recognize in detail abnormal states and the conditions of vehicles and people within the measurement range while reducing processing resources, and issue alarms in stages. It is possible to realize highly accurate anomaly detection such as reduction of alarms and false alarms.

Japanese Patent Application Laid-Open No. 2002-190090

However, Patent Literature 1 identifies an area where an abnormality in traffic flow is expected by analyzing video from a camera and detecting time-varying changes in traffic volume. In the case where measurement targets are limited to vehicles and people with various traveling directions and actions are to be measured, there is a problem that an appropriate recognition area cannot be set.

Therefore, the present invention provides a state recognition apparatus for recognizing a measurement range and a state of a measurement object, based on interaction information between the measurement objects extracted from the general flow information of the movement of the measurement object. It is an object of the present invention to provide a state recognition device capable of executing advanced state recognition processing while reducing the processing resources of the entire device by selecting in advance a measurement range and a measurement target for executing the above.

In order to achieve the above object, the state recognition apparatus of the present invention includes a motion information generation unit that generates motion information of a measurement target based on information acquired by a measurement device, a recognition target candidate selection unit that selects recognition target candidates from the entire range; an interaction extraction unit for extracting action information; a recognition target determination unit for determining a recognition target for which state recognition processing is to be executed based on the interaction information from the recognition target candidates ; and a state recognizing unit for execution.

According to the state recognition device of the present invention, it is possible to select in advance a measurement range and a measurement target for executing state recognition processing, and to execute advanced state recognition processing while reducing the processing resources of the entire device.

Functional block diagram of the state recognition device of the first embodiment Functional block diagram of the people flow information acquisition unit Diagram for explaining processing in the human position measurement unit Diagram explaining the processing in the flow line creation part Diagram for explaining the processing in the people flow information generation unit Functional block diagram of recognition area candidate selection unit Diagram for explaining the processing in the people flow information comparison unit Functional block diagram of human flow interaction extraction unit Diagram for explaining the processing in the positional relationship calculation unit Functional block diagram of the state recognition device of the second embodiment

Hereinafter, specific embodiments of the state recognition device of the present invention will be described with reference to the drawings.

First, using the functional block diagram of FIG. 1, the outline of the state recognition device 1 according to the first embodiment of the present invention will be explained. In the present embodiment, a monocular camera 2 is used as a measuring device to be connected, and a state recognition device 1 in which a person H is used as an object to be measured will be described. Instead, the measuring device may be another sensor such as a stereo camera or a distance sensor, and the measurement target may be a vehicle or the like. The state recognition device 1 is actually a computer including hardware such as an arithmetic device such as a CPU, a main storage device such as a semiconductor memory, an auxiliary storage device such as a hard disk, and a communication device. Then, while referring to the database recorded in the auxiliary storage device, the arithmetic unit executes the program loaded in the main storage device, thereby realizing each function described later. Description will be made while omitting it as appropriate.

The state recognition device 1 extracts the interaction information Ii from the people flow information Ih of the entire measurement range of the image data P captured by the camera 2, and based on this, determines in advance a narrow recognition area A to which advanced state recognition processing is applied. After that, by executing advanced state recognition processing on the determined recognition area A, the device aims to reduce processing resources. In FIG. 1, the state recognition device 1 and the camera 2 are configured separately, but the state recognition device 1 may be incorporated into the camera 2 to integrate the two.

As shown in FIG. 1, the state recognition apparatus 1 of the present embodiment includes a people flow information acquisition unit 3 that acquires people flow information Ih by analyzing image data P captured by a camera 2, and a people flow information acquisition unit 3 outputs Based on the people flow information Ih acquired by the recognition area candidate selection unit 4 and the people flow information acquisition unit 3, the recognition area candidate selection unit 4 selects the recognition area candidate C as a candidate for executing the state recognition processing by calculating the amount of change in the people flow from the people flow information Ih. a human flow interaction extraction unit 5 for extracting the recognition area candidate C selected by the recognition area candidate selection unit 4 or the interaction information Ii between the person H in the recognition area candidate C and other persons; A recognition area determination unit 6 that determines the recognition area A from the interaction information Ii acquired by 5, and a state recognition unit 7 that performs state recognition processing on the determined recognition area A. The details of each unit will be described below.
<People Flow Information Acquisition Unit 3>
The details of the people flow information acquisition unit 3 (motion information generation unit) that generates the people flow information Ih, which is the movement information when the measurement target is a person, will be described with reference to the functional block diagram shown in FIG. 2 . As shown here, the people flow information acquisition unit 3 measures the position of the person H by executing an image analysis or the like on the image acquisition unit 31 that acquires the image data P from the camera 2 and the acquired image data P. A person position measurement unit 32, a flow line creation unit 33 that creates a flow line L from the measured position information of the person H, and from the position information of the person H and the flow line information acquired from the person position measurement unit 32 and the flow line creation unit 33 A people flow information generator 34 is provided to generate people flow information Ih. Hereinafter, each part of the person position measurement part 32, the flow line creation part 33, and the people flow information generation part 34 will be described in order.
<Human Position Measuring Unit 32>
First, the human position measuring unit 32 will be described with reference to FIG. In FIG. 3, P is the image data acquired from the camera 2, H is the person to be measured, F is the rectangular person area surrounding the person H, B is the foot position of the person H on the image coordinates, and S3 is the _three -dimensional image. Space, B3 is the foot position of the person H on the _{three -} dimensional space S3 _, S2 is the xz plane at the position of y = ₀ on the _{three -} dimensional space S3, B2 is the person on the xz plane S2 The foot position of H is shown.

First, the person position measuring unit 32 detects the person H in the image by executing an image processing algorithm such as person detection on the image data P. FIG. As a person detection method, a well-known person detection method such as storing a large number of sample images of the person H in advance and searching whether or not there is a part similar to the sample image in the acquired image may be used. , is not particularly limited. Alternatively, instead of detecting the entire person, a method such as face detection that can detect a partial area of the person H from the image may be used. Next, the foot position B(u, v) of the person H on the image coordinates is determined from the person area F in the image extracted by the person detection. As a determination method, there is a method in which the bottom center of the circumscribing rectangle of the person region F is set as the foot position, or a method in which v There is a method of setting the image coordinates (u', v'-h) obtained by adding a constant value h to the direction as the foot position, and the like, but there is no particular limitation. After that, the image coordinates (u, v) of the foot position B in the image data P are converted to the foot position B ₃ (x, y, z) in the three-dimensional space S ₃ , and on the xz plane S ₂ of y = 0 is measured as the position of the person _H. Conversion from the foot position B (u, v) to the foot position B ₃ (x, y, z) is generally performed using camera parameters such as internal parameters (focal length, etc.) and external parameters (installation position, installation angle, etc.). method. As a method for deriving camera parameters, general camera calibration may be used, and camera parameters can be estimated by manually giving world coordinates corresponding to image coordinates in advance.
<Flow line creation unit 33>
FIG. 4 shows a processing flow for generating the flow line L of the person H by the flow line generation unit 33. As shown in FIG. The flow line creation unit 33 generates a flow line L by combining the positions 28 of the same person measured by the person position measurement unit 32 .

For example, when the person H is imaged by the camera ₂ for t seconds from the time T, the flow line L can be generated by plotting the foot position B2 of the person H at _each time on the same xz plane S2 and joining them together. can. In addition to this example, any method other than the above method may be used as long as it is possible to generate a flow line L that enables the position information and traveling direction of the person H to be grasped in a certain time period.
<People Flow Information Generation Unit 34>
Next, the people flow information generator 34 will be described with reference to FIG. The people flow information generator 34 uses the flow lines La, Lb, . . . , Ln for each person Ha, Hb, .

A method for creating the people flow information Ih will be specifically described. First, as shown in the upper part of FIG. 5, the xz plane S2 of the measurement range is _divided into a plurality of areas (for example, 4×4=16 areas). Then, from the flow line information, the movement direction of the person H in each area is classified into eight directions shown in the middle of FIG. ), the people flow information Ih in the measurement range shown in the lower part of FIG. 5 is obtained. In this people flow information Ih, an area where the direction of people flow D does not exist indicates an area where no person passes through. It is also possible to identify areas that do not pass through. Note that, as a method for generating the people flow information Ih, other methods may be used other than this example as long as the method is capable of roughly grasping the people flow information Ih. As the people flow information Ih, the number of people H who have passed through each area may be counted in addition to the direction of the people flow, and the number of people H passing through each area may be added.
<Recognition area candidate selection unit 4>
FIG. 6 is a functional block diagram of the recognition area candidate selection unit 4. As shown in FIG. The recognition area candidate selection unit 4 includes a people flow information storage unit 41 that stores the people flow information Ih acquired from the people flow information acquisition unit 3 in a people flow information history database 42, and a people flow information storage unit 41 that stores the latest people flow information _Ih acquired from the people flow information acquisition unit 3. and the past people flow information _Ihold stored in the people flow information history database 42, and based on the comparison result of the people flow information comparison unit 43, an area with a change in people flow is output as a recognition area candidate C. A recognition area candidate output unit 44 is provided. The people flow information comparison unit 43 and the recognition area candidate output unit 44 will be described in detail below.
<People Flow Information Comparing Unit 43, Recognition Area Candidate Output Unit 44>
The people flow information comparison unit 43 will be described with reference to FIG. The people flow information comparison unit 43 compares the latest people flow information Ih _new acquired from the people flow information acquisition unit 3 with the past people flow information _Ihold stored in the people flow information history database 42 to calculate the amount of change in people flow ΔIh.

As a comparison method used by the people flow information comparison unit 43, for example, there is a method of digitizing the direction of people flow in each area of the people flow information Ih and comparing the frequency of appearance of each numerical value. Specifically, as illustrated in the upper part of FIG. 7, after creating people flow information Ih′ that is a numerical representation of people flow information Ih, the appearance frequency for each direction of people flow in the entire measurement range is calculated based on this people flow information Ih′. A histogram (FIG. 7(a)) is created. Then, it is a method of calculating the amount of change in people flow ΔIh by calculating the difference between the histogram based on the latest people flow information _Ih'new and the histogram based on the past people flow information _Ih'old .

As another comparison method used by the people flow information comparison unit 43, for example, there is a method of comparing new and old people flow information Ih' obtained by digitizing the direction of people flow in each area. Specifically, as shown in FIG. 7B, the latest people flow information _Ih'new and the past people flow information _Ih'old are compared, and 1 is assigned to areas where there is a change in people flow, and 0 is assigned to areas where there is no change. This is a method for calculating the amount of change in the flow of people ΔIh.

The recognition area candidate output unit 44 uses the people flow change amount ΔIh calculated by the people flow information comparison unit 43 to output an area where the people flow changes as a recognition area candidate C. FIG. For example, when using the amount of change in the flow of people _ΔIh in _FIG .

As the past people flow information Ih _old acquired from the people flow information history database 42, the most recent information in the past can be used as it is, or the average can be calculated each time the people flow information Ih is acquired from the people flow information acquisition unit 3. There are a method of using the people flow information Ih _ave updated by, for example, a method of using the average of the people flow information Ih at regular time intervals, etc., and there is no particular limitation. Also, the people flow information Ih may be expressed using not only the direction of people flow but also the flow of people. For example, in the case of FIG. A method of using the combined values as a histogram may also be used. In addition to the method described in this example, other methods may be used as long as they can compare the people flow information Ih. In addition to the method of comparing the people flow information Ih based on the position of the person H as in this example, other methods may be used as long as they are capable of roughly detecting an area where a change in people flow is expected. may For example, a method such as comparing the optical flow of the entire image may be used.
<Human flow interaction extraction unit 5>
FIG. 8 shows a block configuration of the human flow interaction extraction unit 5. As shown in FIG. As shown here, the people flow interaction extraction unit 5 uses the latest people flow information Ih _new acquired from the people flow information acquisition unit 3 and the position information of the recognition area candidate C output from the recognition area candidate selection unit 4 to perform recognition. A people flow interaction calculation unit 51 that calculates people flow interaction for area candidate C, and a people flow interaction output unit 52 that outputs the calculated people flow interaction are provided. The people flow interaction calculation unit 51 also includes a positional relationship calculation unit 51a, a flow line change calculation unit 51b, and an attention level calculation unit 51c in order to calculate people flow interaction. Each function of the people flow interaction calculator 51 and the people flow interaction output unit 52 will be described below.
<Positional relationship calculator 51a>
The positional relationship calculator 51a will be described with reference to FIG. In FIG. 9, B ₃ a, B ₃ b, and B ₃ c are the latest foot positions of the persons Ha, Hb, and Hc in the three-dimensional space S ₃ , and B ₂ a, B ₂ b, and B ₂ c are the xz plane. _The latest foot positions of persons Ha, Hb, and Hc in S2, C1 and _C2 are recognition area candidates, _O1 and _{O2 are} the centers of the respective recognition area candidates, and _O1Ha , _O1Hb and _O1Hc are The Euclidean distance between the foot positions B ₂ a, B ₂ b, and B ₂ c of each person and the center O ₁ of the recognition area candidate C ₁ , th, is an example of the Euclidean distance threshold.

The positional relationship calculator 51a calculates and accumulates Euclidean distances between the centers of each person and each recognition area candidate at a certain time, and generates a graph as shown in FIG. 9(c). For example, as shown by the solid line in FIG. 9C, there is a period in which the Euclidean distance O ₁ Ha between the person Ha and the center O ₁ of the recognition area candidate C ₁ is equal to or less than the threshold value _th . It can be seen that there is a period that is quite close to . Also, as indicated by the dashed line, there is a point where the Euclidean distance O ₁ Hb between the person Hb and the center O ₁ of the recognition area candidate C ₁ is 0, so the person Hb moves the center O ₁ of the recognition area candidate C ₁ to I know you passed.

The positional relationship calculator 51a calculates the positional relationship between the person H and the recognition area candidate based on the Euclidean distance variation information. The calculation method is not particularly limited as long as it can calculate a value that can quantitatively express the degree of approach of the person H to the recognition area candidate C. FIG. For example, there is a method of directly using the points plotted at positions smaller than a predetermined Euclidean distance threshold th as the degree of proximity. In addition, there is a method of using graph inclination information between certain plots. As exemplified by O ₁ Ha and O ₁ Hb in FIG. 9C, it is possible to determine whether or not the person H is approaching the recognition area candidate C by obtaining the positive/negative of the slope of the graph. By calculating the moving speed of the person H, it is possible to determine whether or not the person H is actively moving with respect to the recognition area candidate C. Therefore, the sum of the moving speeds can be used as a criterion for determining the degree of approach. In addition, instead of using all plots in the graph of each person, a plurality of thresholds other than the threshold th described above are set, and plot points above a certain threshold are not used when calculating the degree of approach, or 2 A method such as using only plot points between two thresholds may be used.
<Flow line change calculation unit 51b>
The flow line change calculation unit 51 b calculates the amount of change in the flow line L of the person H with respect to the recognition area candidate C from the flow line information acquired from the people flow information acquisition unit 3 . As a method of calculating the change in the line of flow L, there is a method of utilizing the direction of the line of flow L as shown in the middle of FIG. For example, the direction of the flow line L of each person is stored every time, and the degree of change in the direction of the flow line L near the recognition area candidate C is set as the amount of change in the flow line L. By setting a value range in which the amount of change is 0 if the moving direction of H remains in the depth direction, and that the amount of change takes the maximum value if the moving direction changes to the opposite hand direction, the flow line of the person H can be obtained. The amount of change in L can be calculated. In addition, the past people flow information _Ihold in the measurement range may also be used _. There is also a method of using a determination criterion such that the amount of change in the flow line L increases when the person moves in a direction that is significantly different from the direction of the past people flow information _Ihold in the vicinity of .
<Attention degree calculation unit 51c>
The attention degree calculation unit 51c is a function of calculating the degree of attention to the recognition area candidate C from the information of the line-of-sight direction of the person H with respect to the recognition area candidate C. FIG. The method of acquiring the line-of-sight direction information is not particularly limited, such as a method of estimating the line-of-sight direction from the image data P, a method of using a line-of-sight sensor, or the like.

As a method of estimating the line-of-sight direction from the image data P, for example, a method of performing head detection on the image, extracting the head region, and estimating from the ratio of black to the head region may be used. If there are very few black areas with respect to the inner area, the direction of the camera, if there are many black areas, the direction opposite to the camera, if there are many black areas on the left side, the right direction, and if there are many black areas on the right side, the left direction. It is possible to estimate the viewing direction.

In this way, the attention degree calculation unit 51c determines that the person H is paying more attention to the recognition area candidate C when the estimated viewpoint direction is the direction of the recognition area candidate C with respect to the person H. The degree of attention to the recognition area candidate C can be quantitatively calculated.
<People Flow Interaction Output Unit 52>
The people flow interaction output unit 52 aggregates the people flow interaction information Ii for each recognition area candidate calculated by the people flow interaction calculation unit 51, and outputs the final degree of people flow interaction for each recognition area candidate C. FIG.

As a method of calculating the degree of interaction, all or part of the interaction information Ii calculated by each of the positional relationship calculation unit 51a, flow line change calculation unit 51b, and attention level calculation unit 51c is used. For example, from the positional relationship calculation unit 51a, the sum of Euclidean distances between the recognition area candidate C and the person H in the measurement range, and from the flow line change calculation unit 51b, the degree of change in the movement direction of each person near the recognition area candidate C and the sum of the attention degrees of the person H to the recognition area candidate C from the attention degree calculation unit 51c. Alternatively, there is a method of determining the sum obtained using a weighting factor or the like as the interaction degree of the human flow with respect to the recognition area candidate C. FIG.

Further, the final interaction information Ii may be determined using the correlation of the human flow interaction output from each of the positional relationship calculation unit 51a, the flow line change calculation unit 51b, and the attention level calculation unit 51c. , basically, it is assumed that the line of sight of the person H is in the same direction as the traveling direction, and from the information of the positional relationship calculation unit 51a and the flow line change calculation unit 51b, it is estimated that the person H does not approach the recognition area candidate C Regardless, for the recognition area candidate C with a high degree of attention output from the attention degree calculation unit 51c, a method of increasing the final degree of human interaction may be used.
<Recognition area determination unit 6>
The recognition area determination unit 6 is a processing target determination unit that determines an execution target of the state recognition processing in the state recognition unit 7. When the target of the state recognition processing is an area as in the present embodiment, human flow interaction The recognition area candidate C output by the output unit 52 and having a high degree of human interaction is determined as the recognition area A to be processed by the state recognition unit 7 . Note that the recognition area A determined by the recognition area determination unit 6 is not limited to one, and not only the recognition area candidate C with the maximum degree of human flow interaction, but also a plurality of subsequent recognition area candidates with the degree of human flow interaction. C may be determined as the recognition area A. On the other hand, if the maximum human flow interaction degree is smaller than a predetermined threshold value, the recognition area A may not be determined. Also, the number of recognition area candidates to be output may be set by referring to specification information such as the CPU and memory of the state recognition device 1 .
<State Recognition Unit 7>
The state recognition unit 7 performs advanced state recognition processing on the recognition area A determined by the recognition area determination unit 6 . As a result, the area to be processed by the state recognizing unit 7 is narrowed from the entire measurement range to the recognition area A, so that processing resources required for state recognition can be greatly reduced even when an advanced algorithm is used. The content of the state recognition processing includes processing for detecting abnormal conditions (traffic jams, etc.) in the measurement range, processing for detecting abnormal behavior of the measurement target (fallen person, violent person, broken vehicle, etc.), and processing for detecting abnormal behavior around the measurement target. This is one of the processes of detecting an object and determining its attributes (person's age, gender, person's belongings, etc.), but since these processes themselves are the same as conventional processes, detailed description thereof will be omitted.

In the above embodiment, the human flow interaction extraction unit 5 and the recognition area determination unit 6 extract the degree of interaction between the recognition area candidate C and the person H, and determine the recognition area A in which the state recognition unit 7 is executed. Although the method has been described, the person H who executes the state recognition section 7 may be determined by extracting the degree of interaction between persons. For example, in the recognition area candidate selection unit 4, immediately after the change in the flow of people is detected, the person H in the recognition area candidate C is set as the center position, and the people flow interaction extraction unit 5 detects the person H in the recognition area candidate C and the surrounding area. A method of calculating the degree of interaction with the person H and outputting the person H whose final sum of the degrees of interaction is higher than a threshold value as an object to be executed by the state recognition unit 7 may be employed. In this method, it is not necessary for the person H, who is the central position of the human flow interaction extractor 5, to always be in the recognition area candidate C. Instead of the area selected by the unit 4, a range within a certain distance from the person H serving as the central position of the human flow interaction extraction unit 5 may be output as the final recognition area A. In addition, it is not necessary for the people flow interaction calculation unit 51 to output the interaction information Ii for all persons H, and when a plurality of persons are newly detected at the same time, the information is randomly generated so that the number of persons is within a predetermined number. A method of selecting the person H may be used.

In the first embodiment of the present invention, with the configuration described above, the state recognition apparatus that recognizes the measurement range and the state of the measurement object detects the mutual interaction between the measurement objects extracted from the flow information of the general movement of the measurement object. By selecting in advance the measurement range and the measurement target for executing the state recognition processing based on the action information Ii, it is possible to execute advanced state recognition processing while reducing the processing resources of the entire apparatus.

In the first embodiment, the state recognition processing executed by the state recognition unit 7 includes a function for recognizing an abnormal state in a measurement range, a function for recognizing abnormal behavior of a measurement target, a high-precision face recognition function, and a person recognition process. The function of recognizing the attribute of H is not particularly limited, and the processing is not limited to advanced processing such that the processing resources of the entire apparatus are insufficient to execute the entire measurement range.

Next, a state recognition device 1A according to a second embodiment of the present invention will be described using the functional block diagram of FIG. Duplicate descriptions of common points with the first embodiment will be omitted.

The state recognition device 1A shown in FIG. 10 selects a recognition area A for executing advanced state recognition processing and the cameras 2 to be used at that time based on interaction information Ii extracted from people flow information Ih acquired by a plurality of cameras 2. It is a pre-selected device.

The state recognition device 1A of FIG. 10 is different from the state recognition device 1 of the first embodiment in that the units from the people flow information acquisition unit 3 to the state recognition unit 7 are provided in the same number as the cameras 2, and that the information The point is that the shared unit 8 and the state recognition command unit 9 are provided.

The information sharing unit 8 includes an installation position information database 81 that stores installation position information of a plurality of cameras, a people flow information database 82 that stores people flow information Ih acquired from each camera image, and output from the recognition area determination unit 6 in each camera. A recognition area information database 83 that stores recognition area information and a recognition target information database 84 that stores recognition target information are provided. command whether or not to execute Details of the information sharing unit 8 and the state recognition command unit 9 will be described below.
<Information sharing unit 8>
The information sharing unit 8 shares and holds the information of the plurality of cameras 2 and the analysis results of the camera images. Each piece of information is described in detail.

The installation position information database 81 holds information on the installation position and angle of view of the camera in the measurement range. The information held here is, for example, a drawing such as a two-dimensional map of the entire measurement range that is prepared in advance, and information that allows the camera's shooting area to be grasped from the position where the camera is installed, the direction of the camera, and the angle of view. If so, it is not particularly limited.

The people flow information database 82 holds information on the flow line L of each person acquired by the people flow information acquisition unit 3 and the direction of the people flow within the measurement range in each camera image. By sharing this information and connecting the results between cameras, analysis using the flow line information of the person H in the recognition area candidate selection unit 4, the human flow interaction extraction unit 5, and the recognition area determination unit 6 in the latter stage can be executed with high accuracy.

The recognition area information database 83 and the recognition target information database 84 hold position information such as the recognition area A determined by the recognition area determining unit 6 and the person H who is the recognition target.
<State recognition command unit 9>
Based on the information obtained from the information sharing section 8, the state recognition commanding section 9 determines the camera 2 for executing the state recognition processing. As the determination method, there are a method of selecting the camera closest to the recognition area A and the recognition target, a method of determining by considering information obtained from the installation position information database 81 and the people flow information database 82, etc., and there is no particular limitation. . To give an example of the latter method, from the installation position of the camera 2, the shooting direction, and the latest crowd flow information Ih _new , it is possible to determine whether or not there is an obstruction caused by a person H or the like between the camera 2 and the recognition area A or the recognition target. There is a method of determining whether or not there is any obstruction, and instructing the execution of state recognition processing to a camera that is expected to be unoccluded.

In the second embodiment of the present invention described above, in the state recognition device that recognizes the measurement range using a plurality of measurement devices (camera 2) and the state of the measurement target, the flow of the approximate movement of the measurement target Based on the interaction information Ii between the measurement targets extracted from the information, by selecting in advance the measurement range, the measurement target, and the measurement device for executing the state recognition processing, it is possible to reduce the processing resources of the entire device and achieve an advanced state. Recognition processing can be performed.

In the second embodiment, an example has been described in which each camera has the function of selecting the recognition area A and the recognition target and the function of executing advanced state recognition processing, but the function may be assigned to each camera. For example, when two cameras are used, one camera executes the functions of the people flow information acquisition unit 3, the recognition area candidate selection unit 4, the people flow interaction extraction unit 5, and the recognition area determination unit 6 to determine the recognition target. By making the camera dedicated to determining and the other camera dedicated to executing only advanced state recognition processing, the processing resources of the latter camera can be reduced and more advanced state recognition processing can be applied. Also good.

In the second embodiment, the interaction information Ii output from the human flow interaction extraction unit 5 may also be held by the information sharing unit 8 and shared by each camera. For example, when the recognition area determining unit 6 determines the recognition target, if the interaction between the flow of people and the recognition target is too small for the first camera to determine the final recognition target, By handing over the degree of interaction between cameras, the final degree of interaction exceeds the threshold due to the sum of the degree of interaction with the second camera, etc., and recognition that was overlooked with only one camera. A state recognition process can be executed on the target by the second camera and subsequent cameras.

1, 1A state recognition device,
2 cameras,
3 people flow information acquisition unit,
31 video acquisition unit,
32 human position measurement unit,
33 flow line creation unit,
34 people flow information generation unit,
4 recognition area candidate selection unit,
41 people flow information storage unit,
42 people flow information history database,
43 People Flow Information Comparison Department,
44 recognition area candidate output unit,
5 human flow interaction extraction unit,
51 People Flow Interaction Calculation Unit,
51a positional relationship calculator,
51b flow line change calculation unit,
51c attention degree calculation unit,
52 people flow interaction output unit,
6 recognition area determination unit,
7 state recognition unit,
8 Information sharing unit;
81 installation location information database,
82 people flow information database,
83 recognition area information database,
84 recognition target information database,
9 State recognition command part

Claims (9)

a motion information generation unit that generates motion information of a measurement target based on information acquired by the measurement device;
a recognition target candidate selection unit that selects a recognition target candidate from the entire measurement range of the measuring device based on the motion information;
an interaction extraction unit that extracts interaction information between the space and the measurement target or interaction information between the measurement targets using the relationship between the motion information and the recognition target candidate ;
a recognition target determination unit that determines, from the recognition target candidates, a recognition target for executing state recognition processing based on the interaction information;
a state recognition unit that executes state recognition processing on the recognition target;
A state recognition device characterized by comprising: In the state recognition device according to claim 1 ,
The state recognition processing executed by the state recognition unit is any of processing for detecting an abnormal state in the measurement range of the measuring device, processing for detecting abnormal behavior of the measurement target, and processing for detecting an attribute of the measurement target. A state recognition device characterized by: In the state recognition device according to claim 1 or claim 2 ,
State recognition characterized by comprising a plurality of the measuring devices, and determining a measuring device to be used when executing the state recognition process based on shared information in which information acquired by the plurality of measuring devices is shared. Device. In the state recognition device according to claim 3 ,
The state recognition device, wherein the shared information is any one of installation position information of the measurement device, movement information of the measurement target, or information of the recognition target. In the state recognition device according to claim 3 or claim 4 ,
The state recognition device, wherein the plurality of measurement devices include at least one dedicated measurement device for executing state recognition processing. In the state recognition device according to any one of claims 1 to 5 ,
The state recognition device, wherein the motion information is information on the position and movement direction of the object to be measured. In the state recognition device according to any one of claims 1 to 6 ,
The interaction information extracted by the interaction extraction unit is
A state recognition apparatus characterized by being information on any one of a distance between a space and said object to be measured, a distance between said objects to be measured, a moving speed of said object to be measured, a line of flow, an amount of change in line of flow, and a line of sight. In the state recognition device according to any one of claims 1 to 7 ,
The state recognition device, wherein the measurement device is one of a monocular camera, a stereo camera, and a distance sensor. Based on the information acquired by the measurement device, generate motion information of the measurement target,
selecting a recognition target candidate from the entire measurement range of the measurement device based on the motion information;
extracting interaction information between the space and the measurement target or interaction information between the measurement targets using the relationship between the motion information and the recognition target candidate ;
determining a recognition target for executing state recognition processing from the recognition target candidates based on the interaction information;
A state recognition method characterized by executing state recognition processing on the recognition target.

Images